1768 results about "Sight line" patented technology

The invention provides a driving behavior intelligent identification method and system based on video analysis, and the method comprises the steps: obtaining a driver face video stream and a road environment video stream collected by a vehicle-mounted camera, and reading the driving information recorded by a whole vehicle communication network; recognizing an eyelid closing state, a sight line direction and a head posture in the driver face video stream based on a posture recognition model, and performing fatigue distraction analysis to obtain driver state information; performing motion trail analysis on the road environment video stream and the driving information, and performing driving risk assessment in combination with the driver state information to obtain driving assessment information; and performing early warning construction according to the driving evaluation information, generating early warning prompt information, and synchronously writing the early warning prompt information, the driving evaluation information and the driver state information into a safety data protection memory. The fatigue and distraction states of the driver can be recognized more accurately, and the accuracy of state judgment is improved.

Various aspects of the present disclosure relate to a use equipment (UE) configured with multiple artificial intelligence (AI) models each of which has been configured (e.g., trained) based on training data sets corresponding to one or more of different conditions, such as location of the UE, orientation of the UE, whether the UE is indoors or outdoors, whether the UE is line-of-sight or non-line-of-sight with a base station, and so forth. A network entity (e.g., a gNB) configures the UE with a set of reference signals for measurement of at least one quantity. The UE generates the measurement report based at least in part on the set of reference signals and one of the multiple AI models, and transmits the measurement report to the network entity (e.g., gNB).

The invention relates to the technical field of vehicles, in particular to an intelligent cabin man-machine cooperative control method and device, a vehicle, a medium and a product. Performing fusion processing on the voice signal data, the visual signal data and the tactile signal data of the current driver by using a preset dynamic weight distribution model to obtain a multi-modal fusion control instruction; and under the condition that the multi-mode fusion control instruction meets a preset sight-voice dual-mode condition, synchronously issuing the multi-mode fusion control instruction to a plurality of target execution devices in the intelligent cabin so as to drive each target execution device to perform cooperative control. Therefore, by constructing a cooperative control mechanism of multi-mode sensing and dynamic weight distribution, the problems that the false wake-up rate is high, the voice and graphical interface cooperative efficiency is low and the interaction mode cannot adapt to a complex driving environment due to a single or separated interaction mode in the related technology are effectively solved.

This invention discloses a 2D gaze point estimation method based on the pupil-corneal reflection vector method. It utilizes the star ray method for pupil center localization and adds a classification pupil detection mechanism for error correction. Corneal feature extraction is performed to obtain the center coordinates of the corneal reflected light spot. Step 4: Establish a gaze mapping model, and assign the detected pupil-corneal reflection vector V(V) to each frame of the image. x V y The gaze coordinates G(G) mapped onto the screen x G y This invention enables 2D gaze point estimation based on the pupil-corneal reflection vector method. Compared with existing technologies, this invention can improve the accuracy of 2D gaze point estimation.

The invention provides a three-dimensional flow velocity field inversion method and system based on coherence-assisted SAR adaptive offset tracking, and a storage medium, so as to improve the problem of decoherence caused by time heterogeneity in a synthetic aperture radar interferometry (InSAR) technology, and relates to the technical field of electric power engineering geological disaster monitoring. The method comprises the following steps: establishing a space-time baseline network between SAR images; obtaining coherence data after geocoding; selecting a pair of SAR images from the space-time baseline network as a reference unit, calculating an offset result through a pixel offset tracking technology for the reference unit, and determining an optimal matching window parameter of the reference unit; establishing a mapping function between coherence and optimal matching window parameters; according to the established mapping function, calculating an optimal matching window parameter of each remaining SAR image pair; and in combination with the line-of-sight average deformation rate fields of the ascending and descending rails, inverting a surface three-dimensional flow velocity deformation field of the target area through a geometric vector decomposition model.

The invention relates to a radar synthetic wind speed resolving method, device and equipment and a storage medium, and is applied to the field of radar speed measurement, the radar synthetic wind speed resolving method comprises the steps that radar sight wind speed and the rotation angle of a radar are acquired, and the rotation angle is the angle of a radar roll angle; performing radar beam azimuth calculation according to the rotation angle to obtain a radar beam vector; and performing synthetic wind speed calculation according to the radar sight wind speed and the radar beam vector, and generating scalar wind speed and wind direction. The method has the technical effect that the calculation accuracy of the synthetic wind speed is improved.

The invention discloses a high arch dam global deformation analysis method and system fusing InSAR and measuring point data, particularly relates to the technical field of radar interference deformation monitoring, and is used for solving the problem of inaccurate shielding failure compensation. According to the method, data stability is guaranteed by obtaining InSAR image data and extracting sight line direction displacement in combination with coherence indexes, a key measuring point deformation sequence is collected, an InSAR result is calibrated based on stable reference points, data consistency is improved, a projection correction coefficient is constructed by using a dam body three-dimensional structure model, the sight line displacement is inversely solved into structural normal displacement, and the accuracy of the dam body three-dimensional structure model is improved. Obtaining an initial deformation value of the real structure meaning; according to the method, a spatial coupling map of measuring points and InSAR pixels is constructed for a shielding failure area, deformation compensation of the failure area is completed based on an edge weight, finally the deformation compensation result is remapped to a three-dimensional structure model, measuring point data restoration is performed on a high arch dam, and structural plane-level deformation visualization and trend analysis are realized more quickly and more accurately.

The invention discloses an eye movement tracking method and system based on multi-modal fusion, and particularly relates to the technical field of eye movement tracking, and the method comprises the steps: collecting a face image, an eye image and a head movement parameter of a user through an RGB camera, an infrared camera and an IMU which are synchronously arranged; the signals are processed in parallel, sight line direction vectors based on the face and the eyes and reliability measurement of the sight line direction vectors are extracted respectively, and the head space posture is calculated; dynamically selecting high-confidence cooperation, master-slave compensation or conflict arbitration strategies according to reliability measurement, and fusing to generate an initial sight direction; and mapping the direction to a display plane to obtain an initial fixation coordinate, performing optimization calibration in combination with visual saliency analysis of screen content, and outputting a final fixation point coordinate. The system comprises a signal acquisition and processing module, a parallel computing module, a multi-strategy fusion decision-making module and an output optimization module. According to the invention, through multi-modal information fusion and dynamic strategy selection, the precision and robustness of eye movement tracking are significantly improved.

The invention relates to the technical field of virtual reality and indoor and outdoor design, in particular to a VR indoor and outdoor design virtual material selection system based on eye movement tracking. Comprising a VR display and eye movement tracking module used for presenting a virtual design space to a user and capturing eye movement data of the user in real time; the fixation point recognition and target object judgment module is used for calculating real-time fixation point coordinates of the user based on the eye movement data; the dynamic material menu management module is used for outputting the target object information according to the judgment module; the sight line interaction and material rendering module is used for receiving a selection instruction of a user on a specific material in the dynamic material menu through a sight line; and the central processing module is used for coordinating and controlling data streams and execution logics of the modules. By integrating the eye movement tracking technology and the dynamic material menu management, the intelligent material selection interaction driven by the fixation point is realized.

The invention discloses a dynamic rendering method and system for virtual reality, a storage medium and electronic equipment, and relates to the technical field of virtual reality, augmented reality and computer graphics. Therefore, the frequency and time sequence change of the sight focus of the user hitting the surface of the virtual object in the virtual scene are analyzed, so that the relative motion trend and observation intention of the user and the object are pre-judged, predictive and adaptive scheduling of rendering resources is realized, and optical feedback conforming to the physical law, namely, dynamic material, is triggered at the sight focus, so that the real-time performance of the user is improved. Therefore, the interaction fineness in the process of interaction between the user and the VR system is improved.

The invention discloses a virtual-real fusion dynamic calibration method based on binocular eye movement tracking. The method comprises the following steps: step 1, constructing a virtual-real fusion dynamic calibration system based on binocular eye movement tracking; step 2, establishing a calibration coordinate system; 3, solving a conversion relation between the coordinate system of the eye movement tracking camera and the coordinate system of the AR equipment through offline joint calibration; 4, collecting eye movement tracking parameters and modeling user characteristics; 5, constructing a virtual display camera model, and solving an internal parameter matrix and external parameter transformation of a virtual display camera, so that the virtual display camera is geometrically consistent with the real watching behavior of the user; step 6, constructing a conversion relation between an external tracking sensor coordinate system and a virtual display coordinate system, so as to convert the position of the marker to a virtual display camera coordinate system; and step 7, evaluating a system registration error, carrying out on-line fine tuning on key external parameters, carrying out sight line prediction compensation, and adaptively adjusting the pose of a virtual object in a virtual display camera coordinate system.

The invention provides a space intelligent non-planar reflection generation method based on surface normal guidance. Extracting multi-layer visual representation from an original RGB image containing a non-planar reflection medium scene, and identifying a non-planar reflection area by combining brightness distribution of the original RGB image and a spatial symmetry relation; analyzing non-linear deformation features of textures in the non-planar reflection area, and predicting pixel-level three-dimensional surface normal vectors and confidence distribution in combination with multi-layer visual representation to construct a surface normal graph; determining a line-of-sight vector based on a camera imaging model, and executing geometrical optical mapping operation on the line-of-sight vector, the surface normal graph and the confidence distribution to construct a reflection offset field; extracting an intermediate semantic feature representation corresponding to the non-planar reflection region, and generating a pre-distortion reflection feature representation according to the reflection offset field and the intermediate semantic feature representation; and injecting the pre-distortion reflection feature representation, the multi-layer visual representation and the initial reflection area mask into a generative model for image reconstruction, and generating a non-planar reflection image.

The invention relates to the technical field of elevators, in particular to an elevator traction machine vibration reconstruction method and system based on a millimeter wave radar. The method comprises the steps that in the elevator running process, echo signals of a millimeter wave radar are collected; firstly, a continuous phase sequence is obtained through echo signals; generating a rough vibration signal in a radar sight line direction according to a phase-displacement mapping relation, calculating a physical confidence coefficient, constructing a dual-channel physical prior tensor, synthesizing a three-axis reconstruction vibration signal tensor by using a PGGN physical gating generation network, and finally performing physical rationality evaluation on the three-axis reconstruction vibration signal tensor. And adjusting the gating parameters of the anisotropic physical gating unit according to the evaluation result. On the premise that a contact sensor does not need to be installed on the surface of the traction machine for a long time, physical consistency reconstruction of the three-axis vibration signals of the elevator traction machine is achieved.

The invention discloses a VR panoramic glasses data synchronization and transmission method based on multi-sensor fusion, and relates to the technical field of computer network and multimedia real-time transmission. Time alignment and unified modeling are carried out on various sensor data such as eye movement, inertia measurement, gestures and environment through extended Kalman filtering; a user head position, a posture, a sight line direction and a gesture action are coded into a single user behavior vector, so that a subsequent system does not face original sensor streams with different formats any more, but makes a decision based on a behavior space with a clear structure, thereby remarkably reducing posture jitter and sight line deviation accumulative errors; a long short-term memory network is introduced to carry out time sequence modeling on behavior vectors and environment contexts, interactive hotspot areas in a virtual scene are predicted, and hotspot distribution is mapped into priority weights of different data streams, so that sensor streams related to fast actions in areas near a sight line can still obtain a relatively high service level when the bandwidth is insufficient.

The invention discloses a predictive vehicle control system and method for multi-modal consciousness recognition and electronic equipment, and relates to the field of remote monitoring, and the system comprises a multi-modal sensing module, an intention prediction module, a cross-domain pre-control module and a false trigger suppression module. The multi-mode sensing module is used for collecting three-mode data of sight, gestures and voice and dynamically distributing weights based on environmental parameters; the intention prediction module is used for performing time-space synchronization verification on the three-mode data and calculating comprehensive confidence in a grading manner; the cross-domain pre-control module outputs pre-control instructions to a cabin domain, a vehicle body domain and a power domain based on the effective intention; the false triggering suppression module is used for dynamically adjusting a confidence coefficient threshold value based on a driving scene and has a failure backspacing learning function; wherein each module performs data interaction based on a vehicle-mounted Ethernet / CAN bus.

The invention provides a dual-unmanned aerial vehicle cooperation method and a cooperation control system, and relates to the technical field of unmanned aerial vehicles. The method comprises the steps that a flight task issued by a ground control part is acquired from an unmanned aerial vehicle, first flight data and second flight data are acquired, the first flight data comprise main image data, and the second flight data comprise auxiliary image data; the slave unmanned aerial vehicle determines a space vector between the master unmanned aerial vehicle and the slave unmanned aerial vehicle as a baseline; the slave unmanned aerial vehicle constructs an accompanying flight control model based on the obstacle influence data, the baseline length corresponding to the baseline and the observation sight line included angle, generates an accompanying flight control instruction enabling the accompanying flight control model to reach the flight condition, controls flight according to the accompanying flight control instruction, and the obstacle influence data is used for describing the obstacle avoidance risk of the flight route to the obstacle; the ground control unit fuses the main image data and the auxiliary image data to obtain an observation result of the object to be observed. The flight stability of the multiple unmanned aerial vehicles is improved, and the efficiency of surveying and mapping tasks is improved.

A communication method including emitting laser pulses as a signal using one or more transmitters at an atmospheric point. Plasma is generated at the atmospheric point to radiate omnidirectional light. The plasma from the atmospheric point is detected using one or more receivers. The one or more receivers have a direct line-of-sight to the atmospheric point to receive the signal from the one or more transmitters via the omnidirectional light generated by the plasma. The signal is demodulated from the transmitted plasma at each receiver, thereby transmitting information encoded on the signal.

The invention discloses a data processing and interaction control system based on dynamic eye position tracking, and particularly relates to the technical field of man-machine interaction and eye movement tracking, which comprises an image acquisition module, a data processing module, an interaction control module and an equipment interface module. The image acquisition module controls a near-infrared camera to capture an eye video; the data processing module executes dynamic calibration, triggers calibration points based on a historical gaze point distribution sparse region, updates a sight line mapping model online by adopting a recursive least square method, and recognizes an interaction intention through a state machine in combination with a gaze point sequence and an eye movement event mode; the interaction control module carries out logic binding on the eye movement intention and signals such as head postures, gestures and voice, and a composite control instruction is generated through complementary fusion, concurrent binding and conflict arbitration; and the device interface module standardizes the instruction and then outputs the instruction to the controlled device. The system realizes non-intrusive dynamic calibration, effectively improves tracking precision and interaction robustness, and is suitable for a multi-modal fusion control scene.

The invention discloses a landscape object identification and explanation method and system, a medium and a product, and relates to the technical field of augmented reality. The method comprises the following steps: when the head motion state of a user is stable, triggering a camera to collect and recognize a single-frame image, matching a recognition result with a regional knowledge base, and generating a candidate object list and coarse-grained poses which are sorted according to spatial probability and historical popularity; performing feature point matching on a current frame and an ideal frame in the video stream, and calculating a fine-grained pose of a target landscape in the candidate object list relative to a camera coordinate system according to a matched target feature point; according to the fine-grained pose and the real-time eye movement data, determining a drop point part of the sight line of the user on the target landscape, forming a multi-dimensional query vector according to the drop point part, the intention of the user and the environment context parameters, and generating multimedia explanation content; and carrying out fusion rendering on the multimedia explanation content and the real-time video stream, and outputting and displaying. By implementing the technical scheme, the personalized multimedia explanation content can be generated.

The invention relates to the technical field of complex industrial equipment monitoring and digital twinning, and discloses a gas turbine data visualization and intelligent interaction system which comprises a sensor array, a computing unit, a force feedback device, an eye tracker and a display terminal. The system constructs a three-dimensional physical field and corrects a velocity vector by using Kriging interpolation and Euler equation momentum balance, and generates a variable radius flow tube reflecting the compressibility of fluid; calculating mass, momentum and energy conservation residual errors to construct a generalized potential energy field, and generating gradient guide force and self-adaptive damping force by driving force feedback equipment; and dynamically adjusting solving precision and rendering parameters through sight tracking. According to the method, the flow field is reconstructed through physical constraint, multi-modal feedback is introduced, the problem that discrete data feature distortion and single visual interaction are difficult to perceive physical consistency is solved, and high-fidelity reduction of the gas turbine flow field and visual touch synchronous accurate diagnosis of fault types are achieved.

The invention belongs to the field of intelligent driving interaction, discloses an HUD interaction method and system combining DMS and ADAS functions, and constructs a multi-mode state sensing system by obtaining a real-time image of a driver and steering wheel grip data. According to the method, eye, pupil and head posture recognition based on deep learning is complementary with a grip distribution matrix, so that fatigue behaviors, distraction behaviors and abnormal holding behaviors can be detected earlier and more accurately, and the precision and robustness of driver state recognition are effectively improved. According to the method, after the fatigue degree of the driver is analyzed, the cognitive load of the driver is dynamically evaluated in a finer-grained mode by constructing the composite indexes based on the eyelid opening degree, the sight line deviation, the grip variation coefficient and the respiratory rhythm. The evaluation mode can reflect the attention, mental load and information processing capability of the driver in real time, so that the system can generate a prompt strategy which better conforms to human factors according to the current state of the driver.

The invention relates to the technical field of vehicle-mounted human-computer interaction and graphic rendering, and discloses a method and a device for optimizing picture fluency of a vehicle-mounted display screen. The method comprises the following steps: collecting multi-source data and preprocessing the multi-source data to obtain a denoised sight line position and light intensity data; fusing the data and determining real-time sight focus coordinates of the driver through light correction; if the focus is located in the navigation area, preferentially calculating a rendering thread of the focus, judging the residence time and calibrating the locking precision; the navigation area frame rate is improved according to the locking state, and secondary area transition data are prepared; residual computing resources are obtained, a degradation rendering scheme is determined, and resource balanced distribution is achieved; extracting a pre-rendering frame when a total load peak value is detected, and obtaining seamless transition data; and finally, injecting the pre-rendered frame into an output buffer area and fusing the real-time coordinates to generate a smooth picture frame sequence. According to the method, dynamic resource allocation and smooth picture output can be realized, and the vehicle-mounted display fluency and the driving safety are remarkably improved.

The present disclosure relates to a multi-mode vehicle window control method and device. The method combines voice recognition and computer vision technologies, detects a user's voice wake-up word using a vehicle-mounted microphone, turns on the vehicle-mounted camera to detect the user's face and head, analyzes the user's gaze, face pose, and head posture to estimate the user's attention region, and aligns the attention region with a vehicle body coordinate system. Intelligent control of a vehicle window is realized according to specific coordinates of an attention area on the vehicle window. Even if the user's face or eyes are not detected, the vehicle window may be operated accurately by determining the user's intention through the face or head pose.

The invention discloses a Rhatish culture VRAR virtual-real fusion interaction method based on eye movement tracking, and particularly relates to the technical field of virtual reality and augmented reality man-machine interaction. The method comprises the following steps: acquiring an eye image of a user through a near-infrared image acquisition unit, and inputting the eye image into a double-branch convolutional neural network model to estimate a two-dimensional fixation point of a screen; performing nonlinear correction on a fixation point based on the user personalized sight line offset correction model established in the offline calibration stage; in the online interaction stage, the corrected fixation point is combined with equipment pose data to be converted into three-dimensional space rays, the distance and angle relation between the three-dimensional space rays and an interaction target is calculated, and an interaction instruction is triggered when a threshold condition is continuously met. According to the method and the system, accurate, stable and natural eye movement interaction of the virtual contents of the Rhatish culture and the entity exhibits in the VR / AR environment is realized, and the user experience and the system robustness are improved.

The invention relates to the cross technical field of medical education and mixed reality technology, in particular to a traditional Chinese medicine meridian point visualization system based on mixed reality technology, which comprises a data fusion and personalized calibration module, a virtual-real fusion interaction module, an interaction teaching module, an intelligent evaluation module and a meridian qi and blood simulation module. By fusing medical image data, depth sensing data and a traditional Chinese medicine acupoint selection rule, a personalized three-dimensional human body model is constructed, and acupoints are accurately calibrated; real-time registration and shielding processing of the virtual model and a real scene are achieved through mixed reality equipment; various natural interaction modes such as gestures, voices and sight lines are supported, and an integrated function of learning, training and assessment is provided; and the meridian qi-blood running state can be dynamically simulated based on the midnight-noon ebb-flow theory. The system has the advantages that the problems that traditional Chinese medicine teaching is poor in intuition and interactivity, cannot be personalized, is inconvenient to examine and the like are solved, and immersive, interactive and intelligent teaching of traditional Chinese medicine meridian point knowledge is achieved.

Systems and methods track a user's line-of-sight relative to objects through a transparent LCD display to provide information about the objects to be displayed on the display proximate the line-of-sight between the object being viewed and the user's head to create an augmented reality environment for the user. A method comprising: determining a position of an object; determining a position of a transparent display; determining a line-of-sight extending between a user's eye and the object through the transparent display; determining where the line-of-sight intersects the transparent display; and displaying object information corresponding to the object via the transparent display proximate where the line-of-sight intersects the transparent display.

The invention discloses a multi-platform SAR (Synthetic Aperture Radar) data and ground monitoring image fused earth surface deformation identification method, and belongs to the technical field of remote sensing monitoring and geological disaster early warning. According to the method, SAR images before and after deformation of a target area are obtained through a spaceborne, airborne or foundation SAR platform, and radar sight direction and azimuth direction deformation fields are extracted through preprocessing and offset tracking or interference processing; synchronously utilizing an unmanned aerial vehicle or ground photography equipment to obtain a high-resolution optical image, and calculating an east-west horizontal deformation field and a south-north horizontal deformation field through an image matching technology; after unifying the multi-source deformation fields to the same spatial resolution, constructing a deformation observation equation set based on a three-dimensional deformation decomposition model by combining radar geometric parameters, and inverting three-dimensional deformation in the vertical, east-west and north-south directions of the earth surface by adopting a weighted fusion algorithm (such as Helmert variance component estimation to determine a weight matrix); and finally, performing time sequence optimization on the deformation field by using Kalman filtering or a stress-strain model to generate a space-time continuous three-dimensional deformation product.

The invention provides intelligent glasses, a shooting control method, electronic equipment and a storage medium, and the intelligent glasses are characterized in that the lower end of a miniature stable assembly is arranged on a glasses frame, a camera module is installed at the front end of the glasses frame through the upper end of the miniature stable assembly, and the miniature stable assembly is used for adjusting the shooting angle of the camera module. The micro stabilizing assembly has at least two rotational degrees of freedom which are perpendicular to each other; the posture sensor is arranged on the glasses frame and used for detecting head posture changes of a wearer in real time and outputting posture data; and the control processing module is electrically connected with the attitude sensor and the micro stable assembly, and is used for receiving the attitude data, calculating an angle correction value based on the attitude data, and driving the micro stable assembly to dynamically adjust the shooting angle of the camera module based on the angle correction value. Therefore, the shooting angle is always consistent with the actual sight direction of the wearer. Viewing offset caused by wearing slippage and posture change is effectively eliminated, and picture instability is remarkably improved.

The invention provides a virtual reality display method and system, a computer and a storage medium. The method comprises the following steps: collecting sight movement data and head movement data of a user; predicting a fixation point area at the next moment based on the historical movement track of the fixation point area; according to the scene depth information, the sight line moving speed and the head motion data of the fixation point area at the next moment, predicting the predicted focal plane depth at the next moment, and dynamically adjusting the configuration of the micro-lens array in the optical assembly to match the predicted focal plane depth. The depth of a focal plane is predicted through fusion data of eye movement tracking and an inertial measurement unit, the configuration of a micro-lens array is dynamically adjusted, meanwhile, a display picture is divided into a fixation point area, a fuzzy display area and an atmosphere rendering area, and based on differentiation rendering processing, the convergence adjustment conflicts of vision are remarkably relieved, the graphic calculation load is reduced, and the display quality is improved. And a self-adaptive high-definition display effect is realized.

The invention discloses an eye movement tracking stability enhancing method, system and device and a storage medium, and belongs to the field of human-computer interaction and computer vision. The method comprises the following steps: acquiring an eye image and an original sight fixation point coordinate; performing quantitative evaluation on the image from five dimensions of definition, contrast and the like to generate an eye image quality evaluation grade; calculating the moving speed and acceleration of the current point based on the historical sight fixation point sequence; fusing, and judging and correcting the original coordinates. Through multi-modal information fusion and hierarchical adaptive correction, noise jump caused by poor image quality can be accurately distinguished and filtered out, meanwhile, the real eye movement dynamic state is effectively reserved, and the output stability and reliability of an eye movement tracking system are remarkably improved.