782 results about "Gaze" patented technology

In some embodiments, while displaying a content entry user interface element and while a first content entry tool is selected, an electronic device detects a first movement of a predefined portion of a user while the predefined portion has a first shape. In some embodiments, in response to detecting the first movement, in accordance with a determination that a gaze of the user was directed toward the content entry user interface element when the first movement was detected, the electronic device enters first content, based on the first content entry tool, corresponding to the first movement into the content entry user interface element. In some embodiments, in accordance with a determination that the gaze of the user was directed toward a menu user interface element when the first movement was detected, the electronic device selects a second content entry tool for entering content into the content entry user interface element.

In some embodiments, an electronic device emphasizes and / or deemphasizes user interfaces based on the gaze of a user. In some embodiments, an electronic device defines levels of immersion for different user interfaces independently of one another. In some embodiments, an electronic device resumes display of a user interface at a previously-displayed level of immersion after (e.g., temporarily) reducing the level of immersion associated with the user interface. In some embodiments, an electronic device allows objects, people, and / or portions of an environment to be visible through a user interface displayed by the electronic device. In some embodiments, an electronic device reduces the level of immersion associated with a user interface based on characteristics of the electronic device and / or physical environment of the electronic device.

The invention relates to the field of multi-modal interaction analysis, in particular to a multi-modal interaction method and system of a digital human agent. The method comprises the following steps: acquiring a real-time face image and a voice signal input stream of an interactive user based on an intelligent agent; performing real-time micro-expression recognition and deep emotion analysis based on the real-time facial image to obtain real-time emotion features of the user; performing time sequence evolution analysis on the real-time emotion characteristics of the user, performing holographic user emotion deep mining, and constructing a user emotion holographic characteristic spectrum; carrying out adaptive acoustic gain processing on the voice signal input stream, and carrying out voice-emotion association analysis based on the user emotion holographic characteristic spectrum to generate a voice-emotion linkage mapping spectrum; and carrying out eyeball fixation point migration tracking based on the user emotion holographic feature map and the real-time face image, and generating a user interaction depth intention signal. Through the real-time deep semantic understanding and emotion perception ability, the intelligent agent interaction intelligence and response accuracy are improved.

In some embodiments, while a keyboard is visible in a three-dimensional environment, the computer system detects a gaze of a user move from a position away from a first key of the keyboard to the first key. In some embodiments, in response to detecting the gaze of the user moving from the position away from the first key to the first key, in accordance with a determination the one or more criteria are satisfied, the computer system initiates a process to select a first character corresponding to the first key for entry. In some embodiments, in response to detecting the gaze of the user moving from the position away from the first key to the first key, in accordance with a determination that the one or more criteria are not satisfied, the computer system forgoes initiating the process.

The invention relates to the technical field of image recognition, in particular to a lightning arrester fault thermal imaging picture recognition method and system combined with gaze prediction, and the method comprises the steps: carrying out the down-sampling of an original thermal imaging image to a fixed size, sequentially passing through a multi-layer convolution and a Spatial Softmax layer, and outputting a predicted gaze point track sequence, extracting a key area and a non-key area according to the intensity of the fixation point track sequence; reconstructing the key area to obtain a compressed and recombined target area; inputting the non-key region into a variational auto-encoder to obtain a low-dimensional potential feature vector; and inputting the image-structure joint feature representation into an EffiCroprViT model, and finally obtaining a lightning arrester fault classification result. Local and global features are efficiently fused, the calculation complexity is reduced, background noise interference is effectively suppressed, real-time and accurate identification and early warning of the fault state of power equipment are realized, and the fault classification method has the advantages of high efficiency, high reliability and high reliability. Therefore, the safety and stability of power grid operation are ensured.

The invention discloses a lightweight display interface rendering optimization system, which relates to the technical field of image processing, and comprises an acquisition module, an interface analysis module, a behavior analysis module and an attention analysis and rendering module, dynamically calculating an effective view radius by utilizing a visual tunneling effect; the interface analysis module performs character string fuzzy matching in combination with the context input by the user, identifies the search intention of the user and improves the weight of a corresponding component; the attention analysis module performs multi-modal weighted fusion on the physiological fixation data and the sparse content saliency thermodynamic diagram; according to the method, by recognizing the semantic intention and the physiological fixation point of the user, on the premise that core visual experience continuity is guaranteed, GPU load and video memory bandwidth occupation are remarkably reduced, and balance of high-performance display and low-power-consumption operation is achieved.

The invention discloses an ideological and political large-scale course teaching attention assessment data acquisition method based on sight tracking. The method comprises the following steps: S1, high-definition large-scene eye movement tracking photographing equipment captures eye images of students in a course teaching process in real time through an infrared camera; s2, performing pupil area detection and pupil center fitting on the eye image to obtain sight tracking data; s3, performing gaze hotspot analysis and gaze duration statistics according to the sight tracking data to quantify attention distribution and change trend of the students and visually display the attention distribution and change trend; and S4, generating a teaching quality evaluation report and teaching optimization suggestions according to the gazing hotspot analysis and gazing duration statistical results. The classroom teaching effect can be objectively evaluated in real time, subjective evaluation errors are reduced, the accuracy and efficiency of teaching feedback are improved, the method is suitable for various scenes such as traditional classrooms, online education and experiment teaching, and the method has the industrial application prospect of education digital transformation.

The invention discloses a cheating detection method and system applied to online interview, relates to the technical field of computer vision, and solves the problems that in the prior art, multiple behavior characteristics in interview personnel are often adopted for monitoring in the whole process, multi-data analysis in the presence of non-cheating behaviors causes waste of computing resources, and the monitoring efficiency is high. And the cheating detection method is low in efficiency. A gaze anomaly coefficient is generated according to gaze data; when the fixation abnormal coefficient is greater than the abnormal threshold value, generating a multi-modal feature according to the monitoring video; generating a multi-modal decision weight according to the interview stage label; fusing the multi-modal decision-making weight and the multi-modal features to obtain multi-modal decision-making data; and inputting the multi-modal decision data into the cheating judgment model to obtain a monitoring result, carrying out tendency evaluation on cheating monitoring, and carrying out multi-modal dynamic fusion when the cheating tendency exists so as to accurately judge the cheating condition, so that the cheating monitoring efficiency and the cheating monitoring accuracy are improved.

Apparatuses, systems, and techniques are presented to estimate user gaze. In at least one embodiment, one or more neural networks are used to determine coarse and fine gaze estimates for one or more users.

The invention relates to the technical field of mixed reality and artificial intelligence fusion application, in particular to a mixed reality intelligent inspection method and system based on multi-modal fusion. According to the technical scheme, the method comprises the following steps: synchronously acquiring a user view image and voice stream data through mixed reality equipment; performing target detection on the view image by adopting an improved YOLOv10 model, and outputting a candidate target area and a confidence score # imgabs0 # thereof; voice input is transferred through a Whisper voice recognition model, and a semantic vector # imgabs1 # is generated by using a BERT semantic embedding model. According to the invention, through a multi-mode cooperative sensing mechanism of fusing image, voice and space gazing information, in combination with context semantic optimization and lightweight terminal deployment, intelligent target identification and natural interaction with high precision, real-time performance and strong robustness are realized in a complex industrial inspection scene.

The invention discloses a high-altitude falling simulation system based on VR equipment, and relates to the technical field of virtual reality simulation, and the system comprises a data collection module which collects the pupil distance and initial sight line data of a user; the visual disturbance module is used for dynamically adjusting local visual disturbance of a corresponding area in the high-altitude virtual scene based on the gaze data to form a visual disturbance effect; the psychological index calculation module is used for calculating an anxiety index and a focus drift speed by adopting a time sequence convolutional neural network according to the visual disturbance effect and the eye movement behavior of the user to obtain an anxiety index and a focus drift speed index; and the disturbance adjustment module adaptively adjusts the frequency and amplitude of the visual disturbance effect by adopting a method of combining fuzzy logic control and adaptive gain adjustment to obtain disturbance rhythm parameters. According to the method, the personalized matching ability and training effectiveness in psychological behavior intervention and virtual high-altitude adaptation training are improved.

The invention relates to the technical field of intelligent education, in particular to an interactive English teaching method and system based on AI vision, and the method comprises the steps: obtaining the voice and mouth dynamic image frame segments of a student, extracting an offset frame segment to generate a semantic motion disjunction interval, analyzing the time sequence consistency of a behavior signal and a motion cause word triggering frame segment, and calculating the motion matching degree. Time synchronism of the speech output and the visual action is evaluated. According to the invention, through real-time analysis of the voice and mouth dynamic image of the student, the alignment offset of the pronunciation and the mouth action is identified, and in combination with behavior signals such as eye fixation and head rotation of the student, the interactivity between the language action and the behavior of the learner is comprehensively evaluated, so that the efficient alignment of the language expression and the standard template is ensured; by analyzing the distribution frequency and the matching degree of the voice behaviors, a personalized teaching path is optimized. In addition, delay frames and contour offset are deeply analyzed, the synchronism of voice and visual actions is enhanced, and the real-time feedback and interaction effect in the learning process is improved.

The invention discloses a new media promotion method and system based on the Internet, and belongs to the technical field of new media promotion, and the method specifically comprises the steps that the distribution density and the stop position of people in a waiting area are monitored in real time, load distribution data in a lift car are collected, and the vibration frequency and the start-stop state in the elevator running process are recorded; when it is detected that the density of people in the waiting area exceeds a set threshold value, switching to duration compressed advertisement content, and when the load distribution of the lift car displays that one side is crowded, displaying interactive advertisements in the free area; capturing a user eyeball movement track, calculating an advertisement interface fixation thermodynamic diagram, identifying a user gesture operation by using a millimeter wave radar, mapping a specific gesture into an advertisement content switching instruction, and generating a personalized advertisement recommendation scheme; and the advertisement interaction data is converted into a pushing strategy adjustment basis of adjacent elevators, a propagation path is established according to the flow rule of people in the elevator room, and when the advertisement click rate of a certain elevator exceeds a set value, a content recommendation signal is sent to the associated elevator.

The invention belongs to the field of virtual reality teaching application, and provides a teaching atmosphere automatic adaptation method and system in an immersive teaching environment. The method comprises the following steps: acquiring an eye movement image; recognizing a fixation point; carrying out ROI tracking; carrying out ROI boundary fusion; adaptively optimizing the object; adjusting the brightness of the ROI; and watching object interaction. According to the method, the immersion and interactivity of a future classroom can be improved, the use experience of an immersive virtual environment is facilitated, and deep fusion of an intelligent teaching environment and self cognition of a user is promoted.

Various implementations assess user interactions to trigger user interface responses. Some implementations associate user gaze with user interface elements using one or more criteria that enable interpreting the user's gaze in ways that are consistent with user intentions and / or expectations. In some implementations, a user's gaze is associated with a user interface element when the user gazes at the user interface element. When the gaze leaves the user interface element (e.g., intentionally or unintentionally), the association may be maintained or discontinued using one or more criteria to be consistent with user intentions and / or expectations. In some implementations, this involves determining whether to continue associating a user's gaze with a user interface element so that the gaze-to-element association is maintained in certain circumstances (e.g., blinks and saccades) but discontinued in other circumstances (e.g., when the user is fixating outside of a zone around the element for more than a threshold time / number of frames).

The invention relates to the technical field of deep faking attribution and detection methods, in particular to a deep faking attribution and detection method based on a gaze guide CLIP model, and the method specifically comprises the following steps: collecting face images, constructing a deep faking attribution and detection data set, enabling each face image in the data set to have a corresponding faking attribution label and a faking detection label, preprocessing images in the data set, and then dividing into a training set and a test set; constructing a zero-sample deep counterfeit attribution and detection model, inputting the preprocessed data set into the model, and performing model processing to obtain predicted image counterfeit detection category features and predicted image counterfeit attribution category features; and performing optimization training on the deep counterfeit attribution and detection model through the training set to obtain an optimized and trained model, and testing the optimized and trained model by using data in the test set. According to the invention, the source of the deep pseudo image is traced through the deep learning method, and the faked attribution and detection can be carried out more accurately.

The invention discloses a nursing scene data enhancement and generation method for intention recognition model training, and belongs to the technical field of artificial intelligence and smart old-age care. The method aims at solving the problems that in the prior art, high-quality nursing scene training data is deficient, and the obtaining cost is high. According to the core technical scheme, the method comprises the steps that firstly, a staring sequence and other context information (such as time, place and physiological signals) of a user are processed through a multi-modal information fusion model, and a structured initial situation vector is generated; secondly, inputting the vector into a scene generation model combined with a nursing knowledge base, and automatically generating a batch of basic nursing scene data with intention labels; key points are that a data enhancement module is introduced, and a plurality of innovative strategies such as situation element disturbance, virtual physiological data synthesis, gaze path variation and virtual scene deduction are adopted to deeply process basic data, so that the diversity and complexity of a data set are greatly enriched; and finally, combining the basic data with the enhanced data to construct a final comprehensive training data set. According to the method, large-scale and high-fidelity training data can be generated in a low-cost and high-efficiency manner, and the accuracy and robustness of the intention recognition model in a real nursing environment are remarkably improved.

The invention discloses a vehicle atmosphere lamp control method and device based on intention recognition and electronic equipment. The method comprises the steps that fixation point information and expression information of a driver and passengers are obtained through recognition according to face image information; determining interaction feature data according to the fixation point information and the interaction information, determining emotional feature data according to the expression information and the physiological state information, and determining scene feature data according to the driving state information and the vehicle exterior environment information; inputting the interaction feature data, the emotion feature data and the scene feature data into a pre-trained atmosphere lamp control intention recognition model to obtain a target control intention and a corresponding confidence coefficient; and determining target atmosphere lamp parameters according to the target control intention, the confidence coefficient and a pre-constructed atmosphere lamp parameter mapping library, and performing atmosphere lamp control on the current vehicle according to the target atmosphere lamp parameters. According to the invention, the efficiency and accuracy of vehicle atmosphere lamp control are improved, refined and personalized atmosphere lamp control is realized, and the method can be applied to the technical field of intelligent cabins.

Photometric stereo techniques enable using a single camera to perform an enrollment process for creating a user-specific anatomical model of an eye for gaze tracking. The user-specific anatomical model includes information about a user's center of vision at multiple dilation states of the eye, which can be used to enhance the accuracy of gaze tracking techniques. Accurate gaze tracking techniques enable the use of gaze tracking at close range, for example, gaze tracking within a head-mounted display device.

The invention discloses an image processing and training method and device, equipment, a head-mounted display and a medium. Acquiring fixation point information, an original image and a fixation point area image; generating a first input tensor based on the gaze point information, generating a second input tensor based on the gaze point region image, and connecting the first input tensor and the second input tensor to generate a first intermediate tensor; taking the first input tensor as a guide tensor, and performing weight distribution of channel weight and space weight on the first intermediate tensor based on a channel attention mechanism and a space attention mechanism to generate a second intermediate tensor; extracting multi-scale fuzzy features of a preset fuzzy image to generate a third intermediate tensor fuzzy image, and generating the third intermediate tensor fuzzy image based on a pre-calibrated point spread function of each sub-region in the fixation point region; connecting the second intermediate tensor and the third intermediate tensor, and inputting a fourth intermediate tensor generated by the connection into an encoder and a decoder to generate a pre-corrected image after confusion spot correction; and generating a to-be-displayed image based on the pre-corrected image and the original image.

The invention provides a camera focusing method and device for video transparent transmission equipment. Gaze information from the gaze tracking subsystem and depth information from the depth tracking system may be utilized to determine a depth to focus. A combination of depth information and gaze information is used.

An information processing apparatus is an information processing apparatus including a processor, the processor is configured to acquire first image data, output the first image data to a display, acquire visual line data, detect, as a gaze position, a visual line position of a case where a state where a variation distance of the visual line position with respect to a first image indicated by the first image data is equal to or shorter than a first distance continues for a first time or longer based on the visual line data, and set a first region based on two or more gaze positions.

The invention relates to the field of intelligent perception, and mainly relates to a user interaction intention perception method and system based on intelligent glasses, and the method comprises the steps: obtaining the abnormal degree of an eye jump drop point according to the main eye jump and the subsequent correction eye jump times after detecting that the current eye jump drop point of a user enters an interaction region of an interactive object; acquiring a pupil diameter change sequence after the main eye jump, and acquiring a pupil transient response score based on the pupil diameter change sequence; after a certain time from the falling point time of the main eye jump, acquiring a real-time double-variable contour ellipse area based on information of a plurality of latest fixation points, and acquiring a fixation stability score based on the real-time double-variable contour ellipse area; obtaining a real-time gaze residence time according to the falling point time of the main eye jump, and obtaining a residence time evaluation score based on the real-time gaze residence time; fusing the abnormal degree, the pupil instantaneous response score, the stability score and the evaluation score to obtain a dynamic interaction intention score of the user, and determining the interaction intention of the user based on the intention score. The perception accuracy of the user interaction intention is improved.

The invention relates to the technical field of vehicle-mounted human-computer interaction and graphic rendering, and discloses a method and a device for optimizing picture fluency of a vehicle-mounted display screen. The method comprises the following steps: collecting multi-source data and preprocessing the multi-source data to obtain a denoised sight line position and light intensity data; fusing the data and determining real-time sight focus coordinates of the driver through light correction; if the focus is located in the navigation area, preferentially calculating a rendering thread of the focus, judging the residence time and calibrating the locking precision; the navigation area frame rate is improved according to the locking state, and secondary area transition data are prepared; residual computing resources are obtained, a degradation rendering scheme is determined, and resource balanced distribution is achieved; extracting a pre-rendering frame when a total load peak value is detected, and obtaining seamless transition data; and finally, injecting the pre-rendered frame into an output buffer area and fusing the real-time coordinates to generate a smooth picture frame sequence. According to the method, dynamic resource allocation and smooth picture output can be realized, and the vehicle-mounted display fluency and the driving safety are remarkably improved.

The embodiment of the invention provides a control method and device, computer equipment, a computer readable storage medium and a computer program product, and belongs to the field of human-computer interaction. The control method comprises the following steps: acquiring eye movement data of a target object under the condition of running the target application program; determining a target fixation point of the target object on the client screen according to the eye movement data; generating a target control instruction according to the target fixation point; and in response to the target control instruction, controlling the target application program to execute a corresponding target operation. According to the technical scheme, the immersion and the convenience degree of using the target application program by the target object can be improved, and the use experience of the target object is enhanced.

The invention relates to a multi-modal data intelligent labeling system based on computer vision. The system comprises a reference point alignment module used for acquiring a feature extraction result of multi-modal data, performing reference point alignment processing on the feature extraction result, and generating a consistent reference point set; the fixation point prediction module is used for performing dynamic prediction processing on a fixation point in the visual data according to the reference point set to generate a dynamic fixation point position; the modal generalization module is used for performing feature space mapping processing through a cross-modal learning framework according to the dynamic fixation point position to obtain a cross-modal labeling result; and the annotation optimization module is used for carrying out quality evaluation on the annotation result through a statistical model and consistency check to generate an optimized annotation result. By adopting the system, efficient and accurate labeling of multi-modal data can be realized, and the labeling efficiency and quality are remarkably improved.

This disclosure describes, in part, techniques for identifying interactions between users and inventory locations. For instance, system(s) may receive image data generated by an imaging device located within a facility. The system(s) may then analyze the image data in order to determine one or more gaze points, where the gaze point(s) indicate location(s) on an inventory location for which a user was looking over a period of time. The system(s) may then generate gaze data indicating period(s) of time that the user was looking at the portion(s) of the inventory location. In some examples, the gaze data represents a heatmap. The system(s) may then use planogram data to determine identifier(s) of item(s) located at the portion(s) of the inventory location. Additionally, the system(s) may generate metrics data representing at least the identifier(s) of the item(s) and the period(s) of time that the user was looking at the item(s).

A method includes determining an eye gaze location of a user and generating a spatial foveation map based on the eye gaze location. The method also includes receiving an image, forming a spatially foveated image using the image and the spatial foveation map, and transmitting the spatially foveated image to a wearable device. The method further includes spatially defoveating the spatially foveated image to produce a spatially defoveated image and displaying the spatially defoveated image.

The invention provides a head-eye posture fused driver state grading intervention method and system, and the method comprises the steps: dividing a fixation region which comprises a safety region, an alarm region and a dangerous region based on an in-vehicle three-dimensional coordinate system; calculating an initial line-of-sight vector according to the driver face key points, obtaining a final line-of-sight vector by combining line-of-sight deviation angle correction, and obtaining a fixation point coordinate based on the eye point coordinate and the final line-of-sight vector; based on fixation point coordinates and fixation area matching, the driver fixation area category is judged; and acquiring the current vehicle speed and the fixation duration in the fixation area, inputting the current vehicle speed and the fixation duration into a pre-trained decision model to obtain a graded intervention strategy, and executing an intervention action according to the strategy. The accuracy of the fixation point position is ensured through fusion of the head posture and the sight line deviation; a grading intervention strategy is obtained by combining the fixation point position, the fixation duration and the vehicle speed, interference to normal driving is reduced, effective early warning can be achieved in the risk accumulation stage, and the method has adaptability to complex vehicle use scenes.

In some embodiments, a computer system enables a user to invoke display of transport controls (and / or other controls associated with controlling playback of content) using gaze inputs, gesture inputs, or a combination of these. In some embodiments, in response to detecting a first user input, the computer system displays a first set of controls in a reduced-prominence state (e.g., in a manner that is not unduly distracting to the user), and in response to detecting a second user input, the computer system displays a second set of controls in an increased-prominence state (e.g., in a more visually prominent state). The second set of controls optionally includes more controls than the first set of controls.