104 results about "Gaze directions" patented technology

Provided herein are system, apparatus, device, method and / or computer program product embodiments, and / or combinations and sub-combinations thereof, for improving object detection based on gaze direction. In the method, a first image of surroundings of a vehicle is received. A second image of a driver of the vehicle is also received. The second image captured contemporaneously with the first image. Based on the second image, a gaze direction of the driver is determined. Based on the gaze direction, a region of the first image being viewed by the driver is determined. An object detection algorithm is applied to recognize an object in the surroundings of the vehicle at a greater level of detail within the region.

To reliably determine the direction of the subject's gaze. [Solution] The gaze direction detection device 1 comprises a camera 10 that acquires a face image by capturing the face of subject A, and an image processing device 20 that processes the face image acquired by the camera 10 to determine the gaze direction of subject A. The image processing device 20 detects the three-dimensional positions of the right pupil, left pupil, and nose based on the face image, identifies the three-dimensional position of the intersection of a second straight line that passes through the three-dimensional position of the nose and intersects perpendicularly with a first straight line connecting the three-dimensional positions of the right pupil and the three-dimensional positions of the left pupil, and the first straight line, and determines the gaze direction of subject A using the first distance between the three-dimensional position of the intersection and the three-dimensional position of the right pupil, the second distance between the three-dimensional position of the intersection and the three-dimensional position of the left pupil, and the third distance between the three-dimensional position of the intersection and the three-dimensional position of the nose.

This disclosure relates to a gaze estimation method, apparatus, device, and medium. The method includes: acquiring a target eye image corresponding to a target eye whose gaze direction needs to be estimated; wherein the target eye image includes images of the target eye captured by different target cameras at the same time; based on the target eye image, acquiring three-dimensional pupil contour point information and the three-dimensional gaze direction of the target eye using a preset target network model; determining the three-dimensional pupil position of the target eye based on the three-dimensional pupil contour point information; and obtaining a gaze estimation result of the target eye based on the three-dimensional pupil position and the three-dimensional gaze direction of the target eye. This disclosure can effectively ensure the reliability of the final gaze estimation result.

An apparatus for assessing the fine motor skills of a target user includes: a pressure sensing module configured to capture pressure data associated with pressure exerted by small muscles during the performance of the fine motor skill by the target user; a motion tracking module configured to capture motion data associated with movement of the small muscles during the performance of the fine motor skill by the target user; an eye-tracking module configured to capture gaze data associated with the gaze direction of the target user during the performance of the fine motor skill; and a performance evaluation module configured to determine, in real time, one or more metrics associated with the accuracy of the target user's fine motor skills based on the captured motion data, the captured gaze data, and the captured pressure data.

The invention relates to a method for operating an assistance system (2) of a vehicle (1) comprising the steps of: issuing an offer to a driver (14) of the vehicle (1) to perform an automated driving maneuver, wherein a steering system (9) for automated lateral guidance of the vehicle (1) is controlled during the execution of the automated driving maneuver; checking whether the driver (14) has accepted the offer, wherein acceptance describes a turning of the gaze direction (31) and / or the head orientation towards a predetermined area (22, 23, 25, 26); detecting an operating action and / or a body posture of the driver (14) during the acceptance check; judging the acceptance as plausible if the operating action corresponds to a predetermined operating action and / or if the body posture corresponds to a predetermined body posture; and issuing a control signal to perform the automated driving maneuver.if the assumption is deemed plausible.

Embodiments here observe the driver and determine an orientation of an articulating camera based on driver gaze tracking and external object detection. A fixed camera collects image data of the vehicle's surroundings, and an articulating camera collects image data of the driver. The gaze direction is used to identify what objects in the vehicle's surroundings the driver is looking at. Based on the position of the object, an orientation of the articulating camera may be determined.

This invention discloses a vision-electrophysiological multimodal eye-tracking zoom control system and zoom glasses, relating to the field of zoom glasses technology. The zoom optical element has at least one control area. The multimodal sensing unit includes a vision detection module, an EOG detection module, and an ambient light sensing module, used to collect the wearer's eye state information, periocular potential information, and ambient light information, respectively, when the glasses are worn. A cross-modal adaptive calibration unit is used to calibrate the eye state information using periocular potential information when the ambient light information does not meet preset lighting conditions. The focus control unit is used to calculate the eye's gaze direction and gaze distance based on the eye state information, determine the target control area based on the gaze direction, generate focus control parameters based on the gaze distance, and drive the target control area to focus using the focus control parameters. This invention improves zoom performance by coordinating visual signals and EOG signals.

A three-dimensional (3D) display device may comprise a display panel including a plurality of subpixels, a variable focus lens positioned in a front position of the display panel, an optical lens positioned in a front position of the variable focus lens, a gaze detector detecting a gaze of a user positioned in the front position of the display panel to generate a gaze detection signal, an object controller generating an object information signal for an object displayed on the display panel, and a focus control circuit controlling a focus of the variable focus lens based on an emission control signal, the gaze detection signal, and the object information signal.

A steering adjustment device and method are provided to suppress steering operations by the driver in directions in which the driver is not fully aware of the situation during automatic driving. The steering adjustment device detects the driver's gaze direction from a facial image representing the driver's face. When the driver's operation on the steering operation receiving unit is detected during automatic driving control of the vehicle's steering, the device determines whether the driver's gaze direction matches the steering direction of the operation. The device adjusts the reaction force so that the reaction force corresponding to the operation on the steering operation receiving unit when the gaze direction matches the steering direction is less than the reaction force when the gaze direction does not match the steering direction.

This invention discloses an eye feature detection and light source matching method based on an event camera, belonging to the field of eye-tracking detection. The method includes first placing an event camera and a ring-shaped light source assembly in front of the eye being tested, controlling the ring-shaped light source assembly to switch from off to on, and simultaneously acquiring event stream data generated by brightness changes in the eye area through the event camera; constructing a global grayscale image and a light spot grayscale image based on the event stream data, then coarsely locating the pupil contour and extracting candidate corneal reflective spots; then obtaining the precise pupil contour and pupil center position based on the candidate corneal reflective spots; finally, based on a preset light spot geometric distribution model, eliminating abnormal spots among the candidate corneal reflective spots, and identifying the correspondence between each remaining candidate corneal reflective spot and the light source in the ring-shaped light source assembly. This invention achieves pupil and corneal reflective spot detection and light source matching based on event data, providing structured feature input for subsequent eye tracking and gaze direction estimation.

This invention belongs to the field of intelligent tour guide technology. It discloses a method and device for interactive cultural relic tours based on data-real fusion. By collecting environmental parameters within the display case and visitor interaction data, it achieves a personalized experience for cultural relic displays. Based on the intersection of the gaze direction vector and the ray of the cultural relic's 3D model, a gaze heat map of the cultural relic's surface is generated, and areas of interest are identified. Multi-scale decorative feature extraction is performed on the areas of interest, constructing a semantic hierarchy tree of the decorative features. Based on this hierarchy tree and the distribution density of areas of interest, a zone-specific rendering strategy is generated, employing different rendering precision for different areas. Simultaneously, combined with environmental parameters and a holographic optical path refractive index drift model, the rendering results are compensated and corrected in real time. Based on the semantic complexity of the decorative features and visitor interaction behavior, the depth of the explanation content and the language type are determined and presented through an interactive interface. This invention provides visitors with an immersive and personalized cultural relic tour experience.

This display device may comprise: a camera; a panel in which at least a partial area operates as a mirror or a display; a memory storing one or more instructions; and at least one processor including processing circuitry, wherein, when the one or more instructions are executed individually or collectively by the at least one processor, the display device: controls the camera to capture an image of a user to obtain a user image; obtains location information of the user; obtains information about a gaze direction of the user on the basis of the user image; and controls the panel such that at least a partial area of the panel operates as the mirror or the display on the basis of the location information and the information about the gaze direction of the user.

Aspects of the present disclosure include a method comprising receiving a video stream of a user during an online session, initiating a calibration by providing for presentation on a display a calibration element positioned at a first position on the display, detecting a gaze direction and / or head rotation of the user based on the video stream, determining a mapping between the first position and the gaze direction and / or head rotation, initiating a verification check during the session by providing for presentation on the display a visual stimulus element randomly positioned at a second position on the display, determining a change in the gaze direction and / or head rotation during the verification check, and verifying whether the user is in front of the display by determining whether the user reacted to the visual stimulus element based on the mapping, the change, and the second position. The verification check occurs after the calibration.

The present invention relates to a head mountable device (100) comprising an eye-tracking system (200), comprising a first detecting subsystem (210) configured to detect at least one first gaze parameter of a respective eye to be tracked when a user wears the head mountable device (100) so as to generate a first signal, wherein the first detecting subsystem (210) is of a low power monitoring type, and a second detecting subsystem (220) configured to detect at least one second gaze parameter of the respective eye so as to generate a second signal, the second detecting subsystem (220) comprises a plurality of detecting elements (221, 222), a processing and control unit (230) associated to the first detecting subsystem (210) and to the second detecting subsystem (220) configured to estimate a region of interest (400) among a predetermined plurality of regions of interest on the basis of said first signal, wherein the estimated region of interest (400) comprises the direction of the gaze of a respective eye to be tracked, the processing and control unit (230) is configured to selectively activate one or more of the detecting elements (221, 222) on the basis of the estimated region of interest (400) for subsequently estimating the direction of the gaze of the respective eye to be tracked inside the estimated region of interest (400) on the basis of said second signal with optimized hardware and computing energy consumption.

The present invention relates to a system, server, device, and method for calculating risk assessment indicators, wherein a wearable image analysis device worn by a user of a means of transportation or mounted to be synchronized with the user's gaze direction, or an image-based risk assessment device linked with a vehicle camera, black box, vehicle-mounted ADAS camera, mobile device, on-board unit (OBU), vehicle-to-everything (V2X) terminal, etc., identifies a driving context from a front image, location information, sensor information, or vehicle status information, determines a risk event according to differential safety standards for each context and outputs a warning, and measures response data including whether to return to a normal state, the time required for return, the number of warning repetitions, the duration of ignoring, and the amount of behavioral change based on the time of warning output or the time of risk event confirmation. The above system generates risk assessment indicators, behavior-based safety scores, or premium rate calculation factors by cumulatively analyzing risk event data and response data, and reflects warning responsiveness independently of the frequency of risk event occurrences. Furthermore, it can enhance the precision and fairness of risk assessment by utilizing warning-free driving indicators, Time to Collision (TTC) correction models, multimodal dynamic weights, adverse weather preprocessing, individual baseline correction, and user-level continuous responsiveness profiles. In addition, the present invention can reduce data tampering and personal information exposure through the calculation of complete indicators within the device, the output of interaction data packets, verification of event packet integrity based on a Trusted Execution Environment (TEE), dual trajectory cross-verification, and on-device anonymization, and depending on the embodiment, it can also be applied to the calculation of risk assessment indicators based on wearable devices, mobile devices, or automotive devices.

This invention discloses a method for joint estimation of gaze direction and gaze region in a monocular vehicle-mounted camera based on geometrically normalized feature fusion. The method includes the following steps: acquiring a driver's face image captured by a monocular vehicle-mounted camera; obtaining the original representation and the geometrically normalized representation; performing multi-scale feature extraction on both the original and geometrically normalized representations to obtain original multi-scale features and normalized multi-scale features; performing feature fusion on the original and normalized multi-scale features to obtain a fused feature representation; performing joint estimation based on the fused feature representation to output the driver's gaze direction estimation result and gaze region classification result; and introducing geometric consistency constraints to correct or optimize the driver's gaze direction estimation result and gaze region classification result to obtain the final output result. This invention improves the accuracy and stability of gaze direction and gaze region classification in monocular vehicle-mounted scenes, and has strong engineering practical value and promising application prospects.

A method for determining the attention of a driver of a vehicle, wherein: - the driver's gaze direction is determined; - an attention value is continuously determined that is representative of the driver's attention to a predefined area, wherein the attention value is determined by: - ​​determining, depending on the determined gaze direction, whether the driver is looking into the predefined area; - if it is determined that the driver is looking into the predefined area, determining the attention value using a predefined loading function, where the loading function is a predefined exponential function; and - if it is determined that the driver is not looking into the predefined area, determining the attention value using a predefined unloading function.

This invention discloses an adaptive adjustment method and system for a virtual environment, relating to the field of virtual reality technology. The method includes: performing background subtraction processing on a target image sequence when a user is experiencing a virtual environment; extracting detailed features from the target image sequence, and identifying key points using target anchor boxes based on the detailed feature information; determining the coordinates of three-dimensional key points based on the target key point information for user action behavior analysis; extracting a user eye image sequence from the target image sequence to determine eyeball image information, and analyzing the gaze direction based on the eyeball image information; analyzing the user's attention focus using image brightness distribution based on the gaze direction information; and adjusting elements of the virtual environment based on the user's action behavior information and user attention focus information. This invention enables more fine-grained virtual environment adjustment, improving the quality and naturalness of the user's immersive experience in the virtual environment.

Provided herein are system, apparatus, device, method and / or computer program product embodiments, and / or combinations and sub-combinations thereof, for predicting a driver path of a vehicle. In the method, an image of a driver of the vehicle is received. Based on the image, a gaze direction of the driver is determined. Based at least in part on the gaze direction, a path of the vehicle is predicted. The vehicle is controlled based on the predicted path.

Various implementations disclosed herein include devices, systems, and methods that track an eye characteristic (e.g., gaze direction, eye orientation, etc.). For example, an example process may include producing light that reflects off a retina of an eye, receiving an image of a portion of the retina from an image sensor, the image corresponding to a plurality of reflections of the light scattered from the retina of the eye, obtaining a representation of the eye corresponding to a first accommodative state, the representation representing at least some of the portion of the retina, and tracking an eye characteristic based on a comparison of the image of the portion of the retina with the representation of the eye.

This disclosure relates to a gaze estimation method, apparatus, device, and medium. The method includes: acquiring a target eye image corresponding to a target eye whose gaze direction needs to be estimated; the target eye image is an image of the target eye captured by a target camera; acquiring intrinsic and extrinsic parameters of the target camera, and obtaining camera information based on the intrinsic and extrinsic parameters; performing at least one target processing operation based on the target eye image and the camera information, and obtaining a gaze estimation result of the target eye image based on the output information of the last target processing operation; the target processing includes: extracting key target information based on the input information corresponding to the target processing, and fusing the key target information and specified information from the input information corresponding to the target processing to obtain the output information corresponding to the target processing. This disclosure helps to obtain more realistic and reliable gaze estimation results to better meet requirements.

The application discloses a kind of intelligent zoom liquid crystal glasses systems based on time eyeball tracking positioning, at least including variable optical element, eye movement perception unit and focusing control unit, variable optical element is set in the optical imaging path of glasses, with at least one control area;Eye movement perception unit is used to collect the eyeball state information of wearer when wearing glasses;Focusing control unit is used to calculate the gaze direction and gaze distance of eyeball according to the eyeball state information, determine target control area according to the gaze direction, generate focusing control parameter according to the gaze distance, and drive target control area to focus through the focusing control parameter.The application continuously models binocular eye movement information under time sequence, realizes automatic focusing control without relying on external ranging device, reduces the burden of eye adjustment, and improves visual clarity and wearing comfort.

A head-mounted device may have a head-mounted support structure. Gaze tracking systems may be supported by the support structure so that the gaze of a user may be monitored. Lenses may be supported by the support structure. Display systems may provide computer-generated images to the user while the user is viewing real-world objects through the lenses. The gaze tracking systems may include image-sensor-based systems such as glint-based systems. A glint-based gaze tracking system may include light-emitting devices that emit light beams that create eye glints on the surface of a user's eyes and may include an image sensor that measures the eye glints to gather information on the user's gaze. A low-power gaze tracking system may be included in the head-mounted device. The low-power gaze tracking system may use light detectors to measure the magnitudes of respective light reflections of the light beams.

A method for operating an operating system (12), in particular of a motor vehicle (10), which comprises an input field (14) with several touch-sensitive control surfaces (16), comprising the steps of: - detecting an area in which a user (20) touches the input field (14); - determining, based on the detected area, which of the control surfaces (16) the user (20) is touching; - detecting the gaze direction (26) of the user (20); - determining, based on the detected gaze direction (26), which of the control surfaces (16) the user (20) is looking at; - selecting one of the control surfaces (16) of the input field (14) taking into account the information about which of the control surfaces (16) the user (20) is touching and which of the control surfaces (16) the user (20) is looking at; - executing a function assigned to the selected control surface (16), characterized in that - if it is detected that none of the control surfaces (16) are being touched by the user (20) is touched,the control surface (16) is selected for which it has been determined that the user (20) has directed their gaze, and – if it is detected that the control surface (16) touched by the user (20) and the control surface (16) on which the user (20) has directed their gaze differ, – the control surface (16) on which the user (20) has directed their gaze is selected only if the control surface (16) touched by the user (20) and the control surface (16) on which the user (20) has directed their gaze are no further apart than a predetermined distance (28), and – the control surface (16) touched by the user (20) is selected if the control surface (16) touched by the user (20) and the control surface (16) on which the user (20) has directed their gaze are further apart than the predetermined distance (28).

Provided herein are system, device, apparatus, method, and / or computer program product embodiments, and / or combinations and sub-combinations thereof, for predicting a driver path of a vehicle. In the method, an image of a driver of the vehicle is received. Based on the image, a gaze direction of the driver is determined. Based at least in part on the gaze direction, a path of the vehicle is predicted. The vehicle is controlled based on the predicted path.

The application provides a multi-stage semantic emotion-driven digital human action generation method, and relates to the technical field of digital human action generation. The application extracts audio features and text semantic features and completes time alignment and fusion through multi-modal preprocessing of input audio and text to obtain a multi-modal feature sequence. Then, the sequence is used to identify semantic intent and analyze emotion to obtain a semantic intent sequence and an emotion curve containing categories, intensity and change trend. Subsequently, a global action script containing action type, intensity, duration and priority is generated according to a preset action mapping rule. The script is encoded into a feature vector and input into a lightweight neural network to obtain an action parameter sequence containing facial expressions, mouth shapes, skeletal actions and gaze directions. After time sequence smoothing and physical constraint checking, the final digital human driving parameters are output, which can realize low-power real-time operation, smooth action, accurate mouth shape synchronization and emotion and semantic matching.

Provided is a binocular visual function system for evaluating binocular single vision. The system comprises: a line-of-sight guiding unit that directs line-of-sight directions of the left eye and the right eye of a subject in a second direction different from a first direction while fixing the head of the subject in the first direction; and a processing unit that presents images spatially separately to the left eye and the right eye of the subject the line-of-sight directions of which have been guided in the second direction by the line-of-sight guiding unit. The images has feature amounts as indices indicating visual characteristics. The feature amount of a left eye image, which is the image presented to the left eye, is different from the feature amount of a right eye image, which is the image presented to the right eye.

The invention relates to a method for operating an assistance system (2) of a vehicle (1), comprising the steps of: outputting, to a driver (14) of the vehicle (1), an offer to carry out an automated driving manoeuvre, wherein, when the automated driving manoeuvre is carried out, a steering system (9) is actuated for automated lateral guidance of the vehicle (1); checking whether the driver (14) has accepted the offer, wherein the acceptance is represented by a turning of the gaze direction (31) and / or of the head orientation towards a predetermined region (22, 23, 25, 26); detecting an operating action and / or a body posture of the driver (14) while the acceptance is being checked; assessing the acceptance as plausible if the operating action corresponds to a predetermined operating action and / or if the body posture corresponds to a predetermined posture; and outputting a control signal for carrying out the automated driving manoeuvre if the acceptance is assessed as plausible.

Various implementations disclosed herein include devices, systems, and methods that modify a video to enable replay of the video with a depth-based effect based on gaze of a user during capture or playback of the video. For example, an example process may include tracking a gaze direction during a capture or a playback of a video, wherein the video includes a sequence of frames including depictions of a three-dimensional (3D) environment, identifying a portion of the 3D environment depicted in the video based on the gaze direction, determining a depth of the identified portion of the 3D environment, and modifying the video to enable replay of the video with a depth-based effect based on the depth of the identified portion.