369 results about "Gaze directions" patented technology

The invention provides a driving behavior intelligent identification method and system based on video analysis, and the method comprises the steps: obtaining a driver face video stream and a road environment video stream collected by a vehicle-mounted camera, and reading the driving information recorded by a whole vehicle communication network; recognizing an eyelid closing state, a sight line direction and a head posture in the driver face video stream based on a posture recognition model, and performing fatigue distraction analysis to obtain driver state information; performing motion trail analysis on the road environment video stream and the driving information, and performing driving risk assessment in combination with the driver state information to obtain driving assessment information; and performing early warning construction according to the driving evaluation information, generating early warning prompt information, and synchronously writing the early warning prompt information, the driving evaluation information and the driver state information into a safety data protection memory. The fatigue and distraction states of the driver can be recognized more accurately, and the accuracy of state judgment is improved.

Various implementations disclosed herein include devices, systems, and methods that initiate an action based on a detected gaze direction oriented towards a target area (e.g., a hot corner / zone). For example, an example process may include producing a reflection by directing light towards an eye using an illuminator, receiving sensor data from a sensor, wherein a direction of sensing by the sensor (e.g., the optical axis of the camera) and a direction from the eye to a target area are approximately aligned, determining a reflective property (e.g., a spectral property) of the reflection based on the sensor data, detecting that a gaze direction of the eye is approximately oriented towards the target area (e.g., a hot zone / spot) based on the reflective property, and initiating an action based on detecting that the gaze direction is approximately oriented towards the target area.

An electronic device that includes a display and an eye tracker configured to collect eye tracking data regarding a gaze of one or more of a user's eyes across the display. The electronic device also includes processing circuitry that is operatively coupled to the display and configured to generate pixel data for frames of content based on the eye tracking data such that the content is configured to be shown on the display in a dynamically foveated manner. The processing circuitry is configured to determine dither blocks, each of which corresponds to a subset of the plurality of pixels. The processing circuitry is configured to apply a dither pattern to the frames of the plurality of frames of content independent of the gaze of one or more of the user's eyes and based on the dither blocks.

An audiovisual system uses a spatial position detection module to process audio signals and determine the location and orientation of a speaking participant. Based on this data, the system dynamically controls sensors to optimize audio and video capture. Behavioral and contextual information may also be used to train intelligence models for improved system performance. Further, sensor arrays may be utilized to identify gaze vectors of participants to select a camera sensor of the sensor array. Meeting content collected with the sensor array may be organized into meeting metrics in accordance with an analytics strategy before training an intelligence module with the meeting metrics. Meeting content may also be configured into digital tiles in accordance with a tile strategy. At least one of the digital tiles may be altered in response to a meeting condition detected by the sensor array.

A light-transmitting substrate is deployed with a first of two major surfaces in facing relation to an eye of a viewer and guides light by internal reflection between the two major surfaces. An optical coupling-out configuration couples image light, that corresponds to a collimated image and that is guided by internal reflection between the two major surfaces, out of the light-transmitting substrate. A first optical coupling configuration collimates light from the eye to produce collimated light, and couples the collimated light into the light-transmitting substrate for guiding by internal reflection. A second optical coupling configuration couples the collimated light out of the light-transmitting substrate toward an optical sensor that senses the coupled-out light. A processing system derives current gaze direction of the eye by processing signals from the optical sensor.

The present invention relates to a blind spot display device for construction equipment including a detector provided in the construction equipment to detect an object around the construction equipment; a display provided in front of eyes of a driver of the construction equipment; a gaze tracker tracking a direction of a gaze of the driver; and a controller acquiring information from the detector and the gaze tracker and analyzing information on a direction consistent with the gaze of the driver, wherein the controller displays a marker for the object included in the direction of the gaze of the driver on the display.

A retinal projection display system includes at least one visible light source for projecting a visible light image, an infrared light source for projecting infrared light, a scanning mirror having a field of view larger than the visible light image, a reflective surface on which the visible light image is projected and on which the infrared light is reflected at least partially towards an eye of a user, wherein the reflective surface is larger than the visible light image, at least one infrared photodetector for receiving reflected infrared light that reflects off of the eye of the user, and a hardware computation module comprising a processor and a memory, the hardware computation module configured to determine a gaze direction of the user based at least in part on the reflected infrared light.

Eye tracking is performed by determining an initial pupil position of a user in relation to a lens situated in front of the user, detecting a change in pupil position in relation to the lens to an updated pupil position in relation to the lens, and determining that the updated pupil position is outside a bounding box associated with the lens. The updated pupil position is a replacement pupil position with a replacement pupil position within the bounding box associated with the lens, and the updated pupil position is utilized for eye-tracking functionality. Eye tracking is also performed by determining that a first pixel associated with a gaze direction is outside a visibility region, identifying a replacement pixel within the visibility region, determining an updated gaze angle based on the replacement pixel, and performing eye tracking using the updated gaze angle.

The invention provides an iris collection method and device based on eye movement prediction, equipment and a storage medium, and the method comprises the steps: collecting image frames and event streams in a mixed manner through an event camera and an RGB camera, obtaining eye feature point coordinates through key point detection, and converting the event streams into an event frame sequence; performing spatial-temporal feature extraction and filtering processing on the eye feature point coordinates and the event frame sequence by using a time sequence prediction network, and predicting eyeball positions and gazing directions at future moments; calculating the optimal optical axis direction of iris imaging according to the predicted eyeball position and gazing direction, and converting the optimal optical axis direction into control parameters such as a pan-tilt angle and a lens focal length of an imaging system; and synchronously adjusting the imaging system and triggering acquisition when the prediction time arrives. According to the method, active iris tracking acquisition is realized by predicting the eyeball movement track, the problem of response lag of traditional passive tracking is solved, and the success rate of iris acquisition is improved.

Provided herein are system, apparatus, device, method and / or computer program product embodiments, and / or combinations and sub-combinations thereof, for improving object detection based on gaze direction. In the method, a first image of surroundings of a vehicle is received. A second image of a driver of the vehicle is also received. The second image captured contemporaneously with the first image. Based on the second image, a gaze direction of the driver is determined. Based on the gaze direction, a region of the first image being viewed by the driver is determined. An object detection algorithm is applied to recognize an object in the surroundings of the vehicle at a greater level of detail within the region.

An adaptive display buffering method for enhanced visual comfort includes: original display data is obtained from an off-screen display buffer; an encoded vector is generated based on the original display data, and content feature scoring is performed based on the encoded vector; content features of the original display data are modified based on the content feature scoring and a user gaze direction, and a content regulator applies different modification methods to a foveal region and a peripheral region; and modified display data is stored into an on-screen display buffer. An adaptive display buffering system for enhanced visual comfort is also provided.

To reliably determine the direction of the subject's gaze. [Solution] The gaze direction detection device 1 comprises a camera 10 that acquires a face image by capturing the face of subject A, and an image processing device 20 that processes the face image acquired by the camera 10 to determine the gaze direction of subject A. The image processing device 20 detects the three-dimensional positions of the right pupil, left pupil, and nose based on the face image, identifies the three-dimensional position of the intersection of a second straight line that passes through the three-dimensional position of the nose and intersects perpendicularly with a first straight line connecting the three-dimensional positions of the right pupil and the three-dimensional positions of the left pupil, and the first straight line, and determines the gaze direction of subject A using the first distance between the three-dimensional position of the intersection and the three-dimensional position of the right pupil, the second distance between the three-dimensional position of the intersection and the three-dimensional position of the left pupil, and the third distance between the three-dimensional position of the intersection and the three-dimensional position of the nose.

This disclosure relates to a gaze estimation method, apparatus, device, and medium. The method includes: acquiring a target eye image corresponding to a target eye whose gaze direction needs to be estimated; wherein the target eye image includes images of the target eye captured by different target cameras at the same time; based on the target eye image, acquiring three-dimensional pupil contour point information and the three-dimensional gaze direction of the target eye using a preset target network model; determining the three-dimensional pupil position of the target eye based on the three-dimensional pupil contour point information; and obtaining a gaze estimation result of the target eye based on the three-dimensional pupil position and the three-dimensional gaze direction of the target eye. This disclosure can effectively ensure the reliability of the final gaze estimation result.

An apparatus for assessing the fine motor skills of a target user includes: a pressure sensing module configured to capture pressure data associated with pressure exerted by small muscles during the performance of the fine motor skill by the target user; a motion tracking module configured to capture motion data associated with movement of the small muscles during the performance of the fine motor skill by the target user; an eye-tracking module configured to capture gaze data associated with the gaze direction of the target user during the performance of the fine motor skill; and a performance evaluation module configured to determine, in real time, one or more metrics associated with the accuracy of the target user's fine motor skills based on the captured motion data, the captured gaze data, and the captured pressure data.

Disclosed is a method, system and device for estimating the three-dimensional gaze direction of a human eye based on appearance. The method comprises: acquiring an image or video containing a face; performing face detection in the image or video frame, introducing a face key point detection algorithm to detect and locate the face key points; cutting out the face image and left and right eye images according to the face detection result and the positions of the key points; and estimating the three-dimensional gaze of the human eye in the image using a gaze estimation model, wherein the training process of the gaze estimation model comprises: using a Base-CNNs network to extract features of the face image to obtain a full-face image feature vector, using two asymmetric branch structures of an EE-Net network to extract features of the left and right eye images to obtain double-eye image feature vectors, estimating a head posture vector in combination with the face key point detection algorithm, and finally using a full connection layer to regress the full-face image feature vector, the double-eye image feature vector and the head posture vector for the final gaze estimation task.

The invention relates to a method for displaying the space behind a vehicle on demand, comprising the following method steps: A) Taking multiple rear-view images behind a vehicle, B) Analysis of rear-view images, identifying objects and / or situations relevant to driving safety, C) Capturing the eyes of the driver of the vehicle, D) Analysis of the eye-space images, whereby the driver's gaze direction and the gaze duration associated with the respective gaze direction are identified, E) Display of the area behind the vehicle on a display element, wherein the rear-view images are displayed on the display element when, in process step B), relevant objects and / or situations are detected for which a rated hazard potential exceeds a limit value, or when the viewing duration detected in process step D), with the driver's gaze directed towards the display element, is longer than a limit viewing duration. The invention also relates to a display device and a vehicle with a display device.

A method for detecting an emotional state of a user includes obtaining a first data stream indicative of facial appearance and gaze direction of the user as the user is viewing a scene, determining, based on the first data stream, facial expression feature information indicative of emotional facial expression of the user, obtaining a second data stream indicative of visual content in a field of view of the user, determining, based on the second data stream, visual feature information indicative of visual content in the scene, determining emotional state information based on analyzing the facial expression feature information determined based on the first data stream and the visual feature information determined based on the second data stream, and performing an operation with respect to the emotional state information, wherein the emotional state information is indicative of the emotional state of the user.

The invention relates to a method for operating an assistance system (2) of a vehicle (1) comprising the steps of: issuing an offer to a driver (14) of the vehicle (1) to perform an automated driving maneuver, wherein a steering system (9) for automated lateral guidance of the vehicle (1) is controlled during the execution of the automated driving maneuver; checking whether the driver (14) has accepted the offer, wherein acceptance describes a turning of the gaze direction (31) and / or the head orientation towards a predetermined area (22, 23, 25, 26); detecting an operating action and / or a body posture of the driver (14) during the acceptance check; judging the acceptance as plausible if the operating action corresponds to a predetermined operating action and / or if the body posture corresponds to a predetermined body posture; and issuing a control signal to perform the automated driving maneuver.if the assumption is deemed plausible.

Embodiments here observe the driver and determine an orientation of an articulating camera based on driver gaze tracking and external object detection. A fixed camera collects image data of the vehicle's surroundings, and an articulating camera collects image data of the driver. The gaze direction is used to identify what objects in the vehicle's surroundings the driver is looking at. Based on the position of the object, an orientation of the articulating camera may be determined.

A method (800) of displaying information (400-406, 500-512) on a head-up display (114) of a vehicle (100) comprises determining (802) a direction of gaze (204, 206) of a driver (200) of the vehicle (100) and selecting (804) a display mode from a group of display modes based on the direction of gaze (204, 206). Each display mode of the group of display modes defines information (400-406, 500-512) to be displayed on the head-up display (114). The group of display modes comprises at least a normal mode and an extended mode. Information (500-512) displayed in the extended mode differs from information (400-406) displayed in the normal mode. The method further comprises displaying the information (400-406, 500-512) on the head-up display (114) according to the selected display mode.

Disclosed is a display apparatus with at least one display or projector; a gaze-tracking means; and at least one processor configured to process gaze-tracking data, collected by the gaze-tracking means, to determine a gaze direction of a user; identify a gaze region and a peripheral region within an image that is to be displayed by the at least one display or projector, based on the gaze direction; apply at least one image restoration technique on the image in an iterative manner such that M iterations of the at least one image restoration technique are applied on the gaze region, and N iterations of the at least one image restoration technique are applied on the peripheral region, M being different from N; and control the at least one display or projector to display the image having the at least one image restoration technique applied thereon.

An electronic device is provided. The electronic device includes a display, memory, including one or more storage media, storing instructions, and one or more processors communicatively coupled to the display and the memory, wherein the instructions, when executed by the one or more processors, cause the electronic device to detect a first event while a first screen is displayed, display a first user interface including information related to a first function corresponding to the first event on a portion of the first screen, identify a direction of a gaze of a user, while the first user interface is displayed, display a second user interface, based on the direction of the gaze corresponding to a location of the first user interface, and determine whether to execute the first function corresponding to the first event, based on a first user input detected while the second user interface is displayed.

To support elimination of a gap in awareness and recognition among participants who engage with information in a situation where the information is provided in real time. An information providing device has a configuration including: a communication device configured to perform data communication with another apparatus; and an arithmetic device configured to execute processing of acquiring, from an observation unit configured to observe a situation in which information is explained by a provider of the information, information regarding an explanation position of the provider in the information, processing of acquiring, from a gaze analysis unit configured to estimate a gaze direction of a receiver who receives the explanation of the information, information regarding a gaze position of the receiver, and processing of specifying and outputting an information exchange gap between the provider and the receiver based on the information regarding each of the explanation position and the gaze position.

In one implementation, a method of processing input for multiple devices is performed by a first electronic device one or more processors and non-transitory memory. The method includes determining a gaze direction. The method includes selecting a target electronic device based on determining that the gaze direction is directed to the target electronic device. The method includes receiving, via an input device, one or more inputs. The method includes processing the one or more inputs based on the target electronic device.

An electronic device, a method and a computer program product provide techniques for automating self-image capturing, via a camera, when the device is in an always on camera mode. The method includes determining if a captured image stream contains a face of a user. In response to determining the image stream contains the face of the first user, the method includes determining whether an eye gaze direction of the user corresponds to a direction of the camera. In response to determining the user is looking towards the direction of the camera, the method includes determining if the electronic device is in an orientation that corresponds with the user taking a self-image. In response to determining the electronic device is in a selfie-taking orientation, while the user is looking towards the direction of the camera, the method includes autonomously capturing, via the camera, a self-photo image and storing the self-photo image.

A virtual eye test can be conducted to evaluate peripheral vision in a virtual reality (VR) environment. The test can be conducted using an electronic device that includes a head-mounted display (HMD) and a camera. The electronic device can generate a VR user interface corresponding to a three-dimensional virtual environment and render the VR user interface on the HMD. The electronic device can simulate one or more spatial task scenarios and while simulating these scenarios, in real time, track gaze direction and peripheral responses to one or one or more using the camera. The device can then evaluate the gaze direction and peripheral responses for assessing peripheral vision performance.

A head-mounted device may have a head-mounted support structure. Gaze tracking systems may be supported by the support structure so that the gaze of a user may be monitored. Lenses may be supported by the support structure. Display systems may provide computer-generated images to the user while the user is viewing real-world objects through the lenses. The gaze tracking systems may include image-sensor-based systems such as glint-based systems. A glint-based gaze tracking system may include light-emitting devices that emit light beams that create eye glints on the surface of a user's eyes and may include an image sensor that measures the eye glints to gather information on the user's gaze. A low-power gaze tracking system may be included in the head-mounted device. The low-power gaze tracking system may use light detectors to measure the magnitudes of respective light reflections of the light beams.

Examples relate to a system (110), a method, and a computer program for a surgical imaging system (100), and to a corresponding surgical imaging system comprising such a system for a surgical imaging system. The system (110) is configured to obtain an eye-tracking sensor signal from an eye-tracking system (130). The system is configured to determine a gaze of a user of a surgical imaging device (120) of the surgical imaging system on a portion of a field of view of the surgical imaging device based on the eye-tracking sensor signal. The system is configured to adjust the field of view of the surgical imaging device based on the gaze of the user on the portion of the field of view.

This invention discloses a vision-electrophysiological multimodal eye-tracking zoom control system and zoom glasses, relating to the field of zoom glasses technology. The zoom optical element has at least one control area. The multimodal sensing unit includes a vision detection module, an EOG detection module, and an ambient light sensing module, used to collect the wearer's eye state information, periocular potential information, and ambient light information, respectively, when the glasses are worn. A cross-modal adaptive calibration unit is used to calibrate the eye state information using periocular potential information when the ambient light information does not meet preset lighting conditions. The focus control unit is used to calculate the eye's gaze direction and gaze distance based on the eye state information, determine the target control area based on the gaze direction, generate focus control parameters based on the gaze distance, and drive the target control area to focus using the focus control parameters. This invention improves zoom performance by coordinating visual signals and EOG signals.

A system and method for combining short-wave infrared (SWIR) imaging with image processing algorithms to measure pupil size dynamics in closed eye conditions, either in response to an external stimulus (such as light-induced PLR) or occurring spontaneously during sleep or anesthesia. The system, with deep learning-based analysis, can further estimate gaze direction during eye movements and achieve robust pupillometry. The system has at least one electromagnetic radiation source for radiating electromagnetic energy in a spectral range of 700 nm to 2,000 nm, and an imaging device for detecting electromagnetic energy and sending image data to a processor which performs algorithmic and deep- learning based analysis of image data to continuously track changes in pupil diameter and gaze direction over time.