Wearable head-mounted eye movement tracking device, method and system, and processing device

Abstract

A wearable head-mounted eye movement tracking device, method and system, and a processing device. The wearable head-mounted eye movement tracking device comprises an outer device frame (10), wherein an opening is formed in the middle of the outer device frame (10), the opening vertically penetrates the outer device frame (10), and two sides of the outer device frame (10) are respectively provided with ear supports (11) which protrude downwards, so as to fix the outer device frame (10) on the head of a dog; and the wearable head-mounted eye movement tracking device further comprises at least one eyeball tracking camera (13), wherein the eyeball tracking camera (13) is used for acquiring an eye image which includes an eye of the dog, and the eye image is configured to be sent to a processing device which is in communication connection with the wearable head-mounted eye movement tracking device, such that the processing device performs recognition on the eye image and obtains a target pupil diameter of the eye of the dog. The wearable head-mounted eye movement tracking device can track, without spatial restrictions, eye information of large-sized dogs which have relatively great freedom of movement, thereby improving the accuracy of collected eye information of dogs.

Description

Wearable head eye tracking device, method, processing device and system Technical Field

[0001] The present application relates to the field of life science technology, and in particular to a wearable head eye tracking device, method, processing device and system. Background Art

[0002] A dog's eyes can reveal a lot of information. For example, a sudden dilation of an animal's pupils can indicate fear. Therefore, monitoring changes in eye information (i.e., eye tracking) is extremely important. Currently, eye trackers are commonly used to track dog eyes.

[0003] However, most dogs are large in size and highly mobile. The eye trackers used in related technologies can either only detect dogs in a fixed state, or can only detect dogs with fixed cameras deployed in a preset space. Either method limits the dog's free range of movement, subjecting the dog to greater spatial restrictions. Therefore, it is impossible to know the dog's eye information in various scenarios, affecting the accuracy of the collected dog eye information.

[0004] Summary of the Invention

[0005] The main purpose of the embodiments of the present application is to propose a wearable head eye tracking device, method, processing device and system, which can track the eye information of large dogs with greater freedom of movement without spatial restrictions, thereby improving the accuracy of the collected dog eye information.

[0006] To achieve the above objectives, a first aspect of an embodiment of the present application provides a wearable head eye tracking device, comprising:

[0007] A device frame, wherein an opening is provided in the middle of the device frame, the opening being continuous from top to bottom, and downwardly protruding ear supports are provided on both sides of the device frame, the ear supports being used to be mounted on the ears of a dog to secure the device frame to the dog's head, and a protruding camera support is also provided on the front side of the device frame;

[0008] at least one eye-tracking camera, disposed on the camera bracket, and configured to photograph a dog wearing the wearable head eye-tracking device to obtain an eye image including the dog's eyes;

[0009] The eye image is used to be sent to a processing device that is communicatively connected to the wearable head eye tracking device, so that the processing device can identify the pupil of the dog's eye in the eye image and extract the target pupil diameter of the dog's eye.

[0010] In some embodiments, the device frame is provided with an adjustable headband, the headband is arranged around the opening, and the headband is provided with a fixing component, and the fixing component is used to fix the adjusted size of the headband.

[0011] In some embodiments, the camera bracket is a small-volume bracket, and the eye-tracking camera is provided at an end of the camera bracket away from the device frame, and an infrared device is provided inside the eye-tracking camera.

[0012] In some embodiments, the device frame is further provided with an elastic contact surface, and the elastic contact surface is arranged around the headband so that the device frame contacts the dog when fixed to the dog's head.

[0013] In some embodiments, the outer frame of the device is further provided with a panoramic camera, which is aligned with the direction of the dog's eyes to record the dog's movement trajectory and external environment information corresponding to the eye image.

[0014] To achieve the above objectives, a second aspect of an embodiment of the present application provides a wearable head eye tracking method, which is applied to a processing device, wherein the processing device is communicatively connected to a wearable head eye tracking device, wherein the wearable head eye tracking device includes a device frame and at least one eye tracking camera, wherein an opening is provided in the middle of the device frame, the opening being vertically through-hole, and downwardly protruding ear supports are provided on both sides of the device frame, the ear supports being configured to be mounted on the ears of a dog to secure the device frame to the dog's head, and a protruding camera support is further provided on the front side of the device frame, the eye tracking camera being provided on the camera support and configured to photograph the dog wearing the wearable head eye tracking device to obtain an eye image including the dog's eyes;

[0015] The method comprises:

[0016] Acquire the eye image sent by the wearable head eye tracking device;

[0017] The pupil of the dog's eye in the eye image is identified, and a target pupil diameter of the dog's eye is extracted.

[0018] In some embodiments, identifying the pupil of the canine eye in the eye image and extracting the target pupil diameter of the canine eye includes:

[0019] Inputting the eye image into a pre-trained pupil recognition model to extract first diameter information of the canine eye in the eye image;

[0020] performing image segmentation processing on the eye image according to a preset segmentation threshold, and extracting second diameter information of the canine eye in the eye image;

[0021] The target pupil diameter of the canine eye is obtained by fusing the first diameter information and the second diameter information.

[0022] In some embodiments, inputting the eye image into a pre-trained pupil recognition model to extract first diameter information of the canine eye in the eye image includes:

[0023] Inputting the eye image into a pre-trained deep learning model to obtain the pupil center point of the eye image;

[0024] Taking the pupil center point as a reference, randomly select a pupil outer circle feature point in the pupil area, determine other pupil outer circle feature points according to a preset interval angle, and repeat this operation until the number of pupil outer circle feature points reaches a preset number;

[0025] A first ellipse fitting process is performed on the eye image according to the pupil center point and the pupil outer circle feature points to obtain first diameter information of the dog's eye in the eye image.

[0026] In some embodiments, performing image segmentation processing on the eye image according to a preset segmentation threshold to extract the second diameter information of the canine eye in the eye image includes:

[0027] Segmenting the eye image into a plurality of sub-images, and determining that the sub-image is a canine eye image set if any of the sub-images is within a preset segmentation threshold;

[0028] A second ellipse fitting process is performed on the canine eye image set to obtain second diameter information of the canine eyes in the eye images.

[0029] In some embodiments, obtaining a target pupil diameter of a canine eye based on the fusion of the first diameter information and the second diameter information includes:

[0030] Calculating a first noise of the first diameter information, and determining a first state vector and a first covariance matrix corresponding to a current moment according to the first noise;

[0031] Based on the first diameter information, the first state vector and the first covariance matrix, obtaining the first state vector and the first covariance matrix corresponding to the next moment after the update;

[0032] Calculating a second noise of the second diameter information, and determining a second state vector and a second covariance matrix corresponding to a current moment according to the second noise;

[0033] Obtaining the second state vector and the second covariance matrix updated at the next moment based on the second diameter information, the second state vector, and the second covariance matrix;

[0034] The target pupil diameter is determined according to the updated first covariance matrix and the second covariance matrix.

[0035] In some embodiments, determining the target pupil diameter according to the updated first covariance matrix and the second covariance matrix includes:

[0036] Calculating a target covariance corresponding to a current moment based on the first noise and the second noise;

[0037] Calculating a target state vector corresponding to a current moment based on the first state vector and the second state vector, and calculating a target covariance matrix corresponding to the current moment based on the first covariance matrix and the second covariance matrix;

[0038] Based on the target covariance, obtaining the target covariance and the target covariance matrix updated at the next moment;

[0039] A pupil weight value is determined according to a ratio value of the target covariance matrix, the target state vector is adjusted according to the pupil weight value, and the target pupil diameter is determined according to the adjusted target state vector.

[0040] In some embodiments, the wearable head eye tracking device further comprises a panoramic camera, wherein the panoramic camera is aligned with the dog's eye orientation to record the dog's movement trajectory and external environment information corresponding to the eye image;

[0041] The panoramic camera is used to send the collected eye image and the external environment information to the processing device, so that the processing device processes the eye image and the external environment information to obtain image presentation information for display.

[0042] To achieve the above-mentioned objectives, the third aspect of the embodiments of the present application proposes a processing device, which includes a memory and a processor, the memory storing a computer program, and when the processor executes the computer program, it implements the wearable head eye tracking device described in the first aspect embodiment and the wearable head eye tracking method described in the second aspect embodiment.

[0043] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes an eye tracking system, which includes the wearable head eye tracking device described in the embodiment of the first aspect and the processing device described in the embodiment of the third aspect.

[0044] Embodiments of the present application provide a wearable head eye-tracking device, method, processing device, and system, wherein the wearable head eye-tracking device includes a device frame, an opening is provided in the middle of the device frame, and the opening is through-hole, and downwardly protruding ear brackets are provided on both sides of the device frame, and the ear brackets are used to be mounted on the ears of a dog to fix the device frame on the dog's head. A protruding camera bracket is also provided on the front side of the device frame; and the wearable head eye-tracking device also includes at least one eye-tracking camera, which is provided on the camera bracket, and the eye-tracking camera is used to photograph the dog wearing the wearable head eye-tracking device to obtain an eye image including the dog's eyes; wherein the eye image is used to be sent to a processing device that is communicatively connected to the wearable head eye-tracking device, so that the processing device identifies the pupil of the dog's eye in the eye image and extracts the target pupil diameter of the dog's eye. In this way, dogs with larger bodies and larger ranges of movement can move freely in any scenario after being equipped with the wearable head eye tracking device. At the same time, the wearable head eye tracking device can collect eye information of the dog in real time during the activity, thereby completing the tracking of the dog's eye information without spatial restrictions and improving the accuracy of the collected dog's eye information.

[0045] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG1 is a schematic diagram of an optional structure of a wearable head eye tracking device provided in an embodiment of the present application;

[0047] FIG2 is a schematic diagram of an optional hardware connection of a wearable head eye tracking device provided in an embodiment of the present application;

[0048] FIG3 is an optional flow chart of a wearable head eye tracking method provided in an embodiment of the present application;

[0049] FIG4 is a flowchart of an implementation of step S102 in FIG3 ;

[0050] FIG5 is a flowchart of an implementation of step S201 in FIG4 ;

[0051] FIG6 is a flowchart of an implementation of step S202 in FIG4 ;

[0052] FIG7 is another optional flow chart of the wearable head eye tracking device provided in an embodiment of the present application;

[0053] FIG8 is a flowchart of an implementation of step S203 in FIG4 ;

[0054] FIG9 is a flowchart of an implementation of step S505 in FIG8 ;

[0055] FIG10 is a schematic diagram of the hardware structure of the processing device provided in an embodiment of the present application.

[0056] Reference numerals: device frame 10 , ear bracket 11 , camera bracket 12 , eye tracking camera 13 , panoramic camera 14 . DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] The study of canine eye information is an exploration of the canine visual system, and has important implications for fields such as biomedicine, ecology, and behavior. By observing subtle changes in a dog's eyes, we can often glean insights into their emotions, arousal, cognitive abilities, and attentional states. For example, a sudden dilation of a dog's pupils may be due to fear. Therefore, monitoring changes in eye information (i.e., eye tracking) is crucial in fields such as brain science research, psychology, and psychiatry. Currently, eye trackers are commonly used to track canine eye information.

[0060] However, most dogs are large in size and highly mobile. The eye trackers used in related technologies can either only detect dogs in a fixed state, or can only detect dogs with fixed cameras deployed in a preset space. Either method limits the dog's free range of movement, subjecting the dog to greater spatial restrictions. Therefore, it is impossible to know the dog's eye information in various scenarios, affecting the accuracy of the collected dog eye information.

[0061] For example, dogs are connected to fixed observation devices via wires for detection. This significantly limits the dog's range of movement. Furthermore, the complex restraints can easily cause the dog to panic, resulting in the collected eye information not being in its natural state, thereby reducing the accuracy of the collected eye information. Alternatively, when a dog is within a predetermined space, multiple cameras can be deployed to capture eye information. However, this approach forces the dog to remain in the same scene, lacking detection of other scenes, similarly reducing the accuracy of the collected eye information.

[0062] Based on this, the embodiments of the present application provide a wearable head eye tracking device, method, processing device and system, which can track the eye information of dogs with larger body size and greater freedom of movement without spatial restrictions, thereby improving the accuracy of the collected dog eye information.

[0063] The wearable head eye tracking device, method, processing device and system provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the wearable head eye tracking device in the embodiments of the present application is described.

[0064] As shown in Figure 1, Figure 1 is an optional structural schematic diagram of a wearable head eye tracking device provided in an embodiment of the present application, wherein the wearable head eye tracking device includes a device frame 10, an opening is provided in the middle of the device frame 10, and the opening is connected from top to bottom, and downwardly protruding ear brackets 11 are respectively provided on both sides of the device frame 10, and the ear brackets 11 are used to be mounted on the ears of a dog to fix the device frame 10 on the head of the dog. A protruding camera bracket 12 is also provided on the front side of the device frame 10.

[0065] In some embodiments, the main body of the wearable head eye tracking device (hereinafter referred to as the "tracking device") is a device frame 10, wherein the device frame 10 is through-through from top to bottom, so that it can be easily put on the dog's head. In the related art, in order to enable the eye tracking device to be fixed on the dog's head, a longitudinal headband is usually added to the eye tracking device to fix the eye tracking device. However, such an eye tracking device increases the dog's sense of foreignness and is not conducive to the collection of eye images. To this end, the wearable head eye tracking device in the embodiment of the present application adds ear brackets 11 (also called ear fixers) on both sides. Compared with the eye tracking device in the related art, the use of the ear brackets 11 can, on the one hand, stabilize the tracking device on the dog's head, and on the other hand, minimize the contact between the tracking device and the dog, increase the dog's wearing comfort, reduce the degree of foreign matter on the head, and make the dog as natural as possible to collect its eye information.

[0066] It is understandable that the device frame 10 that is through from top to bottom can also expose more skin tissue of the dog, making it easier for the operator to perform related operations such as electrode implantation.

[0067] Furthermore, the tracking device can also be equipped with a global positioning system (GPS) to record the movement trajectory of the dog when wearing the tracking device, wherein the movement trajectory is composed of multiple footprints on a satellite map. The GPS positioning system records the movement trajectory of the dog when it moves freely to assist in analyzing the dog's eye information. For example, when the dog passes through a park or a crowded place, it can be learned from the collected eye information that the dog's pupil is obviously enlarged and the outer circle of the pupil is rounder, indicating that the dog will be more excited when passing through these places.

[0068] In some embodiments, the tracking device is further provided with at least one eye tracking camera 13, which is disposed on the camera bracket 12. The eye tracking camera 13 is used to photograph the dog wearing the wearable head eye tracking device to obtain an eye image including the dog's eyes.

[0069] The eye image is used to be sent to a processing device that is communicatively connected to a wearable head eye tracking device, so that the processing device can identify the pupil of the dog's eye in the eye image and extract the target pupil diameter of the dog's eye.

[0070] In some embodiments, only one eye-tracking camera 13 may be provided. In this case, the eye-tracking camera 13 can capture both the dog's left and right eyes, allowing both eyes to appear in the same eye image. Alternatively, as shown in FIG1 , two eye-tracking cameras 13 may be provided, with the left eye-tracking device capturing the dog's left eye and the right eye-tracking device capturing the dog's right eye. This ensures specific capture and facilitates subsequent eye information processing.

[0071] Furthermore, the wearable head eye tracking device can be wirelessly connected to a processing device that recognizes and processes the received eye images. Furthermore, the processing device can be located in other environments, such as the operator's laboratory. This reduces the weight of the tracking device on the dog, reducing the burden on the dog and facilitating its free movement.

[0072] Furthermore, in order to ensure wearing comfort and stability, the device frame 10, ear bracket 11 and camera bracket 12 can be made of lightweight materials, such as acrylonitrile-butadiene-styrene copolymers (ABS).

[0073] Furthermore, as shown in Figure 2, Figure 2 is an optional hardware connection diagram of the wearable head eye tracking device provided in an embodiment of the present application. In order to save line connections, the three video streams of the panoramic camera, the left eye tracking camera and the right eye tracking camera can be integrated together, and the three USB video stream sub-signals are transmitted to the processing device so that the processing device can perform data processing on the received signals.

[0074] Furthermore, the processing device can be a smart phone, tablet computer, laptop computer, desktop computer, etc. or an independent physical server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. It should be noted that the processing device here can also have many other forms, but is not limited to the above forms, and can be specifically set according to actual operating conditions.

[0075] Furthermore, since the eye image contains the dog's eyes, the processing device can obtain information related to the dog's eyes, such as pupil position and pupil diameter, after processing the eye image. By analyzing the pupil position and pupil diameter at different times and in different scenarios, the dog's behavior and potential emotions can be studied.

[0076] Furthermore, the various parts of the wearable head eye tracking device provided in the embodiment of the present application adopt a quick plug-and-unplug design to facilitate the removal and replacement of related components, thereby improving the convenience of maintaining and upgrading the tracking device.

[0077] It should be noted that the wearable head eye tracking device provided in the embodiments of the present application can also be used on other larger animals, such as gorillas, pigs or cows. The embodiments of the present application are only described as preferred embodiments and are not specifically limited.

[0078] It should be noted that in each specific embodiment of this application, when it is necessary to perform relevant processing based on data related to the wearer's identity or characteristics, such as wearer information, wearer behavior data, wearer historical data, and wearer location information, the permission or consent of the wearer's corresponding manager will be obtained first. For example, when it is necessary to collect dog eye images for identification and data processing, the permission or consent of the owner will be obtained. Moreover, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when the embodiments of this application need to obtain sensitive personal information of relevant managers, the necessary wearer-related data for the normal operation of the embodiments of this application will be obtained only after obtaining the separate permission or consent of the relevant managers.

[0079] In some embodiments, the device frame 10 is provided with an adjustable headband, which is arranged around the opening and is provided with a fixing component for fixing the adjusted size of the headband.

[0080] Furthermore, the headband (not shown in the figure) can be made of a stretchable material to facilitate adaptation to dogs of different sizes. For example, when a dog needs to be equipped with the wearable head eye tracking device, the headband can be used to roughly estimate the size of the dog's head, and the headband can be fixed to a certain adjustable size using a fixing component. In this way, the wearable head eye tracking device can provide a comfortable head circumference size for different dogs, and at the same time, the wearable head eye tracking device can be stabilized on the heads of dogs of different sizes.

[0081] Furthermore, the headband is arranged around the opening, reducing additional space occupied and preventing the vision of the dog equipped with the tracking device from being blocked.

[0082] In some embodiments, the camera bracket 12 is a small-volume bracket. An eye-tracking camera 13 is provided at one end of the camera bracket 12 away from the device frame 10 , and an infrared device is provided inside the eye-tracking camera 13 .

[0083] In some embodiments, the camera bracket 12 can be a slender small-volume bracket, and the small-volume bracket extends outward from the device frame 10 in a certain arc. It can be understood that the small-volume bracket can reduce the weight of the dog on the one hand, and avoid obstruction of the dog's field of vision on the other hand, thereby improving the accuracy of the collected eye images.

[0084] Furthermore, eye-tracking camera 13 is typically mounted on the end of camera bracket 12 away from device frame 10. This allows eye-tracking camera 13 to maintain a certain shooting distance to ensure a complete image of the dog's eyes. Furthermore, eye-tracking camera 13 is also equipped with an infrared device. This allows the camera to clearly capture the dog's eyes even at night, when the dog is active. This allows research on canine eye information to be conducted not only during the day but also at night, a period typically overlooked by humans. This increases the diversity of canine eye research scenarios and improves the quality of collected eye information.

[0085] In some embodiments, the device frame 10 is further provided with an elastic contact surface, which is arranged around the headband so that the device frame 10 contacts the dog when it is fixed to the dog's head.

[0086] Furthermore, the elastic contact surface can be made of sponge material or rubber to enhance the comfort of the dog wearing the tracking device. In this way, the presence of the tracking device on the dog's head can be minimized, and the dog is closer to the state when not wearing the tracking device when moving freely, thereby enhancing the authenticity and accuracy of the collected dog's eye information.

[0087] Furthermore, the tracking device is made of waterproof and sweat-proof materials to ensure that the tracking device can still maintain reliability and stability in humid environments.

[0088] In some embodiments, the device frame 10 is further provided with a panoramic camera 14, which is aligned with the direction of the dog's eyes to record the dog's movement trajectory and external environment information corresponding to the eye image.

[0089] Furthermore, the external environment information recorded by the panoramic camera 14 can be an image of the surrounding environment taken when the dog moves to a certain place. In addition, the aforementioned GPS positioning system can also be installed in the panoramic camera 14. In this way, the movement trajectory recorded in the panoramic camera 14 can be the dog's movement footprint. Combining the movement footprint and the environmental image, the range and scene of the dog's free movement can be determined, which is convenient for assisting the analysis of the dog's eye information.

[0090] Furthermore, the panoramic camera used in the embodiment of the present application can have a resolution of 1280×720 and a frame rate of 30fps; while the eye tracking camera can have a resolution of 640×480 and a frame rate of 30fps. In this way, the captured images can be clear and stable, and the accuracy of eye tracking can be avoided from being affected by factors such as image blur and jitter.

[0091] As shown in Figure 3, Figure 3 is an optional flow chart of the wearable head eye tracking method provided by an embodiment of the present application, wherein the processing device is communicatively connected to the wearable head eye tracking device, the wearable head eye tracking device includes a device frame 10 and at least one eye tracking camera 13, an opening is provided in the middle of the device frame 10, and the opening is through-through from top to bottom, and downwardly protruding ear brackets 11 are provided on both sides of the device frame 10, and the ear brackets 11 are used to be mounted on the ears of the dog to fix the device frame 10 on the head of the dog, and a protruding camera bracket 12 is also provided on the front side of the device frame 10, and the eye tracking camera 13 is provided on the camera bracket 12, and is used to shoot the dog wearing the wearable head eye tracking device to obtain an eye image including the dog's eyes; the method in Figure 3 may include but is not limited to steps S101 to S102.

[0092] Step S101: Acquire an eye image sent by a wearable head eye tracking device.

[0093] In some embodiments, a processing device connected to the tracking device can receive an eye image sent by the tracking device, and the processing device can identify and process the canine eye portion in the eye image to obtain eye information for analyzing the canine eye.

[0094] Furthermore, the tracking device can obtain the eye image of the dog in real time and send the eye image to the processing device connected thereto in real time, so that the processing device can process the eye image in real time.

[0095] Step S102 : Identify the pupil of the dog's eye in the eye image and extract the target pupil diameter of the dog's eye.

[0096] In some embodiments, the processing device needs to identify the dog's eyes, especially the pupil part, in the eye image, extract the target pupil diameter, and analyze the dog's eye information based on the target pupil diameter.

[0097] Furthermore, the target pupil diameter can reflect the physiological or emotional impact of the surrounding environment on a dog. For example, when the surrounding optical fiber intensity is too high, a dog's pupils tend to constrict to protect their eyes from damage. Alternatively, when a dog's emotions fluctuate, their pupil diameter often dilates. Furthermore, when a dog suffers from a subtle illness, even without any unusual changes in the surrounding environment, their pupils may become abnormally dilated, constricted, or asymmetrical. In other words, identifying and processing the target pupil diameter allows for further analysis of the dog's eye information, and acquiring highly accurate eye images is crucial for achieving this further analysis.

[0098] Furthermore, other parts of the dog's eyes can be identified and processed based on the acquired eye images, thereby enhancing the multifaceted analysis of the dog's eye information. The identified and processed parts can also include the condition around the eyes and the sclera. Specifically, it is possible to check whether there are tears or other foreign matter around the dog's eyes and record the amount of tears and foreign matter, and check the color of the sclera and its blood vessel conditions. For example, the sclera is normally white. If it is another color different from the normal, it indicates that the dog's eyes are likely to be injured or otherwise stimulated.

[0099] As shown in FIG. 4 , FIG. 4 is a flowchart of an implementation of step S102 in FIG. 3 . In some embodiments, step S102 may include steps S201 to S203 .

[0100] Step S201: Input the eye image into a pre-trained pupil recognition model to extract first diameter information of the dog's eye in the eye image.

[0101] In some embodiments, the pupil recognition model is trained based on a deep learning method. The pupil recognition model can be a deep learning model obtained by combining any one or part of a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), a support vector machine (SVM), or a variational autoencoder (VAE).

[0102] Furthermore, deep learning models can be trained using a pre-set training set. For example, a pre-defined model, Residual Neural Network (ResNet), can be used as the basic model, and the training set can be used as input. TensorFlow can be used as the training environment, and the model training can be iterated one million times until the model converges to obtain a trained model.

[0103] Furthermore, the eye-tracking camera 13 can track and record changes in the dog's eyes in the form of video, and can capture image frames at preset intervals. The interval can be 5 seconds, which avoids the excessive computing power and resource waste caused by analyzing eye images frame by frame. Alternatively, a K-means clustering method can be used to extract 400 eye images from the recorded eye video for identification and processing. This method of extracting eye images can also be used to generate a training set for the pupil recognition model.

[0104] It should be noted that the interval duration can be specifically set according to actual conditions, and the embodiments of the present application do not impose any specific restrictions.

[0105] Step S202 : performing image segmentation processing on the eye image according to a preset segmentation threshold value, and extracting second diameter information of the dog's eye in the eye image.

[0106] In some embodiments, in addition to using a deep learning method to process the eye image, an image segmentation method can also be used to simultaneously identify and process the eye image and obtain the second diameter information of the canine eye in the eye image.

[0107] Furthermore, the preset segmentation threshold can be a grayscale threshold. First, the acquired eye image is grayscale processed to obtain a grayscale value image corresponding to the eye image, and the grayscale value of each pixel is analyzed to see whether it reaches the preset grayscale threshold, so as to determine the corresponding pupil area of ​​the dog's eye. Then, the target pupil diameter is determined from the pupil area, wherein the method for determining the target pupil diameter can be an edge detection algorithm, Hough transform or other methods. The specific method can be set according to actual conditions, and the embodiments of the present application do not impose specific restrictions.

[0108] Step S203 , obtaining the target pupil diameter of the dog's eye by fusing the first diameter information and the second diameter information.

[0109] In some embodiments, the first diameter information obtained through deep learning and the second diameter information obtained through image segmentation can be fused to obtain the final target pupil diameter. It is understood that pupil diameters obtained solely through deep learning or image processing will have certain errors. Fusion of the two can minimize this error, making the resulting target pupil diameter closer to the true value and improving the accuracy of canine eye information.

[0110] What needs to be explained in more detail is that although both deep learning and image segmentation methods can be used alone to calculate the pupil diameter of dogs, each has its own advantages and disadvantages: ① The deep learning method does not need to set thresholds and parameters in a specific environment. As long as the training set is extensive, it can adapt to scenes with a variety of lighting, resolutions, and contrasts. However, the disadvantage is that if the training set is not extensive enough (i.e., scenes with poor matching with the training set), the recognition error is extremely large; ② The advantage of the image segmentation method is that once the threshold is set, the pupil accuracy obtained is very high when the light and contrast are stable, but the error is extremely large when the light changes. The embodiment of the present application uses deep learning and image segmentation methods to identify and process the same eye image respectively, and can obtain the corresponding first diameter information and second diameter information. Then, based on the first diameter information and the second diameter information, the fusion processing can make up for the error caused by using only one of the processing methods, that is, it achieves complementary advantages and improves the accuracy of the final target pupil diameter of the dog.

[0111] As shown in FIG. 5 , FIG. 5 is a flowchart of an implementation of step S201 in FIG. 4 . In some embodiments, step S201 may include steps S301 to S303 .

[0112] Step S301: Input the eye image into a pre-trained deep learning model to obtain the pupil center point of the eye image.

[0113] In some embodiments, after the eye image is input into a pre-trained deep learning model, the deep learning model may first perform a preprocessing operation on the eye image, wherein the preprocessing operation may include region cropping. For example, the eye image captured by the eye tracking camera 13 may contain irrelevant background parts other than the dog's eyes. The deep learning model may first locate the dog's eye area, and then perform region cropping to obtain the eye image including the pupil area after preprocessing. In this way, the background parts irrelevant to the target can be removed, avoiding interference with the subsequent processing of the eye information, thereby improving the quality of the eye image.

[0114] Furthermore, the pupil center of the eye image can be determined from the pupil area. The pupil center represents the position of the dog's pupil in the eye. The position of the pupil center in the dog's eye is of great significance to the study of dog eye information. For example, the pupil center of a dog is usually located in the center of the iris. If the dog develops a lesion, the pupil center will deviate from the center of the iris. At this time, it can be known that the dog is very likely to have a physiological abnormality.

[0115] Step S302: Taking the pupil center point as a reference, randomly select a pupil outer circle feature point in the pupil area, determine other pupil outer circle feature points according to a preset interval angle, and repeat this operation until the number of pupil outer circle feature points reaches a preset number.

[0116] In some embodiments, studying the pupil size in addition to the center point is also crucial. For example, a dog's pupil dilates continuously when excited, constricts continuously when frightened, and dilates and constricts abnormally when sick. Therefore, determining pupil size, or pupil diameter, is crucial as a foundation for subsequent research.

[0117] Furthermore, the pupil area is usually significantly different from other areas. After determining the pupil center point, any feature point on the outermost side of the pupil area can be selected as the pupil outer circle feature point, and then other pupil outer circle feature points can be determined at preset interval angles.

[0118] Furthermore, the preset interval angle can be 45 degrees, so that 8 feature points of the outer circle of the pupil can be obtained. The reasons for selecting 8 feature points are: ① At least 3 points are required to perform an ellipse fitting operation on the pupil; ② If only 3 feature points are set, when the feature point recognition error is too large, it will affect the fitting accuracy of the pupil diameter; ③ The 8 feature points can complement each other without having too many feature points, thereby avoiding increasing the workload of manually constructing a deep learning feature detection model.

[0119] It should be noted that the interval angle can be set according to actual conditions. This application only describes the preferred embodiment and does not impose any specific limitations.

[0120] Step S303 : performing a first ellipse fitting process on the eye image based on the pupil center point and the pupil outer circle feature points to obtain first diameter information of the dog's eye in the eye image.

[0121] In some embodiments, the first ellipse fitting process can be performed using a least squares method, gradient descent, elliptical Hough transform, or random sampling consensus algorithm. This process ultimately yields first diameter information for the canine eye. The primary reason for using ellipse fitting is that, when processing pupil images, the pupil may not be perfectly circular due to various factors (such as lighting, angle, and occlusion). Therefore, using ellipse fitting allows for a more accurate estimation of pupil shape and position, thereby improving the accuracy of subsequent eye analysis and processing.

[0122] Furthermore, since the pupil shape usually obtained after the first ellipse fitting process is elliptical, the major axis of the ellipse can be uniformly selected as the first diameter information, and the first diameter information can be used to reflect the size of the dog's pupil.

[0123] As shown in FIG. 6 , FIG. 6 is a flowchart of an implementation of step S202 in FIG. 4 . In some embodiments, step S202 may include steps S401 to S402 .

[0124] Step S401 : segmenting the eye image into multiple sub-images. If any sub-image is within a preset segmentation threshold, the sub-image is determined to be a canine eye image set.

[0125] In some embodiments, the original eye image may first be preprocessed, wherein the preprocessing operations include denoising, smoothing, and enhancement, and then the acquired eye image may be first segmented into multiple sub-image blocks using an image segmentation method. Whether the segmented sub-image is within a preset segmentation threshold is determined to determine whether the sub-image is a canine eye image set.

[0126] Step S402 : performing a second ellipse fitting process on the dog eye image set to obtain second diameter information of the dog eyes in the eye images.

[0127] In some embodiments, corresponding feature points may be extracted from a set of canine eye images, and a second ellipse fitting process may be performed based on the feature points to obtain second diameter information of the canine eye in the eye image.

[0128] Furthermore, the feature point extraction method of the eye image after image segmentation may be a feature point determination method similar to that in steps S301 to S302 above, or may be other feature point determination methods such as corner detection, spot detection or edge detection.

[0129] For example, as shown in Figure 7, Figure 7 is another optional flow chart of the wearable head eye tracking device provided in an embodiment of the present application. After receiving the eye image, the processing device can process the eye image based on the deep learning method and the image segmentation method respectively to obtain the first diameter information and the second diameter information corresponding to the eye image. Then, the Kalman filter is used to remove noise respectively, and the first diameter information and the second diameter information after noise removal are fused to finally obtain a high-precision canine pupil diameter.

[0130] As shown in FIG. 8 , FIG. 8 is a flowchart of an implementation of step S203 in FIG. 4 . In some embodiments, step S203 may include steps S501 to S505 .

[0131] Step S501: Calculate the first noise of the first diameter information, and determine the first state vector and the first covariance matrix corresponding to the current moment according to the first noise.

[0132] In some embodiments, after respectively obtaining the first diameter information obtained based on the deep learning method and the second diameter information obtained based on the image segmentation method, the two diameter information can be fused to achieve error complementation of the two processing methods and improve the accuracy of the target pupil diameter obtained in the end. ROI-SegIndicates the first diameter information and uses z dlc Indicates the second diameter information.

[0133] Furthermore, the calculated first diameter information and second diameter information usually have errors compared to actual values. This error is usually called noise, which can be eliminated by using Kalman filtering.

[0134] Furthermore, in the embodiments of the present application, noise (both the first noise and the second noise) is a random interference signal introduced during the acquisition, transmission, or processing of the eye image. It is caused by various factors, such as limitations of the device itself, interference during signal transmission, environmental influences, etc. Therefore, it is necessary to calculate the first noise corresponding to the first diameter information and use the first noise to determine the first state vector and first covariance matrix at the current moment.

[0135] Among them, the first state vector refers to the estimated value of the current state obtained by predicting and updating the system state in the estimation process of the Kalman filter. It is a key concept in Kalman filtering and is used to represent the current state of the system; and the first covariance matrix refers to the error covariance matrix of the current state obtained by predicting and updating the system state in the estimation process of the Kalman filter. It is an indicator that describes the degree of deviation between the estimated state and the true state, reflecting the uncertainty of state estimation.

[0136] For example, first, the system state is estimated using a Kalman filter (KF), and smooth data is generated after eliminating extreme values. This method includes two steps: prediction and update, wherein the prediction step uses the following formula: p k =ap k-1 a+q (4)

[0137] In the formula, r is the observation noise, q is the process noise, z∈R n×1 is the observed value (first diameter information and second diameter information), is the posterior state estimate at the current moment k, a is the state transition constant from moment k-1 to moment k, which can be set to 1 (assuming the current state is consistent with the previous state), p k is the posterior estimation error of the state estimation, which is calculated recursively. In order to initialize KF, Set z0 to z0 and p0 to 1.

[0138] Step S502: obtaining the first state vector and the first covariance matrix corresponding to the next moment after the update based on the first diameter information, the first state vector and the first covariance matrix;

[0139] In some embodiments, after obtaining the first diameter information, the first state vector and the first covariance matrix at the current moment, it is necessary to predict and update the state vector and covariance matrix at the next moment based on this information. In this way, the estimation accuracy of the system state is gradually improved by continuously fusing the prior estimates and observations.

[0140] Exemplarily, the update step uses the following formula: p k ←(1-g k )p k (7)

[0141] Among them, g k is the Kalman gain at the current time k, which is used to balance the current observation with the prediction based on the previous state.

[0142] Secondly, the covariance of the observation noise largely determines the performance of the KF fusion algorithm by affecting the fusion gain. Therefore, the estimated state is combined with the observation value to dynamically recalculate the observation noise. By adaptively adjusting the fusion gain, the weight is tilted towards more reliable measurement sources. The noise estimation equation is:

[0143] Where ε is an infinitesimal value, so that z k +ε>0.

[0144] In order to make the observation noise of the two measurements comparable, it can be normalized as follows:

[0145] Step S503: Calculate the second noise of the second diameter information, and determine the second state vector and the second covariance matrix corresponding to the current moment according to the second noise.

[0146] It is understandable that the calculation method of the second noise, the second state vector and the second covariance matrix here is similar to the calculation method of the first noise, the first state vector and the first covariance matrix in the above step S501, and will not be repeated here.

[0147] Step S504: Based on the second diameter information, the second state vector and the second covariance matrix, obtain the second state vector and the second covariance matrix updated at the next moment.

[0148] It can be understood that the calculation method for predicting and updating the second state vector and the second covariance matrix here is similar to the calculation method for predicting and updating the first state vector and the first covariance matrix in the above step S502, and will not be repeated here.

[0149] Step S505 : determining the target pupil diameter according to the updated first state vector, the first covariance matrix, the second state vector, and the second covariance matrix.

[0150] In some embodiments, the diameter information and related data obtained by using two image processing methods are fused to compensate for the error caused by a single image processing method, thereby compensating for the problem of low accuracy of the target pupil diameter obtained.

[0151] Furthermore, the updated first state vector and the first covariance matrix can be predicted and updated again until the preset update stop condition is reached to obtain the first diameter information for calculating the final target pupil diameter. In addition, the second diameter information is also the same, which will not be repeated here.

[0152] Furthermore, the update stopping condition may be that the first state vector and / or the first covariance matrix is ​​smaller than a certain threshold, or the prediction update may be stopped when a specified number of iterations is reached.

[0153] As shown in FIG. 9 , FIG. 9 is a flowchart of an implementation of step S505 in FIG. 8 . In some embodiments, step S505 may include steps S601 to S604 .

[0154] Step S601: Calculate and obtain the target covariance corresponding to the current moment based on the first noise and the second noise.

[0155] In some embodiments, the fusion stage further includes two steps: updating and predicting. The formula for the prediction step is as follows: P k =AP k-1 A T +Q (13)

[0156] Among them, z ROI-Seg and z dlc are two independent pupil measurement values ​​obtained by image segmentation and deep learning methods, respectively. and are the observation noise estimated in the pre-fusion stage; R k is the covariance matrix between each pair of measurements at the current time k; Q is the covariance of the process noise; is the posterior state estimate at the current moment k; A is the state transition constant from k-1 to k, which can also be set to 1; P k is the posterior covariance matrix of the estimated state at time k, and, when initialized, we can set for P0 is 1.

[0157] Step S602 : Calculate the target state vector corresponding to the current moment based on the first state vector and the second state vector, and calculate the target covariance matrix corresponding to the current moment based on the first covariance matrix and the second covariance matrix.

[0158] In some embodiments, P k -1 is the covariance matrix obtained at the previous moment. Here, it is the first diameter information and the second diameter information obtained according to the deep learning method and the image segmentation method respectively. By fusing the first diameter information and the second diameter information and predicting and updating them, the target covariance matrix corresponding to the current moment can be obtained.

[0159] Step S603, based on the target covariance, obtaining the updated target covariance and target covariance matrix at the next moment;

[0160] In some embodiments, after obtaining the target covariance matrix corresponding to the current moment, the target covariance matrix at the current moment can be used as the update basis for the next moment according to the above formula (13), and the target covariance matrix corresponding to the next moment can be determined by adding it to the target covariance.

[0161] For example, after obtaining the target covariance matrix, the target covariance matrix at the next moment can be updated. Specifically, the updating step uses the following formula: P k ←(1-G k C)P k (18)

[0162] Among them, G k is the Kalman fusion gain at the current moment k, which is an adaptive weight used to adjust the weight of the measurement source on the target pupil diameter of the final fusion output.

[0163] Step S604 , determining a pupil weight value according to the ratio value of the target covariance matrix, adjusting the target state vector according to the pupil weight value, and determining a target pupil diameter according to the adjusted target state vector.

[0164] In some embodiments, as shown in the above formula (16), the Kalman fusion gain, that is, the pupil weight value, can be determined using the proportional value of the target covariance matrix, and as shown in the above formula (17), the target state vector obtained is adjusted using the pupil weight value, and the adjusted target state vector is determined to be the target pupil diameter.

[0165] In some embodiments, the wearable head eye tracking device also includes a panoramic camera 14, wherein the panoramic camera 14 is aligned with the direction of the dog's eyes to record the dog's movement trajectory and the external environment information corresponding to the eye image; the panoramic camera 14 is used to send the collected eye image and external environment information to the processing device, so that the processing device processes the eye image and external environment information to obtain image display information for display.

[0166] Furthermore, the processing device that receives the eye images and external environment information sent by the panoramic camera 14 can be the same processing device that identifies and processes the dog's pupils; or, another processing device can be selected to specifically display the panoramic images captured by the panoramic camera 14. In this way, it is convenient to compare the images of the places where the dog has passed and the eye images captured by the eye tracking camera 13, providing a more comprehensive information basis for the study of dog's eye information.

[0167] Furthermore, the eye-tracking camera and the panoramic camera in the embodiment of the present application both record in real time, and the eye images recorded by the eye-tracking camera and the external environment information recorded by the panoramic camera can be sent to the processing device in real time, so that the processing device can use the real-time data stream to perform data processing. In this way, the real-time nature and speed of eye image acquisition and processing are high.

[0168] The present application also provides a processing device comprising a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned wearable head eye tracking device and the aforementioned wearable head eye tracking method. The processing device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0169] As shown in FIG10 , FIG10 is a schematic diagram of the hardware structure of a processing device provided in an embodiment of the present application, and the processing device includes:

[0170] The processor 701 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0171] The memory 702 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 702 and is called by the processor 701 to execute the wearable head eye tracking device of the embodiments of this application.

[0172] Input / output interface 703, used to implement information input and output;

[0173] Communication interface 704, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0174] Bus 705 , which transmits information between various components of the device (e.g., processor 701 , memory 702 , input / output interface 703 , and communication interface 704 );

[0175] The processor 701 , the memory 702 , the input / output interface 703 and the communication interface 704 are connected to each other in communication within the device via a bus 705 .

[0176] In addition, the embodiment of the present application also provides a wearable head eye tracking system, which includes a wearable head eye tracking device and a processing device. Among them, the wearable head eye tracking device included in the wearable head eye tracking system is basically the same as the specific embodiment of the wearable head eye tracking device mentioned above, and will not be repeated here. Under the premise of meeting the requirements of the embodiment of the present application, the wearable head eye tracking system can also be provided with other functional modules to realize other related functions.

[0177] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0178] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0179] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0180] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0181] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0182] It should be understood that in this application, "at least one (item)" and "several" refer to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0184] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0186] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0187] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A wearable head eye tracking device, characterized in that: include: An equipment frame, wherein an opening is provided in the middle of the equipment frame, the opening is through-through from top to bottom, and ear brackets protruding downward are respectively provided on both sides of the equipment frame, the ear brackets are used to be mounted on the ears of the dog to fix the equipment frame on the head of the dog, and a protruding camera bracket is also provided on the front side of the equipment frame; At least one eye tracking camera, disposed on the camera bracket, the eye tracking camera being used to photograph the dog wearing the wearable head eye tracking device to obtain an eye image including the dog's eyes; The eye image is used to be sent to a processing device that is communicatively connected to the wearable head eye tracking device, so that the processing device can identify the pupil of the canine eye in the eye image and extract the target pupil diameter of the canine eye.

2. The wearable head eye tracking device according to claim 1, characterized in that: The device frame is provided with a headband with an adjustable size, the headband is arranged around the opening, and the headband is provided with a fixing component, and the fixing component is used to fix the adjustment size of the headband.

3. The wearable head eye tracking device according to claim 2, characterized in that: The camera bracket is a small-volume bracket, and the eye-tracking camera is arranged at one end of the camera bracket away from the outer frame of the device, and an infrared device is arranged inside the eye-tracking camera.

4. The wearable head eye tracking device according to claim 3, characterized in that: The device outer frame is also provided with an elastic contact surface, and the elastic contact surface is arranged around the headband so that the device outer frame can contact the dog when fixed on the dog's head.

5. The wearable head eye tracking device according to claim 4, characterized in that: The outer frame of the device is also provided with a panoramic camera, which is aligned with the direction of the dog's eyes to record the dog's movement trajectory and external environment information corresponding to the eye image.

6. A wearable head eye tracking method, characterized in that: Applied in a processing device, the processing device is communicatively connected with a wearable head eye tracking device, the wearable head eye tracking device comprises a device frame and at least one eye tracking camera, an opening is provided in the middle of the device frame, the opening is through-through from top to bottom, ear brackets protruding downward are provided on both sides of the device frame, the ear brackets are used to be mounted on the ears of a dog to fix the device frame on the head of the dog, a camera bracket protruding is also provided on the front side of the device frame, the eye tracking camera is provided on the camera bracket, and is used to photograph a dog wearing the wearable head eye tracking device to obtain an eye image including the dog's eyes; The method comprises: Acquire the eye image sent by the wearable head eye tracking device; The pupil of the canine eye in the eye image is identified, and the target pupil diameter of the canine eye is extracted.

7. The wearable head eye tracking method according to claim 6, characterized in that: The step of identifying the pupil of the canine eye in the eye image and extracting a target pupil diameter of the canine eye includes: Inputting the eye image into a pre-trained pupil recognition model to extract first diameter information of the canine eye in the eye image; Performing image segmentation processing on the eye image according to a preset segmentation threshold, and extracting second diameter information of the canine eye in the eye image; The target pupil diameter of the canine eye is obtained by fusing the first diameter information and the second diameter information.

8. The wearable head eye tracking method according to claim 7, characterized in that: The step of inputting the eye image into a pre-trained pupil recognition model to extract first diameter information of a canine eye in the eye image includes: Inputting the eye image into a pre-trained deep learning model to obtain a pupil center point of the eye image; Taking the pupil center point as a reference, randomly select a pupil outer circle feature point in the pupil area, determine other pupil outer circle feature points according to a preset interval angle, and repeat this operation until the number of pupil outer circle feature points reaches a preset number; According to the pupil center point and the pupil outer circle feature points, a first ellipse fitting process is performed on the eye image to obtain first diameter information of the dog's eye in the eye image.

9. The wearable head eye tracking method according to claim 8, characterized in that: The step of performing image segmentation processing on the eye image according to a preset segmentation threshold to extract the second diameter information of the canine eye in the eye image includes: Segmenting the eye image into a plurality of sub-images, and if any of the sub-images is within a preset segmentation threshold, determining that the sub-image is a canine eye image set; A second ellipse fitting process is performed on the canine eye image set to obtain second diameter information of the canine eyes in the eye images.

10. The wearable head eye tracking method according to claim 9, characterized in that: The step of obtaining a target pupil diameter of a canine eye according to the fusion of the first diameter information and the second diameter information includes: Calculating a first noise of the first diameter information, and determining a first state vector and a first covariance matrix corresponding to a current moment according to the first noise; Based on the first diameter information, the first state vector and the first covariance matrix, obtaining the first state vector and the first covariance matrix corresponding to the next moment after updating; Calculating the second noise of the second diameter information, and determining the second state vector and the second covariance matrix corresponding to the current moment according to the second noise; Based on the second diameter information, the second state vector and the second covariance matrix, obtaining the second state vector and the second covariance matrix updated at the next moment; The target pupil diameter is determined according to the updated first covariance matrix and the second covariance matrix.

11. The wearable head eye tracking method according to claim 10, characterized in that: Determining the target pupil diameter according to the updated first covariance matrix and the second covariance matrix includes: Calculating a target covariance corresponding to a current moment based on the first noise and the second noise; Based on the first state vector and the second state vector, a target state vector corresponding to the current moment is calculated, and based on the first covariance matrix and the second covariance matrix, a target covariance matrix corresponding to the current moment is calculated; Based on the target covariance, obtaining the target covariance and the target covariance matrix updated at the next moment; A pupil weight value is determined according to a ratio value of the target covariance matrix, and the target state vector is adjusted according to the pupil weight value, and the target pupil diameter is determined according to the adjusted target state vector.

12. The wearable head eye tracking method according to claim 11, characterized in that: The wearable head eye tracking device also includes a panoramic camera, wherein the panoramic camera is aligned with the dog's eye orientation to record the dog's movement trajectory and external environment information corresponding to the eye image; The panoramic camera is used to send the collected eye image and the external environment information to the processing device, so that the processing device processes the eye image and the external environment information to obtain image presentation information for display.

13. A processing device, characterized in that: The processing device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the wearable head eye tracking device described in any one of claims 1 to 5 and the wearable head eye tracking method described in any one of claims 6 to 12.

14. A wearable head eye tracking system, characterized in that: It comprises the wearable head eye tracking device according to any one of claims 1 to 5, and the processing device according to claim 13.

Images