A security and rescue method based on eye tracking and augmented reality technology

Abstract

The present invention discloses a security and rescue method based on eye tracking and augmented reality technology, comprising: obtaining a plurality of eye tracking models constructed based on different eye feature information to establish an eye tracking model set; obtaining a user's eye feature information; determining a basic eye tracking model from the eye tracking model set based on the user's eye feature information; modifying the basic eye tracking model based on the user's eye feature information to obtain a target eye tracking model; establishing a mapping relationship between the user's eyes and an original image directly in front of the user based on the target eye tracking model; performing recognition processing on the original image to obtain a recognition result; and providing rescue assistance to the user based on the recognition result and the mapping relationship. This solution can provide rescue assistance to rescue personnel, ensuring their personal safety while improving rescue efficiency and safeguarding people's lives.

Description

Technical Field

[0001] The present invention relates to the field of security and rescue technology, and in particular to a security and rescue method based on eye tracking and augmented reality technology. Background Art

[0002] Currently, with the advancement of science and technology, industrial products are gradually replacing human applications in various fields, improving people's production and living standards. However, due to the immaturity of science and technology and the non-standard operation of some users, safety accidents related to industrial products are frequent, and security and rescue are becoming increasingly important. With the advancement of industrial production equipment, the level of safety accidents has also increased, but the deployment and rescue technology has reached a bottleneck. The personal safety of rescue workers cannot be guaranteed, the rescue efficiency is low, and the safety of human life and property is facing a huge threat. Summary of the Invention

[0003] The present invention aims to at least partially address one of the technical problems of the aforementioned technologies. To this end, the present invention aims to provide a security and rescue method based on eye tracking and augmented reality technology, which aims to ensure the personal safety of rescuers by providing them with rescue assistance based on augmented reality technology, while also improving rescue efficiency.

[0004] To achieve the above objectives, the present invention provides a security and rescue method based on eye tracking and augmented reality technology, including:

[0005] Obtain several eye tracking models based on different eye feature information and establish an eye tracking model set;

[0006] Obtain user's eye feature information;

[0007] Determining a basic eye tracking model in the eye tracking model set according to the eye feature information of the user;

[0008] Modifying the basic eye tracking model according to the eye feature information of the user to obtain a target eye tracking model;

[0009] Establishing a mapping relationship between the user's eyes and an original image directly in front of the user based on the target eye tracking model;

[0010] Performing recognition processing on the original image to obtain a recognition result;

[0011] Rescue assistance is provided to the user according to the recognition result and the mapping relationship.

[0012] Preferably, after obtaining the target eye tracking model, the method further includes:

[0013] A correspondence between the user's eye feature information and the target eye tracking model is established and stored.

[0014] Preferably, the eye feature information includes at least one of iris pattern, iris area and pupil area.

[0015] Furthermore, the ratio of the user's iris area to the pupil area is calculated in real time;

[0016] Comparing the ratio value with a preset value to obtain a comparison result;

[0017] The user's eye state is determined based on the comparison result and output.

[0018] Furthermore, we can obtain the ambient lighting information;

[0019] The ratio value is corrected according to the ambient lighting information.

[0020] Preferably, performing recognition processing on the original image includes:

[0021] Dividing the original image into a plurality of image blocks;

[0022] Dividing the image blocks into a first type of image blocks and a second type of image blocks according to the target eye tracking model; the first type of image blocks are image blocks including the user's gaze point, and the second type of image blocks are image blocks not including the user's gaze point;

[0023] Performing image recognition on the first-category image blocks to obtain first identification information, and adding the first identification information to the first-category image blocks to obtain first-category target image blocks;

[0024] Performing image recognition on the second-category image blocks to obtain second identification information, and adding the first identification information to the second-category image blocks to obtain second-category target image blocks;

[0025] The first type of target image blocks and the second type of target image blocks are combined to obtain a target image and output it.

[0026] Preferably, obtaining a target eye tracking model includes:

[0027] Determining a basic field of view range according to the basic eye tracking model;

[0028] Acquire the size of the user's eye area, and adjust the basic field of view range according to the size of the user's eye area to obtain a first field of view;

[0029] Obtaining the position of the pupil of the user when looking straight ahead, establishing a field of view rectangular coordinate system with the position of the pupil of the user when looking straight ahead as the coordinate origin, the horizontal direction as the abscissa direction, and the vertical direction as the ordinate direction;

[0030] Dividing the first field of view into four first-level sub-fields of view according to the four quadrants of the field of view rectangular coordinate system, and obtaining a first-level center point of each first-level sub-field of view;

[0031] Determine the midpoint of the line connecting two adjacent primary center points as the secondary center point;

[0032] Keeping the field of view rectangular coordinate system unchanged, the first field of view is re-divided according to the coordinate origin of the field of view rectangular coordinate system, the primary center point, and the secondary center point to obtain a second field of view including a plurality of secondary sub-fields of view;

[0033] Select several secondary center points as calibration stimulation points, and select several random points in the second visual field as verification stimulation points;

[0034] Performing a first-level correction on the basic eye tracking model according to the first feedback information of the user on the calibration stimulation point to obtain a first-level correction model;

[0035] Verifying the first-level correction model according to the second feedback information of the user on the verification stimulation point to obtain a first-level correction error value;

[0036] Comparing the first-level corrected error value with a preset error value to obtain a comparison result;

[0037] When it is determined that the first-level correction error value is greater than the preset error value, the first-level correction model is corrected twice according to the calibration stimulation point to obtain a second-level correction model; the above process is repeated and iterated until the i-th level correction error value is less than the preset error value;

[0038] The i-level correction model obtained by the i-level correction error value being smaller than the first-level preset error value is used as the target eye tracking model.

[0039] Furthermore, an upper limit k of the number of iterations for correction of the basic eye tracking model is set;

[0040] After each iterative correction of the basic eye tracking model, determining whether the current number of iterations exceeds the upper limit k of the corrected number of iterations, and obtaining a determination result;

[0041] When it is determined that the current number of iterations exceeds the upper limit k of the number of modified iterations, the basic eye tracking model is reacquired for modification.

[0042] Preferably, verifying the first-level correction model according to the second feedback information of the user on the verification stimulation point to obtain the first-level correction error value includes:

[0043] Establishing a spherical coordinate system with the verification stimulus point as the spherical coordinate origin;

[0044] Calculating the theoretical position of the user's pupil when the user is gazing at the verification stimulus point according to the first-level correction model, and obtaining the theoretical position coordinates of the theoretical position of the user's pupil according to the spherical coordinate system;

[0045] Obtaining the actual position of the pupil of the user when the user is gazing at the verification stimulus point, and obtaining the actual position coordinates of the user's pupil according to the spherical coordinate system;

[0046] The first-level correction error value is obtained by substituting the theoretical position coordinates of the user's pupil and the actual position coordinates of the user's pupil into a correction error value calculation algorithm.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. Based on augmented reality technology, it provides rescue workers with visual enhancement assistance to ensure their personal safety and improve rescue efficiency.

[0049] 2. Through eye tracking technology, the rescuer's field of view is divided into multiple blocks. According to the different blocks the rescuer is looking at, different visual enhancement rescue assistance is provided to the rescuer, thereby improving the rescue efficiency of the rescuer.

[0050] 3. Through eye tracking technology, the rescuer's eye information is extracted, and based on the user's eye feature information, the rescuer's eye condition is output in real time to further estimate the rescuer's physical condition and ensure the rescuer's personal safety.

[0051] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0052] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0054] Figure 1is a flow chart of a security and rescue method based on eye tracking and augmented reality technology according to an embodiment of the present invention;

[0055] Figure 2 is a flow chart of outputting a user's eye status in real time according to user's eye information according to one embodiment of the present invention;

[0056] Figure 3 3 is a schematic diagram of a method for correcting a first field of view into a second field of view including a plurality of secondary sub-fields of view according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0058] like Figure 1 As shown, the present invention provides a security and rescue method based on eye tracking and augmented reality technology, including S11-S17:

[0059] S11, obtaining a plurality of eye tracking models constructed based on different eye feature information, and establishing an eye tracking model set;

[0060] S12, obtaining user's eye feature information;

[0061] S13, determining a basic eye tracking model in the eye tracking model set according to the eye feature information of the user;

[0062] S14, modifying the basic eye tracking model according to the eye feature information of the user to obtain a target eye tracking model;

[0063] S15. Establishing a mapping relationship between the user's eyes and the original image directly in front of the user based on the target eye tracking model;

[0064] S16, performing recognition processing on the original image to obtain a recognition result;

[0065] S17. Providing rescue assistance to the user according to the recognition result and the mapping relationship.

[0066] The working principle of the above technical solution is: using a deep learning network based on AI technology to train multiple general eye tracking models established according to different eye feature information, establish an eye tracking model set, obtain the eye feature information of the current user (that is, the current rescuer), the eye feature information includes eye size, eye contour, pupil pattern and pupil area and other information of the current user's eyes; according to the eye feature information of the current user, determine the eye tracking model suitable for the user; because the model is a general eye tracking model with poor pertinence, the eye tracking model is corrected according to the eye feature information of the user to ensure the eye tracking model. This correction process can be performed automatically by the computer according to the eye feature information, or it can be participated in by the user; in the eye tracking After the eye tracking model is corrected, a target eye tracking model is obtained. The target eye tracking model establishes a mapping relationship between the user's eyes and the original image in front of the user's eyes, and the user's eye information determines the direction of the user's line of sight, thereby determining the area the user is looking at and the area not looking at. While determining the user's gaze area, the image in front of the user's eyes is recognized. Based on the image recognition result and combined with the mapping relationship, rescue assistance is provided to the user. For example, when the image directly in front of the user is a burning fire scene containing a large amount of burning wood, the eye tracking model is used to determine the fire area observed by the user and the fire area not observed. When the position not looked at by the user contains dangerous factors (such as burning wood, combustible materials that may cause explosions), the user is promptly reminded of the dangerous factors in the area outside the line of sight.

[0067] The beneficial effects of the above technical solution are as follows: obtaining and correcting the eye tracking model based on the user's eye feature information can improve the applicability of the eye tracking model to the user and improve its accuracy; constructing augmented reality technology based on image recognition technology and eye tracking technology; and providing users with visual enhancement-type rescue assistance based on the augmented reality technology, thereby ensuring the user's personal safety while improving rescue efficiency.

[0068] According to some embodiments of the present invention, the eye feature information includes at least one of iris pattern, iris area, and pupil area.

[0069] The working principle of the above technical solution is as follows: eye feature information includes at least one of iris pattern, iris area and pupil area, among which the iris pattern is used to determine the user's identity. The theoretical basis of this process is that each person's iris pattern is different; the iris area and pupil area are used for subsequent calculations.

[0070] The beneficial effects of the above technical solution are: the user identity is determined with high accuracy through iris patterns, and the matching speed of the eye tracking model is fast, saving a lot of time.

[0071] like Figure 2As shown, the present invention provides a flow chart for outputting the user's eye status in real time according to the user's eye information, including S21-S24:

[0072] S21. Calculate in real time the ratio of the user's iris area to the pupil area;

[0073] S22, obtaining ambient lighting information, and correcting the ratio value according to the ambient lighting information;

[0074] S23, comparing the ratio value with a preset value to obtain a comparison result;

[0075] S24. Determine the eye status of the user according to the comparison result and output it.

[0076] The working principle of the above technical solution is as follows: Under normal circumstances, the ratio of human iris area to pupil area is relatively fixed. When the physical condition changes dramatically, the ratio value will change relatively. For example, when a human is frightened or extremely weak, the pupil ratio will increase relatively. By analyzing the user's eye state, such as the iris area and pupil area, the user's physical condition can be predicted and output to remind the user to pay attention to the physical condition. At the same time, the target of receiving the output can also be a cloud server, reminding other rescuers to pay attention to the user's condition; since human pupils will change with changes in light, it is necessary to obtain ambient light information and correct the ratio value according to the ambient light information. Step S22 is exactly this process.

[0077] The beneficial effects of the above technical solution are: judging the user's eye condition based on partial feature information of the user's eyes and outputting it, the user's eye condition can be used to predict the user's physical condition, etc., and the personal safety of rescuers can be guaranteed by detecting the user's eye condition.

[0078] According to some embodiments of the present invention, performing recognition processing on the original image includes:

[0079] Dividing the original image into a plurality of image blocks;

[0080] Dividing the image blocks into a first type of image blocks and a second type of image blocks according to the target eye tracking model; the first type of image blocks are image blocks including the user's gaze point, and the second type of image blocks are image blocks not including the user's gaze point;

[0081] Performing image recognition on the first-category image blocks to obtain first identification information, and adding the first identification information to the first-category image blocks to obtain first-category target image blocks;

[0082] Performing image recognition on the second-category image blocks to obtain second identification information, and adding the first identification information to the second-category image blocks to obtain second-category target image blocks;

[0083] The first type of target image blocks and the second type of target image blocks are combined to obtain a target image and output it.

[0084] The working principle of the above technical solution is: the target eye tracking model can determine the position of the user's gaze point when the user looks at the original image, divide the original image into several image blocks, and take the image block containing the user's focus point as the first type of image block, and the remaining image blocks as the second type of image blocks, identify the first type of image blocks and the second type of image blocks, obtain first identification information and second identification information, mark the first type of identification information and the second type of identification information on the first type of image blocks and the second type of image blocks respectively, obtain first type of target image blocks and second type of target image blocks, combine the first type of target image blocks and the second type of target image blocks and output them.

[0085] The beneficial effects of the above technical solution are: recognizing the original image directly in front of the user, providing visual enhancement type rescue assistance, which is helpful for security rescue; dividing the image before image recognition helps to improve the image recognition rate, thereby saving rescue time; classifying and identifying image blocks according to the user's gaze point, so that the user can allocate image recognition computing power with emphasis according to the application scenario, improve environmental adaptability, and indirectly ensure rescue efficiency.

[0086] According to some embodiments of the present invention, modifying the basic eye tracking model according to the eye feature information of the user to obtain a target eye tracking model includes:

[0087] Determining a basic field of view range according to the basic eye tracking model;

[0088] Acquire the size of the user's eye area, and adjust the basic field of view range according to the size of the user's eye area to obtain a first field of view;

[0089] Obtaining the position of the pupil of the user when looking straight ahead, establishing a field of view rectangular coordinate system with the position of the pupil of the user when looking straight ahead as the coordinate origin, the horizontal direction as the abscissa direction, and the vertical direction as the ordinate direction;

[0090] Dividing the first field of view into four first-level sub-fields of view according to the four quadrants of the field of view rectangular coordinate system, and obtaining a first-level center point of each first-level sub-field of view;

[0091] Determine the midpoint of the line connecting two adjacent primary center points as the secondary center point;

[0092] Keeping the field of view rectangular coordinate system unchanged, the first field of view is re-divided according to the coordinate origin of the field of view rectangular coordinate system, the primary center point, and the secondary center point to obtain a second field of view including a plurality of secondary sub-fields of view;

[0093] Select several secondary center points as calibration stimulation points, and select several random points in the second visual field as verification stimulation points;

[0094] Performing a first-level correction on the basic eye tracking model according to the first feedback information of the user on the calibration stimulation point to obtain a first-level correction model;

[0095] Verifying the first-level correction model according to the second feedback information of the user on the verification stimulation point to obtain a first-level correction error value;

[0096] Comparing the first-level corrected error value with a preset error value to obtain a comparison result;

[0097] When it is determined that the first-level correction error value is greater than the preset error value, the first-level correction model is corrected twice according to the calibration stimulation point to obtain a second-level correction model; the above process is repeated and iterated until the i-th level correction error value is less than the preset error value;

[0098] The i-level correction model obtained by the i-level correction error value being smaller than the first-level preset error value is used as the target eye tracking model.

[0099] like Figure 3 As shown, the present invention provides a method for correcting a first field of view into a second field of view including a plurality of secondary sub-fields of view, including 3-1 to 3-3:

[0100] 3-1. Obtaining the size of the user's eye area, adjusting the basic field of view according to the size of the user's eye area to obtain a first field of view, and simultaneously obtaining a center point a of the first field of view;

[0101] 3-2. With the center point a as the coordinate origin, establish a horizontal x-axis and a vertical y-axis, and divide the first field of view into four quadrants according to the four quadrants divided by the horizontal and vertical axes to obtain four primary sub-fields of view. The dividing lines are shown as dotted lines.

[0102] 3-3. Obtaining the primary center point of the primary sub-field of view. The primary center point of the primary sub-field of view is indicated by mark b.

[0103] 3-4. Determine the midpoint of the line connecting two adjacent primary center points as the secondary center point, the secondary center point is indicated by mark c;

[0104] 3-5. Keeping the horizontal and vertical axes unchanged, re-divide the first field of view according to the center point of the first field of view, the first-level center point, and the second-level center point to obtain a second field of view including 9 second-level sub-fields of view. The dividing lines of the second-level sub-fields of view are shown as dotted lines.

[0105] It should be noted that Figure 3 The method provided includes two divisions of the first field of view. The first process of dividing the first field of view into several first-level sub-fields of view using the horizontal and vertical axes is linear, while the process of dividing the first sub-field of view according to the center point of the first field of view, the first-level center point, and the second-level center point is nonlinear. Therefore, the second-level center point does not coincide with the center point of the second-level sub-field of view, and the second-level center point is not consistent with the center point of the second-level sub-field of view (which will be used as calibration stimulus points later) and needs to be distinguished. In actual application, the process of dividing the first field of view into several second fields of view includes several division processes, and these divisions of the first field of view can be linear or nonlinear. Figure 3 The methods given are merely examples to illustrate this solution and do not constitute a limitation of this solution.

[0106] The working principle of the above technical solution is as follows: obtaining the basic field of view range in the target eye tracking model, adjusting the basic field of view range according to the eye size in the user's eye feature information, obtaining a first field of view, dividing the first field of view into a second field of view containing several secondary sub-fields of view, and obtaining the center point of the second field of view as a calibration stimulus point, and randomly obtaining several points from the second field of view as verification stimulus points; correcting the basic eye tracking model according to the user's eye reaction to the calibration stimulus point. During the correction process, judging whether the correction is effective based on the user's reaction to the verification stimulus point (the correction error value will be calculated in this process); if the correction is invalid, iteratively correcting the basic eye tracking model according to the calibration stimulus point until the correction is completed (that is, the correction error value is lower than the preset correction error value).

[0107] The beneficial effects of the above technical solution are: the basic eye tracking model is modified in a targeted manner according to the user's eye information to obtain the target eye tracking model, thereby improving the accuracy of the target eye tracking model and thus improving the rescue efficiency.

[0108] According to some embodiments of the present invention, in the process of modifying the basic eye tracking model, an upper limit k of the number of modification iterations of the basic eye tracking model is set;

[0109] After each iterative correction of the basic eye tracking model, determining whether the current number of iterations exceeds the upper limit k of the corrected number of iterations, and obtaining a determination result;

[0110] When it is determined that the current number of iterations exceeds the upper limit k of the number of modified iterations, the basic eye tracking model is reacquired for modification.

[0111] The working principle of the above technical solution is: during the process of correcting the basic eye tracking model, if the number of correction iterations exceeds a preset value, it means that the eye tracking model is not suitable for the user, and the basic eye tracking model is re-acquired to perform correction.

[0112] The beneficial effects of the above technical solution are: users can adjust the upper limit k of the number of iterations according to their needs, thereby adjusting the accuracy of the rescue assistance provided by this solution according to their needs, avoiding the situation where high-precision rescue assistance is used for low-precision rescue tasks and wasting time and resources.

[0113] According to some embodiments of the present invention, verifying the first-level correction model based on the second feedback information of the user on the verification stimulation point to obtain the first-level correction error value includes:

[0114] Establishing a spherical coordinate system with the verification stimulus point as the spherical coordinate origin;

[0115] Calculating the theoretical position of the user's pupil when the user is gazing at the verification stimulus point according to the first-level correction model, and obtaining the theoretical position coordinates of the theoretical position of the user's pupil according to the spherical coordinate system;

[0116] Obtaining the actual position of the pupil of the user when the user is gazing at the verification stimulus point, and obtaining the actual position coordinates of the user's pupil according to the spherical coordinate system;

[0117] The first-level correction error value is obtained by substituting the theoretical position coordinates of the user's pupil and the actual position coordinates of the user's pupil into a correction error value calculation algorithm.

[0118] Furthermore, the correction error value calculation method includes:

[0119] Since there are several verification stimulation points, there are also several corresponding theoretical position coordinates of the user's pupil, which are recorded as (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3)... (Xn, Yn, Zn), where n is a counting number. According to the properties of the spherical coordinate system, X represents the elevation angle between the theoretical position of the user's pupil and the origin of the spherical coordinate system, and its unit is degree. Y represents the azimuth angle between the theoretical position of the user's pupil and the origin of the spherical coordinate system, and its unit is degree. Z represents the user's pupil. The distance between the theoretical position and the origin of the spherical coordinates is in meters; extracting the elevation angle values ​​in the theoretical position coordinates of the plurality of user pupils, establishing a theoretical elevation angle value set {X1, X2, X3...Xn}, normalizing the theoretical elevation angle value set to obtain a theoretical elevation angle normalized value set {NX1, NX2, NX3...NXn}; calculating the root mean square average value NX of the theoretical elevation angle normalized values ​​according to the theoretical elevation angle normalized value set {NX1, NX2, NX3...NXn}. rms :

[0120]

[0121] According to the same method, the root mean square average value NY of the theoretical azimuth normalized value is obtained rms And the theoretical distance normalization value NZ rms ;

[0122] Similarly, according to the actual position coordinates of the user's pupil, the root mean square average value of the actual elevation angle normalized value NA rms , the root mean square average value NB of the actual azimuth normalized value rms And the actual distance normalized value NC rms ;

[0123] According to NX rms ,NY rms , NZ rms , NA rms NB rms and NC rms Substitute the corrected error value into the calculation formula to obtain the corrected error value. The corrected error value formula is: Among them, α is the elevation error weight, β is the azimuth error weight, γ is the distance error weight, α+β+γ=1, according to the imaging properties of the human eye, α<β<γ; the α, β and γ values ​​can be adjusted according to the user's eye feature information.

[0124] The working principle of the above technical solution is: a spherical coordinate system is established with the verification stimulus point as the origin of the spherical coordinate system, and the coordinates of the theoretical position of the user's pupil when the user responds to the verification stimulus point are calculated based on the corrected eye tracking algorithm. At the same time, the coordinates of the actual position of the user's pupil when the user responds to the verification stimulus point are obtained, and the theoretical position coordinates and the actual position coordinates are substituted into the correction error value calculation algorithm to calculate the correction error value.

[0125] The beneficial effects of the above technical solution are as follows: the correction error is quantified, and the correction effect on the basic eye tracking model is judged based on the numerical value of the correction error. This method is simple to implement and highly efficient, which reduces the workload of verifying the correction error and improves work efficiency.

[0126] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A security and rescue method based on eye tracking and augmented reality technology, characterized in that: include: Obtain several eye tracking models based on different eye feature information and establish an eye tracking model set; Obtain user's eye feature information; Determining a basic eye tracking model in the eye tracking model set according to the eye feature information of the user; Modifying the basic eye tracking model according to the eye feature information of the user to obtain a target eye tracking model; Establishing a mapping relationship between the user's eyes and an original image directly in front of the user based on the target eye tracking model; Performing recognition processing on the original image to obtain a recognition result; providing rescue assistance to the user according to the recognition result and the mapping relationship; The basic eye tracking model is modified according to the eye feature information of the user to obtain a target eye tracking model, including: Determining a basic field of view range according to the basic eye tracking model; Acquire the size of the user's eye area, and adjust the basic field of view range according to the size of the user's eye area to obtain a first field of view; Obtaining the position of the pupil of the user when looking straight ahead, establishing a field of view rectangular coordinate system with the position of the pupil of the user when looking straight ahead as the coordinate origin, the horizontal direction as the abscissa direction, and the vertical direction as the ordinate direction; Dividing the first field of view into four first-level sub-fields of view according to the four quadrants of the field of view rectangular coordinate system, and obtaining a first-level center point of each first-level sub-field of view; Determine the midpoint of the line connecting two adjacent primary center points as the secondary center point; Keeping the field of view rectangular coordinate system unchanged, the first field of view is re-divided according to the coordinate origin of the field of view rectangular coordinate system, the primary center point, and the secondary center point to obtain a second field of view including a plurality of secondary sub-fields of view; Select several secondary center points as calibration stimulation points, and select several random points in the second visual field as verification stimulation points; Performing a first-level correction on the basic eye tracking model according to the first feedback information of the user on the calibration stimulation point to obtain a first-level correction model; Verifying the first-level correction model according to the second feedback information of the user on the verification stimulation point to obtain a first-level correction error value; Comparing the first-level corrected error value with a preset error value to obtain a comparison result; When it is determined that the first-level correction error value is greater than the preset error value, performing a second correction on the first-level correction model according to the calibration stimulation point to obtain a second-level correction model; repeating the process of obtaining the target eye tracking model and iterating until the i-th level correction error value is less than the preset error value; The i-level correction model obtained by having the i-level correction error value be smaller than the preset error value is used as the target eye tracking model.

2. The security and rescue method based on eye tracking and augmented reality technology according to claim 1, characterized in that: After obtaining the target eye tracking model, it also includes: A correspondence between the user's eye feature information and the target eye tracking model is established and stored.

3. The security and rescue method based on eye tracking and augmented reality technology according to claim 1, characterized in that: The eye feature information includes at least one of iris pattern, iris area and pupil area.

4. The security and rescue method based on eye tracking and augmented reality technology according to claim 3, characterized in that: Also includes: Calculating in real time the ratio of the user's iris area to the pupil area; Comparing the ratio value with a preset value to obtain a comparison result; The user's eye state is determined based on the comparison result and output.

5. The security and rescue method based on eye tracking and augmented reality technology according to claim 4, characterized in that: Also includes: Get ambient lighting information; The ratio value is corrected according to the ambient lighting information.

6. The security and rescue method based on eye tracking and augmented reality technology according to claim 1, characterized in that: Performing recognition processing on the original image includes: Dividing the original image into a plurality of image blocks; Dividing the image blocks into a first type of image blocks and a second type of image blocks according to the target eye tracking model; the first type of image blocks are image blocks including the user's gaze point, and the second type of image blocks are image blocks not including the user's gaze point; Performing image recognition on the first-category image blocks to obtain first identification information, and adding the first identification information to the first-category image blocks to obtain first-category target image blocks; Performing image recognition on the second-category image blocks to obtain second identification information, and adding the first identification information to the second-category image blocks to obtain second-category target image blocks; The first type of target image blocks and the second type of target image blocks are combined to obtain a target image and output it.

7. The security and rescue method based on eye tracking and augmented reality technology according to claim 1, characterized in that: Also includes: Set the upper limit k of the number of iterations for correcting the basic eye tracking model; After each iterative correction of the basic eye tracking model, determining whether the current number of iterations exceeds the upper limit k of the corrected number of iterations, and obtaining a determination result; When it is determined that the current number of iterations exceeds the upper limit k of the number of modified iterations, the basic eye tracking model is reacquired for modification.

8. The security and rescue method based on eye tracking and augmented reality technology according to claim 1, characterized in that: Performing a first-level correction on the basic eye tracking model according to the first feedback information of the user on the calibration stimulation point includes: Calculating, based on the basic eye tracking model, a theoretical position of the user's pupil when the user gazes at the calibration stimulation point, and establishing a first association relationship between the theoretical position of the user's pupil and the calibration stimulation point; Acquiring the actual position of the user's pupil when the user gazes at the calibration stimulation point, and establishing a second association relationship between the actual position of the user's pupil and the calibration stimulation point; According to the second association relationship, the first association relationship is modified to achieve a first-level correction of the basic eye tracking model.

9. The security and rescue method based on eye tracking and augmented reality technology according to claim 1, characterized in that: Verifying the first-level correction model according to the second feedback information of the user on the verification stimulation point to obtain a first-level correction error value includes: Establishing a spherical coordinate system with the verification stimulus point as the spherical coordinate origin; Calculating the theoretical position of the user's pupil when the user is gazing at the verification stimulus point according to the first-level correction model, and obtaining the theoretical position coordinates of the theoretical position of the user's pupil according to the spherical coordinate system; Obtaining the actual position of the pupil of the user when the user is gazing at the verification stimulus point, and obtaining the actual position coordinates of the user's pupil according to the spherical coordinate system; The first-level correction error value is obtained by substituting the theoretical position coordinates of the user's pupil and the actual position coordinates of the user's pupil into a correction error value calculation algorithm.

Images