Helmet display servo system and method based on reinforcement learning

Abstract

The present invention relates to a reinforcement learning-based helmet display tracking system, comprising a helmet body, a reflection module, an external environment observation camera, a projection module, and a main control board. The external environment observation camera is mounted on the outer surface of the helmet and connected to the main control board, and is used to collect video data of the external environment, determine whether the road in the external environment is a structured road, and transmit the judgment result and the external environment audio and video data to the main control board. The reflection module is controlled and connected to the main control board, which controls the movement of the reflection module in two degrees of freedom. The projection module is used to project image information onto the reflection module. When located on a structured road, the main control board adjusts the reflection module using the two degrees of freedom, so that the image information is observed by the human eye after passing through the reflection module. The human eye also observes the image information while observing the external environment.

Description

Technical Field

[0001] The present invention belongs to the field of display technology, and specifically relates to a follow-up system for a helmet display, which is a system in which the display in the helmet follows the human eye, thereby dynamically adjusting the helmet display with the human eye observation effect as the main goal. Background of the Invention

[0002] A motorcycle rider's helmet is a tool to protect the rider's life safety. Many riders need to lower their heads to observe the mobile phone navigation or make and receive calls while driving, which has an adverse impact on the rider's life safety.

[0003] If there is a product based on VR\AR technology that projects the mobile phone navigation information and phone information required by the rider in front of the helmet, making it easier for the rider to understand the information and avoiding the rider having to look down, then the momentary danger caused by looking down can be reduced.

[0004] Existing helmet display solutions primarily use a projection module to project the desired content onto a reflective panel in a corner of the rider's field of view, allowing the user to observe the reflected content. Alternatively, a display screen is placed directly in this corner, allowing the user to observe the content on the screen. While this structure is simple and easy to manufacture, the disadvantage is that the user must constantly observe the content on the panel or display screen.

[0005] If there is a helmet display solution that can project the required display content directly into the observation range of the human eye and minimize the movement distance of the human eye during observation, then compared with the existing technology, it can reduce the problem of riders' inattention caused by observing navigation and phone information. Summary of the Invention

[0006] Based on existing artificial intelligence technology, the present invention proposes a rider-oriented helmet display following system based on reinforcement learning. The position of the helmet display image is a following system that dynamically adjusts the helmet display image based on the human eye's observation effect.

[0007] The present invention discloses a helmet display tracking system based on reinforcement learning, comprising a helmet body, a reflection module, an external environment observation camera, a projection module, and a main control board. The reflection module, external environment observation camera, projection module, and main control board are all mounted on the helmet body. The external environment observation camera is mounted on the outer surface of the helmet and connected to the main control board, and is configured to collect video data of the external environment, determine whether the road in the external environment is a structured road, and transmit the determination result and audio and video data of the external environment to the main control board. The reflection module is controlled and connected to the main control board, which controls the movement of the reflection module in two degrees of freedom: the pitch angle βin and the roll angle γin of the reflection module. The projection module is configured to project image information onto the reflection module. When the vehicle is on a structured road, the main control board adjusts the reflection module using the two degrees of freedom, so that the image information is observed by the human eye after passing through the reflection module. The human eye also observes the image information while observing the external environment.

[0008] Furthermore, the reinforcement learning-based helmet display tracking system of the present invention also includes an eyeball observation camera; the eyeball observation camera is connected to the main control board, and the eyeball observation camera is used to record video data of the human eye and transmit the video data of the human eye to the main control board, and the main control board extracts the position of the pupil in each frame of the video; the pitch angle βin and roll angle γin of the reflection module are adjusted by the main control board so that the 1-norm distance DSP between the center of the stable point SP1 of the pupil when the rider's eyes observe the external environment and the center of the stable point SP2 of the pupil when the rider's eyes observe image information is minimized.

[0009] Furthermore, the calculation process of the 1-norm distance DSP between the centers of the stable point SP1 and the stable point SP2 is as follows:

[0010] The classic k-means algorithm is used to cluster the pupil locations in each frame of the image, and two regions are clustered. One region close to the center of the Iep graph represents the stable point SP1 = {SP1x, SP1y} of the pupil when the rider's eyes observe the external environment; the other region represents the stable point SP2 = {SP2x, SP2y} of the pupil when the rider's eyes observe image information during normal driving.

[0011] Calculate the 1-norm distance DSP between the centers of the stable points SP1 and SP2

[0012] DSP=||SP2x-SP1x||+||SP2y-SP1y||.

[0013] Furthermore, a method for minimizing the 1-norm distance DSP between the centers of the stable point SP1 and the stable point SP2 is as follows: based on the Q-Learning algorithm, the minimum value of DSP is found by adjusting the position of the image information, that is, adjusting the pitch angle βin and roll angle γin of the reflection module.

[0014] Furthermore, in the Q-Learning algorithm, the reward r is set and the pitch angle βin and roll angle γin are cyclically calculated.

[0015] Adjustment: When the reward r is 0 for n consecutive times, it is considered that the minimum distance DSP is found;

[0016] r is represented by:

[0017]

[0018] Wherein, DSPnew(j+1) is the DSPnew distance of the j+1th iteration, and DSPnew(j) is the DSPnew distance of the jth iteration.

[0019] Furthermore, the method for determining whether a road in the external environment is a structured road is: using an adaptive region of interest calculation method to separate the road area from the non-road area, and then performing structured road determination.

[0020] Furthermore, when the rider is wearing a helmet and lowering his head, the region of interest is the lower half of the external environment video image; when the rider is wearing a helmet and raising his head, the region of interest is set to the lower half of the external environment video image from bottom to top. The area with pixel height is Cout, where Cout is the monitoring angle of the external observation camera, β out It is the angle at which the rider looks up.

[0021] The reinforcement learning-based helmet display tracking method of the present invention is based on the tracking system of the present invention, and the method comprises the following steps:

[0022] Step 1: Using the external environment observation camera installed on the helmet, collect a video dataset V1 of the external environment being a structured road. The video dataset includes multiple video segments, where V1(i) represents the i-th video segment in the video dataset V1, and the length of each video segment is greater than 5 seconds. Denoise the video dataset V1 of the structured road to obtain a denoised video dataset V3 of the structured road.

[0023] Step 2: extracting the eyeball change video set VV1 corresponding to the video data set V3, wherein the eyeball change video set VV1 is an eyeball change video when the external environment is stable;

[0024] Step 3, extracting the pupil position in each frame of the eyeball change video set VV1 to form an image Iep;

[0025] Step 4: Use the classic k-means algorithm to cluster the pupil positions in each frame of the image, and cluster out two regions. One of the regions close to the center point of the Iep graph represents the stable point SP1 = {SP1x, SP1y} of the pupil when the rider's eyes observe the external environment during normal driving;

[0026] Another area shows that when the rider is driving normally, the stable point of the pupil SP2 = {SP2x, SP2y} when the rider's eyes observe image information (such as VR\AR path planning information), and the 1-norm distance DSP of the center of SP1 and SP2 is calculated.

[0027] DSP=||SP2x-SP1x||+||SP2y-SP1y||

[0028] Step 5: Based on the Q-Learning algorithm, the minimum value of DSP is found by adjusting the position of the image information, that is, adjusting the pitch angle βin and the roll angle γin of the reflection module.

[0029] Furthermore, in step 1, the structured road video dataset V1 is subjected to denoising processing to remove video segments with large speed changes, thereby obtaining a denoised structured road video dataset V3.

[0030] The specific steps include:

[0031] Step 1.1: Use the nine-axis acceleration sensor on the helmet to collect the velocity Vxyz corresponding to each frame in the structured road video dataset V1.

[0032]

[0033] in, represents the speed of the helmet movement corresponding to the t-th frame, and the t-th frame is located in the i-th video;

[0034] Indicates the speed of the helmet moving in the x direction corresponding to the tth frame;

[0035] Indicates the speed of the helmet moving in the y direction corresponding to the tth frame;

[0036] Indicates the speed of the helmet moving in the z direction corresponding to the tth frame;

[0037] The x-direction refers to the direction of the line from the left eye to the right eye when the person wears the helmet. The y-direction refers to the direction starting from the middle point of the eyes, perpendicular to the x-direction and pointing horizontally forward. The z-direction refers to the direction starting from the middle point of the eyes, perpendicular to the x-direction and pointing vertically upward.

[0038] Step 1.2: Calculate the average speed value Avg(Vxyz(i)) and variance var(Vxyz(i)) of each video segment, where Avg(Vxyz(i)) represents the average speed value of the i-th video segment, and var(Vxyz(i)) represents the speed variance value of the i-th video segment.

[0039] Step 1.3: Determine whether the helmet velocity Vxyz(ti) corresponding to the t-th frame in the i-th video segment meets the 3S quality standard. The expression is as follows:

[0040]

[0041] Traverse all frames in the i-th video. If there is a frame in the i-th video that does not conform to the above expression, delete the frame;

[0042] Step 1.4, traverse all video segments in the video dataset V1, repeat step 1.3, and finally obtain the video dataset V2;

[0043] Step 1.5, retaining video segments of more than n seconds in the video dataset V2 as the denoised video dataset V3, where n=5 in the present invention;

[0044] Beneficial effect: The image projection position is changed according to the range of human eye observation, actively adapting to the human eye observation range. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a front view of a schematic diagram of various hardware relationship modules in an embodiment of the present invention;

[0046] Figure 2 is a side view of a schematic diagram of various hardware relationship modules in an embodiment of the present invention;

[0047] Figure 3 is a schematic diagram of the structural relationship of various hardware modules in an embodiment of the present invention;

[0048] Figure 4 is a flow chart showing the relationship between various algorithm modules in an embodiment of the present invention;

[0049] Among them, 1-helmet body, 2-goggles, 3-helmet ear area, 4-power port, 5-backup power supply, 6-projection module, 7-reflection module, 8-eyeball observation camera, 9-external observation camera, 10-earphones. DETAILED DESCRIPTION

[0050] In the reinforcement learning-based helmet display tracking system of the present invention, the angle of image projection is adaptively changed according to the gaze direction of the human eye, reducing the movement distance required for human eye observation, thereby adapting to human eye observation.

[0051] When determining the direction of human eye gaze, due to the problem of riders' eye usage habits, for example, some riders are accustomed to observing with peripheral vision, some riders have different gaze points and eyeballs, and some riders have esotropia, lateral strabismus, exotropia, etc. Therefore, the traditional method of simply calculating the focus point cannot solve the problem faced by the present invention. At this time, it is necessary to first solve the problem of human eye observation tracking.

[0052] The reinforcement learning-based helmet display servo system of the present invention includes a hardware system and a software system.

[0053] 1. Hardware system

[0054] like Figure 1 As shown, the helmet display tracking system based on reinforcement learning includes a helmet body 1, goggles 2, a main control board, a power port 4, a backup power supply 5, a projection module 6, a reflection module 7, an eye observation camera 8, an external observation camera 9, an external light sensor, an earphone 10, a button, and a prompt light;

[0055] The main control board, power port, external light sensor, buttons and indicator lights are all located on the surface of the helmet body and in the ear area 3 of the helmet;

[0056] 1.1, the helmet body 1 protects the rider's head safety and is worn on the rider's head like a common helmet;

[0057] 1.2. The goggles 2 are connected to the helmet body 1. Like ordinary goggles 2, they are positioned on the rider's face. They are transparent or translucent panels that can be moved up and down by the rider. The goggles 2 are rotatably connected to the helmet body 1, and a photoelectric sensor is provided at the connection between the goggles 2 and the helmet body 1. The photoelectric sensor is connected to the main control board and is used to detect the current position of the goggles 2, including the upward and downward states.

[0058] When the goggles 2 are in the lowered position, blocking the front of the user's face, the photoelectric sensor activates the servo system of the present invention through the main control board; if the goggles 2 are in the upper position, the photoelectric sensor deactivates the servo system of the present invention through the main control board. This function can also be canceled by the rider.

[0059] 1.3. The main control board, power port, backup power supply 54, button, and indicator light are all located on the side of the helmet body 1. The power port is connected to the main control board and can operate in both external power and backup power 5 modes. The power port is a Type-C interface. When not in use, an external rubber cover can be placed over the port to prevent water ingress.

[0060] Among them, the backup power supply 5 is set on the outer surface of the helmet, protected by a plastic shell, and can be plugged in and replaced. It can be a 5V10000mA power bank or other configurations. It can be charged by the power port and can also provide power to the main control board and other circuits.

[0061] 1.4. The external light sensor is located on the surface of the helmet body 1 and is connected to the main control board. When the rider wears the helmet in and out of areas with large light variations, such as tunnels, the main control board adjusts the projection brightness based on the external light, dimming dark areas and brightening bright areas to reduce eye discomfort. In this embodiment, the BH1750FVI module for Bluetooth communication is used.

[0062] 1.5, earphones 10 are set inside the helmet, corresponding to the position of the ears, for receiving information from the main control board and prompting the rider to operate;

[0063] 1.6, the warning light is on the outside of the helmet, used to remind the rider and outsiders of its position and status.

[0064] 1.7. The buttons include wired buttons and wireless buttons. The wired buttons are located on the right side of the helmet. The wireless buttons use Bluetooth communication and are located on the handlebars of the rider's bike.

[0065] The projection module 6, the two-degree-of-freedom reflection module 7, the eyeball observation camera 8, and the road observation camera are all arranged inside the helmet body 1. The eyeball observation camera 8 is located in the jaw guard portion of the helmet body 1 and faces the eyeball, which is conducive to the stability of the eyeball observation.

[0066] 1.8, projection module 6 is located on the helmet surface and faces 2DOF reflective module 7. Controlled by the main control board, it projects the rider's desired information onto 2DOF reflective module 7. 2DOF reflective module 7 can be a reflective lens, projecting information onto goggles 2, or it can be a projection screen for direct observation, creating a VR / AR effect. In this embodiment, the m100smart micro-projector core from Shenzhen Weimai Technology Co., Ltd. is used.

[0067] 1.9. The 2-DOF reflector module 7 is located at the lower front of the helmet interior. Its 2-DOF angle transformation is controlled by the main control board, which also provides feedback. The main control board modifies the image quality on the goggles 2 by changing the 2-DOF angle. It also receives feedback on the current 2-DOF angle, controlling the image distortion to maintain a normal state when projected onto the goggles 2, thus accommodating human observation. In this embodiment, the reflector uses a 400-700nm visible light reflector produced by a platinum-radium laser. The 2-DOF reflector module 7 is driven by two rotary motors, each with two degrees of freedom. The reduction motor consists of a 24GA100-N10VA-001 DC brushed motor and an inverted gearbox. The reduction motor also consists of the same model motor and a tower-shaped gearbox. The gearbox reduction ratio is 1:50. Compared to brushless DC motors, which have longer lifespans and wider applications, brushed DC motors were chosen due to their compact size, which meets the requirements for the compact design of the anti-shake scoop. The angle encoder installed on the motor load shaft is KMZ60, which is specifically used for detecting the rotation angle of the DC motor. These two degrees of freedom correspond to the pitch angle βin and roll angle γin of the reflector module 7 toward the helmet, where in represents the camera facing inward. The general rotation scheme has three degrees of freedom, but this scheme uses two degrees of freedom. The reason is that if the reflector module 7 in this scheme is made with three degrees of freedom, the structural size requirements will be relatively large, making it difficult to process and manufacture. Therefore, this scheme fixes the inward angle of the reflector module 7 at the front and bottom of the helmet, so the yaw angle αin is physically fixed to 0, retaining only the pitch angle βin and roll angle γin, which can reduce the volume.

[0068] In the present invention, the pitch angle βout is the pitch angle of the helmet relative to the inertial coordinate system of the human eye, which is the plane (XOYout) formed by the line from the left eye to the right eye (Xout axis) and the line perpendicular to this line and parallel to the front line (Yout axis). Therefore, the present invention directly combines the βout angle with the monitoring angle Cout of the external observation camera 9 to obtain the region of interest. This calculation method is simple and reliable. Assuming that the vertical pixels captured by the camera are Hout

[0069] The eyeball observation camera 8 is located in the front of the helmet, facing inward toward the eyeball. It is used to observe the state of the human eye and provide it to the main control board. In this embodiment, it is a customized pinhole camera with a resolution of 5 million pixels, and the data is stored on the main control board.

[0070] 1.10. External observation camera 9 is located inside the helmet, facing outward. It observes external conditions and provides data to the main control board for analysis. In this embodiment, it is a Steelmate dashcam with a resolution of 5 megapixels. It records sound and stores data on an 8GB SD card on the main control board.

[0071] 1.11, the main control board includes a high-performance computing module, a GPS positioning module, a nine-axis acceleration sensor (speed, movement direction, and angle detection), Bluetooth, a 5G communication module, an SD card, etc.

[0072] The high-performance computing module is used to provide computing resources for the algorithm. In the embodiment of the present invention, an STM32+GPU board is used.

[0073] The GPS positioning module is used to issue an alarm in case of an accident.

[0074] After obtaining data, the nine-axis acceleration sensor is used to match the video observed by the road observation camera and then analyze and use it. In this embodiment, the DFRobot BMX160 is used.

[0075] Bluetooth is used to connect to an external wireless (Bluetooth) button and receive button operation information.

[0076] The 5G communication module is used for communication with external networks.

[0077] The SD card is a module for storing driving records. This embodiment uses an 8G micro SD card.

[0078] The main control board receives input signals from the eye observation camera 8, the external observation camera 9, the external light sensor, and the buttons. It also controls the projection module's image and other information output in two degrees of freedom. These images and other information are then reflected by the reflection module 7 and observed by the human eye. It also controls the output of the earphones 10 and the indicator light.

[0079] 2. Software and algorithm system

[0080] The software and algorithm system includes a human eye stability analysis module and a following module.

[0081] 2.1 Human Eye Stability Analysis Module

[0082] The primary function of the Eye Stability Analysis Module is to analyze eye stability after acquiring data. To accomplish this, the module collects data on eye changes, external environment data, and data from a nine-axis acceleration sensor. The external environment data refers to information about the road the rider travels.

[0083] Normally, when human vision is stable and reliable, the pupil should remain stable and largely motionless, except for blinking. This is the desired effect of VR / AR systems. If the pupil remains unstable and constantly moving, this may indicate that the projection position of the present invention is not suitable for human observation, and the tracking system is unreliable. However, it may also be due to the complex external environment and constant interference. Therefore, the present invention eliminates the instability caused by these external interference factors and then analyzes the stability of human observation.

[0084] The analysis of external change data mainly uses external observation cameras 9 to observe external images. The principle is to first determine whether the road is a structured road. If it is not a structured road, but an unstructured road, such as a crowded alley, it is difficult to determine the stability, so this data is eliminated.

[0085] This solution uses two cameras: an external observation camera 9 and an eye observation camera 8. The external observation camera 9 serves two functions: first, it acts as a dashcam, recording various driving information; second, it is used in the data culling process of the present invention, determining whether the external road is structured and removing videos with head movements. From the remaining videos, the time frames are extracted, and the information from the eye observation camera 8 in the corresponding time frames is extracted for further eye movement analysis. The eye observation camera 8 is designed to continuously face the rider's eyes and observe eye movements.

[0086] 2.1.1 Structured Road Recognition

[0087] To minimize computational complexity, this paper adopts the solution from "Research on Structured Road Recognition Algorithms Based on Image Processing" by Lin Lianghao of Harbin University of Science and Technology. This solution primarily relies on traditional algorithms, resulting in a low computational load. Despite this low computational load, this paper still uses a small modification because it only needs to detect stable straight roads, thereby extracting videos of vehicles traveling on structured roads (such as highways and urban expressways).

[0088] In the proposal of "Research on Structured Road Recognition Algorithm Based on Image Processing", first, for the preprocessing of road images, an adaptive region of interest calculation method is proposed to separate the road area from the non-road area, eliminate noise interference and improve the efficiency of lane line recognition; to address the problem of unclear lane lines under different light intensities, it is proposed to perform corresponding grayscale stretching on road images with different light intensities to make the transformed lane lines easier to identify.

[0089] The present invention improves the adaptive region of interest calculation method of the original solution. Since the adaptive region of interest calculation method thereof will occupy a certain amount of calculation, the nine-axis acceleration sensor in the hardware of the present invention can provide, for example, the pitch angle β out Information, pitch angle β out In the present invention, the pitch angle of the helmet relative to the plane (XOYout) formed by the line from the left eye to the right eye (Xout axis) and the line perpendicular to this line and parallel to the front line (Your axis) is the pitch angle. Simply put, it is the angle of the rider's head up and down, which is a variable. Therefore, the present invention directly uses β out The angle is directly combined with the monitoring angle Cout range of the external observation camera 9 to obtain the area of ​​interest. This calculation method is simple and reliable. Assuming that the vertical pixel captured by the camera is Hout, the method is as follows:

[0090] When β out When <=0, the rider is in the state of lowering his head. In this state, the region of interest is directly set to the area of ​​Hout / 2 pixels height of the lower half of the camera photographed by the external observation camera 9 (the width remains unchanged), thereby reducing the amount of calculation.

[0091] When β out When it is >0, it is the state of the rider looking up. In this state, the region of interest is set to the lower half of the camera taken from the external observation camera 9 from the bottom up. The area with the height of pixels (the width remains unchanged) is used to reduce the amount of calculation.

[0092] Secondly, an improved Canny algorithm is used to detect edges in road images. To address the traditional Canny edge detection algorithm's issues with weakening important edges and poor adaptability, bilateral filtering is used instead of Gaussian filtering. Multiple directional gradient templates are used to calculate image gradients, and the maximum inter-class variance method is used to determine the image's high and low thresholds. This effectively improves the algorithm's denoising performance and edge detection accuracy.

[0093] Finally, the solution in "Research on Structured Road Recognition Algorithm Based on Image Processing" is used for the recognition of straight roads.

[0094] When a structured road is identified, it is recorded. If there is a structured road for 5 consecutive seconds, the video of the straight road captured by the external observation camera 9 is retained and recorded as V1(i), which represents the video of the i-th structured road. Multiple videos of structured roads constitute the video set V1. The number of frames of each i-th video starts from the smallest frame number. To the maximum frame number f stands for frame;

[0095] Here, for example, if there are 3 structured road videos within 5 minutes Assuming 10 frames per second, we can further assume that Indicates frame 0, represents the 50th frame, then, V1(1) = {0-50}, indicating that the first video segment in the video set V1 includes images from the 0th frame to the 50th frame; Indicates the 70th frame, represents the 170th frame, then, V1(2) = {70~170}, which means that the second video segment in the video set V1 includes images from the 70th frame to the 170th frame; Indicates the 300th frame, Represents the 500th frame, then, V1(3) = {300~500}, which means that the third video in the video set V1 includes images from the 300th frame to the 500th frame, that is, it can be understood that the first video segment is a 5-second video from the 0th to the 5th second, the second video segment is a 10-second video from the 7th to the 17th second, and the third segment is a 20-second video from the 30th to the 50th second, totaling 35 seconds of video.

[0096] 2.1.2 Remove videos with head movements

[0097] Extract the velocity module from the nine-axis accelerometer in the hardware to obtain the velocities Vx, Vy, and Vz of the line from the left eye to the right eye (X-axis), the line perpendicular to the X-axis and parallel to the front line (Y-axis) from the center point of the eyes, and the line perpendicular to the X-axis and directly upward (Z-axis) from the center point of the eyes. Add the absolute values ​​(i.e., eliminate the direction) to obtain the helmet movement velocity Vxyz:

[0098]

[0099] in, Represents the speed of the helmet movement corresponding to the t-th frame, and the t-th frame is located in the i-th video. Calculate the speed average Avg(Vxyz(i)) and variance var(Vxyz(i)) in the i-th video of the video set V1.

[0100] Here, following the example from the previous section, It is one of the frames in the 35-second video (V1(1), V1(2), V1(3)). Assuming 10 frames per second, The maximum value at this time is 350 frames.

[0101] Then, in the frame images of each video, according to the 3S quality standard, that is, the following standard, only all videos that meet this area are retained, and other frames that do not meet the requirements are eliminated to obtain the V2 video set.

[0102]

[0103] Then, in V2, only the video with no frames deleted for 5 consecutive seconds is retained as the video V3 that can be used for analysis without external interference factors, and the time number ti3 of its video frame is obtained, that is, from the external image video taken by the external observation camera 9 with a length of ti1max, the video of the corresponding time frame ti3 of VV in the eye change video taken by the eye observation camera 8 is obtained, thereby obtaining an eye observation camera 8 video set VV1.

[0104] Here, following the example in the previous section, assuming that multiple frames appear in V1(3) that are deleted according to the 3S quality standard in this section, and that the frames do not meet the requirement of 5 consecutive seconds, then V1(3) will be deleted, leaving only two segments, V1(1) and V1(2). At this moment, V1(1) and V1(2) constitute V3. V3(1) = {0-50}, V3(2) = {70-170)}. At this moment, the time number of ti3 is that the first video segment is the video from 0 to 5 seconds, and the second video segment is the video from 7 to 17 seconds. Then, in the corresponding same time period of the eye video captured by the eye observation camera 8, the video from 0 to 5 seconds and the video from 7 to 17 seconds are found. This is the data set VV1 of the eye in the external stable state within these 5 minutes.

[0105] 2.1.3 Eye Movement Extraction

[0106] When removing information from external instabilities, the present invention analyzes eye stability, first extracting the eye position from the data. This paper improves upon Xia Yang's proposal in "Research on Human Gaze Point Estimation Methods Based on Head-Mounted Eye Tracking Systems" from Huazhong University of Science and Technology, analyzing the pupil center. The original paper used an improved Canny algorithm to extract image edges, applied morphological methods to obtain independent contours, and filtered the contours based on pupil features in the image. Finally, an improved RANSAC algorithm was used to fit the pupil ellipse, combining it with confidence assessment criteria to obtain the final pupil ellipse and pupil center.

[0107] Since the solution of fitting pupil ellipse by RANSAC algorithm has high calculation accuracy, it consumes a lot of computing resources. Therefore, the present invention uses the elliptical version of the simple classic Hough transform to replace the RANSAC algorithm.

[0108] In this way, the VV1 data extracted from the eyeball change video captured by the eyeball observation camera 8 in the previous step is used as input to obtain the position of the eyeball in each frame captured by the eyeball observation camera 8. Then put these points at different times on a picture Iep. The number of points on Iep is the same as the total number of frames in V3. The position of each point is The Iep graph is then clustered using the classic k-means algorithm to identify two regions. One region close to the center of the Iep graph represents the stable pupil point SP1 = {SP1x, SP1y} when the rider's eyes observe the external environment during normal driving. The other region represents the stable pupil point SP2 = {SP2x, SP2y} when the rider's eyes observe VR / AR information (such as path planning information) during normal driving. The 1-norm distance DSP between the centers of SP1 and SP2 is then calculated.

[0109] DSP=||SP2x-SP1x||+||SP2y-SP1y||

[0110] The goal of the present invention is to minimize the distance DSP by following the algorithm in the subsequent steps, so that the rider can reduce the eye movement DSP when using the VR\AR helmet of the present invention.

[0111] 2.2 Display follower module

[0112] 2.2.1 Basic Debounce Following Control Algorithm AC1

[0113] Among existing control schemes for tracking systems, the display tracking module has the largest application in high-end movie cameras and spoons for Parkinson's patients. It can minimize random jitter while retaining the primary motion required by the rider. Its control principle is essentially the control problem of an inverse multi-degree-of-freedom robot, a specific implementation control scheme. This solution is a transplant of the solution from Xiang Bingxin's paper "Design of a Feeding Utensil Control System for Hand Tremor Patients" at Huazhong University of Science and Technology, replacing the anti-shake spoon's task with the anti-shake task of the two-degree-of-freedom reflection module 7.

[0114] Similar to Xiang Bingxin's 3-DOF control solution, this anti-shake task uses a micro brushed DC motor with a gearbox as the actuator to achieve low speed and high torque output. This invention selects a dead zone model as the backlash nonlinearity model and employs the inertia method to dynamically model the anti-shake and establish state-space equations. To address the need to maintain relative stillness between the 2-DOF reflection module 7 and the goggles 2 under conditions of continuous vibration interference and load transitions, a system framework employs an outer loop to control acceleration and an inner loop to control position. The inner loop uses backstepping to derive the control law to compensate for backlash nonlinearity and achieve rapid position tracking response. The outer loop implements an adaptive fuzzy PI algorithm for anti-shake control. This solution's input signals are derived from a 9-axis accelerometer module (for speed, motion direction, and angle detection). Algorithmically, since this invention only has two degrees of freedom (pitch angle βin and roll angle γin), computational overhead and execution time are reduced. A simplified version of Xiang Bingxin's solution (from 3 to 2 degrees of freedom) can be used to implement the basic anti-shake following control algorithm AC1.

[0115] During execution, the input signals of this solution come from a 9-axis accelerometer module (for speed, direction of motion, and angle detection), as well as an angle encoder installed on the motor's load shaft, specifically for detecting the rotation angle of the DC motor. The objects that can be controlled are the pitch angle βin and roll angle γin of the reflection module 7 toward the helmet. These are controlled by corresponding micro DC brushed motors with gearboxes as actuators: Motor 1 corresponds to the pitch angle, and Motor 2 corresponds to the roll angle. The calculation unit is the main control board, and the control cycle is to form a closed-loop control system.

[0116] This control scheme is only a control scheme that executes a clear control target and can debounce when maintaining normal operation. It can only serve as an execution-level control algorithm.

[0117] 2.2.2 Optimal Control Algorithm AC2 Based on Reinforcement Learning

[0118] The present invention not only faces a short-term, specific, non-interfering, reverse multi-degree-of-freedom execution problem, but also a longer-term, interfering, motion control problem. This is because there is a problem with the rider's eye usage habits. For example, some riders are used to observing with peripheral vision, and some riders have different eye gaze points and eyeball directions (because some have esotropia, lateral strabismus, exotropia, etc.), plus there is still a certain amount of residual jitter. Therefore, the system is faced with a control scheme without a precise target. Faced with this control problem without a precise target, classic solutions such as PID and fuzzy control are no longer applicable to this problem. Therefore, the present invention adopts a data-driven, reinforcement learning solution.

[0119] The goal of this control scheme is to minimize DSP, which is the distance between the first stable point DSP1 of the human eye extracted in 2.1.3 (the stable point of the pupil when the rider's eyes observe the outside during normal driving) and the second stable point DSP2 (the stable point of the pupil when the rider's eyes observe VR\AR information (such as road information) during normal driving). This data is obtained according to the algorithm process in 2.1. When the new data set of each new VV1 data set reaches 5 minutes, a new round of DSPnew is calculated for this new 5-minute data set. That is to say, the data sampling period changes according to external conditions. The present invention aims to minimize the value of DSPnew detected in each control cycle.

[0120] The objects that can be controlled by this solution are the pitch angle βin and roll angle γin of the reflection module 7 toward the helmet. They are controlled by corresponding micro DC brushed motors with gearboxes as actuators, with Motor 1 corresponding to the pitch angle and Motor 2 corresponding to the roll angle.

[0121] The computing unit of this solution is the main control board.

[0122] The Q-Learning algorithm is a classic reinforcement learning algorithm and a model-free reinforcement learning algorithm based on value functions. For details, please refer to the paper "Deep Reinforcement Learning with Double Q-leaming". Its main idea is to solve the optimal strategy by iteratively updating the action-value function. The specific process of the Q-learning algorithm is as follows: Figure 4 As shown, where E represents the number of iterations. In this invention, E is set to 50 times; r represents the reward, which is

[0123]

[0124] Here, DSPnew(j+1) is the DSPnew distance at the j+1th iteration, and DSPnew(j) is the DSPnew distance at the jth iteration. The present invention aims to make DSPnew smaller and smaller, that is, 1 / DSPnew(j+1) is larger than 1 / DSPnew(j). Therefore, r(j+1) should be as large as possible.

[0125] In addition, in order to reduce unnecessary jitter near the optimal value, the present invention further adds a judgment condition, that is, when DSPnew(j) and DSPnew(j+1) do not change much, r(j+1) is set to 0.

[0126] Right now:

[0127]

[0128] In the Q-Learning algorithm, a Q-table must be established, with rows representing each state (status) and columns representing each action (action). The value represents the reward for a particular action in a particular state. A value function Q(status, action), abbreviated as Q(s, a), is maintained for each state-action pair. This function represents the long-term reward for taking action (action) in state (status). Here, state (status) is the pitch angle βin and roll angle γin of the reflection module 7 toward the helmet, and action is the rotation angle of the pitch angle βin and roll angle γin. At each step, the present invention selects an action a based on the current state (status) and the Q-function, i.e., an ε-greedy strategy based on the current Q-table. The AC1 model is used to execute a, and the feedback from the environment is observed. This feedback typically includes the new state s′ and the reward r.

[0129] Then, the classic Bellman optimality equation is used to update the Q value:

[0130] Q(s,a)←Q(s,a)+α[r+γmax Qa′(s′,a′)-Q(s,a)]

[0131] Where α is the learning rate, γ is the discount factor, and max Qa′(s′, a′) represents the maximum Q value of all possible actions a′ in state s′.

[0132] In the present invention, α is equal to 1, γ is equal to 0.5, and action a is to increase the pitch angle βin or roll angle γin by 0.5 degrees. Under such a setting, the action space is actually within the range of plus or minus 25 degrees of the original basic setting of the pitch angle βin and plus or minus 25 degrees of the original basic setting of the roll angle γin, with a maximum change of 0.5 degrees. The basic de-jitter following control algorithm AC1 is used to achieve control, and continuous fine-tuning is performed to find the minimum pitch angle βin and roll angle γin in the state of DSPnew, so that the distance between the first stable point of the human eye (the stable point of the pupil when the rider's eyes observe the outside during normal driving) and the second stable point (the stable point of the pupil when the rider's eyes observe image information (such as VR\AR road information) during normal driving) is minimized.

[0133] In order to reduce the testing process, when r(j+1) is 0 for four consecutive times, that is, when the four combination states of βin and γin are all 0, the present invention considers that the optimal point has been found and ends the control at this moment.

[0134] By implementing the present invention, a following system for dynamically adjusting the helmet display can be realized, which is oriented to the rider's helmet display and takes the human eye observation effect as the main goal.

[0135] Its benefit is that by judging the direction of human eye attention, the projection range can be changed, reducing the movement distance of the human eye due to observation, adapting to human eye observation, and reducing the distraction caused by the human eye in observing the projection information.

Claims

1. A helmet display servo system based on reinforcement learning, characterized in that: It includes a helmet body, a reflection module, an external environment observation camera, a projection module, a main control board and an eye observation camera; The reflection module, external environment observation camera, projection module and main control board are all arranged on the helmet body; The external environment observation camera is arranged on the outer surface of the helmet and is connected to the main control board. It is used to collect video data of the external environment, determine whether the road in the external environment is a structured road, and transmit the judgment result and the audio and video data of the external environment to the main control board; The reflection module is controlled and connected to the main control board, and the main control board controls the movement of the reflection module in two degrees of freedom. The two degrees of freedom refer to the pitch angle of the reflection module. and roll angle ; The projection module is used to project image information onto the reflection module. When located on a structured road, the main control board adjusts the reflection module through the two degrees of freedom so that the image information is observed by the human eye after passing through the reflection module. The human eye observes the image information while observing the external environment; The eyeball observation camera is connected to the main control board, and is used to record video data of the human eye and transmit the video data of the human eye to the main control board, and the main control board extracts the position of the pupil in each frame of the video; Adjust the pitch angle of the reflector module through the main control board and roll angle , so that the 1-norm distance DSP between the center of the pupil of the stable point SP1 when the rider's eyes observe the external environment and the center of the pupil of the stable point SP2 when the rider's eyes observe image information is minimized; The calculation process of the distance DSP is as follows: The classic k-means algorithm is used to cluster the pupil locations in each frame of the image, and two regions are clustered. One of the regions close to the center point of the Iep graph represents the stable point SP1={SP1x, SP1y} of the pupil when the rider's eyes observe the external environment; Another area represents the stable point SP2={SP2x, SP2y} of the pupil when the rider observes image information during normal driving. Calculate the 1-norm distance DSP between the centers of SP1 and SP2 。 2. The helmet display servo system based on reinforcement learning according to claim 1, characterized in that: The method to minimize the distance DSP is: based on the Q-Learning algorithm, by adjusting the position of the image information, that is, adjusting the pitch angle of the reflection module and roll angle ,turn up The minimum value of .

3. The helmet display servo system based on reinforcement learning according to claim 2, characterized in that: In the Q-Learning algorithm, set the reward r and loop the pitch angle and roll angle Adjustment is made. When the reward r value is 0 for n consecutive times, it is considered that the minimum distance is found. ; r is represented by: ； in, is the DSPnew distance of the j+1th iteration, is the DSPnew distance of the jth iteration.

4. The helmet display servo system based on reinforcement learning according to claim 1, characterized in that: The method for determining whether a road in an external environment is a structured road is as follows: using an adaptive region of interest calculation method to separate the road area from the non-road area, and then performing structured road determination.

5. The helmet display servo system based on reinforcement learning according to claim 4 is characterized in that: When the rider is wearing a helmet and lowering his head, the region of interest is the lower half of the external environment video image; When the rider is wearing a helmet and looking up, the region of interest is set as the area from bottom to top in the external environment video image. pixel-height area, where is the monitoring angle of the external observation camera, It is the angle at which the rider looks up.

6. A helmet display tracking method based on reinforcement learning, the method being based on the tracking system according to claim 1, characterized in that: The method comprises the following steps: Step 1: Use the external environment observation camera set on the helmet to collect a video dataset V1 of the external environment being a structured road. The video dataset includes multiple video segments, and V1(i) represents the i-th video segment in the video dataset V1. Denoising is performed on the structured road video dataset V1 to obtain the denoised structured road video dataset V3 Step 2: extracting the eyeball change video set VV1 corresponding to the video data set V3, wherein the eyeball change video set VV1 is an eyeball change video when the external environment is stable; Step 3, extracting the pupil position in each frame of the eyeball change video set VV1 to form an image Iep; Step 4: Use the classic k-means algorithm to cluster the pupil positions in each frame of the image, and cluster out two regions. One of the regions close to the center point of the Iep graph represents the stable point SP1={SP1x, SP1y} of the pupil when the rider's eyes observe the external environment during normal driving; Another area represents the stable point SP2={SP2x, SP2y} of the pupil when the rider observes image information (e.g., VR / AR path planning information) during normal driving, and calculates the 1-norm distance DSP between the centers of SP1 and SP2. ； Step 5: Based on the Q-Learning algorithm, the position of the image information is adjusted, that is, the pitch angle of the reflection module is adjusted. and roll angle turn up The minimum value of .

7. The method for following a helmet display based on reinforcement learning according to claim 6, characterized in that: In step 1, denoising is performed on the structured road video dataset V1 to remove video segments with large speed changes, thereby obtaining a denoised structured road video dataset V3. The denoising process specifically includes the following steps: Step 1.1: Use the acceleration sensor on the helmet to collect the velocity Vxyz corresponding to each frame in the structured road video dataset V1. ； in, represents the speed of the helmet movement corresponding to the t-th frame, and the t-th frame is located in the i-th video; Indicates the speed of the helmet moving in the x direction corresponding to the tth frame; Indicates the speed of the helmet moving in the y direction corresponding to the tth frame; represents the speed of the helmet moving in the z direction corresponding to the tth frame; The x-direction refers to the direction of the line from the left eye to the right eye when the person wears the helmet. The y-direction refers to the direction starting from the middle point of the eyes, perpendicular to the x-direction and pointing horizontally forward. The z-direction refers to the direction starting from the middle point of the eyes, perpendicular to the x-direction and pointing vertically upward. Step 1.2: Calculate the average speed of each video and variance ,in represents the average speed of the i-th video, Represents the speed variance value of the i-th video; Step 1.3: Determine the helmet speed corresponding to the tth frame in the i-th video segment Whether it complies with the 3S quality standard, the expression is as follows: ； Traverse all frames in the i-th video. If there is a frame in the i-th video that does not conform to the above expression, delete the frame; Step 1.4, traverse all video segments in the video dataset V1, repeat step 1.3, and finally obtain the video dataset V2; Step 1.5: retain the video segments that are continuous for more than n seconds in the video dataset V2 as the denoised video dataset V3.

Images