A remote control car control system and method based on eye tracking technology

Abstract

The present invention discloses a remote control system and method for a remote-controlled car based on eye tracking technology, the system comprising an eye tracking system, a gaze intention recognition system and a car long and short step motion control system; the eye tracking system comprises a graphical user interface module and an eye movement data acquisition module; the gaze intention recognition system comprises a gaze area integration module and an audio interaction module; the gaze area integration module updates the gaze area in real time; the audio interaction module prompts through a beep audio when the coordinates of the integrated gaze area center point are closest to one of the arrow center points; the car long and short step control module controls the car to move in the direction indicated by the target arrow according to the target arrow and displacement step corresponding to the gaze intention. The present invention obtains the gaze point as an eye movement signal based on the eye tracking technology, and controls the direction of the car by gaze; the car is quickly and efficiently controlled from the car's perspective, effectively reducing the user's operation intensity and time, and avoiding the problem of perspective conversion.

Description

Technical Field

[0001] The present invention belongs to the technical field of intersection of cognitive neuroscience and image processing, and specifically relates to a remote control car control system and method based on eye tracking technology. Background Art

[0002] Motion control is a basic function for human-computer interaction to realize the operation of complex systems, and it has important application value in designing motion scenes for human-computer interaction. The remote control car system can replace people to go deep into dangerous or extreme environments, avoid casualties, and increase the opportunity for remote interaction with dangerous environments. The common control method of remote control cars is handle control, which requires the use of both hands as the operation medium and has a certain learning cost. When operating complex machines remotely, both hands need to be used at the same time to collaborate on multiple devices. Frequent switching between devices can easily lead to reduced efficiency, increased workload and fatigue. Moreover, when using a panoramic view to control the remote control car, it is easily affected by the perspective conversion, and the perspective of the car's direction is not intuitive, which affects the operation performance. In order to achieve fast and efficient motion control of the remote control car, reduce the difficulty of operation and thus improve efficiency, consider using the interaction method of eye movement signals.

[0003] Eye movement signals are a type of signal that directly reflects the brain's intention and can be obtained in a natural state. The interactive method using eye movement signals is simple, fast, and easy for users to master. It does not require human limb movement, thereby reducing the physical burden on people. When the user faces the graphical user interface, the image of the eyeballs of both eyes can be obtained based on the infrared camera and pupil-corneal reflection vector method in the eye tracker. The position of the pupil center can be obtained through image processing, and the position where the line of sight falls in the graphical user interface can be obtained through coordinate transformation. The present application aims to propose an efficient remote control car control system using eye movement signals. Summary of the invention

[0004] Purpose of the invention: In view of the deficiencies of the above-mentioned background technology, the present invention provides a remote control car control system and method based on eye tracking technology, which can capture the user's gaze position in real time, identify the user's gaze intention in combination with audio interaction, and replace the operation of both hands with the user's real-time eye movement to achieve motion control of the remote control car, thereby solving the problem that the traditional remote control car interaction method is inconvenient to operate and requires changing the viewing angle.

[0005] Technical solution: The present invention provides a remote control car control system based on eye tracking technology, including: an eye tracking system, a gaze intention recognition system and a car long and short step motion control system; the eye tracking system includes a graphical user interface module and an eye movement data acquisition module; the graphical user interface module presents the picture of the car's perspective and eight arrows for interaction; the eye movement data acquisition module collects the user's eye movement signals in real time by an eye tracker, and extracts the gaze point coordinates through clustering algorithm processing; the gaze intention recognition system includes a gaze area integration module and an audio interaction module; the gaze area integration module integrates the gaze area through a heat map generation submodule, a connection area division submodule and a gaze area extraction submodule, and updates the gaze area in real time; the audio interaction module prompts through a beep audio when the coordinates of the integrated gaze area center point are closest to one of the arrow center points; the gaze area integration module obtains The user's gaze target arrow and gaze duration are taken. When the user continues to gaze at the same target arrow, the audio interaction module gives two consecutive beep audio prompts to indicate that a long-term gaze intention to the arrow has been detected, and visual feedback is given by lighting up the target arrow on the user interface in red. When the center point of the gaze area is close to the center point of one of the arrows and a beep audio prompt is obtained, the user promptly shifts his or her gaze to the center of the interface. At this time, the center point of the gaze area integrated by the gaze area integration module is closest to the center point of the interface, which is recognized as the user's short-term gaze intention, and visual feedback is given by lighting up the target arrow on the user interface in yellow. The car's long and short step control module controls the car to move in the direction indicated by the target arrow according to the target arrow and displacement step corresponding to the gaze intention. A long-term gaze intention corresponds to a large-step displacement, and a short-term gaze intention corresponds to a small-step displacement.

[0006] Furthermore, the gaze point coordinates obtained by the eye tracking system are transmitted to the heat map generation submodule; the gaze point heat map is generated in the heat map generation submodule and transmitted to the connected area division submodule; the connected area division submodule obtains the connected area division that merges adjacent gaze point areas, and transmits it to the gaze area extraction submodule; the gaze area extraction module extracts the gaze area from different connected areas; the heat map generation submodule constructs a gaze point heat map using the gaze point coordinates obtained by the eye tracking system, superimposes five consecutive gaze point coordinate maps, and uses a Gaussian smoothing algorithm to smooth the gaze point coordinate distribution map to form a gaze point heat map; the connected area division submodule Figure 2 The binarization process highlights the outline of the target of interest, and adjacent fixations are merged into a large connected area, while isolated fixations form small connected areas on their own, thereby obtaining the division of the connected areas of the fixation point heat map. The fixation area extraction submodule searches for the minimum circumscribed rectangle of the connected area in the binary fixation point heat map, classifies different connected areas into different rectangular frames, removes the rectangular frames with an area smaller than a threshold, and uses the remaining rectangular frames as fixation areas.

[0007] Furthermore, the method for the eye tracking system to obtain the gaze point coordinates is: after presenting the graphical user interface, the eye tracker collects the sampling point coordinates of the user's gaze at the graphical user interface, the computer updates the accumulated collected data set at intervals of 100ms, and performs density-based clustering analysis on the eye movement data set to obtain the division of clusters, and selects the center point of the largest cluster as the gaze point coordinates.

[0008] Furthermore, the implementation process of the long and short step motion control system of the trolley to control the long and short step movement of the trolley is as follows: if the gaze intention recognition module recognizes the user's long-term gaze intention on the target arrow, the long and short step motion control system of the trolley controls the trolley to move in long steps in the direction indicated by the target arrow; if the gaze intention recognition module recognizes the user's short-term gaze intention on the target arrow, the long and short step motion control system of the trolley controls the trolley to move in short steps in the direction indicated by the target arrow; when the target arrow points to up, down, left or right on the screen, the trolley is controlled to move forward, backward, left or right accordingly; when the target arrow points to the upper left, upper right, lower left or lower right on the screen, the trolley is controlled to move to the front left, front right, back left or back right accordingly.

[0009] Based on the same inventive concept, the present invention also provides a remote control car control method based on eye tracking technology, comprising the following steps:

[0010] (1) Setting the parameters of the infrared camera, setting the pupil and corneal reflection detection thresholds in the infrared imaging image, obtaining the positions of the pupil and corneal center points, and obtaining the eye movement coordinates corresponding to the gaze point based on the RANSAC method;

[0011] (2) Complete gaze point calibration, display calibration points at the four corners and the center of the screen, and the user gazes at the calibration points in turn to obtain the corresponding gaze point coordinates and establish a mapping relationship between the gaze point area and the screen;

[0012] (3) Calibration points are displayed at the four corners and the center of the screen, and the coordinates of the gaze points of the user are collected. The deviation between the gaze points collected in this step and the calibration point coordinates obtained in the calibration process of step (2) is calculated. When the average deviation is less than 1 degree of visual angle, the calibration step is completed and the process proceeds to step (4). Otherwise, the calibration process of step (2) is repeated until the requirements are met.

[0013] (4) A graphical user interface is displayed on the screen, with the dynamic image from the car's perspective being displayed in the center of the interface;

[0014] (5) collecting eye movement signals of the user looking at the graphical user interface in real time, updating the coordinate data set of the sampling points of the human eye on the screen at a certain time interval, performing density-based clustering analysis on the collected data set to obtain cluster divisions, and extracting the center point of the largest cluster as the gaze point coordinate;

[0015] (6) using the real-time updated gaze point coordinates to integrate the gaze area through the heat map generation submodule, the connection area division submodule, and the gaze area extraction submodule;

[0016] (7) The center points of the eight arrows and the center point of the interface are regarded as nine reference points, and the distances from the center point of the gaze area to each reference point are calculated. When the center point of the integrated gaze area is closest to the center point of one of the arrows, a beep audio prompt is used to prompt the user;

[0017] (8) When the user continues to gaze at one of the arrows, the audio interaction module gives two consecutive beep audio prompts, indicating that the gaze intention recognition system has detected the long-term gaze intention on the arrow, and provides visual feedback by lighting up the target arrow on the user interface in red. At the same time, the long and short step motion control system of the car controls the car to move in the direction indicated by the target arrow with a large step displacement; when the target arrow points to the top, bottom, left, or right on the screen, the car is controlled to move forward, backward, left, or right accordingly; when the target arrow points to the upper left, upper right, lower left, or lower right on the screen, the car is controlled to move to the left front, right front, left back, or right back accordingly;

[0018] (9) After the center point of the gaze area integrated in step (7) is closest to the center point of one of the arrows and a beep audio prompt is received, the user promptly shifts his or her gaze to the center of the interface. The gaze area integration module then integrates the center point of the gaze area closest to the center point of the interface. The gaze intention recognition system recognizes this as the user's short-term gaze intention and provides visual feedback by lighting up the target arrow on the user interface in yellow. At the same time, the car's long and short step motion control system controls the car to move in the direction indicated by the target arrow with a small step displacement. The correspondence between the direction of the target arrow on the screen and the car's moving direction is the same as in step (7).

[0019] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention captures the user's eye movement signals facing the graphical user interface in real time and detects the gaze point coordinates that can reflect the user's line of sight, integrates the user's gaze areas of interest from the gaze point coordinates through the heat map generation submodule, the communication area division submodule, and the gaze area extraction submodule, identifies the user's gaze intention in combination with the audio prompt, controls the car to move in the direction indicated by the target arrow according to the target arrow and the displacement step corresponding to the gaze intention, and replaces the operation of both hands with the user's real-time eye movement to realize the motion control of the remote-controlled car, making the user's motion control of the car faster and more natural, and the operation easier; in addition, since the picture of the car's perspective is echoed in the center of the graphical user interface, the remote-controlled car control system of the present application avoids the problem of switching perspectives and can improve the interactive performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of the control system structure of a remote-controlled car based on eye tracking technology;

[0021] Figure 2 This is the signal flow chart of the remote control car control system based on eye tracking technology;

[0022] Figure 3 for the graphical user interface and its dimensional drawings;

[0023] Figure 4 Example graph of gaze points generated for an eye tracking system;

[0024] Figure 5 To integrate the example diagram of the fixation area;

[0025] Figure 6 This is the wireless communication node diagram of the car;

[0026] Figure 7 This is a diagram of the system test results. DETAILED DESCRIPTION

[0027] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0028] The present invention proposes a remote control car control system based on eye tracking technology. Figure 1As shown, it consists of an eye tracking system, a gaze intention recognition system, and a long and short step motion control system for the car. Among them, the eye tracking system includes a graphical user interface module and an eye movement data acquisition module. The graphical user interface module presents a graphical user interface, and the center of the interface echoes the dynamic picture of the car's perspective, and there are eight arrows around the picture for interaction; the eye movement data acquisition module uses an eye tracker to collect the user's eye movement signals in real time, and extracts the gaze point coordinates through clustering algorithm processing. The gaze intention recognition system includes a gaze point area integration module and an audio interaction module. The gaze point area integration module integrates the gaze area from the real-time updated gaze point coordinates through the heat map generation submodule, the connection area division submodule, and the gaze area extraction submodule; the audio interaction module prompts through a beep audio when the coordinates of the integrated gaze area center point are closest to the center point of one of the arrows; the gaze intention recognition system obtains the user's gaze target arrow and gaze duration. When the user continues to gaze at the same target arrow, the audio interaction module gives two consecutive beep audio prompts to indicate that the gaze intention recognition system has detected a long-term gaze intention on the arrow, and visual feedback is given by lighting up the target arrow on the user interface in red; when the center point of the gaze area integrated by the gaze area integration module is close to the center point of one of the arrows and receives a beep audio prompt, the user promptly shifts his or her gaze to the center of the interface, and the center point of the gaze area integrated by the gaze area integration module is closest to the center point of the video area. The gaze intention recognition system recognizes it as the user's short-term gaze intention, and visual feedback is given by lighting up the target arrow on the user interface in yellow. The long and short step motion control system of the car controls the car to move in the direction indicated by the target arrow according to the target arrow and displacement step corresponding to the gaze intention. Long-term gaze corresponds to large-step displacement, and short-term gaze corresponds to small-step displacement.

[0029] Figure 2is a system signal flow chart. In the eye tracking system, the graphical user interface module presents the graphical user interface, and the dynamic picture of the car's perspective is echoed in the center of the interface; the eye data acquisition module collects the coordinates of the gaze sampling points of the human eye on the screen when the user faces the graphical user interface and performs density-based clustering analysis on the collected sampling point coordinate data set at a time interval of 100 milliseconds to update the detected gaze point coordinates. In the gaze intention recognition system, the gaze area integration module integrates the gaze area through the heat map generation submodule, the connection area division submodule, and the gaze area extraction submodule with the gaze point coordinates obtained from the eye tracking system; then calculates the distance between the coordinates of the center point of the gaze area and the nine reference points composed of the center points of eight arrows and the center point of the screen. If the gaze point is closest to one of the arrow centers, the target arrow is selected and the user is prompted by a beep audio; the user's gaze duration intention for the target arrow is obtained according to the user's action after receiving the first beep audio prompt. If the user continues to gaze at the same arrow, the system gives a second beep audio prompt, and the gaze intention recognition system recognizes it as the user's long-term gaze intention; if the user immediately shifts his gaze to the center of the interface, the gaze intention recognition system recognizes it as the user's short-term gaze intention. The long and short step motion control system of the car controls the car to move in the direction indicated by the target arrow according to the target arrow and displacement step corresponding to the gaze intention. Long-term gaze corresponds to long-step displacement, and short-term gaze corresponds to short-step displacement.

[0030] Figure 3 The following is a graphical user interface and its dimensions. The center of the interface displays the dynamic image of the car's perspective, and there are eight arrows for interaction around the video area. The resolution of the screen presenting the graphical user interface is 1280×1024 pixels, and the refresh rate is 60Hz. The size of the video area is 940×752 pixels. The center point of the video rectangular area and the center points of the eight arrows constitute a total of nine reference points. The horizontal distance between adjacent reference points is 555 pixels, and the vertical distance between adjacent reference points is 448 pixels.

[0031] The fixation points generated by the eye tracking system are as follows Figure 4 As shown in the figure, the sampling point coordinate data set updated at intervals of 100 milliseconds is analyzed by a density-based clustering algorithm to obtain the division of clusters, and the center point of the largest cluster is selected as the gaze point coordinate. The sampling frequency of the eye tracker is set to 1000Hz. The implementation of the density-based clustering algorithm requires setting two parameters - the neighborhood radius and the minimum number of points. In this system, the neighborhood radius is set to 20 pixels and the minimum number of points is set to 80. If a data point has no less than the minimum number of points within the neighborhood of the specified radius, it is called a core point. The algorithm scans the entire data set, finds any core point, and expands the points that are densely connected to these data points to obtain the division of clusters. Figure 3In the example, the sampling points marked with circles are clusters that are classified as one type, and the sampling points marked with crosses are not classified as clusters and are considered noise. The cluster marked with circles contains 91 sampling points. The center point of the cluster is extracted as the focus point and marked with a five-pointed star in the figure.

[0032] Integrate the fixation area such as Figure 5 As shown, the integration of fixation areas is achieved by updating the fixation point coordinates in real time through the fixation point heat map analysis method. The fixation area integration module consists of heat map generation, connected area division and fixation area extraction submodules. The fixation point coordinates obtained by the eye tracking subsystem are transmitted to the heat map generation submodule; the fixation point heat map is generated in the heat map generation submodule and transmitted to the connected area division submodule; the connected area division that merges adjacent fixation point areas is obtained in the connected area division submodule, and transmitted to the fixation area extraction submodule; the fixation area is extracted from different connected areas in the fixation area extraction module. The heat map generation submodule constructs a fixation point heat map using the fixation point coordinates obtained by the eye tracking subsystem, superimposes five consecutive fixation point coordinate maps, and uses the Gaussian smoothing algorithm to smooth the fixation point coordinate distribution map to form a fixation point heat map, as shown in Figure 5 As shown in (a). The submodule of connecting the area divides the gaze point into Figure 2 The binary heat map is generated by highlighting the outline of the target of interest. Figure 5 As shown in (b), adjacent fixations are merged into a large connected area, and isolated fixations form small connected areas on their own, thus obtaining the division of the connected area of ​​the fixation point heat map. The fixation area extraction submodule searches for the minimum bounding rectangle of the connected area in the binary fixation point heat map, such as Figure 5 In (c), rectangular boxes 1 and 2 are used to classify different connected areas into different rectangular boxes. The rectangular boxes with an area smaller than the threshold are removed, and the remaining rectangular boxes are used as the fixation areas, such as Figure 5 As shown in (d).

[0033] The process of controlling the long and short step movement of the car by the long and short step motion control subsystem is as follows: if the gaze intention recognition module recognizes the user's long-term gaze intention on the target arrow, the long and short step motion control subsystem controls the car to move in the direction indicated by the target arrow with long steps; if the gaze intention recognition module recognizes the user's short-term gaze intention on the target arrow, the long and short step motion control subsystem controls the car to move in the direction indicated by the target arrow with short steps. When the target arrow points to the top, bottom, left, or right on the screen, the car is controlled to move forward, backward, left, or right accordingly; when the target arrow points to the upper left, upper right, lower left, or lower right on the screen, the car is controlled to move to the left front, right front, left back, or right back accordingly.

[0034] Figure 6 Figure 1 is a diagram of wireless communication nodes of the car. Figure 6The ROS node communication shown in the figure. The car publishes the camera image topic at 30 frames per second. At the same time, the computer subscribes to the compressed image to complete the echo of the dynamic picture of the car's perspective on the user interface. The movement of the target arrow and displacement step corresponding to the gaze intention is considered to be the car's service to the computer. When the system detects the user's gaze intention, the computer immediately requests the car for service. When the car receives the request, it immediately responds and provides service.

[0035] Based on the same inventive concept, the present invention also proposes a remote control car control method based on eye tracking technology, comprising the following steps:

[0036] (1) Set the parameters of the infrared camera, set the pupil and corneal reflection detection thresholds in the infrared imaging image, obtain the positions of the pupil and corneal center points, and obtain the eye movement coordinates corresponding to the gaze point based on the RANSAC method.

[0037] (2) After gaze point calibration is completed, calibration points are displayed at the four corners and the center of the screen. The user gazes at the calibration points in turn, obtains the corresponding gaze point coordinates, and establishes a mapping relationship between the gaze point area and the screen.

[0038] (3) Calibration points are displayed at the four corners and the center of the screen, and the coordinates of the gaze points of the user are collected. The deviation between the gaze points collected in this step and the calibration point coordinates obtained during the calibration process in step (2) is calculated. When the average deviation is less than 1 degree of the visual angle, the calibration step is completed and the process proceeds to step (4). Otherwise, the calibration process in step (2) is repeated until the requirements are met.

[0039] (4) A graphical user interface is displayed on the screen, with the dynamic image from the car's perspective being displayed in the center of the interface.

[0040] (5) The eye movement signals of the user looking at the graphical user interface are collected in real time, and the coordinate data set of the sampling points of the human eye on the screen is updated at intervals of 100 ms. The collected data set is subjected to density-based clustering analysis to obtain the cluster division, and the center point of the largest cluster is extracted as the gaze point coordinate.

[0041] (6) The gaze point coordinates updated in real time are used to integrate the gaze area through the heat map generation submodule, the connected area division submodule, and the gaze area extraction submodule.

[0042] (7) The center points of the eight arrows and the center point of the interface are regarded as nine reference points. The distance from the center point of the gaze area to each reference point is calculated. When the center point of the integrated gaze area is closest to the center point of one of the arrows, the user is prompted by a beep audio.

[0043] (8) When the user continues to gaze at one of the arrows, the audio interaction module gives two consecutive beep audio prompts, indicating that the gaze intention recognition system has detected a long-term gaze intention on the arrow, and provides visual feedback by lighting up the target arrow on the user interface in red. At the same time, the long and short step motion control system of the car controls the car to move in the direction indicated by the target arrow with a large step displacement; when the target arrow points to the top, bottom, left, or right on the screen, the car is controlled to move forward, backward, left, or right accordingly; when the target arrow points to the upper left, upper right, lower left, or lower right on the screen, the car is controlled to move to the left front, right front, left back, or right back accordingly.

[0044] (9) After the center point of the gaze area integrated in step (7) is closest to the center point of one of the arrows and a beep audio prompt is received, the user promptly shifts his or her gaze to the center of the interface. The gaze area integration module then integrates the center point of the gaze area closest to the center point of the interface. The gaze intention recognition system recognizes this as the user's short-term gaze intention and provides visual feedback by lighting up the target arrow on the user interface in yellow. At the same time, the car's long and short step motion control system controls the car to move in the direction indicated by the target arrow with a small step displacement of 25 cm. The correspondence between the direction of the target arrow on the screen and the direction of travel of the car is the same as in step (7).

[0045] Figure 7 The following is the result of the system test experiment. The test experiment requires the subjects to look at the target arrow in a clockwise order of upward, right, downward, and left to make the car drive a route composed of squares to return to the origin, and then look at the target arrow in a clockwise order of upper left, upper right, lower right, and lower left to make the car drive a route composed of diamonds to return to the origin. Completing this 8-time interactive task is considered a set of experiments. During the experiment, the subjects completed 16 sets of experiments by alternating between long-term and short-term gazes. A total of 7 subjects participated in the system test experiment. Figure 7 (a) shows that the average accuracy of target detection when the subjects used short-term fixation and long-term fixation to control the movement of the car was 95.54% and 99.55% respectively, and the t-test showed that the results under the two control conditions were statistically different (p<0.5); Figure 7 (b) shows the classification accuracy of the subjects using short-term and long-term gazes on different arrows to control the movement of the car. It can be found that when using long-term gaze to control the movement of the car, a higher classification accuracy can be obtained at each arrow, while the classification accuracy of short-term gazes for different target arrows is mostly lower than the accuracy of identifying long-term gazes. This may be because the method of identifying short-term gazes involves a rapid shift of vision, which is related to the subject's concentration and familiarity.

Claims

1. A remote control car control system based on eye tracking technology, characterized in that: The control system includes an eye tracking system, a gaze intention recognition system and a long and short step motion control system for the trolley; the eye tracking system includes a graphical user interface module and an eye movement data acquisition module; the graphical user interface module presents the picture of the trolley's perspective and eight arrows for interaction; the eye movement data acquisition module collects the user's eye movement signals in real time by an eye tracker, and extracts the gaze point coordinates through clustering algorithm processing; the gaze intention recognition system includes a gaze area integration module and an audio interaction module; The fixation area integration module integrates the fixation area through the heat map generation submodule, the connection area division submodule and the fixation area extraction submodule, and updates the fixation area in real time; the audio interaction module prompts through beep audio when the coordinate of the center point of the integrated fixation area is closest to the center point of one of the arrows; The gaze area integration module obtains the user's gaze target arrow and gaze duration. When the user continues to gaze at the same target arrow, the audio interaction module gives two consecutive beep audio prompts to indicate that a long-term gaze intention on the arrow has been detected, and visual feedback is given by lighting up the target arrow on the user interface in red. When the center point of the gaze area is close to the center point of one of the arrows and a beep audio prompt is given, the user promptly shifts his or her gaze to the center of the interface. At this time, the center point of the gaze area integrated by the gaze area integration module is closest to the center point of the interface, and the user's short-term gaze intention is recognized, and visual feedback is given by lighting up the target arrow on the user interface in yellow. The car's long and short step length control module controls the car to move in the direction indicated by the target arrow according to the target arrow and the displacement step length corresponding to the gaze intention. A long-term gaze intention corresponds to a large-step displacement, and a short-term gaze intention corresponds to a small-step displacement. The gaze point coordinates acquired by the eye tracking system are transmitted to the heat map generation submodule; The heat map generation submodule generates a gaze point heat map and transmits it to the connected area division submodule; the connected area division submodule obtains the connected area division that merges adjacent gaze point areas and transmits it to the gaze area extraction submodule; the gaze area extraction module extracts gaze areas from different connected areas; the heat map generation submodule constructs a gaze point heat map using the gaze point coordinates obtained by the eye tracking system, superimposes five consecutive gaze point coordinate maps, and uses a Gaussian smoothing algorithm to smooth the gaze point coordinate distribution map to form a gaze point heat map; the connected area division submodule binarizes the gaze point heat map to highlight the outline of the target of interest, merges adjacent gaze points into a large connected area, and isolated gaze points form small connected areas on their own, thereby obtaining the division of the connected area of ​​the gaze point heat map; the gaze area extraction submodule classifies and divides different connected areas into different rectangular frames by searching for the minimum circumscribed rectangle of the connected area in the binary gaze point heat map, removes the rectangular frames with an area smaller than a threshold, and the remaining rectangular frames are used as gaze areas; The method for the eye tracking system to obtain the gaze point coordinates is as follows: after presenting a graphical user interface, the eye tracker collects the sampling point coordinates of the user's gaze on the graphical user interface, the computer updates the accumulated collected data set at intervals of 100ms, performs density-based clustering analysis on the eye movement data set, obtains the division of clusters, and selects the center point of the largest cluster as the gaze point coordinates.

2. The remote control system of a car based on eye tracking technology according to claim 1, characterized in that: The implementation process of the long and short step motion control system of the trolley to control the long and short step movement of the trolley is as follows: if the gaze intention recognition module recognizes the user's long-term gaze intention on the target arrow, the long and short step motion control system of the trolley controls the trolley to move in long steps in the direction indicated by the target arrow; if the gaze intention recognition module recognizes the user's short-term gaze intention on the target arrow, the long and short step motion control system of the trolley controls the trolley to move in short steps in the direction indicated by the target arrow; when the target arrow points to up, down, left, or right on the screen, the trolley is controlled to move forward, backward, left, or right accordingly; when the target arrow points to the upper left, upper right, lower left, or lower right on the screen, the trolley is controlled to move to the front left, front right, back left, or back right accordingly.

3. A remote control car control method based on eye tracking technology using the system as claimed in any one of claims 1-2, characterized in that: The following steps are involved: (1) Setting the parameters of the infrared camera, setting the pupil and corneal reflection detection thresholds in the infrared imaging image, obtaining the positions of the pupil and corneal center points, and obtaining the eye movement coordinates corresponding to the gaze point based on the RANSAC method; (2) Complete gaze point calibration, display calibration points at the four corners and the center of the screen, and the user gazes at the calibration points in turn to obtain the corresponding gaze point coordinates and establish a mapping relationship between the gaze point area and the screen; (3) Calibration points are displayed at the four corners and the center of the screen, and the coordinates of the gaze points of the user are collected. The deviation between the gaze points collected in this step and the calibration point coordinates obtained in the calibration process of step (2) is calculated. When the average deviation is less than 1 degree of visual angle, the calibration step is completed and the process proceeds to step (4). Otherwise, the calibration process of step (2) is repeated until the requirements are met. (4) A graphical user interface is displayed on the screen, with the dynamic image from the car's perspective being displayed in the center of the interface; (5) collecting eye movement signals of the user looking at the graphical user interface in real time, updating the coordinate data set of the sampling points of the human eye on the screen at a certain time interval, performing density-based clustering analysis on the collected data set to obtain cluster divisions, and extracting the center point of the largest cluster as the gaze point coordinate; (6) using the real-time updated gaze point coordinates to integrate the gaze area through the heat map generation submodule, the connection area division submodule, and the gaze area extraction submodule; (7) The center points of the eight arrows and the center point of the interface are regarded as nine reference points, and the distances from the center point of the gaze area to each reference point are calculated. When the center point of the integrated gaze area is closest to the center point of one of the arrows, a beep audio prompt is used to prompt the user; (8) When the user continues to gaze at one of the arrows, the audio interaction module gives two consecutive beep audio prompts, indicating that the gaze intention recognition system has detected the long-term gaze intention on the arrow, and provides visual feedback by lighting up the target arrow on the user interface in red. At the same time, the long and short step motion control system of the car controls the car to move in the direction indicated by the target arrow with a large step displacement; when the target arrow points to the top, bottom, left, or right on the screen, the car is controlled to move forward, backward, left, or right accordingly; when the target arrow points to the upper left, upper right, lower left, or lower right on the screen, the car is controlled to move to the left front, right front, left back, or right back accordingly; (9) After the center point of the gaze area integrated in step (7) is closest to the center point of one of the arrows and a beep audio prompt is received, the user promptly shifts his or her gaze to the center of the interface. The gaze area integration module then integrates the center point of the gaze area closest to the center point of the interface. The gaze intention recognition system recognizes this as the user's short-term gaze intention and provides visual feedback by lighting up the target arrow on the user interface in yellow. At the same time, the car's long and short step motion control system controls the car to move in the direction indicated by the target arrow with a small step displacement. The correspondence between the direction of the target arrow on the screen and the car's moving direction is the same as in step (7).

Images