Vehicle control method and apparatus based on eye tracking, electronic device, and storage medium

Abstract

The present application discloses a vehicle control method, system and apparatus based on eye tracking, an electronic device, and a storage medium. The method comprises: collecting eye movement state information of a passenger on a traveling vehicle; on the basis of the eye movement state information, determining whether the passenger is in a carsickness state; and if the passenger is in the carsickness state, controlling a driving state of the vehicle, wherein the controlling a driving state of the vehicle comprises controlling the speed and steering of the vehicle, and controlling the vehicle environment. By means of the solution, a carsickness state of a passenger is determined by observing an eye movement state of the passenger, and the speed and steering of a vehicle, and the vehicle environment are adjusted in time, so that accumulation of carsickness and discomfort of the passenger is mitigated, and the passenger can be kept within the tolerance range so as to complete the riding process of the vehicle, and thus the tolerance capacity of the passenger is enhanced, further reducing the future carsickness.

Description

Eye tracking vehicle control method, device, electronic device and storage medium Technical Field

[0001] The present application relates to the field of eye tracking, and in particular to an eye tracking vehicle control method, an eye tracking vehicle control system, an eye tracking vehicle control device, an electronic device, and a storage medium. Background Art

[0002] Motion sickness, commonly known as motion sickness, occurs when the accelerations of a vehicle, such as jolting, swaying, or spinning, stimulate the passenger's vestibular nerves, causing the body to perceive the accelerations. However, due to limited vision, the passenger cannot perceive these accelerations through observation. This mismatch between visual and physical perception can lead to motion sickness, causing symptoms such as vertigo and vomiting. Furthermore, excessive vehicle speeds subject the vehicle to constant acceleration, potentially disrupting the balance mechanism in the cochlea, causing discomfort and dizziness. Poor ventilation and air circulation in the vehicle can also contribute to dizziness.

[0003] When people experience motion sickness, the vestibular organs in the inner ear sense the vehicle's motion and rotation and send signals to the brain. The brain interprets these signals and triggers a series of bodily responses to help the body adapt to these changes. These include eye movements, which help the body maintain balance and stability. There are two main types of eye movements caused by motion sickness. The first is pupil constriction. Motion sickness can cause disturbances in the autonomic nervous system, including pupil constriction. This occurs when the body attempts to reduce the amount of light entering the eye by reducing pupil size, thereby alleviating motion sickness symptoms. The second type is the corneal reflex. Motion sickness can affect the corneal reflex, making it less sensitive or inconsistent.

[0004] Traditional methods of preventing and treating motion sickness include medication, acupuncture, meditation, etc., but the efficacy of these methods is not clear due to individual differences. In addition, the actual situation is that passengers who are in a state of motion sickness cannot immediately leave the vehicle environment and still need to use the vehicle to reach their destination.

[0005] Therefore, an eye-tracking vehicle control solution is needed. The motion sickness state can be tracked through eye movement status, and the motion sickness state can be improved by changing the vehicle driving, driving status, and riding environment. The solution can try to reduce the passengers' motion sickness without leaving the riding state, and help them endure to their destination.

[0006] Summary of the Invention

[0007] The object of the present invention is to provide an eye-tracking vehicle control method, an eye-tracking vehicle control system, an eye-tracking vehicle control device, an electronic device and a storage medium to solve at least one of the above-mentioned technical problems.

[0008] The present invention provides the following solutions:

[0009] According to one aspect of the present invention, there is provided an eye-tracking vehicle control method, the eye-tracking vehicle control method comprising:

[0010] Collect eye movement status information of passengers in a moving vehicle;

[0011] determining whether the passenger is in a motion sickness state according to the eye movement state information;

[0012] If the passenger is in a state of motion sickness, the vehicle's driving state is controlled;

[0013] The control of the vehicle driving state includes controlling the vehicle speed, steering and the interior environment of the vehicle.

[0014] Furthermore, the controlling of vehicle speed, steering and in-vehicle environment includes:

[0015] Collect passenger eye movement information to control vehicle speed, steering, and the in-car environment;

[0016] determining whether the passenger's motion sickness condition has been alleviated based on the eye movement state information;

[0017] If the passenger is relieved of motion sickness, the vehicle driving status is marked;

[0018] Vehicle control prompt information is generated according to the marked vehicle driving state.

[0019] According to a second aspect of the present invention, an eye-tracking vehicle control system is provided, the eye-tracking vehicle control system comprising: a control unit module, an eye-tracking module, a vehicle data module, and a vehicle control module;

[0020] The eye tracking module is used to collect passenger's eye movement status information;

[0021] The vehicle data module is used to collect vehicle driving status information;

[0022] The control unit module is configured to process the passenger's eye movement state information and the vehicle's driving state information to generate instructions for controlling the vehicle's driving;

[0023] The vehicle control module is used to control the driving state of the vehicle according to the instruction for controlling the driving of the vehicle;

[0024] The eye tracking module collects the passenger's eye movement status information and sends it;

[0025] The vehicle data module collects and sends the vehicle's driving status information;

[0026] The control unit module receives data from the eye tracking module and the vehicle data module, generates and sends instructions for controlling the vehicle's movement based on the passenger's eye movement status information and the vehicle's driving status information;

[0027] The vehicle control module receives data from the control unit module and controls the driving state of the vehicle according to the instructions for controlling the driving of the vehicle.

[0028] Furthermore, the control unit module includes: a data analysis module, a motion sickness determination module and a preventive measures module;

[0029] The data analysis module is used to process the passenger's eye movement state information and the vehicle's driving state information;

[0030] The motion sickness determination module is used to determine whether the passenger's eye movement state is an eye movement state of motion sickness;

[0031] The preventive measures module is used to generate a vehicle driving state that alleviates the passenger's motion sickness according to the passenger's motion sickness state;

[0032] The data analysis module receives the data from the eye tracking module, processes the eye movement state information of the passenger and the vehicle driving state information, generates feature data, and sends it;

[0033] The motion sickness determination module receives the data from the data analysis module and determines whether the passenger's eye movement state is an eye movement state of motion sickness according to the characteristic data;

[0034] If the eye movement state of the passenger is the eye movement state of motion sickness, sending a judgment result that the passenger is in a motion sickness state;

[0035] The preventive measures module receives the data from the motion sickness determination module and generates the vehicle driving state for alleviating the passenger's motion sickness according to the determination result that the passenger is in a motion sickness state;

[0036] According to the vehicle driving state for alleviating passenger motion sickness, a vehicle driving control instruction is generated.

[0037] Furthermore, the eye tracking module includes:

[0038] Recording passenger's eye movement data based on optical sensors;

[0039] According to the correlation between eye movement state and motion sickness, characteristic data of eye movement state is collected;

[0040] The characteristic data of the eye movement state includes characteristic data of pupil diameter, characteristic data of eye movement velocity, characteristic data of eye movement duration, and characteristic data of corneal reflection angle.

[0041] Furthermore, the vehicle data module includes:

[0042] Extract characteristic data of vehicle driving status based on its impact on motion sickness;

[0043] The characteristic data of the vehicle driving state includes vehicle speed, steering action information and environmental information of vehicle interior temperature and oxygen content.

[0044] Furthermore, the data analysis module includes:

[0045] The characteristic data of the passenger's eye movement state and the characteristic data of the vehicle's driving state are used as data samples to build and train a machine learning model;

[0046] Construct and train a machine learning model as described above to generate a prediction model;

[0047] Importing the prediction model into the motion sickness judgment module;

[0048] According to the result of the motion sickness determination module determining the characteristic data, driving the preventive measure module to generate a vehicle driving control instruction;

[0049] The vehicle data module feeds back characteristic data of the vehicle driving state for executing the vehicle driving control instruction;

[0050] The eye tracking module feeds back characteristic data of the eye movement state of the vehicle data module when executing the vehicle driving control instruction;

[0051] The data analysis module iterates model training and updates the prediction model based on the feedback feature data samples.

[0052] According to three aspects of the present invention, an eye-tracking vehicle control device is provided, the eye-tracking vehicle control device comprising:

[0053] An information collection module, used to collect eye movement status information of passengers in a moving vehicle;

[0054] A motion sickness determination module, configured to determine whether a passenger is in a motion sickness state based on the eye movement state information;

[0055] The travel control module is used to control the vehicle's driving state if the passenger is in a state of motion sickness, wherein controlling the vehicle's driving state includes controlling the vehicle's speed, steering, and the vehicle's interior environment.

[0056] According to four aspects of the present invention, there is provided an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0057] A computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the eye-tracking vehicle control method.

[0058] According to five aspects of the present invention, a computer-readable storage medium is provided, comprising: a computer program that can be executed by an electronic device is stored therein, and when the computer program is run on the electronic device, the electronic device executes the steps of the eye-tracking vehicle control method.

[0059] Through the above solution, the following beneficial technical effects are achieved:

[0060] This application determines the passenger's motion sickness status by observing the passenger's eye movements, and promptly adjusts the vehicle's speed, steering, and interior environment to alleviate the passenger's accumulated motion sickness discomfort, allowing the passenger to maintain a tolerance range and complete the vehicle ride, thereby exercising the passenger's tolerance ability and further reducing the feeling of motion sickness during future rides.

[0061] This application uses a machine learning method to use the characteristic data of eye movement status and the characteristic data of vehicle driving status as sample data for model training to form a prediction model, so as to detect and accurately assess the passenger's motion sickness early, and predict the improvement of the motion sickness condition by changing the vehicle's driving status, the in-vehicle environment, etc., iterate the model training, and establish a reliable prediction model between vehicle driving and vehicle riding.

[0062] This application uses model training to gradually adjust the association weights of the characteristic data of eye movement status, the characteristic data of vehicle driving status and motion sickness, further optimizes the prediction of motion sickness and improves the motion sickness state by controlling the vehicle speed, steering action information and the temperature and oxygen content in the car. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] FIG1 is a flow chart of an eye-tracking vehicle control method provided by one or more embodiments of the present invention.

[0064] FIG2 is a structural diagram of an eye-tracking vehicle control system provided by one or more embodiments of the present invention.

[0065] FIG3 is a structural diagram of an eye-tracking vehicle control device provided by one or more embodiments of the present invention.

[0066] FIG4 is a schematic diagram of an eye-tracking vehicle control system architecture according to a specific embodiment of the present invention.

[0067] FIG5 is a schematic diagram of collecting eye tracking data according to a specific embodiment of the present invention.

[0068] FIG6 is a schematic diagram of a machine learning method for training a motion sickness prediction model according to a specific embodiment of the present invention.

[0069] FIG7 is a schematic diagram of a motion sickness prediction model guiding vehicle control according to a specific embodiment of the present invention.

[0070] FIG8 is a block diagram of an electronic device structure of an eye-tracking vehicle control method provided by one or more embodiments of the present invention. DETAILED DESCRIPTION

[0071] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0072] FIG1 is a flow chart of an eye-tracking vehicle control method provided by one or more embodiments of the present invention.

[0073] The eye tracking vehicle control method shown in FIG1 includes:

[0074] Step S1, collecting eye movement status information of passengers in a moving vehicle;

[0075] Step S2, judging whether the passenger is in a motion sickness state according to the eye movement state information;

[0076] Step S3: If the passenger is in a motion sickness state, control the vehicle's driving state;

[0077] Step S4, controlling the vehicle driving state includes controlling the vehicle speed, steering and the interior environment of the vehicle.

[0078] Specifically, based on the principle of motion sickness, which is caused by a misalignment of information fed back to the brain by the vestibular and visual organs, eye movement characteristics differ from those observed in a non-motion sick state. By monitoring eye movements while the vehicle is moving, it is determined whether a passenger is experiencing motion sickness. If a passenger is experiencing motion sickness, the vehicle's driving state can be controlled to alleviate the condition. Besides motion sickness being caused by a misalignment of information fed back to the brain by the vestibular and visual organs, motion sickness caused by vehicle motion can be alleviated by reducing vehicle speed, turning intensity, and mitigating the difficulty the balance organs adapt to bumpy roads. There is also the issue of "smell sickness," which can be alleviated through ventilation and increased oxygen levels in the vehicle. Since the cause of motion sickness is not clearly detected when a passenger boards a vehicle, attempts can be made to control vehicle motion and ventilation based on the passenger's motion sickness status, while also observing eye movements to determine whether changes in vehicle motion or ventilation are having a mitigating effect. If so, the current vehicle driving state control strategy is effective for the passenger experiencing motion sickness.

[0079] In this embodiment, controlling vehicle speed, steering, and the in-vehicle environment includes:

[0080] Collect passenger eye movement information to control vehicle speed, steering, and the in-car environment;

[0081] Determine whether the passenger's motion sickness has been alleviated based on eye movement status information;

[0082] If the passenger is relieved of motion sickness, the vehicle driving status is marked;

[0083] Generate vehicle control prompt information based on the marked vehicle driving status.

[0084] Specifically, the vehicle's driving state that alleviates motion sickness is marked and associated with the passenger whose motion sickness has improved. In other words, the current vehicle's driving state control strategy is effective for passengers experiencing motion sickness. The vehicle's driving state control strategy can be associated with the passenger's identity information, or the vehicle's driving state control strategy and passenger motion sickness data can be used as sample data for model building and training.

[0085] FIG2 is a structural diagram of an eye-tracking vehicle control system provided by one or more embodiments of the present invention.

[0086] The eye tracking vehicle control system shown in FIG2 includes: a control unit module, an eye tracking module, a vehicle data module, and a vehicle control module;

[0087] Eye tracking module, used to collect passenger's eye movement status information;

[0088] Vehicle data module, used to collect vehicle driving status information;

[0089] A control unit module is used to process the passenger's eye movement state information and the vehicle's driving state information to generate instructions for controlling the vehicle's driving;

[0090] A vehicle control module is used to control the driving state of the vehicle according to the instructions for controlling the driving of the vehicle;

[0091] The eye tracking module collects the passenger's eye movement status information and sends it;

[0092] The vehicle data module collects the vehicle's driving status information and sends it;

[0093] The control unit module receives data from the eye tracking module and the vehicle data module, generates and sends instructions for controlling the vehicle's movement based on the passenger's eye movement status information and the vehicle's driving status information;

[0094] The vehicle control module receives data from the control unit module and controls the driving state of the vehicle according to the instructions for controlling the driving of the vehicle.

[0095] Specifically, the eye movement state information of the passenger and the driving state information of the vehicle are collected correspondingly through the eye tracking module and the vehicle data module. The control unit module is loaded with a model of how the vehicle driving state affects the passenger's motion sickness state, and generates instructions for controlling the vehicle's driving, wherein the passenger's motion sickness state is indirectly judged by the passenger's eye movement state. The instructions for controlling the vehicle's driving are used to improve the passenger's motion sickness state. The vehicle control module controls the vehicle's driving state according to the instructions for controlling the vehicle's driving. When the driving state is changed, the vehicle data module collects the vehicle's driving state information and feeds back the vehicle's driving state after executing the instructions for controlling the vehicle's driving. The eye tracking module collects the passenger's eye movement state information and feeds back whether the vehicle's driving state after executing the instructions for controlling the vehicle's driving has improved the passenger's motion sickness state.

[0096] In this embodiment, the control unit module includes: a data analysis module, a motion sickness determination module and a preventive measures module;

[0097] A data analysis module is used to process passenger eye movement status information and vehicle driving status information;

[0098] A motion sickness judgment module is used to judge whether the passenger's eye movement state is the eye movement state of motion sickness;

[0099] A preventive measures module, for generating a vehicle driving state to alleviate the passenger's motion sickness according to the passenger's motion sickness state;

[0100] The data analysis module receives the data from the eye tracking module, processes the eye movement state information of the passenger and the vehicle driving state information, generates feature data, and sends it;

[0101] The motion sickness judgment module receives data from the data analysis module and judges whether the passenger's eye movement state is an eye movement state of motion sickness based on the characteristic data;

[0102] If the passenger's eye movement state is an eye movement state of motion sickness, then a judgment result that the passenger is in a motion sickness state is sent;

[0103] The preventive measures module receives data from the motion sickness determination module and generates a vehicle driving state to alleviate the passenger's motion sickness based on the determination result that the passenger is in a motion sickness state;

[0104] Generate vehicle driving control instructions based on the vehicle driving status to alleviate passengers' motion sickness.

[0105] Specifically, the control unit module runs a data analysis module, a motion sickness determination module, and a preventive measures module. The data analysis module is used to process passenger eye movement status information and vehicle driving status information, and analyze the correlation between the vehicle driving status and the passenger's eye movement status. The preventive measures module determines the passenger's motion sickness status and the severity of motion sickness based on the passenger's eye movement status. The preventive measures module takes different preventive measures based on the passenger's motion sickness status and the severity of motion sickness, such as generating a vehicle driving status that alleviates the passenger's motion sickness and converting it into a vehicle driving control instruction. The data analysis module receives data from the motion sickness determination module, which is equivalent to indirectly receiving data from the data analysis module. Based on the characteristic data analyzed by the data analysis module, it controls the vehicle driving status accordingly. For example, if the data analysis module analyzes that an eye movement frequency greater than 60 times is characteristic data of dizziness, the preventive measures module instructs the vehicle to slow down or open the window to pass, so that the eye movement frequency drops below 60 times.

[0106] In this embodiment, the eye tracking module includes:

[0107] Recording passenger's eye movement data based on optical sensors;

[0108] According to the correlation between eye movement state and motion sickness, characteristic data of eye movement state is collected;

[0109] The characteristic data of the eye movement state includes characteristic data of pupil diameter, characteristic data of eye movement velocity, characteristic data of eye movement duration, and characteristic data of corneal reflection angle.

[0110] Specifically, an optical sensor, such as a camera, is placed in front of the passenger, or a VR device is worn on the passenger to record the passenger's eye movement data. Feature data of eye movement states that are associated with motion sickness are selected and used as feature data for determining motion sickness. For example, feature data of pupil diameter, eye movement velocity, eye movement duration, and corneal reflection angle are used.

[0111] In this embodiment, the vehicle data module includes:

[0112] Extract characteristic data of vehicle driving status based on its impact on motion sickness;

[0113] The characteristic data of the vehicle's driving status include vehicle speed, steering action information, and environmental information such as vehicle temperature and oxygen content.

[0114] Specifically, motion sickness is not only related to vehicle motion but also to non-vehicle motion factors like odor and oxygen levels. We extract characteristic data related to vehicle driving conditions, such as vehicle speed and steering, and environmental data like interior temperature and oxygen levels, as characteristic data. By modifying these characteristic data, we can alleviate or intensify motion sickness.

[0115] In this embodiment, the data analysis module includes:

[0116] The characteristic data of the passenger's eye movement state and the characteristic data of the vehicle's driving state are used as data samples to build and train a machine learning model;

[0117] Generate a prediction model based on building and training a machine learning model;

[0118] Import the prediction model into the motion sickness judgment module;

[0119] According to the result of the motion sickness judgment module judging the characteristic data, the driving preventive measures module generates a vehicle driving control instruction;

[0120] The vehicle data module feeds back characteristic data of the vehicle driving state for executing the vehicle driving control command;

[0121] The eye tracking module feeds back characteristic data of the eye movement state for the vehicle data module to execute the vehicle driving control command;

[0122] The data analysis module iterates model training and updates the prediction model based on the feedback feature data samples.

[0123] Specifically, machine learning technology is used to take the characteristic data of the passenger's eye movement status and the characteristic data of the vehicle's driving status as data samples, build and train a machine learning model, generate a prediction model, and import it into the motion sickness judgment module to convert the passenger's eye movement status data into data for judging the passenger's motion sickness status.

[0124] Due to individual and environmental differences, as well as the distinction between "odor sickness" and "vestibular sickness" as the causes of motion sickness, the feedback feature data samples can be used to iteratively train the model, update the prediction model, and improve prediction accuracy. For example, by collecting feedback feature data samples multiple times, the change in the vehicle's operating state corresponds to the change in eye movement state, indirectly determining whether the motion sickness state has changed. During the iterative process, the collected sample data includes both valid and invalid data. For example, if the prediction model for "odor sickness" does not improve significantly after slowing down the vehicle, the eye movement data for "odor sickness" can be used as valid data for the prediction model for "vestibular sickness."

[0125] FIG3 is a structural diagram of an eye-tracking vehicle control device provided by one or more embodiments of the present invention.

[0126] The eye tracking vehicle control device shown in FIG3 includes: an information acquisition module, a motion sickness determination module, and a travel control module;

[0127] An information collection module, used to collect eye movement status information of passengers in a moving vehicle;

[0128] A motion sickness judgment module is used to judge whether a passenger is in a motion sickness state based on eye movement status information;

[0129] The travel control module is used to control the vehicle's driving state if the passenger is in a state of motion sickness. Controlling the vehicle's driving state includes controlling the vehicle's speed, steering, and the vehicle's interior environment.

[0130] It is worth noting that although this system only discloses an information collection module, a motion sickness judgment module, and a travel control module, what the present invention wants to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with the existing technology to form an infinite number of embodiments or technical solutions. In other words, this system is open rather than closed. Just because this embodiment only discloses individual basic functional modules, it cannot be considered that the scope of protection of the claims of the present invention is limited to the above-mentioned basic functional modules.

[0131] Through the above solution, the following beneficial technical effects are achieved:

[0132] This application determines the passenger's motion sickness status by observing the passenger's eye movements, and promptly adjusts the vehicle's speed, steering, and interior environment to alleviate the passenger's accumulated motion sickness discomfort, allowing the passenger to maintain a tolerance range and complete the vehicle ride, thereby exercising the passenger's tolerance ability and further reducing the feeling of motion sickness during future rides.

[0133] This application uses a machine learning method to use the characteristic data of eye movement status and the characteristic data of vehicle driving status as sample data for model training to form a prediction model, so as to detect and accurately assess the passenger's motion sickness early, and predict the improvement of the motion sickness condition by changing the vehicle's driving status, the in-vehicle environment, etc., iterate the model training, and establish a reliable prediction model between vehicle driving and vehicle riding.

[0134] This application uses model training to gradually adjust the association weights of the characteristic data of eye movement status, the characteristic data of vehicle driving status and motion sickness, further optimizes the prediction of motion sickness and improves the motion sickness state by controlling the vehicle speed, steering action information and the temperature and oxygen content in the car.

[0135] FIG4 is a schematic diagram of an eye-tracking vehicle control system architecture according to a specific embodiment of the present invention.

[0136] FIG5 is a schematic diagram of collecting eye tracking data according to a specific embodiment of the present invention.

[0137] FIG6 is a schematic diagram of a machine learning method for training a motion sickness prediction model according to a specific embodiment of the present invention.

[0138] FIG7 is a schematic diagram of a motion sickness prediction model guiding vehicle control according to a specific embodiment of the present invention.

[0139] In a specific embodiment, as shown in FIG4 , an eye-tracking vehicle control system architecture is disclosed. This architecture can be used in a motion sickness prevention system based on AR or VR eye-tracking technology. The system can monitor the user's eye movements and vehicle operation data in real time, determine the user's motion sickness level based on the characteristics of the eye movements, and take corresponding prevention and treatment measures based on the vehicle's operation data to effectively solve the motion sickness problem of users in the back seat.

[0140] The system mainly includes: eye tracker, vehicle data collector, control unit and user interface.

[0141] Regarding the eye tracker, the user's eye movement is detected through optical principles, including but not limited to pupil diameter, eye movement speed, eye movement duration, corneal reflection and other characteristics, and the detected data is transmitted to the control unit.

[0142] Regarding the vehicle data collector, it collects data from the vehicle terminal and the on-board TBOX, including but not limited to driving speed, acceleration, window opening status, etc., and transmits the collected data to the control unit.

[0143] Regarding the control unit, quantitative indicators are extracted based on the eye movement data detected by the eye tracker to determine the user's degree of motion sickness, and corresponding prevention and control measures are taken based on the degree of motion sickness and the current operating data of the vehicle.

[0144] Regarding the user interface, the eye tracking data and the preventive and therapeutic measures taken by the control unit are displayed to the user, including but not limited to prompts on the degree of motion sickness, instructions on preventive and therapeutic measures, etc.

[0145] Callable subroutines or subroutine systems are set in the control unit, including a data analysis subroutine, a motion sickness judgment subroutine, and a prevention and control measure subroutine.

[0146] Regarding the data analysis subroutine, the eye movement data detected by the eye tracker is processed and analyzed to extract feature quantities related to motion sickness, such as pupil diameter, amplitude, speed, acceleration of eye movement, and duration of eye movement.

[0147] Regarding the motion sickness judgment subroutine, based on the feature quantity extracted by the data analysis subroutine, a machine learning algorithm or other algorithm is used to judge the user's motion sickness level, including no motion sickness, mild, moderate, and severe motion sickness.

[0148] Regarding the prevention and control measures subroutine, the output of the motion sickness determination subroutine is combined with vehicle operation data to take appropriate prevention and control measures, including but not limited to stopping AR / VR entertainment, adjusting the virtual environment, playing soothing music, sending instructions to open the rear window, providing deep breathing guidance, reminding the driver to drive smoothly, and reducing the speed.

[0149] In another specific embodiment, as shown in Figures 5, 6, and 7, a user wears an AR or VR device, which includes an eye tracker, a control unit, and a user interface. The eye tracker uses optical principles to detect the user's eye movement data, including pupil diameter, eye movement velocity, eye movement duration, corneal reflex, and other characteristics. The eye tracker transmits the detected eye movement data to the control unit. A vehicle data collector collects vehicle-side and onboard TBOX data, including but not limited to driving speed, acceleration, window opening status, etc., and transmits the collected data to the control unit. The control unit calls a data analysis subroutine to process and analyze the eye movement data and extract feature quantities related to motion sickness. The control unit calls a motion sickness determination subroutine to use a machine learning algorithm to determine the user's degree of motion sickness based on the feature quantities. A model is trained based on previously collected data and classified and analyzed with the current feature quantities to determine the user's degree of motion sickness. The control unit calls a preventive measures subroutine to take corresponding preventive measures based on the output of the motion sickness determination subroutine combined with vehicle operation data. Including but not limited to stopping AR / VR entertainment, adjusting the virtual environment, playing soothing music, sending instructions to open the rear window, providing deep breathing guidance, reminding the driver to drive smoothly, reducing the speed, etc.; the user interface shows the user eye tracking data and the prevention and control measures taken by the control unit.

[0150] Among them, the eye movement feature quantification methods include: pupil diameter, eye movement speed, and eye movement frequency.

[0151] Regarding pupil diameter, through image processing technology, pupil diameter information can be extracted from the user's eye photos.

[0152] Eye movement velocity can be calculated by calculating the ratio of the pixel distance of eye movement to the time. For example, if the user's eyes move 100 pixels in 1 second, then the eye movement velocity is 100 pixels / second.

[0153] Eye movement frequency can be calculated by counting the number of eye movements per unit time. For example, if a user moves their eyes 60 times in 1 minute, their eye movement frequency is 60 times per minute.

[0154] FIG8 is a block diagram of an electronic device structure of an eye-tracking vehicle control method provided by one or more embodiments of the present invention.

[0155] As shown in FIG8 , the present application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0156] A computer program is stored in the memory. When the computer program is executed by the processor, the processor performs the steps of an eye-tracking vehicle control method.

[0157] The present application also provides a computer-readable storage medium storing a computer program executable by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of an eye-tracking vehicle control method.

[0158] The present application also provides a vehicle, comprising:

[0159] An electronic device for implementing the steps of an eye-tracking vehicle control method;

[0160] a processor that runs a program, and when the program runs, executes the steps of the eye-tracking vehicle control method based on data output by the electronic device;

[0161] The storage medium is used to store a program, which, when running, executes the steps of the eye-tracking vehicle control method for data output from the electronic device.

[0162] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0163] The electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory. The operating system can be any one or more computer operating systems that control electronic devices through processes, such as the Linux operating system, the Unix operating system, the Android operating system, the iOS operating system, or the Windows operating system. In the embodiments of the present invention, the electronic device can be a handheld device such as a smartphone or a tablet computer, or an electronic device such as a desktop computer or a portable computer, which is not particularly limited in the embodiments of the present invention.

[0164] The execution subject of the electronic device control in the embodiment of the present invention can be an electronic device, or a functional module in the electronic device that can call a program and execute the program. The electronic device can obtain the firmware corresponding to the storage medium. The firmware corresponding to the storage medium is provided by the supplier. The firmware corresponding to different storage media can be the same or different, and is not limited here. After the electronic device obtains the firmware corresponding to the storage medium, it can write the firmware corresponding to the storage medium into the storage medium, specifically, burn the firmware corresponding to the storage medium into the storage medium. The process of burning the firmware into the storage medium can be implemented using existing technology and will not be described in detail in the embodiment of the present invention.

[0165] The electronic device can also obtain a reset command corresponding to the storage medium. The reset command corresponding to the storage medium is provided by the supplier. The reset commands corresponding to different storage media can be the same or different, and are not limited here.

[0166] In this case, the storage medium of the electronic device is a storage medium in which the corresponding firmware is written. The electronic device can respond to the reset command corresponding to the storage medium in which the corresponding firmware is written, thereby resetting the storage medium in which the corresponding firmware is written according to the reset command corresponding to the storage medium. The process of resetting the storage medium according to the reset command can be implemented in the existing technology and will not be described in detail in the embodiments of the present invention.

[0167] For the convenience of description, the above devices are described as various units and modules according to their functions. Of course, when implementing this application, the functions of each unit and module can be implemented in the same or multiple software and / or hardware.

[0168] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art in the art to which the present invention pertains. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with those in the context of the prior art and, unless specifically defined, will not be interpreted in an idealized or overly formal sense.

[0169] For simplicity of description, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because certain steps can be performed in other orders or simultaneously according to the embodiments of the present invention. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0170] From the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solution of the present application, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application or certain portions of the embodiments.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An eye movement tracking vehicle control method, characterized in that, The described eye movement tracking vehicle control method includes: Collect the eye movement state information of passengers on the moving vehicle; Judge whether the passenger is in a motion sickness state according to the eye movement state information; If the passenger is in a motion sickness state, control the vehicle driving state; The control of the vehicle driving state includes controlling the vehicle speed, steering and controlling the interior environment of the vehicle.

2. The eye movement tracking vehicle control method according to claim 1, wherein The control of the vehicle speed, steering and controlling the interior environment of the vehicle includes: Collect the eye movement state information of the passenger for controlling the vehicle speed, steering and controlling the interior environment of the vehicle; Judge whether the passenger's motion sickness state is relieved according to the eye movement state information; If the passenger's motion sickness state is relieved, mark the vehicle driving state; Generate vehicle control prompt information according to the marked vehicle driving state.

3. An eye movement tracking vehicle control system, characterized in that, The described eye movement tracking vehicle control system includes: a control unit module, an eye movement tracking module, a vehicle data module and a vehicle control module; The eye movement tracking module is used to collect the eye movement state information of passengers; The vehicle data module is used to collect the driving state information of the vehicle; The control unit module is used to process the eye movement state information of the passenger and the driving state information of the vehicle, and generate an instruction for controlling the vehicle driving; The vehicle control module is used to control the vehicle driving state according to the instruction for controlling the vehicle driving; The eye movement tracking module collects the eye movement state information of passengers and sends it; The vehicle data module collects the driving state information of the vehicle and sends it; The control unit module receives the data of the eye movement tracking module and the vehicle data module, generates an instruction for controlling the vehicle driving according to the eye movement state information of the passenger and the driving state information of the vehicle, and sends it; The vehicle control module receives the data of the control unit module and controls the vehicle driving state according to the instruction for controlling the vehicle driving.

4. The eye movement tracking vehicle control system according to claim 3, wherein The control unit module includes: a data analysis module, a motion sickness judgment module and a preventive measure module; The data analysis module is used to process the eye movement state information of the passenger and the driving state information of the vehicle; The motion sickness judgment module is used to judge whether the eye movement state of the passenger is the eye movement state under the motion sickness state; The preventive measure module is used to generate a vehicle driving state for relieving the passenger's motion sickness according to the passenger's motion sickness state; The data analysis module receives the data of the eye movement tracking module, processes the eye movement state information of the passenger and the driving state information of the vehicle, generates characteristic data, and sends it; The motion sickness judgment module receives the data of the data analysis module and judges whether the eye movement state of the passenger is the eye movement state under the motion sickness state according to the characteristic data; If the eye movement state of the passenger is the eye movement state under the motion sickness state, send the judgment result that the passenger is in the motion sickness state; The preventive measure module receives the data of the motion sickness judgment module and generates the vehicle driving state for relieving the passenger's motion sickness according to the judgment result that the passenger is in the motion sickness state; Generate an instruction for controlling the vehicle driving according to the vehicle driving state for relieving the passenger's motion sickness.

5. The eye movement tracking vehicle control system according to claim 4, characterized in that, The eye movement tracking module includes: Record the data of the eye movement state of the passenger according to the optical sensor; Collect the characteristic data of the eye movement state according to the correlation between the eye movement state and the motion sickness phenomenon; The characteristic data of the eye movement state includes the characteristic data of pupil diameter, the characteristic data of eye movement speed, the characteristic data of eye movement duration, and the characteristic data of corneal reflection angle.

6. The eye movement tracking vehicle control system according to claim 4, characterized in that The vehicle data module includes: Extract the characteristic data of the vehicle driving state according to the influence on the motion sickness phenomenon; The characteristic data of the vehicle driving state includes vehicle speed, the action information of steering, and the environmental information of the temperature and oxygen content in the vehicle.

7. The eye movement tracking vehicle control system according to claim 4, characterized in that The data analysis module includes: Use the characteristic data of the passenger's eye movement state and the characteristic data of the vehicle driving state as data samples to construct and train a machine learning model; Generate a prediction model according to the construction and training of the machine learning model; Import the prediction model into the motion sickness judgment module; According to the result of judging the characteristic data by the motion sickness judgment module, drive the prevention measure module to generate a vehicle driving control instruction; The vehicle data module feeds back the characteristic data of the vehicle driving state for executing the vehicle driving control instruction; The eye movement tracking module feeds back the characteristic data of the eye movement state for the vehicle data module to execute the vehicle driving control instruction; The data analysis module performs iterative model training and updates the prediction model according to the fed-back characteristic data samples.

8. An eye movement tracking vehicle control device, characterized in that, The eye movement tracking vehicle control device includes: An information acquisition module for acquiring the eye movement state information of passengers on a moving vehicle; A motion sickness judgment module for judging whether a passenger is in a motion sickness state according to the eye movement state information; A trip control module for controlling the vehicle driving state if the passenger is in a motion sickness state, and the controlling the vehicle driving state includes controlling the vehicle speed, steering, and controlling the vehicle interior environment.

9. An electronic device, characterized in that, Includes: A processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the eye movement tracking vehicle control method according to any one of claims 1 to 2.

10. A computer-readable storage medium, characterized in that, Includes: It stores a computer program executable by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of the eye movement tracking vehicle control method according to any one of claims 1 to 2.

Images