Monitoring method and system for human-vehicle information fusion, and medium

Abstract

The invention relates to a human-vehicle information fusion monitoring method and system and a medium, and the method comprises the steps: U1, collecting the data information of the motion state of a vehicle in real time, obtaining the analysis data information of the sweat of a driver in real time based on a vehicle-mounted sweat analysis sensor, obtaining the face image data information of the driver in real time based on a vehicle-mounted camera, and carrying out the real-time monitoring of the motion state of the vehicle; data information of the heart rate of the driver is obtained in real time based on a vehicle-mounted heart rate sensor; and U2, on the basis of the analysis data information of the sweat of the driver, the face image data information of the driver and the data information of the heart rate of the driver, constructing a driver state analysis model based on a Gaussian kernel function, and representing the state of the driver to obtain data information of the state of the driver. According to the invention, the accurate understanding of the in-vehicle information acquisition module on the emotion of the driver is improved, the safety of the vehicle in the long-distance driving process is improved, and hidden dangers brought to the safety by driving fatigue are prevented.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle monitoring, and in particular to a monitoring method, system and medium for integrating human and vehicle information. Background Art

[0002] As a means of transportation, cars have brought great convenience to our lives and work, but they have also brought about traffic safety issues. Countless traffic accidents occur across the country every year, causing significant casualties and property losses. With the development of technology, the performance and safety alarm systems of cars have become more and more perfect. However, many traffic accidents still occur every year, which is closely related to the physical health and mental state of the driver.

[0003] However, it does not classify human physiological data, and often only issues an alarm when the human physiological data reaches the dangerous range, resulting in the driver not knowing whether the current human physiological data has already approached the edge of the dangerous range; when the driver's human physiological data reaches the dangerous range, the vehicle safety system issues an alarm, which cannot play the role of reminding the driver. At the same time, the existing vehicle safety system cannot distinguish the identities of different drivers, resulting in confusion in the human physiological data of different drivers.

[0004] At the same time, driver fatigue driving monitoring equipment based on video information has been integrated into the driving recorder on long-distance transport vehicles. Based on this recorder, the driver's fatigue status can be monitored in real time. However, since the fatigue status of different drivers is different, the monitoring accuracy is greatly reduced, resulting in this type of fatigue driving monitoring equipment cannot become a necessary equipment at the automotive level. Summary of the invention

[0005] In view of the above problems, the present invention provides a monitoring method, system and medium for human-vehicle information fusion, which not only improves the in-vehicle information collection module's accurate understanding of the driver's emotions, but also improves the safety of the car during long-distance driving to prevent safety hazards caused by driving fatigue.

[0006] In order to achieve the above-mentioned purpose and other related purposes, the technical solution provided by the present invention is as follows:

[0007] A monitoring method for human-vehicle information fusion, the method comprising:

[0008] U1. Collect the vehicle's motion status data in real time, obtain the driver's sweat analysis data in real time based on the vehicle-mounted sweat analysis sensor, obtain the driver's facial image data in real time based on the vehicle-mounted camera, and obtain the driver's heart rate data in real time based on the vehicle-mounted heart rate sensor;

[0009] U2. Based on the analysis data information of the driver's sweat, the driver's facial image data information and the driver's heart rate data information, a driver state analysis model based on a Gaussian kernel function is constructed to characterize the driver's state and obtain data information on the driver's state;

[0010] U3. Based on the data information of the driver's state and the data information of the vehicle's motion state, an improved multimodal feature fusion algorithm integrating the golden sine factor is used to fuse the driver's state and the vehicle's motion state to obtain the fused data information of the driver's and vehicle's state;

[0011] U4. Based on the fused data information on the status of the person and the vehicle, a person and vehicle monitoring model based on dynamic learning is constructed to monitor the status of the person and the vehicle, and obtain monitoring data information on the status of the person and the vehicle.

[0012] Furthermore, in step U2, the construction of a driver state analysis model based on a Gaussian kernel function to characterize the driver's state includes:

[0013] U21. Based on the analysis data information of the driver's sweat and the facial image data information of the driver, construct a first Gaussian kernel function G of the driver's state,

[0014]

[0015] Wherein, x1 is the analysis data information of the driver's sweat, x2 is the facial image data information of the driver, α1, α2 and α3 are the first weight factors of the driver's state analysis, and the first analysis state of the driver is characterized to obtain the data information of the first analysis state of the driver;

[0016] U22. Based on the driver's facial image data information and the driver's heart rate data information, establish the driver's state second Gaussian kernel function H,

[0017]

[0018] Wherein, x2 is the driver's facial image data information, x3 is the driver's heart rate data information, β1, β2 and β3 are the second weight factors of the driver's state analysis, and the second analysis state of the driver is characterized to obtain the data information of the driver's second analysis state;

[0019] U23. Based on the data information of the second analysis state of the driver and the data information of the first analysis state of the driver, establish the driver's state analysis function Q,

[0020]

[0021] Among them, q1 is the data information of the driver's second analysis state, q2 is the data information of the driver's first analysis state, δ1, δ2 and δ3 are the balance constant factors of the driver's state analysis, which characterize the driver's state and obtain the data information of the driver's state.

[0022] Furthermore, the balance yield factors δ1, δ2 and δ3 of the driver state analysis are:

[0023]

[0024] Among them, q1 is the data information of the driver's second analysis state, and q2 is the data information of the driver's first analysis state.

[0025] Furthermore, the constraints of the first weight factors α1, α2 and α3 of the driver state analysis are:

[0026]

[0027] The constraints of the second weight factors β1, β2 and β3 of the driver state analysis are:

[0028]

[0029] Among them, the value range of β1 is (0,1), the value range of β2 is (0,1), and the value range of β3 is (1,2).

[0030] Furthermore, in step U3, the improved multimodal feature fusion algorithm integrating the golden sine factor is used to fuse the driver's state and the vehicle's motion state, including:

[0031] U31. Based on the data information of the driver's state, construct a feature extraction function W of the driver's state integrating the golden sine factor,

[0032]

[0033] Among them, r1 is the data information of the driver's state, γ1, γ2 and γ3 are the integrated golden sine factors of the driver's state, and the characteristics of the driver's state are extracted to obtain the characteristic data information of the driver's state;

[0034] U32. Based on the data information of the motion state of the vehicle, a feature extraction function R of the motion state of the vehicle integrating the golden sine factor is constructed,

[0035]

[0036] Among them, r2 is the data information of the vehicle's motion state, η1, η2 and η3 are the integrated golden sine factors of the vehicle's motion state, and the characteristics of the vehicle's motion state are extracted to obtain the characteristic data information of the vehicle's motion state;

[0037] U33. Based on the characteristic data information of the vehicle's motion state and the characteristic data information of the driver's state, a multi-modal feature fusion function P is established.

[0038]

[0039] Among them, y1 is the characteristic data information of the vehicle's motion state, y2 is the characteristic data information of the driver's state, λ1, λ2 and λ3 are multimodal feature fusion factors, which fuse the driver's state and the vehicle's motion state to obtain the fused data information of the human and vehicle states.

[0040] Furthermore, the constraint function f of the multimodal feature fusion factors λ1, λ2 and λ3 is,

[0041]

[0042] The value range of the constraint function f is (2,3).

[0043] Furthermore, the integrated golden sine factors γ1, γ2 and γ3 of the driver's state are,

[0044]

[0045] The integrated golden sine factors η1, η2 and η3 of the vehicle's motion state are,

[0046]

[0047] Among them, r1 is the data information of the driver's state, and r2 is the data information of the vehicle's motion state.

[0048] Furthermore, in step U4, the construction of a human-vehicle monitoring model based on dynamic learning to monitor the status of the human-vehicle includes:

[0049] U41. Based on the data information of the fused state of the person and the vehicle, a data set of the fused state of the person and the vehicle is constructed;

[0050] U42. Input the fused data set of the human-vehicle status into the human-vehicle monitoring model based on dynamic learning for training and learning, and determine the human-vehicle monitoring function S,

[0051]

[0052] Among them, z is the fused data set of the human-vehicle status, μ1, μ2 and μ3 are dynamic learning factors for human-vehicle monitoring, and the trained human-vehicle monitoring model based on dynamic learning is obtained;

[0053] U43. Based on the trained human-vehicle monitoring model based on dynamic learning, the fused data information of the human-vehicle status is input, the human-vehicle status is monitored, and the monitoring data information of the human-vehicle status is obtained.

[0054] In order to achieve the above-mentioned purpose and other related purposes, the present invention also provides a monitoring system for human-vehicle information fusion, including a computer device, which is programmed or configured to execute the steps of any one of the human-vehicle information fusion monitoring methods.

[0055] In order to achieve the above-mentioned purpose and other related purposes, the present invention also provides a computer-readable storage medium, on which is stored a computer program programmed or configured to execute any one of the monitoring methods for human-vehicle information fusion.

[0056] The present invention has the following positive effects:

[0057] 1. The present invention characterizes the driver's state by constructing a driver state analysis model based on the Gaussian kernel function, and fuses the driver's state and the vehicle's motion state by combining an improved multimodal feature fusion algorithm that integrates the golden sine factor. This not only improves the in-vehicle information acquisition module's accurate understanding of the driver's emotions, but also improves the safety of the car during long-distance driving to prevent the hidden dangers of driving fatigue to safety.

[0058] 2. The present invention monitors the status of people and vehicles by constructing a human-vehicle monitoring model based on dynamic learning. This not only solves the problem that the existing solutions only focus on one type of information or one type of information of the human target, which cannot truly reflect a certain emotional state of the human body, but also the entire monitoring process can be adaptively adjusted according to different driving conditions, thereby improving the accuracy and robustness of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0060] Figure 2 A schematic diagram of a process of constructing a driver state analysis model based on a Gaussian kernel function according to the present invention;

[0061] Figure 3 It is a flow chart of an improved multi-modal feature fusion algorithm integrating golden sine factor of the present invention;

[0062] Figure 4The present invention is a flow chart of constructing a human-vehicle monitoring model based on dynamic learning. DETAILED DESCRIPTION

[0063] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0064] Example 1: Figure 1 As shown, a monitoring method for human-vehicle information fusion, the method comprising:

[0065] U1. Collect the vehicle's motion status data in real time, obtain the driver's sweat analysis data in real time based on the vehicle-mounted sweat analysis sensor, obtain the driver's facial image data in real time based on the vehicle-mounted camera, and obtain the driver's heart rate data in real time based on the vehicle-mounted heart rate sensor;

[0066] U2. Based on the analysis data information of the driver's sweat, the driver's facial image data information and the driver's heart rate data information, a driver state analysis model based on a Gaussian kernel function is constructed to characterize the driver's state and obtain data information on the driver's state;

[0067] U3. Based on the data information of the driver's state and the data information of the vehicle's motion state, an improved multimodal feature fusion algorithm integrating the golden sine factor is used to fuse the driver's state and the vehicle's motion state to obtain the fused data information of the driver's and vehicle's state;

[0068] U4. Based on the fused data information on the status of the person and the vehicle, a person and vehicle monitoring model based on dynamic learning is constructed to monitor the status of the person and the vehicle, and obtain monitoring data information on the status of the person and the vehicle.

[0069] In this embodiment, if Figure 2 As shown, in step U2, the construction of the driver state analysis model based on the Gaussian kernel function to characterize the driver's state includes:

[0070] U21. Based on the analysis data information of the driver's sweat and the facial image data information of the driver, construct a first Gaussian kernel function G of the driver's state,

[0071]

[0072] Wherein, x1 is the analysis data information of the driver's sweat, x2 is the facial image data information of the driver, α1, α2 and α3 are the first weight factors of the driver's state analysis, and the first analysis state of the driver is characterized to obtain the data information of the first analysis state of the driver;

[0073] U22. Based on the driver's facial image data information and the driver's heart rate data information, establish the driver's state second Gaussian kernel function H,

[0074]

[0075] Wherein, x2 is the driver's facial image data information, x3 is the driver's heart rate data information, β1, β2 and β3 are the second weight factors of the driver's state analysis, and the second analysis state of the driver is characterized to obtain the data information of the driver's second analysis state;

[0076] U23. Based on the data information of the second analysis state of the driver and the data information of the first analysis state of the driver, establish the driver's state analysis function Q,

[0077]

[0078] Among them, q1 is the data information of the driver's second analysis state, q2 is the data information of the driver's first analysis state, δ1, δ2 and δ3 are the balance constant factors of the driver's state analysis, which characterize the driver's state and obtain the data information of the driver's state.

[0079] In this embodiment, the balance yield factors δ1, δ2 and δ3 of the driver state analysis are,

[0080]

[0081] Among them, q1 is the data information of the driver's second analysis state, and q2 is the data information of the driver's first analysis state.

[0082] In this embodiment, the constraints of the first weight factors α1, α2 and α3 of the driver state analysis are:

[0083]

[0084] The constraints of the second weight factors β1, β2 and β3 of the driver state analysis are:

[0085]

[0086] Among them, the value range of β1 is (0,1), the value range of β2 is (0,1), and the value range of β3 is (1,2).

[0087] Example 2: Based on the monitoring method for human-vehicle information fusion in Example 1, the present invention is further illustrated and described below.

[0088] like Figure 1 As shown, a monitoring method for human-vehicle information fusion, the method comprising:

[0089] U1. Collect the vehicle's motion status data in real time, obtain the driver's sweat analysis data in real time based on the vehicle-mounted sweat analysis sensor, obtain the driver's facial image data in real time based on the vehicle-mounted camera, and obtain the driver's heart rate data in real time based on the vehicle-mounted heart rate sensor;

[0090] U2. Based on the analysis data information of the driver's sweat, the driver's facial image data information and the driver's heart rate data information, a driver state analysis model based on a Gaussian kernel function is constructed to characterize the driver's state and obtain data information on the driver's state;

[0091] U3. Based on the data information of the driver's state and the data information of the vehicle's motion state, an improved multimodal feature fusion algorithm integrating the golden sine factor is used to fuse the driver's state and the vehicle's motion state to obtain the fused data information of the driver's and vehicle's state;

[0092] U4. Based on the fused data information on the status of the person and the vehicle, a person and vehicle monitoring model based on dynamic learning is constructed to monitor the status of the person and the vehicle, and obtain monitoring data information on the status of the person and the vehicle.

[0093] In this embodiment, if Figure 3 As shown, in step U3, the improved multimodal feature fusion algorithm using the integrated golden sine factor to fuse the driver's state and the vehicle's motion state includes:

[0094] U31. Based on the data information of the driver's state, construct a feature extraction function W of the driver's state integrating the golden sine factor,

[0095]

[0096] Among them, r1 is the data information of the driver's state, γ1, γ2 and γ3 are the integrated golden sine factors of the driver's state, and the characteristics of the driver's state are extracted to obtain the characteristic data information of the driver's state;

[0097] U32. Based on the data information of the motion state of the vehicle, a feature extraction function R of the motion state of the vehicle integrating the golden sine factor is constructed,

[0098]

[0099] Among them, r2 is the data information of the vehicle's motion state, η1, η2 and η3 are the integrated golden sine factors of the vehicle's motion state, and the characteristics of the vehicle's motion state are extracted to obtain the characteristic data information of the vehicle's motion state;

[0100] U33. Based on the characteristic data information of the vehicle's motion state and the characteristic data information of the driver's state, a multi-modal feature fusion function P is established.

[0101]

[0102] Among them, y1 is the characteristic data information of the vehicle's motion state, y2 is the characteristic data information of the driver's state, λ1, λ2 and λ3 are multimodal feature fusion factors, which fuse the driver's state and the vehicle's motion state to obtain the fused data information of the human and vehicle states.

[0103] In this embodiment, the constraint function f of the multimodal feature fusion factors λ1, λ2 and λ3 is,

[0104]

[0105] The value range of the constraint function f is (2,3).

[0106] In this embodiment, the integrated golden sine factors γ1, γ2 and γ3 of the driver's state are,

[0107]

[0108] The integrated golden sine factors η1, η2 and η3 of the vehicle's motion state are,

[0109]

[0110]

[0111] Among them, r1 is the data information of the driver's state, and r2 is the data information of the vehicle's motion state.

[0112] In this embodiment, if Figure 4 As shown, in step U4, the construction of a human-vehicle monitoring model based on dynamic learning to monitor the status of the human-vehicle includes:

[0113] U41. Based on the data information of the fused state of the person and the vehicle, a data set of the fused state of the person and the vehicle is constructed;

[0114] U42. Input the fused data set of the human-vehicle status into the human-vehicle monitoring model based on dynamic learning for training and learning, and determine the human-vehicle monitoring function S,

[0115]

[0116] Among them, z is the fused data set of the human-vehicle status, μ1, μ2 and μ3 are dynamic learning factors for human-vehicle monitoring, and the trained human-vehicle monitoring model based on dynamic learning is obtained;

[0117] U43. Based on the trained human-vehicle monitoring model based on dynamic learning, the fused data information of the human-vehicle status is input, the human-vehicle status is monitored, and the monitoring data information of the human-vehicle status is obtained.

[0118] In this embodiment, the present invention provides a monitoring system for human-vehicle information fusion, including a computer device, which is programmed or configured to execute the steps of any one of the human-vehicle information fusion monitoring methods.

[0119] In this embodiment, the present invention provides a computer-readable storage medium, on which is stored a computer program programmed or configured to execute any one of the human-vehicle information fusion monitoring methods.

[0120] Any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0121] In summary, the present invention not only improves the in-vehicle information collection module's accurate understanding of the driver's emotions, but also improves the safety of the car during long-distance driving to prevent potential safety hazards caused by driving fatigue.

[0122] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A monitoring method for human-vehicle information fusion, characterized in that: The method comprises: U1. Collect the vehicle's motion status data in real time, obtain the driver's sweat analysis data in real time based on the vehicle-mounted sweat analysis sensor, obtain the driver's facial image data in real time based on the vehicle-mounted camera, and obtain the driver's heart rate data in real time based on the vehicle-mounted heart rate sensor; U2. Based on the analysis data information of the driver's sweat, the driver's facial image data information and the driver's heart rate data information, a driver state analysis model based on a Gaussian kernel function is constructed to characterize the driver's state and obtain data information on the driver's state; U3. Based on the data information of the driver's state and the data information of the vehicle's motion state, an improved multimodal feature fusion algorithm integrating the golden sine factor is used to fuse the driver's state and the vehicle's motion state to obtain the fused data information of the driver's and vehicle's state; U4. Based on the fused data information on the status of the person and the vehicle, a person and vehicle monitoring model based on dynamic learning is constructed to monitor the status of the person and the vehicle, and obtain monitoring data information on the status of the person and the vehicle.

2. The monitoring method for human-vehicle information fusion according to claim 1 is characterized in that: In step U2, the construction of a driver state analysis model based on a Gaussian kernel function to characterize the driver's state includes: U21. Based on the analysis data information of the driver's sweat and the facial image data information of the driver, construct a first Gaussian kernel function G of the driver's state, Wherein, x1 is the analysis data information of the driver's sweat, x2 is the facial image data information of the driver, α1, α2 and α3 are the first weight factors of the driver's state analysis, and the first analysis state of the driver is characterized to obtain the data information of the first analysis state of the driver; U22. Based on the driver's facial image data information and the driver's heart rate data information, establish the driver's state second Gaussian kernel function H, Wherein, x2 is the driver's facial image data information, x3 is the driver's heart rate data information, β1, β2 and β3 are the second weight factors of the driver's state analysis, and the second analysis state of the driver is characterized to obtain the data information of the driver's second analysis state; U23. Based on the data information of the second analysis state of the driver and the data information of the first analysis state of the driver, establish the driver's state analysis function Q, Among them, q1 is the data information of the driver's second analysis state, q2 is the data information of the driver's first analysis state, δ1, δ2 and δ3 are the balance constant factors of the driver's state analysis, which characterize the driver's state and obtain the data information of the driver's state.

3. The monitoring method for human-vehicle information fusion according to claim 2 is characterized in that: The balance yield factors δ1, δ2 and δ3 of the driver state analysis are, Among them, q1 is the data information of the driver's second analysis state, and q2 is the data information of the driver's first analysis state.

4. The monitoring method for human-vehicle information fusion according to claim 2 is characterized in that: The constraints of the first weight factors α1, α2 and α3 of the driver state analysis are: The constraints of the second weight factors β1, β2 and β3 of the driver state analysis are: Among them, the value range of β1 is (0,1), the value range of β2 is (0,1), and the value range of β3 is (1,2).

5. The monitoring method for human-vehicle information fusion according to claim 1 is characterized in that: In step U3, the improved multimodal feature fusion algorithm integrating the golden sine factor is used to fuse the driver's state and the vehicle's motion state, including: U31. Based on the data information of the driver's state, construct a feature extraction function W of the driver's state integrating the golden sine factor, Among them, r1 is the data information of the driver's state, γ1, γ2 and γ3 are the integrated golden sine factors of the driver's state, and the characteristics of the driver's state are extracted to obtain the characteristic data information of the driver's state; U32. Based on the data information of the motion state of the vehicle, a feature extraction function R of the motion state of the vehicle integrating the golden sine factor is constructed, Among them, r2 is the data information of the vehicle's motion state, η1, η2 and η3 are the integrated golden sine factors of the vehicle's motion state, and the characteristics of the vehicle's motion state are extracted to obtain the characteristic data information of the vehicle's motion state; U33. Based on the characteristic data information of the vehicle's motion state and the characteristic data information of the driver's state, a multi-modal feature fusion function P is established. Among them, y1 is the characteristic data information of the vehicle's motion state, y2 is the characteristic data information of the driver's state, λ1, λ2 and λ3 are multimodal feature fusion factors, which fuse the driver's state and the vehicle's motion state to obtain the fused data information of the human and vehicle states.

6. The monitoring method for human-vehicle information fusion according to claim 5 is characterized by: The constraint function f of the multimodal feature fusion factors λ1, λ2 and λ3 is, The value range of the constraint function f is (2,3).

7. The monitoring method for human-vehicle information fusion according to claim 5 is characterized by: The integrated golden sine factors γ1, γ2 and γ3 of the driver's state are, The integrated golden sine factors η1, η2 and η3 of the vehicle's motion state are, Among them, r1 is the data information of the driver's state, and r2 is the data information of the vehicle's motion state.

8. The monitoring method for human-vehicle information fusion according to claim 1 is characterized in that: In step U4, the construction of a human-vehicle monitoring model based on dynamic learning to monitor the status of the human-vehicle includes: U41. Based on the data information of the fused state of the person and the vehicle, a data set of the fused state of the person and the vehicle is constructed; U42. Input the fused data set of the human-vehicle status into the human-vehicle monitoring model based on dynamic learning for training and learning, and determine the human-vehicle monitoring function S, Among them, z is the fused data set of the human-vehicle status, μ1, μ2 and μ3 are dynamic learning factors for human-vehicle monitoring, and the trained human-vehicle monitoring model based on dynamic learning is obtained; U43. Based on the trained human-vehicle monitoring model based on dynamic learning, the fused data information of the human-vehicle status is input, the human-vehicle status is monitored, and the monitoring data information of the human-vehicle status is obtained.

9. A monitoring system for human-vehicle information fusion, comprising a computer device, characterized in that: The computer device is programmed or configured to execute the steps of the human-vehicle information fusion monitoring method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program programmed or configured to execute the human-vehicle information fusion monitoring method as described in any one of claims 1 to 8.

Images