A vehicle fusion control method and device, electronic equipment and storage medium

Abstract

This invention provides a vehicle fusion control method, device, electronic device, and storage medium. The method includes acquiring vehicle operation monitoring data and driver control perception data; determining a target safety level among multiple preset safety levels based on the operation monitoring data; if the target safety level is a first-class preset safety level, matching preset perception features based on thermal radiation perception data and limb motion perception data to obtain feature matching results; and controlling vehicle functions based on the feature matching results and a preset permissible operation strategy corresponding to the target safety level; if the target safety level is a second-class preset safety level, activating a preset driving focus mode; determining an emergency level based on changes in the target monitoring parameters and control perception data; and responding to emergencies based on the emergency level. This invention improves driving safety by limiting vehicle function responses through preset permissible operation strategies, thereby reducing traffic accidents caused by improper driver operation or negligence.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a vehicle fusion control method, device, electronic device and storage medium. Background Technology

[0002] With the development of automotive intelligence, in-car Bluetooth phone calls have become a standard feature in modern cars. However, answering calls mainly relies on physical buttons or voice control, which presents certain operational complexities and risks of accidental triggering, especially during driving, which can easily distract the driver and affect driving safety.

[0003] While related technologies use cameras to capture driver gestures for call answering, these camera-based solutions are highly dependent on lighting conditions and suffer from high false recognition rates. Furthermore, they do not consider driver authentication or safety mechanisms for hazardous scenarios, posing certain safety risks. Summary of the Invention

[0004] This invention provides a vehicle fusion control method, device, electronic device, and storage medium to solve the technical problems of the above-mentioned gesture control process being unable to achieve identity verification and safety control mechanisms in dangerous scenarios.

[0005] In one embodiment of this application, a vehicle fusion control method is provided, comprising: acquiring vehicle operation monitoring data and driver control perception data, wherein the operation monitoring data includes multiple types of safety monitoring parameters, and the control perception data includes thermal radiation perception data and limb motion perception data; determining a target safety level among multiple preset safety levels based on the operation monitoring data, wherein different preset safety levels correspond to different combinations of different types of safety monitoring parameters; if the target safety level is a first type of preset safety level, then matching preset perception features based on the thermal radiation perception data and the limb motion perception data to obtain a feature matching result, and performing vehicle function control based on the feature matching result and a preset permitted operation strategy corresponding to the target safety level; if the target safety level is a second type of preset safety level, then activating a preset driving focus mode, and determining an emergency level based on the target monitoring parameters and the perception changes in the control perception data, and performing an emergency response based on the emergency level, wherein the target monitoring parameters include at least one type of safety monitoring parameter.

[0006] In one embodiment of this application, determining a target safety level among multiple preset safety levels based on the operational monitoring data includes: if the first activation state parameter of the electronic stability control system is inactive and the vehicle speed is less than a preset speed threshold, then the target safety level is determined to be the highest safety level; if the second activation state parameter of the automatic emergency braking system is inactive and the lateral acceleration is less than a preset acceleration threshold, then the target safety level is determined to be the medium safety level; if the first activation duration of the electronic stability control system is greater than or equal to a preset first duration threshold, and / or the second activation duration of the automatic emergency braking system is greater than or equal to a preset second duration threshold, then the target safety level is determined to be the lowest safety level; wherein, the types of safety monitoring parameters include multiple types of dynamic parameters, multiple types of stability signal parameters, and multiple types of safety signal parameters, the types of dynamic parameters include the vehicle speed and the lateral acceleration, the types of stability signal parameters include the first activation state parameter and the first activation duration of the electronic stability control system, the types of safety signal parameters include the second activation state parameter and the second activation duration of the automatic emergency braking system, and each preset safety level includes the lowest safety level, the medium safety level, and the highest safety level.

[0007] In one embodiment of this application, determining the emergency level based on the perceived changes in target monitoring parameters and control perception data includes: determining the gesture control state based on heart rate fluctuations obtained from steering wheel grip force data, and determining the body temperature change value and body temperature change rate based on the thermal radiation perception data; if the body temperature change value is greater than or equal to a preset temperature change threshold, the gesture control state is a first abnormal state, and the target activation state parameter of the target safety control system is activated, then the emergency level is determined to be the lowest level; if the body temperature change rate is greater than or equal to a preset change rate threshold, the gesture control state is a second abnormal state, the vehicle deceleration is greater than a preset deceleration threshold, and the target activation state parameter of the target safety control system is activated, then the emergency level is determined to be the highest level; wherein, the control perception data also includes the steering wheel grip force data, the target safety control system includes at least one of an automatic emergency braking system and an electronic stability control system, the target monitoring parameters include the vehicle deceleration and the target activation state parameter, and the gesture control state includes one of a normal control state, the first abnormal state, and the second abnormal state, with the abnormality levels of the first abnormal state and the second abnormal state increasing sequentially.

[0008] In one embodiment of this application, emergency response based on the emergency level includes: if the emergency level is the lowest level, dialing an emergency alarm number and delaying the confirmation of dialing by a preset third time threshold; if the emergency level is the highest level, uploading vehicle-side data to the cloud and triggering emergency rescue services.

[0009] In one embodiment of this application, vehicle function control is performed based on the feature matching result and a preset permitted operation strategy corresponding to the target security level, including: if the feature matching result is successful, then the perception control operation corresponding to the limb motion perception data is determined as the target control operation; if the target security level is the highest security level, and the target control operation satisfies the first permitted operation strategy corresponding to the highest security level, then the target control operation is responded to, wherein the first permitted operation strategy includes at least one of answering all incoming calls and waking up voice control; if the target security level is the medium security level, and the target control operation satisfies the second permitted operation strategy corresponding to the medium security level, then the target control operation is responded to, wherein the second permitted operation strategy includes answering only incoming calls from preset contacts; wherein the preset permitted operation strategy is either the first permitted operation strategy or the second permitted operation strategy, and each preset security level includes the lowest security level, the medium security level, and the highest security level.

[0010] In one embodiment of this application, if the target security level is a first type of preset security level, then a feature matching result is obtained by matching preset sensing features based on the thermal radiation sensing data and the limb motion sensing data, including: if the target security level is the highest security level, then a first feature vector is generated based on the thermal radiation sensing data and the limb motion sensing data, and the first feature vector is matched with the first sensing feature to obtain a feature matching result; if the target security level is a medium security level, then a second feature vector is generated based on the limb motion sensing data, and the second feature vector is matched with the second sensing feature to obtain a feature matching result; wherein, the first type of preset security level includes the highest security level and the medium security level, the first feature vector includes a dorsal hand vein thermal radiation spectrum feature and a hand gesture feature to be detected, the first sensing feature includes at least one preset vein thermal radiation spectrum feature and at least one preset first hand gesture feature, the second feature vector includes a hand gesture feature to be detected, the second sensing feature includes at least one preset second hand gesture feature, and each preset security level includes the lowest security level, the medium security level, and the highest security level.

[0011] In one embodiment of this application, generating a first feature vector based on the thermal radiation sensing data and the limb motion sensing data includes: extracting the thermal radiation distribution map of the dorsal veins of the hand from the thermal radiation sensing data; reconstructing three-dimensional point cloud data of the finger joints based on the finger motion sensing data, wherein the limb motion sensing data includes upper limb motion sensing data and the finger motion sensing data, and the sensing wavelength corresponding to the upper limb motion sensing data is greater than the sensing wavelength corresponding to the finger motion sensing data; and generating a first feature vector based on the thermal radiation distribution map of the dorsal veins of the hand, the three-dimensional point cloud data of the finger joints, and the upper limb motion sensing data.

[0012] In one embodiment of this application, a vehicle fusion control device is provided, comprising: a data acquisition module for acquiring vehicle operation monitoring data and driver control perception data, wherein the operation monitoring data includes multiple types of safety monitoring parameters, and the control perception data includes thermal radiation perception data and limb motion perception data; a safety level determination module for determining a target safety level among multiple preset safety levels based on the operation monitoring data, wherein different preset safety levels correspond to different combinations of different types of safety monitoring parameters; a vehicle control module for, if the target safety level is a first type of preset safety level, matching preset perception features based on the thermal radiation perception data and the limb motion perception data to obtain a feature matching result, and controlling vehicle functions based on the feature matching result and a preset permitted operation strategy corresponding to the target safety level; and an emergency control module for, if the target safety level is a second type of preset safety level, activating a preset driving focus mode, determining an emergency level based on the perception changes of the target monitoring parameters and the control perception data, and performing an emergency response based on the emergency level, wherein the target monitoring parameters include at least one type of safety monitoring parameter.

[0013] In one embodiment of this application, an electronic device is provided, the electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device enables the vehicle fusion control method as described in any of the above embodiments.

[0014] In one embodiment of this application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer processor, the computer performs the vehicle fusion control method described in any of the above embodiments.

[0015] The beneficial effects of the embodiments of the present invention are as follows: The present invention provides a vehicle fusion control method, device, electronic device and storage medium. The embodiments of the present invention determine a target safety level among multiple preset safety levels by combining different types of safety monitoring parameters, and perform graded control of vehicle functions and emergency response control based on the target safety level and the driver's control perception data, thereby improving driving safety. Furthermore, in the graded control of vehicle functions, the response of vehicle functions can be restricted according to the preset allowed operation strategy corresponding to the target safety level, thereby reducing traffic accidents caused by improper operation or negligence of the driver.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0018] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown;

[0019] Figure 2 A schematic flowchart of a vehicle fusion control method according to an embodiment of this application is shown;

[0020] Figure 3 A system schematic diagram illustrating an embodiment of the vehicle fusion control method according to this application is shown;

[0021] Figure 4 A block diagram of a vehicle fusion control device according to an embodiment of this application is shown;

[0022] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0023] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0024] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the shape, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0025] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0026] Please see Figure 1 , Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown. For example... Figure 1 As shown, the system architecture may include vehicle sensors 101, a target safety control system 102, and a computer device 103. The computer device 101 may be at least one of a microcomputer, an embedded computer, etc. The target safety control system includes an Electronic Stability Control (ESC) system and an Automatic Emergency Braking (AEB) system. Vehicle sensors 101 are used to obtain control perception data and dynamic parameters. The target safety control system 102 transmits the stability signal parameters corresponding to the electronic stability control system and the safety signal parameters corresponding to the automatic emergency braking system to the computer device 103 via a Controller Area Network with Flexible Data-rate (CAN FD). The computer device 103 performs hierarchical vehicle fusion control, and the operational monitoring data includes dynamic parameters, stability signal parameters, and safety signal parameters.

[0027] For example, computer device 103 acquires vehicle operation monitoring data and driver control perception data. The operation monitoring data includes multiple types of safety monitoring parameters, and the control perception data includes thermal radiation perception data and limb motion perception data. Based on the operation monitoring data, a target safety level is determined from multiple preset safety levels. Different preset safety levels correspond to different combinations of different types of safety monitoring parameters. If the target safety level is a first type of preset safety level, preset perception features are matched based on the thermal radiation perception data and limb motion perception data to obtain feature matching results. Vehicle function control is performed based on the feature matching results and the preset permitted operation strategy corresponding to the target safety level. If the target safety level is a second type of preset safety level, a preset driving focus mode is activated, and an emergency level is determined based on the perception changes of the target monitoring parameters and control perception data. An emergency response is performed based on the emergency level. The target monitoring parameters include at least one type of safety monitoring parameter.

[0028] In related technologies, gesture control processes cannot achieve identity verification and security control mechanisms in dangerous scenarios.

[0029] To address the aforementioned technical problems, this application provides a vehicle fusion control method, apparatus, electronic device, and storage medium. The implementation details of the technical solutions in the embodiments of this application are described in detail below.

[0030] Please see Figure 2 , Figure 2 A schematic flowchart of a vehicle fusion control method according to an embodiment of this application is shown. Figure 2 As shown, in an exemplary embodiment, the vehicle fusion control method includes at least steps S210 to S240, which are described in detail below:

[0031] Step S210: Obtain vehicle operation monitoring data and driver control perception data.

[0032] The operational monitoring data includes various safety monitoring parameters, while the control and sensing data includes thermal radiation sensing data and limb movement sensing data.

[0033] Step S220: Determine the target safety level from multiple preset safety levels based on the operation monitoring data.

[0034] Different preset security levels correspond to different combinations of security monitoring parameters.

[0035] In one embodiment of this application, a target safety level is determined from multiple preset safety levels based on operational monitoring data, including: if the first activation state parameter of the electronic stability control system is inactive and the vehicle speed is less than a preset speed threshold, then the target safety level is determined to be the highest safety level; if the second activation state parameter of the automatic emergency braking system is inactive and the lateral acceleration is less than a preset acceleration threshold, then the target safety level is determined to be the medium safety level; if the first activation duration of the electronic stability control system is greater than or equal to a preset first duration threshold, and / or the second activation duration of the automatic emergency braking system is greater than or equal to a preset second duration threshold, then the target safety level is determined to be the lowest safety level; wherein, the types of safety monitoring parameters include multiple types of dynamic parameters, multiple types of stability signal parameters, and multiple types of safety signal parameters, the types of dynamic parameters include vehicle speed and lateral acceleration, the types of stability signal parameters include the first activation state parameter and the first activation duration of the electronic stability control system, the types of safety signal parameters include the second activation state parameter and the second activation duration of the automatic emergency braking system, and each preset safety level includes the lowest safety level, the medium safety level, and the highest safety level.

[0036] In one embodiment of this application, the highest safety level is detected by combining the vehicle speed in the dynamic parameters and the first activation state parameter in the stability signal parameters; the medium safety level is detected by combining the lateral acceleration in the dynamic parameters and the second activation state parameter in the safety signal parameters; and the lowest safety level is detected by combining the first activation duration in the stability signal parameters and / or the second activation duration in the safety signal parameters, so as to obtain the target safety level.

[0037] In one embodiment of this application, multiple preset safety levels are predefined using a weighted evaluation model based on safety monitoring parameters. For example, the weight allocation is as follows: stability signal parameters corresponding to ESC account for 40%, safety signal parameters corresponding to AEB account for 30%, lateral acceleration accounts for 20%, and vehicle speed accounts for 10%.

[0038] In one embodiment of this application, the preset speed threshold can be set to 80 km / h, the preset acceleration threshold can be set to 0.3 times the gravitational acceleration (0.3g), and both the preset first duration threshold and the preset second duration threshold can be set to 5 seconds. This is just an example, and this application does not impose any restrictions on the actual values ​​of various thresholds.

[0039] In one embodiment of this application, ESC not being activated indicates that the vehicle's handling stability is high, and there is no need for ESC intervention to maintain stable driving. Combined with the limitation of a preset speed threshold, this can be considered as being at the highest safety level.

[0040] In one embodiment of this application, if the second activation state parameter of the automatic emergency braking system is not activated and the lateral acceleration is less than a preset acceleration threshold, it indicates that the vehicle is performing a sharp turn or sudden braking, and at this time it can be considered to be in a medium safety level.

[0041] In one embodiment of this application, if the first activation duration of the electronic stability control system is greater than or equal to a preset first duration threshold, and / or the second activation duration of the automatic emergency braking system is greater than or equal to a preset second duration threshold, it indicates that the vehicle is in an emergency driving state. At this time, it can be considered to be at the lowest safety level, and the driver should focus on driving.

[0042] Step S230: If the target safety level is the first type of preset safety level, then the preset sensing features are matched according to the thermal radiation sensing data and the limb motion sensing data to obtain the feature matching result, and the vehicle function is controlled according to the feature matching result and the preset allowed operation strategy corresponding to the target safety level.

[0043] In one embodiment of this application, if the target safety level is a first-class preset safety level, then preset sensing features are matched based on thermal radiation sensing data and limb motion sensing data to obtain a feature matching result. This includes: if the target safety level is the highest safety level, then a first feature vector is generated based on the thermal radiation sensing data and limb motion sensing data, and the first feature vector is matched with the first sensing feature to obtain a feature matching result; if the target safety level is a medium safety level, then a second feature vector is generated based on the limb motion sensing data, and the second feature vector is matched with the second sensing feature to obtain a feature matching result; wherein, the first-class preset safety level includes the highest safety level and the medium safety level, the first feature vector includes a dorsal hand vein thermal radiation spectrum feature and a hand gesture feature to be detected, the first sensing feature includes at least one preset vein thermal radiation spectrum feature and at least one preset first hand gesture feature, the second feature vector includes a hand gesture feature to be detected, the second sensing feature includes at least one preset second hand gesture feature, and each preset safety level includes the lowest safety level, the medium safety level, and the highest safety level.

[0044] In one embodiment of this application, in order to reduce computational load and improve the control response speed of vehicle functions, identity verification and gesture control verification are performed in the highest security level scenario; in the medium security level scenario, only gesture control verification is performed.

[0045] In one embodiment of this application, the preset second gesture feature and the preset first gesture feature may be the same or different.

[0046] In one embodiment of this application, when the preset second gesture feature and the preset first gesture feature are the same, data security and driving safety can be improved by providing non-sensitive vehicle functions.

[0047] In one embodiment of this application, when the preset second gesture feature and the preset first gesture feature are different, the function control authority of the preset first gesture feature is greater than the function control authority of the preset first gesture feature, thereby improving data security and driving safety.

[0048] In one embodiment of this application, matching preset sensing features based on thermal radiation sensing data and limb motion sensing data to obtain feature matching results further includes: generating a first feature vector based on the thermal radiation sensing data and limb motion sensing data, the first feature vector including a dorsal hand vein thermal radiation map feature and a hand gesture feature to be tested; if the dorsal hand vein thermal radiation map feature successfully matches a preset vein thermal radiation map feature, the first matching result is determined as successful authentication; if the dorsal hand vein thermal radiation map feature fails to match multiple preset vein thermal radiation map features, the first matching result is determined as failed authentication; if the first matching result is successful authentication and the hand gesture feature to be tested successfully matches a preset third hand gesture feature, the feature matching result is determined as successful control verification of the first type; if the first matching result fails (successful authentication) and the hand gesture feature to be tested successfully matches a preset fourth hand gesture feature, the feature matching result is determined as successful control verification of the second type; wherein, the first feature vector includes a dorsal hand vein thermal radiation map feature and a hand gesture feature to be tested, and the function control authority of the preset third hand gesture feature is greater than the function control authority of the preset fourth hand gesture feature. The above embodiments perform gesture control verification for different function control permissions after identity verification, thereby improving data security and driving safety.

[0049] In one embodiment of this application, generating a first feature vector based on thermal radiation sensing data and limb motion sensing data includes: extracting the thermal radiation distribution map of the dorsal veins of the hand from the thermal radiation sensing data; reconstructing three-dimensional point cloud data of the finger joints based on the finger motion sensing data, wherein the limb motion sensing data includes upper limb motion sensing data and finger motion sensing data, and the sensing wavelength corresponding to the upper limb motion sensing data is greater than the sensing wavelength corresponding to the finger motion sensing data; and generating a first feature vector based on the thermal radiation distribution map of the dorsal veins of the hand, the three-dimensional point cloud data of the finger joints, and the upper limb motion sensing data.

[0050] In one embodiment of this application, thermal radiation sensing data is obtained through an infrared thermal imaging sensor. The infrared thermal imaging sensor is an infrared thermal imaging array with a spatial resolution of not less than 160×120 pixels.

[0051] In one embodiment of this application, a convolutional network is used to extract the thermal radiation distribution map of the veins on the back of the hand from the thermal radiation sensing data.

[0052] In one embodiment of this application, upper limb motion sensing data is obtained through a 24 GHz dual-frequency millimeter-wave radar. Leveraging its ±60° wide beam characteristic, it can capture a wide range of upper limb motion trajectories with a detection radius of 1.5 meters (m). Finger motion sensing data is obtained through a 60 GHz dual-frequency millimeter-wave radar. Utilizing its short wavelength characteristic of 5 millimeters (mm), it achieves a spatial resolution of 0.8 mm, thereby accurately reconstructing the three-dimensional point cloud data of the finger joints with a positioning error of <1.5 mm, providing sub-millimeter-level motion trajectory features for fine gesture recognition.

[0053] In one embodiment of this application, the upper limb movement trajectory includes range-based movement data such as the hand sliding steering wheel trajectory, the upper arm joint movement angle, and the distance of the hand from the center console.

[0054] In one embodiment of this application, finger motion sensing data includes micro-finger movement data such as lightly tapping the steering wheel and lightly swiping the steering wheel.

[0055] In one embodiment of this application, an infrared sensor and a dual-band radar array are arranged in at least one of the steering wheel, the driver's seat, and the top of the driver's cabin to capture the driver's thermal radiation perception data and limb motion perception data in a non-contact manner.

[0056] In one embodiment of this application, vehicle function control is performed based on feature matching results and preset permitted operation strategies corresponding to the target security level, including: if the feature matching result is successful, then the perception control operation corresponding to the limb motion perception data is determined as the target control operation; if the target security level is the highest security level and the target control operation satisfies the first permitted operation strategy corresponding to the highest security level, then the target control operation is responded to, the first permitted operation strategy including at least one of answering all incoming calls and waking up voice control; if the target security level is the medium security level and the target control operation satisfies the second permitted operation strategy corresponding to the medium security level, then the target control operation is responded to, the second permitted operation strategy including only answering incoming calls from preset contacts; wherein, the preset permitted operation strategy is either the first permitted operation strategy or the second permitted operation strategy, and each preset security level includes the lowest security level, the medium security level, and the highest security level.

[0057] In one embodiment of this application, when a vehicle owner binds a vehicle, preset sensory features are recorded. Among them, three to five preset gestures for answering a phone call can be used, regardless of whether the person is the vehicle owner or not; the preset vein thermal radiation pattern feature corresponding to the back of the hand represents the information of the vehicle owner, and preset contacts set by the driver are also recorded, including emergency contacts.

[0058] In one embodiment of this application, when the driver's voice cannot wake up the voice control, the voice control can be woken up by gesture.

[0059] In one embodiment of this application, successful matching also includes successful control verification of the first type and successful control verification of the second type.

[0060] In one embodiment of this application, if the current security level is medium, calls from emergency contacts can be answered, while other calls are not answered; when a new target security level of the highest security level is subsequently detected, calls are answered through the steps for the highest security level.

[0061] In one embodiment of this application, when answering a call via a vehicle Bluetooth phone, even if the feature matching result is successful and the preset allowed operation policy corresponding to the preset security level is met, the call can still be blocked in road sections where traffic rules prohibit answering vehicle Bluetooth phones.

[0062] Step S240: If the target safety level is the second-class preset safety level, then the preset driving focus mode is activated, and the emergency level is determined based on the perceived changes of the target monitoring parameters and control perception data, and an emergency response is carried out according to the emergency level.

[0063] Among them, the target monitoring parameters include at least one type of safety monitoring parameters.

[0064] In one embodiment of this application, the second type of preset security level includes the lowest security level. The preset driving focus mode includes blocking all incoming calls.

[0065] In one embodiment of this application, the detection conditions and preset permitted operation strategies for different preset security levels are as follows:

[0066] Table 1. Detection conditions and preset permissible operation strategies for different preset security levels.

[0067]

[0068]

[0069] In one embodiment of this application, in order to improve the accuracy of determining the emergency level and the accuracy of emergency response, it is necessary to increase the detection frequency of operational monitoring data and control sensing data.

[0070] In one embodiment of this application, determining the emergency level based on perceived changes in target monitoring parameters and control perception data includes: determining the gesture control state based on heart rate fluctuations obtained from steering wheel grip force data, and determining the body temperature change value and body temperature change rate based on thermal radiation perception data; if the body temperature change value is greater than or equal to a preset temperature change threshold, the gesture control state is a first abnormal state, and the target activation state parameter of the target safety control system is activated, then the emergency level is determined to be the lowest level; if the body temperature change rate is greater than or equal to a preset change rate threshold, the gesture control state is a second abnormal state, the vehicle deceleration is greater than a preset deceleration threshold, and the target activation state parameter of the target safety control system is activated, then the emergency level is determined to be the highest level; wherein, the control perception data also includes steering wheel grip force data, the target safety control system includes at least one of an automatic emergency braking system and an electronic stability control system, the target monitoring parameters include vehicle deceleration and target activation state parameters, and the gesture control state includes one of a normal control state, the first abnormal state, and the second abnormal state, with the abnormality levels of the first abnormal state and the second abnormal state increasing sequentially.

[0071] In one embodiment of this application, the target activation state parameter includes at least one of a first activation state parameter and a second activation state parameter.

[0072] In one embodiment of this application, the driver's state of emergency is indirectly determined by a steering wheel grip force sensor. By comparing changes in heart rate fluctuations with corresponding preset fluctuation thresholds, it can be determined whether the gesture control state is in a first abnormal state or a second abnormal state.

[0073] In one embodiment of this application, during an emergency, the driver's biometrics differ from normal, with a surge in the secretion of stress hormones such as adrenaline, leading to an increase or fluctuation in body temperature, which in turn alters the thermal radiation spectrum characteristics of the veins on the back of the hand. Furthermore, in the initial stage of stress hormone secretion, the redistribution of blood to the brain, heart, etc., may cause a decrease in the local temperature of the skin surface of the limbs. Muscle activity during an emergency response may cause this local temperature to rise. Moreover, due to the sweat glands and blood vessels in the palm, the temperature change in the palm is more pronounced than that in the back of the hand. Therefore, the body temperature change value can be determined by combining the changes in palm and back of hand temperature, or it can be determined directly based on the changes in palm temperature. The changes in palm and back of hand temperature are obtained based on thermal radiation sensing data.

[0074] In one embodiment of this application, the body temperature change value is used to characterize the absolute value of the body temperature difference, and the preset temperature change threshold can be set to 1.5 degrees Celsius (°C). That is, when the body temperature difference is within ±1.5°C, the emergency level can be determined as the lowest level by combining the gesture control state as the first abnormal state and the target activation state parameters corresponding to ESC and AEB as activated.

[0075] In one embodiment of this application, the preset rate of change threshold can be set to 3°C / s, and the preset deceleration threshold can be set to 0.4g.

[0076] In one embodiment of this application, emergency response is performed based on the emergency level, including: if the emergency level is the lowest level, an emergency alarm call is dialed and the call is confirmed after a preset third time threshold is delayed; if the emergency level is the highest level, vehicle-side data is uploaded to the cloud and emergency rescue services are triggered.

[0077] In one embodiment of this application, the preset third duration threshold can be set to 10s; the emergency alarm number is 122; and the emergency rescue service is the SOS rescue service provided by the Emergency Call System.

[0078] In one embodiment of this application, the triggering conditions and response actions for the emergency level are as follows:

[0079] Table 2. Triggering conditions and response actions for emergency levels

[0080]

[0081] In one embodiment of this application, a yellow alert can be used to indicate the lowest level of emergency, and a red alert can be used to indicate the highest level of emergency.

[0082] In one embodiment of this application, please refer to Figure 3 , Figure 3 A system schematic diagram illustrating an embodiment of the vehicle fusion control method according to this application is shown. Figure 3As shown, the system includes an information input module, a biological data acquisition module, and a hierarchical control module; the information input module includes a gesture input unit and an infrared information input unit; the biological data acquisition module includes a hand movement acquisition unit, a thermal imaging data acquisition unit, and a grip strength data acquisition unit; the hierarchical control module includes a hierarchical control unit and an emergency response unit. The system includes a gesture input unit for setting preset target gesture features, including preset first and second gesture features, or preset third gesture features and preset third gestures; an infrared information input unit for inputting preset venous thermal radiation spectrum features; a hand motion acquisition unit for acquiring driver's limb movement perception data; a thermal imaging data acquisition unit for acquiring driver's thermal radiation perception data; and a grip force data acquisition unit for acquiring driver's steering wheel grip force data. A hierarchical control unit is used to restrictive hierarchical control of vehicle functions based on the target safety level, thermal radiation perception data, and limb movement perception data, such as three levels of call answering control: answering all incoming calls, answering calls from emergency contacts, and blocking incoming calls. An emergency response unit is used to trigger the determination of the emergency level when the target safety level is the lowest, thereby triggering an emergency response before a collision occurs. Through the functions provided by the above system, this application can restrict the response of vehicle functions according to preset permissible operation strategies corresponding to the target safety level, reducing traffic accidents caused by improper operation or negligence of the driver; and it can avoid the inability to send effective rescue information due to communication failures after a collision.

[0083] Please see Figure 4 , Figure 4 A block diagram of a vehicle fusion control device according to an embodiment of this application is shown. This device can be applied to... Figure 1 The implementation environment shown is specifically configured in computer device 103. This device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0084] like Figure 4 As shown, a vehicle fusion control device 400 according to an embodiment of this application includes: a data acquisition module 401, a safety level determination module 402, a vehicle control module 403, and an emergency control module 404.

[0085] The data acquisition module 401 is used to acquire vehicle operation monitoring data and driver control perception data. The operation monitoring data includes multiple types of safety monitoring parameters, and the control perception data includes thermal radiation perception data and limb motion perception data.

[0086] The safety level determination module 402 is used to determine the target safety level from multiple preset safety levels based on operation monitoring data. Different preset safety levels correspond to different combinations of safety monitoring parameters.

[0087] The vehicle control module 403 is used to match preset sensing features based on thermal radiation sensing data and limb motion sensing data if the target safety level is the first type of preset safety level, obtain the feature matching result, and control the vehicle function according to the feature matching result and the preset allowed operation strategy corresponding to the target safety level.

[0088] The emergency control module 404 is used to activate a preset driving focus mode if the target safety level is the second type of preset safety level, and to determine the emergency level based on the perceived changes of the target monitoring parameters and control perception data, and to perform an emergency response based on the emergency level. The target monitoring parameters include at least one type of safety monitoring parameter.

[0089] It should be noted that the vehicle fusion control device and the vehicle fusion control method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the vehicle fusion control device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0090] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the vehicle fusion control method provided in the above embodiments.

[0091] Please see Figure 5 , Figure 5 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 5 The computer system 500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0092] like Figure 5As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage portion 508 into Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0093] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0094] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this application.

[0095] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0097] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0098] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the vehicle fusion control method provided in the various embodiments described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0099] In the above embodiments, unless otherwise specified, the use of ordinal numbers such as "first" and "second" to describe common objects only indicates that they refer to different instances of the same object, rather than indicating that the objects being described must be in a given order, whether temporally, spatially, sequentially, or in any other way.

[0100] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A vehicle fusion control method characterized by, The method includes: The system acquires vehicle operation monitoring data and driver control perception data. The operation monitoring data includes multiple types of safety monitoring parameters, and the control perception data includes thermal radiation perception data and limb movement perception data. Based on the operational monitoring data, a target security level is determined from multiple preset security levels, and different preset security levels correspond to different combinations of different types of security monitoring parameters; If the target safety level is the first type of preset safety level, then the preset sensing features are matched according to the thermal radiation sensing data and the limb motion sensing data to obtain the feature matching result, and the vehicle function is controlled according to the feature matching result and the preset allowed operation strategy corresponding to the target safety level. If the target safety level is the second type of preset safety level, then the preset driving focus mode is activated, and the emergency level is determined according to the perceived changes of the target monitoring parameters and the control perception data. An emergency response is then carried out according to the emergency level. The target monitoring parameters include at least one type of safety monitoring parameters. Based on the operational monitoring data, a target security level is determined from multiple preset security levels, including: If the first activation state parameter of the electronic stability control system is inactive and the vehicle speed is less than a preset speed threshold, the target safety level is determined to be the highest safety level; if the second activation state parameter of the automatic emergency braking system is inactive and the lateral acceleration is less than a preset acceleration threshold, the target safety level is determined to be the medium safety level; if the first activation duration of the electronic stability control system is greater than or equal to a preset first duration threshold, and / or the second activation duration of the automatic emergency braking system is greater than or equal to a preset second duration threshold, the target safety level is determined to be the lowest safety level; wherein, the types of safety monitoring parameters include multiple types of dynamic parameters, multiple types of stability signal parameters, and multiple types of safety signal parameters, the types of dynamic parameters include the vehicle speed and the lateral acceleration, the types of stability signal parameters include the first activation state parameter and the first activation duration of the electronic stability control system, the types of safety signal parameters include the second activation state parameter and the second activation duration of the automatic emergency braking system, and each preset safety level includes the lowest safety level, the medium safety level, and the highest safety level; Vehicle function control is performed based on the feature matching results and the preset permitted operation strategy corresponding to the target safety level, including: If the feature matching result is a successful match, then the perception control operation corresponding to the limb motion perception data is determined as the target control operation; If the target security level is the highest security level, and the target control operation satisfies the first permitted operation policy corresponding to the highest security level, then the target control operation is responded to. The first permitted operation policy includes at least one of answering all incoming calls and waking up voice control. If the target security level is medium security level, and the target control operation satisfies the second permitted operation policy corresponding to the medium security level, then the target control operation is responded to. The second permitted operation policy includes only answering calls from preset contacts. The preset allowed operation policy is either the first allowed operation policy or the second allowed operation policy, and each preset security level includes the lowest security level, the medium security level, and the highest security level.

2. The vehicle fusion control method according to claim 1, characterized by, The emergency level is determined based on changes in target monitoring parameters and the perceived control sensing data, including: The hand gesture control state is determined based on the heart rate fluctuations obtained from the steering wheel grip force data, and the body temperature change value and rate of change are determined based on the thermal radiation sensing data. If the body temperature change value is greater than or equal to the preset temperature change threshold, the gesture control state is the first abnormal state, and the target activation state parameter of the target safety control system is activated, then the emergency level is determined to be the lowest level. If the rate of change of body temperature is greater than or equal to a preset rate of change threshold, the gesture control state is a second abnormal state, the vehicle deceleration is greater than a preset deceleration threshold, and the target activation state parameter of the target safety control system is activated, then the emergency level is determined to be the highest level. The control perception data further includes the steering wheel grip force data; the target safety control system includes at least one of an automatic emergency braking system and an electronic stability control system; the target monitoring parameters include the vehicle deceleration and the target activation state parameters; and the gesture control state includes one of a normal control state, a first abnormal state, and a second abnormal state, with the abnormality levels of the first abnormal state and the second abnormal state increasing sequentially.

3. The vehicle fusion control method according to claim 2, characterized by, Emergency response shall be conducted according to the aforementioned emergency level, including: If the emergency level is the lowest level, then dial the emergency alarm number and confirm the dialing after a preset third time threshold. If the emergency level is the highest level, the vehicle data will be uploaded to the cloud and emergency rescue services will be triggered.

4. The vehicle fusion control method according to any one of claims 1 to 3, characterized by, If the target safety level is a first-class preset safety level, then based on the thermal radiation sensing data and the limb motion sensing data, preset sensing features are matched to obtain feature matching results, including: If the target safety level is the highest safety level, then a first feature vector is generated based on the thermal radiation sensing data and the limb motion sensing data, and the first feature vector is matched with the first sensing feature to obtain the feature matching result; If the target security level is medium security level, then a second feature vector is generated based on the limb motion perception data, and the second feature vector is matched with the second perception feature to obtain the feature matching result; Wherein, the first type of preset security level includes the highest security level and the medium security level, the first feature vector includes the dorsal hand vein thermal radiation spectrum feature and the hand gesture feature to be detected, the first sensing feature includes at least one preset vein thermal radiation spectrum feature and at least one preset first hand gesture feature, the second feature vector includes the hand gesture feature to be detected, the second sensing feature includes at least one preset second hand gesture feature, and each preset security level includes the lowest security level, the medium security level and the highest security level.

5. The vehicle fusion control method according to claim 4, characterized by, A first feature vector is generated based on the thermal radiation sensing data and the limb motion sensing data, including: Extract the thermal radiation distribution map of the veins on the back of the hand from the thermal radiation sensing data; Reconstructing three-dimensional point cloud data of finger joints based on finger motion perception data, wherein the limb motion perception data includes upper limb motion perception data and finger motion perception data, and the perception wavelength corresponding to the upper limb motion perception data is greater than the perception wavelength corresponding to the finger motion perception data; A first feature vector is generated based on the thermal radiation distribution map of the veins on the back of the hand, the three-dimensional point cloud data of the finger joints, and the motion perception data of the upper limb.

6. A vehicle fusion control device characterized by comprising: The device includes: The data acquisition module is used to acquire vehicle operation monitoring data and driver control perception data. The operation monitoring data includes multiple types of safety monitoring parameters, and the control perception data includes thermal radiation perception data and limb movement perception data. A safety level determination module is used to determine a target safety level from multiple preset safety levels based on the operational monitoring data. Different preset safety levels correspond to different combinations of safety monitoring parameters. Determining the target safety level from multiple preset safety levels based on the operational monitoring data includes: if the first activation state parameter of the electronic stability control system is inactive and the vehicle speed is less than a preset speed threshold, then the target safety level is determined to be the highest safety level; if the second activation state parameter of the automatic emergency braking system is inactive and the lateral acceleration is less than a preset acceleration threshold, then the target safety level is determined to be the medium safety level; if the first activation duration of the electronic stability control system is greater than or equal to a preset first duration threshold, and / or... If the second activation duration of the automatic emergency braking system is greater than or equal to a preset second duration threshold, then the target safety level is determined to be the lowest safety level. The types of safety monitoring parameters include multiple types of dynamic parameters, multiple types of stability signal parameters, and multiple types of safety signal parameters. The types of dynamic parameters include the vehicle speed and the lateral acceleration. The types of stability signal parameters include the first activation state parameter and the first activation duration of the electronic stability control system. The types of safety signal parameters include the second activation state parameter and the second activation duration of the automatic emergency braking system. Each preset safety level includes the lowest safety level, the medium safety level, and the highest safety level. A vehicle control module is configured to, if the target safety level is a first-class preset safety level, perform a matching of preset sensing features based on the thermal radiation sensing data and the limb motion sensing data to obtain a feature matching result, and perform vehicle function control based on the feature matching result and a preset permitted operation strategy corresponding to the target safety level; wherein, performing vehicle function control based on the feature matching result and the preset permitted operation strategy corresponding to the target safety level includes: if the feature matching result is a successful match, determining the sensing control operation corresponding to the limb motion sensing data as the target control operation; if the target safety level is the highest safety level, and the... If the target control operation satisfies the first permitted operation policy corresponding to the highest security level, then the target control operation is responded to. The first permitted operation policy includes at least one of answering all incoming calls and waking up voice control. If the target security level is a medium security level, and the target control operation satisfies the second permitted operation policy corresponding to the medium security level, then the target control operation is responded to. The second permitted operation policy includes answering only incoming calls from preset contacts. The preset permitted operation policy is either the first permitted operation policy or the second permitted operation policy, and each preset security level includes the lowest security level, the medium security level, and the highest security level. An emergency control module is used to activate a preset driving focus mode if the target safety level is a second-class preset safety level, determine the emergency level based on the perceived changes of the target monitoring parameters and the control perception data, and perform an emergency response based on the emergency level. The target monitoring parameters include at least one type of safety monitoring parameters.

7. An electronic device, comprising: The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the vehicle fusion control method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the vehicle fusion control method according to any one of claims 1 to 5.

Images