Intelligent vehicle control method, device and equipment based on driver state monitoring
Through the dual recognition of driver heart rate data of smart bracelets and cameras, the problem of insufficient accuracy of camera monitoring in the prior art is solved, and higher monitoring accuracy and safety guarantees are achieved.
Patent Information
- Application Number
- CN202510249254.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-04
- Publication Date
- 2025-06-27
AI Technical Summary
When the existing driver monitoring system is monitored by cameras, it is susceptible to factors such as lighting conditions, driver wearing glasses or occlusion, resulting in insufficient accuracy of misidentification and monitoring, which limits the reliability and practicality of the system.
The smart bracelet and camera are used to double-identify the driver's heart rate data. By obtaining the heart rate characteristic data of the bracelet and the driver's image data collected by the camera, the two data are weighted and fused based on the weight coefficient to generate the target heart rate data, and decide whether to take safety measures based on the comparison results of the target heart rate data and the warning threshold.
Through the integration of dual data sources, the accuracy and reliability of driver heart rate monitoring are improved, heart rate abnormalities can be captured in a timely manner, and appropriate safety measures are taken to ensure the safety of drivers and passengers.
Smart Images
Figure CN120207349A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of driver monitoring systems, and in particular, to an intelligent vehicle control method, device, and equipment based on driver state monitoring. Background Art
[0002] With the continuous progress of technology, the automotive industry is undergoing unprecedented changes. As the core of future transportation, intelligent vehicles are rapidly emerging and becoming the mainstream in the market. Intelligent vehicles not only inherit the convenience of traditional vehicles but also greatly enhance the driving experience and safety through innovative functions such as autonomous driving technology and intelligent cockpit systems. In addition, intelligent vehicles achieve real-time communication between vehicles and the outside world through vehicle networking technology, further improving the convenience and efficiency of travel.
[0003] Although intelligent vehicle technology brings many conveniences, human factors are still one of the main causes of traffic accidents. According to statistics, behaviors such as driver fatigue, distraction, and drunk driving are important reasons for traffic accidents. With the advent of an aging society, the risk of drivers losing their driving ability due to sudden health problems is also increasing. Therefore, real-time monitoring of the driver's state is crucial for preventing accidents and ensuring driving safety. The Driver Monitoring System (DMS) can significantly reduce the accident risk by analyzing the driver's behavior and physiological indicators in real time and issuing alarms and taking intervention measures in a timely manner.
[0004] Currently, the Driver Monitoring System (DMS) mainly monitors the driver's state through in-vehicle cameras. However, this camera-based monitoring technology has certain limitations. The monitoring effect of the camera is easily affected by factors such as lighting conditions, the driver wearing glasses or being blocked, resulting in misidentification. Moreover, the deployment position and angle of the camera may also affect the monitoring accuracy. These limitations restrict the reliability and practicality of the DMS system. Summary of the Invention
[0005] In view of this, one or more embodiments of the present disclosure provide an intelligent vehicle control method, device, and equipment based on driver state monitoring, which can accurately monitor the driver's heart rate state and timely control the vehicle to execute safety measures.
[0006] On the one hand, the present disclosure provides an intelligent vehicle control method based on driver state monitoring, the method includes: obtaining heart rate characteristic data of a driver's bracelet, and determining first heart rate data based on the heart rate characteristic data; obtaining driver image data, and determining second heart rate data based on the driver image data; weighted fusing the first heart rate data and the second heart rate data based on a weight coefficient to generate target heart rate data; determining whether to control the vehicle to execute safety measures according to the comparison result between the target heart rate data and a warning threshold.
[0007] On the other hand, the present disclosure also provides an intelligent vehicle control device based on driver state monitoring, which includes: a first acquisition unit for acquiring heart rate characteristic data of a driver's bracelet and determining first heart rate data based on the heart rate characteristic data; a second acquisition unit for acquiring driver image data and determining second heart rate data based on the driver image data; a data fusion unit for weighted-fusing the first heart rate data and the second heart rate data based on a weight coefficient to generate target heart rate data; and an intelligent control unit for determining whether to control the vehicle to execute safety measures according to a comparison result between the target heart rate data and a warning threshold.
[0008] On the other hand, the present disclosure also provides an electronic device, which includes a memory and a processor. The memory is used for storing a computer program, and when the computer program is executed by the processor, the above-mentioned intelligent vehicle control method based on driver state monitoring is implemented.
[0009] On the other hand, the present disclosure also provides a computer-readable storage medium, which is used for storing a computer program, and when the computer program is executed by a processor, the above-mentioned intelligent vehicle control method based on driver state monitoring is implemented.
[0010] The technical solutions provided by one or more embodiments of the present disclosure can double-identify the driver's heart rate data through an intelligent bracelet and a camera, ensuring the reliability of the data source. By using a weight coefficient to weighted-fuse the heart rate data obtained from two channels, the accuracy and availability of the target heart rate data are ensured, and the accuracy of driver heart rate monitoring is improved. According to the comparison result between the target heart rate data and the warning threshold, the abnormal situation of the driver's heart rate can be accurately captured, and safety measures can be taken in a timely manner to ensure the safety of the driver and passengers. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The features and advantages of the embodiments of the present disclosure will be more clearly understood by referring to the accompanying drawings. The drawings are schematic and should not be construed as imposing any limitation on the present disclosure. In the drawings:
[0012] Figure 1 A step schematic diagram of an intelligent vehicle control method based on driver state monitoring in one embodiment of the present disclosure is shown;
[0013] Figure 2 An architecture schematic diagram of an intelligent vehicle control system based on driver state monitoring in one embodiment of the present disclosure is shown;
[0014] Figure 3 A software framework schematic diagram of a cockpit domain controller in one embodiment of the present disclosure is shown;
[0015] Figure 4The flowchart of the intelligent vehicle control method based on driver state monitoring in an embodiment of the present disclosure is shown;
[0016] Figure 5 The schematic diagram of the functional modules of the intelligent vehicle control device based on driver state monitoring in an embodiment of the present disclosure is shown;
[0017] Figure 6 The schematic diagram of the structure of the electronic device in an embodiment of the present disclosure is shown. Detailed implementation manners
[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.
[0019] Please refer to Figure 1 , the intelligent vehicle control method based on driver state monitoring provided by an embodiment of the present disclosure can be applied to a vehicle and may include the following steps.
[0020] S1: Obtain the heart rate characteristic data of the driver's bracelet, and determine the first heart rate data based on the heart rate characteristic data.
[0021] In this embodiment, the driver's bracelet usually has a heart rate monitoring function or service. The vehicle can wirelessly obtain the heart rate characteristic data of the driver monitored by the driver's bracelet through Bluetooth, Wi-Fi, etc. After performing certain data processing (such as outlier cleaning, noise filtering, data smoothing, feature extraction, etc.) on the heart rate characteristic data, the first heart rate data of the driver can be obtained.
[0022] In some embodiments, the driver's bracelet can be matched and connected through a Bluetooth driver. By invoking the heart rate monitoring service of the driver's bracelet, the heart rate characteristic data can be obtained. By parsing the heart rate characteristic data, the first heart rate data can be determined.
[0023] In a practical application example, the in-vehicle Bluetooth module can be paired and authenticated with the driver's smart bracelet, which can include an authentication and security authentication process to ensure the accuracy and security of the connection. For example, the PasskeyEntry method can be used for matching and connection, that is, the matching code of the smart bracelet needs to be input for the vehicle's Bluetooth module for pairing and connection. After establishing a connection with the smart bracelet, the vehicle can instruct the smart bracelet to enable the heart rate monitoring service and receive the heart rate characteristic data collected by the smart bracelet. After performing one or more analysis means such as outlier cleaning, noise filtering, data smoothing, feature extraction, numerical conversion, and Kalman filtering on the heart rate characteristic data, the first heart rate data can be determined.
[0024] S2: Obtain the driver image data, and based on the driver image data, determine the second heart rate data.
[0025] In this embodiment, by using the in-vehicle camera, the vehicle can photograph or record the driver to generate driver image data. Based on the driver image data, by using some image analysis methods (such as imaging photoplethysmography, facial feature extraction and processing, deep learning model recognition, etc.), the second heart rate data of the driver can be determined.
[0026] In some embodiments, the in-vehicle camera can be accessed through the camera driver. By using the in-vehicle camera, the driver image data can be collected. Based on the driver image data, the driver's facial image can be determined. By analyzing the color change information of the driver's facial image, the second heart rate data can be determined.
[0027] In a practical application example, through the camera driver, the camera permission of the in-vehicle camera can be requested. On the basis of obtaining the camera permission, the video stream collection function of the in-vehicle camera can be started to obtain the driver image data. For the collected driver image data, face detection can be performed through tools such as OpenCV and MediaPipe to determine the driver's facial image. For determining the driver's facial image, the region of interest (ROI) can also be located to further improve the data processing efficiency and the accuracy of the calculation result. For example, in the driver's facial image, the regions of interest such as the tip of the nose, the forehead, and the cheeks are located. For the original driver's facial image or the driver's facial image with the region of interest located, the color channels can be separated, and the fluctuation of the color change can be calculated, and then the second heart rate data can be determined.
[0028] It should be noted that the beating of the human heart can cause periodic changes in the blood flow under the facial skin, and these changes will lead to weak periodic changes in the color of the reflected light on the facial skin. Although these changes are difficult to detect by the human eye, they can be captured by a camera. In particular, the green light channel is the most sensitive to changes in blood absorption. Therefore, by analyzing the green light channel signal of the facial image, the heart rate information can be extracted.
[0029] S3: Based on the weight coefficient, weighted fusion of the first heart rate data and the second heart rate data is performed to generate target heart rate data.
[0030] In this embodiment, weighted fusion of the first heart rate data and the second heart rate data to generate target heart rate data can effectively reduce misjudgment caused by noise or errors in a single data source, thereby improving the accuracy and reliability of overall heart rate monitoring. The weight coefficient can be a normalization coefficient.
[0031] In this embodiment, the weight coefficient can be preset and is used to characterize the importance of the heart rate data collected from two channels. For example, it can be set that the weight coefficient of the first heart rate data is higher than that of the second heart rate data, indicating that the heart rate data collected by the smart bracelet is more reliable, and the heart rate data obtained by the in-vehicle camera is used as an auxiliary reference; it can be set that the weight coefficient of the second heart rate data is higher than that of the first heart rate data, indicating that the heart rate data obtained by the in-vehicle camera is more reliable, and the heart rate data collected by the smart bracelet is used as an auxiliary reference; it can also be set that the weight coefficient of the first heart rate data is the same as that of the second heart rate data, indicating an equal reference to the heart rate data collected from the two channels.
[0032] In this embodiment, the weight coefficient can be dynamically changed, and the weight coefficients of the two heart rate monitoring data can be dynamically adjusted according to the real-time data quality of the two heart rate monitoring data. Judging the data quality of the heart rate monitoring data can be achieved by one or more means such as analyzing the signal-to-noise ratio, analyzing the signal strength, and detecting the number of outliers.
[0033] In some embodiments, the first signal quality of the first heart rate data can be detected. According to the first signal quality, the first weight value of the first heart rate data can be determined. After detecting the second signal quality of the second heart rate data, according to the second signal quality, the second weight value of the second heart rate data can be determined. Based on the first weight value and the second weight value, the weight coefficient can be updated. Using the updated weight coefficient, weighted fusion of the first heart rate data and the second heart rate data is performed to generate target heart rate data.
[0034] In a practical application example, the first signal quality of the first heart rate data and the second signal quality of the second heart rate data can be detected by one or more of the means such as analyzing the signal-to-noise ratio, analyzing the signal strength, and detecting the number of outliers. Using the first weight value and the second weight value, the first signal quality and the second signal quality can be quantitatively evaluated. By normalizing the first weight value and the second weight value, the latest weight coefficient can be obtained.
[0035] In a practical application example, the formula for weighted fusion of the first heart rate data and the second heart rate data is: Xt = K * HR(DMS) + (1 - K) * HR(wristband). Where Xt is the finally calculated target heart rate, HR(DMS) is the second heart rate measured by the DMS camera, HR(wristband) is the first heart rate measured by the smart wristband, and K is the dynamic weight. If the signal quality of the DMS camera is high, then "K" is larger and "1 - K" is smaller; if the signal quality of the smart wristband is high, then "K" is smaller and "1 - K" is larger.
[0036] In some embodiments, both the first timestamp of the first heart rate data and the second timestamp of the second heart rate data can be determined. According to the first timestamp and the second timestamp, the first heart rate data and the second heart rate data can be weighted and fused to generate the target heart rate data.
[0037] In a practical application example, based on the smart wristband and the DMS camera, two paths of data, namely the smart wristband heart rate data and the DMS heart rate data, can be extracted. Through the timestamp matching principle, the DMS heart rate data and the smart wristband heart rate data can be data-fused. The mechanism of timestamp matching can eliminate the fusion error caused by different data sampling rates.
[0038] S4: Determine whether to control the vehicle to perform safety measures according to the comparison result between the target heart rate data and the warning threshold.
[0039] In this embodiment, the target heart rate data can accurately reflect the current heart rate state of the driver. Through the comparison result between the target heart rate data and the warning threshold, the abnormal heart rate condition of the driver can be accurately captured. If there is an abnormal heart rate condition of the driver (such as sudden arrest), the vehicle can be controlled to take safety measures in time to ensure the safety of the driver and passengers.
[0040] In some embodiments, when the target heart rate data is lower than the first threshold, the vehicle can be controlled to send a warning message. The first threshold can be set according to actual applications. When the target heart rate data is lower than the first threshold, it indicates that the driver's state is poor and the driver needs to be reminded to take a rest actively. For example, warning text, warning images, and warning animations can be sent to the vehicle display screen, warning voices and warning sound effects can be played, and the seat warning vibration can be triggered. The content of the warning text or warning voice can be "Your heart rate is abnormal. Please stop and rest."
[0041] In some embodiments, when the target heart rate data is lower than the second threshold, the vehicle can be controlled to take emergency avoidance. The second threshold can be set according to actual applications. When the target heart rate data is lower than the second threshold, it indicates that the driver's state is critical and the driver may have difficulty taking a rest actively, so the vehicle needs to be controlled to perform automated avoidance means. For example, the vehicle can be controlled to decelerate, the horn can be used to warn pedestrians, and an emergency call can be triggered.
[0042] In some embodiments, the first threshold and the second threshold can be set simultaneously, and the value of the second threshold can be lower than the value of the first threshold. In this way, a gradient threshold can be formed to correspond to different critical scenarios.
[0043] In some embodiments, controlling the vehicle to take emergency avoidance can include, but is not limited to, taking the following measures: controlling the vehicle's power system to decelerate the whole vehicle; controlling the intelligent driving domain controller of the vehicle to perform lane centering driving; controlling the body domain controller of the vehicle to open all the vehicle windows; controlling the body domain controller of the vehicle to trigger the external warning horn; controlling the vehicle's on-vehicle communication module to make an emergency call.
[0044] Please refer to Figure 2 , an intelligent vehicle control system based on driver state monitoring provided by an embodiment of the present disclosure mainly consists of an intelligent bracelet, a DMS camera, an intelligent cockpit, a gateway, a body domain controller, an intelligent driving domain controller, and a power system.
[0045] The main node cockpit domain controller mainly consists of a Bluetooth antenna, a Bluetooth module, a deserializer, a processor, a microcontroller unit (MCU), and a controller area network (CAN) transceiver. The downstream nodes of the whole vehicle consist of a gateway, a body domain controller, an intelligent driving domain controller, and a power system. The gateway is mainly used for CAN message routing. The main function of the body domain controller is to control the opening and closing of the windows and trigger the external horn for warning. The main function of the intelligent driving domain controller is to perform intelligent driving functions such as lane centering. The main function of the power system is to perform deceleration and parking operations.
[0046] In this embodiment, please refer toFigure 3 and Figure 4 the cockpit domain controller may have Figure 3 the software framework shown in Figure 4 and may execute the method flow of the intelligent vehicle control method based on driver state monitoring shown in
[0047] Specifically, the BSP (Board Support Package) is a set of software components designed for a specific hardware platform. It includes hardware initialization code, device drivers, and an operating system adaptation layer. Its main purpose in this embodiment is to manage the Bluetooth driver and the camera driver.
[0048] After the Bluetooth service (BT Service) in the Android system runs the Bluetooth driver of the BSP, it can obtain the first heart rate data of the intelligent bracelet worn by the driver. The Bluetooth service can send the first heart rate data to the heart rate monitoring application in the Android system. After the DMS algorithm service in the Android system runs the camera driver of the BSP, it can obtain the driver image data and can parse the second heart rate data using the driver image data. The DMS algorithm service can also send the second heart rate data to the heart rate monitoring application.
[0049] The heart rate monitoring application can fuse the first heart rate data and the second heart rate data to obtain the target heart rate data. The heart rate monitoring application can compare the target heart rate data with the first threshold and can also compare the target heart rate data with the second threshold. When it is detected that the target heart rate data is lower than the first threshold, the heart rate monitoring application can issue a warning to prompt the driver to rest. When it is detected that the target heart rate data is lower than the second threshold, the heart rate monitoring application can notify the vehicle decision service (Carservice) in the Android system to initiate a vehicle control behavior.
[0050] The vehicle decision service can make a decision and generate a vehicle control request, such as one or more requests for controlling the window (opening all the vehicle windows), reducing the vehicle speed, lane centering, sounding the horn outside the vehicle, etc. The vehicle control request generated by the vehicle decision service can be transmitted to the vehicle control service (IC service) in the QNX system. The vehicle control service can convert the vehicle control request into a specific vehicle control signal and can clarify the signal value of the vehicle control signal.
[0051] The vehicle control signal generated by the vehicle control service can also be transmitted to the MCU. The MCU can convert the vehicle control signal into the signal format of the CAN signal and transmit it to the corresponding body domain controller, intelligent driving domain controller, powertrain, and in-vehicle communication module (Tbox) through the gateway. After that, the body domain controller can control the opening and closing of the window and trigger the external horn for warning; the intelligent driving domain controller can perform the lane centering task; the powertrain can reduce the vehicle speed and stop the vehicle; and the in-vehicle communication module can send warning messages.
[0052] In this embodiment, the Android system focuses on the information processing function and can more effectively process image data, parse heart rate data, fuse heart rate data, and generate accurate vehicle control requests. The QNX system provides high real-time performance and security and can generate stable and safe vehicle control signals after receiving vehicle control requests. The combined use of the Android system and the QNX system provides high performance, high security, and high flexibility for the intelligent cockpit system. Through the Hypervisor technology, these two operating systems can be flexibly configured and managed on the same cockpit domain controller. The Hypervisor virtualizes hardware resources (such as CPU, memory, I / O) and provides independent operating environments for different operating systems, realizing efficient resource management and secure isolation.
[0053] In this embodiment, the MCU can integrate a CAN controller to directly convert the vehicle control signal into a CAN signal, reducing the dependence on external dedicated protocol conversion chips and lowering the hardware cost. The MCU can flexibly configure parameters such as the format, baud rate, and frame structure of the CAN signal through software programming to adapt to different communication requirements. For example, the configuration can be completed through simple serial port instructions, which is convenient for development and debugging. The MCU can achieve low-latency and high-efficiency data transmission when processing signal conversion, especially in a vehicle control system that requires quick response, ensuring the timely transmission of signals.
[0054] The technical solution provided by one or more embodiments of the present disclosure can double-identify the driver's heart rate data through the smart bracelet and the camera, ensuring the reliability of the data source. By using the weight coefficient to weighted-fuse the heart rate data obtained from the two channels, the accuracy and availability of the target heart rate data are ensured, improving the accuracy rate of driver heart rate monitoring. According to the comparison result between the target heart rate data and the warning threshold, the abnormal situation of the driver's heart rate can be accurately captured, and safety measures can be taken in a timely manner to ensure the safety of the driver and passengers.
[0055] Please refer to Figure 5 , the present disclosure also provides an intelligent vehicle control device based on driver state monitoring, and the device includes:
[0056] The first acquisition unit 100 is configured to acquire the heart rate characteristic data of the driver's bracelet, and determine the first heart rate data based on the heart rate characteristic data;
[0057] The second acquisition unit 200 is configured to acquire the driver image data, and determine the second heart rate data based on the driver image data;
[0058] The data fusion unit 300 is configured to weighted-fuse the first heart rate data and the second heart rate data based on the weight coefficient to generate the target heart rate data;
[0059] The intelligent control unit 400 is configured to determine whether to control the vehicle to execute safety measures according to the comparison result between the target heart rate data and the warning threshold.
[0060] In one embodiment, the first acquisition unit 100 is specifically configured to match and connect to the driver's bracelet through a Bluetooth driver; call the heart rate monitoring service of the driver's bracelet to acquire the heart rate characteristic data; and analyze the heart rate characteristic data to determine the first heart rate data.
[0061] In one embodiment, the second acquisition unit 200 is specifically configured to access the in-vehicle camera through a camera driver; use the in-vehicle camera to collect the driver image data; determine the driver's facial image based on the driver image data; and analyze the color change information of the driver's facial image to determine the second heart rate data.
[0062] In one embodiment, the data fusion unit 300 is specifically configured to detect the first signal quality of the first heart rate data, and determine the first weight value of the first heart rate data according to the first signal quality; detect the second signal quality of the second heart rate data, and determine the second weight value of the second heart rate data according to the second signal quality; update the weight coefficient based on the first weight value and the second weight value; and use the updated weight coefficient to weighted-fuse the first heart rate data and the second heart rate data to generate the target heart rate data.
[0063] In one embodiment, the data fusion unit 300 is further configured to determine the first timestamp of the first heart rate data; determine the second timestamp of the second heart rate data; and weighted-fuse the first heart rate data and the second heart rate data according to the first timestamp and the second timestamp to generate the target heart rate data.
[0064] In one embodiment, the intelligent control unit 400 is specifically configured to control the vehicle to send a warning message when the target heart rate data is lower than the first threshold; and control the vehicle to perform emergency avoidance when the target heart rate data is lower than the second threshold.
[0065] In one embodiment, the intelligent control unit 400 includes an emergency avoidance sub-unit 401. The emergency avoidance sub-unit 401 is configured to perform at least one of the following functions: controlling the power system of the vehicle to decelerate the whole vehicle; controlling the intelligent driving domain controller of the vehicle to perform lane centering driving; controlling the body domain controller of the vehicle to open all the vehicle windows; controlling the body domain controller of the vehicle to trigger the external warning horn; controlling the on-vehicle communication module of the vehicle to make an emergency call.
[0066] Each unit described in the above embodiment can be specifically implemented by a computer chip or by a product with a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0067] For the convenience of description, the above devices are described by function as various units respectively. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0068] Please refer to Figure 6 , the present disclosure also provides an electronic device, which includes a memory and a processor. The memory is used to store a computer program, and when the computer program is executed by the processor, the intelligent vehicle control method based on driver state monitoring described above is implemented.
[0069] The present disclosure also provides a computer-readable storage medium, which is used to store a computer program, and when the computer program is executed by a processor, the intelligent vehicle control method based on driver state monitoring described above is implemented.
[0070] Among them, the processor can be a Central Processing Unit (CPU). The processor can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. chips, or a combination of the above types of chips.
[0071] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to the methods in the embodiments of the present disclosure. By running the non-transitory software programs, instructions, and modules stored in the memory, the processor can execute various functional applications and data processing of the processor, that is, implement the methods in the above method embodiments.
[0072] The memory may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created by the processor and the like. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.
[0073] Those skilled in the art can understand that to implement all or part of the processes in the above method embodiments, it can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the above method embodiments. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memories.
[0074] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the embodiments of the device, equipment, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0075] The above description is only for the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.
[0076] While embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations fall within the scope defined by the appended claims.
Claims
1. An intelligent vehicle control method based on driver status monitoring, characterized in that: The method comprises: Acquire heart rate characteristic data of the driver's wristband, and determine first heart rate data based on the heart rate characteristic data; Acquire driver image data, and determine second heart rate data based on the driver image data; Based on a weight coefficient, weightedly fuse the first heart rate data and the second heart rate data to generate target heart rate data; According to the comparison result between the target heart rate data and the warning threshold, it is determined whether to control the vehicle to execute safety measures.
2. The method according to claim 1, characterized in that The step of obtaining the heart rate characteristic data of the driver's wristband and determining the first heart rate data based on the heart rate characteristic data includes: The driver's wristband is matched and connected via Bluetooth driver; Calling the heart rate monitoring service of the driver's wristband to obtain the heart rate characteristic data; The heart rate characteristic data is analyzed to determine the first heart rate data.
3. The method according to claim 1, characterized in that The acquiring the driver image data and determining the second heart rate data based on the driver image data includes: Access the vehicle camera through the camera driver; Using the vehicle-mounted camera, collecting the driver image data; Determining a driver's facial image based on the driver image data; The color change information of the driver's facial image is analyzed to determine the second heart rate data.
4. The method according to claim 1, characterized in that The step of weighting and fusing the first heart rate data and the second heart rate data based on a weight coefficient to generate target heart rate data includes: detecting a first signal quality of the first heart rate data, and determining a first weight value of the first heart rate data according to the first signal quality; detecting a second signal quality of the second heart rate data, and determining a second weight value of the second heart rate data according to the second signal quality; Based on the first weight value and the second weight value, updating the weight coefficient; The first heart rate data and the second heart rate data are weightedly fused using the updated weight coefficient to generate the target heart rate data.
5. The method according to claim 1 or 4, characterized in that: The weighted fusion of the first heart rate data and the second heart rate data to generate target heart rate data includes: determining a first timestamp of the first heart rate data; determining a second timestamp of the second heart rate data; The first heart rate data and the second heart rate data are weighted and fused according to the first timestamp and the second timestamp to generate the target heart rate data.
6. The method according to claim 1, characterized in that The step of determining whether to control the vehicle to execute safety measures according to a comparison result between the target heart rate data and the warning threshold comprises: When the target heart rate data is lower than a first threshold, controlling the vehicle to send a warning message; When the target heart rate data is lower than a second threshold, the vehicle is controlled to perform emergency avoidance.
7. The method according to claim 6, characterized in that The controlling the vehicle to perform emergency avoidance includes at least one of the following: Controlling the power system of the vehicle to decelerate the entire vehicle; Controlling the intelligent driving domain controller of the vehicle to perform lane centering driving; Controlling the body domain controller of the vehicle to open all vehicle windows; Controlling the body domain controller of the vehicle to trigger an external warning horn; Control the vehicle-mounted communication module of the vehicle to make an emergency call.
8. An intelligent vehicle control device based on driver status monitoring, characterized in that: The device comprises: A first acquisition unit, configured to acquire heart rate characteristic data of a driver's wristband, and determine first heart rate data based on the heart rate characteristic data; a second acquiring unit, configured to acquire driver image data, and determine second heart rate data based on the driver image data; a data fusion unit, configured to weightedly fuse the first heart rate data and the second heart rate data based on a weight coefficient to generate target heart rate data; The intelligent control unit is used to determine whether to control the vehicle to execute safety measures based on the comparison result between the target heart rate data and the warning threshold.
9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.