Air curtain control method and device based on driver state monitoring and electronic equipment

Abstract

The invention discloses an air curtain control method and device based on driver state monitoring and electronic equipment. The air curtain control method comprises the steps that face image time sequence data of a target driver and image time sequence data of a road in front of a vehicle are obtained; according to the face image time sequence data, eye muscle time sequence features, eyeball pupil time sequence features and head inclination amplitude time sequence features of the target driver are extracted, and according to the road image time sequence data, obstacle orientation time sequence features of the road in front of the vehicle are extracted; inputting the eye muscle time sequence characteristics, the eyeball pupil time sequence characteristics, the head inclination amplitude time sequence characteristics and the obstacle orientation time sequence characteristics into a pre-trained collision risk prediction model to obtain a collision risk prediction value; and determining a corresponding target air curtain unfolding strategy according to the collision risk prediction value, and controlling the vehicle air curtain according to the target air curtain unfolding strategy. The method improves the accuracy and timeliness of air curtain unfolding control, and can be applied to the technical field of vehicle control.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to an air curtain control method, device and electronic equipment based on driver status monitoring. Background Art

[0002] In the field of vehicle safety, Driver Monitoring Systems (DMS) are becoming an increasingly important tool for improving driving safety. DMS monitors the driver's condition in real time, such as fatigue and distraction, and issues warning signals to reduce the likelihood of accidents. Furthermore, as part of passive safety measures, intelligent air curtain systems deploy rapidly in the event of a collision, protecting occupants' heads from impact.

[0003] Existing intelligent curtain airbag systems typically rely on a fixed deployment strategy, whereby the curtain airbag deploys in a predetermined manner upon detecting a dangerous situation (e.g., a collision). However, this approach has the following drawbacks: 1) The response time is long, and the air curtain cannot be deployed in a timely manner; 2) Lack of flexibility. The deployment strategy of the air curtain cannot be adjusted according to different risk levels, which may result in insufficient or excessive protection for occupants in some cases.

[0004] 3) The system has a high false trigger rate, which increases unnecessary operating costs and easily causes passengers to feel uneasy.

[0005] Therefore, there is an urgent need for a more intelligent and flexible air curtain deployment control solution to improve vehicle safety and user driving experience. Summary of the Invention

[0006] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0007] To this end, an object of an embodiment of the present invention is to provide an air curtain control method based on driver status monitoring, which improves the accuracy and timeliness of air curtain deployment control, thereby improving vehicle safety and user driving experience.

[0008] Another object of an embodiment of the present invention is to provide an air curtain control device based on driver status monitoring.

[0009] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include: In one aspect, an embodiment of the present invention provides a method for controlling an air curtain based on driver status monitoring, comprising the following steps: Obtaining facial image time series data of the target driver and road image time series data in front of the vehicle; Extracting the target driver's eye muscle timing features, eyeball pupil timing features, and head tilt amplitude timing features based on the facial image timing data, and extracting the obstacle orientation timing features on the road ahead of the vehicle based on the road image timing data; Inputting the eye muscle timing characteristics, the eyeball pupil timing characteristics, the head tilt amplitude timing characteristics, and the obstacle orientation timing characteristics into a pre-trained collision risk prediction model to obtain a collision risk prediction value; A corresponding target air curtain deployment strategy is determined according to the collision risk prediction value, and the vehicle air curtain is controlled according to the target air curtain deployment strategy.

[0010] Furthermore, in one embodiment of the present invention, extracting the target driver's eye muscle timing features, eyeball pupil timing features, and head tilt amplitude timing features based on the facial image timing data specifically includes: Performing edge detection on each frame of facial image in the facial image time series data to obtain eye contours and head contours; Extracting a plurality of consecutive frames of eye region images from the facial image time series data according to the eye contour; Extracting periocular muscle texture features and determining eyelid opening and closing degrees based on the eye region image, and performing time series processing on the periocular muscle texture features and the eyelid opening and closing degrees corresponding to a plurality of consecutive frames of the eye region image to obtain the eye muscle time series features; Binarizing the eye region image and detecting the pupil contour, determining the pupil position and pupil diameter based on the pupil contour, and performing time series processing on the pupil positions and pupil diameters corresponding to a plurality of consecutive frames of the eye region image to obtain the eyeball pupil time series feature; The corresponding head tilt amplitude is determined according to the head contour, and the head tilt amplitude time series feature is determined according to the head tilt amplitude corresponding to a plurality of consecutive frames of the facial images.

[0011] Furthermore, in one embodiment of the present invention, extracting the time series features of the obstacle orientation on the road ahead of the vehicle based on the road image time series data specifically includes: determining a plurality of consecutive frames of road image information according to the road image time series data; performing obstacle detection on the road image information to obtain an obstacle direction and an obstacle distance, and determining an obstacle orientation feature based on the obstacle direction and the obstacle distance; The obstacle orientation time series feature is generated according to the obstacle orientation features corresponding to multiple consecutive frames of road image information.

[0012] Furthermore, in one embodiment of the present invention, the collision risk prediction model is trained by the following steps: Obtaining time series samples of the driver's facial image and the road image in front of the vehicle in historical driving scenes; Extracting eye muscle temporal feature samples, eye pupil temporal feature samples, and head tilt amplitude temporal feature samples based on the facial image temporal sample, and extracting obstacle orientation temporal feature samples based on the road image temporal sample; Constructing training samples based on the eye muscle temporal feature samples, the eye pupil temporal feature samples, the head tilt amplitude temporal feature samples, and the obstacle orientation temporal feature samples, determining collision risk labels corresponding to the training samples through manual labeling, and constructing a training data set based on the training samples and the collision risk labels; The training data set is input into a pre-built CNN-LSTM hybrid neural network for training to obtain the trained collision risk prediction model.

[0013] Furthermore, in one embodiment of the present invention, the CNN-LSTM hybrid neural network includes an input layer, a CNN convolutional layer, a feature fusion layer, an LSTM layer, an attention layer and an output layer, the input layer is used to input the training sample, the CNN convolutional layer is used to perform feature extraction on the training sample to obtain multiple local time series features, the feature fusion layer is used to perform feature fusion on the local time series features to obtain fused time series features, the LSTM layer is used to generate a hidden state sequence based on the fused time series features, the attention layer is used to dynamically assign weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism, and the output layer is used to map the hidden state sequence after dynamic weight assignment to a collision risk prediction result.

[0014] Furthermore, in one embodiment of the present invention, inputting the training data set into a pre-built CNN-LSTM hybrid neural network for training to obtain the trained collision risk prediction model specifically includes: Input the training sample through the input layer; The CNN convolutional layer extracts the eye muscle temporal feature samples, the eye pupil temporal feature samples, the head tilt amplitude temporal feature samples, and the obstacle orientation temporal feature samples to obtain eye muscle local features, eye pupil local features, head tilt amplitude local features, and obstacle orientation local features; The feature fusion layer performs feature fusion on the eye muscle local feature, the eye pupil local feature, the head tilt amplitude local feature, and the obstacle orientation local feature to obtain a fused time series feature; Generate a hidden state sequence according to the fused time series features through the LSTM layer; Dynamically weighting each dimension of the hidden state sequence based on a multi-head self-attention mechanism through the attention layer; Mapping the hidden state sequence after dynamic weight allocation into a collision risk prediction result through the output layer; Determine a loss value according to the collision risk prediction result and the corresponding collision risk label; The parameters of the CNN-LSTM hybrid neural network are updated according to the loss value through a back propagation algorithm to obtain the trained collision risk prediction model.

[0015] Furthermore, in one embodiment of the present invention, determining a corresponding target air curtain deployment strategy according to the collision risk prediction value, and controlling the vehicle air curtain according to the target air curtain deployment strategy, specifically includes: According to the collision risk prediction value, a matching is triggered in a preset air curtain deployment strategy mapping table to obtain the corresponding target air curtain deployment strategy; triggering a corresponding collision risk warning signal according to the collision risk prediction value; determining a corresponding curtain airbag deployment area and curtain airbag inflation pressure according to the target curtain airbag deployment strategy when the duration of the collision risk warning signal reaches a preset first threshold; and controlling the vehicle curtain airbag according to the curtain airbag deployment area and the curtain airbag inflation pressure; or, Acquire radar detection data in front of the vehicle, determine a current collision risk value based on the radar detection data, and when the difference rate between the current collision risk value and the collision risk prediction value is less than or equal to a preset second threshold, determine the corresponding air curtain deployment area and air curtain inflation pressure based on the target air curtain deployment strategy, and control the vehicle air curtain based on the air curtain deployment area and the air curtain inflation pressure.

[0016] On the other hand, an embodiment of the present invention provides an air curtain control device based on driver status monitoring, comprising: A data acquisition module is used to acquire facial image time series data of the target driver and road image time series data in front of the vehicle; a feature extraction module, configured to extract the target driver's eye muscle timing features, eye pupil timing features, and head tilt amplitude timing features based on the facial image timing data, and extract the obstacle orientation timing features on the road ahead of the vehicle based on the road image timing data; a collision risk prediction module, configured to input the eye muscle timing characteristics, the eye pupil timing characteristics, the head tilt amplitude timing characteristics, and the obstacle orientation timing characteristics into a pre-trained collision risk prediction model to obtain a collision risk prediction value; The air curtain deployment control module is used to determine a corresponding target air curtain deployment strategy according to the collision risk prediction value, and control the vehicle air curtain according to the target air curtain deployment strategy.

[0017] In another aspect, an embodiment of the present invention provides an electronic device, including: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned air curtain control method based on driver state monitoring.

[0018] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program executable by a processor, wherein the computer program implements the above-mentioned air curtain control method based on driver status monitoring when executed by the processor.

[0019] On the other hand, an embodiment of the present invention further provides a computer program product, including a computer program, which implements the above-mentioned air curtain control method based on driver state monitoring when executed by a processor.

[0020] The advantages and benefits of the present invention will be described in part in the following description and will become apparent from the following description or learned through practice of the present invention: An embodiment of the present invention obtains facial image time series data of a target driver and road image time series data in front of a vehicle, extracts eye muscle time series features, eye pupil time series features, and head tilt amplitude time series features of the target driver based on the facial image time series data, extracts obstacle orientation time series features on the road in front of the vehicle based on the road image time series data, inputs the eye muscle time series features, eye pupil time series features, head tilt amplitude time series features, and obstacle orientation time series features into a pre-trained collision risk prediction model to obtain a collision risk prediction value, determines a corresponding target air curtain deployment strategy based on the collision risk prediction value, and controls the vehicle air curtain based on the target air curtain deployment strategy. The embodiment of the present invention extracts eye muscle timing features, eye pupil timing features, and head tilt amplitude timing features based on the driver's facial image timing data, extracts obstacle orientation timing features based on the road image timing data in front of the vehicle, outputs a collision risk prediction value at the current moment based on a pre-trained collision risk prediction model, determines a corresponding air curtain deployment strategy based on the collision risk prediction value, and controls the vehicle air curtain according to the air curtain deployment strategy, so that the air curtain deployment can be controlled in a timely and accurate manner when the vehicle is about to collide, thereby improving the accuracy and timeliness of the air curtain deployment control, and also improving the safety of the vehicle and the user's driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A flowchart of a method for controlling an air curtain based on driver status monitoring provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a CNN-LSTM hybrid neural network provided in an embodiment of the present invention; Figure 3 A structural block diagram of an air curtain control device based on driver status monitoring provided by an embodiment of the present invention; Figure 4 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the objects, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the appended claims.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0025] The existing air curtain deployment control scheme has the following disadvantages: 1) Long response time: In existing technologies, the response time of air curtain systems is generally long, usually exceeding 100 milliseconds, and they are unable to react in the shortest possible time, resulting in the possibility of not being able to protect occupants in time in an emergency.

[0026] 2) Fixed deployment strategy: Existing curtain airbag deployment strategies are typically fixed and cannot be flexibly adjusted to different risk levels. This fixed strategy may result in over- or underprotection of occupants in certain situations, affecting the effectiveness of the system.

[0027] 3) High false trigger rate: Existing systems are prone to false triggering when processing DMS signals, increasing unnecessary operating costs and causing passenger anxiety. This not only reduces system reliability but can also negatively impact the driving experience.

[0028] This invention primarily enhances the intelligence of vehicle safety systems, particularly the linkage control between the Driver Monitoring System (DMS) and the intelligent curtain airbag system. By analyzing the driver's fatigue, distraction, and road conditions in real time, the system automatically determines the current collision risk level and adjusts the curtain airbag deployment strategy accordingly. This not only improves the responsiveness and accuracy of curtain airbag deployment control but also reduces the probability of false triggering, providing more reliable safety for drivers and passengers. It can be applied to long-distance driving, urban traffic congestion, and driving in adverse weather conditions, ensuring effective protection for occupants in a variety of complex environments.

[0029] The air curtain control method based on driver status monitoring provided in the embodiment of the present invention can be applied to a terminal, a server, or software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a vehicle-mounted terminal, etc., but is not limited thereto; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the air curtain control method based on driver status monitoring, etc., but is not limited to the above forms.

[0030] The present invention can be used in a wide variety of general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0031] It should be noted that in various specific embodiments of the present invention, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present invention needs to obtain the user's sensitive personal information, it will obtain the user's separate permission or consent through a pop-up window or jump to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present invention will be obtained.

[0032] Reference Figure 1 The embodiment of the present invention provides an air curtain control method based on driver status monitoring, which specifically includes the following steps: S101, obtaining time-series data of a target driver's facial image and time-series data of a road image in front of the vehicle; S102, extracting the target driver's eye muscle timing features, eyeball pupil timing features, and head tilt amplitude timing features based on the facial image timing data, and extracting the obstacle orientation timing features on the road ahead of the vehicle based on the road image timing data; S103, inputting the eye muscle timing characteristics, the eye pupil timing characteristics, the head tilt amplitude timing characteristics, and the obstacle orientation timing characteristics into a pre-trained collision risk prediction model to obtain a collision risk prediction value; S104: Determine a corresponding target air curtain deployment strategy according to the collision risk prediction value, and control the vehicle air curtain according to the target air curtain deployment strategy.

[0033] The embodiment of the present invention extracts eye muscle timing features, eye pupil timing features, and head tilt amplitude timing features based on the driver's facial image timing data, extracts obstacle orientation timing features based on the road image timing data in front of the vehicle, outputs a collision risk prediction value at the current moment based on a pre-trained collision risk prediction model, determines a corresponding air curtain deployment strategy based on the collision risk prediction value, and controls the vehicle air curtain according to the air curtain deployment strategy, so that the air curtain deployment can be controlled in a timely and accurate manner when the vehicle is about to collide, thereby improving the accuracy and timeliness of the air curtain deployment control, and also improving the safety of the vehicle and the user's driving experience.

[0034] As an optional embodiment, the target driver's eye muscle timing features, eye pupil timing features, and head tilt amplitude timing features are extracted based on the facial image timing data, which specifically includes: S1021, performing edge detection on each frame of facial image in the facial image time series data to obtain eye contours and head contours; S1022, extracting multiple frames of eye region images from the facial image time series data according to the eye contour; S1023, extracting periocular muscle texture features and determining eyelid opening and closing degrees based on the eye region image, performing time series processing on the periocular muscle texture features and eyelid opening and closing degrees corresponding to multiple consecutive frames of eye region images to obtain eye muscle time series features; S1024, performing binarization processing on the eye region image and detecting the pupil contour, determining the pupil position and pupil diameter based on the pupil contour, and performing time series processing on the pupil positions and pupil diameters corresponding to multiple frames of eye region images to obtain eyeball pupil time series features; S1025. Determine the corresponding head tilt amplitude according to the head contour, and determine the head tilt amplitude time series feature according to the head tilt amplitude corresponding to the multiple consecutive frames of facial images.

[0035] Specifically, the driver's facial image is captured in real time using an on-board camera (such as one in front of the steering wheel or on the dashboard), ensuring both lighting adaptability and angular coverage. Multiple receiving units (such as millimeter-wave radar or Bluetooth sensors) are used to locate the driver's position, dynamically adjusting the camera angle to ensure complete coverage of the entire head area. Edge detection and texture analysis are performed on the original image to segment the eye and head contours. A "neighborhood fit index" is iteratively calculated using a sliding window, merging highly similar regions to generate precise eye position coordinates. The resulting cropped eye region image (including structures such as the eyelid, iris, and pupil) is then output.

[0036] The feature extraction process is as follows: 1) Timing characteristics of eye muscles: Based on the eye area image, the periocular muscle texture features (such as changes in crow's feet wrinkles) are extracted and the eyelid opening and closing degree is analyzed. The periocular muscle texture features and eyelid opening and closing degrees corresponding to multiple consecutive frames of eye area images are time-series processed to obtain the eye muscle time-series features.

[0037] 2) Eye pupil timing characteristics: The eye region is binarized, and the pupil contour is detected using the findContours function of OpenCV. The convex hull of the contour is calculated, and the pupil center coordinates and pupil diameter are determined through the minimum circumscribed rectangle (minAreaRect). The changes in the pupil center coordinates and pupil diameter in consecutive frames are recorded to obtain the temporal characteristics of the eye pupil.

[0038] 3) Head tilt amplitude time series characteristics: The head tilt amplitude can be determined based on the head contour corresponding to the facial image taken from the front, and the head tilt amplitude can be determined based on the head contour corresponding to the facial image taken from the side. The head tilt amplitude can be obtained by combining the two. The head tilt amplitude corresponding to multiple consecutive frames of facial images can be used to determine the temporal characteristics of the head tilt amplitude.

[0039] As an optional embodiment, extracting the time series features of the obstacle orientation on the road ahead of the vehicle based on the road image time series data specifically includes: S1026, determining a plurality of consecutive frames of road image information according to the road image time series data; S1027: Detect obstacles on the road image information to obtain obstacle directions and distances, and determine obstacle orientation features based on the obstacle directions and distances. S1028: Generate obstacle orientation temporal features based on obstacle orientation features corresponding to multiple frames of road image information.

[0040] Specifically, multiple continuous road image information is determined based on the road image time series data. By inputting the road image information into the obstacle detection model, the direction of the obstacle relative to the vehicle and the distance from the vehicle can be obtained, thereby determining the obstacle orientation characteristics. The obstacle orientation characteristics corresponding to the multiple continuous road image information are time-series processed to generate the obstacle orientation time series characteristics.

[0041] As an optional implementation, the collision risk prediction model is trained by the following steps: S201, obtaining a time series sample of a driver's facial image and a time series sample of a road image in front of the vehicle in a historical driving scene; S202, extracting eye muscle temporal feature samples, eye pupil temporal feature samples, and head tilt amplitude temporal feature samples based on the facial image temporal sample, and extracting obstacle orientation temporal feature samples based on the road image temporal sample; S203: constructing training samples based on the eye muscle temporal feature samples, the eye pupil temporal feature samples, the head tilt amplitude temporal feature samples, and the obstacle orientation temporal feature samples, determining collision risk labels corresponding to the training samples through manual labeling, and constructing a training dataset based on the training samples and the collision risk labels; S204: Input the training data set into a pre-built CNN-LSTM hybrid neural network for training to obtain a trained collision risk prediction model.

[0042] Specifically, the driver's facial image time series samples and the road image time series samples in front of the vehicle under historical driving scenarios are obtained (with user authorization in advance); eye muscle time series feature samples, eye pupil time series feature samples and head tilt amplitude time series feature samples are extracted based on the facial image time series samples, and obstacle orientation time series feature samples are extracted based on the road image time series samples; manual analysis is used to determine whether the user in the sample has behaviors such as fatigue driving or inattention, and the collision risk is evaluated in combination with the road conditions ahead to obtain a collision risk label; after obtaining a sufficient amount of training data under different historical driving scenarios, a training dataset can be constructed; the training dataset is input into the pre-constructed CNN-LSTM hybrid neural network for training to obtain a trained collision risk prediction model.

[0043] Further as an optional implementation, the CNN-LSTM hybrid neural network includes an input layer, a CNN convolutional layer, a feature fusion layer, an LSTM layer, an attention layer and an output layer. The input layer is used to input training samples, the CNN convolutional layer is used to extract features from the training samples to obtain multiple local time series features, the feature fusion layer is used to fuse the local time series features to obtain fused time series features, the LSTM layer is used to generate a hidden state sequence based on the fused time series features, the attention layer is used to dynamically assign weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism, and the output layer is used to map the hidden state sequence after dynamic weight assignment to a collision risk prediction result.

[0044] like Figure 2 The figure shows a structural diagram of the CNN-LSTM hybrid neural network provided by an embodiment of the present invention. It can be seen that the embodiment of the present invention sets four CNN convolutional layers to extract local temporal features of the eye muscle temporal feature samples, the eye pupil temporal feature samples, the head tilt amplitude temporal feature samples and the obstacle orientation temporal feature samples respectively. The obtained eye muscle local features, eye pupil local features, head tilt amplitude local features and obstacle orientation local features will enter the feature fusion layer for feature fusion, and the final fused temporal features will be input into the LSTM layer for subsequent recognition.

[0045] As an optional implementation, the training data set is input into a pre-built CNN-LSTM hybrid neural network for training to obtain a trained collision risk prediction model, which specifically includes: S2041. Input training samples through the input layer; S2042, extracting features from the eye muscle temporal feature samples, the eye pupil temporal feature samples, the head tilt amplitude temporal feature samples, and the obstacle orientation temporal feature samples through a CNN convolutional layer to obtain eye muscle local features, eye pupil local features, head tilt amplitude local features, and obstacle orientation local features; S2043, performing feature fusion on the eye muscle local features, the eye pupil local features, the head tilt amplitude local features, and the obstacle orientation local features through a feature fusion layer to obtain a fused time series feature; S2044, generating a hidden state sequence based on the fused time series features through the LSTM layer; S2045. Dynamically assign weights to each dimension of the hidden state sequence based on the multi-head self-attention mechanism through the attention layer; S2046. Mapping the hidden state sequence after dynamic weight allocation to a collision risk prediction result through the output layer; S2047. Determine a loss value based on the collision risk prediction result and the corresponding collision risk label; S2048. Update the parameters of the CNN-LSTM hybrid neural network through the back propagation algorithm according to the loss value to obtain a trained collision risk prediction model.

[0046] Specifically, the fused time series features are input into the LSTM layer to generate a corresponding hidden state sequence, and dynamic weights are assigned to the various dimensions of the hidden state sequence based on the multi-head self-attention mechanism, and then mapped into a collision risk prediction result; a loss value is determined based on the difference between the collision risk prediction result and the collision risk label, and the selection of the loss function is not limited in this embodiment of the present invention; the parameters of the CNN-LSTM hybrid neural network are updated based on the loss value through the back propagation algorithm, and then the next round of iterative training begins. When the preset convergence condition is reached (the number of iterations reaches a threshold and the loss value is lower than the preset threshold), the training is stopped to obtain a trained collision risk prediction model.

[0047] As an optional embodiment, a target air curtain deployment strategy is determined according to the collision risk prediction value, and the vehicle air curtain is controlled according to the target air curtain deployment strategy, which specifically includes: S1041: triggering a match in a preset curtain airbag deployment strategy mapping table to obtain a corresponding target curtain airbag deployment strategy based on the collision risk prediction value; S1042: triggering a corresponding collision risk warning signal based on the collision risk prediction value; when the duration of the collision risk warning signal reaches a preset first threshold, determining a corresponding curtain airbag deployment area and curtain airbag inflation pressure based on a target curtain airbag deployment strategy; and controlling the vehicle curtain airbag based on the curtain airbag deployment area and curtain airbag inflation pressure; or, S1043. Obtain radar detection data in front of the vehicle, determine the current collision risk value based on the radar detection data, and when the difference between the current collision risk value and the collision risk prediction value is less than or equal to a preset second threshold, determine the corresponding air curtain deployment area and air curtain inflation pressure based on the target air curtain deployment strategy, and control the vehicle air curtain based on the air curtain deployment area and air curtain inflation pressure.

[0048] Specifically, the corresponding collision risk level can be determined according to the collision risk prediction value. For example, when the collision risk prediction value is greater than or equal to 0.7, it is determined to be high risk. When the collision risk prediction value is greater than or equal to 0.4 and less than 0.7, it is determined to be medium risk. When the collision risk is less than 0.4, it is determined to be low risk. According to different collision risk levels, the corresponding target air curtain deployment strategy is matched in the preset air curtain deployment strategy mapping table. For example, when the risk is high, the air curtain is deployed in the third level, the volume increases by 15%, and the inflation pressure increases by 10%. When the risk is medium, the air curtain is deployed at the baseline. When the risk is low, the air curtain is not deployed and only the seat belts are pre-tightened.

[0049] In addition, to prevent the air curtain from being triggered by mistake and affecting the driver's normal driving, the embodiment of the present invention is also provided with a false triggering mechanism: used to reduce the probability of false triggering of the system: 1) The collision risk warning signal is triggered based on the collision risk prediction value. When the collision risk warning signal duration reaches a preset first threshold, the curtain airbag is deployed. For example, the DMS signal must last for more than 300 milliseconds to trigger the curtain airbag deployment.

[0050] 2) The system acquires radar data from the vehicle ahead and determines the current collision risk based on this data. When the difference between the current collision risk value and the predicted collision risk value is less than or equal to a preset second threshold, the curtain airbag deployment is triggered. For example, deployment is triggered only when the difference between the two values ​​is no more than 25%. The accuracy of the triggering conditions is ensured by cross-validating the triggering conditions with millimeter-wave radar data.

[0051] It should be noted that the embodiment of the present invention provides a method for controlling the deployment of air curtains when a collision is about to occur. Even if the air curtain cannot be deployed in advance before the collision due to insufficient duration of the collision risk warning signal or failure to pass cross-validation with the millimeter-wave radar data, the air curtain will still be deployed according to the existing air curtain deployment strategy at the moment of the collision. That is to say, the embodiment of the present invention adds a function of triggering the deployment of air curtains in advance on the basis of the existing method of deploying air curtains at the moment of collision. When the corresponding conditions are met, it can be triggered in advance, thereby improving the timeliness of the air curtain deployment control.

[0052] In some optional embodiments, the embodiments of the present invention may also make structural improvements to the air curtain, for example: the structural layer is locally thickened by 1.2 mm and laser crush grooves are added, which increases the bending stiffness by 18%; the depth of the weakening line of the interior layer is controlled at 0.3±0.05 mm, and the tear force control accuracy reaches ±5N; the connection points adopt a buckle and crush groove spatial coupling design, which reduces the risk of abnormal noise by 90%.

[0053] The above describes the method steps of the embodiment of the present invention. It can be recognized that the embodiment of the present invention extracts the eye muscle timing features, eye pupil timing features, and head tilt amplitude timing features based on the driver's facial image timing data, extracts the obstacle orientation timing features based on the road image timing data in front of the vehicle, outputs the collision risk prediction value at the current moment based on the pre-trained collision risk prediction model, determines the corresponding air curtain deployment strategy based on the collision risk prediction value, and controls the vehicle air curtain according to the air curtain deployment strategy, so that the air curtain deployment can be controlled in a timely and accurate manner when the vehicle is about to collide, thereby improving the accuracy and timeliness of the air curtain deployment control, and also improving the safety of the vehicle and the user's driving experience.

[0054] Compared with the prior art, the embodiments of the present invention also have the following advantages: 1) Fast Response Time: Embodiments of this invention significantly shorten the air curtain system's response time, with a response delay of less than 50 milliseconds, significantly lower than the industry standard of 100 milliseconds. This improvement enables the system to react more quickly in emergencies, providing immediate protection and significantly improving occupant safety.

[0055] 2) Flexible Deployment Strategies: By incorporating a risk assessment algorithm based on DMS warning signals, the system can flexibly adjust the curtain airbag deployment strategy based on different risk levels. For example, in high-risk situations, the curtain airbags will adopt a three-stage deployment strategy, increasing their volume to provide enhanced protection; while in low-risk situations, only pre-tensioning the seatbelts will suffice. This flexibility not only improves system effectiveness but also reduces resource waste.

[0056] 3) Low false trigger rate: This embodiment of the present invention significantly reduces the probability of false triggers by implementing a dual safety mechanism, including a DMS signal duration threshold and cross-validation with millimeter-wave radar data. Test results show a false trigger rate of less than 0.1%, significantly improving system reliability and user experience.

[0057] 4) Precise Hardware Innovation: At the hardware level, this embodiment utilizes localized thickening and laser-incorporated crush grooves to enhance flexural rigidity. Precise control of the weakening line depth enables precise management of tearing forces. Furthermore, the spatial coupling between the buckle and the crush groove effectively reduces the risk of abnormal noise, further enhancing system stability and safety.

[0058] Reference Figure 3 , an embodiment of the present invention provides an air curtain control device based on driver status monitoring, comprising: A data acquisition module is used to acquire facial image time series data of the target driver and road image time series data in front of the vehicle; A feature extraction module is used to extract the target driver's eye muscle timing features, eye pupil timing features, and head tilt amplitude timing features based on the facial image timing data, and to extract the obstacle orientation timing features on the road ahead of the vehicle based on the road image timing data; The collision risk prediction module is used to input the eye muscle timing characteristics, eye pupil timing characteristics, head tilt amplitude timing characteristics, and obstacle orientation timing characteristics into a pre-trained collision risk prediction model to obtain a collision risk prediction value; The air curtain deployment control module is used to determine the corresponding target air curtain deployment strategy based on the collision risk prediction value, and control the vehicle air curtain according to the target air curtain deployment strategy.

[0059] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0060] Reference Figure 4 , an embodiment of the present invention provides an electronic device, including: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned air curtain control method based on driver state monitoring.

[0061] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0062] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program executable by a processor. When the computer program is executed by the processor, the computer program implements the above-mentioned air curtain control method based on driver status monitoring.

[0063] A computer-readable storage medium according to an embodiment of the present invention can execute an air curtain control method based on driver status monitoring provided by an embodiment of the method of the present invention, can execute any combination of implementation steps of the embodiment of the method, and has the corresponding functions and beneficial effects of the method.

[0064] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the above-mentioned air curtain control method based on driver state monitoring when executed by a processor.

[0065] It is understandable that the contents of the above method embodiments are all applicable to the present program product embodiments, the functions specifically implemented by the present program product embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0066] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0067] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0068] The terms "first," "second," "third," "fourth," and the like (if any) in the description of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or apparatus.

[0069] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0070] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present invention set forth in the claims using ordinary skills without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0071] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the above methods in various embodiments of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0072] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0073] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable media on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other media, and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0074] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0075] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0076] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0077] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A curtain airbag control method based on driver status monitoring, characterized in that: The following steps are involved: Obtaining facial image time series data of the target driver and road image time series data in front of the vehicle; Extracting the target driver's eye muscle timing features, eyeball pupil timing features, and head tilt amplitude timing features based on the facial image timing data, and extracting the obstacle orientation timing features on the road ahead of the vehicle based on the road image timing data; Inputting the eye muscle timing characteristics, the eyeball pupil timing characteristics, the head tilt amplitude timing characteristics, and the obstacle orientation timing characteristics into a pre-trained collision risk prediction model to obtain a collision risk prediction value; A corresponding target air curtain deployment strategy is determined according to the collision risk prediction value, and the vehicle air curtain is controlled according to the target air curtain deployment strategy.

2. The air curtain control method based on driver status monitoring according to claim 1, characterized in that: The step of extracting the target driver's eye muscle timing features, eyeball pupil timing features, and head tilt amplitude timing features based on the facial image timing data specifically includes: Performing edge detection on each frame of facial image in the facial image time series data to obtain eye contours and head contours; Extracting a plurality of consecutive frames of eye region images from the facial image time series data according to the eye contour; Extracting periocular muscle texture features and determining eyelid opening and closing degrees based on the eye region image, and performing time series processing on the periocular muscle texture features and the eyelid opening and closing degrees corresponding to a plurality of consecutive frames of the eye region image to obtain the eye muscle time series features; Binarizing the eye region image and detecting the pupil contour, determining the pupil position and pupil diameter based on the pupil contour, and performing time series processing on the pupil positions and pupil diameters corresponding to a plurality of consecutive frames of the eye region image to obtain the eyeball pupil time series feature; The corresponding head tilt amplitude is determined according to the head contour, and the head tilt amplitude time series feature is determined according to the head tilt amplitude corresponding to a plurality of consecutive frames of the facial images.

3. The air curtain control method based on driver status monitoring according to claim 1, characterized in that: The step of extracting the obstacle orientation time series features on the road ahead of the vehicle based on the road image time series data specifically includes: determining a plurality of consecutive frames of road image information according to the road image time series data; performing obstacle detection on the road image information to obtain an obstacle direction and an obstacle distance, and determining an obstacle orientation feature based on the obstacle direction and the obstacle distance; The obstacle orientation time series feature is generated according to the obstacle orientation features corresponding to multiple consecutive frames of road image information.

4. The air curtain control method based on driver status monitoring according to claim 1, characterized in that: The collision risk prediction model is trained by the following steps: Obtaining time series samples of the driver's facial image and the road image in front of the vehicle in historical driving scenes; Extracting eye muscle temporal feature samples, eye pupil temporal feature samples, and head tilt amplitude temporal feature samples based on the facial image temporal sample, and extracting obstacle orientation temporal feature samples based on the road image temporal sample; Constructing training samples based on the eye muscle temporal feature samples, the eye pupil temporal feature samples, the head tilt amplitude temporal feature samples, and the obstacle orientation temporal feature samples, determining collision risk labels corresponding to the training samples through manual labeling, and constructing a training data set based on the training samples and the collision risk labels; The training data set is input into a pre-built CNN-LSTM hybrid neural network for training to obtain the trained collision risk prediction model.

5. The air curtain control method based on driver status monitoring according to claim 4, characterized in that: The CNN-LSTM hybrid neural network includes an input layer, a CNN convolutional layer, a feature fusion layer, an LSTM layer, an attention layer and an output layer. The input layer is used to input the training sample. The CNN convolutional layer is used to extract features from the training sample to obtain multiple local time series features. The feature fusion layer is used to fuse the local time series features to obtain fused time series features. The LSTM layer is used to generate a hidden state sequence based on the fused time series features. The attention layer is used to dynamically assign weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism. The output layer is used to map the hidden state sequence after dynamic weight assignment into a collision risk prediction result.

6. The air curtain control method based on driver status monitoring according to claim 5, characterized in that: Inputting the training data set into a pre-built CNN-LSTM hybrid neural network for training to obtain the trained collision risk prediction model specifically includes: Input the training sample through the input layer; The CNN convolutional layer extracts the eye muscle temporal feature samples, the eye pupil temporal feature samples, the head tilt amplitude temporal feature samples, and the obstacle orientation temporal feature samples to obtain eye muscle local features, eye pupil local features, head tilt amplitude local features, and obstacle orientation local features; The feature fusion layer performs feature fusion on the eye muscle local feature, the eye pupil local feature, the head tilt amplitude local feature, and the obstacle orientation local feature to obtain a fused time series feature; Generate a hidden state sequence according to the fused time series features through the LSTM layer; Dynamically weighting each dimension of the hidden state sequence based on a multi-head self-attention mechanism through the attention layer; Mapping the hidden state sequence after dynamic weight allocation into a collision risk prediction result through the output layer; Determine a loss value according to the collision risk prediction result and the corresponding collision risk label; The parameters of the CNN-LSTM hybrid neural network are updated according to the loss value through a back propagation algorithm to obtain the trained collision risk prediction model.

7. The air curtain control method based on driver status monitoring according to any one of claims 1 to 6, characterized in that: The step of determining a corresponding target air curtain deployment strategy according to the collision risk prediction value, and controlling the vehicle air curtain according to the target air curtain deployment strategy, specifically includes: According to the collision risk prediction value, a matching is triggered in a preset air curtain deployment strategy mapping table to obtain the corresponding target air curtain deployment strategy; triggering a corresponding collision risk warning signal according to the collision risk prediction value; determining a corresponding curtain airbag deployment area and curtain airbag inflation pressure according to the target curtain airbag deployment strategy when the duration of the collision risk warning signal reaches a preset first threshold; and controlling the vehicle curtain airbag according to the curtain airbag deployment area and the curtain airbag inflation pressure; or, Acquire radar detection data in front of the vehicle, determine a current collision risk value based on the radar detection data, and when the difference rate between the current collision risk value and the collision risk prediction value is less than or equal to a preset second threshold, determine the corresponding air curtain deployment area and air curtain inflation pressure based on the target air curtain deployment strategy, and control the vehicle air curtain based on the air curtain deployment area and the air curtain inflation pressure.

8. An air curtain control device based on driver status monitoring, characterized in that: include: A data acquisition module is used to acquire facial image time series data of the target driver and road image time series data in front of the vehicle; a feature extraction module, configured to extract the target driver's eye muscle timing features, eye pupil timing features, and head tilt amplitude timing features based on the facial image timing data, and extract the obstacle orientation timing features on the road ahead of the vehicle based on the road image timing data; a collision risk prediction module, configured to input the eye muscle timing characteristics, the eye pupil timing characteristics, the head tilt amplitude timing characteristics, and the obstacle orientation timing characteristics into a pre-trained collision risk prediction model to obtain a collision risk prediction value; The air curtain deployment control module is used to determine a corresponding target air curtain deployment strategy according to the collision risk prediction value, and control the vehicle air curtain according to the target air curtain deployment strategy.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the air curtain control method based on driver state monitoring as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, an air curtain control method based on driver state monitoring according to any one of claims 1 to 7 is implemented.

Images