Vehicle control method, vehicle, and computer-readable storage medium

Abstract

The embodiment of the application provides a kind of vehicle control method, vehicle and computer readable storage medium, it is related to vehicle control technical field.Therein, vehicle control method includes: obtaining multiple target images in preset time window, wherein, multiple target images are used to describe driver behavior;Using pre-trained distraction detection model, multiple target images are handled, and multiple behavior recognition results are obtained;Based on multiple behavior recognition results, determine target behavior, wherein, target behavior is used to represent the driver behavior with time sequence consistency in preset time window;Based on target behavior, determine driver distraction risk level;Based on driver distraction risk level, determine vehicle control strategy, and according to vehicle control strategy, vehicle control is carried out.The application solves the technical problem that the driver distraction behavior detection accuracy is low in the related art, which leads to inaccurate vehicle control and affects driving safety.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and more specifically, to a vehicle control method, a vehicle, and a computer-readable storage medium. Background Technology

[0002] Driver distraction detection is one of the core safety functions in intelligent cockpits and advanced driver assistance systems (ADAS), and its accuracy is directly related to driving safety and life protection. Current mainstream driver distraction detection methods are mostly based on deep convolutional neural networks, which classify behavior by extracting local features from the driver's face and limbs.

[0003] However, in real-world driving environments, existing detection methods struggle to effectively suppress strong interference sources such as window reflections, dashboard textures, swaying ornaments, and alternating light and dark conditions. These methods easily misinterpret background motion as distracting actions, leading to numerous false alarms. Furthermore, existing detection methods are insufficient in recognizing complex distractions (such as "operating the central control screen while turning one's head"), resulting in inaccurate risk assessments.

[0004] There is currently no good solution to the above problems. Summary of the Invention

[0005] This application provides a vehicle control method, a vehicle, and a computer-readable storage medium to at least solve the technical problem in the related art where the low accuracy of driver distraction detection leads to inaccurate vehicle control and thus affects driving safety.

[0006] According to one aspect of the embodiments of this application, a vehicle control method is provided, comprising: acquiring multiple target images within a preset time window, wherein the multiple target images are used to describe driver behavior; processing the multiple target images using a pre-trained distraction detection model to obtain multiple behavior recognition results, wherein the pre-trained distraction detection model includes: a first feature extraction subnetwork, a second feature extraction subnetwork, a feature fusion subnetwork, and a behavior classification subnetwork; for any target image among the multiple target images, the first feature extraction subnetwork is used to extract local features of any target image to obtain a first feature extraction result, and the second feature extraction subnetwork is used to extract features of any target image. The global features are used to obtain the second feature extraction result. The feature fusion subnetwork is used to adaptively weight and fuse the first and second feature extraction results to obtain the fusion result. The behavior classification subnetwork is used to classify the fusion result to obtain the behavior recognition result corresponding to any target image. Multiple target images correspond one-to-one with multiple behavior recognition results. Based on multiple behavior recognition results, the target behavior is determined, where the target behavior is used to characterize the driver behavior with temporal consistency within a preset time window. Based on the target behavior, the driver distraction risk level is determined. Based on the driver distraction risk level, the vehicle control strategy is determined, and the vehicle is controlled according to the vehicle control strategy.

[0007] Furthermore, the first feature extraction subnetwork includes: a convolutional neural network based on a residual structure, wherein different attention modules are deployed in multiple residual block groups of the convolutional neural network.

[0008] Furthermore, the second feature extraction sub-network includes a location encoding module and a relation encoder, wherein the location encoding module is used to add location encoding, and the relation encoder is used to extract the global semantic relations of any target image based on the processing results of the location encoding module.

[0009] Furthermore, the feature fusion subnetwork includes a gating network, which is configured to: concatenate the first feature extraction result and the second feature extraction result to obtain a joint feature extraction result; process the joint feature extraction result using the gating network to obtain a weight allocation result, wherein the weight allocation result is used to characterize the relative importance of the first feature extraction result and the second feature extraction result; and perform weighted fusion of the first feature extraction result and the second feature extraction result based on the weight allocation result to obtain a fusion result.

[0010] Furthermore, based on multiple behavior recognition results, the target behavior is determined, including: performing statistics on multiple behavior recognition results to obtain statistical results, wherein the statistical results are used to describe the frequency of occurrence of multiple driver behaviors corresponding to the multiple behavior recognition results respectively; and determining the target behavior based on the statistical results.

[0011] Furthermore, based on the target behavior, the driver's distraction risk level is determined, including: obtaining the duration of the target behavior and a preset risk baseline; and based on the duration and the preset risk baseline, determining the driver's distraction risk level corresponding to the target behavior.

[0012] Furthermore, the driver distraction risk levels include: a first risk level, a second risk level, a third risk level, and a fourth risk level. The driving risk corresponding to the first risk level is lower than that corresponding to the second risk level, the second risk level is lower than that corresponding to the third risk level, and the third risk level is lower than that corresponding to the fourth risk level. Based on the driver distraction risk level, a vehicle control strategy is determined, including: in response to the driver distraction risk level being the first risk level, the vehicle control strategy includes triggering visual cues; in response to the driver distraction risk level being the second risk level, the vehicle control strategy includes triggering auditory cues; in response to the driver distraction risk level being the third risk level, the vehicle control strategy includes triggering both auditory and vibration cues; and in response to the driver distraction risk level being the fourth risk level, the vehicle control strategy includes triggering visual cues, auditory cues, vibration cues, and sending an assistance request to the vehicle's advanced driver assistance system.

[0013] Further, acquiring multiple target images within a preset time window includes: acquiring multiple initial images within the preset time window, wherein the multiple initial images are used to characterize the images to be processed collected by the vehicle-mounted camera to describe the driver's behavior; and performing region extraction and size normalization and standardization on the multiple initial images to obtain multiple target images.

[0014] According to another aspect of the embodiments of this application, a vehicle control system is also provided, including: an acquisition module, configured to acquire multiple target images within a preset time window, wherein the multiple target images are used to describe driver behavior; and a processing module, configured to process the multiple target images using a pre-trained distraction detection model to obtain multiple behavior recognition results, wherein the pre-trained distraction detection model includes: a first feature extraction subnetwork, a second feature extraction subnetwork, a feature fusion subnetwork, and a behavior classification subnetwork. For any target image among the multiple target images, the first feature extraction subnetwork is used to extract local features of any target image to obtain a first feature extraction result, and the second feature extraction subnetwork is used to extract global features of any target image. The system obtains a second feature extraction result. A feature fusion subnetwork is used to adaptively weight and fuse the first and second feature extraction results to obtain a fusion result. A behavior classification subnetwork is used to classify the fusion result to obtain a behavior recognition result corresponding to any target image. Multiple target images correspond one-to-one with multiple behavior recognition results. A first determination module is used to determine the target behavior based on multiple behavior recognition results. The target behavior is used to characterize driver behavior with temporal consistency within a preset time window. A second determination module is used to determine the driver distraction risk level based on the target behavior. A third determination module is used to determine the vehicle control strategy based on the driver distraction risk level and perform vehicle control according to the vehicle control strategy.

[0015] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the executable program, wherein the executable program executes the methods in various embodiments of this application when it runs.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0017] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0018] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0019] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the methods in various embodiments of this application.

[0020] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0021] In this embodiment, multiple target images within a preset time window are acquired, wherein the multiple target images are used to describe driver behavior; a pre-trained distraction detection model is used to process the multiple target images to obtain multiple behavior recognition results. The pre-trained distraction detection model includes: a first feature extraction subnetwork, a second feature extraction subnetwork, a feature fusion subnetwork, and a behavior classification subnetwork. For any target image among the multiple target images, the first feature extraction subnetwork is used to extract local features of any target image to obtain a first feature extraction result; the second feature extraction subnetwork is used to extract global features of any target image to obtain a second feature classification result. The feature extraction results are processed by a feature fusion subnetwork, which adaptively weights and fuses the first and second feature extraction results to obtain a fusion result. A behavior classification subnetwork is used to classify the fusion result into behaviors, obtaining a behavior recognition result corresponding to any target image. Multiple target images correspond one-to-one with multiple behavior recognition results. Based on multiple behavior recognition results, target behaviors are determined, whereby the target behaviors characterize driver behaviors with temporal consistency within a preset time window. Based on the target behaviors, the driver distraction risk level is determined. Based on the driver distraction risk level, a vehicle control strategy is determined, and vehicle control is performed according to the vehicle control strategy. This application first acquires multiple target images within a preset time window and inputs them into a pre-trained distraction detection model, achieving continuous and temporal perception of driver behavior, thus avoiding the problem of misjudgment caused by instantaneous interference in single-frame detection. Secondly, the pre-trained distraction detection model comprises multiple sub-networks: a first feature extraction sub-network, a second feature extraction sub-network, a feature fusion sub-network, and a behavior classification sub-network. The first feature extraction sub-network extracts local features from multiple target images, while the second feature extraction sub-network extracts global features. The collaboration between these two sub-networks overcomes the shortcomings of traditional methods that rely solely on single-scale features, significantly improving the completeness of driver behavior feature representation. The feature fusion sub-network adaptively weights and fuses the identified local and global features, dynamically adjusting the contribution ratio of local and global information. This effectively addresses the issues of semantic separation between local and global features in existing technologies, as well as the inability of fixed fusion weights to adapt to complex scenarios, enhancing the robustness of the distraction detection model to the changing in-vehicle environment. Simultaneously, the behavior classification sub-network outputs the behavior recognition result for each target image based on the fusion result. Based on multiple behavior recognition results, driver behaviors with temporal consistency within a preset time window are identified, i.e., continuous distraction behaviors are used as target behaviors to avoid transient interference. Furthermore, based on the target behavior, the driver's distraction risk level is determined, and the corresponding vehicle control strategy is determined according to the driver's distraction risk level, thereby achieving effective risk assessment and precise vehicle control.Therefore, this application achieves the technical effect of improving the accuracy of driver distraction detection, thereby enabling precise vehicle control to ensure driving safety, and solves the technical problem in related technologies where low accuracy of driver distraction detection leads to inaccurate vehicle control and thus affects driving safety. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0023] Figure 1 This is a flowchart of a vehicle control method according to an embodiment of this application;

[0024] Figure 2 This is a flowchart of another vehicle control method according to an embodiment of this application;

[0025] Figure 3 This is an implementation architecture diagram of a vehicle control system according to an embodiment of this application;

[0026] Figure 4 This is a structural block diagram of a vehicle control system according to an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] According to an embodiment of this application, an embodiment of a vehicle control method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] This embodiment provides a vehicle control method. Figure 1 This is a flowchart of a vehicle control method according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0032] Step S10: Acquire multiple target images within a preset time window, wherein the multiple target images are used to describe the driver's behavior;

[0033] Step S11: A pre-trained distraction detection model is used to process multiple target images to obtain multiple behavior recognition results. The pre-trained distraction detection model includes: a first feature extraction sub-network, a second feature extraction sub-network, a feature fusion sub-network, and a behavior classification sub-network. For any target image among the multiple target images, the first feature extraction sub-network is used to extract local features of any target image to obtain a first feature extraction result. The second feature extraction sub-network is used to extract global features of any target image to obtain a second feature extraction result. The feature fusion sub-network is used to adaptively weight and fuse the first feature extraction result and the second feature extraction result to obtain a fusion result. The behavior classification sub-network is used to classify the fusion result to obtain the behavior recognition result corresponding to any target image. Multiple target images correspond one-to-one with multiple behavior recognition results.

[0034] Step S12: Based on multiple behavior recognition results, determine the target behavior, wherein the target behavior is used to characterize the driver behavior that has temporal consistency within a preset time window;

[0035] Step S13: Determine the driver's distraction risk level based on the target behavior;

[0036] Step S14: Determine the vehicle control strategy based on the driver's distraction risk level, and control the vehicle according to the vehicle control strategy.

[0037] The aforementioned preset time window refers to a fixed-length video capture interval set during vehicle operation to capture the continuity and dynamic evolution of driver behavior.

[0038] Optionally, the preset time window is set to 0.5 to 2 seconds, corresponding to 30 to 60 frames of images.

[0039] Optionally, the preset time window is designed based on the fact that typical human distraction behaviors (such as operating a mobile phone or talking to passengers) mostly last for more than 500 milliseconds, while non-risk actions such as brief blinking and slight head movements usually last less than 200 milliseconds. Therefore, the preset time window needs to cover the minimum effective duration of actual distraction behaviors, while avoiding warning delays due to excessively long windows.

[0040] The aforementioned target images refer to a subset of images specifically used for driver state analysis, obtained from the original video frames captured by the vehicle-mounted driver monitoring camera after cropping and standardization of the Region of Interest (ROI).

[0041] Optionally, the target image covers the driver's face, neck, shoulders, and upper arm movement areas. Spatial cropping effectively eliminates interfering information such as dashboard, window reflections, ornaments, and external scenery, ensuring that the subsequent model focuses on behavior-related visual features.

[0042] In one optional embodiment, RGB images of the area directly in front of the driver are continuously acquired at a frequency of 30 frames per second. Each frame is preprocessed to obtain multiple target images. Furthermore, a sliding window mechanism is employed to form a dynamic sequence of the most recent T frames (e.g., T=15) of target images. Each new frame replaces the oldest frame, creating a continuously updated preset time window data stream that provides temporal input for subsequent behavior recognition.

[0043] Optionally, to cope with extreme lighting environments such as tunnel entry and exit, low light at night, and backlight, an adaptive brightness compensation algorithm or infrared-assisted imaging can be used to ensure that the target image still has clear facial and hand textures under different lighting conditions.

[0044] The aforementioned first feature extraction subnetwork refers to a convolutional neural network architecture employing hierarchical attention enhancement. This first feature extraction subnetwork is used to extract fine-grained local spatial features from images, such as eyelid opening and closing, hand position, and minute head tilt angles.

[0045] The aforementioned second feature extraction subnetwork refers to a network architecture centered on a lightweight visual Transformer encoder. The function of the second feature extraction subnetwork is to model the semantic dependencies between non-local regions in an image, such as the collaborative relationship between "right hand on the central control screen" and "eyes looking away from the road."

[0046] The aforementioned feature fusion subnetwork refers to an adaptive weighting module based on a gating mechanism. The role of the feature fusion subnetwork is to dynamically balance the contribution ratios of local and global features, avoiding information conflicts or redundancy caused by simple concatenation or average fusion.

[0047] The aforementioned behavior classification subnetwork refers to a multilayer perceptron network composed of fully connected layers, ReLU activation, and Dropout (a widely used regularization technique mainly used to prevent overfitting of neural networks during training). It is used to map the fused feature vector (i.e. the fusion result) to a probability distribution of predefined behavior categories and output the behavior label corresponding to the highest probability.

[0048] The aforementioned target behavior refers to the stable and continuous driver behavior category extracted by performing temporal analysis on the behavior recognition results of multiple consecutive frames within a preset time window.

[0049] The purpose of identifying the target behavior is to confirm the "true intentional behavior" after filtering out transient noise, rather than to identify it as an accidental misjudgment in a single frame. The target behavior must meet the condition of temporal consistency, that is, it must remain of the same category within a certain time range, thereby distinguishing it from interfering actions such as blinking, brief head turning, and screen reflection.

[0050] In one optional embodiment, the frequency of occurrence of each behavior category within a preset time window is counted, and the category with the most occurrences is selected as the dominant behavior label of the current time window, i.e., the target behavior.

[0051] Optionally, if single-frame classification results are used directly for risk assessment, they are susceptible to false alarms due to random interference such as image blurring, lighting flicker, and rapid blinking. By using temporal smoothing and consistency detection, real distraction behaviors such as "continuous mobile phone operation" and "prolonged head-turning conversation" can be effectively identified, significantly reducing the false alarm rate, making the warnings closer to the intuitive judgment of human drivers about dangerous behaviors, and improving the system's reliability.

[0052] The aforementioned driver distraction risk level refers to a quantitative safety threat index calculated based on the inherent dangerous attributes of the target behavior and its duration. The driver distraction risk level is used to guide the selection of the intensity of subsequent intervention strategies.

[0053] In one alternative embodiment, the driver distraction risk level is divided into four levels: low risk, medium risk, high risk, and emergency risk.

[0054] Optionally, based on the identified target behavior, a preset inherent risk baseline value is assigned. For example, "operating a mobile phone" is high risk (base value 4), "talking to passengers" is medium risk (base value 2), and "adjusting the rearview mirror" is low risk (base value 1). The duration of the target behavior is continuously accumulated, and the risk level is calculated using a non-linear function. For example, the risk level increases by one level when the duration exceeds 3 seconds, and by another level when it exceeds 6 seconds. In addition, to avoid frequent fluctuations in the level, a "state maintenance" mechanism is introduced: the risk level is only updated when the target behavior is stably sustained for more than 2 seconds; if the behavior is interrupted, the level gradually decays.

[0055] The aforementioned vehicle control strategy refers to a tiered response mechanism triggered based on the current level of driver distraction risk, encompassing human-machine interaction prompts, multimodal sensory warnings, and coordinated intervention from Advanced Driver Assistance Systems (ADAS). The vehicle control strategy adheres to the principle of "gradual and minimal intervention," ensuring safety while avoiding driver fatigue and aversion.

[0056] The aforementioned vehicle control refers to sending commands to the human-machine interface, audio system, seat vibration module, and ADAS controller via the vehicle communication bus to achieve active intervention in the vehicle's internal and external systems.

[0057] In one optional implementation, a four-level vehicle control strategy is determined based on the driver's distraction risk level. For example, when the driver's distraction risk level is low, a light-colored icon is used to indicate the current behavioral status on the digital instrument panel or vehicle head-up display (e.g., displaying "Attention: Operating the central control screen"), without interfering with driving. When the driver's distraction risk level is medium, a short (e.g., 0.5 seconds) warning sound (e.g., a "beep") is triggered, and the instrument panel icon flashes yellow. When the driver's distraction risk level is high, a continuous low-frequency warning sound (e.g., a 2-second loop) is activated, and vibration motors on both sides of the driver's seat are activated to generate 20Hz pulse vibrations, forming a multi-channel coordinated warning (visual, auditory, and tactile). When the driver's distraction risk level is critical, in addition to the above, a "severe driver inattention" signal is sent to the ADAS controller via the vehicle communication bus, triggering the lane keeping assist system to increase its correction efforts, or a "ready to intervene" warning is sent to the Automatic Emergency Braking (AEB) system. If there is no improvement in behavior within the next 2 seconds, deceleration assist is activated.

[0058] It should be noted that all control commands must meet the vehicle driving status verification (such as vehicle speed > 5km / h) before they can be executed to prevent accidental triggering when the vehicle is parked.

[0059] In this embodiment, multiple target images within a preset time window are acquired, wherein the multiple target images are used to describe driver behavior; a pre-trained distraction detection model is used to process the multiple target images to obtain multiple behavior recognition results. The pre-trained distraction detection model includes: a first feature extraction subnetwork, a second feature extraction subnetwork, a feature fusion subnetwork, and a behavior classification subnetwork. For any target image among the multiple target images, the first feature extraction subnetwork is used to extract local features of any target image to obtain a first feature extraction result; the second feature extraction subnetwork is used to extract global features of any target image to obtain a second feature classification result. The feature extraction results are processed by a feature fusion subnetwork, which adaptively weights and fuses the first and second feature extraction results to obtain a fusion result. A behavior classification subnetwork is used to classify the fusion result into behaviors, obtaining a behavior recognition result corresponding to any target image. Multiple target images correspond one-to-one with multiple behavior recognition results. Based on multiple behavior recognition results, target behaviors are determined, whereby the target behaviors characterize driver behaviors with temporal consistency within a preset time window. Based on the target behaviors, the driver distraction risk level is determined. Based on the driver distraction risk level, a vehicle control strategy is determined, and vehicle control is performed according to the vehicle control strategy. This application first acquires multiple target images within a preset time window and inputs them into a pre-trained distraction detection model, achieving continuous and temporal perception of driver behavior, thus avoiding the problem of misjudgment caused by instantaneous interference in single-frame detection. Secondly, the pre-trained distraction detection model comprises multiple sub-networks: a first feature extraction sub-network, a second feature extraction sub-network, a feature fusion sub-network, and a behavior classification sub-network. The first feature extraction sub-network extracts local features from multiple target images, while the second feature extraction sub-network extracts global features. Together, they overcome the shortcomings of traditional methods that rely solely on single-scale features, significantly improving the completeness of driver behavior feature representation. The feature fusion sub-network adaptively weights and fuses the identified local and global features, dynamically adjusting the contribution ratio of local and global information. This effectively addresses the issues of semantic separation between local and global features in existing technologies, as well as the inability of fixed fusion weights to adapt to complex scenarios, enhancing the robustness of the distraction detection model to the changing in-vehicle environment. Simultaneously, the behavior classification sub-network outputs the behavior recognition result for each target image based on the fusion result. Based on multiple behavior recognition results, driver behaviors with temporal consistency within a preset time window are identified, i.e., continuous distraction behaviors are used as target behaviors to avoid instantaneous interference. Furthermore, based on the target behavior, the driver's distraction risk level is determined, and the corresponding vehicle control strategy is determined according to the driver's distraction risk level, thereby achieving effective risk assessment and precise vehicle control.Therefore, this application achieves the technical effect of improving the accuracy of driver distraction detection, thereby enabling precise vehicle control to ensure driving safety, and solves the technical problem in related technologies where low accuracy of driver distraction detection leads to inaccurate vehicle control and thus affects driving safety.

[0060] The vehicle control method in the embodiments of this application will be further described below.

[0061] Optionally, the first feature extraction subnetwork includes: a convolutional neural network based on a residual structure, wherein different attention modules are deployed in multiple residual block groups of the convolutional neural network.

[0062] Optionally, the first feature extraction subnetwork uses a convolutional neural network with a residual structure as its backbone. Optionally, an optimized ResNet-50 (a residual neural network with 50 layers) will be used as an example for explanation.

[0063] The key feature of ResNet-50 lies in the design of "Residual Blocks." Each residual block consists of two or three convolutional layers, with "Skip Connections" introduced in between. This means the input feature map is directly added element-wise to the output feature map after convolution. This mechanism solves the performance degradation problem caused by gradient vanishing during deep network training, allowing the network to be trained stably to deeper layers and extract richer semantic information. In the embodiments of this application, ResNet-50 is divided into four stages (Stage 1 to Stage 4), each containing several residual blocks. Each stage corresponds to a different spatial resolution and semantic abstraction level, providing a structural basis for the differentiated deployment of subsequent attention modules.

[0064] Optionally, in Stage 2 (corresponding to a feature map size of 56×56), a Convolutional Block Attention Module (CBAM) is deployed. CBAM consists of two sub-modules connected in series: a Channel Attention Module and a Spatial Attention Module.

[0065] The channel attention module generates two channel description vectors through global average pooling and global max pooling, respectively. These vectors are then fused through a shared fully connected layer and a ReLU activation function to finally output a channel weight vector. This vector is used to weight each channel of the original feature map, thereby enhancing the response to discriminative channels (such as face and hand textures) and suppressing irrelevant channels (such as dashboard background textures).

[0066] The spatial attention module obtains two two-dimensional spatial masks by performing average pooling and max pooling along the channel dimension, respectively. After being stitched together, they are passed through a 7×7 convolutional layer to generate a spatial weight map. This map can accurately focus on the most discriminative spatial regions in the image, such as the driver's eyes, palms, or head deflection areas.

[0067] Deploying CBAM in Stage 2 aims to suppress complex in-vehicle background interference, such as window reflections, swaying of suspended objects, and center console textures, in the early stages of feature extraction, achieving initial target focusing through "localization first, enhancement later".

[0068] Optionally, in Stage 3 (feature map size 28×28), an alternating integration strategy is adopted, deploying the Bottleneck Attention Module (BAM) in some residual blocks and the Efficient Pyramid Split Attention Module (EPSA) in another part of the residual blocks.

[0069] BAM is an improved form of CBAM. It introduces a "bottleneck structure" in the channel attention module and spatial attention module, that is, using 1×1 convolution to reduce dimensionality and then restore it, thereby reducing computational overhead while maintaining high perceptual ability. The advantage of BAM lies in its more refined feature recalibration capability, especially suitable for features at the medium semantic level, such as driver shoulder contours and arm poses.

[0070] The EPSA module is designed to solve the problem of multi-scale semantic fusion. The key to EPSA is dividing the input feature map equally along the channel dimension, forming multiple sub-channel groups. Each group is processed by a 3×3 convolution with a different dilation rate. For example, one group uses a convolution with a dilation rate of 1 (receptive field of 3×3), another uses a convolution with a dilation rate of 2 (receptive field of 7×7), and a third uses a convolution with a dilation rate of 3 (receptive field of 11×11). This "pyramid-like" receptive field expansion mechanism allows the model to simultaneously capture local details (such as eyelid micro-movements) and medium-range context (such as the spatial relationship between the hand and the central control screen). Subsequently, all sub-group features are concatenated and then fused through a 1×1 convolution. Finally, a lightweight channel attention mechanism (such as channel squeezing and excitation) performs a weighted summation, thus achieving a closed loop of "multi-scale perception-adaptive fusion".

[0071] The deployment of EPSA in Stage 3 enables the network to understand complex actions at multiple scales in the middle semantic stage, overcoming the problem of poor scale adaptability caused by traditional convolutional neural networks that can only rely on fixed convolutional kernels.

[0072] Optionally, in Stage 4 (feature map size 14×14), a Squeeze-and-Excitation (SE) module is deployed. The SE module has a minimalist structure: first, it compresses the two-dimensional features of each channel into a scalar through Global Average Pooling (GAP), forming a channel statistical vector; then, it learns the non-linear dependencies between channels through two fully connected layers (with ReLU activation in between); finally, it outputs channel weights through the Sigmoid function to perform channel-level reweighting on the original feature map. The SE module does not introduce additional spatial information processing and focuses on selecting channel responses with high discriminative power from high-level semantic features, such as abstract behavioral representations like "gaze deviation" and "head rotation".

[0073] The SE module is deployed in Stage 4 because the feature maps at this stage have highly abstract semantics and low spatial resolution, making spatial attention unnecessary, while channel-level feature discrimination is crucial for behavior classification. The lightweight nature of SE also avoids introducing too many parameters into deep networks, meeting the efficiency constraints of automotive embedded deployments.

[0074] Optionally, the ResNet-50 can be modified to obtain the first feature extraction sub-network, as follows: For Stage 1, the original structure is maintained (7×7 convolutions, stride of 2, followed by max pooling), and the output feature map size is 112×112. For Stage 2, this stage contains 3 residual blocks. After the 3×3 convolutional layer of each residual block, a CBAM module is inserted. CBAM performs channel attention and spatial attention calculations sequentially to recalibrate the features. In this shallow layer, the spatial attention weight map can significantly improve the response of potential target regions such as faces and hands, while suppressing the background. For Stage 3, this stage contains 4 residual blocks and adopts an alternating integration strategy. That is, in the 1st and 3rd residual blocks, a BAM module is inserted after the 3×3 convolution to further optimize the spatial and channel responses of the features. At the output of the 2nd and 4th residual blocks, an EPSA module is connected. EPSA first groups the input features evenly along the channel dimension. Each group is passed through a 3×3 convolutional layer with different dilation rates to obtain different receptive fields. Then, the outputs are concatenated and fused using a 1×1 convolution. Finally, a channel attention mechanism similar to SE is used to adaptively weight the fused features, outputting features that incorporate multi-scale contextual information. For Stage 4, this stage contains three residual blocks. After the last 1×1 convolutional layer in each residual block and before the summation of short-circuit connections, an SE module is inserted. The SE module learns the importance weights of each channel through global average pooling and two fully connected layers, and performs channel weighting on the feature maps, thereby performing feature selection at a high-level semantic level.

[0075] After the first feature extraction sub-network processes the target image, an initial feature map F_cnn with a size of 7×7×2048 is obtained. Global average pooling is then performed on the initial feature map F_cnn to obtain a 2048-dimensional feature vector v_cnn. At the same time, F_cnn is retained for subsequent global relation modeling.

[0076] It's easy to understand that the deployment of the aforementioned attention modules is not a random stacking, but rather a design based on "shallow layers emphasizing spatial attention, mid-layer layers emphasizing multi-scale attention, and deep layers emphasizing channel attention," forming a progressive feature enhancement pipeline that synergistically enhances spatial, multi-scale, and channel attention. In the shallow layer (Stage 2), CBAM effectively suppresses background clutter through spatial attention, allowing the network to focus on key areas of the human body. In the mid-layer (Stage 3), BAM strengthens local responses, while EPSA integrates local and mid-range context through multi-scale convolutional branches to address the scale diversity of driver actions. In the deep layer (Stage 4), SE focuses on refining semantic channels, making abstract features closer to the essence of behavioral categories. This heterogeneous, hierarchical, and task-oriented attention deployment strategy enables the network to achieve proactive suppression of interference from complex environments, simultaneous modeling of multi-scale actions, and precise extraction of high-order semantics without significantly increasing the number of parameters. In summary, this application constructs an interpretable, hierarchical attention enhancement mechanism that runs through the entire feature extraction process by systematically deploying complementary heterogeneous attention modules at multiple stages of the residual network.

[0077] Optionally, the second feature extraction sub-network includes a location encoding module and a relation encoder, wherein the location encoding module is used to add location encoding, and the relation encoder is used to extract global semantic relations of any target image based on the processing results of the location encoding module.

[0078] First, the role of the location encoding module is to assign a unique "coordinate semantics" to each spatial location in the initial feature map F_cnn output by the first feature extraction sub-network, so that the relation encoder can distinguish the semantic roles of different spatial regions.

[0079] Optionally, the feature map F_cnn extracted by the optimized ResNet-50 deep network (i.e., the first feature extraction sub-network) has a size of 7×7×2048. This means that the feature map already possesses highly abstract semantic information while retaining the 7×7 spatial structure. At this point, each feature map location (e.g., row 3, column 5) corresponds to a 2048-dimensional semantic vector, representing the comprehensive features of that spatial region, but it does not yet possess the ability to perceive "where this location is in the image". The location encoding module generates an encoding vector with the same dimension as the feature vector for each 7×7 spatial location through a learnable two-dimensional location embedding matrix. This vector is automatically learned by the model during training, rather than being manually designed (e.g., sinusoidal function encoding).

[0080] For example, the location encoding module constructs a 7×7×2048 tensor, where the 2048-dimensional vector at position (i, j) represents the relative spatial location semantics of the "region in row i and column j of the image". This tensor is element-wise added to the input F_cnn feature map at corresponding positions to obtain a "location-aware feature sequence". This operation is mathematically equivalent to injecting the relative coordinate information of each spatial token in the original image. For example, the "hand region in the upper left corner" and the "central control screen region in the lower right corner" have different location embeddings after encoding, allowing the subsequent attention mechanism to determine whether they should be semantically associated based on the location differences.

[0081] Optionally, the relation encoder can be a lightweight Transformer encoder, which models the global semantic dependency between any two spatial regions using a self-attention mechanism based on the feature sequence with positional information output by the positional encoding module.

[0082] In one alternative embodiment, the relation encoder consists of only two stacked Transformer layers, each containing two sub-modules: a Multi-Head Self-Attention (MHSA) layer and a Feed-Forward Network (FFN) layer, both employing residual connections and layer normalization structures.

[0083] In the multi-head self-attention layer, the input 49 2048-dimensional feature vectors (7×7=49 tokens) are first processed through three independent linear projection matrices to generate a query matrix Q, a key matrix K, and a value matrix V. Each matrix is ​​split into h=8 sub-heads, each processing a 256-dimensional sub-vector. Within each sub-head, the dot product of Q and K is calculated, then divided by the square root of the sub-vector dimension to obtain an attention score. This score is then normalized to a probability distribution using the Softmax function, representing the "attention level" of each token to every other token. This probability distribution is then multiplied by V to obtain a weighted output vector. Essentially, this process allows features at each spatial location to "actively inquire" about features at all other locations, thereby establishing cross-regional semantic connections. For example, a token representing the "right-hand region" can establish a strong correlation with the "central control screen region" or the "gaze direction region" through attention weights, even if they are spatially far apart. This mechanism overcomes the problem of limited receptive fields in convolutional neural networks, achieving true global perception.

[0084] The feedforward network layer consists of two fully connected layers connected in the middle by the GELU activation function (a non-linear activation function based on the Gaussian cumulative distribution function), which performs non-linear transformation and expansion on the features of the attention output, further enhancing its expressive power.

[0085] After each sub-layer (i.e., a single Transformer layer), residual connections (adding the input directly to the output) and layer normalization (standardizing the mean and variance of the feature vector of each sample) are performed to stabilize training and accelerate convergence.

[0086] After processing through two Transformer layers, the 49 tokens undergo deep interaction and information reorganization, outputting a 49×2048 feature sequence with richer semantics and clearer relationships. To obtain the global semantic vector v_trans, global average pooling is performed on these 49 tokens, averaging the 49 values ​​in each dimension, ultimately outputting a 2048-dimensional vector, v_trans. This vector no longer represents a local region, but rather a "relationship representation" that integrates the relationships between all parts of the entire image, such as the semantic condensation of the complex behavior of "the hand is operating the screen while the gaze is not looking forward."

[0087] It is easy to understand that the location encoding module and the relation encoder together constitute an efficient, lightweight, and learnable global semantic relation modeling engine. By injecting spatial location semantics into the feature map F_cnn and using a minimalist Transformer architecture to achieve long-range dependency modeling between arbitrary regions, it effectively makes up for the shortcomings of traditional vision models in global semantic understanding.

[0088] Optionally, the feature fusion subnetwork includes a gating network, which is configured to: concatenate the first feature extraction result and the second feature extraction result to obtain a joint feature extraction result; process the joint feature extraction result using the gating network to obtain a weight allocation result, wherein the weight allocation result is used to characterize the relative importance of the first feature extraction result and the second feature extraction result; and perform weighted fusion of the first feature extraction result and the second feature extraction result based on the weight allocation result to obtain a fusion result.

[0089] First, the first and second feature extraction results are concatenated to obtain the joint feature extraction result. The first feature extraction result, v_cnn, is a 2048-dimensional feature vector output by a hierarchical multi-attention enhanced ResNet-50 backbone network after global average pooling. It highly focuses on fine-grained visual cues such as texture, edges, and pose in local regions, such as the degree of eyelid opening and closing, the physical contact position of the hand on the central control screen, and the local offset of head orientation. The second feature extraction result, v_trans, captures long-range associations across regions and semantic units, such as "hand position" and "gaze direction," and "head turn" and "steering wheel grip." The concatenation operation links these two feature vectors with the same dimension and complementary semantics along the channel dimension, forming a 4096-dimensional joint feature extraction result.

[0090] The essence of the concatenation operation is to construct a unified semantic space, enabling subsequent gating networks to simultaneously observe complete information of both "local details" and "global semantics" within a continuous high-dimensional representation, thus providing a comprehensive basis for subsequent weight judgments. The concatenation operation itself is a linear operation, without parameters or nonlinear transformations. Its key lies in maintaining the integrity of the original information of the two feature streams and avoiding information distortion caused by early fusion.

[0091] Secondly, a gating network is used to process the joint feature extraction results to obtain weight allocation results, which represent the relative importance of the first and second feature extraction results. The gating network is a lightweight, parameter-efficient neural network module.

[0092] The joint feature extraction results can be processed using a gating network to obtain the weight allocation results, which can be implemented in the following steps.

[0093] First, the 4096-dimensional joint feature vector is input into a fully connected layer, which compresses the feature dimension to 256. This process essentially performs feature compression and nonlinear recombination on the original high-dimensional joint information, introducing nonlinear expressive power and avoiding the inability of simple linear projection to capture complex interactions. Subsequently, this 256-dimensional feature is processed by the ReLU activation function to obtain activated features. The ReLU function sets negative values ​​to zero and retains positive values, thereby enhancing the network's expressive power, mitigating gradient vanishing, and improving training stability. Next, the activated features are input into a second fully connected layer, further compressing the dimension to 2, and outputting two unnormalized scalar values, corresponding to the "original preference scores" of the first feature extraction result (v_cnn) and the second feature extraction result (v_trans), respectively. These scores do not yet have probabilistic meaning; they only represent relative advantages. Subsequently, the two scores are non-linearly compressed using the Sigmoid activation function, mapping them to the (0, 1) interval to obtain two normalized scalars g1 and g2 between 0 and 1. These represent the model's initial estimate of the "trust level" of local and global features under the current frame input conditions. The closer the normalized scalar value is to 1, the more critical the feature flow; the closer it is to 0, the weaker the contribution of the feature flow. The key to the design of this gated network lies in its "lightweight" nature: it contains only two fully connected layers, with no additional convolutions or pooling, resulting in a very small number of parameters and extremely low inference latency, fully meeting the stringent real-time requirements of automotive embedded systems.

[0094] Furthermore, after obtaining g1 and g2, normalization processing is required to ensure they satisfy the probability constraints: α = g1 / (g1 + g2), β = g2 / (g1 + g2), thus ensuring α + β = 1. α and β are the weight allocation results corresponding to the first and second feature extraction results.

[0095] Finally, the results of the first and second feature extractions are weighted and fused to obtain the fused result, F_fused = α × v_cnn + β × v_trans, which is a 2048-dimensional weighted composite vector. It is not a simple average of the two features, but a dynamically weighted sum that adaptively adjusts according to the scenario. This fusion method preserves the original dimensions and semantic structure of the two feature streams, resulting in a semantically richer and more robust composite representation received by the subsequent classifier, effectively avoiding the risk of failure of a single feature stream in extreme scenarios.

[0096] It is easy to understand that by utilizing the feature fusion sub-network, a four-step collaborative process of "splicing-gating-normalization-fusion" is achieved, constructing a lightweight, adaptive, and semantically aware dual-stream feature fusion paradigm.

[0097] Optionally, the target behavior is determined based on multiple behavior recognition results, including the following steps:

[0098] Step S121: Statistical analysis is performed on multiple behavior recognition results to obtain statistical results, wherein the statistical results are used to describe the frequency of occurrence of multiple driver behaviors corresponding to the multiple behavior recognition results respectively;

[0099] Step S122: Based on the statistical results, determine the target behavior.

[0100] In one optional embodiment, multiple behavior recognition results (i.e., multiple behavior labels) corresponding to multiple target images within a preset time window are counted to form a frequency vector, which is the "statistical result". For example, if "operating the phone with the right hand" appears 18 times, "talking to passengers" appears 9 times, and "driving safely" appears 3 times in a window of T=30 frames, then the statistical result is [3, 18, 9, 0, 0, ..., 0], where each dimension corresponds to the cumulative frequency of each type of behavior.

[0101] Furthermore, based on the statistical results, the most frequent behaviors are selected as the target behaviors.

[0102] Through the above steps, the system achieves accurate transformation from instantaneous and easily disturbed single-frame classification results to stable and semantically clear target behaviors, significantly improving the system's decision robustness and interpretability in real and complex driving scenarios.

[0103] Optionally, based on the target behavior, the driver's distraction risk level is determined, including the following steps:

[0104] Step S131: Obtain the duration of the target behavior and the preset risk baseline value;

[0105] Step S132: Based on the duration and preset risk baseline, determine the driver distraction risk level corresponding to the target behavior.

[0106] The duration mentioned above is the cumulative value of the target behavior occurring continuously in the time series, rather than the detection result of a single frame.

[0107] For example, for a sliding behavior buffer queue of length T (e.g., T=10 frames, corresponding to approximately 0.33 seconds, sampled at 30 FPS), the duration is incremented whenever the behavior label of the current frame matches that of the previous frame; once the behavior changes (e.g., from "operating the phone" to "looking forward"), the duration is reset to zero. This mechanism effectively filters out false triggers caused by detection jitter or brief gaze shifts, ensuring that risk assessment is initiated only for persistent distracting behaviors.

[0108] The aforementioned preset risk baseline values ​​are static risk weights pre-assigned to each defined driver distraction behavior. For example, "operating a mobile phone" involves visual, manual, and cognitive distractions, so its preset risk baseline value is set at 2.8; "eating" requires both hands off the steering wheel and the driver's gaze is often off the road, so its preset risk baseline value is 2.2; "adjusting the rearview mirror," although a brief action, still poses a potential risk if it occurs at high speed, so its preset risk baseline value is 1.5; and "looking left and right," if extremely short (such as checking for vehicles behind), has a preset risk baseline value of 1.0. These preset risk baseline values ​​were calculated by a team of safety experts through multi-dimensional weighted calculations in a real accident database and simulated driving experiments. After cross-validation, they were solidified into system parameters and are not dynamically adjusted based on individuals or scenarios, ensuring the objectivity and consistency of the assessment.

[0109] Optionally, after obtaining the duration and preset risk baseline, a nonlinear risk mapping function is used to make a comprehensive judgment to obtain the driver distraction risk level corresponding to the target behavior.

[0110] Optionally, determining the driver's distraction risk level can be implemented by setting multiple risk level threshold ranges. Risk level R=1 (low risk) corresponds to "base value × duration" < 1.5; risk level R=2 (medium risk) corresponds to: 1.5 ≤ base value × duration < 4.0; risk level R=3 (high risk) corresponds to: 4.0 ≤ base value × duration < 8.0; risk level R=4 (emergency risk) corresponds to: base value × duration ≥ 8.0. For example, if the target behavior is "operating a mobile phone" (preset risk base value 2.8), and the duration is 1.8 seconds, then the calculated score = 2.8 × 1.8 = 5.04, falling within the R=3 range, thus determining the driver's distraction risk level as high risk.

[0111] In the above steps, by introducing a collaborative assessment mechanism with two dimensions of "duration" and "preset risk baseline", the limitations of traditional static classification based solely on behavior type are overcome. This enables quantitative modeling of the "intensity-time" coupling effect of distraction behavior, significantly improving the semantic depth and scenario adaptability of risk assessment.

[0112] Optionally, the driver distraction risk level includes: a first risk level, a second risk level, a third risk level, and a fourth risk level. The driving risk corresponding to the first risk level is lower than that corresponding to the second risk level, the driving risk corresponding to the second risk level is lower than that corresponding to the third risk level, and the driving risk corresponding to the third risk level is lower than that corresponding to the fourth risk level. Based on the driver distraction risk level, a vehicle control strategy is determined, including the following steps:

[0113] Step S141, in response to the driver distraction risk level being the first risk level, determines the vehicle control strategy including: triggering visual cues;

[0114] Step S142, in response to the driver distraction risk level being the second risk level, determines the vehicle control strategy including: triggering auditory cues;

[0115] Step S143, in response to the driver distraction risk level being the third risk level, determine the vehicle control strategy including: triggering auditory cues and vibration cues;

[0116] Step S144, in response to the driver distraction risk level being the fourth risk level, determines the vehicle control strategy including: triggering visual cues, auditory cues, vibration cues, and sending an assistance request to the vehicle's advanced driver assistance system.

[0117] Optionally, the first risk level corresponds to minor, occasional, and low-hazard distractions, such as briefly turning the head to check the rearview mirror, slightly adjusting the air conditioning knob, or looking away from the road for less than 1.5 seconds. When the driver's distraction risk level is the first risk level, to avoid excessively interfering with the driver's cognitive load, a "non-intrusive" visual cue is used as the primary intervention. For example, in the digital instrument panel or head-up display (HUD), a low-brightness, light blue, or gradient semi-transparent icon (such as a flashing eye outline) is used to provide a cue in a non-critical area at the edge of the driver's field of vision (such as the upper left or upper right corner). This icon does not obstruct road information, and its brightness automatically adapts to the ambient light (based on feedback from the in-vehicle light sensor).

[0118] Optionally, the second risk level corresponds to low-to-medium intensity, persistently increasing distracting behaviors, such as frequent touchscreen use, engaging in non-urgent conversations with passengers, or having one's gaze deviate from the road for more than 1.5 seconds but less than 3 seconds, and the behavior is repetitive (e.g., the same behavior appears in more than three consecutive frames). At this point, visual cues are insufficient to elicit sufficient alertness, and "auditory cues" are activated as the second level of intervention. The auditory cues use "semantic prompts" instead of traditional sharp alarms. The prompts are played by the in-vehicle audio system, using a soft pulse tone in the 2.5kHz~3.5kHz frequency band (similar to three short "beep" sounds, spaced 200ms apart). The volume is dynamically adjusted according to the in-vehicle noise level, with a maximum of no more than 75dB to avoid causing auditory fatigue or startle. In one optional embodiment, a "sound source localization" technology is employed to achieve spatial directionality of sound through a multi-speaker array (such as A-pillar, center console, and dashboard speakers): when the driver's head is turned to the left and distracted, a prompt tone is played from the right speaker with a slight delay (e.g., 5ms), creating an auditory guidance that "comes from the opposite direction," naturally guiding the driver to turn straight ahead. Furthermore, the semantic encoding of the prompt tone represents an association between behavior and consequences; for example, when "phone operation" is detected, the prompt tone is "Please look ahead"; when "talking to someone" is detected, the prompt tone is "Focusing on driving is safer."

[0119] Optionally, the third risk level applies to highly persistent and dangerous distracting behaviors, such as single distractions lasting more than 3 seconds, behaviors categorized as "using a mobile phone," "eating," "applying makeup," or "smoking," or continued distraction even when the vehicle is in high-risk conditions such as high-speed driving, curves, or lane changes in congestion. In this case, a single auditory cue is insufficient to break the driver's locked attention; a multimodal collaborative wake-up mechanism must be activated, i.e., simultaneous triggering of auditory and tactile vibration cues. The auditory cue at this stage is upgraded to a "continuous pulse tone" (two short "beep-beep" sounds per second), with the volume increased to 80dB and superimposed with a slight low-frequency resonance (approximately 80Hz) to enhance auditory penetration. The vibration cues are achieved by a linear resonant actuator built into the seat or a seatbelt tensioner, generating regular pulse vibrations with a period of 1.2 seconds and an amplitude of 0.3g, with the vibration position precisely mapped to the driver's contact point. For example, if right-hand distraction is detected, the left seat back vibrates; if head turn is detected, the corresponding shoulder seatbelt vibrates.

[0120] Optionally, the fourth risk level corresponds to extremely dangerous distracted behavior, typically manifested as: distracted behavior lasting more than 5 seconds, behavior categories such as "closing eyes," "looking down at a mobile phone for more than 2 seconds," and "severe deviation from the road direction." At this point, the driver is on the verge of "functional incapacity," and active safety intervention must be initiated to form the ultimate protection of "human-machine co-governance." The vehicle control strategy corresponding to the fourth risk level is a four-fold collaborative intervention. Specifically, the visual cue is upgraded to a full-screen red flashing warning (covering the central area of ​​the head-up display and the central control screen, lasting for 3 seconds, and cannot be canceled); the auditory cue is a continuous high-frequency alarm sound; the vibration cue is upgraded to a high-intensity, non-periodic pulse, simulating the tactile sensation of "sudden loss of control" to stimulate instinctive reactions; in addition, a structured control request message is sent to the advanced driver assistance system via the vehicle communication bus, and its protocol format includes: distraction type code (e.g., 0x15 = mobile phone operation), duration (unit: seconds), confidence level (>95%), current vehicle speed, steering angle, distance to the vehicle in front, and other data. Upon receiving the request, the ADAS system automatically activates or enhances its functions: if lane keeping assist is already enabled, it enhances steering torque compensation; if not enabled, it automatically activates temporary lane centering assist; if a collision risk is detected ahead, it triggers emergency braking assist pre-charge and illuminates the hazard warning lights, while simultaneously announcing via voice, "System intervention, focus on driving immediately."

[0121] The above steps, through the construction of a four-level progressive risk response mechanism, have achieved a qualitative leap from "passive detection" to "intelligent guidance" and then to "proactive safety collaboration." This mechanism not only fully considers the physiological rhythm of human attention recovery and the complementarity of multimodal perception, but also greatly improves the accuracy, acceptability, and effectiveness of intervention through key technical features such as spatial cues, semantic feedback, tactile positioning, and deep integration with ADAS.

[0122] Optionally, acquiring multiple target images within a preset time window includes the following steps:

[0123] Step S101: Obtain multiple initial images within a preset time window, wherein the multiple initial images are used to characterize the images to be processed collected by the vehicle camera to describe the driver's behavior.

[0124] Step S102: Region extraction and size normalization and standardization are performed on multiple initial images to obtain multiple target images.

[0125] The aforementioned acquisition of multiple initial images within a preset time window refers to the vehicle-mounted camera continuously capturing the original RGB video stream of the driver's face and upper body at a fixed frame rate, and extracting a continuous image sequence with a fixed time span from it. Optionally, the multiple initial images are a continuous image sequence corresponding to the most recent T frames.

[0126] The aforementioned region extraction refers to locating the driver's key behavioral regions from each initial image using efficient target detection algorithms.

[0127] Optionally, a lightweight face detector is employed to quickly output face bounding boxes in each frame. Based on these bounding boxes, a rectangular region is expanded by a fixed ratio (e.g., 1.5 times larger in both width and height) to ensure the inclusion of the head, shoulders, upper limbs, and hand movement range. After region extraction, each frame is cropped, retaining only the expanded region. This removes over 70% of irrelevant background information (such as car doors, rooftops, and trees outside the window) from the original image, significantly reducing the computational load and interference sources in subsequent processing.

[0128] The aforementioned size normalization refers to uniformly scaling the cropped region image to a fixed resolution required by the distraction detection model input, such as 224×224 pixels.

[0129] The above standardization refers to converting pixel values ​​from the original 0-255 format to a numerical range that conforms to the input of a neural network.

[0130] In one optional embodiment, raw video data of the driver's area is continuously acquired within a preset time window. For each frame acquired, a lightweight face detection model is activated, outputting the coordinates of the driver's facial center and the positions of key points (such as the eyes, nose tip, and chin). Based on these key points, a rectangular region is dynamically expanded, with a width 2.5 times the width of the face and a height 3 times the height of the face, ensuring that it includes the range of hand interactions on the central control screen or the edge of the window. Furthermore, this region is uniformly scaled to 224×224 pixels and pixel values ​​are normalized.

[0131] Through the above steps, an efficient transformation from the original video stream to a high-quality image sequence that is structured, semantically focused, and uniformly distributed is achieved, significantly reducing the computational burden of subsequent feature extraction.

[0132] It should be noted that the SE module mentioned above can be replaced by the ECA-Net module or the GSoP module, aiming to achieve efficient channel attention with lower computational cost. The CBAM / BAM module can be replaced by simplified variants of Coordinate Attention, GCNet, or Non-Local Networks to achieve different forms of spatial-channel joint attention. The multi-scale acquisition method of the EPSA module can be achieved by using parallel convolutional branches with different kernel sizes, or by using deformable convolutions instead of dilated convolutions. Its final attention fusion can also adopt SE-based channel attention or self-attention-based spatial-channel hybrid attention. The ResNet backbone can be replaced by EfficientNet, RegNet, ConvNeXt, or lightweight MobileNetV3 and ShuffleNetV2. The integration position of the attention module needs to be adaptively adjusted according to the structural characteristics of the new network. The "two-stream" architecture can evolve into a more compact hybrid unit. For example, an "Attentive CNN-TransformerBlock" can be designed, directly embedding lightweight multi-head attention layers into the residual blocks of a CNN, replacing the original 3×3 convolutions, forming a basic building block integrating local computation and global perception. Gated adaptive fusion can be simplified to fixed-ratio fusion or complicated into deep fusion based on cross-attention, where one feature path serves as the query and the other as the key and value, enabling in-depth interaction. The visual processing stream can serve as a perception unit, performing feature-level early fusion or decision-level late fusion with data from millimeter-wave radar (detecting vital signs and micro-motions), infrared cameras (solving low-light problems), and in-vehicle microphones (detecting abnormal sounds or voice fatigue), building a more robust multimodal driver state monitoring system. The backbone network (feature extraction part) of this application can be designed as shared, with multiple lightweight task-specific branches, simultaneously performing distraction detection, fatigue estimation, head pose estimation, gesture recognition, etc., achieving efficient reuse of hardware resources and forming a unified in-cabin perception solution. To adapt to platforms with different computing power, smaller student models can be distilled from the complete model, or the trained model can be pruned and quantized to significantly reduce model size and inference latency while maintaining performance as much as possible, enabling it to be deployed on a variety of hardware from high-end domain controllers to low-cost microcontroller units.

[0133] Based on the above analysis, the technical solutions proposed in this application can achieve the following technical effects: Improved robustness of detection in complex environments: By hierarchically deploying spatial attention modules such as CBAM and BAM, background interference from car windows, dashboards, etc., is effectively suppressed in the early stages of feature extraction, enhancing attention to key areas such as the driver's face and hands. The multi-scale fusion capability of the EPSA module enables the network to adapt to both subtle facial expressions and large-scale body movements, enhancing the model's adaptability to challenging scenarios such as lighting changes and partial occlusion. Enhanced semantic depth of behavioral understanding: The CNN-Transformer hybrid architecture combines the advantages of local feature extraction from convolutional networks with the global relational modeling capabilities of Transformers. This allows the system to not only recognize the local features of "hands in the central control screen area" but also understand the associated context of "eyes simultaneously deviating from the road ahead," achieving more accurate judgment of complex distraction behaviors. To optimize computational efficiency and meet real-time requirements, this approach utilizes computationally efficient CBAM / BAM in shallow layers and fewer parameters in deeper layers using SE. A simplified two-layer Transformer encoder and a lightweight gating network for feature fusion are employed. These designs result in a model with only about 15% more parameters than the original ResNet-50, significantly lower than the parameter increase from directly using the standard ViT-Base model, thus providing feasibility for deployment on automotive embedded platforms. The proposed architecture, characterized by "hierarchical attention-enhanced CNN + lightweight Transformer relationship modeling + adaptive fusion," exhibits good architectural versatility and functional scalability. By replacing or adding prediction heads, this architecture can be extended to various intelligent cockpit visual perception tasks such as driver fatigue detection, emotion recognition, gesture interaction, and in-cabin object detection, providing a feasible technical path for building a low-cost, high-performance unified in-cabin perception platform.

[0134] Optionally, Figure 2 This is a flowchart of another vehicle control method according to an embodiment of this application, such as... Figure 2 As shown, the vehicle control method can be implemented as follows.

[0135] Image Acquisition and Preprocessing: The camera continuously acquires images at a preset frame rate of 30 FPS. Each frame undergoes preprocessing, including region of interest (ROI) extraction and size normalization. Specifically, a high-efficiency face detector is first run to quickly locate the driver's face. Based on this, a region of interest image containing the driver's head, shoulders, and the main active areas of the arms is cropped according to a preset aspect ratio. Further, the cropped ROI image is scaled to a fixed size of 224×224 pixels required by the network input, and then pixel values ​​are normalized, for example, by converting to a range of [0, 1] or by normalizing using pre-calculated mean and standard deviation.

[0136] Hierarchical multi-attention feature extraction: The preprocessed image is input into the modified CNN backbone network to obtain the first feature extraction result.

[0137] Lightweight Global Relationship Modeling: A global analysis is performed on the feature map output by the modified CNN backbone network to obtain the second feature extraction result. Specifically, the feature map F_cnn (7×7×2048) is flattened in spatial dimension, and each spatial location of 7×7 is regarded as a "feature token", resulting in 49 tokens, each of which is a 2048-dimensional vector. Learnable positional encodings are added to the feature map sequence, and then it is fed into a lightweight Transformer encoder. This encoder consists of only two layers, each containing a multi-head self-attention layer and a feedforward network layer. The token sequence output by the Transformer encoder is average-pooled to obtain a 2048-dimensional feature vector v_trans representing global semantic relationships.

[0138] Gated adaptive feature fusion: Input v_cnn and v_trans into the gated adaptive fusion module to obtain the fusion result F_fused.

[0139] Behavior Classification and Risk Assessment: Driver behavior is determined based on the fusion result. Based on multiple driver behaviors, a target behavior is identified, and the risk level is determined based on the target behavior. Specifically, F_fused is input into the prediction head. The prediction head first consists of a fully connected layer that maps 2048-dimensional features to 512 dimensions, followed by a ReLU activation and Dropout layer, and then another fully connected layer that maps the 512 dimensions to N dimensions, where N is the number of behavior categories to be classified. Then, the Softmax function is used to convert the N-dimensional vector into a probability distribution P = [p1, p2, ..., pN]. For example, N=10, corresponding to: 0-safe driving, 1-right-handed phone operation, 2-left-handed phone operation, 3-touchscreen use, 4-talking to passengers, 5-eating, 6-smoking, 7-hair tidying / makeup application, 8-looking around, 9-other. The category with the highest probability is taken as the behavior recognition result for the current frame. Furthermore, a sliding window of length T is maintained to store the behavior recognition results of the most recent T frames. A short-term stable behavior label L_stable is determined using majority voting. Define a risk mapping function R = f(L_stable, Duration), which calculates the current risk level based on the inherent risk baseline value of the behavior label L_stable (e.g., "operating a mobile phone" has a high baseline value, "adjusting the rearview mirror" has a low baseline value) and the duration of the behavior (Duration). For example, several risk thresholds can be set, and the risk level increases by one level when the Duration exceeds a certain threshold.

[0140] Warning Decision and Output: Based on the determined behavior label L_stable and risk level R, the onboard computing unit generates corresponding control commands and sends them to the execution end via the bus. The current status (e.g., "Distracted: Using a mobile phone") and risk level are displayed on the instrument panel or head-up display using icons, colors, or text. For low risk (R=1), only the human-machine interface display is updated; for medium risk (R=2), a brief auditory warning is triggered; for high risk (R=3), a continuous auditory warning is triggered, and seat vibration may be activated; for emergency risk (R=4, such as persistent high risk while the vehicle is in motion), in addition to the above warnings, a request can be sent to the ADAS system suggesting enhanced lane keeping assist or preparation for emergency braking.

[0141] Optionally, Figure 3 This is an implementation architecture diagram of a vehicle control system according to an embodiment of this application, such as... Figure 3 As shown, the vehicle control system includes a data acquisition layer, a core processing unit, an output and execution layer, and a storage and learning module. The data acquisition layer acquires driver and vehicle status information in real time through in-vehicle cameras, in-vehicle microphones, and controller area network (CLAN) bus sensors. The core processing unit runs a multimodal feature extraction and fusion module and a hierarchical intent reasoning and risk assessment module on the in-vehicle artificial intelligence computing platform to achieve hierarchical intent reasoning and risk assessment. The output and execution layer triggers safety interventions from the digital instrument panel / head-up display, in-vehicle speaker sound warnings, seat vibrations, and the advanced driver assistance system controller based on risk level classification. The storage and learning module continuously collects interactive feedback, response feedback, effect feedback, and intervention feedback to optimize personalized profiles and decision-making models online, updating driving habits and the driver's personalized profile database, forming a complete closed loop of perception, decision-making, execution, and optimization. The entire system achieves intelligent processing throughout the entire process, from multi-source information acquisition to intelligent warnings and continuous optimization.

[0142] Optionally, the implementation of this application embodiment may rely on an in-vehicle electronic system, which includes: at least one camera deployed facing the driver for collecting video data of the driver's area; an in-vehicle computing unit, which is typically a domain controller or dedicated computing module integrating an artificial intelligence acceleration unit (such as a GPU or NPU) and is responsible for running the core algorithm of this application; a vehicle bus network (such as CAN FD or Ethernet) for high-speed and reliable data exchange and control command transmission between the algorithm unit and other vehicle systems; a human-machine interface for information display (such as a digital instrument panel, vehicle head-up display, or central control screen); a mechanism for executing warnings (such as a speaker, buzzer, or haptic feedback device); and a communication interface with an advanced driver assistance system (ADAS) or body control module for sending vehicle intervention requests in emergency situations.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0144] According to an embodiment of this application, a system embodiment of a vehicle control system is provided. It should be noted that this system can be used to execute the above-described vehicle control method.

[0145] According to another aspect of the embodiments of this application, a vehicle control system is also provided. Figure 4 This is a structural block diagram of a vehicle control system according to an embodiment of this application, such as... Figure 4 As shown, the vehicle control system 400 includes: an acquisition module 401, used to acquire multiple target images within a preset time window, wherein the multiple target images are used to describe driver behavior; and a processing module 402, used to process the multiple target images using a pre-trained distraction detection model to obtain multiple behavior recognition results, wherein the pre-trained distraction detection model includes: a first feature extraction subnetwork, a second feature extraction subnetwork, a feature fusion subnetwork, and a behavior classification subnetwork. For any target image among the multiple target images, the first feature extraction subnetwork is used to extract local features of any target image to obtain a first feature extraction result, and the second feature extraction subnetwork is used to extract global features of any target image to obtain a second feature extraction result. The feature fusion subnetwork is used to adaptively weight and fuse the first feature extraction result and the second feature extraction result to obtain the fusion result; the behavior classification subnetwork is used to classify the behavior of the fusion result to obtain the behavior recognition result corresponding to any target image, and multiple target images correspond one-to-one with multiple behavior recognition results; the first determination module 403 is used to determine the target behavior based on multiple behavior recognition results, wherein the target behavior is used to characterize the driver behavior with temporal consistency within a preset time window; the second determination module 404 is used to determine the driver distraction risk level based on the target behavior; the third determination module 405 is used to determine the vehicle control strategy based on the driver distraction risk level, and perform vehicle control according to the vehicle control strategy.

[0146] Optionally, the first feature extraction subnetwork includes: a convolutional neural network based on a residual structure, wherein different attention modules are deployed in multiple residual block groups of the convolutional neural network.

[0147] Optionally, the second feature extraction sub-network includes a location encoding module and a relation encoder, wherein the location encoding module is used to add location encoding, and the relation encoder is used to extract global semantic relations of any target image based on the processing results of the location encoding module.

[0148] Optionally, the feature fusion subnetwork includes a gating network, which is configured to: concatenate the first feature extraction result and the second feature extraction result to obtain a joint feature extraction result; process the joint feature extraction result using the gating network to obtain a weight allocation result, wherein the weight allocation result is used to characterize the relative importance of the first feature extraction result and the second feature extraction result; and perform weighted fusion of the first feature extraction result and the second feature extraction result based on the weight allocation result to obtain a fusion result.

[0149] Optionally, the first determining module 403 is further configured to: perform statistics on multiple behavior recognition results to obtain statistical results, wherein the statistical results are used to describe the occurrence frequency of multiple driver behaviors corresponding to the multiple behavior recognition results respectively; and determine the target behavior based on the statistical results.

[0150] Optionally, the second determining module 404 is further configured to: obtain the duration of the target behavior and a preset risk baseline value; and determine the driver distraction risk level corresponding to the target behavior based on the duration and the preset risk baseline value.

[0151] Optionally, the driver distraction risk level includes: a first risk level, a second risk level, a third risk level, and a fourth risk level, wherein the driving risk corresponding to the first risk level is less than the driving risk corresponding to the second risk level, the driving risk corresponding to the second risk level is less than the driving risk corresponding to the third risk level, and the driving risk corresponding to the third risk level is less than the driving risk corresponding to the fourth risk level. The third determining module 405 is further configured to: in response to the driver distraction risk level being the first risk level, determine a vehicle control strategy including: triggering a visual cue; in response to the driver distraction risk level being the second risk level, determine a vehicle control strategy including: triggering an auditory cue; in response to the driver distraction risk level being the third risk level, determine a vehicle control strategy including: triggering an auditory cue and a vibration cue; in response to the driver distraction risk level being the fourth risk level, determine a vehicle control strategy including: triggering a visual cue, an auditory cue, a vibration cue, and sending an assistance request to the vehicle's advanced driver assistance system.

[0152] Optionally, the acquisition module 401 is further configured to: acquire multiple initial images within a preset time window, wherein the multiple initial images are used to characterize the images to be processed collected by the vehicle-mounted camera to describe the driver's behavior; and perform region extraction and size normalization and standardization on the multiple initial images to obtain multiple target images.

[0153] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0154] Optionally, in this embodiment, the executable program performs the following steps when it runs:

[0155] Step S10: Acquire multiple target images within a preset time window, wherein the multiple target images are used to describe the driver's behavior;

[0156] Step S11: A pre-trained distraction detection model is used to process multiple target images to obtain multiple behavior recognition results. The pre-trained distraction detection model includes: a first feature extraction sub-network, a second feature extraction sub-network, a feature fusion sub-network, and a behavior classification sub-network. For any target image among the multiple target images, the first feature extraction sub-network is used to extract local features of any target image to obtain a first feature extraction result. The second feature extraction sub-network is used to extract global features of any target image to obtain a second feature extraction result. The feature fusion sub-network is used to adaptively weight and fuse the first feature extraction result and the second feature extraction result to obtain a fusion result. The behavior classification sub-network is used to classify the fusion result to obtain the behavior recognition result corresponding to any target image. Multiple target images correspond one-to-one with multiple behavior recognition results.

[0157] Step S12: Based on multiple behavior recognition results, determine the target behavior, wherein the target behavior is used to characterize the driver behavior that has temporal consistency within a preset time window;

[0158] Step S13: Determine the driver's distraction risk level based on the target behavior;

[0159] Step S14: Determine the vehicle control strategy based on the driver's distraction risk level, and control the vehicle according to the vehicle control strategy.

[0160] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0161] Optionally, Figure 5 This is a schematic diagram of an electronic device according to an embodiment of this application, such as... Figure 5 As shown, the electronic device 500 may include a memory 510 and a processor 520, wherein the memory 510 is used to store an executable program; and the processor 520 is used to run the program stored in the memory 510, and the program executes the methods in various embodiments of this application when it runs.

[0162] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0163] Optionally, in this embodiment, the executable program can be configured to store an executable program for performing the following steps:

[0164] Step S10: Acquire multiple target images within a preset time window, wherein the multiple target images are used to describe the driver's behavior;

[0165] Step S11: A pre-trained distraction detection model is used to process multiple target images to obtain multiple behavior recognition results. The pre-trained distraction detection model includes: a first feature extraction sub-network, a second feature extraction sub-network, a feature fusion sub-network, and a behavior classification sub-network. For any target image among the multiple target images, the first feature extraction sub-network is used to extract local features of any target image to obtain a first feature extraction result. The second feature extraction sub-network is used to extract global features of any target image to obtain a second feature extraction result. The feature fusion sub-network is used to adaptively weight and fuse the first feature extraction result and the second feature extraction result to obtain a fusion result. The behavior classification sub-network is used to classify the fusion result to obtain the behavior recognition result corresponding to any target image. Multiple target images correspond one-to-one with multiple behavior recognition results.

[0166] Step S12: Based on multiple behavior recognition results, determine the target behavior, wherein the target behavior is used to characterize the driver behavior that has temporal consistency within a preset time window;

[0167] Step S13: Determine the driver's distraction risk level based on the target behavior;

[0168] Step S14: Determine the vehicle control strategy based on the driver's distraction risk level, and control the vehicle according to the vehicle control strategy.

[0169] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0170] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the methods in various embodiments of this application.

[0171] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods described in the various embodiments of this application.

[0172] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0173] In this application, "multiple" refers to two or more.

[0174] In this application, unless otherwise expressly defined, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0175] The terms “first,” “second,” “third,” “fourth,” etc., in this application (if present) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0176] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0177] In the embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0178] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0180] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0181] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A vehicle control method, characterized in that, include: Acquire multiple target images within a preset time window, wherein the multiple target images are used to describe driver behavior; A pre-trained distraction detection model is used to process the multiple target images to obtain multiple behavior recognition results. The pre-trained distraction detection model includes a first feature extraction subnetwork, a second feature extraction subnetwork, a feature fusion subnetwork, and a behavior classification subnetwork. For any target image among the multiple target images, the first feature extraction subnetwork extracts local features of the target image to obtain a first feature extraction result; the second feature extraction subnetwork extracts global features of the target image to obtain a second feature extraction result; the feature fusion subnetwork adaptively weights and fuses the first and second feature extraction results to obtain a fusion result; and the behavior classification subnetwork classifies the fusion result to obtain a behavior recognition result corresponding to the target image. Each of the multiple target images corresponds one-to-one with the multiple behavior recognition results. Based on the multiple behavior recognition results, a target behavior is determined, wherein the target behavior is used to characterize driver behavior that has temporal consistency within the preset time window; Based on the target behavior, determine the driver's distraction risk level; Based on the driver distraction risk level, a vehicle control strategy is determined, and vehicle control is performed according to the vehicle control strategy.

2. The vehicle control method according to claim 1, characterized in that, The first feature extraction subnetwork includes a convolutional neural network based on a residual structure, wherein different attention modules are deployed in multiple residual block groups of the convolutional neural network.

3. The vehicle control method according to claim 1, characterized in that, The second feature extraction sub-network includes a location encoding module and a relation encoder, wherein the location encoding module is used to add location encoding, and the relation encoder is used to extract the global semantic relationship of any target image based on the processing result of the location encoding module.

4. The vehicle control method according to claim 1, characterized in that, The feature fusion subnetwork includes a gated network, and the feature fusion subnetwork is configured as follows: The first feature extraction result and the second feature extraction result are concatenated to obtain a joint feature extraction result; The joint feature extraction results are processed using the gating network to obtain a weight allocation result, wherein the weight allocation result is used to characterize the relative importance of the first feature extraction result and the second feature extraction result; Based on the weight allocation result, the first feature extraction result and the second feature extraction result are weighted and fused to obtain the fusion result.

5. The vehicle control method according to claim 1, characterized in that, Based on the multiple behavior recognition results, the target behavior is determined, including: The multiple behavior recognition results are statistically analyzed to obtain statistical results, wherein the statistical results are used to describe the frequency of occurrence of multiple driver behaviors corresponding to the multiple behavior recognition results respectively; Based on the statistical results, the target behavior is determined.

6. The vehicle control method according to claim 1, characterized in that, Based on the target behavior, the driver's distraction risk level is determined, including: Obtain the duration of the target behavior and the preset risk baseline value; Based on the duration and the preset risk baseline, the driver distraction risk level corresponding to the target behavior is determined.

7. The vehicle control method according to claim 1, characterized in that, The driver distraction risk level includes: a first risk level, a second risk level, a third risk level, and a fourth risk level. The driving risk corresponding to the first risk level is less than the driving risk corresponding to the second risk level, the driving risk corresponding to the second risk level is less than the driving risk corresponding to the third risk level, and the driving risk corresponding to the third risk level is less than the driving risk corresponding to the fourth risk level. Based on the driver distraction risk level, the vehicle control strategy is determined, including: In response to the driver distraction risk level being the first risk level, the vehicle control strategy is determined to include: triggering a visual cue; In response to the driver distraction risk level being the second risk level, the vehicle control strategy is determined to include: triggering an auditory cue; In response to the driver distraction risk level being the third risk level, the vehicle control strategy is determined to include: triggering the auditory and vibration cues; In response to the driver distraction risk level being the fourth risk level, the vehicle control strategy is determined to include: triggering the visual cues, the auditory cues, the vibration cues, and sending an assistance request to the vehicle's advanced driver assistance system.

8. The vehicle control method according to any one of claims 1 to 7, characterized in that, Acquiring the multiple target images within a preset time window includes: Multiple initial images are acquired within the preset time window, wherein the multiple initial images are used to characterize the images to be processed by the vehicle-mounted camera to describe the driver's behavior. Region extraction, size normalization, and standardization are performed on the multiple initial images to obtain the multiple target images.

9. A vehicle, characterized in that, include: Memory, which stores executable programs; A processor for running the executable program, wherein the executable program, when running, performs the vehicle control method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the vehicle control method according to any one of claims 1 to 8.

Images