A Highway Violation Identification Method Based on an Improved SlowFast Network

Abstract

This invention relates to the field of machine learning technology, and in particular to a method for identifying highway traffic violations based on an improved SlowFast network. The improved SlowFast network includes an input layer, a dual-branch feature extraction layer, a feature fusion layer, and a classification output layer. The dual-branch feature extraction layer includes a Slow branch and a Fast branch, and introduces an attention mechanism: the Slow branch uses ResNet-18 as the backbone network, introduces a CBAM spatial attention module after the third residual layer group of ResNet-18, and removes two residual blocks from the fourth residual layer group of ResNet-18; the Fast branch uses 3D MobileNetV2 as the backbone network, uses Depthwise Conv3D to replace standard convolution in MobileNetV2, and introduces an SE attention module after the output layer of the last 3D inverse residual stage of MobileNetV2. It balances lightweight, real-time performance, and high accuracy, perfectly meeting the needs of real-time behavior recognition scenarios deployed on edge devices.

Description

Technical Field

[0001] This invention relates to the field of deep learning and neural network technology, and in particular to a method for identifying highway traffic violations based on an improved SlowFast network. Background Technology

[0002] The SlowFast network is a classic dual-stream video recognition architecture that captures spatial semantic features through a Slow branch (low frame rate) and temporal dynamic features through a Fast branch (high frame rate), and is widely used in behavior recognition scenarios. However, existing SlowFast networks have the following technical problems:

[0003] 1. The backbone network has a large number of parameters (such as the traditional Slow branch using ResNet-50, which has about 25M parameters), high computational complexity, and is difficult to adapt to the real-time inference requirements of edge devices such as roadside units (RSU);

[0004] 2. Feature extraction lacks specificity, fails to focus on key areas of behavior recognition (such as hands and target objects), and is not robust enough to complex scenes (such as backlighting and occlusion).

[0005] 3. The activation function uses a fixed form (such as ReLU), which cannot be dynamically adjusted according to the characteristics of the two branches, resulting in the loss of negative features or the overexposure of positive features, affecting the recognition accuracy.

[0006] Meanwhile, real-time behavior recognition scenarios deployed on edge devices (such as highway driver hands-off recognition and industrial operation specification recognition) have extremely high requirements for lightweight, real-time performance, and robustness of the model, which the existing SlowFast network cannot meet.

[0007] Therefore, there is an urgent need for a lightweight and high-precision improved SlowFast neural network to solve the pain point of real-time behavior recognition in edge device scenarios. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide a method for identifying highway traffic violations based on an improved SlowFast network, which can meet the requirements of lightweight and high accuracy.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0010] A method for identifying highway traffic violations based on an improved SlowFast network, comprising the following steps:

[0011] For any vehicle, acquire the basic safety messages broadcast periodically by the vehicle, parse the basic safety messages, and determine whether the vehicle is in assisted driving mode based on the parsing results.

[0012] If the vehicle is in assisted driving mode, real-time video data of the vehicle is collected by the camera and the video data is input into the pre-trained hands-free recognition model, which then outputs a recognition result indicating whether hands-free driving has occurred.

[0013] If the identification result indicates that hands-free driving has occurred, an early warning mechanism is triggered, controlling the nearest audible and visual warning device in front of the vehicle to provide an audible and visual reminder, and simultaneously pushing warning information to the nearest law enforcement personnel in front of the vehicle.

[0014] The off-hand recognition model is based on an improved SlowFast network, which includes an input layer, a dual-branch feature extraction layer, a feature fusion layer, and a classification output layer.

[0015] The dual-branch feature extraction layer includes a Slow branch and a Fast branch, and introduces an attention mechanism:

[0016] The Slow branch uses ResNet-18 as the backbone network, introduces the CBAM spatial attention module after the third residual layer group of ResNet-18, and removes two residual blocks in the fourth residual layer group of ResNet-18.

[0017] The Fast branch uses 3D MobileNetV2 as the backbone network, replaces standard convolution with Depthwise Conv3D in MobileNetV2, and introduces an SE attention module after the output layer of the last 3D inverse residual stage of MobileNetV2.

[0018] The activation function used in the dual-branch feature extraction layer is:

[0019] ;

[0020] Where x is the output of the convolutional layer; The slope of the negative interval is a learnable parameter with an initial value of 0.1, used to control the retention strength of negative features. is the nonlinear coefficient in the positive interval, and is a learnable parameter with an initial value of 0.5, which controls the nonlinear smoothness of the positive feature.

[0021] The beneficial effects of this invention are as follows: The improved SlowFast network and its application in highway violation recognition adopt ResNet-18 with pruned residual blocks in the Slow branch and replaces standard convolution with Depthwise Conv3D in the Fast branch, which can effectively reduce the number of model parameters and adapt to real-time inference on edge devices; the introduction of an attention module focuses on key areas of behavior recognition, improving robustness in complex scenarios; and a DualBranchLU activation function is designed to dynamically adapt the dual-branch feature characteristics through learnable parameters, avoiding the loss of negative features or the overexposure of positive features; it balances lightweight, real-time performance and high accuracy, perfectly meeting the needs of real-time behavior recognition scenarios deployed on edge devices. Attached Figure Description

[0022] Figure 1 This is a structural example diagram of the improved SlowFast network according to an embodiment of the present invention;

[0023] Figure 2 This is a flowchart illustrating the application of the improved SlowFast network in highway violation identification according to an embodiment of the present invention.

[0024] Figure 3 This is a structural example diagram of a roadside unit according to an embodiment of the invention;

[0025] Label Explanation:

[0026] 1. A roadside unit; 2. A processor; 3. A memory. Detailed Implementation

[0027] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0028] Before detailing the embodiments of this application, some related concepts will first be explained:

[0029] Basic Safety Message (BSM): In the C-V2X (Cellular Vehicle-to-Everything) technology standard system, GB / T45315-2025 requires vehicles to periodically broadcast Basic Safety Messages (BSM).

[0030] Hands-off driving: The driver fails to maintain "effective control" and "continuous monitoring" of the vehicle, resulting in an inability to respond promptly to sudden road conditions and violating the driver's core safety obligations under "human-machine co-driving". This includes a lack of physical control: both hands are completely off the steering wheel, or only "symbolic contact" is made with non-hand parts (such as knees or elbows), making it impossible to complete steering, braking, or other operations within 0.5-1 seconds (the limit of human emergency response).

[0031] For example, if a driver takes both hands off the steering wheel to look at their phone, eat, or use their knees to brace the steering wheel in an attempt to "trick the vehicle's system," it is considered as taking their hands off the wheel—even if there is physical contact with the steering wheel, if the driver cannot quickly apply force to control it, it is still considered "ineffective control."

[0032] Driving automation levels are divided into L0-L5:

[0033] Level 0: Emergency Assistance (No Automation). The system only provides warnings or brief interventions (such as Automatic Emergency Braking (AEB) and Lane Departure Warning (LDW), but does not participate in vehicle control. The driver must take the lead in driving throughout the entire process; the system serves only as a safety supplement.

[0034] Level 1: Partial driver assistance. The system can independently control a single driving dimension (lateral or longitudinal), such as acceleration, braking, or steering, but requires continuous monitoring and timely intervention from the driver.

[0035] Level 2: Combined driving assistance. The system can control both lateral and longitudinal directions simultaneously (such as automatic acceleration / deceleration + steering), but the driver must continuously monitor road conditions and respond to takeover requests at any time.

[0036] Level 3: Conditional automated driving. The system can independently complete all driving tasks in specific scenarios (such as highways) without the driver needing to continuously monitor it, but must take over promptly when requested by the system.

[0037] Level 4: Highly automated driving. The system drives fully autonomously within its Design Operating Domain (ODD) without driver intervention and can automatically handle emergencies (such as pulling over to the side of the road).

[0038] Level 5: Fully automated driving. The system can autonomously complete driving tasks in any environment (including extreme weather and complex road conditions) without human intervention, and the driver becomes a passenger.

[0039] Roadside Unit (RSU) is the core roadside equipment in the Vehicle-to-Everything (V2X) technology system. Essentially, it is an intelligent roadside node that integrates perception, communication, computing, and control, and is responsible for connecting the vehicle-mounted unit, cloud management platform, and roadside perception equipment.

[0040] For reference Figure 1 Embodiment 1 of the present invention is as follows:

[0041] The improved SlowFast network includes an input layer, a two-branch feature extraction layer, a feature fusion layer, and a classification output layer;

[0042] The dual-branch feature extraction layer includes a Slow branch and a Fast branch;

[0043] The Slow branch uses ResNet-18 as the backbone network, introduces the CBAM spatial attention module after the third residual layer group of ResNet-18, and removes two residual blocks in the fourth residual layer group of ResNet-18.

[0044] In ResNet-18, the input undergoes initial 7×7 convolutions and pooling before being fed into four residual layers (Layer 1 to Layer 4). Layer 4 is located at the deepest layer of the network, with an input feature map size of 14×14 (assuming the input is 224×224), and the output size is compressed to 7×7.

[0045] Layer 4 contains two BasicBlocks (each Block contains two 3×3 convolutional layers). The first convolutional layer of the first Block uses downsampling with a stride of 2, halving the feature map size (14×14 → 7×7) and doubling the number of channels (256 → 512). The second Block does not use downsampling and only extracts features through convolution.

[0046] In this embodiment, two BasicBlocks of Layer 4 are deleted, and a total of six Blocks from Layer 1 to Layer 3 are retained. After adjustment, the total number of layers in the Slow branch is reduced from 18 to 14 (1 initial convolution layer, 2 layers / block of the six Blocks in Layers 1-3, 1 global pooling layer, and 1 fully connected layer).

[0047] The original Layer 4 output was 7×7, and after removal, the Layer 3 output is 14×14 (because the first block of Layer 3 downsamples the size from 28×28 to 14×14, and the second block does not downsample). The final feature map shape is [256, 14, 14], the number of channels is reduced by 50%, and the spatial size is increased by 100%.

[0048] Meanwhile, the two blocks of Layer 4 account for about 15% of the total parameters of ResNet-18 (about 1.2M parameters), and removing them can reduce the inference time of edge devices.

[0049] That is, based on ResNet-18, this embodiment further removes two residual blocks of Layer 4, thereby reducing the number of parameters; a CBAM spatial attention module is added after Layer 3: generating an attention map of the steering wheel region (by pre-training the steering wheel position prior with labeled data), forcing the model to focus on the contact area between the hand and the steering wheel; the convolution kernel is 3×3×3 (time×space), with a stride of (1,2,2), balancing feature extraction and resolution.

[0050] The Fast branch uses 3D MobileNetV2 as the backbone network, replaces standard convolution with Depthwise Conv3D in MobileNetV2, and introduces an SE attention module after the output layer of the last 3D inverse residual stage of MobileNetV2.

[0051] MobileNetV2 consists of an initial convolutional layer, multi-stage inverted residual blocks, and a feature output layer.

[0052] The initial convolutional layer performs 3D convolution on the input video frame sequence (e.g., [B,3,32,112,112], i.e. [batch, channel, time frame, height, width]), compressing the spatial dimension and increasing the channel dimension (e.g., from 3 channels to 16 channels), laying the foundation for feature extraction.

[0053] Multi-stage inverted residual blocks: The core feature extraction part of the network is divided into multiple "stages," each containing 1 to 4 3D inverted residual blocks. In this embodiment, after the output of the inverted residual block of the last stage, before entering Global Average Pooling (3D), an SE attention module is introduced. The features output by the last layer are high-level semantic features. Taking hands-off recognition as an example, the output features already contain key information such as the spatial relationship between the hand and the steering wheel, and the dynamic trend of hand detachment (e.g., a certain channel specifically corresponds to the "action of the hand leaving the steering wheel"). Adding SE attention at this time can accurately strengthen these channels related to hands-off recognition and suppress irrelevant channels such as dashboard noise and passenger body noise, achieving the highest efficiency.

[0054] In this embodiment, based on MobileNetV2, depthwise Conv3D is used instead of standard 3D convolution, which can reduce the number of parameters by 80%; only the last layer adds lightweight SE attention to focus on the dynamic area of ​​the hand (such as wrist movements); the temporal convolution kernel is 3×1×1 to prioritize capturing temporal dynamics, and the spatial convolution kernel is 1×3×3 to simplify the calculation.

[0055] Taking hands-off recognition as an example, the Slow branch uses ResNet-18 and introduces the CBAM spatial attention module, which can focus on key features such as the driver's hand area, while removing some residual blocks to reduce computational load while maintaining feature extraction capabilities. The Fast branch uses 3D MobileNetV2 as its backbone and replaces standard convolution with Depthwise Conv3D to achieve lightweighting. The SE attention module enhances the extraction of key dynamic features of hands-off behavior (such as the instantaneous action of the hand leaving the steering wheel). The improved dual-branch network balances recognition accuracy and real-time performance: the lightweight design adapts to the limited computing resources of roadside units and avoids recognition delays caused by model complexity (delays may cause accidents in high-speed scenarios); the optimization of the attention module and backbone network enables the model to accurately capture diverse hands-off behaviors in high-speed driving (such as different steering wheels of different car models and different hand movement angles), significantly reducing the false negative and false positive rates, and providing reliable recognition results to support subsequent warning mechanisms.

[0056] The activation function used in the dual-branch feature extraction layer is:

[0057] ;

[0058] Where x is the output of the convolutional layer; The slope of the negative interval is a learnable parameter with an initial value of 0.1, used to control the retention strength of negative features. is the nonlinear coefficient in the positive interval, and is a learnable parameter with an initial value of 0.5, which controls the nonlinear smoothness of the positive feature.

[0059] Specifically, in the Slow branch, the range of values ​​for the slope in the negative interval is expressed as follows:

[0060] ;

[0061] To preserve more spatial details in the negative space (such as the shadow on the edge of the steering wheel);

[0062] ;

[0063] The nonlinearity in the positive interval is smoother, which enhances the differentiation of subtle features (such as fingertips touching the steering wheel).

[0064] In the Fast branch, the range of values ​​for the slope in the negative interval and the nonlinear coefficient in the positive interval is expressed as follows:

[0065] ;

[0066] To reduce noise in the negative region (such as video motion blur);

[0067] ;

[0068] The nonlinearity is steeper in the positive interval, accelerating the response to rapid dynamics (such as a sudden detachment of the hand).

[0069] The proposed custom DualBranchLU activation function significantly improves the generalization ability and scene adaptability of the off-hand recognition model compared to a fixed activation function. This function dynamically adjusts features through two learnable parameters: α (slope in the negative interval) and β (nonlinear coefficient in the positive interval). In the negative interval, α controls the intensity of negative feature retention, capturing blurred or weak features in high-speed scenes (such as hand contours in low light at night or weak features when the hand is partially occluded), preventing such features from being directly discarded. In the positive interval, β controls the nonlinear smoothness, mitigating recognition bias caused by feature overexposure in extreme scenes such as strong light and reflections (e.g., when the steering wheel is directly illuminated by midday sunlight, hand features are easily confused with the background). During model training, α and β can be adaptively optimized based on diverse high-speed driving data (different driver hand movements, different steering wheel styles), enabling the model to overcome the adaptability limitations of a fixed activation function and maintain high recognition accuracy in complex and variable high-speed scenes, reducing erroneous judgments in special scenarios.

[0070] Meanwhile, the attention module uses Sigmoid as the activation function, which can generate an attention weight map of 0-1, accurately suppressing background areas (such as the dashboard) and enhancing steering wheel / hand features. At the same time, the output range is clear, which makes it easy to multiply with the original features to achieve weighted enhancement, effectively highlighting key features of the driver's hands, steering wheel and other key areas, and suppressing background interference (such as interior decorations and scenery outside the window).

[0071] In the feature fusion layer, since the feature dimensions and time steps of the two branches are different, they need to be aligned before fusion to avoid information loss.

[0072] Alignment of two branch dimensions:

[0073] Slow branch: The time step is changed from 4 to 8 and the spatial dimension is changed from 7×7 to 1×1 by adaptive 3D pooling (AdaptiveAvgPool3d((8,1,1))), and the output shape is [B,512,8,1,1].

[0074] Fast branch: The time step is changed from 32 to 8 and the spatial dimension is changed from 7×7 to 1×1 by adaptive 3D pooling (AdaptiveAvgPool3d((8,1,1))), and the output shape is [B,128,8,1,1].

[0075] In this embodiment, the time step is uniformly set to 8, matching the key time segments of the release action.

[0076] Feature splicing and optimization:

[0077] Flatten the bi-branch feature into [B, 512×8] and [B, 128×8], and concatenate them to obtain [B, 512×8 + 128×8 = 5120]. Then, use:

[0078] BatchNorm2d: Standardizes features to accelerate training convergence;

[0079] Dropout(0.3): Suppresses overfitting;

[0080] 1×1 convolution: Compresses the dimension to 256, reducing subsequent computation.

[0081] The feature fusion layer uses LeakyReLU as the activation function;

[0082] The first fully connected layer of the classification output layer uses LeakyReLU as the activation function, and the second fully connected layer uses Softmax as the activation function.

[0083] In this embodiment, the feature fusion layer uses LeakyReLU as the activation function. This addresses the "dead neuron" problem of ReLU (LeakyReLU retains a gradient of 0.1 when the input is negative, preventing gradient vanishing in deep networks). Furthermore, traffic monitoring scenarios often involve low-contrast scenes (such as at night), and LeakyReLU enhances the propagation of weak features. In other words, by using LeakyReLU, the feature fusion layer solves the problem of negative gradient vanishing, ensuring that weak features (such as partially occluded hand gestures) are not lost during the fusion of features from the Slow and Fast branches.

[0084] Taking hand-drop recognition as an example, the classification output layer only needs to determine whether the hand has been dropped (1) or not (0). The design of the classification output layer needs to be simple and output a reliable confidence score. It consists of two fully connected layers (FC) and an activation function.

[0085] First layer FC: Input 256 → Output 64, activation function is LeakyReLU(0.1);

[0086] Second layer FC: Input 64 → Output 2 (binary classification), activation function is Softmax (dim=1);

[0087] Output: [B,2], corresponding to "probability of not selling" and "probability of selling", with the maximum probability taken as the judgment result. Furthermore, only "probability of selling ≥ 0.9" is considered valid evidence (to reduce the risk of misjudgment).

[0088] In this embodiment, compared to the traditional SlowFast network, Softmax is used as the activation function in the second fully connected layer of the classification output layer. In the binary classification scenario, the sum of the probabilities output by Softmax is 1, which makes it easy to directly read the drop probability (e.g., 0.95), meeting the quantization requirements; at the same time, compared to Sigmoid, the Softmax function can more clearly distinguish between the two probabilities, avoiding ambiguous judgments due to low probabilities in both classes.

[0089] For reference Figure 2 Embodiment two of the present invention is as follows:

[0090] The application of the improved SlowFast network in highway traffic violation identification includes the following steps:

[0091] For any vehicle, acquire the basic safety messages broadcast periodically by the vehicle, parse the basic safety messages, and determine whether the vehicle is in assisted driving mode based on the parsing results.

[0092] In this embodiment, the continuous basic safety messages are parsed to obtain the control parameters therein, and the vehicle is judged to be in assisted driving state based on the control parameters.

[0093] In this embodiment, the control parameters include:

[0094] Longitudinal control parameters: speed, longitudinal acceleration, and brake pedal status.

[0095] Lateral control parameters: steer angle, turn signal, yaw rate.

[0096] Determining whether the vehicle is in assisted driving mode based on the control parameters includes:

[0097] (1) If the longitudinal acceleration falls into the first preset range and the duration reaches the first time threshold, the speed fluctuation is less than the speed threshold, and the brake pedal is not depressed, then the vehicle is considered to be in assisted driving state.

[0098] In this embodiment, if the longitudinal acceleration is within ±0.5m / s² for more than 2 seconds, the speed fluctuation is less than 2km / h, and there is no active operation of the brake pedal (brake_pedal_status is "not pressed"), then the vehicle is considered to be in assisted driving mode.

[0099] Longitudinal control is one of the core functions of Level 2 assisted driving (e.g., adaptive cruise control stably controls longitudinal acceleration and vehicle speed, eliminating the need for frequent braking). In contrast, manual driving involves frequent adjustments to acceleration based on road conditions (e.g., acceleration / deceleration during overtaking or following), and the brake pedal is used frequently with significant speed fluctuations. This system precisely matches the longitudinal operating characteristics of Level 2 assisted driving by ensuring longitudinal acceleration falls within a pre-set range and remains within the specified threshold, speed fluctuations are less than a threshold, and the brake pedal is not depressed. The duration of the condition avoids misjudgments caused by coincidental instantaneous parameter variations, and the absence of brake pedal depressing further excludes manual intervention by the driver. This judgment logic aligns with real-world driving scenarios, effectively distinguishing between assisted and manual driving, preventing the misjudgment of brief periods of stable driving in manual mode as assisted driving, improving the accuracy of pre-judgment, and providing precise data for subsequent monitoring.

[0100] (2) Perform sliding window analysis on the control parameters obtained from the parsing of the continuous basic safety messages, calculate the standard deviation of the longitudinal acceleration, and if the standard deviation of the longitudinal acceleration is less than the preset first standard deviation threshold, the vehicle is identified as being in an assisted driving state.

[0101] In this embodiment, a sliding window analysis is performed on continuous BSM data (such as the most recent 10 frames, about 1 second) to calculate the standard deviation of longitudinal acceleration. If the standard deviation of longitudinal acceleration is <0.2m / s², the vehicle is considered to be in assisted driving state.

[0102] In real-world high-speed scenarios, even when a vehicle is in assisted driving mode, slight road bumps, crosswinds, and other external factors can cause instantaneous fluctuations in longitudinal acceleration. Relying solely on parameters from a single time period can easily lead to misinterpretations of these fluctuations as manual driving. Conversely, manual driving may also exhibit brief periods of stable acceleration, which can easily be misinterpreted as assisted driving. Sliding window analysis can smooth continuous longitudinal acceleration data, reducing instantaneous interference. Standard deviation reflects the dispersion of the data; in assisted driving, the vehicle's longitudinal control is stable, resulting in a small acceleration standard deviation, while in manual driving, the acceleration dispersion is high, leading to a large standard deviation. By using the condition of "standard deviation less than a preset threshold," the essential characteristic of "long-term stability" in assisted driving can be accurately captured, rather than relying on instantaneous data. This supplementary condition is particularly suitable for complex scenarios such as high-speed sections with slight bumps and crosswinds, effectively filtering out interfering factors and ensuring the reliability of assisted driving status judgments.

[0103] (3) Obtain the lane curvature of the lane where the vehicle is located;

[0104] If the steering angle continues to fall within the second preset range, the steering angle signal is not activated, and the yaw angle data matches the theoretical yaw rate corresponding to the lane curvature, then the vehicle is considered to be in assisted driving mode.

[0105] In this embodiment, if the steering angle is periodically adjusted within ±3°; the steering angle signal is not activated (turn_signal is 0); and the yaw rate matches the lane curvature (the lane curvature radius is obtained through a high-precision map, and the theoretical yaw rate threshold is calculated), then the vehicle is considered to be in assisted driving mode.

[0106] The core functions of Level 2 assisted driving also include lane keeping (lateral control). Its steering angle remains stable within a small range (only used for fine-tuning lane position), the steering angle signal is not activated (no lane change intention), and the yaw angle matches the theoretical value corresponding to the lane curvature (the vehicle travels stably along the lane). In contrast, during manual driving, the driver frequently adjusts the steering angle (e.g., correcting direction, changing lanes). Changing lanes activates the steering angle signal, and the yaw angle is prone to mismatch with lane curvature due to manual operation deviations. By combining lateral parameters with lane curvature, the lateral control characteristics of assisted driving can be accurately identified, avoiding misjudging vehicles that are "longitudinally stable but laterally manually controlled" (e.g., maintaining speed but frequently correcting direction during manual driving) as assisted driving. It also excludes non-assisted driving scenarios where "lateral parameters are abnormal but longitudinally stable." This comprehensive judgment logic better aligns with the actual characteristics of Level 2 assisted driving's "lateral and longitudinal coordinated control," significantly improving the comprehensiveness and accuracy of assisted driving status judgment.

[0107] (4) Perform sliding window analysis on the control parameters obtained from the continuous parsing of the basic safety messages, calculate the rate of change of the steering angle, and if the rate of change of the steering angle is less than the preset rate of change threshold and the duration reaches the second time threshold, then the vehicle is identified as being in assisted driving state.

[0108] A sliding window analysis is performed on continuous BSM data (such as the most recent 10 frames, about 1 second) to capture dynamic change trends and calculate the rate of change of steering angle (Δsteer / Δt). If it is <2° / s and lasts for more than 0.5 seconds, the vehicle is considered to be in assisted driving mode.

[0109] The rate of change of steering angle directly reflects the stability of steering operation: when using assisted driving to maintain lane position, only small and slow adjustments to the steering angle are needed, resulting in a low and stable rate of change; in manual driving, even if the driver intends to maintain the lane, frequent and rapid adjustments to the steering angle may occur due to driving habits (such as small and rapid turns when manually correcting the direction), resulting in a higher rate of change. Sliding window analysis can smooth out instantaneous steering angle fluctuations (such as small changes in steering angle caused by slight road surface disturbances), avoiding misjudgments; the condition that the rate of change is less than a preset threshold and continuously reaches a second time threshold ensures the long-term stability of steering operation and eliminates coincidental situations of brief, slow steering in manual driving. For example, in high-speed curve scenarios, assisted driving will stably adjust the steering angle according to the curve curvature, resulting in a low rate of change; while when manually cornering, the driver may repeatedly make small adjustments to the steering angle, resulting in large fluctuations in the rate of change. This supplementary condition can accurately capture the stable and slow characteristics of assisted driving lateral control, further reducing the misjudgment rate and ensuring the accuracy of assisted driving state judgment.

[0110] If the vehicle is in assisted driving mode, real-time video data of the vehicle is collected by a camera, and the video data is input into a hands-free recognition model based on deep learning. The hands-free recognition model then outputs a recognition result indicating whether hands-free driving has occurred.

[0111] If the identification result indicates that hands-free driving has occurred, an early warning mechanism is triggered, controlling the nearest audible and visual warning device in front of the vehicle to provide an audible and visual reminder, and simultaneously pushing warning information to the nearest law enforcement personnel in front of the vehicle.

[0112] In this embodiment, the synergistic effect of the activation functions in each layer enables the model to stably and accurately identify various high-speed scenarios (day / night, different vehicle models), thereby improving the robustness of the overall system.

[0113] The training data for the drop-hand recognition model is generated by labeling historical surveillance video data of drop-hand behaviors that have been manually identified, characterized, and punished.

[0114] Furthermore, since traffic monitoring needs to avoid "single-frame misjudgment" (such as misidentification of hands due to lens reflection), this embodiment further introduces timing post-processing:

[0115] Timing smoothing: The majority vote is taken for the judgment results of 5 consecutive frames (about 0.5 seconds). The final output is "disengaged" only when ≥3 frames are judged as disengaged.

[0116] Visualization: Output labeled video frames—mark the steering wheel area with a red box, the hand area with a green box, and the probability of dropping hands with yellow text (e.g., "Probability of dropping hands: 0.95") for easy verification.

[0117] If the identification result indicates that hands-free driving has occurred, an early warning mechanism is triggered, controlling the nearest audible and visual warning device in front of the vehicle to provide an audible and visual reminder, while simultaneously sending a warning message to the nearest law enforcement officer in front of the vehicle.

[0118] The hands-free recognition model uses an improved SlowFast network as described in Embodiment 1 above.

[0119] Embodiment 3 of the present invention is as follows:

[0120] The application of the improved SlowFast network in highway violation identification differs from Embodiment 1 in that, in this embodiment, it considers that the on-board unit (OBU) of some vehicles can use on-board cameras to monitor physiological data such as PERCLOS eye-closing duration and yawning frequency, and embeds this data into the BSM. In this embodiment, during BSM parsing, eye-closing duration and yawning frequency are also acquired, and an additional fatigue monitoring model is set up.

[0121] The fatigue monitoring model adopts a three-stage architecture of dual-branch feature extraction, cross-attention fusion, and fatigue level prediction. It identifies whether the driver is fatigued based on image data collected by road cameras and the duration of eye closure and yawning frequency obtained from BSM.

[0122] Specifically, in the dual-branch feature extraction, the visual branch uses a simplified MobileNetV2 model to extract features from image data. The simplified MobileNetV2 model removes the last two inverted residual blocks, retains the 14×14 feature map output, replaces the standard convolution with a depthwise separable convolution, and adds facial region attention: heatmaps are generated using a pre-trained facial keypoint model (such as MTCNN) to guide the network to focus on the eyes (60% weight), mouth (30% weight), and head (10% weight).

[0123] The temporal branch uses a lightweight bidirectional LSTM (Bi-LSTM) + temporal convolution (TCN) model structure for feature extraction.

[0124] For the output data after feature extraction from the visual and temporal branches, a cross-attention mechanism is used for feature fusion. The visual features are flattened to [B, 128×14×14=25088], and then compressed to [B, 64] through a linear layer; the temporal features are [B, 16], and then expanded to [B, 64] through a linear layer. The attention weights from visual to temporal (e.g., the attention of the visual feature "mouth open" to the temporal feature "yawning frequency") and the weights from temporal to visual (e.g., the attention of the temporal feature "PERCLOS=0.7" to the visual feature "eye closed") are calculated. The weighted fusion outputs a joint feature of [B, 128].

[0125] The output layer outputs interpretable fatigue levels (0-5, 0 = alert, 5 = severe fatigue), rather than a simple binary classification, facilitating graded warnings using RSU. Structure: 2-layer fully connected network (64→32→6), using Softmax activation function, outputting a probability distribution of 6 levels; Loss functions: Focal Loss (addressing the imbalance problem due to a small number of fatigue samples) and level-ordered loss (ensuring the model understands the semantic relationship of "level 3 > level 2").

[0126] In this embodiment, the roadside unit also provides graded warnings based on the output of the fatigue monitoring model. When the fatigue level is greater than or equal to 1 and less than 3, the driver is warned of fatigue driving through the roadside audible and visual warning devices. If the fatigue level is 3 or above, it is also considered as hands-free driving (involving another definition of hands-free driving: lack of attention monitoring, where the hands are lightly touching the steering wheel but the driver has lost the "continuous perception ability" of road conditions), and the process for handling hands-free driving begins.

[0127] If the identification result indicates that hands-free driving has occurred, an early warning mechanism is triggered, controlling the nearest audible and visual warning device in front of the vehicle to provide an audible and visual reminder, while simultaneously sending a warning message to the nearest law enforcement officer in front of the vehicle.

[0128] Please refer to Figure 3 Embodiment four of the present invention is as follows:

[0129] A roadside unit 1 includes a processor 2, a memory 3, and a computer program stored in the memory 3 and executable on the processor 2. When the processor 2 executes the computer program, it implements the steps of the improved SlowFast network described in Embodiment 2 in the application of highway violation identification based on the improved SlowFast network described in Embodiment 1 through the hardware carrier of the roadside unit 1 (RSU).

[0130] The improved SlowFast network is applied to highway traffic violation identification through the hardware platform of Roadside Unit 1 (RSU), providing reliable hardware support for the implementation of the technical solution and ensuring the practical feasibility and efficiency of the monitoring function. As the core roadside device for vehicle-to-everything (V2X), Roadside Unit 1 possesses integrated capabilities of perception, communication, computing, and control. On the one hand, it can directly receive Basic Safety Messages (BSMs) broadcast by vehicles without relying on active uploads from onboard devices, avoiding driver circumvention of onboard monitoring (such as tampering with onboard data), resulting in more objective and real-time data acquisition. On the other hand, Roadside Unit 1 can directly control highway-deployed cameras (for video acquisition) and audio-visual alarm devices (for triggering on-site warnings), while simultaneously pushing warning information to nearby law enforcement personnel through a communication module, forming a complete closed loop of data acquisition—status assessment—hands-free identification—warning enforcement. Compared to solutions relying on cloud or onboard devices, the localized deployment of Roadside Units reduces data transmission latency (which can lead to untimely warnings in high-speed scenarios) and can directly link with roadside infrastructure, ensuring rapid implementation and transforming the technical solution into actual highway traffic safety monitoring capabilities.

[0131] In summary, the improved SlowFast network of this invention and its application in highway violation identification address the highway safety pain point caused by drivers' excessive reliance on assisted driving. It constructs a comprehensive regulatory system encompassing accurate screening of assisted driving status, efficient identification of hands-free driving, and dual-warning linkage enforcement, resulting in significant overall benefits. The pre-processing stage accurately distinguishes between assisted driving and manual driving by analyzing the horizontal and vertical control parameters of continuous BSMs, combined with sliding window analysis of standard deviation and rate of change, avoiding misjudgments, missed judgments, and resource waste. Hands-free driving identification employs an improved SlowFast network, with a lightweight Slow branch enhanced spatial attention and a Fast branch optimized for dynamic feature extraction, coupled with a custom DualBranchLU activation function, balancing the low computational requirements of roadside units with recognition accuracy in complex scenarios (day / night, strong light / shadow). The warning mechanism links roadside audio-visual equipment with law enforcement personnel, providing real-time alerts to drivers to correct hands-free driving and forming a closed-loop regulatory system. Local deployment of roadside units reduces data transmission latency, ensuring real-time performance throughout the entire process. The overall solution addresses issues such as lack of supervision, inaccurate identification, and untimely warnings in assisted driving, effectively curbing over-reliance and improving the safety of human-machine co-driving on highways.

[0132] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A high-speed road illegal behavior recognition method based on an improved SlowFast network, characterized in that, The method comprises the steps of: For any vehicle, obtaining a basic safety message periodically broadcast by the vehicle, parsing the basic safety message, and determining whether the vehicle is in an assisted driving state according to the parsing result; If the vehicle is in the assisted driving state, collecting real-time video data of the vehicle through a camera, and inputting the video data into a pre-trained hands-off driving recognition model to output a recognition result of whether there is hands-off driving; If the recognition result is that there is hands-off driving, triggering a warning mechanism to control a sound and light warning device near the front of the vehicle to sound and light, and pushing warning information to law enforcement personnel near the front of the vehicle; The improved SlowFast network comprises an input layer, a double-branch feature extraction layer, a feature fusion layer, and a classification output layer. The double-branch feature extraction layer comprises a Slow branch and a Fast branch, and an attention mechanism is introduced: The Slow branch uses ResNet-18 as a backbone network, introduces a CBAM spatial attention module after the third residual layer group of ResNet-18, and removes two residual blocks in the fourth residual layer group of ResNet-18; The Fast branch uses three-dimensional MobileNetV2 as a backbone network, uses DepthwiseConv3D instead of standard convolution in MobileNetV2, and introduces an SE attention module after the output layer of the last 3D reverse residual stage of MobileNetV2; The activation function used by the double-branch feature extraction layer is: ； where x is the output of the convolution layer; is the negative interval slope, is a learnable parameter with an initial value of 0.1, used to control the retention strength of the negative features; is the positive interval nonlinear coefficient, is a learnable parameter with an initial value of 0.5, used to control the nonlinear smoothness of the positive features.

2. The method of claim 1, wherein the improved SlowFast network-based expressway illegal behavior recognition method is characterized in that, In the Slow branch of the double-branch feature extraction layer, the value range of the negative interval slope of the activation function DualBranchLU is [0.1, 0.3], and the value range of the positive interval nonlinear coefficient is [0.3, 0.7].

3. The method of claim 1, wherein the method is based on an improved SlowFast network. In the Fast branch of the double-branch feature extraction layer, the value range of the negative interval slope of the activation function DualBranchLU is [0.05, 0.1], and the value range of the positive interval nonlinear coefficient is [0.8, 1.2].

4. The method of claim 1, wherein the method is based on an improved SlowFast network. The feature fusion layer uses LeakyReLU as an activation function; The first full connection layer of the classification output layer uses LeakyReLU as an activation function, and the second full connection layer uses Softmax as an activation function.

5. The method of claim 1, wherein the method is based on an improved SlowFast network. Parsing the basic safety message and determining whether the vehicle is in an assisted driving state according to the parsing result comprises: Parsing the continuous basic safety message, obtaining the control parameters therein, and determining whether the vehicle is in an assisted driving state according to the control parameters.

6. The method of claim 5, wherein the improved SlowFast network-based expressway illegal behavior recognition method is characterized in that, The control parameters include longitudinal acceleration, speed, and brake pedal state; Determining whether the vehicle is in an assisted driving state according to the control parameters comprises: If the longitudinal acceleration falls within a first preset interval and the duration reaches a first time threshold, the speed fluctuation is less than a speed threshold, and the brake pedal state is not depressed, the vehicle is determined to be in an assisted driving state.

7. The method of claim 6, wherein the improved SlowFast network-based expressway illegal behavior recognition method is characterized in that, The judgment of whether the vehicle is in the assisted driving state according to the control parameter further comprises: The control parameter obtained by analyzing the continuous basic safety messages is subjected to sliding window analysis, the standard deviation of the longitudinal acceleration is calculated, and if the standard deviation of the longitudinal acceleration is less than a preset first standard deviation threshold, it is determined that the vehicle is in the assisted driving state.

8. The high-speed road illegal behavior recognition method based on the improved SlowFast network according to claim 5, characterized in that, The control parameter comprises a steering angle signal, yaw angle data and steering angle data; The judgment of whether the vehicle is in the assisted driving state according to the control parameter comprises: The lane curvature of the lane where the vehicle is located is obtained; If the steering angle continuously falls into the second preset interval, the steering angle signal is not activated, and the yaw angle data matches the theoretical yaw angle speed corresponding to the lane curvature, it is determined that the vehicle is in the assisted driving state.

9. The method of claim 8, wherein the improved SlowFast network-based highway illegal behavior recognition method is characterized in that, The judgment of whether the vehicle is in the assisted driving state according to the control parameter further comprises: The control parameter obtained by analyzing the continuous basic safety messages is subjected to sliding window analysis, the rate of change of the steering angle is calculated, and if the rate of change of the steering angle is less than a preset rate of change threshold and the duration reaches a second time threshold, it is determined that the vehicle is in the assisted driving state.

Images