Vehicle illegal behavior identification method based on computer vision

Abstract

The invention relates to the technical field of computer vision recognition, particularly discloses a vehicle illegal behavior recognition method based on computer vision, and aims to solve the problems that a traditional method is low in recognition precision, high in false alarm rate and difficult to model dynamic illegal behaviors in a complex traffic scene. The method comprises the following steps: acquiring a high-frame-rate high-definition video stream and performing time synchronization; vehicle detection and trajectory generation are realized through an improved YOLOv7 model and a multi-hypothesis tracking algorithm; fusing the vehicle track, the lane attribution, the signal lamp state and the road topology to construct a multi-dimensional space-time behavior feature map; performing end-to-end illegal behavior discrimination by using a space-time convolutional neural network and gating cycle unit fusion model; and automatically generating a structured illegal evidence packet with a digital signature. According to the technical scheme, high-precision recognition of behaviors such as red light running, line pressing and illegal lane changing can be achieved under the conditions of dense traffic flow, shielding and complex illumination, and the integrity of law enforcement data and the real-time performance of the system are remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a vehicle illegal behavior recognition method based on computer vision BACKGROUND

[0002] With the accelerated promotion of intelligent transportation systems and the digitalization of urban governance, the automatic recognition technology of vehicle illegal behavior based on computer vision has become a key support for improving the efficiency of traffic law enforcement and the level of road safety. This technology relies on video monitoring, image analysis and artificial intelligence algorithms, aiming to detect and collect evidence on typical illegal behaviors such as running red lights, line driving, illegal lane changing, and occupying emergency lanes in real time. The core lies in accurately extracting the vehicle motion trajectory, position state and behavior semantics from complex dynamic traffic scenes, and then realizing high-precision and low-false alarm illegal judgment.

[0003] Among them, the recognition of vehicle illegal behavior highly depends on the accurate modeling of spatio-temporal continuity and the fusion analysis of multi-dimensional behavior features in the traffic scene. The ideal solution needs to stably track the target vehicle and analyze its driving intention and operation compliance under complex conditions such as dense traffic, obstruction interference, light change and weather disturbance. However, the existing technology still faces multiple bottlenecks in practical application.

[0004] The existing technology generally adopts detection logic based on rule threshold or shallow visual features, which is difficult to effectively model the dynamic evolution process of vehicle behavior, resulting in low recognition accuracy for continuous illegal behavior (such as long-time line driving or gradual lane changing). At the same time, the mainstream method relies on single perspective or static image frame for judgment, lacks deep integration of multi-frame time sequence information and spatial context relationship, and is easily affected by instantaneous obstruction or image blur interference, resulting in misjudgment. In addition, the existing system usually decouples the detection and recognition modules for processing, and fails to realize end-to-end behavior semantic understanding, making it difficult to distinguish between legal avoidance and real illegal operation in complex intersection or high-density traffic scenes, which seriously restricts the stability and generalization ability of the system in real traffic environment.

[0005] Therefore, there is an urgent need for a vehicle illegal behavior recognition method that can deeply integrate spatio-temporal context, adapt to complex scene changes and have end-to-end behavior discrimination ability. SUMMARY

[0006] The purpose of the present application is to provide a vehicle illegal behavior recognition method based on computer vision, which can effectively solve the problems in the background art.

[0007] To achieve the above purpose, the technical solution adopted by the present application is as follows: A vehicle illegal behavior recognition method based on computer vision, comprising the following specific steps: Step 1: Collect traffic monitoring video stream data, continuously acquire video frame sequences containing vehicle dynamic behavior through high-definition video monitoring equipment deployed at road intersections or key road sections, video frame rate not less than 25 frames per second, resolution not less than 1920 by 1080 pixels, and perform timestamp synchronization and frame alignment processing on the original video stream; Step 2: Perform multi-target vehicle detection and tracking, use a deep learning-based target detection model to locate and classify vehicles in each video frame, output vehicle bounding boxes and confidence scores, and combine an improved multi-hypothesis tracking algorithm to associate the detection results across frames, generating the motion trajectory of each vehicle within a continuous time window; Step 3: Construct a spatio-temporal behavior feature map, based on the vehicle motion trajectory, extract spatial position sequences, speed change curves, acceleration vectors, and lane attribution states, and fuse traffic signal light states, lane line geometry, and road topology information to construct a multi-dimensional behavior feature map containing spatio-temporal context constraints; Step 4: Implement end-to-end illegal behavior semantic discrimination, input the multi-dimensional behavior feature map into a pre-trained spatio-temporal convolutional neural network and gated recurrent unit fusion model, which learns spatial local features and temporal evolution patterns jointly, and outputs probability distributions for red light running, lane weaving, illegal lane changing, and emergency lane occupation; Step 5: Generate structured illegal evidence packages, based on the illegal behavior discrimination results, automatically intercept video clips, key frame images, vehicle trajectory graphs, and behavior feature parameters for the corresponding time period, package them as structured evidence data packages that meet traffic law enforcement standards, and add digital signatures and timestamps to ensure data integrity and legal effectiveness.

[0008] Preferably, the high-definition video monitoring equipment in step 1 uses a global shutter CMOS image sensor with a wide dynamic range of not less than 120 decibels, which can stably output clear video streams in strong backlight, low light at night, and rainy and foggy weather conditions.

[0009] Preferably, the deep learning target detection model in step 2 is an improved YOLOv7 architecture, which introduces a deformable convolution module into the backbone network to enhance the feature extraction capability for occluded vehicles, and uses an adaptive anchor box clustering strategy in the detection head part to optimize the bounding box prediction accuracy for common vehicle types on urban roads.

[0010] Preferably, the improved multi-hypothesis tracking algorithm in step 2 introduces a trajectory confidence decay mechanism, starts trajectory prediction compensation when a target is not detected for 3 consecutive frames, and combines Kalman filtering to smooth the estimation of vehicle motion state, ensuring that the trajectory continuity is not less than 95% in the case of temporary occlusion or image blur.

[0011] Preferably, the lane attribution state in step 3 is obtained by a real-time lane line detection module, which uses a lightweight semantic segmentation network to perform pixel-level classification on road images, outputs the accurate boundaries of each lane in each frame of image, and maps the vehicle position to the bird's eye view coordinate system through perspective transformation to eliminate the influence of perspective distortion.

[0012] Preferably, the traffic signal light state information in step 3 is obtained through data interface docking with the traffic signal control system, or in the absence of interface, it is analyzed in real time by an independent signal light recognition sub-module based on color space threshold and shape template matching, with an accuracy of not less than 98%.

[0013] Preferably, the spatio-temporal convolutional neural network in step 4 includes 4 three-dimensional convolutional layers, each with a kernel size of 3x3x3, for extracting motion patterns within local spatio-temporal cubes, and a gated recurrent unit hidden layer with a dimension of 256 for modeling long-term temporal dependencies, both of which realize cross-modal information interaction through feature concatenation and attention weight fusion.

[0014] Preferably, the pre-trained model in step 4 is trained end-to-end on a dataset containing more than 500,000 labeled trajectories, covering 12 typical traffic scenarios such as urban trunk roads, expressways, school areas, and construction sections, with a balanced sampling ratio of not less than 1:3 for illegal behavior samples.

[0015] Preferably, the structured illegal evidence package in step 5 includes vehicle license plate recognition results, illegal behavior type code, illegal behavior occurrence time, geographic location coordinates, lane number, and system self-check log, all of which are packaged in JSON format and digitally signed using the SM2 algorithm.

[0016] Preferably, it also includes confidence threshold filtering of the identified illegal behavior, triggering the evidence generation process only when the illegal behavior probability is greater than or equal to 0.9, and labeling suspected cases with a confidence between 0.7 and 0.9 as manual review status for secondary confirmation by background audit personnel.

[0017] Preferably, it also includes establishing an illegal behavior recognition performance feedback loop, returning the manual review results and on-site law enforcement confirmation data to the training system for dynamically updating the parameters of the spatio-temporal convolutional neural network and gated recurrent unit fusion model, with an online fine-tuning period of not more than 7 days.

[0018] Preferably, the method is deployed on an edge computing device equipped with a 4-core ARM processor and a dedicated neural network acceleration chip, with a single-channel video stream processing delay of less than 200 milliseconds, supporting simultaneous processing of 8 channels of 1080P video streams, and a total power consumption of less than 30 watts.

[0019] To sum up, the present application includes at least one of the following beneficial technical effects: The present application realizes high-precision and low-false alarm recognition of vehicle illegal behavior by constructing an end-to-end spatio-temporal behavior feature map and a deep fusion neural network discrimination model. Compared with traditional methods that rely on single frame images or simple rule thresholds, the present application can effectively model the dynamic evolution process of vehicle behavior, and still maintain stable recognition performance under dense traffic, instantaneous occlusion and complex lighting conditions. By multi-dimensionally fusing vehicle trajectory, lane attribution, signal light state and road topological information, the discrimination ability for continuous illegal behavior (such as gradual line pressing or hidden lane changing) is significantly improved. At the same time, the automatic generation of structured evidence bags and the digital signature mechanism ensure the integrity and legal effect of law enforcement data, greatly reducing the cost of manual review. The edge deployment scheme further enhances the real-time performance and scalability of the system, and is suitable for the practical application requirements of large-scale urban intelligent transportation systems. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is the overall technical scheme architecture schematic diagram of the vehicle illegal behavior recognition method based on computer vision proposed by the present application; Figure 2 is the core principle framework schematic diagram of end-to-end illegal behavior semantic discrimination in the present application; Figure 3 is the logic flow framework diagram of multi-target vehicle detection and tracking and spatio-temporal behavior feature map construction in the present application; DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the present application more clear and explicit, the present application is further described in detail below in combination with specific embodiments.

[0022] Currently, in the background of the accelerated promotion of intelligent transportation systems and digitalization of urban governance, the automatic recognition technology of vehicle illegal behavior based on computer vision faces the technical bottleneck of being difficult to stably track target vehicles and accurately analyze their driving intention and operation compliance under complex conditions such as dense traffic, occlusion interference, lighting changes and weather disturbances. The existing technology generally adopts detection logic based on rule thresholds or shallow visual features, lacks effective modeling of the dynamic evolution process of vehicle behavior, and relies on a single perspective or static image frame for judgment, which fails to realize end-to-end behavior semantic understanding, resulting in high misjudgment rate and poor stability in complex intersection or high-density traffic scenes.

[0023] In view of the above technical problems, the present application proposes a technical scheme for realizing high-precision and low-false alarm recognition of vehicle illegal behavior by constructing an end-to-end spatio-temporal behavior feature map and a deep fusion neural network discrimination model, and is applied to a vehicle illegal behavior recognition method based on computer vision.

[0024] Reference Figure 1 The overall technical scheme architecture of the present application includes a video stream acquisition unit, a multi-target vehicle detection and tracking unit, a space-time behavior feature map construction unit, an end-to-end illegal behavior semantic discrimination unit, and a structured illegal evidence package generation unit. Each unit works cooperatively to form a complete closed-loop processing link from the original video input to the structured law enforcement evidence output. Specifically, the method includes the following five core steps, each step is tightly coupled with the data bus and control logic inside the system to ensure the real-time and consistency of the processing flow.

[0025] In the above vehicle illegal behavior recognition method, step 1, collect traffic monitoring video stream data, continuously acquire video frame sequences containing vehicle dynamic behavior through high-definition video monitoring devices deployed at road intersections or key road sections, the video frame rate is not less than 25 frames per second, the resolution is not less than 1920 by 1080 pixels, and the original video stream is timestamped and frame-aligned.

[0026] Specifically, the high-definition video monitoring device uses a global shutter CMOS image sensor with a wide dynamic range function, with a dynamic range of not less than 120 decibels, which can stably output clear video stream under strong backlight, night low illumination and rain and fog weather conditions.

[0027] During the video stream acquisition process, the system has a built-in hardware-level timestamp generator that adds nanosecond-level precision timestamp information to each frame of image, ensuring strict time sequence alignment of multiple video sources in subsequent processing.

[0028] The frame alignment process uses an affine transformation model based on feature point matching, selects stable feature points such as road markings and fixed references as alignment references, calculates the geometric transformation matrix between adjacent frames, and performs pixel-level resampling on the current frame to eliminate the frame displacement caused by camera slight jitter or high-speed vehicle motion. This processing ensures that the spatial coordinate system relied on by subsequent trajectory analysis remains continuous and consistent in the time dimension, laying the foundation for high-precision behavior modeling.

[0029] In the above vehicle illegal behavior recognition method, step 2, multi-target vehicle detection and tracking is performed, a deep learning-based target detection model is used to locate and classify vehicles in each video frame, outputting vehicle bounding boxes and confidence scores, and an improved multi-hypothesis tracking algorithm is used to associate the detection results across frames, generating the motion trajectory of each vehicle within a continuous time window.

[0030] Specifically, the deep learning target detection model is an improved YOLOv7 architecture, the backbone network of which introduces a deformable convolution module to enhance the feature extraction capability for partially occluded vehicles, and the detection head part adopts an adaptive anchor box clustering strategy to optimize the boundary box prediction accuracy for common vehicle types on urban roads.

[0031] The deformable convolution module dynamically adjusts the sampling position of the convolution kernel by learning the offset field, so that it can focus on the visible area of the partially occluded vehicle, thereby improving the stability of feature expression.

[0032] The adaptive anchor box clustering strategy dynamically generates an optimal anchor box size set based on the historical detection data of the local traffic scene using the K-means++ algorithm, ensuring that the boundary box prediction for different vehicle types such as sedans, SUVs, and trucks has the smallest regression error. The boundary box information output by the detection model includes center coordinates, width and height dimensions, class labels, and confidence scores. Detection results with confidence scores lower than 0.5 are directly filtered out to reduce the computational burden and misassociation risk of the subsequent tracking module.

[0033] Further, the improved multi-hypothesis tracking algorithm introduces a trajectory confidence decay mechanism. When a target is not detected for 3 consecutive frames, trajectory prediction compensation is started, and Kalman filtering is used to smooth the estimation of the vehicle's motion state, ensuring that the trajectory continuity is not less than 95% in the case of temporary occlusion or image blur.

[0034] The algorithm maintains a trajectory hypothesis pool, each hypothesis corresponding to a potential target identity. At each frame processing, the algorithm calculates the association cost matrix between all detection results and existing trajectory hypotheses. The cost function considers the boundary box overlap, appearance feature similarity, and motion state consistency.

[0035] The optimal matching is solved by the Hungarian algorithm, and new trajectories are initialized for unmatched detection results, while confidence decay is performed for unmatched trajectory hypotheses. The initial confidence value is 1.0, and it is multiplied by the decay factor 0.8 if it is not updated for each frame. When the confidence is lower than 0.3 and there is no match for 3 consecutive frames, the trajectory is terminated. If a match is re-acquired before the confidence drops to 0.3, the confidence is restored to 0.9.

[0036] For trajectories with confidence between 0.3 and 0.9, the system starts the Kalman filter predictor, which predicts the target's position in the next frame based on the uniform speed or uniform acceleration motion model, and uses the predicted position as a virtual detection point for subsequent association, effectively maintaining the continuity of the trajectory during occlusion. The state vector of the Kalman filter includes position, velocity, and acceleration components, and the process noise covariance is dynamically adjusted according to the road type, with a value of 0.1 for urban roads and 0.05 for highways, to adapt to different driving behavior patterns.

[0037] ReferenceFigure 3 The figure shows the logical flow framework of multi-target vehicle detection and tracking and spatio-temporal behavior feature map construction. In Figure 3 , the video frame sequence is input into the detection module, and the detection frame is output. After processing by the tracking module, the trajectory sequence is generated, and then input into the feature extraction module, and finally the spatio-temporal behavior feature map is constructed.

[0038] In the above vehicle illegal behavior recognition method, step 3, constructing a spatio-temporal behavior feature map, based on vehicle motion trajectory, extracting spatial position sequence, speed change curve, acceleration vector and lane attribution state, and fusing traffic signal light state, lane line geometry and road topology information, constructing a multi-dimensional behavior feature map containing spatio-temporal context constraints. Specifically, the lane attribution state is obtained by a real-time lane line detection module. This module uses a lightweight semantic segmentation network to perform pixel-level classification on road images, outputs the accurate boundaries of each lane in each frame of image, and maps the vehicle position to the bird's eye view coordinate system through perspective transformation to eliminate the influence of perspective distortion.

[0039] The lightweight semantic segmentation network is based on MobileNetV3 backbone, followed by ASPP module to capture multi-scale context information, and the output channel number is lane class number plus 1 (background class). The network uses a mixed strategy of synthetic data and real data in the training stage. Synthetic data is generated by a game engine, covering various lighting, weather and road wear conditions. Real data comes from urban road collection. The perspective transformation parameters are estimated online through calibration board or lane geometry constraints. The vehicle center point in the image coordinate system is projected to the bird's eye view coordinate system, so that lane attribution judgment only needs to perform simple coordinate interval comparison, greatly improving the calculation efficiency and accuracy.

[0040] In addition, the traffic signal light state information is obtained by data interface docking with the traffic signal control system, or in the absence of interface conditions, it is analyzed in real time by an independent signal light recognition sub-module based on color space threshold and shape template matching, and the state recognition accuracy is not less than 98%.

[0041] When there is a standard RS485 or TCP / IP interface, the system directly subscribes to the signal light phase state data stream to obtain the accurate switching time of the red, yellow and green three-color lights. In the absence of an interface, the signal light recognition sub-module first locates the signal light region in the image, uses Haar-like features and AdaBoost classifier for rough positioning, and then performs fine recognition in the candidate region.

[0042] The recognition process is performed in the HSV color space. The threshold for the red channel is set to H∈[0,10]∪[170,180], S≥0.5, V≥0.3; the threshold for the green channel is H∈[40,80], S≥0.4, V≥0.3. Simultaneously, circular template matching is used to verify the shape of the LED beads, eliminating interference sources such as billboards and vehicle lights. The system maintains a state machine that votes on the recognition results of five consecutive frames. The traffic light state is updated only when the number of unanimous votes exceeds four, effectively suppressing momentary misidentification.

[0043] The spatiotemporal behavior feature map has a data structure of a three-dimensional tensor with dimensions T×F×C, where T is the time window length (usually 30 frames, corresponding to 1.2 seconds), F is the number of feature fields (usually 16), and C is the number of feature channels (usually 1). The feature vector at each time step t includes: the vehicle's x and y coordinates in the bird's-eye view coordinate system; velocity magnitude and direction angle; acceleration magnitude and direction angle; the current lane number; the lateral distance to the left and right lane lines; the current traffic light status code (0=red, 1=yellow, 2=green); the road type code (0=urban road, 1=highway, 2=school zone); and the coordinate sequence of historical trajectory points (used to capture movement trends). This feature map serves as the sole input to the subsequent discriminative model, fully encapsulating the spatiotemporal context information of vehicle behavior.

[0044] In the above-mentioned vehicle violation identification method, step 4 involves performing end-to-end semantic discrimination of violations by inputting the multi-dimensional behavioral feature map into a pre-trained spatiotemporal convolutional neural network and gated recurrent unit fusion model. This model outputs the probability distribution for violations such as running red lights, driving over the line, illegal lane changes, and occupying emergency lanes by jointly learning spatial local features and temporal evolution patterns.

[0045] Specifically, the spatiotemporal convolutional neural network contains four three-dimensional convolutional layers, each with a kernel size of 3 x 3 x 3, used to extract motion patterns within a local spatiotemporal cube. The gated recurrent unit hidden layer has a dimension of 256 and is used to model long-term temporal dependencies. The two achieve cross-modal information interaction through feature concatenation and attention weight fusion.

[0046] The architecture of the fusion model is designed as follows: the input T×F×C feature map is first reshaped into a four-dimensional tensor of T×1×F×C and then fed into a spatiotemporal convolutional neural network.

[0047] The first 3D convolutional layer has 32 output channels, the second layer has 64, the third layer has 128, and the fourth layer has 256. Each layer is followed by batch normalization and a ReLU activation function. The 3D convolutional kernel slides along the time dimension, enabling it to simultaneously capture joint patterns of spatial features (such as lane position) and temporal dynamics (such as speed changes), such as the typical red-light running behavior pattern of "continuously moving forward while the light is red." The output of the spatiotemporal convolutional neural network is globally averaged into a 256-dimensional spatiotemporal feature vector.

[0048] Simultaneously, the original feature maps are directly input into a single-layer gated recurrent unit (GRU) with a hidden state dimension of 256. The GRU selectively memorizes or forgets historical information through update and reset gate mechanisms, making it particularly adept at modeling long-distance temporal dependencies, such as the gradual lateral movement trajectory of a vehicle before a lane change. The hidden state of the GRU at the last time step is extracted as a 256-dimensional temporal feature vector.

[0049] The two 256-dimensional feature vectors are concatenated into a 512-dimensional fused feature vector. This fused feature vector is then weighted using a learnable attention weight module.

[0050] The attention weighting module consists of two fully connected layers with tanh activation function in between and softmax output layers, generating two weight coefficients that apply to spatiotemporal and temporal features respectively. The final weighted feature vector is fed into a fully connected classification layer with four output nodes, each corresponding to a violation (running a red light, crossing the line, illegal lane change, or using the emergency lane), and the probability distribution is output via a softmax function. Model training uses cross-entropy loss and Focal Loss weighting for violation categories to mitigate the imbalance between positive and negative samples. Specifically: Where pt is the predicted class probability by the model, αt is the class balance factor, and γ is the focusing parameter, set to 2.0. This loss function makes the model pay more attention to samples that are difficult to classify during training, improving its ability to distinguish boundary cases.

[0051] Furthermore, the pre-trained model was trained end-to-end on a dataset containing over 500,000 labeled trajectories. The dataset covers 12 typical traffic scenarios, including urban main roads, highways, school zones, and construction zones, with a balanced sampling ratio of at least 1:3 for illegal behavior samples. The training process used the Adam optimizer, with an initial learning rate of 0.001, decaying to 0.9 every 10 epochs. The model achieved a mAP of 92.5% on the validation set, and the recall rate for the four main types of illegal behaviors all exceeded 90%.

[0052] In the above-mentioned method for identifying vehicle violations, step 5 generates a structured violation evidence package. Based on the violation judgment result, video clips, key frame images, vehicle trajectory maps, and behavioral feature parameters for the corresponding time period are automatically extracted and packaged into a structured evidence data package that conforms to traffic law enforcement standards. A digital signature and timestamp are added to ensure data integrity and legal validity.

[0053] Specifically, the structured violation evidence package includes vehicle license plate recognition results, violation type code, precise time of violation, geographical coordinates, lane number, and system self-check log. All data items are encapsulated in JSON format and digitally signed using the national cryptographic SM2 algorithm.

[0054] The evidence generation process is strictly controlled by a confidence threshold. The evidence generation process is only triggered when the probability of illegal behavior is greater than or equal to 0.9, and suspected cases with a confidence level between 0.7 and 0.9 are marked as requiring manual review and are subject to secondary confirmation by back-end reviewers.

[0055] For cases that are triggered, the system automatically extracts the original video clip from 2 seconds before the violation occurs to 3 seconds after it occurs, maintaining a resolution of 1920 x 1080 and a frame rate of 25 frames per second. The keyframe image selects the frame corresponding to the peak probability of the violation and overlays visual annotations such as vehicle bounding boxes, trajectory lines, lane lines, and traffic light status.

[0056] The vehicle trajectory map is generated in SVG format, clearly showing the vehicle's movement path and lane relationship in a bird's-eye view coordinate system. All data items are organized according to a predefined JSON schema, including a metadata header, main evidence data, and a digital signature. The digital signature uses the national cryptographic algorithm SM2, and the private key is stored in a hardware security module to ensure that the signing process is unforgeable and non-repudiable. The timestamp is provided by an NTP server synchronized by the National Time Service Center, with millisecond-level accuracy.

[0057] In addition, the system includes establishing a closed-loop feedback mechanism for illegal behavior recognition performance. Manual review results and on-site enforcement confirmation data are fed back to the training system to dynamically update the parameters of the spatiotemporal convolutional neural network and gated recurrent unit fusion model. The online fine-tuning cycle of the model does not exceed 7 days. After cleaning and labeling, the feedback data is added to the incremental training set. Knowledge distillation technology is used to specifically correct misjudgments while maintaining the original model performance. The fine-tuning process is performed on the backup computing unit of the edge device, without affecting the real-time performance of the main business process.

[0058] The method is deployed on an edge computing device equipped with a quad-core ARM processor and a dedicated neural network acceleration chip. The single-channel video stream processing latency is less than 200 milliseconds, supporting simultaneous processing of eight 1080P video streams, with a total power consumption of less than 30 watts. The dedicated neural network acceleration chip is hardware-optimized for 3D convolution and gated recurrent unit operations, providing 4 TOPS of INT8 computing power to ensure model inference is completed within 20 milliseconds. Video decoding, detection, tracking, and feature extraction modules are all pipelined optimized to achieve overlap between computation and I / O, maximizing hardware resource utilization.

[0059] refer to Figure 2 This diagram illustrates the core principle framework for semantic discrimination of end-to-end illegal behavior. Figure 2 In this process, the spatiotemporal behavioral feature map is used as input and fed into the spatiotemporal convolutional neural network and the gated recurrent unit respectively. The outputs of the two are fused by attention and then output by the classifier as the probability distribution of illegal behavior.

[0060] The above embodiments illustrate in detail the core technical solution of the present invention and its specific implementation details. Through extreme engineering development, the feasibility of the technical solution and the rigor of the legal text are ensured.

[0061] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects.

[0062] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for identifying vehicle violations based on computer vision, characterized in that: The specific steps include the following: Step 1: Collect traffic monitoring video stream data. Continuously acquire video frame sequences containing vehicle dynamic behavior through high-definition video monitoring equipment deployed at road intersections or key road sections, and perform timestamp synchronization and inter-frame alignment processing on the original video stream. Step 2: Perform multi-target vehicle detection and tracking. Use a deep learning-based target detection model to locate and classify vehicles in each video frame, output vehicle bounding boxes and confidence scores, and combine an improved multi-hypothesis tracking algorithm to correlate the detection results across frames to generate the motion trajectory of each vehicle within a continuous time window. Step 3: Construct a spatiotemporal behavioral feature map. Based on the vehicle motion trajectory, extract the spatial position sequence, velocity change curve, acceleration vector and lane affiliation status, and integrate traffic light status, lane line geometry and road topology information to construct a multidimensional behavioral feature map containing spatiotemporal context constraints. Step 4: Perform end-to-end semantic discrimination of illegal behaviors. Input the multi-dimensional behavioral feature map into the pre-trained spatiotemporal convolutional neural network and gated recurrent unit fusion model. This model outputs the probability distribution of illegal behaviors such as running red lights, driving over the line, illegal lane changing and occupying emergency lane by jointly learning spatial local features and temporal evolution patterns. Step 5: Generate a structured evidence package of violations. Based on the results of the violation judgment, automatically extract video clips, key frame images, vehicle trajectory maps and behavioral feature parameters for the corresponding time period, package them into a structured evidence data package that conforms to traffic law enforcement standards, and attach digital signatures and timestamps to ensure data integrity and legal validity.

2. The method for identifying vehicle violations based on computer vision according to claim 1, characterized in that: The high-definition video surveillance equipment uses a global shutter CMOS image sensor, which has a wide dynamic range function with a dynamic range of not less than 120 decibels.

3. The method for identifying vehicle violations based on computer vision according to claim 1, characterized in that: The deep learning-based object detection model is an improved YOLOv7 architecture, which introduces deformable convolutional modules into its backbone network and adopts an adaptive anchor box clustering strategy in the detection head.

4. The method for identifying vehicle violations based on computer vision according to claim 1, characterized in that: The improved multi-hypothesis tracking algorithm introduces a trajectory confidence decay mechanism, which initiates trajectory prediction compensation when no target is detected for three consecutive frames, and combines Kalman filtering to smoothly estimate the vehicle's motion state.

5. The method for identifying vehicle violations based on computer vision according to claim 1, characterized in that: The lane ownership status is obtained through a real-time lane line detection module. This module uses a lightweight semantic segmentation network to perform pixel-level classification of road images and maps vehicle positions to the bird's-eye view coordinate system through perspective transformation.

6. The method for identifying vehicle violations based on computer vision according to claim 1, characterized in that: The traffic signal status information is obtained through data interface with the traffic signal control system, or by a separate signal recognition submodule that performs real-time analysis based on color space thresholds and shape template matching.

7. The method for identifying vehicle violations based on computer vision according to claim 1, characterized in that: The spatiotemporal convolutional neural network contains four three-dimensional convolutional layers, each with a kernel size of 3 x 3 x 3; the gated recurrent unit has a hidden layer dimension of 256, and the two achieve cross-modal information interaction through feature concatenation and attention weight fusion.

8. The method for identifying vehicle violations based on computer vision according to claim 1, characterized in that: The structured violation evidence package includes vehicle license plate recognition results, violation type code, precise time of violation, geographical coordinates, lane number, and system self-check log. All data items are encapsulated in JSON format and digitally signed using the national cryptographic SM2 algorithm.

9. The method for identifying vehicle violations based on computer vision according to claim 1, characterized in that: Identified illegal activities are filtered using a confidence threshold. The evidence generation process is triggered only when the probability of the illegal activity is greater than or equal to 0.9, and suspected cases with a confidence level between 0.7 and 0.9 are marked as requiring manual review.

10. The method for identifying vehicle violations based on computer vision according to claim 1, characterized in that: Establish a closed-loop feedback mechanism for illegal behavior identification performance, and feed back the results of manual review and on-site law enforcement confirmation data to the training system to dynamically update the parameters of the spatiotemporal convolutional neural network and gated recurrent unit fusion model. The online fine-tuning cycle of the model does not exceed 7 days.

Images