Traffic video edge analysis system based on Yolov5-Edge and Prewitt algorithms

Abstract

The invention discloses a traffic video edge analysis system based on Yolov5-Edge and Prewitt algorithms, and the system achieves the real-time collection, intelligent recognition and encrypted transmission of a vehicle-mounted video through the construction of a layered architecture of an object-side equipment layer, an edge calculation layer and a cloud service layer. According to the system, a lightweight Yolov5-Edge model and a Prewitt edge detection algorithm are fused in edge equipment, multi-channel video parallel processing is realized under finite computing power, and the video inspection efficiency is effectively improved; the frame rate and the number of processing paths are dynamically adjusted through the resource monitoring and load control module, and the real-time performance and stability of AI reasoning are guaranteed; a data management and encryption mechanism is adopted, only abnormal data are stored at an edge end and clouded through RSA and AES encryption modes, and security and compliance of sensitive behavior information in the transmission and storage process are ensured. The system can identify and classify abnormal behaviors of drivers and passengers without depending on a high-performance server.

Description

Technical Field

[0001] This application relates to the field of road traffic safety technology, and in particular to a traffic video edge analysis system based on the YOLOv5-Edge and Prewitt algorithms. Background Technology

[0002] As road traffic safety management evolves towards "full-area visibility and real-time early warning," video AI inspection based on vehicle cameras is gradually becoming an important means of detecting abnormal driver and passenger behavior (such as smoking, using mobile phones, and not wearing seat belts) and reducing accident risks. However, the large number of operating vehicles, the complex driving environment (mountainous areas, cities, and various operating conditions), and the susceptibility of network and camera posture fluctuations have led to technical requirements for high concurrency, strong real-time performance, and high reliability in the acquisition, transmission, and processing of in-vehicle video. This urgently necessitates the support of lightweight edge analysis systems for in-vehicle scenarios.

[0003] There are two main types of solutions for existing traffic video inspection: one relies on traditional edge computing servers, which, although capable of multi-channel processing, are large in size and power consumption, making them unsuitable for vehicle-mounted or mobile inspections; the other uses lightweight edge devices (embedded chips, edge boxes), but common problems include AI models and computing power scheduling not being adapted to traffic scenarios, limited parallel processing capacity per device (usually only 2-3 channels), waste of resources due to full video storage, lack of encryption for sensitive behavioral data, and difficulty in balancing the inference speed and accuracy of deep learning object detection algorithms (such as the general YOLOv5) on the edge side, leading to bottlenecks such as high detection latency and false positives.

[0004] Therefore, in edge intelligent inspection of vehicle traffic videos, the limited efficiency of parallel processing of massive multi-channel videos, the high security risks of sensitive behavioral data in the transmission and storage process, and the difficulty in balancing accuracy and real-time performance in abnormal behavior recognition have become urgent problems to be solved. Summary of the Invention

[0005] This application provides a traffic video edge analysis system based on the Yolov5-Edge and Prewitt algorithms, aiming to solve the problems of limited efficiency in parallel processing of massive multi-channel video in the edge intelligent inspection of vehicle traffic videos, high security risks in the transmission and storage of sensitive behavioral data, and difficulty in balancing accuracy and real-time performance in abnormal behavior recognition in existing technologies.

[0006] A traffic video edge analysis system based on the Yolov5-Edge and Prewitt algorithms, the system comprising: The device layer is used to collect in-vehicle video data and transmit it to the edge computing layer via a gateway; The edge computing layer, deployed on embedded edge devices, includes: a video acquisition and scheduling module, used for frame extraction and task allocation of multiple in-vehicle video streams; The AI ​​inference module is used to detect human targets and perform behavior recognition based on a lightweight instance segmentation model. The seat belt recognition module is used to determine whether a person is wearing a seat belt based on template similarity and edge features. The resource monitoring and load control module is used to adjust the inspection parameters and the number of video streams based on the system's computing power usage. The data management and encryption module is used to perform local storage, cleanup, and encrypted uploading of abnormal data; The cloud service layer is used to receive encrypted abnormal data, perform verification, storage, and output to upper-layer interfaces.

[0007] Optionally, in the above scheme, the edge computing layer is deployed on a Jetson-nano embedded hardware platform, which has a quad-core CPU and a 128-core GPU, and runs a Linux system, OpenCV and PyTorch framework to support model loading and edge AI inference.

[0008] Optionally, in the above scheme, the AI ​​inference module adopts a Yolov5-Edge instance segmentation model that has been pruned and quantized with INT8. The model includes two branches: detection and segmentation, and adopts an Anchor-Free detection structure to simultaneously output human detection boxes and segmentation masks.

[0009] In the above scheme, optionally, the AI ​​inference module distinguishes between drivers and passengers through coordinate rules. When the human body detection box satisfies x_max>(2 / 3)W and y_min<(1 / 3)H, it is determined to be a driver; otherwise, it is determined to be a passenger. Target segmentation of head, hands, mobile phone, and cigarette butt is performed within the personnel area to realize the recognition of smoking and mobile phone use behaviors. Here, W represents the width and H represents the height.

[0010] Optionally, in the above solution, the seatbelt recognition module includes: The template preprocessing unit is used to apply the corresponding Prewitt operator to the seat belt template images in the diagonal and anti-diagonal directions to extract edge features and generate template feature sets. The feature extraction unit is used to sample the personnel area image in a sliding window manner and extract the window edge features using the Prewitt operator; The similarity analysis unit is used to calculate the cosine similarity between window features and template features and generate feature sequences; The classification and determination unit is used to input the feature sequence into the support vector machine classifier to output the seat belt wearing category.

[0011] Optionally, in the above scheme, the video acquisition and scheduling module determines the vehicle's operating status based on the vehicle's GPS data, adds the corresponding camera video stream to the inspection list only when the vehicle is in operation, and refreshes the video stream list periodically to remove offline or stationary vehicles.

[0012] Optionally, in the above scheme, the video acquisition and scheduling module adopts a dual scheduling strategy of round-robin and priority scheduling, wherein: A regular queue is used to sequentially extract frames from all video streams. Priority queues are used to schedule video streams that meet any of the following conditions: suspected anomaly detected, confirmed anomaly detected, multiple targets detected, or video streams belonging to key vehicles or key routes are detected. The video streams in the priority queue are subjected to the next round of frame extraction at shortened inspection intervals.

[0013] Optionally, in the above scheme, the resource monitoring and load control module sets CPU and GPU usage thresholds. When the CPU usage exceeds 60% or the GPU usage exceeds 75%, the number of parallel inspection paths is automatically reduced from 5 to 3, and the default value is restored under low load conditions. At the same time, the frame extraction interval is dynamically adjusted according to the time period.

[0014] In the above scheme, optionally, the modules of the edge computing layer communicate with each other through a message queue. The video acquisition and scheduling module sends the extracted frame images to the AI ​​inference module, and the output results of the AI ​​inference module are received and recorded by the data management and encryption module.

[0015] Optionally, in the above scheme, the data management and encryption module performs the following operations when it detects abnormal behavior: Extract the abnormal frame image and save a 5-second video clip before and after the abnormality as abnormal data; The abnormal data is encrypted using an RSA public key and uploaded via a mobile communication network; Upon receiving the data, the cloud service layer performs an MD5 integrity check and encrypts the data using the AES-256 algorithm. Non-abnormal video frames are only temporarily stored in memory for 10 seconds and then cleared by a scheduled task; Abnormal data is retained locally for 72 hours by default, and the retention period can be configured.

[0016] Compared with the prior art, this application has at least the following beneficial effects: Based on further analysis and research of existing technical problems, this application recognizes the limitations in the efficiency of parallel processing of massive multi-channel video in edge intelligent inspection of vehicle traffic videos, the high security risks of sensitive behavioral data during transmission and storage, and the difficulty in balancing accuracy and real-time performance in abnormal behavior recognition. This application addresses these challenges by constructing a layered architecture comprising an object-side device layer, an edge computing layer, and a cloud service layer, enabling real-time acquisition, intelligent recognition, and encrypted transmission of vehicle video. The system integrates a lightweight Yolov5-Edge model and the Prewitt edge detection algorithm in edge devices, achieving parallel processing of multiple video streams with limited computing power, effectively improving video inspection efficiency. Resource monitoring and load control modules dynamically adjust the frame rate and number of processing channels to ensure the real-time performance and stability of AI inference. Data management and encryption mechanisms are employed, storing only abnormal data at the edge and uploading it to the cloud using RSA and AES encryption, ensuring the security and compliance of sensitive behavioral information during transmission and storage. This system can identify and classify abnormal behaviors of drivers and passengers without relying on high-performance servers. It solves the problems of low efficiency in processing massive amounts of video, uneven distribution of computing resources, insufficient data security, and poor real-time recognition in existing technologies, and realizes lightweight, intelligent and safe processing of traffic video inspection at the edge. Attached Figure Description

[0017] Figure 1 A diagram illustrating the architecture of a lightweight traffic video edge analysis system provided in one embodiment of this application; Figure 2 Example diagram of a seat belt area template provided in one embodiment of this application; Figure 3 A flowchart of a seatbelt recognition algorithm based on template similarity statistics provided in one embodiment of this application; Figure 4 A flowchart illustrating video inspection analysis and dynamic storage cloud migration based on edge AI technology is provided as an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] In the description of this application: unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," "third," etc., in this application are intended to distinguish the objects referred to and do not have any special meaning in terms of technical connotation (e.g., they should not be construed as an emphasis on importance or order). Expressions such as "including," "comprising," and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).

[0020] Abnormal behavior of drivers and passengers is a key factor in causing or exacerbating traffic accidents. Video AI inspection can promptly and effectively detect safety hazards and real-time risky behaviors. In traffic video inspection scenarios, tens of thousands of vehicles' onboard cameras generate massive amounts of surveillance video data. Due to the influence of mountainous terrain and complex urban road conditions, the videos captured by in-vehicle monitoring cameras are prone to perspective shifts and image repetitions due to collisions, shaking, network issues, etc. Moreover, edge hardware computing power is limited. Traditional inspection solutions face three core pain points: First, the efficiency of processing massive amounts of video is low. A single edge device cannot process multiple video streams in parallel, and performing AI analysis on each video stream can easily lead to hardware overload. Second, there are high security risks in data transmission and storage. Sensitive information such as driver / passenger behavior data is not transmitted encrypted, and storing all video streams leads to a waste of edge hardware storage resources. Third, there is the issue of balancing the accuracy and real-time performance of AI inspection capabilities. Existing edge computing solutions have not been adapted to traffic scenarios in terms of algorithm and computing power allocation, and only achieve basic video recognition, resulting in excessively high delays and false positive rates in abnormal behavior detection.

[0021] Existing traffic video inspection solutions mainly fall into two categories: one is based on traditional edge computing servers, which can handle multiple video streams, but the hardware is bulky and consumes a lot of power, making it unsuitable for vehicle-mounted or mobile inspection scenarios; the other is based on lightweight edge devices (such as ordinary embedded chips and edge boxes), but without optimized AI models and computing power scheduling, a single device can only handle 2-3 video streams and lacks data security mechanisms. In terms of algorithms, while deep learning object detection algorithms (such as YOLOv5) are widely used in behavior recognition, they have not been lightweighted for traffic scenarios, have slow inference speeds, and cannot meet the needs of real-time inspection.

[0022] To address the aforementioned issues, this embodiment proposes a lightweight traffic video edge analysis system based on the YOLOv5-Edge and Prewitt algorithms: 1) Based on the lightweight YOLOv5-Edge AI model and the traditional edge feature Prewitt operator, an innovative seatbelt recognition algorithm based on template similarity statistics is proposed, and an abnormal behavior recognition method is designed in conjunction with the composite AI principle to quickly and accurately identify abnormal behavior; 2) A three-layer lightweight architecture of "device-edge-cloud" is designed to achieve local real-time processing of multiple video streams through edge devices, reducing reliance on cloud computing power; 3) A model lightweighting and computing power balancing algorithm is adopted to quickly complete the "acquisition-inference-return" of multiple video streams and control the computing power load; 4) An "encrypted transmission + dynamic storage closed loop" is constructed, with abnormal data encrypted and uploaded to the cloud using RSA public key, and locally anomaly-free data is automatically deleted at regular intervals, balancing data security and storage costs, and achieving high-precision, high-real-time, and high-security intelligent inspection and analysis of traffic videos.

[0023] In one embodiment, such as Figure 1As shown, a traffic video edge analysis system based on the Yolov5-Edge and Prewitt algorithms is provided. The device layer is used to collect in-vehicle video data and transmit it to the edge computing layer via a gateway. The edge computing layer, deployed on embedded edge devices, includes: a video acquisition and scheduling module, used for frame extraction and task allocation of multiple in-vehicle video streams; The AI ​​inference module is used to detect human targets and perform behavior recognition based on a lightweight instance segmentation model. The seat belt recognition module is used to determine whether a person is wearing a seat belt based on template similarity and edge features. The resource monitoring and load control module is used to adjust the inspection parameters and the number of video streams based on the system's computing power usage. The data management and encryption module is used to perform local storage, cleanup, and encrypted uploading of abnormal data; The cloud service layer is used to receive encrypted abnormal data, perform verification, storage, and output to upper-layer interfaces.

[0024] The overall system architecture of this embodiment comprises three parts: the device layer, the edge computing layer, and the cloud service layer. These three layers are connected through a unified communication gateway to achieve layered acquisition, analysis, and transmission of video and data. The core objective of this system is to enable real-time inspection and abnormal behavior identification of multiple vehicle-mounted videos on low-power, low-computing-power edge devices, ensuring high real-time performance and security in complex traffic scenarios.

[0025] The device layer primarily consists of vehicle-mounted network cameras. These cameras support H.264 encoding and RTSP real-time transmission protocol, and feature automatic exposure adjustment and adaptive lighting to ensure clear image sequences even under complex lighting conditions such as tunnels, nighttime, and backlighting. The device cameras and edge computing devices are directly connected via the same vehicle gateway, with data transmission latency not exceeding 100ms, thus providing highly timely video input for real-time inspections.

[0026] The edge computing layer is deployed on an embedded hardware platform, such as a Jetson-nano device, which integrates a quad-core CPU and a 128-core GPU, and runs a Linux system and the OpenCV and PyTorch frameworks to support the parallel execution of deep learning inference and traditional vision algorithms. The edge computing layer is the core of this invention, and it includes functional units such as a video acquisition and scheduling module, an AI inference module, a seatbelt recognition module, a resource monitoring and load control module, and a data management and encryption module.

[0027] The video acquisition and scheduling module is responsible for extracting frames from multiple camera inputs according to a set strategy and allocating tasks. The module supports a dual-queue mechanism of "round-robin + priority," where all video streams are inspected in a round-robin fashion in the regular queue, while video streams from suspected abnormalities or key vehicles are inspected at high frequency in the priority queue, thus ensuring continuous detection coverage and focusing on key scenes. The module can also combine vehicle GPS status to determine whether it is in operation, performing video inspections only on vehicles in operation to reduce the processing of invalid data.

[0028] The AI ​​inference module employs a lightweight Yolov5-Edge model, optimized through network pruning and INT8 quantization. It features a dual-branch structure for detection and instance segmentation, and utilizes an anchor-free mechanism to simultaneously obtain human detection bounding boxes and segmentation masks during a single inference iteration. The model is optimized for the complex postures of in-vehicle occupants, enabling simultaneous detection of both driver and passenger areas, and differentiation of body parts, heads, hands, mobile phones, cigarette butts, and other targets to identify smoking and mobile phone use.

[0029] The seatbelt recognition module employs a lightweight seatbelt detection algorithm based on the Prewitt edge operator and template similarity statistics. The module first constructs seatbelt template sets along the diagonal and anti-diagonal directions, extracting template edge features using the Prewitt operator. Then, it slides through a fixed-size window on the target person's image, extracting window edge features and calculating cosine similarity with the template features to form a feature sequence. Finally, the feature sequence is input into a support vector machine classifier to output the seatbelt wearing status. This algorithm features low computational complexity and stability, making it particularly suitable for low-computing environments in embedded devices.

[0030] The resource monitoring and load control module monitors CPU and GPU utilization in real time. When the resource load exceeds a set threshold (e.g., CPU > 60% or GPU > 75%), the system automatically reduces the number of video feeds or increases the frame extraction interval. Under low load conditions, it restores the default configuration, achieving a dynamic balance between computing power and task requirements. The module can also automatically adjust the frame rate according to different time periods, such as increasing the frame extraction density during peak traffic hours to enhance detection coverage.

[0031] The data management and encryption module is used for hierarchical storage and secure encryption of data generated during inspections. When the system detects abnormal behavior, it captures the current abnormal frame and a 5-second video clip before and after it, generating structured text (including time, camera number, vehicle identification, and abnormality type). This text is then encrypted using the RSA public-key algorithm and uploaded to the cloud service layer via a 4G / 5G network. Non-abnormal data is temporarily stored in memory for 10 seconds and automatically deleted after confirmation of no abnormalities. Abnormal data is retained locally for 72 hours, with an expired cleanup performed by a scheduled task, thus conserving storage resources while ensuring traceability.

[0032] The cloud service layer is configured with encrypted cloud storage and verification mechanisms. Upon receiving encrypted data, the cloud first checks data integrity using MD5 verification, and then performs secondary encryption using the AES-256 algorithm. The cloud only stores encrypted data; the decryption key is only stored in the corresponding edge device, preventing the risk of data leakage in the cloud. The cloud system also provides an interface to push abnormal information to the traffic management platform in real time, enabling visualized alerts and data retention.

[0033] This embodiment achieves efficient inspection and intelligent analysis of multi-channel video in vehicles by building a layered collaborative architecture on a lightweight edge device, effectively solving the three key problems raised in the background technology: By employing a dual-algorithm framework combining the lightweight Yolov5-Edge model and the Prewitt algorithm, each edge device can process five video streams in parallel within 3 seconds. The multi-process structure and message queue communication mechanism ensure the concurrent execution of video frame extraction, AI inference, and data management, fundamentally alleviating the performance bottleneck caused by single-threaded AI inference in traditional solutions, and significantly improving the real-time performance and system stability of video inspection.

[0034] The system employs a "local dynamic storage + cloud encrypted backup" mechanism, combining RSA transmission encryption and AES-256 cloud encrypted storage for dual protection, ensuring the confidentiality and integrity of driver and passenger behavior data throughout the entire process of collection, transmission, and storage. Simultaneously, a timed cleanup mechanism retains only abnormal data, avoiding redundancy caused by storing all video data, significantly reducing resource usage, and making data management more efficient.

[0035] This embodiment optimizes the YOLOv5 structure at the model level through network pruning, quantization, and Anchor-Free mechanisms, enabling the model to maintain high inference frame rates and stable accuracy on edge devices. By combining Prewitt operator feature extraction and template similarity calculation, a low-computing-power, high-recognition scheme for seatbelt detection is achieved, thus maintaining high detection accuracy and stability while ensuring lightweight operation.

[0036] The traffic video edge analysis system proposed in this embodiment, based on the Yolov5-Edge and Prewitt algorithms, forms a complete closed loop in terms of architecture design, algorithm fusion, computing power scheduling, and data security management. This solution can achieve efficient, stable, and secure abnormal behavior detection and inspection analysis in vehicle-mounted or mobile traffic scenarios, effectively supporting the needs of traffic safety monitoring and intelligent supervision.

[0037] In this embodiment, the edge computing layer is deployed on a Jetson-nano embedded hardware platform, which has a quad-core CPU and a 128-core GPU, and runs a Linux system, OpenCV and PyTorch framework to support model loading and edge AI inference.

[0038] In this embodiment, the AI ​​inference module adopts a Yolov5-Edge instance segmentation model that has been pruned and quantized with INT8. The model includes two branches: detection and segmentation. It uses an Anchor-Free detection structure to simultaneously output human detection boxes and segmentation masks.

[0039] In this embodiment, the AI ​​inference module distinguishes between drivers and passengers using coordinate rules. When the human body detection box satisfies x_max>(2 / 3)W and y_min<(1 / 3)H, it is determined to be a driver; otherwise, it is determined to be a passenger. Target segmentation of the head, hands, mobile phone, and cigarette butt is performed within the personnel area to achieve recognition of smoking and mobile phone use behaviors.

[0040] In this embodiment, the seatbelt recognition module includes: The template preprocessing unit is used to apply the corresponding Prewitt operator to the seat belt template images in the diagonal and anti-diagonal directions to extract edge features and generate template feature sets. The feature extraction unit is used to sample the personnel area image in a sliding window manner and extract the window edge features using the Prewitt operator; The similarity analysis unit is used to calculate the cosine similarity between window features and template features and generate feature sequences; The classification and determination unit is used to input the feature sequence into the support vector machine classifier to output the seat belt wearing category.

[0041] In this embodiment, the video acquisition and scheduling module determines the vehicle's operating status based on the vehicle's GPS data, adds the corresponding camera video stream to the inspection list only when the vehicle is in operation, and refreshes the video stream list periodically to remove offline or stationary vehicles.

[0042] In this embodiment, the video acquisition and scheduling module adopts a dual scheduling strategy of round-robin and priority scheduling, wherein: A regular queue is used to sequentially extract frames from all video streams. Priority queues are used to schedule video streams that meet any of the following conditions: suspected anomaly detected, confirmed anomaly detected, multiple targets detected, or video streams belonging to key vehicles or key routes are detected. The video streams in the priority queue are subjected to the next round of frame extraction at shortened inspection intervals.

[0043] In this embodiment, the resource monitoring and load control module sets CPU and GPU usage thresholds. When CPU usage exceeds 60% or GPU usage exceeds 75%, the number of parallel inspection paths is automatically reduced from 5 to 3, and the default value is restored under low load conditions. At the same time, the frame extraction interval is dynamically adjusted according to the time period.

[0044] In this embodiment, the modules of the edge computing layer communicate with each other through message queues. The video acquisition and scheduling module sends the extracted frame images to the AI ​​inference module, and the output results of the AI ​​inference module are received and recorded by the data management and encryption module.

[0045] In this embodiment, the data management and encryption module performs the following operations when it detects abnormal behavior: Extract the abnormal frame image and save a 5-second video clip before and after the abnormality as abnormal data; The abnormal data is encrypted using an RSA public key and uploaded via a mobile communication network; Upon receiving the data, the cloud service layer performs an MD5 integrity check and encrypts the data using the AES-256 algorithm. Non-abnormal video frames are only temporarily stored in memory for 10 seconds and then cleared by a scheduled task; Abnormal data is retained locally for 72 hours by default, and the retention period can be configured.

[0046] This embodiment aims to address the problems of "low efficiency in processing massive amounts of video, weak AI capability adaptation, and insufficient data management" in existing traffic video inspection and analysis systems. Through lightweight architecture design, AI model optimization, balanced computing power scheduling, and encrypted storage mechanisms, it provides an intelligent inspection and analysis system adapted to large-scale operational vehicle scenarios, achieving the following objectives: 1) Edge devices can process 5 channels of vehicle-mounted video within 3 seconds, improving the real-time performance of inspections; 2) Abnormal data transmission and storage are encrypted throughout the process, reducing data transmission and local storage costs; 3) The accuracy rate of driver / passenger abnormal behavior recognition is ≥95%, and the false judgment rate is ≤3%, providing reliable data support for traffic safety production.

[0047] This embodiment presents a lightweight traffic video edge analysis system based on the Yolov5-Edge and Prewitt algorithms. It consists of five modules: a lightweight edge computing architecture, an AI video analysis model, optimized hardware and software configurations, an intelligent inspection process, and data transmission and management. These modules interact to form a complete inspection process (e.g., ...). Figure 1 (As shown). The three-layer architecture module enables layered data processing and transmission; the AI ​​video analysis model module enables precise inspection video analysis; the hardware and software configuration module optimizes the AI ​​model and computing power allocation; the intelligent inspection process module improves video processing efficiency; and the data transmission and management module ensures data security, together completing the real-time inspection and counting statistics of abnormal behaviors inside operating vehicles.

[0048] Lightweight edge computing architecture: This module is responsible for building the data transmission and processing link from "device to edge to cloud", clarifying the hardware, functions and interaction design of each layer, solving the problems of transmission redundancy, computing power waste and cloud overload in traditional architectures, and realizing efficient flow and secure storage of video from acquisition to anomaly warning. The specific architecture is shown in the table below:

[0049] The lightweight edge computing architecture collaborates across three layers—the device layer, the edge computing layer, and the cloud service layer—to build a complete data transmission and processing link in four steps. The specific steps are as follows: Device acquisition and initial transmission: The vehicle-mounted network camera at the device layer acquires video streams inside the vehicle and automatically adjusts exposure parameters according to light intensity to optimize video clarity in backlight and nighttime scenes; then, it generates an image sequence by periodically sampling and inspecting images, and transmits the image sequence to the edge computing device at the edge computing layer with a transmission delay of ≤100ms through the same gateway directly connected to the edge device; Edge layer processing and resource management: After receiving data, the edge computing devices in the edge computing layer start multiple independent processes, namely ① AI inference process, ② resource monitoring process, ③ data management process, and ④ log recording process, to work together. Abnormal data encryption and uploading: After the data management process in the edge computing layer identifies abnormal data, it calls the RSA public key encryption algorithm to encrypt the abnormal alarm data and uploads the encrypted data to the cloud service layer via the 4G / 5G network.

[0050] Cloud reception and backup alerts: After receiving encrypted data, the public key encrypted cloud server of the cloud service layer verifies the integrity of the data through the MD5 hash value verification mechanism. If the verification fails, the edge device is notified to re-upload. If the verification passes, the data is encrypted and stored using the AES-256 algorithm, and abnormal data is pushed to the traffic management platform in real time. The abnormal alarm is completed in conjunction with the physical alarm device and the cloud management platform.

[0051] AI Video Analysis Model: In AI inspection, the main function is to identify abnormal behaviors of occupants in vehicles. The abnormal behavior analysis model is a lightweight design combining traditional and deep learning algorithms based on business scenario requirements. Together, they identify three types of abnormal behaviors in vehicle camera images: smoking, using a mobile phone, and not wearing a seatbelt. The model mainly consists of three processes: human segmentation, behavioral target analysis, and a seatbelt recognition algorithm based on template similarity statistics. The specific process is as follows: Human body segmentation: Human body detection is based on the YOLOv5-Edge instance segmentation algorithm to classify the human body region and the types of people in the vehicle. The model is trained using a self-labeled traffic scene behavior dataset (containing 100,000+ valid images, covering three abnormal behaviors of people in the vehicle: smoking, playing with mobile phones, and not wearing seat belts). Compared to other single-stage segmentation algorithms, YOLOv5-Edge is deeply optimized for edge scenes. Through network layer fusion and INT8 quantization design, it maintains stable accuracy while reducing the number of parameters by 40%, making it more suitable for resource-constrained edge hardware. YOLOv5-Edge innovatively integrates detection and segmentation branches, accurately locating human bodies in different poses through an anchor-free mechanism and simultaneously outputting pixel-level segmentation masks, avoiding the secondary computational overhead of "detect first, then segment." Especially in densely populated and complex behavior scenes, it reduces mask overlap through improved DIOU-NMS, resulting in higher reliability than single segmentation algorithms. At the same time, it balances lightweight design with fine-grained feature extraction capabilities, improving the segmentation accuracy of human limb movements and local behaviors by 10%-15%, providing higher-quality feature input for behavior recognition.

[0052] In the human detection process, the human body region is first segmented from the video frames using the instance segmentation branch of YOLOv5-Edge to generate a human body mask image, avoiding coordinate misjudgments caused by interference. Then, a coordinate verification rule is used to classify drivers and passengers. Based on the bounding box coordinates (x_min, y_min, x_max, y_max) of the human body mask and the image size (width W, height H) of the vehicle camera, a judgment rule is set: if the bounding box satisfies x_max > W... 2 / 3 and y_min <H If 1 / 3 is the number of passengers, the passenger is identified as the driver (corresponding to the driver's seat); if not, the passenger is identified as the front passenger seat or the rear seat.

[0053] Behavioral target recognition: Also based on the YOLOv5-Edge instance segmentation algorithm, a secondary segmentation is performed on the human body region image segmented in the human body detection process. The main segmentation is of mobile phones, cigarette butts, heads, and hands contained in the human body region. Based on the coordinate association of the head, hand, and mobile phone regions, the hand and cigarette butt regions are segmented as a whole. This can effectively improve the detection and segmentation of smaller targets such as mobile phones and cigarette butts, and reduce the interference of other information in the entire vehicle image.

[0054] Seatbelt recognition algorithm based on template similarity statistics: Seatbelt recognition is implemented in a lightweight manner based on traditional vision algorithms. The algorithm flow is as follows: Figure 2As shown, the main function is to calculate the edge features of each window in the personnel detection image using a sliding window method and the Prewitt edge operator. Simultaneously, it extracts image feature sequences by comparing them with templates. These feature sequences are then used as input to employ a support vector machine classifier to identify whether a seatbelt is being worn. The detailed process is as follows: Data preparation is as follows: Based on prior knowledge of the seat belt direction for drivers / passengers, the module maintains 3 diagonal seat belt template images p_i, 3 anti-diagonal seat belt template images r_i (as shown in Figure 3), 2 pairs of diagonal Prewitt operators p_pw and anti-diagonal operators n_pw, and a personnel region image img; ① Template preprocessing: Using the diagonal operator p_pw, edge features are extracted from the seatbelt template images p_i along each diagonal direction using a sliding window, resulting in the diagonal template feature set p_f; similarly, the inverse diagonal template feature set n_f is obtained. The operators are defined as follows:

[0055] ② Personnel Image Feature Extraction: For the driver / passenger image (img), a sliding window of the size of the seatbelt template (w=48, h=48) with a sliding step of 4 is sampled without filling. Edge features (b_f) are extracted from the image information within the sampling window using both diagonal and inverse diagonal Prewitt operators. Then, cosine similarity is used to calculate the cosine similarity between the edge features (b_f) of the sampling window and each template feature in the template feature list (p_f). Finally, the similarity values ​​obtained from all sliding windows in the image are serialized to obtain the personnel image seatbelt feature sequence. The feature cosine similarity is expressed as:

[0056] Where k is the length of the feature vector, and in this paper, it is the area of ​​the sliding window, which is 48×48.

[0057] ③ Seatbelt Recognition and Classification: The process employs Support Vector Machine (SVM) to identify individuals not wearing seatbelts. Compared to other classifier algorithms, SVM is simpler and more stable, has a lower risk of overfitting under lower-dimensional data complexity, and performs well in higher-dimensional binary classification problems. The seatbelt feature sequence of a single person image is used as input to the SVM classifier, performing binary classification into two categories: seatbelt wearing and seatbelt position. Based on a seatbelt classification threshold, the images are filtered to identify individuals wearing seatbelts and those who may not be wearing them, thus completing the seatbelt wearing recognition task for vehicle images.

[0058] Hardware and software configuration optimization: This module is responsible for AI model lightweighting, computing power allocation, and device initialization configuration, ensuring that edge computing devices can stably complete multi-channel video inspections under limited computing power, supporting system real-time performance and ease of use, and achieving efficient inspections. Specifically, it includes the following: AI Model Lightweighting: Through model pruning and quantization, the size of the YOLOv5-Edge model is compressed from 40MB to 28MB, and the inference speed is increased to 20fps (single-channel video), which is 30% more efficient than the general YOLOv5 model inference, and is adapted to the GPU computing power of edge devices.

[0059] Computing power balancing scheduling: Following the principle of "peak-shifting dynamic adjustment", ① Frame extraction interval adjustment: During off-peak hours, when inspecting 5 video channels, frames are extracted at "1 frame / second", and during peak hours, it is automatically adjusted to "2 frames / second" to ensure that no abnormal behavior is missed; ② Frame extraction channel adjustment: The computing power utilization rate of edge box devices is monitored in real time through scripts. When the load exceeds the threshold (such as CPU > 60% or GPU > 75%), the number of video channels inspected is automatically reduced one by one until the load returns to the safe threshold, so as to achieve dynamic matching between computing power and inspection needs.

[0060] Automatic startup configuration: Configure an initialization script in the edge device. After the device is plugged in, it will automatically perform the following operations: ① Load the preset list of RTSP addresses for vehicle cameras; ② Initialize the YOLOv5-Edge model and inference environment; ③ Configure the cloud public key and local private key; ④ Start the resource monitoring and logging service. This achieves "plug and play" functionality without manual intervention, which facilitates rapid and convenient deployment when deploying a large number of edge devices and reduces the complexity of AI engineering implementation.

[0061] Intelligent Inspection Control: This module is responsible for optimizing video stream filtering, inspection scheduling, and process communication logic, reducing redundant data processing, avoiding missed inspections, and improving inspection efficiency. The specific process is as follows: 1) Video stream filtering: By accessing the vehicle's GPS module through edge devices, the real-time location and driving status data of the vehicle are obtained to determine whether the vehicle is in operation. Only the camera addresses of "in operation vehicles" are included in the video stream list. The video stream list is updated regularly to remove addresses of non-operation vehicles (such as vehicles parked in parking lots or offline) to reduce invalid video processing.

[0062] 2) Inspection and scheduling mechanism: A dual scheduling strategy of "round-robin + priority" is adopted, establishing a regular queue and a priority queue, specifically designed as follows: Polling: For all video stream addresses, they are initially added to the regular queue, the priority queue is polled first, then the regular queue is traversed, and 5 video streams are inspected at a rate of "3 seconds / round" (processing time for each video stream is ≤0.6 seconds). Priority scheduling: Based on behavioral motivations and key video streams identified in video stream analysis, priority columns are changed, as detailed below: ① Prioritize behavioral motivation: If a video stream is detected as "suspected abnormal" during the current inspection (such as a driver's hand being close to their face but not clearly determined to be playing with a mobile phone), the video stream will be added to the priority queue, and the next round of inspection interval will be shortened to 1 second, focusing on monitoring for any clear abnormal behavior to avoid missed detection.

[0063] ② Prioritize abnormal alarms: If an abnormality is detected in a video stream during this inspection, the video stream will be added to the priority queue for continuous and focused monitoring of the abnormal behavior; ③ Prioritize densely populated areas: If multiple people appear in the video stream, add the video stream to the priority queue; ④ Prioritize key vehicles: Based on information such as key vehicles, operating ratings, and key routes in the system data, the corresponding video streams are added to the priority queue.

[0064] The priority queue is internally divided and sorted according to the priority order described above.

[0065] 3) Process Communication Optimization: ZeroMQ lightweight message queue is used to realize communication between the "inspection process" and the "AI inference process". After the inspection process extracts frames, it sends the video frame data to the AI ​​inference process through the message queue. The AI ​​model inference results are returned in real time, achieving a communication latency of less than 50ms for a single video stream, which is 40% lower than the latency of Socket communication, avoiding inspection delays caused by data transmission stutters.

[0066] Data transmission and management: This module is responsible for building a system of "local dynamic storage + cloud encrypted backup" to ensure data security and reduce storage redundancy. The specific process is as follows: 1) Local Dynamic Storage: When the AI ​​model detects abnormal behavior, it immediately executes the following: ① Capture the abnormal frame image (Base64 encoded for easy transmission); ② Save a 5-second video clip before and after the abnormality (H.264 encoded, 1Mbps bitrate, balancing clarity and storage usage); ③ Store the abnormal data locally (default retention time is 72 hours, manual adjustment of retention time is supported). Video frames without abnormal behavior are only temporarily stored in memory for 10 seconds and automatically deleted after confirmation of no abnormality. Expired and redundant data is cleaned up daily at 2:00 AM via a cron job.

[0067] 2) Encrypted Transmission and Cloud Backup: Edge devices encrypt aberration data (structured text + 5 seconds of video) using the RSA public-key encryption algorithm. The public key is pre-distributed to the edge devices from the cloud, while the private key is stored locally on the Jetson-nano (encrypted in the / etc / secret directory, accessible only to the root user). The encrypted data is uploaded to the cloud via a 4G / 5G network. Upon receiving the data, the cloud verifies its integrity using the corresponding public key and then encrypts it using the AES-256 algorithm to prevent unauthorized access.

[0068] This embodiment presents a lightweight traffic video edge analysis system based on the Yolov5-Edge and Prewitt algorithms. Through the collaborative work of five core modules, it achieves the following significant effects in traffic video inspection scenarios: Inspection efficiency is greatly improved: edge computing devices can complete the "frame extraction-inference-result output" of 5 channels of vehicle video within 3 seconds, which is 67% more efficient than traditional lightweight edge solutions (3 channels / 3 seconds per device); during peak hours, by dynamically adjusting the frame interval, it can still maintain an abnormal behavior detection rate of more than 95%, meeting the high-concurrency inspection needs of traffic morning and evening peak hours.

[0069] Data security level meets standards: The system adopts a three-level security mechanism of "local key storage + RSA transmission encryption + AES cloud storage encryption" to meet the national information security level protection 2.0 level 3 standard. Abnormal data has no risk of leakage during transmission and storage, solving the security problem of sensitive behavioral data in traffic scenarios.

[0070] Storage costs are significantly reduced: Local storage only includes abnormal data and critical logs within 24 hours, with no abnormal data deleted in real time. Compared to the full video storage solution, local storage usage is reduced by 60%. The cloud only backs up encrypted abnormal data, reducing storage costs by 75% and significantly reducing investment in hardware and cloud resources.

[0071] Optimized recognition accuracy and ease of use: The accuracy rate of driver / passenger abnormal behavior recognition is ≥95%, and the false judgment rate is ≤3%, providing traffic management departments with a reliable basis for judging violations; the equipment supports "plug and start", requiring no professional technicians to configure, and a single device can be deployed within 10 minutes, adapting to different scales of operating vehicles (from hundreds to tens of thousands of vehicles), with strong scalability.

[0072] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A traffic video edge analysis system based on Yolov5-Edge and Prewitt algorithms, characterized in that, The system includes: The device layer is used to collect in-vehicle video data and transmit it to the edge computing layer via a gateway; The edge computing layer, deployed on embedded edge devices, includes: a video acquisition and scheduling module, used for frame extraction and task allocation of multiple in-vehicle video streams; The AI ​​inference module is used to detect human targets and perform behavior recognition based on a lightweight instance segmentation model. The seat belt recognition module is used to determine whether a person is wearing a seat belt based on template similarity and edge features. The resource monitoring and load control module is used to adjust the inspection parameters and the number of video streams based on the system's computing power usage. The data management and encryption module is used to perform local storage, cleanup, and encrypted uploading of abnormal data; The cloud service layer is used to receive encrypted abnormal data, perform verification, storage, and output to upper-layer interfaces.

2. The system according to claim 1, characterized in that, The edge computing layer is deployed on a Jetson-nano embedded hardware platform, which has a quad-core CPU and a 128-core GPU, and runs a Linux system, OpenCV, and PyTorch framework to support model loading and edge AI inference.

3. The system according to claim 1, characterized in that, The AI ​​inference module uses a Yolov5-Edge instance segmentation model that has been pruned and quantized with INT8. The model includes two branches: detection and segmentation. It adopts an Anchor-Free detection structure to simultaneously output human detection boxes and segmentation masks.

4. The system according to claim 3, characterized in that, The AI ​​inference module distinguishes between drivers and passengers using coordinate rules. When the human detection box satisfies x_max>(2 / 3)W and y_min<(1 / 3)H, it is determined to be a driver; otherwise, it is determined to be a passenger. The module also performs target segmentation of the head, hands, mobile phone, and cigarette butt within the human area to achieve recognition of smoking and mobile phone use behaviors. Here, W represents the width and H represents the height.

5. The system according to claim 1, characterized in that, The seatbelt recognition module includes: The template preprocessing unit is used to apply the corresponding Prewitt operator to the seat belt template images in the diagonal and anti-diagonal directions to extract edge features and generate template feature sets. The feature extraction unit is used to sample the personnel area image in a sliding window manner and extract the window edge features using the Prewitt operator; The similarity analysis unit is used to calculate the cosine similarity between window features and template features and generate feature sequences; The classification and determination unit is used to input the feature sequence into the support vector machine classifier to output the seat belt wearing category.

6. The system according to claim 1, characterized in that, The video acquisition and scheduling module determines the vehicle's operating status based on the vehicle's GPS data. It adds the corresponding camera video stream to the inspection list only when the vehicle is in operation, and refreshes the video stream list periodically to remove offline or stationary vehicles.

7. The system according to claim 1, characterized in that, The video acquisition and scheduling module adopts a dual scheduling strategy of round-robin and priority scheduling, wherein: A regular queue is used to sequentially extract frames from all video streams. Priority queues are used to schedule video streams that meet any of the following conditions: suspected anomaly detected, confirmed anomaly detected, multiple targets detected, or video streams belonging to key vehicles or key routes are detected. The video streams in the priority queue are subjected to the next round of frame extraction at shortened inspection intervals.

8. The system according to claim 1, characterized in that, The resource monitoring and load control module sets CPU and GPU usage thresholds. When CPU usage exceeds 60% or GPU usage exceeds 75%, it automatically reduces the number of parallel inspection paths from 5 to 3, and restores the default value under low load conditions. At the same time, it dynamically adjusts the frame extraction interval according to the time period.

9. The system according to claim 1, characterized in that, The modules of the edge computing layer communicate with each other through message queues. The video acquisition and scheduling module sends the extracted frame images to the AI ​​inference module, and the output results of the AI ​​inference module are received and recorded by the data management and encryption module.

10. The system according to claim 1, characterized in that, The data management and encryption module performs the following operations when it detects abnormal behavior: Extract the abnormal frame image and save a 5-second video clip before and after the abnormality as abnormal data; The abnormal data is encrypted using an RSA public key and uploaded via a mobile communication network; Upon receiving the data, the cloud service layer performs an MD5 integrity check and encrypts the data using the AES-256 algorithm. Non-abnormal video frames are only temporarily stored in memory for 10 seconds and then cleared by a scheduled task; Abnormal data is retained locally for 72 hours by default, and the retention period can be configured.

Images