Evidence obtaining method and system for driving violation of occupying highway emergency lane

Abstract

The invention relates to the technical field of intelligent traffic, and particularly discloses a method and system for obtaining evidences of driving violation of occupying an emergency lane of an expressway, and the method comprises the steps: carrying out the global monitoring of the expressway through a dome camera, so as to obtain a global monitoring video of the expressway; when a main traffic lane in the expressway monitoring video is in a vehicle congestion state, performing close-up monitoring on an emergency lane area of the expressway through a dome camera to obtain an emergency lane area close-up monitoring video; when the vehicle in the emergency lane area close-up monitoring video has the illegal behavior of occupying the emergency lane for driving, capturing the driving process image and the license plate image of the illegal vehicle in the emergency lane through a spherical camera, and synchronously recording a section of illegal process video of the illegal vehicle; and generating an emergency lane driving violation evidence obtaining image according to the process image and the license plate image of the illegal vehicle. According to the invention, automatic identification, tracking, evidence obtaining and evidence chain generation of illegal behaviors can be realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and more specifically, to a method and system for obtaining evidence of illegal driving by occupying the emergency lane of a highway. Background Technology

[0002] As the arteries of the national transportation network, highways are crucial for both traffic efficiency and safety. Emergency lanes, commonly known as "lifelines," are specifically designated for vehicles involved in emergency situations such as engineering rescue, firefighting, medical assistance, or police officers performing urgent duties. No other vehicles are permitted to enter or drive on them except in emergencies. However, during holidays or when highways are congested due to accidents, some drivers... Figure 1 Taking advantage of one's own convenience to illegally occupy the emergency lane is not only illegal but also extremely dangerous. It can easily cause accidents, exacerbate traffic congestion, and set a negative example.

[0003] Currently, the methods for obtaining evidence against such illegal activities have the following limitations: 1. On-site enforcement: Relying on patrols by management departments, this method has extremely limited coverage and poses security risks for law enforcement. It also makes comprehensive and timely monitoring difficult given the rapidly changing traffic congestion situation.

[0004] 2. Fixed-point enforcement: Fixed violation enforcement systems are installed on key road sections. This method is limited by the physical location of poles and cameras, resulting in a fixed monitoring range and blind spots. Drivers familiar with the locations may try to evade penalties by merging back into the main lane, limiting its deterrent effect. Furthermore, large-scale construction of fixed points is costly and unsuitable for full road coverage.

[0005] 3. Semi-automatic monitoring system: Although some systems are connected to surveillance video, they largely rely on manual video inspection at the monitoring center, or require pre-defining the emergency lane area in the video frame. This method is inefficient, highly dependent on manual labor, and the preset static area cannot adapt to changes in the viewing angle after the PTZ camera rotates, resulting in poor flexibility.

[0006] In summary, existing technologies cannot meet the needs for efficient, accurate, comprehensive, and automated off-site evidence collection of vehicles using emergency lanes on highways. Therefore, there is an urgent need for an intelligent evidence collection system that can intelligently sense traffic conditions, automatically identify illegal behaviors, and automatically generate standardized evidence. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method and system for collecting evidence of illegal driving in the emergency lane of a highway. The aim is to minimize human intervention by automatically understanding traffic scenarios, triggering monitoring events, identifying illegal behaviors, and generating a legally compliant chain of evidence through algorithms.

[0008] As a first aspect of the present invention, a method for obtaining evidence of illegal driving in the emergency lane of a highway is provided, comprising: Step S1: Use a high-definition network dome camera to perform global monitoring of the highway at a preset global monitoring position to obtain a global monitoring video of the highway; wherein, the high-definition network dome camera is deployed on the side of the highway. Step S2: Determine whether the main driving lane area in the global monitoring video of the highway is in a state of traffic congestion; when the main driving lane area in the monitoring video of the highway is in a state of traffic congestion, use the high-definition network dome camera to perform close-up monitoring of the emergency lane area of ​​the highway at the emergency lane close-up monitoring preset position to obtain close-up monitoring video of the emergency lane area. Step S3: Determine whether the vehicle in the close-up monitoring video of the emergency lane area has committed a traffic violation by occupying the emergency lane; when the vehicle in the close-up monitoring video of the emergency lane area has committed a traffic violation by occupying the emergency lane, capture three process images and one license plate image of the illegal vehicle driving in the emergency lane through the high-definition network dome camera, and simultaneously record a video of the illegal vehicle's traffic violation in the emergency lane. Step S4: Generate evidence images of the illegal vehicle's violation of driving in the emergency lane based on the three process images and one license plate image of the illegal vehicle.

[0009] Furthermore, the step of determining whether the main driving lane area in the global highway monitoring video is in a state of traffic congestion also includes: Vehicles, main driving lanes, emergency lanes, and road markings were identified and segmented from the global monitoring video of the highway using a deep learning model. The vehicle detection model detects the average speed and density of vehicles in the main driving lane area. When the average speed of vehicles in the main driving lane area is lower than the preset speed threshold and the vehicle density is higher than the preset density threshold for a certain period of time, it is determined that the main driving lane area is in a state of vehicle congestion.

[0010] Furthermore, the deep learning model is a semantic segmentation network based on an encoder-decoder structure. This semantic segmentation network is trained on a large number of labeled highway scene images and can accurately output the road element category to which each pixel belongs.

[0011] Furthermore, the step of using the high-definition network dome camera to perform close-up monitoring of the emergency lane area of ​​the highway from the preset position for close-up monitoring when the main driving lane area in the highway monitoring video is in a state of traffic congestion also includes: When the main driving lane is congested, the high-definition network dome camera is switched from the global monitoring preset position to the emergency lane close-up monitoring preset position, automatically triggering the collection of evidence of illegal driving in the emergency lane.

[0012] Furthermore, the determination of whether a vehicle in the close-up surveillance video of the emergency lane area has committed a traffic violation by occupying the emergency lane also includes: A deep learning-based multi-target tracking algorithm is used to analyze the movement route and speed of each vehicle in the close-up surveillance video of the emergency lane area in real time. If a vehicle's route remains within the emergency lane for a certain period of time and its speed within the emergency lane exceeds the minimum driving speed threshold, the vehicle is deemed to have committed a traffic violation by occupying the emergency lane.

[0013] Furthermore, the process of capturing three images of the illegal vehicle driving in the emergency lane and one license plate image using the high-definition network dome camera also includes: The high-definition network dome camera is controlled to perform rapid automatic focusing and zooming, capture an image of the license plate of the illegal vehicle, and optimize the license plate image based on image enhancement technology based on super-resolution reconstruction.

[0014] Furthermore, the process of generating the emergency lane violation evidence image of the offending vehicle based on three process images and one license plate image also includes: The time, location, violation, license plate number, and legal basis information of the violation are overlaid on the image of the vehicle violating traffic rules by driving in the emergency lane. The hash values ​​of the images and videos of the illegal driving in the emergency lane were calculated and stored in the database.

[0015] As a second aspect of the present invention, a system for collecting evidence of violations of driving in the emergency lane of a highway is provided, comprising: The first front-end sensing unit is used to perform global monitoring of the highway through a high-definition network dome camera at a global monitoring preset position to obtain a global monitoring video of the highway; wherein, the high-definition network dome camera is deployed on the side of the highway. The first central analysis unit is used to determine whether the main driving lane area in the global monitoring video of the highway is in a state of vehicle congestion. The second front-end sensing unit is used to monitor the emergency lane area of ​​the highway through the high-definition network dome camera at the emergency lane close-up monitoring preset position when the main driving lane area in the highway monitoring video is in a state of traffic congestion, so as to obtain the emergency lane area close-up monitoring video. The second central analysis unit is used to determine whether the vehicles in the close-up surveillance video of the emergency lane area have committed illegal driving behavior by occupying the emergency lane. The third front-end sensing unit is used to capture three process images and one license plate image of the vehicle driving in the emergency lane through the high-definition network dome camera when a vehicle in the close-up monitoring video of the emergency lane area is found to be illegally occupying the emergency lane, and simultaneously record a video of the illegal vehicle's illegal driving process in the emergency lane. The third center analysis unit is used to generate evidence images of the illegal vehicle's violation of driving in the emergency lane based on three process images and one license plate image.

[0016] The method and system for collecting evidence of illegal driving in the emergency lane of a highway provided by this invention have the following advantages: (1) Fully automated: From scene understanding, event triggering, behavior recognition to evidence generation, the entire process requires no human intervention, which greatly improves law enforcement efficiency; (2) High intelligence: Based on deep learning, the scene self-understanding and behavior analysis model enables the system to have visual perception and judgment ability similar to humans, with high accuracy and strong adaptability; (3) Flexibility and wide coverage: By utilizing the pan-tilt characteristics of the PTZ camera and through preset position scheduling, a "one-to-many" monitoring mode is realized. A single PTZ camera can cover multiple areas of interest, resulting in low system cost and wide coverage. (4) Solid and reliable chain of evidence: The generated multi-dimensional evidence (pictures, videos) strictly conforms to national standards and introduces anti-tampering technology, and the evidence has strong legal effect; (5) Strong engineering practicality: The system can adapt to complex real highway environments, such as changes in lighting, rain and snow, and vehicle obstruction, and has good robustness. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof.

[0018] Figure 1 The flowchart illustrates the method for obtaining evidence of illegal driving in the emergency lane of a highway, as provided by this invention.

[0019] Figure 2A flowchart illustrating the specific implementation method of the method for collecting evidence of illegal driving in the emergency lane of a highway provided by the present invention. Detailed Implementation

[0020] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the method and system for collecting evidence of illegal driving in the emergency lane of a highway according to the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0021] This embodiment provides a method for obtaining evidence of illegal driving in the emergency lane of a highway, such as... Figure 1 As shown, the method for obtaining evidence of illegal driving in the emergency lane of a highway includes: Step S1: Use a high-definition network dome camera to perform global monitoring of the highway at a preset global monitoring position to obtain a global monitoring video of the highway; wherein, the high-definition network dome camera is deployed on the side of the highway. Step S2: Determine whether the main driving lane area in the global monitoring video of the highway is in a state of traffic congestion; when the main driving lane area in the monitoring video of the highway is in a state of traffic congestion, use the high-definition network dome camera to perform close-up monitoring of the emergency lane area of ​​the highway at the emergency lane close-up monitoring preset position to obtain close-up monitoring video of the emergency lane area. Preferably, such as Figure 2 As shown, determining whether the main driving lane area in the highway global monitoring video is in a state of traffic congestion further includes: Vehicles, main driving lanes, emergency lanes, and road markings were identified and segmented from the global monitoring video of the highway using a deep learning model. It should be noted that this invention completely abandons the method of manually drawing the monitoring area. After the system is initialized or the viewing angle of the high-definition network spherical camera changes, the algorithm will analyze the current video frame in real time. Its core is a semantic segmentation model (DeepLabV3+, UNet, etc.) trained with massive data. This model can perform pixel-level classification on the input image and accurately distinguish the sky, vegetation, road surface, driving lane lines, emergency lane, guardrail, vehicles, etc. Among them, (1) Dynamic extraction of emergency lane: The algorithm automatically outlines the precise contour of the emergency lane by identifying the area in the "road surface" pixels that is continuous with the driving lane in texture and color but separated by a white solid line or the "emergency lane" text mark. This contour is converted into a dynamic and calculable polygon monitoring area in real time. (2) Adaptive viewing angle change: When the PTZ camera rotates due to the execution of preset position call or automatic tracking, the video stream under the new viewing angle will immediately trigger a new round of scene self-understanding and regenerate the monitoring area, thereby perfectly adapting to any posture change of the PTZ camera and solving the pain point of inaccurate static area definition in traditional methods.

[0022] The vehicle detection model detects the average speed and density of vehicles in the main driving lane area. When the average speed of vehicles in the main driving lane area is lower than the preset speed threshold (e.g., 20 km / h) and the vehicle density is higher than the preset density threshold for a certain period of time (e.g., 30 seconds), it is determined that the main driving lane area is in a state of vehicle congestion.

[0023] It should be noted that this invention continuously performs traffic flow analysis on the main driving lane under the default wide-angle view of the PTZ camera. (1) Feature extraction: Using a lightweight vehicle detection model (YOLOv5s), the number, position and speed of vehicles in the main driving lane in the highway video are detected and counted in real time. (2) State judgment: A "congestion index" is defined, which is a weighted function of the reciprocal of the average vehicle speed and the vehicle density. When the index exceeds the threshold continuously (for example, the average vehicle speed is <20 km / h and the vehicle density is >80% for 30 seconds), the system determines that the road segment (main driving lane) has entered a congested state. (3) Event triggering: The judgment of the congestion event will serve as a key trigger to automatically start the "emergency lane occupation behavior monitoring" special task.

[0024] Specifically, the deep learning model is a semantic segmentation network based on an encoder-decoder structure. This semantic segmentation network is trained on a large number of labeled highway scene images and can accurately output the road element category to which each pixel belongs, thereby achieving pixel-level localization and dynamic contour extraction of the lane area.

[0025] Preferably, such as Figure 2As shown, the method of using the high-definition network dome camera to perform close-up monitoring of the emergency lane area of ​​the highway from the preset position of the emergency lane close-up monitoring when the main driving lane area in the highway monitoring video is in a state of traffic congestion also includes: When the main driving lane is congested, the high-definition network dome camera is switched from the global monitoring preset position to the emergency lane close-up monitoring preset position, automatically triggering the collection of evidence of illegal driving in the emergency lane.

[0026] Step S3: Determine whether the vehicle in the close-up monitoring video of the emergency lane area has committed a traffic violation by occupying the emergency lane; when the vehicle in the close-up monitoring video of the emergency lane area has committed a traffic violation by occupying the emergency lane, capture three process images and one license plate image of the illegal vehicle driving in the emergency lane through the high-definition network dome camera, and simultaneously record a video of the illegal vehicle's traffic violation in the emergency lane. Preferably, such as Figure 2 As shown, the process of determining whether a vehicle in the close-up surveillance video of the emergency lane area has committed a traffic violation by occupying the emergency lane also includes: A deep learning-based multi-target tracking algorithm is used to assign a unique ID to each vehicle in the close-up surveillance video of the emergency lane area, and to analyze the movement route and speed of each vehicle in the close-up surveillance video of the emergency lane area in real time. If a vehicle's route remains within the emergency lane for a certain period of time and its speed within the emergency lane exceeds the minimum driving speed threshold, the vehicle is deemed to have committed a traffic violation by occupying the emergency lane.

[0027] Specifically, vehicles that illegally use the emergency lane must meet both of the following conditions: Condition 1: The number of consecutive frames in which the center point or outline area of ​​the vehicle is located within the emergency lane exceeds the preset number of consecutive frames (e.g., 10 frames). Condition 2: The vehicle's speed in the emergency lane exceeds the minimum driving speed threshold (e.g., 5 km / h) to exclude parked vehicles.

[0028] Preferably, the process of capturing three images of the illegal vehicle driving in the emergency lane and one license plate image using the high-definition network dome camera further includes: The high-definition network dome camera is controlled to perform fast automatic focusing and zooming to capture an image of the license plate of the illegal vehicle. The license plate image is then optimized based on super-resolution reconstruction image enhancement technology to improve the recognition rate of the license plate that is blurred due to light, weather or distance.

[0029] Step S4: Generate evidence images of the illegal vehicle's violation of driving in the emergency lane based on the three process images and one license plate image of the illegal vehicle.

[0030] Preferably, the step of generating the emergency lane violation evidence image of the offending vehicle based on three process images and one license plate image further includes: The violation time, location, violation, license plate number, and legal basis information are overlaid on the image of the vehicle violating traffic rules in the emergency lane; the image of the vehicle violating traffic rules in the emergency lane complies with GA / T832 standards. The hash values ​​of the images and videos of the illegal driving in the emergency lane were calculated and stored in the database.

[0031] The method for collecting evidence of illegal driving in the emergency lane of a highway provided by this invention comprehensively applies the following key technologies: First, traffic scene self-understanding and dynamic calibration technology based on multi-task deep neural networks dynamically identifies traffic elements and real-time traffic flow status in the emergency lane area; second, automatic triggering of congestion events and PTZ camera collaborative scheduling technology based on spatiotemporal semantics automatically triggers evidence collection of illegal driving in the emergency lane; and third, a precise identification and tracking model for illegal driving in the emergency lane based on multi-target tracking and route analysis enables large-scale, high-precision, and all-weather off-site automatic evidence collection of illegal driving in the emergency lane of a highway.

[0032] As another embodiment of the present invention, a system for collecting evidence of traffic violations involving the use of emergency lanes on highways is provided, the system comprising: The first front-end sensing unit is used to perform global monitoring of the highway through a high-definition network dome camera at a global monitoring preset position to obtain a global monitoring video of the highway; wherein, the high-definition network dome camera is deployed on the side of the highway. The first central analysis unit is used to determine whether the main driving lane area in the global monitoring video of the highway is in a state of vehicle congestion. The second front-end sensing unit is used to monitor the emergency lane area of ​​the highway through the high-definition network dome camera at the emergency lane close-up monitoring preset position when the main driving lane area in the highway monitoring video is in a state of traffic congestion, so as to obtain the emergency lane area close-up monitoring video. The second central analysis unit is used to determine whether the vehicles in the close-up surveillance video of the emergency lane area have committed illegal driving behavior by occupying the emergency lane. The third front-end sensing unit is used to capture three process images and one license plate image of the vehicle driving in the emergency lane through the high-definition network dome camera when a vehicle in the close-up monitoring video of the emergency lane area is found to be illegally occupying the emergency lane, and simultaneously record a video of the illegal vehicle's illegal driving process in the emergency lane. The third center analysis unit is used to generate evidence images of the illegal vehicle's violation of driving in the emergency lane based on three process images and one license plate image.

[0033] It should be noted that multiple high-definition network dome cameras (referred to as "PTZ cameras") are deployed along the side of the highway. These high-definition network dome cameras are equipped with high-precision pan-tilt units, optical zoom lenses, and high-definition video acquisition capabilities to collect continuous video stream data. The central analysis unit is deployed in the central computer room, which is a computing cluster consisting of several high-performance servers. It carries the core algorithm software of this system. This central analysis unit is responsible for receiving, processing, and analyzing the front-end video stream, performing traffic status judgment, illegal behavior identification, and issuing control commands to the dome cameras.

[0034] It should be noted that, based on the existing fiber optic network of the highway, high-speed, low-latency transmission of video stream data and control signaling is achieved between the front-end sensing unit and the central analysis unit.

[0035] It should be noted that each high-definition network dome camera has at least two key preset positions: (1) Preset position A (global monitoring position): wide-angle view, covering multiple lanes, used for macro traffic condition monitoring.

[0036] (2) Preset position B (emergency lane close-up position): telephoto view, focusing on the emergency lane, used for detailed identification of illegal behavior.

[0037] When the system detects congestion in preset position A mode, it immediately sends a PTZ (Pan-Tilt-Zoom) command to the PTZ camera via the SDK provided by the PTZ camera manufacturer (such as ONVIF, GB / T 28181 protocol, or proprietary SDK), causing it to quickly and accurately rotate to preset position B. This scheduling strategy ensures that monitoring resources are efficiently used for the areas requiring the most attention at critical moments.

[0038] After the PTZ camera switches to a close-up view, the system enters a high-intensity analysis mode: (1) High-precision detection and Re-ID: A more accurate vehicle detection model (YOLOv7, Faster R-CNN) is used to identify all vehicles. At the same time, re-identification technology is used to ensure that vehicles can still be correctly associated after they are briefly out of sight or occluded.

[0039] (2) Robust Multi-Target Tracking (MOT): Integrates advanced trackers such as DeepSORT or ByteTrack, assigns a unique ID to each vehicle, and smoothly records its movement path. The tracking algorithm can effectively handle complex situations such as vehicle intersections and occlusions.

[0040] (3) Behavior Determination Rule Engine: This engine is not a simple area intrusion detection. It comprehensively analyzes the following spatiotemporal logic: ① Spatial Relationship: Calculates the positional relationship between the bottom center point of the vehicle tracking frame or the entire vehicle outline and the dynamically generated polygonal area of ​​the emergency lane in real time. Determines whether the vehicle has "entered" and "traveled" within this area.

[0041] ② Motion Status: The instantaneous speed of the vehicle is calculated through the sequence of motion route points. Vehicles that are legally stopped in the emergency lane due to malfunctions, accidents, etc., are excluded. Only when the speed of a vehicle in the emergency lane exceeds the minimum driving threshold (e.g., 5 km / h) is it considered to be "driving".

[0042] ③ Continuity and uniqueness: The illegal behavior must continue for a certain period of time or distance (for example, driving continuously in the emergency lane for more than 5 seconds or 20 meters) to avoid misjudging the temporary lane-changing behavior.

[0043] At the same time, the system ensures that evidence is collected only once for the same violation by the same vehicle, avoiding duplicate penalties.

[0044] Once a vehicle is confirmed to have violated the law, the system initiates the evidence collection process within milliseconds: (1) Process image capture: Control the PTZ camera to continuously capture three high-resolution images, clearly record the displacement process of the target vehicle in the emergency lane, and form dynamic evidence.

[0045] (2) License Plate Close-up Capture: Control the PTZ camera to perform rapid autofocus and zoom to capture a close-up image containing a clear vehicle license plate. During this process, image super-resolution and deblurring algorithms are integrated to improve the recognizability of license plates under adverse conditions.

[0046] (3) Process video recording: Starting about 5 seconds before the behavior recognition is triggered and ending about 10 seconds after the behavior is confirmed, record a 1080p high-definition video of no less than 15 seconds to fully show the entire process of the illegal behavior.

[0047] (4) Evidence Synthesis and Anti-counterfeiting: The system automatically synthesizes three process images and one license plate image into a "four-in-one" evidence image according to the requirements of GA / T 832-2014 "Technical Specification for Image Evidence Collection of Road Traffic Safety Violations". The system automatically overlays the violation time (accurate to milliseconds), violation location (latitude and longitude), violation behavior, license plate number, and legal basis onto the image. Finally, the generated image and video files are hashed using MD5 or SHA-256, and the hash values ​​are stored in the database to ensure the integrity and immutability of the evidence.

[0048] The system for collecting evidence of illegal driving in emergency lanes on highways provided by this invention uses a high-performance server as the core computing power carrier, runs an advanced algorithm for collecting evidence of illegal driving in emergency lanes, and interacts with a front-end PTZ camera group through a high-speed fiber optic network to achieve automatic identification, tracking, evidence collection and evidence chain generation of illegal behavior.

[0049] The specific implementation scheme of the method and system for collecting evidence of illegal driving in the emergency lane of a highway provided by this invention is as follows: I. System Basic Configuration and Device Initialization: This step is the foundation for the normal operation of the system and aims to organically integrate and optimize the parameters of the front-end physical devices and the back-end algorithm system.

[0050] 1. Fine-grained settings for multi-preset point scenarios The quality of the preset point settings directly determines the efficiency and effectiveness of monitoring. This system configures two types of core preset points for each PTZ camera: Preset monitoring points for large scenes (global perspective): (1) Purpose: To be used for macroscopic traffic flow status perception and timely detection of congestion events.

[0051] (2) Setup requirements: The pan-tilt head should have a small tilt angle and the focal length should be adjusted to the wide-angle end to ensure that the image can cover all the main lanes within the range of the pan-tilt camera.

[0052] (3) Parameter calibration: To ensure the accuracy of subsequent traffic condition analysis, camera parameter calibration needs to be performed at this preset position. By photographing a calibration board of known size or using known road dimensions (such as lane width of 3.75 meters), a mapping relationship between the image pixel coordinate system and the real-world coordinate system is established, providing a basis for subsequent calculations of vehicle speed and density.

[0053] Small scene snapshot preset point (close-up perspective): (1) Purpose: After congestion is triggered, focus on the emergency lane to conduct high-definition identification and evidence collection of illegal behaviors.

[0054] (2) Setup Requirements: The gimbal should be turned so that the emergency lane is centered in the frame, and the focus should be adjusted to the telephoto end. The license plate should be clearly distinguishable (the license plate should be at least 80 pixels wide in the frame), and the boundaries of the emergency lane and adjacent driving lanes should be captured for motion path analysis.

[0055] (3) Adaptive lighting settings: For complex lighting conditions such as night, dusk, and backlight, separate image parameter presets need to be configured under this preset position. For example, turn on the PTZ camera's infrared mode and increase the gain at night, and turn on the wide dynamic range (WDR) function to the strong setting when backlighting to ensure that the image is available all day.

[0056] 2. Adaptive PTZ Camera Polling Strategy Configuration and Task Startup To efficiently utilize limited PTZ camera resources to cover a wide road segment, the system employs an intelligent polling strategy.

[0057] Dynamic polling interval mechanism: (1) Peak / Congestion Mode: During peak traffic hours (e.g., 7:00-9:00, 17:00-19:00) or when the system detects a decrease in overall vehicle speed on a road segment, the polling interval is automatically shortened to 5-8 minutes. The system can even enter "event-driven" mode, where polling is temporarily paused after a PTZ camera detects congestion, and it is ordered to continue monitoring until the congestion is resolved.

[0058] (2) Off-peak / smooth traffic mode: During periods of low traffic, the polling interval can be extended to 20-30 minutes to save system resources and network bandwidth.

[0059] 3. Priority scheduling of pre-defined points based on illegal heatmaps: The system backend will collect historical violation data and generate a heat map of road violations. The polling controller will then prioritize dispatching PTZ cameras to pre-positioned points in areas with high violation rates based on the heat map.

[0060] 4. Anomaly recovery and self-checking mechanism: (1) Task Startup: The polling task is started when the system is powered on or at a scheduled time each day. At the same time, the monitoring center has the authority to manually start polling of any PTZ camera in real time.

[0061] (2) Heartbeat Monitoring: The system continuously monitors the "heartbeat" connection with each PTZ camera. If a network interruption, PTZ camera offline, or SDK connection failure is detected, an alarm will immediately be displayed on the management interface. (3) Automatic recovery: For temporary faults (such as network jitter), the system will automatically reconnect after 3 retry intervals (such as 30 seconds, 60 seconds, and 120 seconds). If the reconnection fails, the PTZ camera will be marked as "faulty" and its polling task will be skipped, while the maintenance personnel will be notified.

[0062] II. PTZ Camera Access and Preset Point Query: This step enables the backend system to uniformly manage heterogeneous front-end devices, which is crucial for system compatibility.

[0063] 1. Multi-brand device compatibility access layer design (1) To address the differences in SDKs among PTZ cameras from different brands (such as Hikvision, Dahua, Uniview, etc.), this system designs a unified device access layer. This layer encapsulates the proprietary SDKs of each manufacturer and provides a unified RESTful API or gRPC interface to upper-layer applications.

[0064] (2) Preparation: In the device management interface, enter the access information of each PTZ camera using “device model + IP address” as the unique identifier.

[0065] 2. Secure SDK Login and Authentication (1) The login process uses a challenge-response authentication mechanism instead of transmitting the password in plaintext. The system sends a random number (challenge) to the PTZ camera, and the PTZ camera uses the random number to perform a hash operation with the password and returns the result (response), thereby preventing the password from being eavesdropped.

[0066] (2) For new PTZ cameras that support OAuth 2.0 or digital certificate authentication, the system will prioritize the more secure authentication method.

[0067] 3. Standardized acquisition and caching of pre-defined point information (1) By calling the unified queryPresets(deviceId) API, the access layer converts it into a call to a specific vendor's SDK (such as Hikvision's NET_DVR_GetPreset).

[0068] (2) The acquired preset point information (preset point number, name, coordinates) is parsed and standardized into an internal system data model and stored in the cache database. This avoids the need to query again every time a poll is conducted, greatly improving efficiency.

[0069] 4. Automated Equipment Compatibility Testing During the system integration phase, an automated test script will run. This script will iterate through all the recorded PTZ cameras, perform the entire process of "login - query preset point - rotate to preset point - capture image - return to position", and generate a test report to quickly locate incompatible devices.

[0070] III. Implementation of the core process for obtaining evidence of illegal activities: This is the core algorithm execution stage of the invention, which realizes the fully automated conversion from video data to illegality determination.

[0071] 1. Highly reliable sensor data acquisition and video stream access Protocol selection: (1) RTSP (Real-time Streaming Protocol): It is preferred for use in local area networks or environments with extremely high network quality. Its latency can be controlled within 200ms, ensuring the real-time performance of control.

[0072] (2) GB / T 28181: Used for scenarios of cross-public network or large-scale cascading monitoring. It has good network penetration and unified management capabilities. Although the latency is slightly higher (500ms-1s), it is more suitable for wide-area deployment.

[0073] Reconnection after disconnection and adaptive bitrate: (1) The system maintains a watchdog thread for each video stream connection to periodically check the frame rate. If the frame rate drops below a threshold (e.g., 15 frames per second) or the stream completely drops, the watchdog will immediately destroy the old connection and attempt to rebuild a new connection according to an exponential backoff strategy (e.g., every 1 second, 2 seconds, 4 seconds...). (2) The system monitors the network bandwidth in real time. When the bandwidth is insufficient, it dynamically sends instructions to the PTZ camera to reduce the video bitrate or resolution in order to prioritize the continuity of the video stream.

[0074] 2. In-depth understanding of YOLOv7 and semantic analysis scenarios Precise detection of traffic elements: (1) A customized detection model based on YOLOv7 was adopted. This model was pre-trained on the COCO dataset and then transferred to tens of thousands of labeled highway images. The labeled categories included: cars, trucks, buses, emergency lanes, driving lane lines (solid / dashed lines), road edges, etc.

[0075] (2) The model can achieve an inference speed of more than 30 frames per second on a Tesla T4 GPU, meeting the real-time requirements. Its detection accuracy (mAP@0.5) for vehicles and lane lines can reach more than 98.5%.

[0076] Intelligent judgment of congestion status: (1) Algorithm: In the video stream of preset points in a large scene, the system periodically (e.g., every 2 seconds) runs YOLOv7 detection and calculates two key metrics: (2) Vehicle density: The number of vehicles in the main lane in the picture is counted and divided by the total area of ​​the lane (pixel level) to obtain the normalized density value.

[0077] (3) Average speed: The vehicle displacement between consecutive frames is estimated by optical flow or a lightweight correlation filter tracker (such as KCF), and then converted into the real speed by combining the camera calibration parameters.

[0078] (4) Trigger threshold: If the average speed of vehicles is lower than 20 km / h and the density is higher than 0.7 (range 0-1) for 5 consecutive cycles (i.e. 10 seconds), it is determined to be a congestion event, and the PTZ camera is immediately triggered to schedule to the preset small scene preset point.

[0079] 3. A precise identification model for driving in the emergency lane. Vehicle key point detection and tracking: (1) After the PTZ camera switches to a close-up view of a small scene, a more complex multi-target tracking (MOT) process is enabled.

[0080] (2) First, YOLOv7 is used to detect all vehicles and regress the 2D bounding boxes and bottom center key points of the vehicles. The bottom center point is more accurate than the vehicle center point in reflecting the contact position between the vehicle and the road surface.

[0081] (3) Subsequently, the DeepSORT tracking algorithm is adopted. It assigns a unique ID to each detected target, uses Kalman filtering to predict its position in the next frame, and then performs correlation matching through Mahalanobis distance and appearance features (using a deep feature extraction network) to form a smooth and continuous motion path.

[0082] Spatial Relationship Modeling and Behavior Determination Rule Engine: (1) Dynamic emergency lane region generation: The system no longer relies on fixed boxes, but runs a lightweight semantic segmentation model (such as U-Net) in real time to segment the “emergency lane” region from the current close-up viewpoint, forming a dynamic polygon mask that changes with the viewpoint.

[0083] (2) Behavior determination logic: For each vehicle in the tracking sequence, the rule engine performs the following judgment: ① Position determination: Calculate whether the center point of the vehicle's bottom is located within the dynamically generated polygonal area of ​​the emergency lane. Use the cv2.pointPolygonTest function to make this determination.

[0084] ② Determine the vehicle's motion status: Calculate the instantaneous speed of the vehicle along its route. Exclude vehicles with speeds below 5 km / h (which may be disabled vehicles or emergency vehicles actually performing a mission).

[0085] ③ Continuous judgment: Only when a vehicle meets the condition of "being in the emergency lane and having a speed higher than the threshold" for 10 consecutive frames (approximately 0.3 seconds) or more will it be ultimately judged as "driving in the emergency lane" as a violation. This setting effectively filters out false judgments of temporary lane changes or driving over the line.

[0086] 5. Phased and multi-dimensional image capture and information collection Triggering and image capture logic: (1) Once the behavior determination engine outputs a positive violation signal, the system immediately starts the high-speed continuous shooting mode.

[0087] (2) Process Image Capture: Control the PTZ camera to capture three high-definition images continuously at the highest frame rate (e.g., 25fps), with an interval of approximately 0.5 seconds, to ensure that the vehicle's movement process within the emergency lane is clearly displayed. The image resolution should be no less than 1920x1080.

[0088] (3) License Plate Close-up Capture: During continuous shooting, the system issues commands in parallel to control the PTZ camera to perform rapid automatic zoom and focus, pulling the focal length to its maximum to capture a close-up image with the vehicle license plate as the clear subject. In this process, an image super-resolution (SR) algorithm is integrated to enhance potentially blurry license plates in software, further improving the recognition rate.

[0089] Video recording of the illegal process: The system employs a circular buffer technique to continuously cache the most recent 10 seconds of video stream. When an illegal act is triggered, the system not only saves the video of the 5 seconds following the trigger but also saves the video of the 10 seconds preceding the trigger from the buffer, together forming a complete video of the illegal act lasting no less than 15 seconds. The video is encoded in H.264 format, with a resolution of 1080p and a frame rate of 25fps.

[0090] IV. Standardized Generation and Application of Evidence Chain: This step ensures that all evidence collection results comply with legal and regulatory requirements and form an irrefutable evidence chain.

[0091] 1. Automated synthesis of four-in-one illegal evidence images (1) Layout and synthesis: The system uses image processing libraries such as OpenCV to seamlessly stitch together three process images (in chronological order) and a close-up image of a license plate in a 2x2 grid layout to generate a synthesized "four-in-one" evidence image.

[0092] (2) Information overlay: On the composite image, strictly in accordance with the requirements of GA / T 832-2014 "Technical Specification for Image Evidence Collection of Road Traffic Safety Violations", the following character information shall be overlaid: ① Time of violation: accurate to milliseconds, format: YYYY-MM-DD HH:MM:SS, such as 2023-10-27 15:30:45.245. The time source is a unified Network Time Protocol (NTP) server to ensure time synchronization of all devices.

[0093] ②Location of violation: Format: XX Expressway XX direction KXXX+XXX meters, such as G50 Shanghai-Chongqing Expressway Shanghai direction K124+150 meters.

[0094] ③ License plate number: The license plate recognition algorithm identifies and automatically fills in the number from the close-up image. The recognition result needs to be verified by confidence level. Those with a confidence level below 95% will be marked as "awaiting manual review".

[0095] ④ Illegal behavior: Driving in the emergency lane in non-emergency situations.

[0096] ⑤ Illegal code: Fixed code 4608A.

[0097] ⑥ Equipment Code: The unique serial number of the PTZ camera used for evidence collection.

[0098] ⑦ Anti-counterfeiting code: A unique serial number generated by the system, which can be queried in conjunction with the backend database.

[0099] (3) Font and Clarity: All characters are in bold, white with a black outline, to ensure readability on any background. The final image is saved as a high-quality JPEG.

[0100] 2. Synchronization and processing of video evidence (1) The video of the illegal process also needs to be overlaid with the same information as the images, such as time, location, and device encoding. The system uses the FFmpeg tool library to hard-encode the text watermark into each frame of the video to prevent it from being stripped off.

[0101] (2) Complete evidence information will be written into the metadata of the video file.

[0102] 3. Tamper-proof technology and assurance of the integrity of the chain of evidence (1) Hash verification: For the final generated "four-in-one" image and illegal process video file, the system calculates its SHA-256 hash value, prints the hash value in plaintext on the evidence image, and stores it in the central database.

[0103] (2) Blockchain Evidence Preservation (Optional Advanced Function): To address potential legal challenges, the system can upload key evidence (image hash, video hash, illegal data) to the judicial blockchain evidence preservation platform to obtain a unique blockchain evidence preservation certificate, further solidifying the generation time and content of the evidence and achieving absolute immutability.

[0104] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for collecting evidence of illegal driving in the emergency lane of a highway, characterized in that, include: Step S1: Use a high-definition network dome camera to perform global monitoring of the highway at a preset global monitoring position to obtain a global monitoring video of the highway; wherein, the high-definition network dome camera is deployed on the side of the highway. Step S2: Determine whether the main driving lane area in the global monitoring video of the highway is in a state of traffic congestion; when the main driving lane area in the monitoring video of the highway is in a state of traffic congestion, use the high-definition network dome camera to perform close-up monitoring of the emergency lane area of ​​the highway at the emergency lane close-up monitoring preset position to obtain close-up monitoring video of the emergency lane area. Step S3: Determine whether the vehicle in the close-up monitoring video of the emergency lane area has committed a traffic violation by occupying the emergency lane; when the vehicle in the close-up monitoring video of the emergency lane area has committed a traffic violation by occupying the emergency lane, capture three process images and one license plate image of the illegal vehicle driving in the emergency lane through the high-definition network dome camera, and simultaneously record a video of the illegal vehicle's traffic violation in the emergency lane. Step S4: Generate evidence images of the illegal vehicle's violation of driving in the emergency lane based on the three process images and one license plate image of the illegal vehicle.

2. The method for collecting evidence of illegal driving in the emergency lane of a highway according to claim 1, characterized in that, The determination of whether the main driving lane area in the global monitoring video of the highway is in a state of traffic congestion also includes: Vehicles, main driving lanes, emergency lanes, and road markings were identified and segmented from the global monitoring video of the highway using a deep learning model. The vehicle detection model detects the average speed and density of vehicles in the main driving lane area. When the average speed of vehicles in the main driving lane area is lower than the preset speed threshold and the vehicle density is higher than the preset density threshold for a certain period of time, it is determined that the main driving lane area is in a state of vehicle congestion.

3. The method for collecting evidence of illegal driving in the emergency lane of a highway according to claim 2, characterized in that, The deep learning model is a semantic segmentation network based on an encoder-decoder structure. This semantic segmentation network is trained on a large number of labeled highway scene images and can accurately output the road element category to which each pixel belongs.

4. The method for collecting evidence of illegal driving in the emergency lane of a highway according to claim 1, characterized in that, The method of using the high-definition network dome camera to perform close-up monitoring of the emergency lane area of ​​the highway from a preset position when the main driving lane area in the highway monitoring video is congested also includes: When the main driving lane is congested, the high-definition network dome camera is switched from the global monitoring preset position to the emergency lane close-up monitoring preset position, automatically triggering the collection of evidence of illegal driving in the emergency lane.

5. The method for collecting evidence of illegal driving in the emergency lane of a highway according to claim 1, characterized in that, The process of determining whether a vehicle in the close-up surveillance video of the emergency lane area has committed a traffic violation by occupying the emergency lane also includes: A deep learning-based multi-target tracking algorithm is used to analyze the movement route and speed of each vehicle in the close-up surveillance video of the emergency lane area in real time. If a vehicle's route remains within the emergency lane for a certain period of time and its speed within the emergency lane exceeds the minimum driving speed threshold, the vehicle is deemed to have committed a traffic violation by occupying the emergency lane.

6. The method for collecting evidence of illegal driving in the emergency lane of a highway according to claim 1, characterized in that, The process of capturing three images of the illegal vehicle driving in the emergency lane and one license plate image using the high-definition network dome camera also includes: The high-definition network dome camera is controlled to perform rapid automatic focusing and zooming, capture an image of the license plate of the illegal vehicle, and optimize the license plate image based on image enhancement technology based on super-resolution reconstruction.

7. The method for collecting evidence of illegal driving in the emergency lane of a highway according to claim 1, characterized in that, The process of generating evidence images of the illegal vehicle's emergency lane driving violation based on three process images and one license plate image also includes: The time, location, violation, license plate number, and legal basis information of the violation are overlaid on the image of the vehicle violating traffic rules by driving in the emergency lane. The hash values ​​of the images and videos of the illegal driving in the emergency lane were calculated and stored in the database.

8. A system for collecting evidence of traffic violations involving the use of emergency lanes on highways, used to implement the method for collecting evidence of traffic violations involving the use of emergency lanes on highways as described in any one of claims 1-7, characterized in that, The system for collecting evidence of violations of occupying the emergency lane on highways includes: The first front-end sensing unit is used to perform global monitoring of the highway through a high-definition network dome camera at a global monitoring preset position to obtain a global monitoring video of the highway; wherein, the high-definition network dome camera is deployed on the side of the highway. The first central analysis unit is used to determine whether the main driving lane area in the global monitoring video of the highway is in a state of vehicle congestion. The second front-end sensing unit is used to monitor the emergency lane area of ​​the highway through the high-definition network dome camera at the emergency lane close-up monitoring preset position when the main driving lane area in the highway monitoring video is in a state of traffic congestion, so as to obtain the emergency lane area close-up monitoring video. The second central analysis unit is used to determine whether the vehicles in the close-up surveillance video of the emergency lane area have committed illegal driving behavior by occupying the emergency lane. The third front-end sensing unit is used to capture three process images and one license plate image of the vehicle driving in the emergency lane through the high-definition network dome camera when a vehicle in the close-up monitoring video of the emergency lane area is found to be illegally occupying the emergency lane, and simultaneously record a video of the illegal vehicle's illegal driving process in the emergency lane. The third center analysis unit is used to generate evidence images of the illegal vehicle's violation of driving in the emergency lane based on three process images and one license plate image.

Images