License plate continuous lane changing monitoring management system based on computer vision and Internet of Things

Abstract

The invention relates to a license plate continuous lane changing monitoring management system based on computer vision and the Internet of Things. The system comprises a vehicle information acquisition layer used for acquiring vehicle images frame by frame and preprocessing the vehicle images to obtain vehicle image data; the vehicle image data comprises a vehicle image frame sequence and a timestamp corresponding to each vehicle image frame; the visual identification layer is used for carrying out license plate positioning and license plate number identification on the vehicle image data by adopting a computer visual model to obtain a license plate identification information group; the license plate identification information group comprises a license plate number in the vehicle image frame, a license plate position corresponding to the license plate number and a timestamp; the continuous lane changing recognition layer is used for carrying out continuous lane changing behavior judgment on the license plate position change of the same license plate number in the vehicle image data and the corresponding timestamp based on the dynamic traffic congestion degree to obtain a continuous lane changing recognition result; the continuous lane change recognition result comprises illegal continuous lane change. By adopting the system, the lane changing behavior can be quickly identified in an environment with complex traffic flow and dynamic change.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle identification, and in particular relates to a license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things. Background Art

[0002] With the continuous growth of urban motor vehicle ownership, road traffic pressure has increased significantly, and traffic violations have seriously affected road safety and traffic efficiency. Frequent and continuous lane changes by motorists not only disrupt traffic order but also easily lead to traffic accidents such as rear-end collisions and collisions, making them a key risk in current urban traffic management.

[0003] Existing traffic monitoring systems generally rely on fixed-point cameras for image acquisition, supplemented by manual review or semi-automatic identification methods to determine traffic violations. However, for scenarios where the behavioral characteristic of "continuous lane changes" requires both timing and continuity, traditional systems face the following major problems:

[0004] Most systems lack the definition and judgment rules for "continuous lane changing" behavior. They can only identify static or instantaneous events such as single lane changes, wrong-way driving, and lane occupation, making it difficult to capture continuous driving behavior characteristics. In addition, license plate recognition modules and behavior judgment modules are often deployed in a decentralized manner, and communication and data transmission rely on central servers, resulting in poor real-time performance and delayed response, making it difficult to achieve "identification and judgment at the same time." Most current violation identification systems do not take into account differences in road congestion conditions, and their judgment rules are one-size-fits-all. They fail to dynamically adjust the lane change judgment threshold, making it easy to make misjudgments or missed judgments. The evidence collection process in existing systems is cumbersome and often relies on manual screening of video clips or pictures, posing challenges to law enforcement efficiency and the legitimacy of evidence. Summary of the Invention

[0005] Based on this, it is necessary to provide a license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things that can monitor continuous lane change behavior in real time to address the above technical problems.

[0006] In a first aspect, the present application provides a license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things, comprising:

[0007] The vehicle information acquisition layer is used to collect vehicle images frame by frame and perform preprocessing to obtain vehicle image data; the vehicle image data includes a sequence of vehicle image frames and the timestamp corresponding to each vehicle image frame;

[0008] The visual recognition layer is used to use a computer vision model to locate and recognize the license plate of the vehicle image data to obtain a license plate recognition information group; the license plate recognition information group includes the license plate number in the vehicle image frame, the license plate position corresponding to the license plate number, and a timestamp;

[0009] The continuous lane change recognition layer is used to determine the continuous lane change behavior of the same license plate number in the vehicle image data based on the dynamic traffic congestion level and the corresponding timestamp, and obtain the continuous lane change recognition result; the continuous lane change recognition result includes illegal continuous lane changes.

[0010] In one embodiment, based on the dynamic traffic congestion level, the continuous lane change behavior is determined by analyzing the license plate position changes and corresponding timestamps of the same license plate number in the vehicle image data, and the continuous lane change recognition result is obtained, including:

[0011] Obtaining a continuous lane change time threshold based on dynamic traffic congestion;

[0012] Performing a difference calculation on the horizontal coordinates of the license plate positions of two adjacent vehicle image frames in the vehicle image frame sequence for the same license plate number to obtain a license plate coordinate change value;

[0013] According to the preset judgment rules and the change value of the license plate coordinates, the driving behavior type of the vehicle is judged; the driving behavior type includes lane changing behavior and straight driving behavior;

[0014] The lane-changing behavior distribution is analyzed according to the behavior type to obtain the continuous lane-changing recognition result.

[0015] In one embodiment, the preset judgment rule corresponds to the following steps:

[0016] If the license plate coordinate change exceeds a preset first displacement threshold, the vehicle is determined to have completed a lane change and the timestamp corresponding to the lane change is recorded;

[0017] If the license plate coordinate change value is less than the preset second displacement threshold, it is determined that the vehicle has completed a straight-line behavior, and the timestamp corresponding to the straight-line behavior is recorded.

[0018] In one embodiment, analyzing the lane change behavior distribution according to the behavior type to obtain the continuous lane change recognition result includes:

[0019] The number and distribution of lane changes within a preset time period are counted. If two or more lane changes occur consecutively, the continuous lane change recognition result is obtained using the following method:

[0020] Obtain the timestamps corresponding to two adjacent lane change behaviors and calculate the difference between the two timestamps to obtain the time interval; the two adjacent lane change behaviors come from a sequence of more than two consecutive lane change behaviors;

[0021] If the time interval is less than the continuous lane change time threshold, the continuous lane change identification result is determined to be an illegal continuous lane change.

[0022] In one embodiment, the number and distribution of lane changes within a preset time period are counted according to the following settings:

[0023] When a vehicle completes a lane change, the number of lane changes is increased by one, and the number of straight-ahead behaviors is reset to zero;

[0024] When a vehicle completes a straight-line behavior, the number of straight-line behaviors is increased by one;

[0025] If the duration of the vehicle's consecutive straight-moving behaviors exceeds the preset time threshold, the number of currently recorded lane changes and straight-moving behaviors will be reset to zero.

[0026] In one embodiment, the dynamic traffic congestion degree is obtained by the congestion determination unit through the following method:

[0027] Obtain the travel time differences of several newly identified vehicles and their corresponding first identification times;

[0028] The travel time differences of newly identified vehicles are added to a fixed-length circular stack in the order of their first identification time, and used to construct the current congestion judgment window according to preset conditions; the congestion judgment window is composed of a time difference set consisting of multiple travel time differences;

[0029] Calculate the average travel time of the time difference set of the congestion judgment window;

[0030] The average travel speed is calculated based on the fixed distance of the monitored section and the average travel time;

[0031] Map the average traffic speed to the traffic congestion level model to obtain the dynamic traffic congestion degree;

[0032] Clear the current loop stack and open a new congestion judgment window to receive the travel time difference data of newly identified vehicles;

[0033] The current congestion judgment window is generated when any of the following preset conditions is met:

[0034] The amount of data in the current loop stack reaches a fixed length;

[0035] The time interval between the first recognition time of the newly recognized vehicle and the first recognition time of the most recently added vehicle in the loop stack exceeds a preset time interval threshold.

[0036] In one embodiment, the travel time difference of a newly identified vehicle and its corresponding first identification time are obtained by the following method:

[0037] In response to a new license plate detection instruction, the front camera unit publishes a first tuple corresponding to the new license plate to a first preset topic via a lightweight message transmission protocol and generates a green light on instruction; the first tuple includes the license plate number and the time of first recognition; the green light on instruction is used to instruct the green light of the front integrated device to turn on; the new license plate detection instruction is generated by retrieving the license plate number from a corresponding license plate number cache library;

[0038] The rear camera unit responds to the detected new license plate instruction and publishes a second tuple corresponding to the new license plate to a second preset topic via a lightweight message transmission protocol; the second tuple includes the license plate number and the second recognition time;

[0039] The cloud computing unit obtains the first tuple and the second tuple by subscribing to the first preset topic and the second preset topic, and calculates the difference based on the first recognition time and the second recognition time to obtain the travel time difference of the newly recognized vehicle and its corresponding first recognition time.

[0040] In one embodiment, the visual recognition layer includes a license plate location unit and a license plate number recognition unit;

[0041] The license plate positioning unit is used to analyze the vehicle image data through the license plate positioning model to determine the license plate position;

[0042] The license plate number recognition unit is used to identify the area corresponding to the license plate position in the vehicle image data through the license plate number recognition model to obtain the license plate number.

[0043] In one embodiment, the license plate location model is obtained by:

[0044] Obtain a data set and perform data enhancement processing on the data set to obtain a first training set; the data set includes real-scene license plate images and their corresponding annotated license plate frames;

[0045] The first training set is used to adjust the parameters of the pre-trained YOLOv8 model, and the parameters are optimized by configuring the loss function weights to obtain the license plate positioning model;

[0046] The license plate recognition model is obtained in the following way:

[0047] Obtain a license plate image and its corresponding license plate character annotation to obtain a second training set; the license plate image is segmented based on the license plate position generated by the license plate positioning model;

[0048] The second training set is used to adjust the parameters of the pre-trained LPRNet model, and the parameters are optimized through the CTC loss function to obtain the license plate recognition model.

[0049] In one embodiment, the system further includes a violation reporting layer, which includes a violation instruction layer, a reporting layer, and a push layer;

[0050] The violation instruction layer is used to generate a photo instruction and a red light on instruction when the continuous lane change recognition result is a violation. The photo instruction is used to instruct the camera of the front integrated device to take a photo of the illegal continuous lane change behavior and provide feedback in the form of violation image evidence in a set image format. The red light on instruction is used to instruct the red light of the front integrated device to turn on.

[0051] The reporting layer is used to receive image evidence of violations and push the violation evidence to the push layer through a lightweight message transmission protocol; the violation evidence includes the violation image evidence, its corresponding license plate number, and the violation timestamp;

[0052] The push layer is used to store violation evidence in logs, analyze the violation evidence, and push it to the traffic police client through the Internet of Things protocol.

[0053] The aforementioned license plate continuous lane change monitoring and management system, based on computer vision and the Internet of Things, utilizes frame-by-frame image acquisition and a visual recognition layer to rapidly identify lane changes in complex and dynamically changing traffic environments. Compared to traditional monitoring systems based on static cameras or time interval calculations, this system offers greater real-time performance and accuracy, enabling it to sensitively respond to rapidly changing traffic conditions and promptly identify dangerous driving behavior. The system dynamically analyzes real-time traffic data and adaptively adjusts lane change decision rules based on varying traffic densities. This ensures efficient traffic monitoring and decision-making in all scenarios, avoiding the inefficiency and misjudgment of static threshold rules in high-density traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 This is an application environment diagram of the license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things;

[0056] Figure 2 This is a structural diagram of the license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things of the present invention;

[0057] Figure 3 A schematic diagram of the process of performing continuous lane change recognition by the continuous lane change recognition layer of the present invention;

[0058] Figure 4The figure is a flow chart of the license plate continuous lane change monitoring and management method based on computer vision and the Internet of Things of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0060] The license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things provided in the embodiment of the present application relies on Internet communication and edge computing and can be applied to Figure 1 In the application environment shown. Among them, the all-in-one machines arranged at the front and rear ends of the monitoring road communicate with the server through the network, and the terminal 103 communicates with the cloud server 104 through the network. The data storage system can store the data that the cloud server 104 needs to process. The data storage system can be integrated on the cloud server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, etc. The all-in-one machine includes a front-end all-in-one machine 101 and a back-end all-in-one machine 102. The front-end all-in-one machine 101 and the back-end all-in-one machine 102 are equipped with camera modules, or are directly connected to road cameras through the Internet of Things communication protocol. The all-in-one machine is also equipped with red and green indicator lights. The cloud server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0061] In an exemplary embodiment, Figure 2 As shown in the figure, a license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things is provided. Figure 1 Take the all-in-one device in the figure as an example to illustrate:

[0062] The license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things includes:

[0063] The vehicle information collection layer 210 is used to collect vehicle images frame by frame and perform preprocessing to obtain vehicle image data; the vehicle image data includes a vehicle image frame sequence and a timestamp corresponding to each vehicle image frame.

[0064] Schematically, the vehicle information acquisition layer 210 is deployed on the system's front-end integrated computer. In conjunction with road surveillance cameras, it continuously captures vehicles entering the monitoring area through frame-by-frame image acquisition. The vehicle information acquisition layer 210 is primarily responsible for image acquisition and data preprocessing of vehicles within the target monitoring area, providing a time-sequential, stable, and clearly structured image input stream for the subsequent license plate recognition and behavior assessment modules. Optionally, the system employs a frame-by-frame acquisition strategy, completing image analysis and behavior assessment while the vehicle is still within the monitoring field of view to support rapid response and real-time law enforcement. This means that all processing tasks are triggered immediately after acquisition, forming a frame-stream processing architecture. Image frames are acquired at fixed intervals and preprocessed to achieve resolution normalization. To ensure computational stability, the system normalizes the image resolution to 854×480 pixels (points). A coordinate filtering mechanism is also implemented to remove frames with drift or detection failures, ensuring compatibility and stability of the subsequent model across different device conditions. Furthermore, each frame is assigned a corresponding timestamp for subsequent behavior timing analysis.

[0065] Furthermore, vehicle image data is organized as a sequence of image frames and a collection of timestamps. Compared to traditional event-triggered photography mechanisms, this system's vehicle information collection layer offers greater continuity and timing accuracy, enabling the capture of subtle motion features during lane changes, improving overall system responsiveness and recognition accuracy.

[0066] The visual recognition layer 220 is used to use a computer vision model to locate and recognize the license plate of the vehicle image data to obtain a license plate recognition information group; the license plate recognition information group includes the license plate number in the vehicle image frame, the license plate position corresponding to the license plate number, and a timestamp.

[0067] The visual recognition layer 220 undertakes the two core tasks of license plate detection and character recognition. Its goal is to accurately identify the license plate number and its position coordinates in the current image from continuous image frames, and synchronously associate the timestamp information to form a license plate recognition information group that can be used for time series tracking and behavior modeling. Schematically, to achieve high-accuracy license plate positioning, the system uses a self-trained deep learning target detection model based on YOLOv8x-LPRNet to perform bounding box detection on the license plate and crop the license plate image area for character recognition. The final recognition result constitutes a license plate recognition information group in the form of a triple of license plate number, license plate position and timestamp, where the license plate position is reflected in the form of image coordinates.

[0068] The continuous lane change recognition layer 230 is used to determine the continuous lane change behavior of the same license plate number in the vehicle image data based on the dynamic traffic congestion level and the corresponding timestamp, and obtain the continuous lane change recognition result; the continuous lane change recognition result includes illegal continuous lane change.

[0069] The continuous lane change recognition layer 230 determines whether a vehicle has violated traffic regulations by continuously changing lanes based on the license plate position trends and recognition timestamps within the image frame sequence, and outputs a final behavior classification result. This module integrates the license plate recognition results with road congestion information dynamically acquired from the cloud server to implement scenario-adaptive behavior rule determination.

[0070] Specifically, lane change behavior is determined based on the change in the license plate's horizontal coordinate, Δx, across consecutive frames. When Δx between two consecutive frames exceeds a set threshold, the system determines that the vehicle has changed lanes. If Δx is less than the set straight-ahead criterion, the vehicle is considered to be traveling in a straight line. The system maintains a counter to record the number of lane changes and the duration of consecutive straight-ahead driving for each vehicle. Furthermore, a time interval threshold, t, corresponding to the road congestion level, is used to determine whether the behavior between lane changes complies with traffic regulations. If the lane change frequency is high and the time interval threshold, t, is not met between lane changes, the system identifies the behavior as an illegal continuous lane change and triggers the photo capture command and image evidence collection logic.

[0071] Optionally, the road congestion level is calculated by the cloud based on the average traffic speed of the current road section and mapped to different congestion levels according to traffic industry standards. Each level corresponds to a set of lane change determination time thresholds.

[0072] Compared with traditional lane change recognition methods, the continuous lane change recognition layer significantly enhances the ability to accurately judge high-frequency lane change behaviors in a short period of time, avoiding misjudgments caused by ignoring changes in traffic conditions. At the same time, it supports flexible adjustment of rules and has good adaptability and scalability.

[0073] In the above-mentioned license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things, the system can capture each dynamic frame data of the vehicle in real time through frame-by-frame acquisition and efficient image processing algorithms, avoiding the image delay problem that may occur in traditional monitoring methods. Combining license plate number recognition and position change determination, the present invention can accurately identify the continuous lane change behavior of the vehicle, especially in cases of complex traffic flow and blurred road signs, while still maintaining a high recognition rate to avoid missed judgments and misjudgments. The introduction of dynamic traffic congestion allows the lane change determination time threshold to be adjusted according to the real-time traffic status. It not only avoids the mistaken injury of violation determination under high density, but also enhances the adaptability of the system.

[0074] In one embodiment, based on the dynamic traffic congestion level, the continuous lane change behavior is determined by analyzing the license plate position changes and corresponding timestamps of the same license plate number in the vehicle image data, and the continuous lane change recognition result is obtained, including:

[0075] S301. Obtain a continuous lane change time threshold according to a dynamic traffic congestion level.

[0076] Traditional methods for analyzing lane-changing behavior mostly adopt a static time threshold method, without considering that the actual road conditions can affect the driving behavior rhythm, which is prone to misjudgment or missed judgment.

[0077] Schematically, the time interval threshold required for analyzing lane-changing behavior is dynamically adjusted according to the change of the real-time road traffic congestion degree. Exemplarily, according to the mapping relationship between the dynamic traffic congestion degree and the continuous lane-changing time threshold, in the non-congested state, the system sets the continuous lane-changing time threshold to 2.0 seconds, while in the severely congested state, the threshold is increased to 3.5 seconds.

[0078] Table 1 Mapping relationship between dynamic traffic congestion degree and continuous lane-changing time threshold

[0079] Speed (50，30] (30，25] (25，15] (15，0] Dynamic traffic congestion No congestion Mild congestion Moderate congestion Severe congestion Continuous lane change time threshold 2 seconds 2.5 seconds 3 seconds 3.5 seconds

[0080] S302. Calculate the difference between the abscissas of the license plate positions in two adjacent vehicle image frames in the vehicle image frame sequence for the same license plate number to obtain the license plate coordinate change value.

[0081] Schematically, by sorting the positioning information of the same license plate number in the frame sequence in time, the abscissa x of the license plate center position in each frame is obtained , , , ,

[0085] ,

[0083] , t-1 ,

[0084] ,

[0082] , and calculate the difference Δx = x t -x t-1 . Among them, the value of Δx is used as an approximate estimate of the vehicle's lateral movement.

[0082] S303. Judge the type of driving behavior of the vehicle according to the license plate coordinate change value according to the preset judgment rule; the types of driving behavior include lane-changing behavior and straight-going behavior.

[0083] After obtaining the abscissa difference Δx between each frame, the system will classify the behavior type according to the set judgment rule. Schematically, if |Δx| > T1, it is determined as a lane-changing behavior; if |Δx| < T2, it is determined as a straight-going behavior; where T1 and T2 are the lane-changing judgment displacement threshold and the straight-going judgment stability threshold respectively. Further, to improve the robustness, the system has a tolerance interval, and when Δx is in the interval [T2, T1], it is not judged and only recorded as a transition state to effectively avoid misjudgment caused by non-actual lateral movement such as picture jitter and optical flow disturbance.

[0084] S304. Analyze the lane-changing behavior distribution according to the behavior type to obtain the continuous lane-changing recognition result.

[0085] After acquiring a series of frame-level behavior labels, the system analyzes the behavior trajectory of each license plate number to determine whether there are any irregular continuous lane changes. Schematically, the system maintains two status counters: the lane change counter (difdic[license plate]), which increments by 1 each time a lane change is detected, and the straight driving counter (samedic[license plate]), which increments by 1 each time a straight driving behavior is detected.

[0086] Specifically, if the lane change counter reaches 2 and there's no straight-ahead time between two consecutive lane changes that meets the set continuous lane change time threshold, t, the system determines the vehicle has violated continuous lane change regulations and outputs a continuous lane change identification result. For example, in a non-congested state, t is set to 2.0 seconds. If a vehicle records two consecutive lane changes within less than 2 seconds, it is considered a continuous lane change violation.

[0087] In one embodiment, the preset judgment rule corresponds to the following steps:

[0088] If the license plate coordinate change value exceeds a preset first displacement threshold, it is determined that the vehicle has completed a lane change behavior, and the timestamp corresponding to the lane change behavior is recorded.

[0089] If the license plate coordinate change value is less than the preset second displacement threshold, it is determined that the vehicle has completed a straight-line behavior, and the timestamp corresponding to the straight-line behavior is recorded.

[0090] Illustratively, the first displacement threshold is the minimum value of the vehicle coordinate change required for a lane change, and the second displacement threshold is the maximum value of the vehicle coordinate change allowed for a vehicle traveling straight. For example, the section of Changning Road west of Zhongshan Park has six 5.49-meter-wide motor vehicle lanes, two 1.83-meter-wide sidewalks, and a 3.66-meter-wide bridleway. After compressing the image to 854pt*480pt, the width of a single motor vehicle lane appears to be approximately 116.45pt. Optionally, if a vehicle completes a lane change, its license plate position will shift laterally by 110pt. This change in the license plate coordinates, i.e., the horizontal coordinate change, can be used to determine whether a single lane change has been completed. The displacement threshold for a vehicle maintaining straight driving is 20pt, which is used to determine whether the vehicle maintains straight driving behavior.

[0091] Preset judgment rules are used to classify a vehicle's motion state in adjacent image frames as either lane change or straight-ahead behavior, thereby establishing the basic decision-making mechanism within the behavior recognition layer. Specifically, the judgment rules use the change in the license plate's horizontal coordinate within the image frame as the classification basis, and record this information in conjunction with timestamp information. For example, the system calculates the horizontal coordinate difference between two consecutive image frames with the same license plate number within the image frame sequence, denoted as Δx. Δx is compared with two displacement thresholds. If Δx exceeds the first displacement threshold, the system determines that the vehicle corresponding to the current frame has undergone a lane change. The system then increments the lane change counter field difdic[plate] corresponding to the license plate number in the behavior dictionary and records the current frame's timestamp in the lane change time list diftime[plate] for that license plate number for subsequent frequency and interval analysis. If Δx is less than the second displacement threshold, the system determines that the vehicle is in a straight-ahead state; the straight-ahead counter field samedic[plate] is incremented, and the current frame's timestamp is recorded in the straight-ahead time sequence sametime[plate].

[0092] To reduce the impact of edge image jitter or recognition errors, when Δx is between two thresholds, the system temporarily suspends behavior classification and only caches the image and coordinates, making supplementary judgments when subsequent behavioral trends become clear. This enhances the robustness of behavior judgment and is particularly suitable for scenarios with low vehicle speeds or high image frame rates, avoiding misjudging slight vehicle jitters as lane changes.

[0093] The above rules not only achieve efficient classification of behavior types, but also provide reliable basic data for the subsequent steps of the system by binding behavior to time. They are simple to implement, have low computational overhead, and high accuracy, making them suitable for real-time deployment on edge devices.

[0094] In one embodiment, analyzing the lane change behavior distribution according to the behavior type to obtain the continuous lane change recognition result includes:

[0095] The number and distribution of lane changes within a preset time period are counted. If two or more lane changes occur consecutively, the continuous lane change recognition result is obtained using the following method:

[0096] Obtain the timestamps corresponding to two adjacent lane change behaviors and calculate the difference between the two timestamps to obtain the time interval; the two adjacent lane change behaviors come from a sequence of more than two consecutive lane change behaviors.

[0097] If the time interval is less than the continuous lane change time threshold, the continuous lane change identification result is determined to be an illegal continuous lane change.

[0098] Schematically, after completing frame-level behavior classification, the system further conducts statistics and analysis on the temporal distribution of lane change behaviors of each vehicle to form a continuous lane change behavior identification strategy. This strategy uses the number of lane change behaviors and the time interval between behaviors as dual constraints, and screens out illegal continuous lane change behaviors with obvious short-term repetitive characteristics from the temporal dimension.

[0099] Specifically, the system maintains a lane change behavior list (diftime[plate]) corresponding to each license plate number. This list records the timestamp sequence of all identified lane changes. After each frame of behavior classification is completed, the system updates this list in real time. During a statistical period (i.e., the vehicle's residence time in the monitoring area), the system determines whether the number of lane changes exceeds two. If not, no further violation analysis is triggered. If more than two consecutive lane changes are recorded, the system enters the continuous lane change determination process.

[0100] In the judgment process, the system extracts the timestamps t of the two most recent lane changes from diftime[plate] i-1 , t i Calculate the time difference between the two:

[0101] Δt=t i -t i-1

[0102] This Δt is a direct reflection of the behavior frequency and represents the time interval between two lane changes of the vehicle. The system compares this interval with the continuous lane change time threshold determined by the road congestion level. If the calculated Δt is less than the continuous lane change time threshold T under the current congestion level, the system will automatically reset the vehicle. 拥堵 , the system determines that the vehicle has short-term and high-frequency continuous lane changing behavior, and classifies the behavior as illegal continuous lane changing, generating a corresponding violation identification result.

[0103] Furthermore, two adjacent lane changes refer to two adjacent lane changes occurring in a continuous lane change sequence, that is, continuous lane changes that are not interrupted by a straight-line behavior that meets the straight-line clearance condition. This is to exclude two non-continuous lane changes in normal driving intervals, ensure that the system only issues penalty instructions for truly short-term violations, and reduce false alarms.

[0104] In one embodiment, the number and distribution of lane changes within a preset time period are counted according to the following settings:

[0105] When a vehicle completes a lane change, the number of lane changes is increased by one, and the number of straight-ahead behaviors is reset to zero.

[0106] When a vehicle completes a straight-line behavior, the number of straight-line behaviors is increased by one.

[0107] If the duration of the vehicle's consecutive straight-moving behaviors exceeds the preset time threshold, the number of currently recorded lane changes and straight-moving behaviors will be reset to zero.

[0108] Schematically, a time-threshold-based behavior clearing mechanism is used to dynamically manage the lane-changing and straight-driving behavior counts for each vehicle, ensuring that the behavior determination process always occurs within a logically clear analysis window and preventing historical behavior from interfering with current recognition. The system maintains two independent behavior counters for each license plate number: difdic[plate] represents the cumulative number of lane changes that occurred during the current time period for the vehicle; and samedic[plate] represents the number of continuous, uninterrupted straight-driving behaviors. Whenever the system determines that a lane change occurs in a frame based on coordinate changes, the difdic[plate] for the vehicle is incremented by one, while samedic[plate] is cleared to zero. Whenever a straight-driving behavior is determined, samedic[plate] is incremented by one, but the current difdic[plate] remains unchanged. At the same time, the system maintains a behavior time window accumulator for each vehicle, which is used to count the cumulative length of time for continuous straight-driving behaviors. Each time a frame of straight-driving behavior is identified, the cumulative straight-driving time ΔT is calculated between the current timestamp and the timestamp of the last lane-changing behavior. 直行累积 , once the continuous straight-line behavior lasts for ΔT 直行累积 When the set clearing time threshold is exceeded, the system simultaneously resets the two behavior counters for the current vehicle, indicating that the historical behavior sequence has been reset and will no longer be counted in the next round of violation analysis. This mechanism ensures that the system triggers a violation only when the lane change is continuous and not interrupted by straight driving.

[0109] Through this state clearing strategy, the system can maintain stable recognition boundaries in scenarios such as multi-vehicle interference, slow traffic flow, and partial vehicle occlusion, preventing residual lane changes from earlier triggering subsequent judgments, greatly improving the accuracy and engineering robustness of continuous behavior recognition.

[0110] In one embodiment, the dynamic traffic congestion degree is obtained by the congestion determination unit through the following method:

[0111] S41. Obtain the travel time differences of several newly identified vehicles and their corresponding first identification times.

[0112] The system receives real-time license plate recognition information from the front and rear cameras, including the time of the first and second vehicle recognition. The system calculates the difference between the two to determine the vehicle's travel time within the monitored road section. It also records the vehicle's front camera shooting timestamp, which serves as the basis for subsequent time-based sorting and window division. Newly recognized vehicles are those whose license plate information is stored in the local license plate cache.

[0113] S42. Add the travel time differences of the newly identified vehicles to a fixed-length circular stack in the order of their first identification time, so as to construct the current congestion judgment window according to preset conditions; the congestion judgment window is composed of a time difference set consisting of multiple travel time differences.

[0114] The current congestion judgment window is generated when any of the following preset conditions is met:

[0115] The amount of data in the current loop stack has reached a fixed length.

[0116] The time interval between the first recognition time of the newly recognized vehicle and the first recognition time of the most recently added vehicle in the loop stack exceeds a preset time interval threshold.

[0117] Schematically, the system uses a fixed-length circular stack structure to cache the travel time differences of newly identified vehicles, and pushes them into the stack in the order of the first recognition time. This structure can both limit the stack capacity and maintain the time sequence, and is used to construct the current congestion judgment window. Each window consists of a time difference set in the stack, called a time difference set window. For example, a circular stack with a fixed length of 50 vehicles is used. When the number of vehicles in the circular stack reaches 50, the 51st vehicle will enter the circular stack by pushing the top of the first vehicle in the stack out of the stack. By setting a fixed length in the circular stack, the controllability of the number of vehicles is guaranteed.

[0118] The release of the stack is the generation of the congestion judgment window, which is triggered when any of the following conditions are met:

[0119] The amount of data stored in the current loop stack has reached a fixed upper limit;

[0120] The time interval between the first recognition time of the newly recognized vehicle and the first recognition time of the vehicle most recently added to the loop stack exceeds a set threshold.

[0121] When any of the above conditions is met, the system determines that the current window data has met the calculation requirements and enters the subsequent judgment process. In particular, considering the situation of sparse traffic, the first recognition time of the new vehicle is compared with the first recognition time of the most recently entered vehicle in the loop stack. If the time interval between the first recognition times of the two vehicles is less than the set threshold of 5 minutes, the two vehicles enter the same loop stack, that is, the same congestion judgment window. If the time interval between the first recognition times of the two vehicles is greater than the set threshold of 5 minutes, the loop stack divides the vehicles in the stack into a congestion judgment window and opens a new congestion judgment window to receive the new vehicle into the loop stack.

[0122] S43: Calculate the average travel time of the time difference set of the congestion judgment window.

[0123] The traffic congestion degree of the section should be evaluated based on the average travel speed, using the following calculation formula:

[0124]

[0125] in, is the average travel speed of the interval section; L is the length of the interval section, excluding intersections; t i is the time it takes for vehicle i to pass through the interval section; when measuring the number of vehicles, the sample size should be determined based on the number of lanes in the interval section, and generally should not be less than 10 times the total number of lanes in the interval section.

[0126] For illustration, it is easier to obtain data using time measurement. When the length of a road section is fixed, it can be directly judged by time, that is, the average travel time is defined as:

[0127]

[0128] in, is the average travel time of the section.

[0129] The average calculation is performed on the current time difference set to obtain the average travel time of the vehicle in the time period.

[0130] S44. Calculate the average travel speed based on the fixed distance of the monitored road section and the average travel time.

[0131] According to the time-distance formula, the average speed can be expressed as:

[0132]

[0133] Furthermore, the travel time difference is introduced to take into account the time change, and we get:

[0134]

[0135] Where Δt is the travel time difference; t start is the first recognition time; t end This is the second identification time.

[0136] S45. Map the average traffic speed to a traffic congestion level model to obtain a dynamic traffic congestion degree.

[0137] Table 2 Correspondence between average traffic speed and traffic congestion

[0138]

[0139] Indicatively, the traffic congestion level model can be specifically adjusted according to different road traffic conditions.

[0140] S46: Clear the current loop stack and open a new congestion determination window to receive the travel time difference data of newly identified vehicles.

[0141] The system clears the current loop stack, marking the completion of this round of window calculation, and starts the construction process of the next round of new windows to receive the time difference data of newly identified vehicles, and clears the local license plate cache to allow vehicles to pass through the same road section multiple times.

[0142] This system enables high-frequency dynamic assessment of local traffic conditions with minimal data structure and computational overhead, without requiring comprehensive historical traffic records. Compared to traditional static threshold setting methods, this solution not only improves the behavior recognition model's adaptability to actual road conditions but also offers excellent scalability and deployment flexibility, making it suitable for different road types and camera distances.

[0143] In one embodiment, the travel time difference of a newly identified vehicle and its corresponding first identification time are obtained by the following method:

[0144] In response to the instruction of detecting a new license plate, the front camera unit publishes the first tuple corresponding to the new license plate to the first preset topic through a lightweight message transmission protocol, and generates a green light on instruction; the first tuple includes the license plate number and the first recognition time; the green light on instruction is used to instruct the green light of the front integrated machine to light up; the instruction of detecting a new license plate is generated by retrieving the license plate number from the corresponding license plate number cache library.

[0145] Schematically, license plates are recognized in successive image frames. When the system identifies a new license plate number that has not been recorded in the current cycle, it searches the local license plate cache for that number. If a match is not found, it indicates a newly recognized vehicle, and the system generates a new license plate command, which serves as a state trigger event.

[0146] In response to the new license plate instruction, the front camera unit extracts the license plate number and its corresponding recognition time to form a first tuple. Furthermore, the system publishes the first tuple to the first preset topic through the lightweight Internet of Things message protocol, and at the same time generates a green light instruction to control the display status of the indicator module on the all-in-one machine to prompt the operator that the vehicle has been successfully recorded by the system. Furthermore, the license plate information corresponding to the local license plate cache is cleared to allow the vehicle to pass through the road section multiple times.

[0147] The rear camera unit responds to the detected new license plate instruction and publishes the second tuple corresponding to the new license plate to the second preset topic through the lightweight message transmission protocol; the second tuple includes the license plate number and the second recognition time.

[0148] Similarly, the rear camera unit is deployed at the end of the monitoring interval. Its task is to detect the license plates of all vehicles leaving the area and generate a second tuple in response to the new license plate instruction. The second tuple is also published to the second preset topic through the lightweight IoT message protocol for subsequent time difference calculation.

[0149] The cloud computing unit obtains the first tuple and the second tuple by subscribing to the first preset topic and the second preset topic, and calculates the difference based on the first recognition time and the second recognition time to obtain the travel time difference of the newly recognized vehicle and its corresponding first recognition time.

[0150] Schematically, a cloud computing unit deployed on a central server or edge gateway node subscribes to the two aforementioned topics. Once a license plate number is identified as appearing in both topics, the system matches the two tuples and calculates the time difference. The time of the first recognition is used as the vehicle's front-end passing time. The system records this time and the passing time difference as the vehicle's dynamic indicators in the current loop stack, providing input data for subsequent road traffic status calculations.

[0151] Optionally, the lightweight message transmission protocol is the MQTT (Message Queuing Telemetry Transport) protocol, which includes a server, a publisher and a subscriber, wherein the publisher is the front camera unit and the rear camera unit, and the subscriber is the cloud computing unit. Furthermore, the Alibaba Cloud platform is used as the cloud server.

[0152] In one embodiment, the visual recognition layer includes a license plate location unit and a license plate number recognition unit;

[0153] The license plate positioning unit is used to analyze the vehicle image data through the license plate positioning model to determine the license plate position.

[0154] Schematically, the license plate location unit detects the vehicle license plate area from continuous image frames. The unit is built using a deep learning model, preferably a target detection network based on the YOLOv8 series. The YOLOv8 model has good small target detection capabilities. The model takes vehicle image data as input and outputs the coordinates of the license plate bounding box in each frame of the image [x min ,y min ,x max ,y max ] to form regional candidate results.

[0155] The license plate number recognition unit is used to identify the area corresponding to the license plate position in the vehicle image data through the license plate number recognition model to obtain the license plate number.

[0156] Schematically, the image region output by the license plate location unit is used as input, and an end-to-end character recognition model is used to parse the license plate number. The preferred model is the lightweight convolutional neural network LPRNet (License Plate Recognition Network), which is designed based on the CTC (Connectionist Temporal Classification) loss function and is suitable for recognition tasks with a variable number of license plate characters and unclear character segmentation locations.

[0157] Through the coordinated operation of the license plate positioning unit and the license plate number recognition unit, the system achieves stable extraction of vehicle identity in continuous image frames, generating triple information of license plate number, license plate position and timestamp, providing basic data guarantee for subsequent motion trajectory analysis and behavior pattern recognition based on license plate number.

[0158] In this embodiment, the visual recognition layer adopts a modular design architecture, has good model replacement capabilities and computing resource adaptability, is suitable for on-demand deployment in all-in-one machines of different performance levels, and supports a remote model update mechanism, providing good technical support for subsequent system capability expansion and algorithm upgrades.

[0159] In one embodiment, the license plate location model is obtained by:

[0160] A data set is obtained and data enhancement processing is performed on the data set to obtain a first training set; the data set includes real-scene license plate images and their corresponding annotated license plate frames.

[0161] The first training set is used to adjust the parameters of the pre-trained YOLOv8 model, and the parameters are optimized by configuring the loss function weights to obtain the license plate positioning model.

[0162] As an example, we constructed a license plate image dataset encompassing diverse scenarios. This dataset is derived from real-world images collected on urban roads, covering typical traffic conditions such as sunny days, nighttime, backlighting, and rain and fog. It includes examples of motor vehicle license plates from different angles, colors, and types. Each image is accompanied by a license plate bounding box annotation file in the standard VOC / COCO format, meaning each example contains the coordinates of a four-point rectangular bounding box and its "plate" label category.

[0163] To enhance the robustness of the model to various complex conditions, the system performs data augmentation on the original data set, including but not limited to random brightness adjustment, affine transformation, blur enhancement, occlusion simulation, traffic density simulation and other methods. The samples after data augmentation are merged with the original data set to form the first training set for training.

[0164] In terms of model selection, YOLOv8x is preferred as the backbone network for target detection. This model has strong multi-scale detection capabilities and high frame rate performance, making it suitable for detecting small targets such as license plates in complex backgrounds. Based on the YOLOv8 weights pre-trained on ImageNet (an image database), the system uses the aforementioned first training set for fine-tuning. During training, the model is optimized by configuring Focal Loss weights to strengthen the focus on difficult samples, setting the loss function ratio of category confidence to regression error, and customizing Anchor Box settings to adapt to the stability of the license plate aspect ratio.

[0165] Furthermore, the loss functions in the YOLOv8 algorithm include:

[0166] Bounding box regression loss (Box Loss): Calculated using CIoU (Complete IoU) loss. The specific calculation formula is as follows:

[0167]

[0168] Among them, b and b gt are the coordinates of the predicted bounding box and the real bounding box respectively; c is the coordinate of b and b gt The diagonal length of the minimum enclosing box of the bounding box; ρ is the function that calculates the distance between the center points of the two bounding boxes; α is the weight coefficient; v is a measure of the consistency of the aspect ratio; IoU is the intersection over union ratio, which is an indicator of the degree of overlap between two bounding boxes. It is used to evaluate the degree of match between the predicted bounding box and the true bounding box; A and B are two bounding boxes, S A∩B is the area where the two borders overlap; S A∪B The area of these two bounding boxes combined.

[0169] From the above formula, we can see that if the predicted bounding box is closer to the true bounding box, the larger the IoU is. The smaller the L box The smaller it is, the more accurate the predicted bounding box position and size are.

[0170] Classification loss: Cross Entropy Loss is used for calculation, and its calculation formula is as follows:

[0171]

[0172] Among them, L class is the cross entropy loss; pi is the value of the i-th element of the true label. In multi-classification problems, it is a one-hot encoded vector, that is, the index position corresponding to the true category is 1, and the rest are 0; q i is the value of the i-th element predicted by the model, that is, the probability that the model predicts that the category is true.

[0173] In practice, if the model predicts a lower probability of the true label, -log(q i ) will be larger, L class The larger it is, the more accurate the classification loss is.

[0174] Objectness Loss: Objectness loss is calculated using Binary Cross Entropy Loss. The specific calculation formula is as follows:

[0175] L obj =-[y obj log(p obj )+(1-y obj )log(1-p obj )]

[0176] Among them, L obj is the existence label of the target (0 or 1); p obj is the target probability of the prediction. When the probability of the predicted target is greater, log(p obj ) is larger, L obj The smaller the target loss is, the more likely it is that all targets can be identified.

[0177] By weighting and summing all the above losses, we can get the total loss of the loss function. The specific formula is as follows:

[0178] L total =ω box L box +ω class L class +ω obj L obj +…

[0179] Among them, ω is the weight of different losses.

[0180] The final trained model achieved an average positioning accuracy of over 96% on the test set and has the ability to run in real time in a frame streaming manner on edge devices.

[0181] The license plate recognition model is obtained in the following way:

[0182] The license plate image and its corresponding license plate character annotation are obtained to obtain a second training set; the license plate image is obtained by segmenting the license plate position generated by the license plate positioning model.

[0183] The second training set is used to adjust the parameters of the pre-trained LPRNet model, and the parameters are optimized through the CTC loss function to obtain the license plate recognition model.

[0184] Schematically, the image data used in the recognition model training stage comes from the license plate area image output by the first-stage positioning model. Specifically, the images in the first training set are automatically extracted from the license plate area through the trained YOLOv8 model, cropped and normalized, and the license plate character recognition training set, i.e., the second training set, is constructed, in which each image sample is equipped with its corresponding character-level annotation.

[0185] The model used is LPRNet (License Plate Recognition Network), which is a lightweight convolutional network without character alignment and is suitable for recognizing license plate character sequences on the end side. The system is based on the official open source LPRNet model and is fine-tuned using the second training set. The CTC (Connectionist Temporal Classification) loss function is introduced during the training process to support end-to-end recognition of variable character lengths. The model uses gradient clipping, learning rate decay, and DropBlock regularization techniques during the convergence process to prevent overfitting and improve generalization capabilities. After the final training, the recognition model can achieve a recognition accuracy of more than 98.3% in the test set with a license plate completeness rate greater than 85% and an occlusion rate less than 30%.

[0186] Through the above two-stage model training process, the system obtains highly customized and engineering deployment-friendly visual recognition capabilities, and demonstrates excellent stability and recognition efficiency in actual road deployment, constituting an important component of the visual recognition layer in the technical solution of the present invention.

[0187] In one embodiment, the system further includes a violation reporting layer, which includes a violation instruction layer, a reporting layer, and a push layer;

[0188] The violation instruction layer is used to generate a photo-taking instruction and a red light-on instruction when the continuous lane change recognition result is an illegal continuous lane change; the photo-taking instruction is used to instruct the camera of the front integrated device to take a photo of the illegal continuous lane change behavior and provide feedback in the form of illegal image evidence according to the set image format; the red light-on instruction is used to instruct the red light of the front integrated device to turn on.

[0189] Schematically, the photographing instruction is used to trigger the camera module on the front integrated machine, such as ESP32-CAM, to perform image capture. The system calls the vehicle behavior occurrence frame or the image of the adjacent key frame, extracts the image frame at the moment of the violation in the set format such as JPG / WEBP, and caches it as the violation image evidence and returns it. The image should include clear lane lines, license plates, and lane-changing paths for subsequent law enforcement confirmation.

[0190] The red light亮起 instruction synchronously controls the status indication module on the front integrated machine to turn on the red light, indicating that the vehicle has been determined to have a violation, and providing visual feedback for system status debugging and offline intervention. Through such responses, the system links the data processing logic with the physical indication mechanism, effectively improving the transparency of front-end recognition and the on-site prompting ability.

[0191] The reporting layer is used to receive the violation image evidence and push the violation evidence to the push layer through the lightweight message transmission protocol; the violation evidence includes the violation image evidence, its corresponding license plate number, and the violation timestamp.

[0192] Specifically, after the photographing operation is completed, the violation image evidence will be encoded and packaged, together with the corresponding license plate number and the violation occurrence timestamp, to form a structured violation evidence information packet. Further, the system uniformly uses the lightweight message transmission protocol to push this evidence to the backend push layer through a dedicated topic. Exemplarily, the evidence information structure is {plate: "沪B12345", timestamp: "2025-04-20 14:21:45", image: "base64_encoded_image_data", type: "continuous lane-changing violation"}. This structure not only facilitates unified parsing and storage but also supports the mixed coding processing of image compression formats and character formats to adapt to network bandwidth difference scenarios. Optionally, the image is base64-encoded after compression.

[0193] The push layer is used to store the violation evidence in the log and push it to the traffic police client after parsing the violation evidence through the Internet of Things protocol.

[0194] The log records all violation evidence information in the log system in real time, supporting operators to query, export, and track.

[0195] The information push pushes the parsed violation evidence to the designated traffic police law enforcement client through Internet of Things access protocols such as HTTP API, CoAP, or platform SDK, ensuring that the law enforcement side can receive the complete data including evidence, behavior type, and illegal time in a timely manner.

[0196] This mechanism can support functions such as automatic alarm, generating penalty suggestions, and linking with the traffic management platform, realizing the intelligent traffic management closed-loop of AI automatic recognition and real-time feedback law enforcement.

[0197] Through the violation reporting layer in this embodiment, the system not only completes the closed-loop response logic from image recognition, behavior judgment, behavior confirmation and law enforcement push, but also provides quantifiable, traceable and auditable violation data support for the refined management of urban traffic.

[0198] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0199] Based on the same inventive concept, the embodiments of the present application also provide a method for implementing the aforementioned computer vision and Internet of Things-based license plate continuous lane change monitoring and management system. The solution to the problem provided by this method is similar to the solution described in the aforementioned system. Therefore, the specific limitations in one or more embodiments of the method for implementing the computer vision and Internet of Things-based license plate continuous lane change monitoring and management system provided below can be found in the above-mentioned limitations on the computer vision and Internet of Things-based license plate continuous lane change monitoring and management system, and will not be repeated here.

[0200] In an exemplary embodiment, Figure 4 As shown, a license plate continuous lane change monitoring and management method based on computer vision and the Internet of Things is provided, including:

[0201] S401 , capturing vehicle images frame by frame and performing preprocessing to obtain vehicle image data; the vehicle image data includes a vehicle image frame sequence and a timestamp corresponding to each vehicle image frame.

[0202] S402. Use a computer vision model to locate and recognize the license plate of the vehicle image data to obtain a license plate recognition information group; the license plate recognition information group includes the license plate number in the vehicle image frame, the license plate position corresponding to the license plate number, and a timestamp.

[0203] S403: Based on the dynamic traffic congestion degree, continuous lane change behavior is determined for the license plate position change and the corresponding time stamp of the same license plate number in the vehicle image data to obtain a continuous lane change recognition result; the continuous lane change recognition result includes an illegal continuous lane change.

[0204] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0205] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0206] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0207] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. A license plate continuous lane change monitoring and management system based on computer vision and the Internet of Things, characterized by: The system comprises: The vehicle information acquisition layer is used to collect vehicle images frame by frame and perform preprocessing to obtain vehicle image data; the vehicle image data includes a sequence of vehicle image frames and a timestamp corresponding to each vehicle image frame; a visual recognition layer for performing license plate location and license plate number recognition on the vehicle image data using a computer vision model to obtain a license plate recognition information group; the license plate recognition information group includes the license plate number in the vehicle image frame, the license plate position corresponding to the license plate number, and the timestamp; The continuous lane change recognition layer is used to determine the continuous lane change behavior of the license plate position changes of the same license plate number in the vehicle image data and the corresponding timestamp based on the dynamic traffic congestion degree, and obtain a continuous lane change recognition result; the continuous lane change recognition result includes an illegal continuous lane change.

2. The system according to claim 1, wherein: The determining of the continuous lane change behavior based on the license plate position change of the same license plate number in the vehicle image data and the corresponding timestamp based on the dynamic traffic congestion degree to obtain the continuous lane change recognition result includes: Obtaining a continuous lane change time threshold based on dynamic traffic congestion; performing a difference calculation on the horizontal coordinates of the license plate positions in two adjacent vehicle image frames in the vehicle image frame sequence for the same license plate number to obtain a license plate coordinate change value; According to the preset judgment rules, the driving behavior type of the vehicle is judged based on the license plate coordinate change value; the driving behavior type includes lane change behavior and straight driving behavior; The continuous lane change recognition result is obtained by analyzing the lane change behavior distribution according to the behavior type.

3. The system according to claim 2, characterized in that The preset judgment rules correspond to the following steps: If the license plate coordinate change value exceeds a preset first displacement threshold, it is determined that the vehicle has completed the lane change behavior once, and a timestamp corresponding to the lane change behavior is recorded; If the license plate coordinate change value is less than a preset second displacement threshold, it is determined that the vehicle has completed the straight-moving behavior once, and a timestamp corresponding to the straight-moving behavior is recorded.

4. The system according to claim 3, characterized in that Analyzing the lane change behavior distribution according to the behavior type to obtain the continuous lane change recognition result includes: The number and distribution of lane changes within a preset time period are counted. If the lane change occurs two or more times in a row, the continuous lane change recognition result is obtained according to the following method: Obtaining the timestamps corresponding to two adjacent lane change behaviors and calculating the difference between the two timestamps to obtain a time interval; the two adjacent lane change behaviors are derived from a sequence of two or more consecutive lane change behaviors; If the time interval is less than the continuous lane change time threshold, the continuous lane change identification result is determined to be the illegal continuous lane change.

5. The system according to claim 3, wherein: Use the following settings to count the number and distribution of lane changes within a preset time period: When the vehicle completes the lane change behavior once, the number of lane changes is increased by one, and the number of straight-ahead behaviors is reset to zero; When the vehicle completes the straight-ahead behavior once, the number of straight-ahead behaviors is increased by one; If the duration of the straight-ahead behavior completed continuously by the vehicle cumulatively exceeds a preset time threshold, the number of the lane change behavior and the straight-ahead behavior currently recorded will be reset to zero.

6. The system according to claim 1, wherein: The dynamic traffic congestion degree is obtained by the congestion judgment unit in the following manner: Obtain the travel time differences of several newly identified vehicles and their corresponding first identification times; The travel time differences of the newly identified vehicles are added to a fixed-length circular stack in the order of their first identification time, and used to construct a current congestion judgment window according to preset conditions; the congestion judgment window is composed of a time difference set consisting of multiple travel time differences; Calculate the average travel time of the time difference set of the congestion judgment window; Calculate the average travel speed based on the fixed distance of the monitored road section and the average travel time; Mapping the average traffic speed to a traffic congestion level model to obtain the dynamic traffic congestion degree; Clearing the current loop stack and opening a new congestion determination window for receiving subsequent travel time difference data of newly identified vehicles; The current congestion judgment window is generated if any of the following preset conditions is met: The amount of data in the current loop stack reaches the fixed length; The time interval between the first recognition time of the newly recognized vehicle and the first recognition time of the vehicle most recently added to the circulation stack exceeds a preset time interval threshold.

7. The system according to claim 6, characterized in that The travel time difference of the newly identified vehicle and its corresponding first recognition time are obtained by the following method: In response to a new license plate detection instruction, the front camera unit publishes a first tuple corresponding to the new license plate to a first preset topic via a lightweight message transmission protocol, and generates a green light on instruction; the first tuple includes the license plate number and the time of first recognition; the green light on instruction is used to instruct the green light of the front integrated device to turn on; the new license plate detection instruction is generated by retrieving the license plate number from a corresponding license plate number cache library; The rear camera unit responds to the detected new license plate instruction and publishes a second tuple corresponding to the new license plate to a second preset topic through a lightweight message transmission protocol; the second tuple includes the license plate number and the second recognition time; The cloud computing unit obtains the first tuple and the second tuple by subscribing to the first preset topic and the second preset topic, and calculates the difference based on the first recognition time and the second recognition time to obtain the travel time difference of the newly recognized vehicle and its corresponding first recognition time.

8. The system according to claim 1, wherein: The visual recognition layer includes a license plate positioning unit and a license plate number recognition unit; The license plate positioning unit is used to analyze the vehicle image data through a license plate positioning model to determine the license plate position; The license plate number recognition unit is used to recognize the area of the vehicle image data corresponding to the license plate position through a license plate number recognition model to obtain the license plate number.

9. The system according to claim 8, characterized in that: The license plate positioning model is obtained by: Acquire a data set and perform data enhancement processing on the data set to obtain a first training set; the data set includes real-scene license plate images and their corresponding annotated license plate frames; Using the first training set to adjust the parameters of the pre-trained YOLOv8 model, and optimizing the parameters by configuring the loss function weights, to obtain the license plate positioning model; The license plate recognition model is obtained by: Acquire a license plate image and its corresponding license plate character annotation to obtain a second training set; the license plate image is segmented based on the license plate position generated by the license plate positioning model; The second training set is used to adjust the parameters of the pre-trained LPRNet model, and the parameters are optimized through the CTC loss function to obtain the license plate recognition model.

10. The system according to any one of claims 1 to 9, characterized in that: The system further comprises a violation reporting layer, which comprises a violation instruction layer, a reporting layer and a push layer; The violation instruction layer is used to generate a photo-taking instruction and a red light-on instruction when the continuous lane change recognition result is the illegal continuous lane change; the photo-taking instruction is used to instruct the camera of the front integrated device to take a photo of the illegal continuous lane change behavior and provide feedback in the form of illegal image evidence in a set image format; the red light-on instruction is used to instruct the red light of the front integrated device to turn on; The reporting layer is used to receive the violation image evidence and push the violation evidence to the push layer through a lightweight message transmission protocol; the violation evidence includes the violation image evidence and its corresponding license plate number and violation timestamp; The push layer is used to store the violation evidence in a log, and after parsing the violation evidence, push it to the traffic police client through the Internet of Things protocol.

Images