License plate recognition system and method and storage medium

By calculating the time synchronization error between devices through a heartbeat message mechanism, the time synchronization problem in the license plate recognition system is solved, the accuracy of license plate recognition is improved, and it is applicable to scenarios such as collision monitoring of height restriction protection frames.

CN119007177APending Publication Date: 2024-11-22GUANGXI TIANDAO INFORMATION TECH +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410998082.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Existing license plate recognition methods cannot solve the time synchronization problem between vibration sensor detection of collision events and license plate recognition in height restriction protection frame collision monitoring scenarios, resulting in delayed license plate recognition results. Furthermore, existing methods are not applicable or too costly in low-power scenarios.

Method used

A heartbeat message mechanism is used to calculate the time synchronization error between devices. By identifying the message interaction between the recognition processor and the collision processor, the time synchronization error between devices is determined, and the collision video is obtained from the camera for license plate recognition.

Benefits of technology

It effectively solves the problem of asynchronous external triggering events and vehicle capture and recognition, improves the accuracy of license plate recognition, and is suitable for scenarios with harsh construction conditions and limited cost control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007177A_ABST
    Figure CN119007177A_ABST
Patent Text Reader

Abstract

The invention provides a license plate recognition system and method and a storage medium, and belongs to the field of license plate recognition, the license plate recognition system comprises a recognition processor and a collision processor, the recognition processor is used for generating heartbeat messages according to a preset period, sending the heartbeat messages to the collision processor and obtaining heartbeat message sending time, the heartbeat message sending moment is the moment when the identification processor sends the heartbeat message; and the collision processor is used for returning a heartbeat response message to the identification processor according to the heartbeat message when receiving the heartbeat message sent by the identification processor. According to the method, the problem that external trigger events and vehicle snapshot recognition are not synchronous is effectively solved, the accuracy of license plate recognition is improved, and the method has a good application effect in some scenes where construction conditions are harsh, cost control is limited and other resources are limited.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention mainly relates to the technical field of license plate recognition, and in particular to a license plate recognition system, method and storage medium. Background Art

[0002] In the scenario of height-limited guardrail collision monitoring, the existing license plate recognition method is to deploy vibration sensors and license plate recognition cameras on site, where the license plate recognition camera is used to continuously recognize license plates and record videos, and the vibration sensor detects collision events of height-limited guardrails or bridge facilities beams. When a collision event is detected, the license plate recognition camera is notified to retrieve the license plate recognition result at the current moment. However, this method cannot solve the time synchronization problem of vibration sensor detection of collision events and license plate recognition. The license plate recognition result retrieved lags behind the time of the collision event, and the vehicle involved cannot be accurately captured in scenes with high vehicle speeds. At the same time, the license plate recognition process of this method is uninterrupted, requiring the camera to have high computing power and high power consumption requirements for the power supply system. It is not suitable for most low-power height-limited guardrail collision monitoring scenarios, and it cannot actually solve the problem of vibration sensor detection of collision events. Another license plate recognition method uses a laser ranging sensor to trigger license plate recognition. If this method uses low-cost ultra-high vehicle laser detection equipment, it will result in a high false alarm rate. If ultra-high vehicle laser detection equipment with a low false alarm rate is used, the cost will be increased, and it is not suitable for deployment at multiple monitoring points. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a license plate recognition system, method and storage medium in view of the deficiencies in the prior art.

[0004] The technical solution of the present invention to solve the above technical problem is as follows: a license plate recognition system includes a recognition processor and a collision processor,

[0005] The identification processor is used to generate a heartbeat message according to a preset period, send the heartbeat message to the collision processor, and obtain a heartbeat message sending time, where the heartbeat message sending time is the time when the identification processor sends the heartbeat message;

[0006] The collision processor is used for returning a heartbeat response message to the identification processor according to the heartbeat message when receiving the heartbeat message sent by the identification processor;

[0007] The identification processor is used to calculate the inter-device time synchronization error between the collision processor and the collision processor according to the time when the heartbeat message is sent and the time corresponding to the reception of the heartbeat response message;

[0008] The collision processor is also used to obtain an original collision signal from the collision detection device and obtain an original collision time, wherein the original collision time is the time when the collision processor obtains the original collision signal from the collision detection device;

[0009] Detecting and analyzing the original collision signal and the original collision time to obtain a collision detection time group, and sending the collision detection time group to the recognition processor;

[0010] The recognition processor is further used to receive the collision detection time group and obtain a time group receiving moment, wherein the time group receiving moment is a moment when the recognition processor receives the collision detection time group;

[0011] Performing a time delay analysis on the time group receiving moment, the time synchronization error and the collision detection time group to obtain a collision occurrence time period, and obtaining a collision occurrence video from a camera according to the collision occurrence time period;

[0012] The collision video is identified and analyzed to obtain vehicle license plate information.

[0013] Another technical solution of the present invention to solve the above technical problem is as follows: a license plate recognition method, comprising the following steps:

[0014] Generate a heartbeat message according to a preset period, send the heartbeat message to the collision processor, and obtain the heartbeat message sending time, where the heartbeat message sending time is the time when the identification processor sends the heartbeat message;

[0015] When receiving a heartbeat message sent by the identification processor, returning a heartbeat response message to the identification processor according to the heartbeat message;

[0016] Calculate the inter-device time synchronization error between the collision processor and the collision processor according to the heartbeat message sending time and the time corresponding to the receiving of the heartbeat response message;

[0017] Obtaining an original collision signal from a collision detection device, and obtaining an original collision time, wherein the original collision time is the time when the collision processor obtains the original collision signal from the collision detection device;

[0018] Performing detection and analysis on the original collision signal and the original collision moment to obtain a collision detection time group;

[0019] Obtaining a time group receiving time, wherein the time group receiving time is a time when the recognition processor receives the collision detection time group;

[0020] Performing a time delay analysis on the time group receiving moment, the time synchronization error and the collision detection time group to obtain a collision occurrence time period, and obtaining a collision occurrence video from a camera according to the collision occurrence time period;

[0021] The collision video is identified and analyzed to obtain vehicle license plate information.

[0022] The beneficial effects of the present invention are as follows: the time synchronization error between devices and the collision processor is calculated through the heartbeat message sending time and the time corresponding to the receiving heartbeat response message, the collision detection time group is obtained by detecting and analyzing the original collision signal and the original collision time, the collision occurrence time period is obtained by analyzing the time group receiving time, the time synchronization error and the delay of the collision detection time group, and the collision occurrence video is obtained from the camera according to the collision occurrence time period, and the vehicle license plate information is obtained by identifying and analyzing the collision occurrence video, which effectively solves the problem of asynchronism between external triggering events and vehicle capture and recognition, improves the accuracy of license plate recognition, and has good application effects in some resource-constrained scenarios such as harsh construction conditions and cost control. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A module block diagram of a license plate recognition system provided by an embodiment of the present invention;

[0024] Figure 2 A flowchart of a license plate recognition method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0026] Figure 1 A module block diagram of a license plate recognition system provided by an embodiment of the present invention.

[0027] like Figure 1 As shown, a license plate recognition system includes a recognition processor and a collision processor.

[0028] The identification processor is used to generate a heartbeat message according to a preset period, send the heartbeat message to the collision processor, and obtain a heartbeat message sending time, where the heartbeat message sending time is the time when the identification processor sends the heartbeat message;

[0029] The collision processor is used for returning a heartbeat response message to the identification processor according to the heartbeat message when receiving the heartbeat message sent by the identification processor;

[0030] The identification processor is used to calculate the inter-device time synchronization error between the collision processor and the collision processor according to the time when the heartbeat message is sent and the time corresponding to the reception of the heartbeat response message;

[0031] The collision processor is also used to obtain an original collision signal from the collision detection device and obtain an original collision time, wherein the original collision time is the time when the collision processor obtains the original collision signal from the collision detection device;

[0032] Detecting and analyzing the original collision signal and the original collision time to obtain a collision detection time group, and sending the collision detection time group to the recognition processor;

[0033] The recognition processor is further used to receive the collision detection time group and obtain a time group receiving moment, wherein the time group receiving moment is a moment when the recognition processor receives the collision detection time group;

[0034] Performing a time delay analysis on the time group receiving moment, the time synchronization error and the collision detection time group to obtain a collision occurrence time period, and obtaining a collision occurrence video from a camera according to the collision occurrence time period;

[0035] The collision video is identified and analyzed to obtain vehicle license plate information.

[0036] It should be understood that the collision detection device may be an ultra-high warning laser radar.

[0037] It should be understood that the heartbeat message is sent by the identification processor regularly according to a preset period.

[0038] Specifically, at the time when the external event originally occurs, that is, when the external physical collision input occurs, the collision detection device will a (i.e., the original collision moment) generates a collision sensor signal (i.e., the original collision signal). The time delay from the external physical collision to the generation of the collision sensor signal can be ignored. The collision event detection algorithm in the on-site event detection device (i.e., the collision processor) is triggered to perform collision event detection. At T c After the collision event is detected at the time (i.e. the target collision time), it is sent to the on-site computing device (i.e. the recognition processor).

[0039] Specifically, the on-site computing device (i.e., the identification processor) is in T b At a certain time (i.e., the time group receiving time), a collision event sent by a field event detection device (i.e., a collision processor) is received.

[0040] In the above embodiment, the time synchronization error between the devices and the collision processor is calculated by the time when the heartbeat message is sent and the time corresponding to the reception of the heartbeat response message, the collision detection time group is obtained by the detection and analysis of the original collision signal and the original collision time, the collision occurrence time period is obtained by the delay analysis of the time group receiving time, the time synchronization error and the collision detection time group, and the collision occurrence video is obtained from the camera according to the collision occurrence time period, and the vehicle license plate information is obtained by the recognition and analysis of the collision occurrence video, which effectively solves the problem of asynchrony between external trigger events and vehicle capture and recognition, improves the accuracy of license plate recognition, and has good application effects in some resource-constrained scenarios such as harsh construction conditions and cost control.

[0041] Optionally, as an embodiment of the present invention, the heartbeat response message carries a heartbeat message receiving time and a heartbeat response message sending time, the heartbeat message receiving time is the time when the collision processor receives the heartbeat message sent by the identification processor, and the heartbeat response message sending time is the time when the collision processor sends the heartbeat response message.

[0042] In the identification processor, the process of calculating the inter-device time synchronization error between the device and the collision processor according to the time when the heartbeat message is sent and the time corresponding to the reception of the heartbeat response message includes:

[0043] Receive the heartbeat response message and obtain the heartbeat response message receiving time, where the heartbeat response message receiving time is the time when the identification processor receives the heartbeat response message;

[0044] The synchronization error between devices is calculated by using the first formula for the heartbeat message sending time, the heartbeat message receiving time, the heartbeat response message sending time, and the heartbeat response message receiving time, and the time synchronization error is obtained. The first formula is:

[0045]

[0046] Wherein, Δt0 is the time synchronization error between devices, T0 is the time when the heartbeat message is sent, T1 is the time when the heartbeat message is received, T2 is the time when the heartbeat response message is sent, and T3 is the time when the heartbeat response message is received.

[0047] Specifically, the on-site computing device (i.e., the identification processor) periodically sends a heartbeat message to detect whether the on-site event detection device (i.e., the collision processor) is online, and records the heartbeat message sending time T0; after receiving the heartbeat message at time T1 (i.e., the heartbeat message receiving time), the on-site event detection device (i.e., the collision processor) sends a heartbeat response message with the heartbeat message receiving time T1 (i.e., the heartbeat message receiving time) and the heartbeat response message sending time T2 (i.e., the heartbeat response message sending time) of the device clock, and the on-site computing device (i.e., the identification processor) receives the heartbeat response message of the on-site event detection device (i.e., the collision processor) at time T3 (i.e., the heartbeat response message receiving time), and calculates the time synchronization error Δt0 between devices (i.e., the time synchronization error between devices) using a set of binary linear equations, and the equations are as follows:

[0048]

[0049] Among them, Δt0 is the synchronization error between devices, Δt td is the heartbeat message transmission delay, T0 is the heartbeat message sending time, T1 is the heartbeat message receiving time, T2 is the heartbeat response message sending time, and T3 is the heartbeat response message receiving time.

[0050] In the above embodiment, the inter-device time synchronization error between the collision processor is calculated based on the time when the heartbeat message is sent and the time corresponding to the reception of the heartbeat response message, thereby improving the accuracy of license plate recognition. It has a good application effect in some resource-constrained scenarios such as harsh construction conditions and cost control.

[0051] Optionally, as an embodiment of the present invention, in the collision processor, the process of detecting and analyzing the original collision signal and the original collision time to obtain the collision detection time group includes:

[0052] S21: filtering the original collision signal to obtain a filtered collision signal;

[0053] S22: performing collision detection on the filtered collision signal by using a pre-built detection model to obtain a detection result of whether a collision exists, and obtain a target collision time, where the target collision time is the time when the detection result of whether a collision exists is obtained;

[0054] S23: If the detection result is a successful collision, the target collision moment and the original collision moment are differentially calculated to obtain a collision event occurrence delay, and the collision event occurrence delay and the target collision moment are sent to the recognition processor together. The collision detection time group includes the collision event occurrence delay and the target collision moment.

[0055] It should be understood that the pre-built detection model may be a support vector machine model (SVM), a random forest model or a BP neural network.

[0056] Specifically, if an abnormal signal is detected, a certain number of collision sensor signal time series are received through a sliding window time Δt1, and then the signal is filtered, machine learning and other algorithms are used to generate the abnormal signal. c The collision event is detected at the time (i.e. the target collision time). This calculation process will generate an algorithm delay Δt2, which is caused by complex factors and has delay jitter, but the total delay Δt2 for collision event detection is a (i.e. the time delay of the collision event) can be calculated by T a (i.e., the original collision moment) and T c (i.e., target collision moment) is obtained as follows:

[0057] Δt a =Δt1+Δt2=T c -T a .

[0058] It should be understood that the on-site event detection device (ie, the collision processor) is c At the time (i.e., the target collision time), the detected collision event is sent to the on-site computing device (i.e., the recognition processor) via wired or wireless means, and the message also carries T c time (i.e., target collision time) and Δt a (i.e. the time delay of the collision event).

[0059] In the above embodiment, the original collision signal and the original collision moment are detected and analyzed to obtain a collision detection time group, which effectively solves the problem of asynchronism between the external trigger event and the vehicle capture and recognition, and improves the accuracy of license plate recognition.

[0060] Optionally, as an embodiment of the present invention, in the identification processor, the process of performing delay analysis on the time group receiving moment, the time synchronization error and the collision detection time group to obtain the collision occurrence time period includes:

[0061] The total delay of the collision event is calculated by the second formula on the collision event occurrence delay, the target collision time, the time synchronization error and the time group receiving time to obtain the total delay of the collision event. The second formula is:

[0062] Δt c =Δt a +Δt b ,

[0063] Among them, Δt b =T b -(T c +Δt0),

[0064] Among them, Δt c is the total delay of the collision event, Δt a is the time delay of the collision event, Δt b is the collision event message transmission delay, T b is the time group receiving time, T c is the target collision moment, Δt0 is the time synchronization error;

[0065] The actual collision time is obtained by calculating the time group receiving time and the total delay of the collision event through the third formula, and the third formula is:

[0066] T′ a =T b -Δt c ,

[0067] Among them, T′ a is the actual collision moment, T b is the receiving time of the time group, Δt c is the total delay of the collision event;

[0068] Returning to the collision processor to re-import the original collision signal until a preset number of collisions is reached, thereby obtaining a total delay of multiple collision events;

[0069] The delay jitter of the collision event is calculated by calculating the total delay of all the collision events through the fourth formula to obtain the delay jitter of the collision event. The fourth formula is:

[0070]

[0071] in,

[0072] Among them, T jitter is the collision event delay jitter, Δt ci is the total delay of the ith collision event, μ is the mean of the total delay of the collision event, and N is the preset number of collisions;

[0073] The difference between the actual collision moment and the collision event delay jitter is taken as the starting moment, the sum of the actual collision moment and the collision event delay jitter is taken as the ending moment, and all moments between the starting moment and the ending moment are the collision occurrence time period.

[0074] It should be understood that the actual collision moment refers to the actual collision moment calculated when the preset number of collisions is reached.

[0075] It should be understood that the on-site computing device (ie, the identification processor) is bAt the time (i.e., the time group receives the collision event sent by the on-site event detection device (i.e., the collision processor), T c (i.e. target collision time) and Δt0 (i.e. time synchronization error) to calculate the message transmission delay Δt b (i.e., collision event message transmission delay), as follows:

[0076] Δt b =T b -(T c +Δt0).

[0077] Specifically, the on-site computing device (i.e., the identification processor) uses T b The time (i.e. the time group receiving time) is taken as the reference, and the time is deviated backward by Δt a +Δt b , calculate the actual collision time T′ under the clock system of this device a (i.e. the actual collision moment), as follows:

[0078] T′ a =T b -(Δt a +Δt b ),

[0079] Considering the collision event delay jitter T jitter (i.e., collision event delay jitter), intercept T′ a -T jitter (i.e. starting time) to T′ a +T jitter (i.e. the end time) between the historical videos.

[0080] In the above embodiment, the delay analysis of the time group receiving time, the time synchronization error and the collision detection time group is performed to obtain the collision occurrence time period, which effectively solves the synchronization problem between the external trigger event and the vehicle capture and recognition, and improves the accuracy of license plate recognition.

[0081] Optionally, as an embodiment of the present invention, in the recognition processor, the process of performing recognition analysis on the collision video to obtain the vehicle license plate information includes:

[0082] Extracting key frames from the collision video to obtain multiple collision video images;

[0083] Perform vehicle recognition on all the collision video images using a pre-built R-CNN high-precision deep learning inference model to obtain collision vehicle images;

[0084] The vehicle license plate information is obtained by performing license plate recognition on the collision vehicle image using a pre-built OCR optical character recognition model.

[0085] It should be understood that the on-site computing equipment (i.e., the recognition processor) extracts frames (i.e., key frame extraction) from the historical video clips in the range (i.e., the collision video), and implements intelligent classification through the trained R-CNN high-precision deep learning inference model (i.e., the pre-built R-CNN high-precision deep learning inference model), identifies the video frames of overheight vehicles or vehicles colliding with the height-limited highway (i.e., the collision vehicle image), and ensures the best license plate clarity.

[0086] Specifically, the on-site computing device (ie, the recognition processor) uses the obtained video frames to complete the license plate recognition through a license plate recognition algorithm based on an end-to-end OCR model (ie, a pre-built OCR optical character recognition model).

[0087] It should be understood that the R-CNN high-precision deep learning reasoning model, that is, R-CNN is a high-precision deep learning target detection algorithm, which realizes the detection of targets in images by combining the region recommendation algorithm and the deep convolutional neural network.

[0088] It should be understood that the OCR optical character recognition model, namely OCR (Optical Character Recognition) is a technology that converts text in an image into computer-editable text. With the development of technology, OCR technology has evolved from traditional image processing and machine learning methods to a more advanced stage based on deep learning.

[0089] In the above embodiment, the vehicle license plate information is obtained by identifying and analyzing the collision video, which effectively solves the synchronization problem between the external trigger event and the vehicle capture and recognition, improves the accuracy of license plate recognition, and has good application effects in some resource-constrained scenarios such as harsh construction conditions and cost control.

[0090] Optionally, as another embodiment of the present invention, vehicle capture and license plate recognition are triggered by events. The existing technologies all try to solve the problem that the two actions of triggering events and license plate recognition are synchronized in real time, and there is a sufficiently small delay and delay jitter between the two. To achieve this goal, ground sensing coils or virtual coils are usually used to trigger vehicle capture and license plate recognition. The former requires the deployment of hardware equipment and requires suitable construction conditions on site, and the hardware cost and construction cost are high; the latter is to detect whether the vehicle enters the area defined by the virtual coil through a video intelligent recognition algorithm. The vehicle entry detection algorithm deployed on the smart camera or on-site computing equipment needs to be online all the time, and there are also high requirements for the quality of the collected images. Fill lights and flash lights must be turned on at night, and the working energy consumption is high, requiring suitable power supply conditions on site.

[0091] In the case of railway bridge and culvert height limit guard frame collision monitoring and early warning and similar scenarios, the on-site construction conditions do not allow more equipment to be deployed on the ground or on the roadside. Moreover, many of them are located in remote areas with no network or electricity, and need to rely on solar power supply systems to provide electricity. The solar panels cannot be too large, and power consumption needs to be strictly controlled. The existing technical solutions that rely on ground sensing coils or virtual coils are not fully capable of this type of scenario.

[0092] In the collision monitoring and early warning scenario of the height-limited protective frame of the railway bridge and culvert, a collision detection device has been deployed on site. When the device detects a collision event, it links the camera to perform vehicle capture and license plate recognition. However, from the occurrence of the collision to the triggering of the license plate recognition, it is necessary to go through the processes of collision signal filtering, event detection, signal transmission, etc., and it is impossible to synchronously trigger the vehicle capture and license plate recognition in real time. The method involved in the present invention is to solve the problem of the asynchrony between the triggering event and the vehicle capture and recognition time in such scenarios. After the on-site equipment issues an over-height warning or collision alarm event, the vehicle involved is retrieved from the historical video within a certain time range, and the license plate is identified.

[0093] Optionally, as another embodiment of the present invention, after receiving an external trigger event, the present invention retrieves the vehicle involved in the incident from historical videos within a certain time range and identifies the license plate.

[0094] The external trigger event can be an overheight warning or collision alarm signal issued by the on-site event detection device in the collision monitoring and early warning scenario of the height-limited protective frame of the railway bridge or culvert, or a collision event detected by the collision alarm algorithm software. Generally, it can also be an external trigger signal issued by the on-site event detection device in other scenarios. Usually, this external trigger event is a non-real-time event that lags behind the moment of vehicle collision and cannot trigger vehicle capture and recognition in real time;

[0095] The on-site event detection equipment refers to a sensor device deployed on-site for sensing and detecting the occurrence of on-site events, which may be an ultra-high warning laser radar or collision detection equipment in the collision monitoring and early warning scenario of the height-limited protective frame of a railway bridge or culvert, or a characteristic event detection equipment in other scenarios;

[0096] The collision alarm algorithm software refers to the software deployed in the collision detection device or other computing device, which receives the collision sensor signal and detects the collision event through algorithms such as signal filtering and machine learning;

[0097] The said retrieval of the involved vehicle from the historical video within a certain time range means that after the algorithm deployed on the on-site computing device receives the external trigger event, it calculates the original occurrence time of the external trigger event based on the current time. This calculation process takes into account the delay and delay jitter of the on-site device sensor signal filtering, event detection, signal transmission, event detection algorithm, etc., and also takes into account the time synchronization error between devices. Then, based on the calculated original occurrence time of the external trigger event, it is offset by a certain range, and frames are extracted from the video clip within this range to retrieve the involved vehicle;

[0098] The on-site computing device may be a smart camera, or other computing device with a certain computing power connected to a common camera;

[0099] Retrieving the involved vehicles means that on the on-site computing equipment, intelligent classification is achieved through the trained R-CNN high-precision deep learning inference model to identify the video frames of overheight vehicles, vehicles that collided with height-limited highways, or characteristic vehicles in other scenarios, and ensure the best license plate clarity;

[0100] The license plate recognition refers to completing the license plate recognition on the on-site computing device by using the video frame obtained from the previous step through a license plate recognition algorithm based on an end-to-end OCR model.

[0101] Through the above process, the asynchrony between external trigger events and vehicle capture and recognition in railway bridge and culvert height limit guard frame collision monitoring and early warning and similar scenarios can be effectively solved. After receiving the external trigger event, the on-site computing equipment retrieves the vehicle involved from the historical video within a certain time range and identifies the license plate.

[0102] Optionally, as another embodiment of the present invention, the present invention can effectively solve the problem of asynchrony between external trigger events and vehicle capture and recognition, and has good application effects in some resource-constrained scenarios such as harsh construction conditions and cost control. The method is flexible in deployment and can be deployed on field equipment or on a back-end server platform.

[0103] The present invention can ensure the success rate of capturing and recognizing license plates of vehicles involved in collisions with height-limited guardrails. Compared with the existing technical means, the vehicle capturing and license plate recognition of the present invention can adapt to the harsh on-site deployment environment of the railway bridge and culvert height-limited guardrail collision monitoring and early warning system. It has a high degree of on-site automation and can effectively improve the intelligent management level of railway height-limited guardrails, improve monitoring and management efficiency, and reduce personnel workload.

[0104] In addition, the present invention has a certain degree of universality and can be applied to any scenario where the external triggering event and the vehicle capture and recognition are not synchronized, such as monitoring and early warning fields such as highway bridges, culverts, and urban elevated roads. With the society's high attention to traffic safety and the increasing demand for intelligent transportation systems, the market potential of related traffic infrastructure collision monitoring and alarm systems will be further expanded, and the resulting social and economic benefits are very significant.

[0105] Optionally, as another embodiment of the present invention, the present invention can effectively solve the problems of railway bridge and culvert height limit guard frame collision monitoring and early warning and similar scenarios. Because the detection of external trigger events is affected by the equipment calculation and processing delay, message transmission delay and delay jitter, and is out of sync with vehicle capture and identification, the on-site computing device restores the original occurrence time of the external trigger event after receiving the external trigger event, and then retrieves the vehicle involved from the historical video within a certain time range and identifies the license plate.

[0106] Figure 2 A flowchart of a license plate recognition method provided by an embodiment of the present invention.

[0107] Optionally, as another embodiment of the present invention, Figure 2 As shown, a license plate recognition method comprises the following steps:

[0108] Generate a heartbeat message according to a preset period, send the heartbeat message to the collision processor, and obtain the heartbeat message sending time, where the heartbeat message sending time is the time when the identification processor sends the heartbeat message;

[0109] When receiving a heartbeat message sent by the identification processor, returning a heartbeat response message to the identification processor according to the heartbeat message;

[0110] Calculate the inter-device time synchronization error between the collision processor and the collision processor according to the heartbeat message sending time and the time corresponding to the receiving of the heartbeat response message;

[0111] Obtaining an original collision signal from a collision detection device, and obtaining an original collision time, wherein the original collision time is the time when the collision processor obtains the original collision signal from the collision detection device;

[0112] Performing detection and analysis on the original collision signal and the original collision moment to obtain a collision detection time group;

[0113] Obtaining a time group receiving time, wherein the time group receiving time is a time when the recognition processor receives the collision detection time group;

[0114] Performing a time delay analysis on the time group receiving moment, the time synchronization error and the collision detection time group to obtain a collision occurrence time period, and obtaining a collision occurrence video from a camera according to the collision occurrence time period;

[0115] The collision video is identified and analyzed to obtain vehicle license plate information.

[0116] Optionally, as an embodiment of the present invention, the heartbeat response message carries a heartbeat message receiving time and a heartbeat response message sending time, the heartbeat message receiving time is the time when the collision processor receives the heartbeat message sent by the identification processor, and the heartbeat response message sending time is the time when the collision processor sends the heartbeat response message.

[0117] The process of calculating the inter-device time synchronization error between the collision processor and the collision processor according to the heartbeat message sending time and the time corresponding to the receiving of the heartbeat response message includes:

[0118] Receive the heartbeat response message and obtain the heartbeat response message receiving time, where the heartbeat response message receiving time is the time when the identification processor receives the heartbeat response message;

[0119] The synchronization error between devices is calculated by using the first formula for the heartbeat message sending time, the heartbeat message receiving time, the heartbeat response message sending time, and the heartbeat response message receiving time, and the time synchronization error is obtained. The first formula is:

[0120]

[0121] Wherein, Δt0 is the time synchronization error between devices, T0 is the time when the heartbeat message is sent, T1 is the time when the heartbeat message is received, T2 is the time when the heartbeat response message is sent, and T3 is the time when the heartbeat response message is received.

[0122] Optionally, as an embodiment of the present invention, the process of detecting and analyzing the original collision signal and the original collision moment to obtain the collision detection time group includes:

[0123] S21: filtering the original collision signal to obtain a filtered collision signal;

[0124] S22: performing collision detection on the filtered collision signal by using a pre-built detection model to obtain a detection result of whether a collision exists, and obtain a target collision time, where the target collision time is the time when the detection result of whether a collision exists is obtained;

[0125] S23: If the detection result is a successful collision, the target collision moment and the original collision moment are differentially calculated to obtain a collision event occurrence delay, and the collision event occurrence delay and the target collision moment are sent to the recognition processor together. The collision detection time group includes the collision event occurrence delay and the target collision moment.

[0126] Optionally, another embodiment of the present invention provides a license plate recognition system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the license plate recognition method described above is implemented. The system may be a computer or other system.

[0127] Optionally, another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the license plate recognition method as described above is implemented.

[0128] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0129] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0132] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0133] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. For such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0134] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A license plate recognition system, It is characterized in that include: Identification handlers and collision handlers, The identification processor is used to generate a heartbeat message according to a preset period, send the heartbeat message to the collision processor, and obtain a heartbeat message sending time, where the heartbeat message sending time is the time when the identification processor sends the heartbeat message; The collision processor is used for returning a heartbeat response message to the identification processor according to the heartbeat message when receiving the heartbeat message sent by the identification processor; The identification processor is used to calculate the inter-device time synchronization error between the collision processor and the collision processor according to the time when the heartbeat message is sent and the time corresponding to the reception of the heartbeat response message; The collision processor is also used to obtain an original collision signal from the collision detection device and obtain an original collision time, wherein the original collision time is the time when the collision processor obtains the original collision signal from the collision detection device; Detecting and analyzing the original collision signal and the original collision time to obtain a collision detection time group, and sending the collision detection time group to the recognition processor; The recognition processor is further used to receive the collision detection time group and obtain a time group receiving moment, wherein the time group receiving moment is a moment when the recognition processor receives the collision detection time group; Performing a time delay analysis on the time group receiving moment, the time synchronization error and the collision detection time group to obtain a collision occurrence time period, and obtaining a collision occurrence video from a camera according to the collision occurrence time period; The collision video is identified and analyzed to obtain vehicle license plate information.

2. The license plate recognition system according to claim 1, It is characterized in that The heartbeat response message carries a heartbeat message receiving time and a heartbeat response message sending time, wherein the heartbeat message receiving time is the time when the collision processor receives the heartbeat message sent by the identification processor, and the heartbeat response message sending time is the time when the collision processor sends the heartbeat response message. In the identification processor, the process of calculating the inter-device time synchronization error between the device and the collision processor according to the time when the heartbeat message is sent and the time corresponding to the reception of the heartbeat response message includes: Receive the heartbeat response message and obtain the heartbeat response message receiving time, where the heartbeat response message receiving time is the time when the identification processor receives the heartbeat response message; The synchronization error between devices is calculated by using the first formula for the heartbeat message sending time, the heartbeat message receiving time, the heartbeat response message sending time, and the heartbeat response message receiving time, and the time synchronization error is obtained. The first formula is: Among them, Δt 0 is the time synchronization error between devices, T 0 The time when the heartbeat message is sent, T 1 is the time when the heartbeat message is received, T 2 The time when the heartbeat response message is sent, T 3 The time when the heartbeat response message is received.

3. The license plate recognition system according to claim 1, It is characterized in that In the collision processor, the process of detecting and analyzing the original collision signal and the original collision time to obtain the collision detection time group includes: S21: filtering the original collision signal to obtain a filtered collision signal; S22: performing collision detection on the filtered collision signal by using a pre-built detection model to obtain a detection result of whether a collision exists, and obtain a target collision time, where the target collision time is the time when the detection result of whether a collision exists is obtained; S23: If the detection result is a successful collision, the target collision moment and the original collision moment are differentially calculated to obtain a collision event occurrence delay, and the collision event occurrence delay and the target collision moment are sent to the recognition processor together. The collision detection time group includes the collision event occurrence delay and the target collision moment.

4. The license plate recognition system according to claim 3, It is characterized in that In the identification processor, the process of performing delay analysis on the time group receiving moment, the time synchronization error and the collision detection time group to obtain the collision occurrence time period includes: The total delay of the collision event is calculated by the second formula on the collision event occurrence delay, the target collision time, the time synchronization error and the time group receiving time to obtain the total delay of the collision event. The second formula is: Δt c =Δt a +Δt b , Among them, Δt b =T b -(T c +Δt 0 ), Among them, Δt c is the total delay of the collision event, Δt a is the time delay of the collision event, Δt b is the collision event message transmission delay, T b is the time group receiving time, T c is the target collision moment, Δt 0 is the time synchronization error; The actual collision time is obtained by calculating the time group receiving time and the total delay of the collision event through the third formula, and the third formula is: T′ a =T b -Δt c , Among them, T′ a is the actual collision moment, T b is the receiving time of the time group, Δt c is the total delay of the collision event; Returning to the collision processor to obtain the original collision signal from the collision detection device again until a preset number of collisions is reached, thereby obtaining a total delay of multiple collision events; The delay jitter of the collision event is calculated by calculating the total delay of all the collision events through the fourth formula to obtain the delay jitter of the collision event. The fourth formula is: in, Among them, T jitter is the collision event delay jitter, Δt ci is the total delay of the ith collision event, μ is the mean of the total delay of the collision event, and N is the preset number of collisions; The difference between the actual collision moment and the collision event delay jitter is taken as the starting moment, the sum of the actual collision moment and the collision event delay jitter is taken as the ending moment, and all moments between the starting moment and the ending moment are the collision occurrence time period.

5. The license plate recognition system according to claim 1, It is characterized in that In the recognition processor, the process of performing recognition analysis on the collision video to obtain the vehicle license plate information includes: Extracting key frames from the collision video to obtain multiple collision video images; Perform vehicle recognition on all the collision video images using a pre-built R-CNN high-precision deep learning inference model to obtain collision vehicle images; The vehicle license plate information is obtained by performing license plate recognition on the collision vehicle image using a pre-built OCR optical character recognition model.

6. A license plate recognition method, It is characterized in that The steps include: Generate a heartbeat message according to a preset period, send the heartbeat message to the collision processor, and obtain the heartbeat message sending time, where the heartbeat message sending time is the time when the identification processor sends the heartbeat message; When receiving a heartbeat message sent by the identification processor, returning a heartbeat response message to the identification processor according to the heartbeat message; Calculate the inter-device time synchronization error between the collision processor and the collision processor according to the heartbeat message sending time and the time corresponding to the receiving of the heartbeat response message; Obtaining an original collision signal from a collision detection device, and obtaining an original collision time, wherein the original collision time is the time when the collision processor obtains the original collision signal from the collision detection device; Performing detection and analysis on the original collision signal and the original collision moment to obtain a collision detection time group; Obtaining a time group receiving time, wherein the time group receiving time is a time when the recognition processor receives the collision detection time group; Performing a time delay analysis on the time group receiving moment, the time synchronization error and the collision detection time group to obtain a collision occurrence time period, and obtaining a collision occurrence video from a camera according to the collision occurrence time period; The collision video is identified and analyzed to obtain vehicle license plate information.

7. The license plate recognition method according to claim 6, It is characterized in that The heartbeat response message carries a heartbeat message receiving time and a heartbeat response message sending time, wherein the heartbeat message receiving time is the time when the collision processor receives the heartbeat message sent by the identification processor, and the heartbeat response message sending time is the time when the collision processor sends the heartbeat response message. The process of calculating the inter-device time synchronization error between the collision processor and the collision processor according to the heartbeat message sending time and the time corresponding to the receiving of the heartbeat response message includes: Receive the heartbeat response message and obtain the heartbeat response message receiving time, where the heartbeat response message receiving time is the time when the identification processor receives the heartbeat response message; The synchronization error between devices is calculated by using the first formula for the heartbeat message sending time, the heartbeat message receiving time, the heartbeat response message sending time, and the heartbeat response message receiving time, and the time synchronization error is obtained. The first formula is: Among them, Δt 0 is the time synchronization error between devices, T 0 The time when the heartbeat message is sent, T 1 is the time when the heartbeat message is received, T 2 The time when the heartbeat response message is sent, T 3 The time when the heartbeat response message is received.

8. The license plate recognition method according to claim 6, It is characterized in that The process of detecting and analyzing the original collision signal and the original collision moment to obtain a collision detection time group includes: S21: filtering the original collision signal to obtain a filtered collision signal; S22: performing collision detection on the filtered collision signal by using a pre-built detection model to obtain a detection result of whether a collision exists, and obtain a target collision time, where the target collision time is the time when the detection result of whether a collision exists is obtained; S23: If the detection result is a successful collision, the target collision moment and the original collision moment are differentially calculated to obtain a collision event occurrence delay, and the collision event occurrence delay and the target collision moment are sent to the recognition processor together. The collision detection time group includes the collision event occurrence delay and the target collision moment.

9. A license plate recognition system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the license plate recognition method according to any one of claims 6 to 8 is implemented.

10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the license plate recognition method according to any one of claims 6 to 8 is implemented.