License plate recognition method based on vehicle face recognition, electronic equipment and storage medium
Through the method based on vehicle face recognition, the optimization of YOLOv3 algorithm and twin network model to identify the license plate number of Mengpian vehicles has been solved, and the problem that regulatory authorities cannot identify Mengpian vehicles in a timely manner has been achieved, which has achieved high accuracy and rapid identification effects, and reduced roads Safety hazards.
Patent Information
- Application Number
- CN202410900839.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-05
- Publication Date
- 2024-11-26
AI Technical Summary
Regulatory authorities are unable to identify and crack down on Mengpian freight vehicles in a timely manner, resulting in road safety hazards and it is difficult for existing technology to quickly obtain information about Mengpian vehicles.
Using a method based on vehicle face recognition, the optimized YOLOv3 algorithm is used to detect the vehicle face, and the twin network model is used to recognize the vehicle face. The license plate number of the Meng brand vehicle is determined through the similarity comparison of vehicle images and the trajectory reverse traceability, and the results are summarized. Send to law enforcement officers.
The accuracy and speed of identification of license plate numbers of Mengpian vehicles has been improved, ensuring that regulatory authorities can timely crack down on Mengpian vehicles and reduce road safety hazards.
Smart Images

Figure CN119027930A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of license plate recognition technology, and in particular to a license plate recognition method based on vehicle face recognition, an electronic device and a storage medium Background Art
[0002] With the continuous improvement of modern transportation convenience, the number of vehicles on the road is also increasing day by day. However, in addition to the convenience of people's lives, it also hides various safety hazards. In particular, freight vehicles often overload on the road in order to reduce transportation costs, and in order to escape the punishment of regulatory authorities, they cover the vehicle license plate when passing through the inspection site, and then reveal the license plate when leaving the inspection site. The regulatory authorities are also unable to quickly obtain the information of the unlicensed vehicles, resulting in the inability to regulate and crack down on the unlicensed vehicles in time. Therefore, there is an urgent need for a method for identifying the license plates of unlicensed freight vehicles, so that regulatory authorities can crack down on unlicensed vehicles in a timely manner and reduce road safety hazards. Summary of the invention
[0003] In order to solve the technical problem raised in the above background technology that the regulatory authorities are unable to crack down on vehicles with concealed license plates in a timely manner, resulting in safety hazards on the roads.
[0004] In a first aspect, the present invention provides a license plate recognition method based on vehicle face recognition, which specifically comprises the following steps:
[0005] S11, when a licensed vehicle passes through a detection station, obtaining a vehicle image;
[0006] S12, inputting the vehicle image into a vehicle face detection model to obtain a vehicle face image;
[0007] S13, inputting the vehicle face image into a trained vehicle face recognition model to obtain a vehicle face recognition result, and obtaining the license plate number of the masked vehicle according to the vehicle face recognition result;
[0008] S14, tracing back the track of the suspected vehicle to determine the license plate number of the masked vehicle;
[0009] S15, sending the license plate number of the covered vehicle to law enforcement personnel, who will then perform law enforcement management.
[0010] Optionally, an optimized YOLOv3 algorithm is used to perform vehicle face detection on the acquired vehicle image.
[0011] Furthermore, the optimization of the YOLOv3 algorithm specifically includes:
[0012] First, insert a BN layer between the fully connected layer and the nonlinear layer;
[0013] The calculation formula before the convolution layer and BN layer fusion is as follows:
[0014] out(j)=x(i)×w(0)+x(i+1)×w(1)+x(i+2)×w(2)+...+x(i+k)×w(k)+b
[0015]
[0016] Among them, x i is the image data, w i is the weight, α is the scaling factor, μ is the mean, δ 2 is the variance, β is the bias, ε is the minimum value (to prevent the denominator from becoming zero), and the convolution formula is as follows:
[0017]
[0018] Secondly, cancel the BN layer and use new weight coefficients and offsets to express the calculation formula;
[0019] The calculation formula is adjusted, and the adjusted formula is as follows:
[0020]
[0021] After adjustment, the new weight coefficients are as follows:
[0022]
[0023] The new offsets are as follows:
[0024]
[0025] The final calculation formula is as follows:
[0026]
[0027] After the above adjustment, the BN layer is cancelled, and new weight coefficients and offsets are used to express the calculation formula.
[0028] Optionally, a twin network model is used as a vehicle face recognition model.
[0029] Furthermore, the S13 specifically includes:
[0030] The extracted face image X1 of the masked vehicle is input into the first sub-network, and then fed into the convolutional layer, pooling layer and fully connected layer in sequence, and finally the first feature vector G is output. W (X1);
[0031] The car face image X2 in the database is input into the second sub-network, and is fed to the convolutional layer, pooling layer and fully connected layer in sequence, and finally the second feature vector G is output.W (X2);
[0032] The formula for calculating the similarity between the vehicle face image X1 and the vehicle face image X2 is as follows:
[0033] E W (X1,X2)=||G W (X1)-G W (X2)||
[0034] The similarity between the vehicle face image X1 and the vehicle face image X2 is compared with a set threshold. If the similarity is greater than the set threshold, the vehicle corresponding to the vehicle face image X2 is determined to be a suspect vehicle.
[0035] Furthermore, the set threshold is set to a numerical range, and all vehicles whose similarities fall within the numerical range are determined to be suspected vehicles.
[0036] Furthermore, the S14 specifically includes the following steps:
[0037] Taking the time when the masked vehicle passes the checkpoint as the starting point, intercept the time window to obtain the driving trajectory of the masked vehicle. At the same time, intercept the time window with the starting point to obtain the driving trajectory of each of the suspected vehicles, and judge whether the driving trajectories of the masked vehicle and the suspected vehicle overlap. If they overlap, it is determined that the overlapping suspected vehicle is the masked vehicle, and the license plate number of the suspected vehicle is obtained as the license plate number of the masked vehicle.
[0038] In a second aspect, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned license plate recognition method based on vehicle face recognition when executing the computer program.
[0039] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned license plate recognition method based on vehicle face recognition is implemented.
[0040] Beneficial effects:
[0041] The present invention uses an optimized YOLOv3 algorithm to perform vehicle face detection on the acquired vehicle image, and fuses the parameters of the BN layer with the convolution layer to improve the speed of the model forward reasoning. Using the optimized YOLOv3 algorithm, the accuracy of vehicle face detection is 93.4%, which meets the requirements of accuracy and calculation speed. The twin neural network is used to identify and compare the vehicle face image. After screening out multiple suspected vehicles, the driving trajectories of multiple suspected vehicles are traced back and compared with the masked vehicles, which can not only reduce the time complexity, but also further improve the accuracy of identifying the license plate number of the masked vehicle. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 A flow chart of a license plate recognition method based on vehicle face recognition provided by the present invention. DETAILED DESCRIPTION
[0043] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0044] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.
[0045] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0046] The specific implementation of the present invention will be further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0047] like Figure 1 As shown, the present invention provides a license plate recognition method based on vehicle face recognition, which specifically includes the following steps:
[0048] S11, when a licensed vehicle passes through a detection station, obtaining a vehicle image;
[0049] The technical solution of the present invention is to perform license plate recognition on covered vehicles. It is a license plate recognition method after covered vehicles have been identified in vehicle images obtained by the detection site. As for how to identify and screen out covered vehicles in all captured images, any method in the existing technology can be used, which will not be repeated here.
[0050] S12, inputting the vehicle image into a vehicle face detection model to obtain a vehicle face image;
[0051] Since vehicle face detection has high requirements for detection accuracy and real-time performance, and the detection accuracy and speed of the YOLOv3 algorithm are very high, the present invention adopts an optimized YOLOv3 algorithm to perform vehicle face detection on the acquired vehicle image.
[0052] When training the YOLOv3 network model, the BN layer is usually placed after the convolution layer. The BN layer can speed up the convergence of the network and control overfitting. After the data is normalized by the BN layer, the problems of gradient disappearance and gradient explosion can be effectively solved. Although the BN layer has a good training effect, it will increase the amount of computation for network forward reasoning, reduce the performance of the model, and also occupy a relatively large amount of memory. Therefore, it is necessary to optimize the YOLOv3 algorithm and fuse the parameters of the BN layer with the convolution layer to increase the speed of the model's forward reasoning.
[0053] The optimization of the YOLOv3 algorithm specifically includes:
[0054] First, a BN layer is inserted between the fully connected layer and the nonlinear layer.
[0055] The calculation formula before the convolution layer and BN layer fusion is as follows:
[0056] out(j)=x(i)×w(0)+x(i+1)×w(1)+x(i+2)×w(2)+...+x(t+k)×w(k)+b
[0057]
[0058] Among them, x i is the image data, w iis the weight, α is the scaling factor, μ is the mean, δ 2 is the variance, β is the bias, ε is the minimum value (to prevent the denominator from becoming zero), and the convolution formula is as follows:
[0059]
[0060] Secondly, cancel the BN layer and use new weight coefficients and offsets to express the calculation formula.
[0061] The calculation formula is adjusted, and the adjusted formula is as follows:
[0062]
[0063] After adjustment, the new weight coefficients are as follows:
[0064]
[0065] The new offsets are as follows:
[0066]
[0067] The final calculation formula is as follows:
[0068]
[0069] After the above adjustment, the BN layer is cancelled, and new weight coefficients and offsets are used to express the calculation formula.
[0070] Using the optimized YOLOv3 algorithm, the accuracy of vehicle face detection is 93.4%, meeting the requirements of accuracy and computing speed.
[0071] In order to compare and identify the face of the covered vehicle in the subsequent steps, the present invention will input the vehicle images collected by each checkpoint into the vehicle face detection model, obtain the vehicle face images of all vehicles captured by the checkpoints, and store the vehicle face images of all vehicles captured by the checkpoints and the associated vehicle information in the database. The vehicle information includes the license plate number, license plate color, vehicle color, vehicle type, vehicle axle number, vehicle brand, etc.
[0072] For the same vehicle, the features of the vehicle face do not remain unchanged. When the vehicle passes through different checkpoints, the vehicle image will be collected and the corresponding vehicle face image will be detected and stored in the database. The database will update the original vehicle face image in the database according to the latest vehicle face image to improve robustness.
[0073] S13, inputting the vehicle face image into a trained vehicle face recognition model to obtain a vehicle face recognition result, and obtaining the license plate number of the masked vehicle according to the vehicle face recognition result;
[0074] In real life, no two faces are exactly alike. Even if they are similar, there are still differences. However, it is very difficult to distinguish between two cars of the same color and model. Even so, there are still some special marks that can be used to distinguish between two cars of the same color and model. For example, there are often some differences in the face area of the vehicle, especially in the windshield area, including the location, color and number of annual inspection marks, as well as decorations, scratches, etc.
[0075] The present invention adopts a twin network model as a vehicle face recognition model.
[0076] The S13 specifically includes:
[0077] The extracted face image X1 of the masked vehicle is input into the first sub-network, and then fed into the convolutional layer, pooling layer and fully connected layer in sequence, and finally the first feature vector G is output. W (X1);
[0078] The car face image X2 in the database is input into the second sub-network, and is fed to the convolutional layer, pooling layer and fully connected layer in sequence, and finally the second feature vector G is output. W (X2);
[0079] The formula for calculating the similarity between the vehicle face image X1 and the vehicle face image X2 is as follows:
[0080] E w (X1,X2)=||G W (X1)-G W (X2)||
[0081] Compare the similarity between the vehicle face image X1 and the vehicle face image X2 with a set threshold, and if the similarity is greater than the set threshold, determine that the vehicle corresponding to the vehicle face image X2 is a suspect vehicle. Specifically, the set threshold is set to a numerical range, and all vehicles whose similarities fall within the numerical range are determined as suspect vehicles.
[0082] S14, tracing back the track of the suspected vehicle to determine the license plate number of the masked vehicle.
[0083] The S14 specifically includes the following steps:
[0084] Taking the time when the masked vehicle passes the checkpoint as the starting point, intercept the time window to obtain the driving trajectory of the masked vehicle. At the same time, intercept the time window with the starting point to obtain the driving trajectory of each of the suspected vehicles, and judge whether the driving trajectories of the masked vehicle and the suspected vehicle overlap. If they overlap, it is determined that the overlapping suspected vehicle is the masked vehicle, and the license plate number of the suspected vehicle is obtained as the license plate number of the masked vehicle.
[0085] By tracing back the driving trajectories of multiple suspected vehicles and comparing them with masked vehicles, not only can the time complexity be reduced, but the accuracy of identifying the license plate numbers of masked vehicles can also be further improved.
[0086] S15, sending the license plate number of the covered vehicle to law enforcement personnel, who will then perform law enforcement management.
[0087] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned license plate recognition method based on vehicle face recognition when executing the computer program.
[0088] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned license plate recognition method based on vehicle face recognition is implemented.
[0089] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A license plate recognition method based on vehicle face recognition, characterized in that: The specific steps include: S11, when a licensed vehicle passes through a detection station, obtaining a vehicle image; S12, inputting the vehicle image into a vehicle face detection model to obtain a vehicle face image; S13, inputting the vehicle face image into a trained vehicle face recognition model to obtain a vehicle face recognition result, and obtaining the license plate number of the masked vehicle according to the vehicle face recognition result; S14, tracing back the track of the suspected vehicle to determine the license plate number of the masked vehicle; S15, sending the license plate number of the covered vehicle to law enforcement personnel, who will then perform law enforcement management.
2. The license plate recognition method based on vehicle face recognition as claimed in claim 1, characterized in that: The optimized YOLOv3 algorithm is used to perform vehicle face detection on the acquired vehicle image.
3. The license plate recognition method based on vehicle face recognition as claimed in claim 2, characterized in that: The optimization of the YOLOv3 algorithm specifically includes: First, insert a BN layer between the fully connected layer and the nonlinear layer; The calculation formula before the convolution layer and BN layer fusion is as follows: out(j)=x(i)×w(0)+x(i+1)×w(1)+x(i+2)×w(2+...+x(i+k)×w(k)+b Among them, x i is the image data, w i is the weight, α is the scaling factor, μ is the mean, δ 2 is the variance, β is the bias, ε is the minimum value (to prevent the denominator from becoming zero), and the convolution formula is as follows: Secondly, cancel the BN layer and use new weight coefficients and offsets to express the calculation formula; The calculation formula is adjusted, and the adjusted formula is as follows: After adjustment, the new weight coefficients are as follows: The new offsets are as follows: The final calculation formula is as follows: After the above adjustment, the BN layer is cancelled, and new weight coefficients and offsets are used to express the calculation formula.
4. The license plate recognition method based on vehicle face recognition as claimed in claim 1, characterized in that: The twin network model is used as the vehicle face recognition model.
5. The license plate recognition method based on vehicle face recognition as claimed in claim 4, characterized in that: The S13 specifically includes: The extracted face image λ1 of the masked vehicle is input into the first sub-network, and then fed into the convolutional layer, pooling layer and fully connected layer in sequence, and finally the first feature vector G is output. W (X1); The car face image X2 in the database is input into the second sub-network, and is fed to the convolutional layer, pooling layer and fully connected layer in sequence, and finally the second feature vector G is output. W (X2); The formula for calculating the similarity between the vehicle face image X1 and the vehicle face image X2 is as follows: E W (X1,X2)=||G W (X1)-G W (X2)| The similarity between the vehicle face image X1 and the vehicle face image X2 is compared with a set threshold. If the similarity is greater than the set threshold, the vehicle corresponding to the vehicle face image X2 is determined to be a suspect vehicle.
6. The license plate recognition method based on vehicle face recognition as claimed in claim 5, characterized in that: The set threshold is set to a numerical range, and all vehicles whose similarities fall within the numerical range are determined to be suspected vehicles.
7. The license plate recognition method based on vehicle face recognition as claimed in claim 1, characterized in that: The S14 specifically includes the following steps: Taking the time when the masked vehicle passes the checkpoint as the starting point, intercept the time window to obtain the driving trajectory of the masked vehicle. At the same time, intercept the time window with the starting point to obtain the driving trajectory of each of the suspected vehicles, and judge whether the driving trajectories of the masked vehicle and the suspected vehicle overlap. If they overlap, it is determined that the overlapping suspected vehicle is the masked vehicle, and the license plate number of the suspected vehicle is obtained as the license plate number of the masked vehicle.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for license plate recognition based on vehicle face recognition as described in any one of claims 1 to 7 is implemented.
9. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a license plate recognition method based on vehicle face recognition as described in any one of claims 1 to 7.
