Non-motor vehicle license plate identification method and system

Abstract

The invention relates to a non-motor vehicle license plate identification method and system. The method comprises the following steps: establishing a non-motor vehicle license plate detection training sample library, a non-motor vehicle license plate classification training sample library and a non-motor vehicle license plate matching template library; the training sample library is used for training a non-motor vehicle license plate target detection model and a non-motor vehicle license plate classification model; obtaining an optimal snapshot effect image of the non-motor vehicle target; acquiring non-motor vehicle rectangular frame information in the image with the optimal snapshot effect based on the non-motor vehicle license plate target detection model; non-motor vehicle license plate classification is carried out through the non-motor vehicle license plate classification model, a license plate area is corrected according to a classification result, and a positive view angle license plate image is obtained; and carrying out character extraction on the positive view angle license plate image, and carrying out non-motor vehicle license plate identification and normalization verification based on the non-motor vehicle license plate matching template library and the extracted characters. According to the invention, the identification of the local non-motor vehicle license plate can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent traffic control, and in particular to a non-motor vehicle license plate recognition method and system. Background Art

[0002] Non-motor vehicle license plates refer to non-motor vehicles such as electric bicycles and electric tricycles. Because the management requirements for non-motor vehicle license plates differ from those for motor vehicle license plates, and there is no unified national standard defining their style and format, the style, color, size, content, and character style of non-motor vehicle license plates are currently determined independently by local traffic management departments. This results in a wide variety of license plate colors, sizes, content, and character styles. Consequently, existing license plate recognition algorithms are designed only for standardized motor vehicle license plates and are unable to accommodate and effectively recognize the diverse styles of non-motor vehicle license plates. Summary of the Invention

[0003] To this end, the present invention provides a non-motor vehicle license plate recognition method and system, which realizes the recognition of local non-motor vehicle license plates through the steps of establishing a template library, detection model, character segmentation, recognition correction, etc., and supports non-motor vehicle data analysis and other services.

[0004] In order to solve the above technical problems, the present invention provides a non-motor vehicle license plate recognition method, comprising: Based on all non-motor vehicle license plate types and corresponding images collected locally, a non-motor vehicle license plate detection training sample library, a non-motor vehicle license plate classification training sample library, and a non-motor vehicle license plate matching template library are established; wherein the non-motor vehicle license plate matching template library records the character position, character type, and character encoding format rules in the non-motor vehicle license plate image; Training a non-motor vehicle license plate target detection model based on the non-motor vehicle license plate detection training sample library; training a non-motor vehicle license plate classification model based on the non-motor vehicle license plate classification training sample library; Extracting a video frame image from a camera video stream, and obtaining an optimal snapshot effect image of a non-motor vehicle target based on the video frame image; Based on the non-motor vehicle license plate target detection model, obtaining non-motor vehicle rectangular frame information in the best capture effect image; Based on the non-motor vehicle rectangular frame information in the best captured image, the non-motor vehicle license plate classification model is used to classify the non-motor vehicle license plate, and the license plate area is corrected according to the classification result to obtain a positive-view license plate image; Characters are extracted from the front-view license plate image, and non-motor vehicle license plate recognition and standardization verification are performed based on the non-motor vehicle license plate matching template library and the extracted characters.

[0005] In one embodiment of the present invention, based on all non-motor vehicle license plate types and corresponding images collected locally, a non-motor vehicle license plate detection training sample library, a non-motor vehicle license plate classification training sample library, and a non-motor vehicle license plate matching template library are established, including: Different non-motor vehicle license plates are divided into different types according to differences including license plate color, size, and character format; For each non-motor vehicle license plate type, original actual non-motor vehicle images taken by a camera are collected, and the non-motor vehicle and its corresponding non-motor vehicle license plate area are annotated to form a non-motor vehicle license plate detection training sample library; wherein the original actual non-motor vehicle images include non-motor vehicles, non-motor vehicle license plates, and background environment information; For each non-motor vehicle license plate type, a picture of the non-motor vehicle license plate area is intercepted from the original actual non-motor vehicle picture to form a non-motor vehicle license plate classification training sample library; For each type of non-motor vehicle license plate, front-view pictures including non-motor vehicle license plate pictures or license plate example pictures are collected, the front-view pictures are annotated, and the annotation information is saved as an XML file to form a non-motor vehicle license plate matching template library; wherein, the annotation information includes the rectangular frame position of characters and graphics, the character type and the encoding format rules of the characters, and the character type includes fixed characters and non-fixed characters.

[0006] In one embodiment of the present invention, the non-motor vehicle license plate target detection model includes a Yolo target detection algorithm; the non-motor vehicle license plate classification model includes an Alexnet target classification algorithm.

[0007] In one embodiment of the present invention, extracting a video frame image from a camera video stream, and obtaining an optimal snapshot effect image of a non-motor vehicle target based on the video frame image includes: Detecting a first non-motor vehicle target from the current frame image using the non-motor vehicle license plate target detection model; In response to the presence of a first non-motor vehicle target in the current frame image, storing the current frame image, recording first non-motor vehicle rectangular frame information corresponding to each first non-motor vehicle target in the current frame image, and extracting corresponding first non-motor vehicle target features; Recording the current frame image as a first frame image, and recording all first non-motor vehicle targets in the current frame image as a current non-motor vehicle set; Detecting the next frame of image, and in response to the presence of a second non-motor vehicle target in the next frame of image, recording second non-motor vehicle rectangular frame information corresponding to each second non-motor vehicle target in the next frame of image, and extracting corresponding second non-motor vehicle target features, wherein the first non-motor vehicle rectangular frame information and the second non-motor vehicle rectangular frame information both include a non-motor vehicle coordinate position rectangular frame and a license plate position rectangular frame; Based on the first non-motor vehicle rectangular frame information, the first non-motor vehicle target feature, the second non-motor vehicle rectangular frame information and the second non-motor vehicle target feature, all the second non-motor vehicle targets are traversed and compared with all the first non-motor vehicle targets in terms of IOU overlap and non-motor vehicle feature similarity, and the comparison results are updated to the current non-motor vehicle set.

[0008] In one embodiment of the present invention, performing a traversal comparison of IOU overlap and non-motor vehicle feature similarity on all second non-motor vehicle targets and all first non-motor vehicle targets, and updating the comparison results to the current non-motor vehicle set, includes: In response to the IOU overlap ratio of the first non-motor vehicle target and the second non-motor vehicle target in the current non-motor vehicle set exceeding a preset overlap threshold and the non-motor vehicle feature similarity exceeding a preset non-motor vehicle similarity threshold, it is determined that the two are the same target, and the rectangular frame area size of the non-motor vehicle coordinate position rectangular frame and / or license plate position rectangular frame of the two is judged, and the target with the larger rectangular frame is updated to the current non-motor vehicle set.

[0009] In one embodiment of the present invention, the preset overlap threshold is 90%; the non-motor vehicle similarity threshold is 90%.

[0010] In one embodiment of the present invention, it further comprises: If, when detecting the next frame of image, it is found that among all the detected second non-motor vehicle targets, there is a second non-motor vehicle target that is different from all the first non-motor vehicles in the current non-motor vehicle set, then the second non-motor vehicle is used as a new target and updated to the current non-motor vehicle set; If all first non-motor vehicle targets in the current non-motor vehicle set do not identify the same target when compared with all second non-motor vehicle targets detected in the next frame of image, then the image in the current non-motor vehicle set is determined to be the best captured image, and is removed from the current non-motor vehicle set after being stored. The above operation is repeated to record and store the best snapshot images of all non-motor vehicles passing through the video screen.

[0011] In one embodiment of the present invention, based on the non-motor vehicle rectangular frame information in the best snapshot effect image, the non-motor vehicle license plate classification model is used to classify the non-motor vehicle license plate, and the license plate area is corrected according to the classification result to obtain a positive-view license plate image, including: According to the classification results, determine the RGB components corresponding to the color of the non-motor vehicle license plate; According to the RGB components, a binarization threshold is set, the license plate area is binarized, and a binary image of the license plate area is cropped; Using a rotating rectangular frame, matching the binary image area, determining the actual rectangular frame of the license plate, and cropping the actual license plate image according to the actual rectangular frame; The actual license plate image is mapped into an orthographic license plate image through rotation transformation and affine transformation.

[0012] In one embodiment of the present invention, character extraction is performed on the front-view license plate image, and non-motor vehicle license plate recognition and standardization verification is performed based on the non-motor vehicle license plate matching template library and the extracted characters, including: Adjusting the size of the front-view license plate image to be consistent with the corresponding license plate template in the non-motor vehicle license plate matching template library; Constructing a character position binary mask template according to the character positions marked in the non-motor vehicle license plate matching template library; The positive-view license plate image and the character position binary mask template are combined through image and operation to extract the position of each character and distinguish between fixed characters and non-fixed characters; For each extracted character, the character encoding format rules set in the non-motor vehicle license plate matching template library are used to limit the character set range of the corresponding position characters, thereby improving the accuracy of license plate recognition, and the characters are identified and compared through the character recognition algorithm, thereby realizing the recognition and standardization verification of non-motor vehicle license plates.

[0013] The present invention also provides a non-motor vehicle license plate recognition system, comprising: A sample and template library establishment module is used to establish a non-motor vehicle license plate detection training sample library, a non-motor vehicle license plate classification training sample library, and a non-motor vehicle license plate matching template library based on all non-motor vehicle license plate types and corresponding images collected locally; wherein the non-motor vehicle license plate matching template library records the character position, character type, and character encoding format rules in the non-motor vehicle license plate image; A model training module is used to train a non-motor vehicle license plate target detection model based on the non-motor vehicle license plate detection training sample library; and to train a non-motor vehicle license plate classification model based on the non-motor vehicle license plate classification training sample library; The best snapshot effect image acquisition module is used to extract video frame images from the camera video stream and obtain the best snapshot effect image of the non-motor vehicle target based on the video frame images; A non-motor vehicle rectangular frame information acquisition module is used to acquire the non-motor vehicle rectangular frame information in the best captured effect image based on the non-motor vehicle license plate target detection model; A front-view license plate image acquisition module is used to classify non-motor vehicle license plates using the non-motor vehicle license plate classification model based on the non-motor vehicle rectangular frame information in the best captured image, and to correct the license plate area according to the classification result to obtain a front-view license plate image; The non-motor vehicle license plate recognition and standardization verification module is used to extract characters from the positive-view license plate image, and perform non-motor vehicle license plate recognition and standardization verification based on the non-motor vehicle license plate matching template library and the extracted characters.

[0014] The above technical solution of the present invention has the following advantages over the prior art: The non-motor vehicle license plate recognition method and system described in the present invention, combined with the steps of establishing a template library, detection model, character segmentation, recognition correction, etc., significantly improves the automatic detection, classification and recognition capabilities of non-motor vehicle license plates in various styles and environments, and provides reliable technology for various application scenarios such as traffic management and data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0016] Figure 1 It is a flow chart of the non-motor vehicle license plate recognition method of the present invention.

[0017] Figure 2 This is a schematic diagram of non-motor vehicle license plate pattern 1.

[0018] Figure 3 This is a schematic diagram of non-motor vehicle license plate pattern 2.

[0019] Figure 4 This is a schematic diagram of non-motor vehicle license plate pattern 3. DETAILED DESCRIPTION

[0020] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0021] In the present invention, "several" means one or more, "multiple" means more than two, "greater than," "less than," "exceeds," etc. are understood to exclude the number itself; "above," "below," "within," etc. are understood to include the number itself. In the description of the present invention, the use of "first" or "second" is solely for the purpose of distinguishing technical features and is not to be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0022] Example 1 Reference Figure 1 As shown, a non-motor vehicle license plate recognition method of this embodiment includes: S1. Based on all non-motor vehicle license plate types and corresponding pictures collected locally, establish a non-motor vehicle license plate detection training sample library, a non-motor vehicle license plate classification training sample library and a non-motor vehicle license plate matching template library; wherein, the non-motor vehicle license plate matching template library records the character position, character type and character encoding format rules in the non-motor vehicle license plate image.

[0023] Specifically, this step includes: S11. Different non-motor vehicle license plates are classified into different types based on differences in plate color, size, and character format. It should be noted that, in addition to the plate number, each non-motor vehicle has differences in plate color, plate size, plate character format, etc., which are considered different plate types. Figure 2 and Figure 3 Shown are two different license plate types.

[0024] S12. For each non-motor vehicle license plate type, collect 2,000 original, actual non-motor vehicle images taken by a camera, and label the non-motor vehicle and its corresponding non-motor vehicle license plate area to form a non-motor vehicle license plate detection training sample library; wherein the original, actual non-motor vehicle images include the non-motor vehicle, the non-motor vehicle license plate, and background environment information; S13. For each non-motor vehicle license plate type, 1,000 images of the non-motor vehicle license plate area are captured from the original actual non-motor vehicle images (including only the license plate portion, excluding the non-motor vehicle, background environment, etc.) to form a non-motor vehicle license plate classification training sample library; S14. For each type of non-motor vehicle license plate, collect front-view images (only including license plates) including non-motor vehicle license plate images or license plate example images, annotate the front-view images, and save the annotation information as an XML file to form a non-motor vehicle license plate matching template library; wherein the annotation information includes the rectangular frame position of characters and graphics, character type, and character encoding format rules, and the character type includes fixed characters and non-fixed characters.

[0025] It should be noted that when annotating the above-mentioned front view pictures, each picture needs to be annotated with the position of the rectangular frame of Chinese, English, numbers, graphics and other characters in the sample picture, the fixed characters and non-fixed characters, the encoding format rules of the characters, and the annotation information is saved in an XML file. Figure 2 The positions marked with "Quanzhou", "Fengze" and "56789" are fixed characters, and "56789" is a non-fixed character; Figure 3 In the figure, the positions of "Quanzhou", "Fengze", "0265" and barrier-free signs are marked, among which "Quanzhou", "Fengze" and barrier-free signs are fixed characters, and "0265" is a non-fixed character; in Picture 4, the positions of "Hangzhou", "electric bicycle" and "3010131" are marked, among which "Hangzhou" and "electric bicycle" are fixed characters, and "3010131" is a non-fixed character.

[0026] S2. Training a non-motor vehicle license plate target detection model based on the non-motor vehicle license plate detection training sample library; training a non-motor vehicle license plate classification model based on the non-motor vehicle license plate classification training sample library.

[0027] Exemplarily, in this embodiment, the non-motor vehicle license plate target detection model includes the YOLO (You Only Look Once) target detection algorithm, which is used to detect non-motor vehicles and their non-motor vehicle license plates in the camera capture image; the non-motor vehicle license plate classification model includes the Alexnet target classification algorithm, which is used to classify the detected non-motor vehicle license plates and confirm the license plate style of the non-motor vehicle.

[0028] S3. Extracting video frame images from the camera video stream, specifically converting the camera video stream into images by intercepting one frame of image from the video per second, and obtaining the best snapshot effect image of the non-motor vehicle target based on the video frame images.

[0029] Specifically, this step includes: S31, detecting a first non-motor vehicle target from the current frame image using the non-motor vehicle license plate target detection model; S32: In response to a first non-motor vehicle target existing in the current frame image, storing the current frame image, recording first non-motor vehicle rectangular frame information corresponding to each first non-motor vehicle target in the current frame image, and extracting corresponding first non-motor vehicle target features; S33, recording the current frame image as a first frame image, and recording all first non-motor vehicle targets in the current frame image as a current non-motor vehicle set; S34, detecting the next frame of image, and in response to the presence of a second non-motor vehicle target in the next frame of image, recording second non-motor vehicle rectangular frame information corresponding to each second non-motor vehicle target in the next frame of image, and extracting corresponding second non-motor vehicle target features, wherein the first non-motor vehicle rectangular frame information and the second non-motor vehicle rectangular frame information both include a non-motor vehicle coordinate position rectangular frame and a license plate position rectangular frame; S35. Based on the first non-motor vehicle rectangular frame information, the first non-motor vehicle target feature, the second non-motor vehicle rectangular frame information, and the second non-motor vehicle target feature, perform a traversal comparison of IOU (Intersection over Union) overlap and non-motor vehicle feature similarity on all second non-motor vehicle targets and all first non-motor vehicle targets, and update the comparison results to the current non-motor vehicle set.

[0030] Specifically, step S35 includes: S351, in response to the IOU overlap ratio of the first non-motor vehicle target and the second non-motor vehicle target in the current non-motor vehicle set exceeding a preset overlap threshold and the non-motor vehicle feature similarity exceeding a preset non-motor vehicle similarity threshold; in this embodiment, the preset overlap threshold is 90%; the non-motor vehicle similarity threshold is 90%.

[0031] S352: Determine that the two are the same target, and determine the rectangular frame area sizes of the non-motor vehicle coordinate position rectangular frames and / or license plate position rectangular frames of the two, and update the target with the larger rectangular frame to the current non-motor vehicle set.

[0032] Specifically, it also includes: S36. If, when detecting the next frame of image, it is found that among all the detected second non-motor vehicle targets, there is a second non-motor vehicle target that is different from all the first non-motor vehicles in the current non-motor vehicle set, then the second non-motor vehicle target is used as a new target and is updated to the current non-motor vehicle set; S37. If all first non-motor vehicle targets in the current non-motor vehicle set do not identify the same target when compared with all second non-motor vehicle targets detected in the next frame of image, then the image in the current non-motor vehicle set is determined to be the best captured image, stored, and then removed from the current non-motor vehicle set. S38, looping the above operations, recording and storing the best snapshot effect images of all non-motor vehicles passing through the video screen.

[0033] S4. Based on the non-motor vehicle license plate target detection model, obtain the non-motor vehicle rectangular frame information in the best snapshot effect image.

[0034] S5. Based on the non-motor vehicle rectangular frame information in the best captured image, classify the non-motor vehicle license plate using the non-motor vehicle license plate classification model, correct the license plate area according to the classification result, and obtain a positive-view license plate image.

[0035] Specifically, this step includes: S51. Determine the RGB components corresponding to the color of the non-motor vehicle license plate according to the classification result; S52, setting a binarization threshold according to the RGB components, binarizing the license plate area, and cropping a binary image of the license plate area; S53, using a rotating rectangular frame to match the binary image area, determine the actual rectangular frame of the license plate, and crop the actual license plate image according to the actual rectangular frame; S54: Map the actual license plate image into an orthographic license plate image through rotation transformation and affine transformation.

[0036] S6. Extract characters from the license plate image from the front view, and perform non-motor vehicle license plate recognition and compliance verification based on the non-motor vehicle license plate matching template library and the extracted characters.

[0037] Specifically, this step includes: S61, adjusting the size of the front-view license plate image to be consistent with the corresponding license plate template in the non-motor vehicle license plate matching template library; S62, constructing a character position binary mask template according to the character positions marked in the non-motor vehicle license plate matching template library; S63, performing an image AND operation on the front-view license plate image and the character position binary mask template to extract the position of each character and distinguish between fixed characters and non-fixed characters; S64: For each extracted character, according to the character encoding format rules set in the non-motor vehicle license plate matching template library, the character set range of the corresponding position character is limited to improve the license plate recognition accuracy, and the characters are recognized and compared through the character recognition algorithm, thereby realizing the recognition and standardization verification of the non-motor vehicle license plate. Figure 2 The fixed characters are limited to "Quanzhou" and "Fengze", and the non-fixed characters are limited to numbers.

[0038] Example 2 Based on the same inventive concept, this embodiment provides a non-motor vehicle license plate recognition system, the principle of which is similar to that of the non-motor vehicle license plate recognition method, and the repeated parts will not be repeated.

[0039] This embodiment provides a non-motor vehicle license plate recognition system, including: A sample and template library establishment module is used to establish a non-motor vehicle license plate detection training sample library, a non-motor vehicle license plate classification training sample library, and a non-motor vehicle license plate matching template library based on all non-motor vehicle license plate types and corresponding images collected locally; wherein the non-motor vehicle license plate matching template library records the character position, character type, and character encoding format rules in the non-motor vehicle license plate image; A model training module is used to train a non-motor vehicle license plate target detection model based on the non-motor vehicle license plate detection training sample library; and to train a non-motor vehicle license plate classification model based on the non-motor vehicle license plate classification training sample library; The best snapshot effect image acquisition module is used to extract video frame images from the camera video stream and obtain the best snapshot effect image of the non-motor vehicle target based on the video frame images; A non-motor vehicle rectangular frame information acquisition module is used to acquire the non-motor vehicle rectangular frame information in the best captured effect image based on the non-motor vehicle license plate target detection model; A front-view license plate image acquisition module is used to classify non-motor vehicle license plates using the non-motor vehicle license plate classification model based on the non-motor vehicle rectangular frame information in the best captured image, and to correct the license plate area according to the classification result to obtain a front-view license plate image; The non-motor vehicle license plate recognition and standardization verification module is used to extract characters from the positive-view license plate image, and perform non-motor vehicle license plate recognition and standardization verification based on the non-motor vehicle license plate matching template library and the extracted characters.

[0040] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0041] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0042] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0043] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0044] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for recognizing a non-motor vehicle license plate, characterized in that: include: Based on all non-motor vehicle license plate types and corresponding images collected locally, a non-motor vehicle license plate detection training sample library, a non-motor vehicle license plate classification training sample library, and a non-motor vehicle license plate matching template library are established; wherein the non-motor vehicle license plate matching template library records the character position, character type, and character encoding format rules in the non-motor vehicle license plate image; Training a non-motor vehicle license plate target detection model based on the non-motor vehicle license plate detection training sample library; training a non-motor vehicle license plate classification model based on the non-motor vehicle license plate classification training sample library; Extracting a video frame image from a camera video stream, and obtaining an optimal snapshot effect image of a non-motor vehicle target based on the video frame image; Based on the non-motor vehicle license plate target detection model, obtaining non-motor vehicle rectangular frame information in the best capture effect image; Based on the non-motor vehicle rectangular frame information in the best captured image, the non-motor vehicle license plate classification model is used to classify the non-motor vehicle license plate, and the license plate area is corrected according to the classification result to obtain a positive-view license plate image; Characters are extracted from the front-view license plate image, and non-motor vehicle license plate recognition and standardization verification are performed based on the non-motor vehicle license plate matching template library and the extracted characters.

2. A non-motor vehicle license plate recognition method according to claim 1, characterized in that: Based on all non-motor vehicle license plate types and corresponding images collected locally, a non-motor vehicle license plate detection training sample library, a non-motor vehicle license plate classification training sample library, and a non-motor vehicle license plate matching template library are established, including: Different non-motor vehicle license plates are divided into different types according to differences including license plate color, size, and character format; For each non-motor vehicle license plate type, original actual non-motor vehicle images taken by a camera are collected, and the non-motor vehicle and its corresponding non-motor vehicle license plate area are annotated to form a non-motor vehicle license plate detection training sample library; wherein the original actual non-motor vehicle images include non-motor vehicles, non-motor vehicle license plates, and background environment information; For each non-motor vehicle license plate type, a picture of the non-motor vehicle license plate area is intercepted from the original actual non-motor vehicle picture to form a non-motor vehicle license plate classification training sample library; For each type of non-motor vehicle license plate, front-view pictures including non-motor vehicle license plate pictures or license plate example pictures are collected, the front-view pictures are annotated, and the annotation information is saved as an XML file to form a non-motor vehicle license plate matching template library; wherein, the annotation information includes the rectangular frame position of characters and graphics, the character type and the encoding format rules of the characters, and the character type includes fixed characters and non-fixed characters.

3. The method for recognizing a non-motor vehicle license plate according to claim 1, wherein: The non-motor vehicle license plate target detection model includes a Yolo target detection algorithm; the non-motor vehicle license plate classification model includes an Alexnet target classification algorithm.

4. The method for recognizing a non-motor vehicle license plate according to claim 1, wherein: Extracting a video frame image from a camera video stream, and obtaining an optimal snapshot effect image of a non-motor vehicle target based on the video frame image, including: Detecting a first non-motor vehicle target from the current frame image using the non-motor vehicle license plate target detection model; In response to the presence of a first non-motor vehicle target in the current frame image, storing the current frame image, recording first non-motor vehicle rectangular frame information corresponding to each first non-motor vehicle target in the current frame image, and extracting corresponding first non-motor vehicle target features; Recording the current frame image as a first frame image, and recording all first non-motor vehicle targets in the current frame image as a current non-motor vehicle set; Detecting the next frame of image, and in response to the presence of a second non-motor vehicle target in the next frame of image, recording second non-motor vehicle rectangular frame information corresponding to each second non-motor vehicle target in the next frame of image, and extracting corresponding second non-motor vehicle target features, wherein the first non-motor vehicle rectangular frame information and the second non-motor vehicle rectangular frame information both include a non-motor vehicle coordinate position rectangular frame and a license plate position rectangular frame; Based on the first non-motor vehicle rectangular frame information, the first non-motor vehicle target feature, the second non-motor vehicle rectangular frame information and the second non-motor vehicle target feature, all the second non-motor vehicle targets are traversed and compared with all the first non-motor vehicle targets in terms of IOU overlap and non-motor vehicle feature similarity, and the comparison results are updated to the current non-motor vehicle set.

5. A non-motor vehicle license plate recognition method according to claim 4, characterized in that: Performing a traversal comparison of IOU overlap and non-motor vehicle feature similarity on all second non-motor vehicle targets and all first non-motor vehicle targets, and updating the comparison results to the current non-motor vehicle set, including: In response to the IOU overlap ratio of the first non-motor vehicle target and the second non-motor vehicle target in the current non-motor vehicle set exceeding a preset overlap threshold and the non-motor vehicle feature similarity exceeding a preset non-motor vehicle similarity threshold, it is determined that the two are the same target, and the rectangular frame area size of the non-motor vehicle coordinate position rectangular frame and / or license plate position rectangular frame of the two is judged, and the target with the larger rectangular frame is updated to the current non-motor vehicle set.

6. A non-motor vehicle license plate recognition method according to claim 5, characterized in that: The preset overlap threshold is 90%; the non-motor vehicle similarity threshold is 90%.

7. The method for recognizing a non-motor vehicle license plate according to claim 5, characterized in that: Also includes: If, when detecting the next frame of image, it is found that among all the detected second non-motor vehicle targets, there is a second non-motor vehicle target that is different from all the first non-motor vehicles in the current non-motor vehicle set, then the second non-motor vehicle is used as a new target and updated to the current non-motor vehicle set; If all first non-motor vehicle targets in the current non-motor vehicle set do not identify the same target when compared with all second non-motor vehicle targets detected in the next frame of image, then the image in the current non-motor vehicle set is determined to be the best captured image, and is removed from the current non-motor vehicle set after being stored. The above operation is repeated to record and store the best snapshot images of all non-motor vehicles passing through the video screen.

8. The method for recognizing a non-motor vehicle license plate according to claim 1, wherein: Based on the non-motor vehicle rectangular frame information in the best captured image, the non-motor vehicle license plate classification model is used to classify the non-motor vehicle license plate, and the license plate area is corrected according to the classification result to obtain a positive-view license plate image, including: According to the classification results, determine the RGB components corresponding to the color of the non-motor vehicle license plate; According to the RGB components, a binarization threshold is set, the license plate area is binarized, and a binary image of the license plate area is cropped; Using a rotating rectangular frame, matching the binary image area, determining the actual rectangular frame of the license plate, and cropping the actual license plate image according to the actual rectangular frame; The actual license plate image is mapped into an orthographic license plate image through rotation transformation and affine transformation.

9. The method for recognizing a non-motor vehicle license plate according to claim 1, wherein: Characters are extracted from the front-view license plate image, and non-motor vehicle license plate recognition and standardization verification are performed based on the non-motor vehicle license plate matching template library and the extracted characters, including: Adjusting the size of the front-view license plate image to be consistent with the corresponding license plate template in the non-motor vehicle license plate matching template library; Constructing a character position binary mask template according to the character positions marked in the non-motor vehicle license plate matching template library; The positive-view license plate image and the character position binary mask template are combined through image and operation to extract the position of each character and distinguish between fixed characters and non-fixed characters; For each extracted character, the character encoding format rules set in the non-motor vehicle license plate matching template library are used to limit the character set range of the corresponding position characters, thereby improving the accuracy of license plate recognition, and the characters are identified and compared through the character recognition algorithm, thereby realizing the recognition and standardization verification of non-motor vehicle license plates.

10. A non-motor vehicle license plate recognition system, characterized in that: include: A sample and template library establishment module is used to establish a non-motor vehicle license plate detection training sample library, a non-motor vehicle license plate classification training sample library, and a non-motor vehicle license plate matching template library based on all non-motor vehicle license plate types and corresponding images collected locally; wherein the non-motor vehicle license plate matching template library records the character position, character type, and character encoding format rules in the non-motor vehicle license plate image; A model training module is used to train a non-motor vehicle license plate target detection model based on the non-motor vehicle license plate detection training sample library; and to train a non-motor vehicle license plate classification model based on the non-motor vehicle license plate classification training sample library; The best snapshot effect image acquisition module is used to extract video frame images from the camera video stream and obtain the best snapshot effect image of the non-motor vehicle target based on the video frame images; A non-motor vehicle rectangular frame information acquisition module is used to acquire the non-motor vehicle rectangular frame information in the best captured effect image based on the non-motor vehicle license plate target detection model; A front-view license plate image acquisition module is used to classify non-motor vehicle license plates using the non-motor vehicle license plate classification model based on the non-motor vehicle rectangular frame information in the best captured image, and to correct the license plate area according to the classification result to obtain a front-view license plate image; The non-motor vehicle license plate recognition and standardization verification module is used to extract characters from the positive-view license plate image, and perform non-motor vehicle license plate recognition and standardization verification based on the non-motor vehicle license plate matching template library and the extracted characters.

Images