License plate snapshot matting method

Abstract

The invention relates to the field of license plate cutout processing, and discloses a license plate snapshot cutout method, which comprises the following steps of: 1, acquiring original data of a picture and inputting the original data into a license plate recognition algorithm; step 2, the license plate recognition algorithm carries out recognition processing on the input picture original data, an image matting result picture and license plate structure data are output, and the license plate structure data comprise initial coordinates of the image matting result picture and width and height data of the image matting result picture; 3, querying a resolution type list supported by the current chip, and screening the minimum adaptive resolution from the list; 4, calculating the display translation distance of the image matting result image according to the minimum adaptive resolution and the width and height data of the image matting result image, and enabling the image matting result image to be centrally displayed under the minimum adaptive resolution; the technical problem that license plate information cannot be completely displayed or a processed picture cannot effectively highlight the license plate information due to the fact that the fixed coding resolution is not matched with a license plate image is solved.

Description

Technical Field

[0001] The present invention relates to the field of license plate extraction processing, and specifically relates to a license plate capture and extraction method. Background Art

[0002] In the fields of traffic management, public safety, and logistics transportation, license plate recognition (ANPR, Automatic Number Plate Recognition) technology has become an indispensable intelligent technology. This technology can automatically extract and recognize license plate information from vehicle images, providing strong support for subsequent vehicle monitoring, tracking, and data analysis. Especially when integrating license plate recognition technology into MDVR (Mobile Digital Video Recorder), in-vehicle devices can achieve automatic collection and processing of vehicle information, greatly improving the accuracy and timeliness of information acquisition.

[0003] The core links of license plate recognition include license plate recognition, capture, and extraction processing. In practical applications, the capture and extraction processes of license plate images are often affected by various factors, resulting in inaccurate extraction of the license plate area. Due to the dynamics during vehicle driving, the position and shooting angle of the license plate often change significantly. This causes the position and size of the license plate in the image to be unstable, increasing the difficulty of license plate capture. The in-vehicle camera is fixedly installed, and when the vehicle is driving, the body posture may change due to factors such as road conditions and driving habits, thereby affecting the shooting effect of the license plate.

[0004] At the same time, due to limitations such as hardware, interface bandwidth, and image algorithm optimization of embedded device chips, the resolution of picture encoding is mostly fixed. This results in that under different shooting conditions, the resolution of the license plate image may not match the actual size of the license plate, thereby causing key information to be lost during extraction, such as partial images of the license plate number being cut off or the license plate not being able to display complete information. In order to ensure that the license plate is fully displayed, a larger resolution is selected for shooting. However, as shown in the appendix Figure 1 When the extracted image is small (i.e., the license plate occupies a small proportion in the image), this causes the overall picture to be large while the license plate is small, and the position of the license plate extraction is close to the upper left corner of the overall picture, unable to effectively highlight the license plate information. Summary of the Invention

[0005] The present invention aims to provide a license plate capture and extraction method to solve the technical problem that in the license plate recognition of in-vehicle devices in the prior art, the fixed encoding resolution does not match the license plate image, resulting in the inability to fully display license plate information or the processed picture being unable to effectively highlight the license plate information.

[0006] To achieve the above object, the present invention adopts the following technical solution: A license plate capture and extraction method, including:

[0007] Step 1: Obtain the original image data and input it into the license plate recognition algorithm for recognition and matting processing. The original image data includes the complete license plate information;

[0008] Step 2: The license plate recognition algorithm performs recognition processing on the input original image data and outputs a matting result image and license plate structure data. The license plate structure data includes the starting coordinates of the matting result image and the width and height data of the matting result image. The starting coordinates are the alignment coordinate points of the matting result image and the image coding resolution;

[0009] Step 3: Query the list of supported resolution types of the current chip and filter out the minimum adapted resolution from the resolution type list. The minimum adapted resolution is the minimum resolution whose width and height are both greater than the width and height of the matting result image;

[0010] Step 4: Calculate the display translation distance of the matting result image based on the minimum adapted resolution and the width and height data of the matting result image, and adjust the starting coordinates of the matting result image to center the display of the matting result image at the minimum adapted resolution;

[0011] Step 5: Use the adjusted starting coordinates of the matting result image and the minimum adapted resolution to generate a matting result display image through the chip coding layer.

[0012] The principle and advantages of this solution are as follows: This solution adapts to the coding capabilities of different in-vehicle device chips by dynamically adjusting the resolution of the image, thus solving the problem of mismatch between the fixed coding resolution and the license plate image; This dynamic adjustment method greatly improves the adaptability and flexibility of the system. By centering the display of the matting result image, the problem of omission or inability to prominently display license plate information during display is avoided. This helps to improve the accuracy and efficiency of license plate recognition, and also enhances the user experience. Selecting the minimum adapted resolution for image coding and display can reduce the consumption of processing resources while ensuring image quality. This is particularly important for in-vehicle devices with limited resources, which can extend the battery life of the device and improve the overall performance. This solution effectively improves the system adaptability, information readability, and resource utilization through the dynamic adjustment of the license plate image resolution and the centering display strategy, so as to effectively cope with mobile digital video recorders or other embedded devices with license plate recognition functions where the shooting angle and distance change greatly.

[0013] Preferably, as an improvement, in screening the minimum adapted resolution, the supported resolution types are arranged in ascending order and stored in an array, and the sorted resolution array is traversed for comparison and screening.

[0014] The beneficial effects of this improvement are as follows: Through sorting, the first resolution that meets the conditions (i.e., both the width and height are greater than the resolution of the keying result image) can be quickly found, avoiding the time-consuming operation of comparing all resolutions one by one. The sorted array makes the search process more orderly, reduces the complexity of the algorithm, and improves the stability and reliability of the program.

[0015] Preferably, as an improvement, in adjusting the starting coordinates of the keying result image, the horizontal display translation distance of the keying result image is half of the difference between the width value of the minimum adaptation resolution and the width value of the keying result image, and the vertical display translation distance of the keying result image is half of the difference between the height value of the minimum adaptation resolution and the height value of the keying result image. The starting coordinates of the keying result image are respectively subtracted by the corresponding horizontal display translation distance and vertical display translation distance.

[0016] The beneficial effects of this improvement are as follows: By calculating half of the difference as the translation distance, the centering position of the keying result image under the minimum adaptation resolution can be simply determined, reducing the complexity of the calculation. This method ensures that the keying result image can be accurately centered and displayed, and at the same time, the calculation process is fast and efficient, suitable for the on-vehicle device environment for real-time processing.

[0017] Preferably, as an improvement, the license plate structure data further includes keying information for guiding the storage location of the keying result image; the keying result display image is saved to the specified storage location according to the keying information corresponding to its keying result image.

[0018] The beneficial effects of this improvement are as follows: By guiding the storage location through the keying information, the data of the keying result image can be managed orderly, avoiding the problems of data chaos and loss. The clarity of the storage location makes subsequent data access and processing more convenient, improving the running efficiency of the program.

[0019] Preferably, as an improvement, the data format of the original image data is YUV.

[0020] The beneficial effects of this improvement are as follows: The YUV format is a commonly used color space in video processing, with good compatibility, facilitating interface docking with the video processing module of on-vehicle devices. The YUV format separates the luminance and chrominance information, enabling separate processing of luminance and chrominance during image processing and improving the processing efficiency.

[0021] Preferably, as an improvement, the resolution type list includes multiple resolution types.

[0022] The beneficial effects of this improvement are as follows: The support for multiple resolution types enables this solution to adapt to more types of on-vehicle devices, improving the versatility and application scope of the solution. According to different device performances and requirements, different resolutions can be selected for image processing and display, improving the flexibility and adaptability of the solution.

[0023] Preferably, as an improvement, the method is applicable to a mobile digital video recorder integrated with a license plate recognition algorithm or other embedded devices with license plate recognition functions.

[0024] The beneficial effects of this improvement are as follows: By applying this solution to these devices, the application fields and functional scopes can be expanded, and the competitiveness and market share of the devices can be improved. Integrating this solution with the license plate recognition algorithms of these devices can achieve seamless docking of functions and overall optimization of the system, improving the stability and performance of the system. Brief Description of the Drawings

[0025] Figure 1 Schematic diagram showing the matte extraction of the prior art for the embodiments of the present invention.

[0026] Figure 2 Schematic diagram showing the adaptation resolution screening for the embodiments of the present invention.

[0027] Figure 3 Schematic diagram showing the matte coordinate adjustment for the embodiments of the present invention. Detailed Embodiments

[0028] The following is a further detailed description through specific embodiments:

[0029] Embodiment

[0030] A license plate capture and matte extraction method includes:

[0031] Step 1: Obtain and input the original image data.

[0032] Obtain the original image data containing the license plate, with the data format being YUV, and input the original image data into the license plate recognition algorithm.

[0033] Among them, specifically, it includes:

[0034] S101. Use a deep learning model based on Faster R-CNN for license plate area detection.

[0035] aster R-CNN (Region-based Convolutional Neural Networks) is a deep learning model for object detection. It uses a convolutional neural network (CNN) to extract image features, generates candidate regions (ROIs) through a region proposal network (RPN), and then classifies and regresses the candidate regions to accurately detect the position and category of the target object.

[0036] License plate area detection process:

[0037] Image preprocessing: Convert the input YUV image to an RGB image, and perform size adjustment and normalization processing.

[0038] Feature extraction: Use a pre-trained convolutional neural network (such as VGG, ResNet, etc.) to extract image features.

[0039] Region Proposal Network (RPN): Generate multiple candidate regions (Anchor Boxes) on the feature map and determine whether these regions contain license plates.

[0040] ROIPooling: Map the candidate regions to the feature map and perform pooling operations to obtain feature vectors of a fixed size.

[0041] Classification and regression: Use a fully connected layer to classify the candidate regions (whether it is a license plate) and perform regression (exact position).

[0042] S102. Perform perspective transformation and affine transformation on the tilted or deformed license plate to improve the accuracy of character recognition.

[0043] Perspective transformation and affine transformation are used to correct the tilted or deformed license plate, making it horizontal or vertical, thereby improving the accuracy of character recognition.

[0044] Transformation process:

[0045] Detect the four corner points of the license plate: During the license plate detection stage, the four corner points of the license plate can be detected simultaneously.

[0046] Calculate the transformation matrix: According to the four detected corner points, calculate the transformation matrix for perspective transformation or affine transformation.

[0047] Apply the transformation: Apply the transformation matrix to the license plate area to obtain the corrected license plate image.

[0048] S103. Use the U-Net semantic segmentation network or a method based on connected component analysis to extract license plate characters.

[0049] U-Net is a convolutional neural network for image segmentation. Its structure is similar to an encoder-decoder, with symmetric contraction and expansion paths. U-Net can accurately segment the target area in the image and is suitable for license plate character segmentation.

[0050] U-Net license plate character segmentation process:

[0051] Input the corrected license plate image: Input the corrected license plate image into the U-Net network.

[0052] Feature extraction: In the contraction path, extract image features through convolution and pooling operations.

[0053] Feature upsampling: In the expansion path, the feature map is restored to the original image size through upsampling and convolution operations.

[0054] Segmentation result: Use the Sigmoid or Softmax function to perform binary classification on the feature map to obtain the character segmentation result.

[0055] The method based on connected component analysis uses the connectivity of pixels in the image to extract character regions. By traversing the pixels in the image, connected components are found and it is determined whether they are characters.

[0056] License plate character segmentation process based on connected component analysis:

[0057] Binarization processing: Binarize the license plate image to obtain a black and white image.

[0058] Connected component detection: Traverse the pixels in the black and white image to find all connected components.

[0059] Character screening: According to features such as the size and shape of the connected components, filter out possible character regions.

[0060] S104. Use CRNN (Convolutional Recurrent Neural Network) for character recognition and combine CTC (Connectionist Temporal Classification) for unsupervised training.

[0061] CRNN is a deep learning model for sequence recognition, which combines the advantages of Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). CRNN can directly extract features from images and perform sequence recognition, and is suitable for license plate character recognition. CTC is an algorithm for sequence label learning, which can handle the alignment problem between the input sequence and the output sequence. In license plate character recognition, CTC can be used to align and train the character sequence output by CRNN with the true label.

[0062] License plate character recognition process:

[0063] Input character segmentation result: Input the segmented character image into the CRNN network.

[0064] Feature extraction: Use the convolutional layer to extract features of the character image.

[0065] Sequence recognition: Use the recurrent layer (such as LSTM, GRU, etc.) to recognize the feature sequence.

[0066] Output character sequence: Use the fully connected layer and the Softmax function to output the probability distribution of the character sequence.

[0067] Define the loss function: Use the CTC loss function to calculate the error between the character sequence output by CRNN and the true label.

[0068] Backpropagation: Perform backpropagation according to the loss function to update the weights of the CRNN network.

[0069] Train the model: Use a large amount of training data to train the CRNN network until the model converges.

[0070] S105. Use license plate format rules (such as province codes, number rules) for error detection and correction.

[0071] Use license plate format rules (such as province codes, number rules) to perform error detection and correction on the recognition results. For example, it can be checked whether the characters in the recognition results conform to the rules of the license plate number (such as length, character type, etc.), and the non-conforming characters can be corrected.

[0072] Error detection and correction process:

[0073] Verify the recognition results: Compare the recognition results with the license plate format rules.

[0074] Error detection: Find out the characters or character combinations that do not conform to the rules.

[0075] Error correction: Correct the error characters according to the license plate format rules.

[0076] S106. Finally, send the recognition results and the original data to the host computer for subsequent processing.

[0077] Pack the data: Pack the recognition results (including license plate number, character segmentation results, etc.) and the original data (including YUV images, corrected license plate images, etc.).

[0078] Send the data: Send the packed data to the host computer through a communication protocol (such as TCP / IP, UDP, etc.).

[0079] Host computer processing: The host computer receives and processes the data for subsequent processing (such as storage, display, alarm, etc.).

[0080] Step 2: License plate recognition and data acquisition;

[0081] The license plate recognition algorithm performs recognition processing on the input original image data; when the license plate recognition is successful, the license plate recognition algorithm outputs and returns the keying result image, as well as the original image data and the license plate structure data in the keying result image. The license plate structure data includes the single-shot capture recognition result and the corresponding information of the recognized vehicle. The license plate structure data is used to guide the keying storage location and the content of the keying corresponding information.

[0082] The recognized vehicle corresponding information includes the coordinates of the keying result image, license plate information, vehicle ID, vehicle attributes, warning line status, alarm line status, safety island status, SafeZone monitoring status, the label number of the alarm segment corresponding to SafeZone, and the overall keying recognition result, etc. The single-shot capture recognition result includes the number of keying result images output from the original image data, the width and height of the keying result images, etc.

[0083] Specifically:

[0084]

[0085]

[0086] Step 3: Determine the resolution of the adapted chip;

[0087] As shown in the Figure 2 appendix, query the list of resolution types supported by the current chip, arrange the supported resolution types in ascending order and store them in an array. Traverse the sorted resolution array, and compare the width and height data of each resolution with the width and height data of the keying result image in turn, and screen out the smallest resolution whose width and height are both greater than the width and height of the keying result image as the minimum adapted resolution of the keying result image.

[0088] Step 4: Adjust the keying coordinates;

[0089] As shown in the Figure 3 appendix, calculate the display translation distance of the keying result image according to the width and height data of the minimum adapted resolution and the width and height data of the keying result image.

[0090] The horizontal display translation distance of the keying result image is the difference between the width value of the minimum adapted resolution and the width value of the keying result image, and divide this difference by two.

[0091] The vertical display translation distance of the keying result image is the difference between the height value of the minimum adapted resolution and the height value of the keying result image, and divide this difference by two.

[0092] In the initial state, the upper left corner of the keying result image is aligned with the upper left corner of the resolution; then subtract the corresponding horizontal display translation distance and vertical display translation distance from the coordinate points where the keying result image overlaps with the resolution, that is, the starting x and y coordinates, to obtain the keying coordinates after adapting to the current chip resolution, and the adjusted keying result image will be centered and displayed at the minimum adapted resolution.

[0093] Step 5: Generate and save the keying;

[0094] Using the coordinates of the adjusted keying result map and the filtered minimum adaptation resolution, generate a keying result display map through the chip coding layer. And save it to the specified storage location according to the license plate structure data corresponding to the keying result map.

[0095] The above are only embodiments of the present invention. Specific technical solutions and / or common knowledge such as characteristics well known in the art are not described in detail herein. It should be noted that for those skilled in the art, without departing from the technical solution of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope required by this application shall be subject to the content of its claims, and the specific implementation manners and the like recorded in the specification can be used to interpret the content of the claims.

Claims

1. A license plate capture and cutout method, characterized in that: include: Step 1: Obtain the original image data and input it into the license plate recognition algorithm for identification and image cutting processing. The original image data includes the complete information of the license plate; Step 2: The license plate recognition algorithm performs recognition processing on the input image raw data, and outputs the matting result image and the license plate structure data, wherein the license plate structure data includes the starting coordinates of the matting result image and the width and height data of the matting result image, and the starting coordinates are the alignment coordinate points of the matting result image and the image coding resolution; Step 3: query the list of resolution types supported by the current chip, and select the minimum adaptive resolution from the list of resolution types, where the minimum adaptive resolution is the minimum resolution whose width and height are both greater than the width and height of the matting result image; Step 4: Calculate the display translation distance of the matting result image according to the minimum adaptation resolution and the width and height data of the matting result image, and adjust the starting coordinates of the matting result image so that the matting result image is displayed in the center at the minimum adaptation resolution; Step 5: Use the adjusted starting coordinates and minimum adaptation resolution of the matting result image to generate a matting result display image through the chip encoding layer.

2. A license plate capture and cutout method according to claim 1, characterized in that: In filtering the minimum adaptive resolution, the supported resolution types are arranged in ascending order and stored in an array, and the sorted resolution array is traversed for comparison and screening.

3. A license plate capture and cutout method according to claim 2, characterized in that: When adjusting the starting coordinates of the matte result image, the horizontal display translation distance of the matte result image is half of the difference between the minimum adaptive resolution width value and the matte result image width value, and the vertical display translation distance of the matte result image is half of the difference between the minimum adaptive resolution height value and the matte result image height value. The starting coordinates of the matte result image are subtracted from the corresponding horizontal display translation distance and vertical display translation distance respectively.

4. A license plate capture and cutout method according to claim 3, characterized in that: The license plate structure data also includes cutout information for guiding the storage location of the cutout result image; the cutout result display image is saved to a designated storage location according to the cutout information corresponding to the cutout result image.

5. A license plate capture and cutout method according to claim 4, characterized in that: The data format of the original image data is YUV.

6. A license plate capture and cutout method according to claim 5, characterized in that: The resolution type list includes multiple resolution types.

7. A license plate capture and cutout method according to any one of claims 1 to 6, characterized in that: The method is applicable to a mobile digital video recorder integrated with a license plate recognition algorithm or other embedded devices with a license plate recognition function.

Images