A vehicle information collection and management method and system

Abstract

The present invention provides a method and system for collecting and managing vehicle information, relating to the technical field of vehicle management. The method includes: acquiring a captured image of a monitored vehicle; obtaining vehicle features of the monitored vehicle according to the captured image; obtaining a license plate contour of the monitored vehicle through an edge detection algorithm according to the vehicle features; processing the license plate contour through morphological operations to obtain an enhanced image of the license plate contour according to the license plate contour; classifying the enhanced image of the license plate contour into an unoccluded area and an occluded area through a deep learning model according to the enhanced image of the license plate contour; repairing according to the unoccluded area through context information to obtain an occluded image of the occluded area; obtaining a license plate number of the monitored vehicle according to the image of the unoccluded area and the occluded image; and entering the license plate number and vehicle model of the monitored vehicle. The present invention realizes the rapid and accurate collection of vehicle information by combining advanced image processing technology and deep learning algorithms.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle management, and more particularly, to a method and system for collecting and managing vehicle information. Background Art

[0002] Vehicle information plays an important role in vehicle monitoring and management. Obtaining vehicle information, such as license plate numbers and vehicle models, helps vehicle management departments better manage vehicles, so as to record and manage the technical conditions, insurance information, and drivers of vehicles, thereby improving traffic safety and preventing traffic accidents.

[0003] Currently, traditional vehicle information collection methods mainly rely on manual recognition and recording, but the efficiency of manual operations is relatively low. Especially in areas with heavy traffic, it is difficult to quickly and accurately record the information of each vehicle. In addition, when the vehicle license plate is deliberately blocked or the image is unclear due to dirt, insufficient light, camera angle, etc., the difficulty of manual recognition further increases, and the error rate also rises. Summary of the Invention

[0004] The problem solved by the present invention is how to improve the efficiency and accuracy of vehicle information collection.

[0005] To solve the above problems, the present invention provides a method for collecting and managing vehicle information, including:

[0006] Obtaining a captured image of a monitored vehicle;

[0007] Based on the captured image, obtaining vehicle features of the monitored vehicle, where the vehicle features include the vehicle model, length-width ratio, and color distribution of the monitored vehicle;

[0008] Based on the length-width ratio and the color distribution, obtaining the license plate contour of the monitored vehicle through an edge detection algorithm;

[0009] Based on the license plate contour, processing the license plate contour through morphological operations to obtain an enhanced image of the license plate contour;

[0010] Based on the enhanced image of the license plate contour, classifying the enhanced image of the license plate contour into an unobstructed area and an obstructed area through a deep learning model;

[0011] Repairing based on the unobstructed area through context information to obtain an obstructed image of the obstructed area;

[0012] Based on the image of the unobstructed area and the obstructed image, obtaining the license plate number of the monitored vehicle;

[0013] Entering the license plate number and the vehicle model of the monitored vehicle.

[0014] Optionally, obtaining the vehicle features of the monitored vehicle according to the acquired image includes:

[0015] Preprocessing the acquired image to obtain the preprocessed acquired image;

[0016] Through image processing technology, extracting features from the preprocessed acquired image to obtain the contours of the monitored vehicle, the contours of the windows, the contours of the headlights, and the contours of the wheels;

[0017] Through a graph convolutional network, obtaining vehicle features according to the contours of the monitored vehicle, the contours of the windows, the contours of the headlights, and the contours of the wheels;

[0018] Classifying the vehicle features according to vehicle management data to obtain the vehicle model corresponding to the monitored vehicle.

[0019] Optionally, obtaining the license plate contour of the monitored vehicle through an edge detection algorithm according to the length-width ratio and the color distribution includes:

[0020] Performing binarization processing on the acquired image to obtain the binarized image of the acquired image;

[0021] According to the binarized image, obtaining multiple geometric features of the monitored vehicle through the findcontours function;

[0022] Obtaining the length-width ratio of each geometric feature;

[0023] Screening the geometric features according to the length-width ratio, the length-width ratio of the geometric features, and the color distribution to obtain the license plate contour.

[0024] Optionally, screening the geometric features according to the length-width ratio, the length-width ratio of the geometric features, and the color distribution to obtain the license plate contour includes:

[0025] According to the color distribution, obtaining the color corresponding to the geometric feature in the acquired image;

[0026] Screening the geometric features according to the length-width ratio, a preset license plate length-width ratio, and a preset color;

[0027] Wherein, when the difference between the length-width ratio of the geometric feature and the preset license plate length-width ratio is less than a preset threshold, and at the same time the color difference between the color and the preset color is less than a preset color difference threshold, it is determined that the contour of the geometric feature is the license plate contour.

[0028] Optionally, processing the license plate contour through morphological operations according to the license plate contour to obtain an enhanced image of the license plate contour includes:

[0029] Segmenting the area where the license plate contour is located in the binary image to obtain a binary license plate image;

[0030] Performing a dilation operation on the binary license plate image to obtain a dilated binary image;

[0031] Performing an erosion operation on the dilated binary image to obtain an eroded binary image, and using the eroded binary image as the enhanced image of the license plate contour.

[0032] Optionally, classifying the enhanced image of the license plate contour into an unoccluded region and an occluded region through a deep learning model, including:

[0033] Inputting the enhanced image into the deep learning model, and obtaining local features of the enhanced image through the convolutional layer of the deep learning model;

[0034] Inputting the local features into the pooling layer of the deep learning model to reduce the spatial dimension of the local features, and then obtaining a feature representation of the enhanced image through the activation function layer;

[0035] Optimizing the feature representation through an intersection over union loss function to obtain the optimized feature representation;

[0036] Performing pixel classification on the enhanced image according to the optimized feature representation to obtain the unoccluded region and the occluded region.

[0037] Optionally, obtaining context information of the unoccluded region, where the context information includes image information of a preset surrounding area of the unoccluded region;

[0038] Obtaining the overall structural feature of the license plate contour and the feature of the unoccluded region according to the image information of the preset surrounding area of the unoccluded region;

[0039] Inputting the overall structural feature and the feature of the unoccluded region into an autoencoder to generate an estimated image of the occluded region;

[0040] Stitching the image of the unoccluded region and the estimated image, and determining whether the stitched image is usable;

[0041] If so, using the estimated image as the occluded image of the occluded region.

[0042] Optionally, obtaining the license plate number of the monitored vehicle based on the image of the non-occluded area and the occluded image includes:

[0043] Stitching the image of the non-occluded area and the occluded image to obtain a clear image corresponding to the license plate contour;

[0044] Performing character segmentation on the clear image to obtain the geometric features of each character in the clear image;

[0045] Identifying each character based on the geometric features to obtain the license plate number of the monitored vehicle.

[0046] Optionally, entering the license plate number and the vehicle model of the monitored vehicle includes:

[0047] Associating the license plate number and the vehicle model to obtain a comprehensive data record of the monitored vehicle;

[0048] Automatically generating a text file corresponding to the comprehensive data record through an automated script, and then entering the text file into the database.

[0049] The present invention also provides a vehicle information collection and management system, including:

[0050] An acquisition unit for acquiring an acquisition image of a monitored vehicle;

[0051] A feature extraction unit for obtaining vehicle features of the monitored vehicle according to the acquisition image, where the vehicle features include the vehicle model, length-width ratio, and color distribution of the monitored vehicle;

[0052] A contour extraction unit for obtaining the license plate contour of the monitored vehicle through an edge detection algorithm according to the length-width ratio and the color distribution;

[0053] An image processing unit for processing the license plate contour through morphological operations according to the license plate contour to obtain an enhanced image of the license plate contour;

[0054] Dividing the enhanced image of the license plate contour into a non-occluded area and an occluded area through a deep learning model according to the enhanced image of the license plate contour;

[0055] An image restoration unit for restoring according to the non-occluded area through context information to obtain an occluded image of the occluded area;

[0056] An identification unit for obtaining the license plate number of the monitored vehicle according to the image of the non-occluded area and the occluded image;

[0057] A data import unit for entering the license plate number and vehicle model of the monitored vehicle.

[0058] In the vehicle information collection and management method and system of the present invention, by collecting images of monitored vehicles, the system can quickly capture the real-time status of the vehicles. Then, image processing technology is used to extract vehicle features, including vehicle model, length-width ratio, and color distribution, which provide basic information for subsequent license plate recognition. Through the edge detection algorithm, the system can accurately identify the outline of the license plate, and morphological operations further enhance the clarity of the license plate outline, providing high-quality input for the recognition of the deep learning model. The application of the deep learning model is the key to improving efficiency and accuracy, which can distinguish between unobstructed areas and obstructed areas in license plate images, which is particularly important in practical applications because license plates may be blocked by other objects, affecting the accuracy of recognition. Using image restoration technology with context information to repair unobstructed areas can effectively restore the blocked part of the license plate, thereby improving the integrity and accuracy of license plate recognition. Finally, by combining unobstructed areas and repaired obstructed areas, the system can accurately obtain the license plate number and enter the license plate number and vehicle model information into the database together. The system can adapt to different environmental and lighting conditions and maintain a high recognition accuracy even under complex or adverse conditions. It not only reduces manual intervention, reduces error rates, but also improves the speed of information processing, making the collection and management of vehicle information more efficient and accurate. By integrating and entering license plate numbers and other information, the system provides a comprehensive data basis for subsequent data analysis and decision-making, realizes rapid response to vehicle information updates, and provides instant data support for traffic management and vehicle scheduling. The present invention combines advanced image processing technology and deep learning algorithms to achieve rapid and accurate collection of vehicle information, provides strong technical support for traffic management and vehicle monitoring, and improves the efficiency and accuracy of vehicle information collection. Description of the Drawings

[0059] Figure 1 It is a flowchart of the vehicle information collection and management method in an embodiment of the present invention;

[0060] Figure 2 It is a schematic structural diagram of a vehicle information collection and management system in an embodiment of the present invention. Detailed Embodiments

[0061] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be given in conjunction with the accompanying drawings.

[0062] Combined with Figure 1 As shown, the present invention provides a vehicle information collection and management method, including:

[0063] Obtain the captured images of the monitored vehicle.

[0064] Specifically, the first step in vehicle information collection is image acquisition, which is usually accomplished by high-definition cameras, generally deployed at key monitoring points such as traffic intersections, parking lot entrances, or toll booths. These cameras can capture images of passing vehicles in real time, providing the raw data for subsequent vehicle feature analysis. The quality of the images directly affects the accuracy of subsequent processing. Therefore, the captured images need to have sufficient resolution and clarity.

[0065] Based on the captured images, obtain the vehicle features of the monitored vehicle, where the vehicle features include the vehicle model, length-width ratio, and color distribution of the monitored vehicle.

[0066] Specifically, after obtaining the vehicle images, the next step is to extract vehicle features. This step involves image processing and computer vision techniques, and algorithms are used to analyze the images to identify specific attributes of the vehicle, such as the vehicle model, length-width ratio, and color distribution. These features are crucial for vehicle identification and classification. For example, machine learning algorithms can be used to train models to identify the features of different vehicle models, or color clustering algorithms can be used to analyze the color distribution of vehicles.

[0067] Based on the length-width ratio and the color distribution, through an edge detection algorithm, obtain the license plate contour of the monitored vehicle.

[0068] Specifically, edge detection is a key technique in image processing for identifying edge information in images. In vehicle information collection, edge detection algorithms are used to identify the contour of the license plate. Commonly used edge detection algorithms include Sobel, Prewitt, Canny, etc. These algorithms determine the position of the edge by calculating the gradient change of the pixels in the image. For example, the Canny algorithm is widely used in license plate detection due to its high detection accuracy and robustness to noise.

[0069] Based on the enhanced image of the license plate contour, through a deep learning model, classify the enhanced image of the license plate contour into unobstructed regions and obstructed regions.

[0070] Specifically, morphological operations are another important class of techniques in image processing, which analyze and process based on the shape and structure of the image. In the processing of the license plate contour, morphological operations such as dilation and erosion are used to enhance the recognizability of the license plate. The dilation operation can fill small gaps between license plate characters, while the erosion operation can remove noise and unnecessary details on the license plate. These operations help improve the accuracy of license plate recognition.

[0071] Based on the enhanced image of the license plate contour, through a deep learning model, classify the enhanced image of the license plate contour into unobstructed regions and obstructed regions.

[0072] Specifically, deep learning technology has made revolutionary progress in the field of image recognition. In vehicle information collection, deep learning models are used to distinguish non-occluded regions and occluded regions in license plate images. This typically involves using a trained convolutional neural network to identify and classify different regions in the image. For example, a deep learning model may be trained to recognize the letters and numbers on a license plate while ignoring incomplete parts of the image due to occlusion or other reasons.

[0073] Repair the occluded region based on the context information according to the non-occluded region to obtain the occluded image of the occluded region.

[0074] Specifically, the image repair technology using context information aims to restore damaged or incomplete parts of an image. In license plate recognition, if a part of the license plate is occluded, the image repair technology can predict and fill in the missing part based on the surrounding context information. This method can improve the integrity and accuracy of license plate recognition. For example, if a certain character on the license plate is partially occluded, the visible characters around and the known license plate format can be used to infer the missing part.

[0075] Obtain the license plate number of the monitored vehicle based on the image of the non-occluded region and the occluded image.

[0076] Specifically, in the preferred embodiment of the present invention, the recognition of the license plate number generally relies on optical character recognition technology. Optical character recognition technology can convert characters in an image into an editable text format. In a license plate recognition system, optical character recognition technology is used to extract the license plate number from the processed license plate image. This typically involves using specific optical character recognition technology to recognize the characters in the image and convert them into a standard license plate number format.

[0077] Enter the license plate number and the vehicle model of the monitored vehicle.

[0078] Specifically, the last step is to enter the recognized license plate number and vehicle model information into the database. This step involves database management technology, including data storage, retrieval, and management. The entry of vehicle information requires not only accuracy but also an efficient data organization method for subsequent query and analysis. For example, vehicle information can be classified and stored according to time, location, or vehicle type for statistical analysis or traffic management decision-making.

[0079] The vehicle information collection and management method of the present invention can quickly capture the real-time state of a vehicle by collecting and monitoring the vehicle's images. Then, image processing techniques are used to extract vehicle features, including vehicle type, length-width ratio, and color distribution, which provide basic information for subsequent license plate recognition. Through the edge detection algorithm, the system can accurately identify the contour of the license plate, and morphological operations further enhance the clarity of the license plate contour, providing high-quality input for the recognition of the deep learning model. The application of the deep learning model is the key to improving efficiency and accuracy, which can distinguish between unobstructed and obstructed areas in the license plate image, which is particularly important in practical applications because the license plate may be blocked by other objects, affecting the accuracy of recognition. Using image restoration technology with context information to repair the unobstructed area can effectively restore the blocked part of the license plate, thereby improving the integrity and accuracy of license plate recognition. Finally, by combining the unobstructed area and the repaired obstructed area, the system can accurately obtain the license plate number and enter the license plate number and vehicle type information into the database together. The system can adapt to different environments and lighting conditions and maintain a high recognition accuracy even under complex or adverse conditions. It not only reduces manual intervention, lowers the error rate, but also improves the speed of information processing, making the collection and management of vehicle information more efficient and accurate. By integrating and entering information such as the license plate number and vehicle type, the system provides a comprehensive data basis for subsequent data analysis and decision-making, realizes rapid response to vehicle information updates, and provides instant data support for traffic management and vehicle scheduling. The present invention combines advanced image processing techniques and deep learning algorithms to achieve rapid and accurate collection of vehicle information, provides strong technical support for traffic management and vehicle monitoring, and improves the efficiency and accuracy of vehicle information collection.

[0080] Optionally, obtaining the vehicle features of the monitored vehicle according to the collected image includes:

[0081] Preprocessing the collected image to obtain the preprocessed collected image;

[0082] Through image processing techniques, extracting features from the preprocessed collected image to obtain the contour of the monitored vehicle, the contour of the window, the contour of the headlight, and the contour of the wheel;

[0083] Through a graph convolutional network, obtaining vehicle features according to the contour of the monitored vehicle, the contour of the window, the contour of the headlight, and the contour of the wheel;

[0084] Classifying the vehicle features according to vehicle management data to obtain the vehicle type corresponding to the monitored vehicle.

[0085] Specifically, first, the process of extracting vehicle features begins with the preprocessing of the collected images. This step is crucial as it can improve the accuracy and efficiency of subsequent processing. Preprocessing usually includes operations such as denoising, image enhancement, and histogram equalization, aiming to remove noise in the images and enhance the recognizability of vehicle features. For example, a median filter can be used to remove salt-and-pepper noise, while histogram equalization can improve the contrast of the image, making features such as the vehicle's contour, windows, headlights, and wheels more prominent. Next, feature extraction is performed on the preprocessed images through image processing techniques. This step can employ edge detection algorithms, such as the Canny operator, to identify the vehicle's contour; use the Hough transform to detect circular components of the vehicle, such as headlights and wheels; and utilize color segmentation techniques to identify the window areas. The key to feature extraction lies in being able to accurately identify and locate each component of the vehicle, providing reliable data for subsequent feature classification. Finally, the application of the graph convolutional network is based on these extracted features. The graph convolutional network is a deep learning model capable of processing graph-structured data and is very suitable for processing interrelated features, such as different parts of a vehicle. By aggregating the feature information of each vehicle part and combining the topological relationships in the graph structure, the graph convolutional network can learn a more rich and robust vehicle feature representation. For example, the graph convolutional network can effectively integrate the features of the vehicle's contour, windows, headlights, and wheels, thereby achieving accurate classification of the vehicle. In a preferred embodiment of the present invention, first, the collected vehicle images are preprocessed, and wavelet transform is used to remove noise and retain edge information. Then, edge detection is performed through the Canny operator to obtain the vehicle's contour; the headlights and wheels are identified through the Hough transform; and the window areas are identified through color space conversion and clustering analysis. These features are then input into a well-trained graph convolutional network model that has been learned on a large number of vehicle images and can identify and classify different types of vehicle models.

[0086] In this embodiment, the efficiency of vehicle information processing is greatly improved through automated feature extraction and classification, reducing the need for manual intervention. Secondly, image preprocessing and advanced image processing techniques improve the accuracy of feature extraction, thereby enhancing the accuracy of vehicle model recognition. The reduction of manual intervention reduces errors caused by human factors and simultaneously improves the processing speed and efficiency. Through continuous learning and optimization of the deep learning model, the system can adapt to various complex environments and conditions and maintain a high-accuracy recognition ability. The present invention not only improves the intelligent level of vehicle information management but also provides strong technical support for fields such as traffic monitoring, vehicle scheduling, and safety supervision.

[0087] Optionally, obtaining the license plate contour of the monitored vehicle according to the length-width ratio and the color distribution through an edge detection algorithm includes:

[0088] Perform binarization processing on the acquired image to obtain the binarized image of the acquired image;

[0089] According to the binarized image, use the findcontours function to obtain multiple geometric features of the monitored vehicle;

[0090] Obtain the aspect ratio of each of the geometric features;

[0091] According to the length-width ratio, the aspect ratio of the geometric feature, and the color distribution, screen the geometric features to obtain the license plate contour.

[0092] Specifically, first perform binarization processing on the acquired image. This is to convert the image into an image with only black and white pixel values, where white represents the foreground (regions of interest such as vehicles or license plates), and black represents the background. Binarization can be achieved by setting a threshold. Pixel points above the threshold are regarded as white, and pixel points below the threshold are regarded as black. This step is crucial for subsequent edge detection and feature extraction because it simplifies the image and makes it easier for the algorithm to identify regions of interest. Secondly, use the findcontours function to perform contour detection on the binarized image. This function can identify all white regions in the image and extract their boundaries (contours). Each contour can be represented by a series of points, and these points define the shape of the object. In this process, multiple geometric features can be obtained, such as the area, perimeter, center point, etc. of each contour. Finally, by analyzing these geometric features, especially their aspect ratios, regions that may be license plates can be screened. The aspect ratio is an important feature because the aspect ratio of a license plate is usually within a certain range. Combining the color distribution information, screen the detected contours. For example, a reasonable aspect ratio range can be set, and only contours that meet this range are considered potential license plates to further confirm the position of the license plate. For example, if the aspect ratio of a geometric feature conforms to the standard ratio of a license plate and the color distribution matches the typical color of a license plate, then this feature is very likely to represent a license plate. In a preferred embodiment of the present invention, first perform binarization processing on the acquired vehicle image. The Otsu method can be used to automatically determine the threshold to adapt to images under different lighting conditions. Then, apply the findcontours function in the OpenCV library to extract the contours of all white regions. By calculating the aspect ratio of each contour, screen out the contours with an aspect ratio between 1.5 and 2.5. This range usually conforms to the aspect ratio of a license plate. In addition, color histogram analysis can also be used for further screening to exclude those contours with color distributions that do not match the license plate color.

[0093] In this embodiment, the image information is greatly simplified by screening the aspect ratio, analyzing the color distribution, and performing binarization processing on the image. The complex image is converted into a black-and-white image that is easier to analyze through thresholding operation. This step can accurately locate all connected white regions in the image, providing rich geometric information for license plate extraction, further improving the accuracy of license plate detection, and reducing the possibility of false detection.

[0094] Optionally, the step of screening the geometric features according to the aspect ratio, the aspect ratio of the geometric features, and the color distribution to obtain the license plate contour includes:

[0095] Obtaining the color corresponding to the geometric feature in the acquired image according to the color distribution;

[0096] Screening the geometric features according to the aspect ratio, the preset license plate aspect ratio, and the preset color;

[0097] Wherein, when the difference between the aspect ratio of the geometric feature and the preset license plate aspect ratio is less than a preset threshold, and at the same time the color difference between the color and the preset color is less than a preset color difference threshold, it is determined that the contour of the geometric feature is the license plate contour.

[0098] Specifically, first, the system analyzes the color of each geometric feature in the acquired image. This can be done by converting each geometric feature in the image to an appropriate color space (such as RGB) to more accurately represent the color and make comparisons. Then, the system screens these geometric features according to the preset license plate aspect ratio and color. The preset aspect ratio is based on the standardized size of the license plate, and the preset color is the typical color of the license plate, such as white or yellow. During the screening process, the system compares the aspect ratio of each geometric feature with the preset license plate aspect ratio. If this difference is less than a preset threshold, it indicates that the geometric feature may be a license plate. At the same time, the system also calculates the color difference between the geometric feature color and the preset color. The calculation of the color difference can be achieved by various methods, such as using the Euclidean distance to measure the proximity of two colors in the color space. If the color difference is also less than the preset color difference threshold, this further increases the possibility that the geometric feature is a license plate. In a preferred embodiment of the present invention, assume that the preset license plate aspect ratio is 3:1, the preset color is white, and the color difference threshold is set to 30 (in the HSV color space). The system first processes the acquired image to extract all possible geometric features. Then, the system calculates the aspect ratio of each geometric feature and compares it with the preset aspect ratio. For geometric features with an aspect ratio between 2.7 and 3.3, the system further analyzes their colors. If the color of the geometric feature is within 30 units of the white color range, the system marks it as a potential license plate. In this way, the system can effectively identify license plates from images.

[0099] In this embodiment, by considering both the aspect ratio and color dimensions simultaneously, the system can more accurately screen out geometric features that conform to license plate characteristics, reducing interference from other non-license plate objects. At the same time, the preset aspect ratio and color thresholds can be adjusted according to the standards of different regions or different types of license plates, enabling the system to adapt to various license plate specifications and colors. By comprehensively considering the shape and color dimensions of geometric features, the possibility of misidentification is reduced, and the accuracy of license plate recognition is improved.

[0100] Optionally, the processing of the license plate contour through morphological operations according to the license plate contour to obtain an enhanced image of the license plate contour includes:

[0101] Segment the area where the license plate contour is located in the binary image to obtain a binary license plate image;

[0102] Perform a dilation operation on the binary license plate image to obtain a dilated binary image;

[0103] Perform an erosion operation on the dilated binary image to obtain an eroded binary image, and use the eroded binary image as the enhanced image of the license plate contour.

[0104] Specifically, in license plate recognition, morphological operations are used to enhance the contour of the license plate and improve the recognition accuracy. First, the collected image is binarized to obtain the binary contour of the license plate. Then, a dilation operation is used to increase the white pixels in the license plate area. This can fill small cracks or broken strokes that may exist inside the license plate, making the license plate area more complete. The dilation operation slides a predefined structuring element in the image and increases the pixel values according to certain rules (usually assigning the value of the foreground pixel to the background pixel covered by the structuring element), thereby achieving the "expansion" effect of the image. Next, an erosion operation is performed on the dilated image. The erosion operation is the opposite of dilation. It refines the contour of the license plate by reducing the white pixels at the edges of the image. The erosion operation uses the same structuring element, but this time assigns the value of the background pixel to the foreground pixel covered by the structuring element. If not all the foreground pixels under the structuring element are white, the pixel value at that position will be eroded to the background value. This helps to remove noise and burrs at the edges of the license plate, making the contour of the license plate clearer and more precise. In the preferred embodiment of the present invention, first, the collected vehicle image is binarized to obtain a clear license plate contour. Then, a 5x5 rectangular structuring element is defined, and the dilation operation is performed on the binary image. After the dilation operation, the erosion operation is performed on the image again using the same structuring element. Through these two consecutive morphological operations, a license plate image with a clearer contour and a more complete interior can be obtained.

[0105] In this embodiment, the dilation operation enhances the internal integrity of the license plate, fills in possible cracks, and by increasing the white pixels in the license plate area, fills in small cracks or broken strokes that may exist inside, making the license plate area more complete and reducing recognition errors caused by image noise or quality problems. The erosion operation removes the noise at the edges of the license plate, making the contour of the license plate clearer and reducing the possibility of misrecognition.

[0106] Optionally, the enhanced image according to the license plate contour is divided into an unoccluded area and an occluded area by a deep learning model, including:

[0107] Input the enhanced image into the deep learning model, and obtain the local features of the enhanced image through the convolutional layer of the deep learning model;

[0108] Input the local features into the pooling layer of the deep learning model to reduce the spatial dimension of the local features, and then through the activation function layer, obtain the feature representation of the enhanced image;

[0109] Optimize the feature representation through the intersection over union loss function to obtain the optimized feature representation;

[0110] According to the optimized feature representation, perform pixel classification on the enhanced image to obtain the unoccluded area and the occluded area.

[0111] Specifically, in the license plate recognition system, first input the enhanced license plate image into the deep learning model. The convolutional layer of the model is responsible for extracting the local features of the image. These convolutional layers identify patterns in the image, such as edges, textures, etc., through a series of learnable filters. For example, the first convolutional layer may focus on identifying simple edge features, while deeper convolutional layers may identify more complex shape and structure features. Subsequently, these local features are fed into the pooling layer, usually using max pooling or average pooling techniques, to reduce the spatial dimension of the features while increasing the invariance to image displacement. This step reduces the spatial size of the data, thereby reducing the number of parameters and the computational complexity. Next, through the activation function layer, introduce non-linearity so that the model can learn and simulate more complex function mappings. Finally, use the intersection over union loss function to optimize the feature representation of the model. IoU is a metric that measures the overlap between the predicted region and the ground truth region. By minimizing the IoU loss between the predicted box and the ground truth box, the position and shape of the license plate can be more accurately located. After optimization, the model can perform pixel-level classification on the enhanced image to distinguish the unoccluded area and the occluded area.

[0112] In this embodiment, through the hierarchical structure of the deep learning model, features can be gradually extracted from simple to complex. This not only improves the generalization ability of the model for different types of license plates, but also enables the model to better adapt to license plate recognition tasks under different environments and conditions. The combined use of the convolutional layer and the pooling layer significantly improves the feature extraction ability, enabling the model to capture richer and more abstract feature representations. Using the intersection over union loss function to optimize the model ensures a high degree of consistency between the predicted region output by the model and the true license plate region, thereby improving the recognition accuracy of the occluded region, reducing recognition errors caused by occlusion, and enhancing the accuracy and robustness of license plate recognition.

[0113] Optionally, obtain the context information of the non-occluded region, where the context information includes the image information of a preset region around the non-occluded region;

[0114] Based on the image information of the preset region around the non-occluded region, obtain the overall structural features of the license plate contour and the features of the non-occluded region;

[0115] Input the overall structural features and the features of the non-occluded region into an autoencoder to generate an estimated image of the occluded region;

[0116] Stitch the image of the non-occluded region with the estimated image, and determine whether the stitched image is usable;

[0117] If so, use the estimated image as the occluded image of the occluded region.

[0118] Specifically, first, the system analyzes the unoccluded region and extracts the overall structural features of the license plate, such as the edges, colors, and textures of the license plate. These features are then used to infer the possible appearance and attributes of the occluded region. Based on the image information of the unoccluded region and its surrounding preset regions, the overall structural features of the license plate contour and the features of the unoccluded region are extracted. An autoencoder is a powerful neural network that can learn an effective representation of data and is particularly suitable for image inpainting tasks. In the inpainting of the occluded region of the license plate, the encoder part of the autoencoder compresses the features of the unoccluded region into a low-dimensional representation, while the decoder part expands this representation back to the dimension of the original image to generate an estimated image of the occluded region. For example, an autoencoder can be designed where both the encoder and decoder use convolutional neural network layers to maintain the spatial hierarchical structure of the image. In practical applications, a pre-trained autoencoder model that has been trained on a large number of unoccluded license plate images can be adopted to learn the typical features of license plates. When an occluded license plate image needs to be inpainted, the unoccluded region is input into the autoencoder to generate an estimated image of the occluded region. Then, this estimated image is stitched with the image of the unoccluded region to form a complete license plate image. If the generated image is visually consistent with the unoccluded region and conforms to the typical structural features of the license plate, the estimated image is considered usable.

[0119] In this embodiment, by using context information and an autoencoder for inpainting the occluded region, the robustness of the license plate recognition system is improved, the integrity of the license plate is restored, and the license plate can be accurately recognized even in the case of partial occlusion, providing more complete and reliable visual information for subsequent license plate recognition. Due to the coherence of the structure and features, the accuracy of the license plate recognition system is improved for the inpainted image, especially when the occluded region is large or the occlusion features are not obvious.

[0120] Optionally, obtaining the license plate number of the monitored vehicle according to the image of the unoccluded region and the occluded image includes:

[0121] Stitch the image of the unoccluded region and the occluded image to obtain a clear image corresponding to the license plate contour;

[0122] Perform character segmentation according to the clear image to obtain the geometric features of each character in the clear image;

[0123] Identify each character according to the geometric features to obtain the license plate number of the monitored vehicle.

[0124] Specifically, stitching the non-occluded region and the occluded image is a crucial step in the license plate recognition process. First, the occluded region estimation image generated by the autoencoder is precisely aligned and fused with the actual non-occluded region image at the edges. This step needs to ensure that the stitched image is visually coherent, without obvious seams or unnatural changes, for subsequent character segmentation and recognition. Stitching techniques can utilize seamless stitching algorithms in image processing, such as stitching methods based on feature point matching, or use image fusion techniques, such as Poisson fusion, to smooth the transition region. Then, character segmentation is performed on the stitched clear image. This step usually involves binarization of the image to enhance the contrast between characters and the background, and then applying connected component analysis or edge detection algorithms to identify and separate each character. For example, the Otsu method can be used for automatic binarization, and then the characters can be located by finding the contours. The geometric features of each character, such as size, shape, and position, will be extracted to prepare for character recognition. In the character recognition stage, deep learning models such as convolutional neural networks can be used to recognize each character in the image. The convolutional neural network model has been pre-trained on a large amount of license plate character data and can recognize different character shapes and styles. The extracted geometric features are input into the convolutional neural network, and the model will output the predicted category of each character.

[0125] In this embodiment, the integrity of the license plate image is ensured by precisely stitching the non-occluded region and the occluded image, providing high-quality input for character segmentation and recognition. Secondly, the accuracy of character segmentation directly affects the final recognition result. By using advanced image processing techniques, the accuracy of segmentation can be improved, reducing the risk of misrecognition.

[0126] Optionally, the inputting of the license plate number and the vehicle model of the monitored vehicle includes:

[0127] Associating the license plate number and the vehicle model to obtain a comprehensive data record of the monitored vehicle;

[0128] Automatically generating a text file corresponding to the comprehensive data record through an automated script, and then inputting the text file into the database.

[0129] Specifically, first, a data structure is needed to store the license plate numbers and vehicle type information of each monitored vehicle. This data structure is usually a record containing multiple fields. For example, in a relational database, it can be a table that includes fields such as license plate number, vehicle type, timestamp, etc. Among them, the writing of automation scripts is an important means to achieve automated data entry. This script can be implemented using various programming languages. The core functions of the script include receiving the license plate number and vehicle type information as input, and then formatting this information into a specific text file format, such as CSV or JSON. In the preferred embodiment of the present invention, a Python script can use the built-in csv module to generate a CSV file, and each line in the file represents the comprehensive data record of a vehicle.

[0130] In this embodiment, the need for manual input is reduced through an automated process, significantly improving the efficiency and speed of data entry. At the same time, the possibility of human input errors is reduced, ensuring the accuracy and consistency of the data. The text files (such as CSV or JSON format) generated by the script are easy to integrate with other systems or applications, providing good flexibility and scalability. It not only improves the speed and quality of data processing, but also provides data support for intelligent transportation and vehicle management.

[0131] As Figure 2 shown, the present invention also provides a vehicle information collection and management system, including:

[0132] A collection unit for acquiring the captured images of the monitored vehicles;

[0133] A feature extraction unit for obtaining the vehicle features of the monitored vehicle according to the captured images, where the vehicle features include the vehicle type, length-width ratio, and color distribution of the monitored vehicle;

[0134] A contour extraction unit for obtaining the license plate contour of the monitored vehicle through an edge detection algorithm according to the length-width ratio and the color distribution;

[0135] An image processing unit for processing the license plate contour through morphological operations according to the license plate contour to obtain an enhanced image of the license plate contour;

[0136] According to the enhanced image of the license plate contour, dividing the enhanced image of the license plate contour into an unobstructed area and an obstructed area through a deep learning model;

[0137] An image repair unit for repairing according to the unobstructed area through context information to obtain the obstructed image of the obstructed area;

[0138] An identification unit for obtaining the license plate number of the monitored vehicle according to the image of the unobstructed area and the obstructed image;

[0139] A data import unit for entering the license plate number and the vehicle model of the monitored vehicle.

[0140] The vehicle information collection and management system of the present invention can quickly capture the real-time state of a vehicle by collecting an image of the monitored vehicle. Then, image processing technology is used to extract vehicle features, including the vehicle model, length-width ratio, and color distribution, which provide basic information for subsequent license plate recognition. Through the edge detection algorithm, the system can accurately identify the contour of the license plate, and morphological operations further enhance the clarity of the license plate contour, providing high-quality input for the recognition of the deep learning model. The application of the deep learning model is the key to improving efficiency and accuracy. It can distinguish between unobstructed and obstructed areas in the license plate image, which is particularly important in practical applications because the license plate may be blocked by other objects, affecting the accuracy of recognition. Using context information to repair the unobstructed area can effectively restore the blocked part of the license plate, thereby improving the integrity and accuracy of license plate recognition. Finally, by combining the unobstructed area and the repaired obstructed area, the system can accurately obtain the license plate number and enter the license plate number and vehicle model information into the database together. This not only reduces manual intervention and error rates but also improves the speed of information processing, making the collection and management of vehicle information more efficient and accurate. The present invention combines advanced image processing technology and deep learning algorithms to achieve rapid and accurate collection of vehicle information, providing strong technical support for traffic management and vehicle monitoring, and improving the efficiency and accuracy of vehicle information collection.

[0141] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0142] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0143] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. A vehicle information collection and management method, characterized in that: include: Acquire the collected image of the monitored vehicle; Obtaining vehicle features of the monitored vehicle according to the collected image, wherein the vehicle features include the model, length-to-width ratio, and color distribution of the monitored vehicle; According to the aspect ratio and the color distribution, the license plate outline of the monitored vehicle is obtained by an edge detection algorithm; According to the license plate contour, the license plate contour is processed by morphological operation to obtain an enhanced image of the license plate contour; According to the enhanced image of the license plate contour, dividing the enhanced image of the license plate contour into a non-occluded area and an occluded area through a deep learning model; The method includes: obtaining context information of the non-occluded area, wherein the context information includes image information of a preset area around the non-occluded area; Obtaining overall structural features of the license plate outline and features of the non-blocked area based on the image information of a preset area surrounding the non-blocked area; Inputting the overall structural features and the features of the non-occluded area into an autoencoder to generate an estimated image of the occluded area; splicing the image of the non-occluded area with the estimated image, and determining whether the spliced ​​image is usable; If yes, using the estimated image as the occluded image of the occluded area; Obtaining the license plate number of the monitored vehicle according to the image of the non-blocked area and the blocked image; Enter the license plate number and vehicle model of the monitored vehicle.

2. The vehicle information collection and management method according to claim 1, characterized in that: The step of obtaining the vehicle characteristics of the monitored vehicle according to the collected image includes: Preprocessing the acquired image to obtain the processed acquired image; By using image processing technology, feature extraction is performed on the processed collected image to obtain the outline of the monitored vehicle, the outline of the window, the outline of the headlight and the outline of the wheel; Obtaining vehicle features according to the outline of the monitored vehicle, the outline of the window, the outline of the headlight, and the outline of the wheel through a graph convolutional network; The vehicle characteristics are classified according to the vehicle management data to obtain the vehicle model corresponding to the monitored vehicle.

3. The vehicle information collection and management method according to claim 2, characterized in that: The step of obtaining the license plate outline of the monitored vehicle by an edge detection algorithm according to the aspect ratio and the color distribution includes: Performing binarization processing on the collected image to obtain a binarized image of the collected image; According to the binary image, a plurality of geometric features of the monitored vehicle are obtained by using a findcontours function; Obtaining the aspect ratio of each of the geometric features; According to the aspect ratio, the aspect ratio of the geometric feature and the color distribution, the geometric feature is screened to obtain the license plate outline.

4. The vehicle information collection and management method according to claim 3, characterized in that: The step of screening the geometric features according to the aspect ratio, the aspect ratio of the geometric features and the color distribution to obtain the license plate outline includes: According to the color distribution, obtaining the color corresponding to the geometric feature in the acquired image; The geometric features are screened according to the aspect ratio, the preset license plate aspect ratio and the preset color; Among them, when the difference between the aspect ratio of the geometric feature and the preset license plate aspect ratio is less than a preset threshold, and the color difference between the color and the preset color is less than a preset color difference threshold, the outline of the geometric feature is judged to be the license plate outline.

5. The vehicle information collection and management method according to claim 3, characterized in that: The method of processing the license plate contour by morphological operation to obtain an enhanced image of the license plate contour includes: Segmenting the area where the license plate contour is located in the binary image to obtain a license plate binary image; Performing an expansion operation on the license plate binary image to obtain an expanded binary image; An erosion operation is performed on the dilated binary image to obtain an eroded binary image, and the eroded binary image is used as the enhanced image of the license plate contour.

6. The vehicle information collection and management method according to claim 1, characterized in that: The method of dividing the enhanced image of the license plate contour into a non-occluded area and an occluded area according to the enhanced image of the license plate contour by a deep learning model includes: Inputting the enhanced image into the deep learning model, and obtaining local features of the enhanced image through the convolution layer of the deep learning model; Inputting the local features into the pooling layer of the deep learning model to reduce the spatial dimension of the local features, and then passing through the activation function layer to obtain the feature representation of the enhanced image; Optimizing the feature representation by using an intersection-over-union loss function to obtain an optimized feature representation; According to the feature representation completed by the optimization, pixel classification is performed on the enhanced image to obtain the non-occluded area and the occluded area.

7. The vehicle information collection and management method according to claim 2, characterized in that: The obtaining the license plate number of the monitored vehicle according to the image of the non-blocked area and the blocked image comprises: splicing the image of the non-blocked area and the blocked image to obtain a clear image corresponding to the license plate outline; Performing character segmentation according to the clear image to obtain geometric features of each character in the clear image; Each of the characters is identified according to the geometric features to obtain the license plate number of the monitored vehicle.

8. The vehicle information collection and management method according to claim 2, characterized in that: The step of inputting the license plate number and the vehicle model of the monitored vehicle includes: Associating the license plate number with the vehicle model to obtain a comprehensive data record of the monitored vehicle; A text file corresponding to the comprehensive data record is automatically generated through an automated script, and then the text file is entered into a database.

9. A vehicle information collection and management system, characterized in that: include: A collection unit, used to obtain a collection image of the monitored vehicle; A feature extraction unit, configured to obtain vehicle features of the monitored vehicle according to the collected image, wherein the vehicle features include the model, length-to-width ratio and color distribution of the monitored vehicle; A contour extraction unit, used to obtain the license plate contour of the monitored vehicle through an edge detection algorithm according to the aspect ratio and the color distribution; An image processing unit, used for processing the license plate contour through morphological operations according to the license plate contour to obtain an enhanced image of the license plate contour; According to the enhanced image of the license plate contour, dividing the enhanced image of the license plate contour into a non-occluded area and an occluded area through a deep learning model; An image restoration unit is used to perform restoration according to the non-occluded area through context information to obtain an occluded image of the occluded area; specifically comprising: acquiring context information of the non-occluded area, wherein the context information includes image information of a preset area around the non-occluded area; Obtaining overall structural features of the license plate outline and features of the non-blocked area based on the image information of a preset area surrounding the non-blocked area; Inputting the overall structural features and the features of the non-occluded area into an autoencoder to generate an estimated image of the occluded area; splicing the image of the non-occluded area with the estimated image, and determining whether the spliced ​​image is usable; If yes, using the estimated image as the occluded image of the occluded area; A recognition unit, used for obtaining the license plate number of the monitored vehicle according to the image of the non-blocked area and the blocked image; A data import unit is used to input the license plate number and the vehicle model of the monitored vehicle.

Images