Intelligent license plate recognition method and system based on image detection

Abstract

The invention relates to the technical field of license plate recognition, and discloses an intelligent license plate recognition method and system based on image detection. The method comprises the following steps: acquiring a vehicle identification image; performing contour extraction and breakpoint detection according to the pixel distribution characteristics of the vehicle identification image to obtain a damaged mask image; extracting neighborhood pixel information of missing characters according to the damaged mask image and the vehicle identification image to obtain a neighborhood feature vector; and inputting the neighborhood feature vector into a pre-trained deep learning model for character inference and fusion verification to obtain final complete vehicle identification information. The method can achieve the intelligent restoration of damaged license plate characters, and remarkably improves the recognition precision in a complex environment.

Description

Technical Field

[0001] This application relates to the field of license plate recognition technology, and in particular to a method and system for intelligent license plate recognition. Background Technology

[0002] Currently, license plate recognition technology, as a core component of intelligent transportation systems, plays an irreplaceable role in key areas such as urban traffic management, security monitoring, and automated vehicle management. With the deepening of smart city construction, the requirements for the accuracy and reliability of license plate recognition systems are becoming increasingly stringent.

[0003] In a current technology, license plate recognition solutions mainly rely on complete and clear image information for character extraction and recognition.

[0004] However, existing technologies suffer from insufficient adaptability. Due to their inability to effectively handle dynamic changes in image quality, existing technologies experience a sharp decline in recognition accuracy when processing license plate images under complex environmental conditions, especially performing poorly in scenarios such as inclement weather, changes in lighting, or physical occlusion.

[0005] A specific problem with existing technologies is that license plate images often exhibit damage during actual acquisition. The complexity and diversity of this damage pose a significant challenge to recognition systems. The irregular distribution of damaged areas makes it difficult for the system to accurately locate and assess the extent of missing information, while the spatial correlation and format regularity between license plate characters require the restoration process to maintain a high degree of consistency and accuracy. Therefore, how to achieve intelligent restoration and complete reconstruction of missing character information based on accurate identification of damaged areas on license plates has become a key factor restricting recognition effectiveness. Summary of the Invention

[0006] This application provides a license plate intelligent recognition method and system based on image detection to solve the problem that existing technologies cannot accurately recognize characters due to damage, dirt or partial missing parts of vehicle markings in complex environments, resulting in a sharp drop in recognition accuracy.

[0007] In a first aspect, to address the aforementioned technical problems, this application of the present invention provides a method for intelligent license plate recognition based on image detection, comprising: Vehicle identification images are acquired using image acquisition equipment, and the vehicle identification images are preprocessed to obtain image data for damage detection. An edge detection algorithm is used to extract the contours of the image data to obtain the boundary coordinate sequence data of the vehicle identifier; The discontinuity of the boundary contour curve is calculated based on the boundary coordinate sequence data. Combining the discontinuity distribution density and curvature change characteristics, the candidate region for damage is determined based on the discontinuity. Calculate the feature parameters of the candidate damaged region. If the feature parameters of the candidate damaged region are greater than the preset damage feature threshold, then use an adaptive region filling algorithm to determine the specific range of damage in the candidate damaged region, and obtain a damage mask image based on the specific range. The vehicle identification image is overlaid with the damaged mask image to extract the neighboring pixel information of the missing characters. The pixel information is then arranged in a fixed dimension to form a neighborhood feature vector. A convolutional neural network model is used to perform sequence inference on the neighborhood feature vectors, and a candidate character set is obtained by combining the license plate format pattern constraints. For the candidate character set and the preset national license plate coding rules (including the correspondence between province abbreviations and character position constraints), iterative matching is performed to obtain the optimal matching character combination. The optimal matching character combination is then used to fill the missing positions to obtain the repaired character sequence. The repaired character sequence is then subjected to multi-dimensional consistency fusion verification with the undamaged characters of the original image. If the verification passes, the final complete vehicle identification information is output.

[0008] Preferably, the step of acquiring vehicle identification images through an image acquisition device and preprocessing the vehicle identification images to obtain image data for damage detection includes: The vehicle's image is captured using an image acquisition device (such as a high-resolution industrial camera, with parameters set to 2048×1536 pixels, shutter speed 1 / 250 second, and aperture F8.0) to obtain the original image data of the vehicle's markings. The original image data is subjected to Gaussian filtering to remove noise interference, and the brightness and contrast are adjusted to obtain a standardized vehicle identification image; The standardized vehicle identification image is processed using an edge detection algorithm to obtain the contour features of the vehicle identification. A normal state reference template for vehicle identification is established using the aforementioned contour features, and the pixel distribution pattern of the normal state reference template is recorded. The pixel grayscale value distribution of the current vehicle identification image is compared and analyzed at the pixel level with the pixel distribution pattern recorded in the normal state reference template, and the Euclidean distance is calculated to obtain the pixel grayscale value difference. If the difference in pixel grayscale values ​​exceeds a preset damage detection threshold, it is preliminarily determined that there is damage, and image data for damage detection is obtained.

[0009] Preferably, the step of using an edge detection algorithm to extract contours from the image data to obtain boundary coordinate sequence data of the vehicle identifier includes: Based on the image data used for damage detection, the Canny edge detection algorithm is used to identify the locations of pixel gradient abrupt changes, and the pixels corresponding to the abrupt changes are marked as edge pixels to obtain a set of edge pixel points; Based on the connection relationship between adjacent pixels in the edge pixel set, the complete contour path is extracted from the image data of the damage detection using the connected component analysis method. The pixels on the complete contour path are traversed and their coordinates are mapped. The Douglas-Peucker algorithm is used to approximate the polygon, and the contour path is represented as a series of ordered pixel coordinate sets to obtain the boundary coordinate sequence data of the vehicle identifier.

[0010] Preferably, the step of calculating the discontinuity of the boundary contour curve based on the boundary coordinate sequence data, and determining the candidate damage region based on the discontinuity, includes: The adjacent boundary coordinates are obtained from the boundary coordinate sequence data, and the Euclidean distance between the adjacent boundary coordinates is calculated. If the Euclidean distance exceeds a preset discontinuity threshold, it is marked as a breakpoint. Based on the breakpoint, the location information of the potential damaged area is obtained. Based on the location information of the potential damaged area, the distribution characteristics of the breakpoints are statistically analyzed. If the distribution characteristics of the breakpoints meet the preset distribution characteristics, the location of the breakpoints is initially determined as a candidate damaged area.

[0011] Preferably, the step of calculating the feature parameters of the candidate damaged region, and if the feature parameters of the candidate damaged region are greater than a preset damage feature threshold, then using an adaptive region filling algorithm to determine the specific extent of damage in the candidate damaged region, and obtaining a damage mask image based on the specific extent, includes: If the breakpoint in the candidate damaged area is detected to meet the preset triggering condition, the edge tracking algorithm is used to identify the boundary position of the breakpoint and obtain the set of boundary points. Based on the connection relationship between adjacent points in the boundary point set, an adaptive region growth algorithm is used to divide the damaged area and determine the preliminary area range. For the initial region range, a smoothing filter operation is applied to smooth the region boundaries to obtain an optimized region shape; The optimized region shape is converted into a black and white mask format using binarization processing to obtain the initial mask image; If there are isolated noise pixels in the initial mask image, the noise is removed by median filtering to obtain a denoised mask image. For the denoised mask image, the integrity and connectivity of the damaged region are verified using connected component analysis, and the verification results are obtained. If the verification result meets the preset verification criteria, the denoised mask image is determined as the final damaged mask image.

[0012] Preferably, the step of overlaying the vehicle identification image with the damaged mask image, extracting the neighborhood pixel information of the missing characters, and arranging the pixel information according to a fixed dimension to form a neighborhood feature vector includes: Obtain the pixel matrix data of the damaged mask image and the vehicle identification image, and determine the coordinate range of the overlapping area of ​​the two images by pixel-by-pixel traversal operation; Based on the coordinate range of the overlapping area, pixel value difference operation is used to identify the boundary contour of the missing character and obtain the coordinate range of the missing character area; Based on the coordinate range of the missing character region, calculate the gradient direction and intensity value of the complete pixels around the missing character region, and construct an initial feature descriptor by combining it with the local binary pattern histogram. If the dimension of the initial feature descriptor exceeds a preset dimension threshold, then the principal component analysis algorithm is applied to reduce the dimension of the initial feature descriptor to obtain an optimized feature representation. The optimized feature representation is then standardized and normalized to obtain the final neighborhood feature vector.

[0013] Preferably, the step of using a convolutional neural network model to perform sequence inference on the neighborhood feature vectors to obtain a candidate character set includes: A convolutional neural network model is used to extract features from the neighborhood feature vector to obtain a high-dimensional feature representation. The high-dimensional feature representation is processed by a pre-trained sequence inference model, mapped to a character probability space, and the character category at the missing position is predicted. The prediction result is then determined as the initial character type of the missing character. If the confidence level of the initial character type is greater than or equal to the preset confidence threshold, then an initial candidate character set is generated based on the initial character type; If the confidence score of the initial character type is less than the preset confidence score threshold, the feature vector is adjusted through the character co-occurrence matrix, and pattern matching and inference are performed again. According to the preset national license plate coding rules, characters that conform to the structural constraints in the coding rules are selected from the initial candidate character set to obtain an optimized candidate character set; The conditional random field model is used to perform sequence labeling and probability modeling on the optimized candidate character set to obtain the probability distribution of the character set; The probability distribution is sorted and truncated to obtain the highest probability character sequences as the final candidate character set.

[0014] Preferably, the step of iteratively matching the candidate character set and the preset national license plate coding rules to obtain the optimal matching character combination, and using the optimal matching character combination to fill the missing positions to obtain the repaired character sequence, includes: Call the preset vehicle identification format template, and determine whether the inference result in the candidate character set meets the character type requirements of the corresponding position in the format template. If it does, it is determined to be a valid character sequence. The legal character sequences are exhaustively enumerated and permuted using an iterative matching algorithm to obtain a set of character sequences to be matched, and the similarity score between each combination in the set of character sequences to be matched and the standard format is calculated. The similarity score and the confidence score are weighted and fused together to determine the character sequence with the highest weighted score as the optimal matching sequence; The characters of the optimal matching sequence are replaced with the original missing positions determined based on the damaged mask image to obtain the repaired complete character sequence; If the repaired complete character sequence conforms to the preset national license plate coding standard, it is taken as the final repaired character sequence.

[0015] Preferably, the step of performing consistency fusion verification between the repaired character sequence and the undamaged characters of the original image, and outputting the final complete vehicle identification information if the verification passes, includes: Based on the repaired character sequence, standardized character outline information is obtained through pixel-level alignment processing; Spatial position matching is performed between the standardized character outline information and the remaining area of ​​the vehicle identification image. If the overlap between the character boundary and the boundary of the remaining area is greater than or equal to a preset threshold, the position matching is determined to be successful. A convolutional neural network is used to extract features from the character sequences that have successfully matched positions, generating a repaired character feature vector. The cosine similarity between the repaired character feature vector and the features of the remaining region is calculated to obtain a feature similarity score. Geometric consistency verification and illumination consistency analysis are performed on the character sequences that successfully match the positions, and geometric consistency scores and illumination consistency scores are obtained respectively. Based on the feature similarity score, geometric consistency score, illumination consistency score, and semantic consistency score obtained from comparison with the vehicle identification database, a weighted fusion algorithm is used to calculate the comprehensive confidence score. If the overall confidence score is greater than or equal to the preset verification threshold, the repaired character sequence is merged with the remaining area of ​​the vehicle identification image using an image fusion algorithm to output the final complete vehicle identification information.

[0016] Secondly, this application of the present invention provides a license plate intelligent recognition system based on image detection, comprising: The data acquisition and preprocessing module is used to acquire vehicle identification images through image acquisition equipment, and to preprocess the vehicle identification images (denoising, brightness and contrast adjustment, standard template comparison) to obtain image data for damage detection. The contour extraction module is used to extract contours from the image data using the Canny edge detection algorithm, and combined with polygon approximation processing to obtain the boundary coordinate sequence data of the vehicle identifier; The damage detection module calculates the discontinuity of the boundary contour curve based on the boundary coordinate sequence data, and determines the candidate damage area by combining the characteristics of the breakpoint distribution. The mask generation module calculates the feature parameters of the candidate damaged region. If the feature parameters of the candidate damaged region are greater than the preset damage feature threshold, the adaptive region filling algorithm is used to determine the specific range of damage in the candidate damaged region. After filtering, denoising and connected component verification, the damaged mask image is obtained. The neighborhood feature extraction module overlays the vehicle identification image with the damaged mask image to extract the neighborhood pixel information of the missing characters. After dimensionality optimization and standardization, a neighborhood feature vector is formed. The character inference module uses a convolutional neural network model to perform sequence inference on the neighborhood feature vectors, and combines license plate coding rules for screening and probability modeling to obtain a candidate character set. The character repair module, based on the candidate character set and the preset national license plate coding rules, obtains the optimal matching character combination through an iterative matching algorithm, and uses the optimal matching character combination to fill the missing positions to obtain the repaired character sequence; The fusion verification and output module is used to perform four-dimensional (feature, geometry, illumination, semantic) consistency fusion verification between the repaired character sequence and the undamaged characters of the original image. If the verification is successful, the final complete vehicle identification information is output.

[0017] Compared with the prior art, this application has the following beneficial effects: (1) This application generates a damaged mask by contour extraction and discontinuity point detection and applies a region filling method, which can accurately locate and define irregular damaged areas, providing a high-precision spatial positioning basis for character repair and solving the problem of inaccurate positioning of damaged areas by traditional methods.

[0018] (2) This application uses a deep learning model to perform pattern matching on the neighborhood feature vectors, and combines the preset template and rule base of the vehicle identification format rules to perform constraints and iterative comparison, thereby realizing intelligent inference and accurate reconstruction of missing characters, which significantly improves the robustness and accuracy of character recognition.

[0019] (3) This application uses a fusion verification module to comprehensively consider multiple dimensions such as feature similarity, geometric consistency, illumination consistency and semantic consistency, to ensure that the repair result is highly consistent with the original image, effectively solving the problem of recognition difficulties caused by dirt and occlusion, and significantly improving the recognition accuracy of vehicle markings. Attached Figure Description

[0020] Figure 1 This is a schematic flowchart of the image detection-based intelligent license plate recognition method provided in the first embodiment of this application; Figure 2 This is a schematic diagram of an image detection-based intelligent license plate recognition system provided in the second embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0022] Reference Figure 1 The first embodiment of this application provides a method for intelligent license plate recognition based on image detection, including the following steps: S11, The acquisition device acquires vehicle identification images, preprocesses the vehicle identification images, and obtains image data for damage detection; S12, the detection algorithm extracts the contours from the image data to obtain the boundary coordinate sequence data of the vehicle identifier; S13, calculate the discontinuity of the boundary contour curve based on the boundary coordinate sequence data, and determine the candidate region of damage based on the discontinuity; S14, calculate the feature parameters of the damaged candidate region. If the feature parameters of the damaged candidate region are greater than the preset damage feature threshold, then use the adaptive region filling algorithm to determine the specific range of damage in the damaged candidate region, and obtain the damage mask image based on the specific range. S15, The vehicle identification image is superimposed on the damaged mask image to extract the neighborhood pixel information of the missing characters, and the pixel information is arranged in a fixed dimension to form a neighborhood feature vector. S16, A convolutional neural network model is used to perform sequence inference on the neighborhood feature vectors to obtain a candidate character set; S17, For the candidate character set and the preset national license plate coding rules, perform iterative matching to obtain the optimal matching character combination, and use the optimal matching character combination to fill the missing positions to obtain the repaired character sequence; S18, perform consistency fusion verification between the repaired character sequence and the undamaged characters of the original image. If the verification passes, output the final complete vehicle identification information.

[0023] In step S11, a vehicle identification image is acquired using an image acquisition device, and the vehicle identification image is preprocessed to obtain image data for damage detection, including: The vehicle's image is captured using an image acquisition device to obtain the original image data of the vehicle's identification. The original image data is subjected to noise removal and brightness and contrast adjustment to obtain a standardized vehicle identification image; The standardized vehicle identification image is processed using an edge detection algorithm to obtain the contour features of the vehicle identification. A normal state reference template for vehicle identification is established using the aforementioned contour features, and the pixel distribution pattern of the normal state reference template is recorded. The pixel grayscale value distribution of the current vehicle identification image is compared and analyzed at the pixel level with the pixel distribution pattern recorded in the normal state reference template to obtain the pixel grayscale value difference value. If the difference in pixel grayscale values ​​exceeds a preset damage detection threshold, it is preliminarily determined that there is damage, and image data for damage detection is obtained.

[0024] It should be noted that the system first starts the high-resolution industrial camera and sets the shooting parameters as follows: Pixel resolution, shutter speed seconds, aperture value ISO sensitivity set to The camera uses an autofocus algorithm that determines the optimal focal length by calculating the maximum gradient value in the image. When the gradient value reaches a threshold... Focusing is complete at the above time.

[0025] After image acquisition, the system begins image standardization of the original RGB image. The specific steps are as follows: First, the image is denoised by applying a Gaussian filter, with the filter kernel size set to 5×5 and the standard deviation... Next, contour recognition is performed using the Canny edge detection algorithm to identify the contour boundaries of vehicle markings. During this process, dual threshold parameters are set: a low threshold of 50 and a high threshold of 150, with a gradient direction tolerance of ±15°. Following this, region filtering is conducted, performing connected component analysis on the detected edge regions to filter out connected regions with an area greater than 500 pixels, which are then considered valid marking regions. These filtered valid marking regions will serve as the foundational data source for establishing a normal-state reference template.

[0026] Based on the filtered valid identification areas, color features and texture features are extracted respectively to form the core content of the pixel distribution pattern of the normal state reference template. Regarding color features, the mean values ​​of the RGB three channels are as follows: The standard deviations are respectively Regarding texture features, the gray-level co-occurrence matrix formula was used to calculate relevant parameters, yielding a contrast value of 0.342, a correlation value of 0.678, an energy value of 0.156, and a uniformity value of 0.234. These feature parameters will serve as the baseline data for the damage detection algorithm in step S13.

[0027] The contrast formula in the gray-level co-occurrence matrix uses: in: It represents the number of gray levels in the image; These are elements in the gray-level co-occurrence matrix, representing gray values. and The probability of appearing at a specific distance and direction.

[0028] The formulas for the mean and correlation in the gray-level co-occurrence matrix are as follows: in for exist The mean of the direction, The standard deviation is denoted as .

[0029] The energy value formula for the gray-level co-occurrence matrix is ​​as follows: in These are elements in the gray-level co-occurrence matrix, representing gray values. and The probability of appearing at a specific distance and direction.

[0030] The formula for the uniformity of the gray-level co-occurrence matrix is ​​as follows: in It is the grayscale difference between adjacent pixels, which measures the intensity of grayscale contrast.

[0031] In step S12, an edge detection algorithm is used to extract the contours of the image data to obtain the boundary coordinate sequence data of the vehicle identifier, including: Based on the image data used for damage detection, the Canny edge detection algorithm is used to identify the locations of pixel gradient abrupt changes, and the pixels corresponding to the abrupt changes are marked as edge pixels to obtain a set of edge pixel points; Based on the connection relationship between adjacent pixels in the edge pixel set, the complete contour path is extracted from the image data of the damage detection using the connected component analysis method. The pixels on the complete contour path are traversed and their coordinates mapped to represent the contour path as a series of ordered pixel coordinate sets, thereby obtaining the boundary coordinate sequence data of the vehicle identifier.

[0032] For example, based on the image data used for damage detection, the system first applies Gaussian smoothing to remove noise, then uses the Canny edge detection algorithm to identify locations of abrupt changes in pixel gradients, obtaining a set of edge pixels. The specific steps are as follows: the algorithm calculates the gradient magnitude and direction of the image and uses a non-maximum suppression algorithm to retain local maximum gradient points. Finally, it performs dual thresholding (low threshold...) High threshold ), with gradients higher than Points marked as strong edge points, with gradients between and Points between strong edge points are marked as weak edge points, and weak edge points are connected according to strong edge points to obtain a set of edge pixels with pseudo-edges removed.

[0033] The system then starts from a strong edge point and searches for its neighboring edge pixels along an 8-connected region until it returns to the starting point or the search path is interrupted, thus connecting discrete edge pixels into one or more complete contour paths. For the vehicle identifier, only the largest and closed contour path is retained, forming the final boundary coordinate sequence data. The final boundary coordinate sequence data is based on... Stored in the form of , where It represents the number of pixels on the contour, providing ordered geometric input for subsequent geometric analysis.

[0034] In step S13, the discontinuity of the boundary contour curve is calculated based on the boundary coordinate sequence data, and candidate damage regions are determined based on the discontinuity, including: The adjacent boundary coordinates are obtained from the boundary coordinate sequence data, and the Euclidean distance between the adjacent boundary coordinates is calculated. If the Euclidean distance exceeds a preset discontinuity threshold, it is marked as a breakpoint. Based on the breakpoint, the location information of the potential damaged area is obtained. Based on the location information of the potential damaged area, the distribution characteristics of the breakpoints are statistically analyzed. If the distribution characteristics of the breakpoints meet the preset distribution characteristics, the location of the breakpoints is initially determined as a candidate damaged area.

[0035] It should be noted that the core feature of this step is the boundary discontinuity point, which is manifested as a sudden increase in the Euclidean distance between adjacent points in the boundary coordinate sequence, directly indicating the interruption position of the contour path.

[0036] The system obtains the boundary coordinate sequence data In the process, adjacent coordinate point pairs are obtained sequentially. and First, calculate the Euclidean distance between adjacent points. The calculation With the preset discontinuity threshold Compare. If Then, the positions of the adjacent point pair are initially marked as "breakpoints".

[0037] Then when three consecutive points are detected The Euclidean distance between them is greater than or equal to If the angle between adjacent point pairs changes by more than 45°, then these points are marked as suspected breakpoint clusters.

[0038] For the aforementioned cluster of suspected breakpoints, an arc is fitted using three consecutive coordinate points, and the radius of curvature of the arc is calculated. When the radius of curvature is less than 8 pixels or the rate of change of curvature exceeds 0.3, the potential damaged area can be identified. Then, the index range of the suspected breakpoint cluster in the boundary coordinate sequence and the location information of the corresponding set of coordinate points, i.e., the potential damaged area, can be obtained.

[0039] Based on the location information of the potential damaged areas, the system statistically analyzes the distribution characteristics of potential breakpoints. The specific steps are as follows: calculate the number of breakpoints within a unit pixel range, determine whether the breakpoints are continuously distributed (i.e., five or more consecutive adjacent point pairs are breakpoints, forming a breakpoint cluster), and finally calculate the area of ​​the bounding rectangle of the breakpoint cluster.

[0040] To effectively filter out minor discontinuities caused by image noise, it is determined whether the area of ​​the bounding rectangle of the breakpoint cluster is greater than or equal to the preset minimum break area. If greater than or equal to Then the breakpoint cluster is determined to be a real candidate region for damage.

[0041] The system defines a valid damaged area as one whose area exceeds 100 square pixels and whose circularity is less than 0.7, conforming to the damage pattern. Therefore, this will be... The area is set to 100 square pixels. In step S14, the feature parameters of the candidate damaged region are calculated. If the feature parameters of the candidate damaged region are greater than a preset damage feature threshold, an adaptive region filling algorithm is used to determine the specific extent of damage in the candidate damaged region. Based on the specific extent, a damage mask image is obtained, including: If the breakpoint in the candidate damaged area is detected to meet the preset triggering condition, the edge tracking algorithm is used to identify the boundary position of the breakpoint and obtain the set of boundary points. Based on the connection relationship between adjacent points in the boundary point set, an adaptive region growth algorithm is used to divide the damaged area and determine the preliminary area range. For the initial region range, a smoothing filter operation is applied to smooth the region boundaries to obtain an optimized region shape; The optimized region shape is converted into a black and white mask format using binarization processing to obtain the initial mask image; If there are isolated noise pixels in the initial mask image, the noise is removed by median filtering to obtain a denoised mask image. For the denoised mask image, the integrity and connectivity of the damaged region are verified using connected component analysis, and the verification results are obtained. If the verification result meets the preset verification criteria, the denoised mask image is determined as the final damaged mask image.

[0042] It should be noted that the final damaged mask image is a binary image, where the pixel value of the damaged area is 1 (white) and the pixel value of the non-damaged area is 0 (black). It is a precise spatial description of the missing or damaged parts of the vehicle identification image.

[0043] For example, if the breakpoint distribution and area parameters of the candidate damaged region obtained through step S13 meet the preset triggering conditions, the system starts the edge tracking algorithm. This algorithm starts from the edge position of the breakpoint cluster, and along the gradient direction, identifies and tracks the actual boundary positions around the discontinuities to obtain a set of boundary points.

[0044] Based on the connectivity between adjacent points in the boundary point set, the system employs an adaptive region growing algorithm to divide the damaged region. The specific steps of the algorithm are as follows: The algorithm selects a seed point from within the candidate damaged region. If adjacent pixels grayscale value or color and area average The difference between them is less than the preset similarity threshold. ,and If it does not belong to the complete boundary of the identified vehicle identifier, then Add to current area This allows us to determine the initial area boundaries.

[0045] Based on the initial region range, a smoothing filter operation is applied to smooth the region boundary, eliminating jagged edges generated during region growth and generating an optimized region shape.

[0046] Subsequently, binarization processing is used to convert the optimized region shape into a black-and-white mask format, wherein the pixels of the damaged region are set to... The non-damaged area is Generate the initial mask image.

[0047] If there are isolated noise pixels (i.e., single white pixels) in the initial mask image, these noise pixels are removed by median filtering to obtain a denoised mask image.

[0048] Based on the denoised mask image, connected component analysis is used to verify the integrity and connectivity of the damaged areas. The detailed steps are as follows: Traverse all pixels of the denoised mask image, starting from the first unlabeled "1-value pixel," recursively or iteratively search for all adjacent "1-value pixels" in eight directions: "up, down, left, right, upper left, upper right, lower left, lower right," forming an independent connected component. Assign a unique numerical label (connected component 1, connected component 2, etc.) to each connected component until all "1-value pixels" are labeled. Calculate the core parameters of each labeled connected component, including the number of pixels, minimum bounding rectangle, and circularity. Compare these parameters with preset filtering conditions: only connected components with an area > 100 square pixels and a circularity < 0.7 are retained, while isolated fragments with too small an area are deleted. Maintain the pixel value of the retained "effective connected components" at 255, and set the pixel values ​​of the remaining areas to 0, thus obtaining the final damaged mask image.

[0049] The adaptive region growth similarity condition adopts the following: in: Adjacent pixels The intensity value; The current area The average intensity value of all pixels in the image; This is a preset similarity threshold. Region growing is only allowed when the pixel intensity difference is less than this threshold.

[0050] In step S15, the vehicle identification image is overlaid using the damaged mask image to extract the neighborhood pixel information of the missing characters. The pixel information is then arranged according to a fixed dimension to form a neighborhood feature vector, including: Obtain the pixel matrix data of the damaged mask image and the vehicle identification image, and determine the coordinate range of the overlapping area of ​​the two images by pixel-by-pixel traversal operation; Based on the coordinate range of the overlapping area, pixel value difference operation is used to identify the boundary contour of the missing character and obtain the coordinate range of the missing character area; Based on the coordinate range of the missing character region, calculate the gradient direction and intensity value of the complete pixels around the missing character region, and construct an initial feature descriptor based on the gradient direction and intensity value; If the dimension of the initial feature descriptor exceeds a preset dimension threshold, then the principal component analysis algorithm is applied to reduce the dimension of the initial feature descriptor to obtain an optimized feature representation. The optimized feature representation is then standardized and normalized to obtain the final neighborhood feature vector.

[0051] For example, obtain the damaged mask image (binary image, ) and vehicle identification images ( The pixel matrix data of the two images is used to determine the coordinate range of the overlapping area of ​​the two images by traversing the pixel matrix data pixel by pixel; then, pixel-by-pixel multiplication is performed on the pixel matrix data to obtain the isolated image data. This operation utilizes masking properties, when... When the original pixel value is retained, (Normal area) Set to zero masking, only retain relevant neighboring or repaired area information.

[0052] Based on the coordinate range of the overlapping area, pixel value difference calculation is used to identify the boundary contour of the missing character; for the isolated image data By traversing all non-zero pixels and recording the coordinates of the leftmost, rightmost, topmost, and bottommost pixels, the smallest bounding rectangle is formed, and the coordinate range of the missing character region is determined.

[0053] Based on the coordinate range of the missing character region, a boundary expansion algorithm is used to set the neighborhood radius, extending outward by 5 pixels. Then, a bilinear interpolation algorithm is used to overlay the vehicle identification image and the damaged mask image pixel-by-pixel. The calculation formula is: the fused pixel value equals the original image pixel value multiplied by the mask transparency plus the mask pixel value multiplied by the mask opacity, where the transparency is set to 0.7. This weighted fusion method highlights the boundary information of the missing character region. Next, the system extracts pixel features within the defined neighborhood, using a sliding window method to traverse the neighborhood region with a 2×2 step size. The mean, variance, and gradient magnitude of the grayscale within each window are calculated, and the grayscale distribution data of the surrounding complete pixels are extracted.

[0054] Based on the grayscale distribution data, the gradient direction is calculated using operators such as the Sobel operator. and gradient intensity value .

[0055] The gradient direction is quantized, and histogram statistics are performed on the gradient intensity values ​​to construct an initial feature descriptor containing spatial relationships. If the dimension of the initial feature descriptor exceeds a preset dimension threshold, principal component analysis is applied to reduce the dimension of the initial feature descriptor, remove redundant information, and obtain optimized features.

[0056] The optimized feature representation is standardized and normalized, mapping the feature values ​​to... The interval is used to eliminate the influence of dimensions and obtain the final neighborhood feature vector.

[0057] The gradient direction quantization and histogram statistics formulas are as follows: in, It is a neighboring region; It is an indicator function; it is 1 when the condition is met and 0 otherwise. It is a direction quantization function.

[0058] In step S16, a convolutional neural network model is used to perform sequence inference on the neighborhood feature vectors to obtain a candidate character set, including: A convolutional neural network model is used to extract features from the neighborhood feature vector to obtain a high-dimensional feature representation. The high-dimensional feature representation is processed by a pre-trained sequence inference model, mapped to a character probability space, and the character category at the missing position is predicted. The prediction result is then determined as the initial character type of the missing character. If the confidence level of the initial character type is greater than or equal to the preset confidence threshold, then an initial candidate character set is generated based on the initial character type; If the confidence score of the initial character type is less than the preset confidence score threshold, then pattern matching and inference are performed again. According to the preset national license plate coding rules, characters that conform to the structural constraints in the coding rules are selected from the initial candidate character set to obtain an optimized candidate character set; The conditional random field model is used to perform sequence labeling and probability modeling on the optimized candidate character set to obtain the probability distribution of the character set; The probability distribution is sorted and truncated to obtain the highest probability character sequences as the final candidate character set.

[0059] It should be noted that the character co-occurrence matrix model was trained using a dataset containing 500,000 license plate character samples. The training parameters were configured as follows: the optimizer was Adam, and the learning rate was set to... Batch size is ,go through After rounds of iterations, the model accuracy reached .

[0060] For example, the system constructs a convolutional neural network model, using the ResNet-50 architecture as the feature extractor. The input layer receives the standardized neighborhood feature vectors, which are processed through four residual blocks to extract deep semantic information and output a 2048-dimensional high-dimensional feature representation.

[0061] The high-dimensional feature representation is processed by a pre-trained sequence inference model (composed of the ResNet-50 feature extractor and a fully connected layer). The high-dimensional feature representation is input into the fully connected layer, and pattern matching is performed through the Softmax function to map it to the character probability space (68 categories in total, including numbers, letters, and province abbreviations). The probability distribution of each character category is output to determine the initial character type and corresponding confidence level of the missing character.

[0062] If the confidence level of a certain character type is greater than or equal to the preset confidence threshold If the initial character type has a confidence level less than a preset threshold, the system calls the character co-occurrence matrix that records the conditional probability of characters appearing adjacently at a specific position, adjusts the classification weights of the neighborhood feature vectors, and then re-executes pattern matching and inference.

[0063] According to the preset national license plate coding rules, characters that do not conform to the format verification rules are removed from the initial candidate character set to obtain an optimized candidate character set.

[0064] A Conditional Random Field (CRF) model is used for sequence labeling and probabilistic modeling of the optimized candidate character set. The CRF model considers the global dependencies of the entire character sequence and calculates the conditional probability distribution of each character label sequence given a feature vector.

[0065] By sorting and truncating the probability distribution, several character sequences with the highest probability are obtained and output as the final candidate character set.

[0066] In step S17, iterative matching is performed on the candidate character set and the preset national license plate coding rules to obtain the optimal matching character combination. The optimal matching character combination is then used to fill the missing positions to obtain the repaired character sequence, including: Call the preset vehicle identification format template, and determine whether the inference result in the candidate character set meets the character type requirements of the corresponding position in the format template. If it does, it is determined to be a valid character sequence. The legal character sequences are exhaustively enumerated and permuted using an iterative matching algorithm to obtain a set of character sequences to be matched, and the similarity score between each combination in the set of character sequences to be matched and the standard format is calculated. The similarity score and the confidence score are weighted and fused together to determine the character sequence with the highest weighted score as the optimal matching sequence; The characters of the optimal matching sequence are replaced with the original missing positions determined based on the damaged mask image to obtain the repaired complete character sequence; If the repaired complete character sequence conforms to the preset national license plate coding standard, it is used as the final repaired character sequence.

[0067] It should be noted that in the specific iterative comparison process, for example, for the recognition result "Beijing A1B3C8", if the confidence level (0.72) of the fourth character "3" is lower than the set threshold (0.85), the system will trigger the repair mechanism. The system calculates that the Hamming distance similarity between the candidate character "8" and the original character "3" is 0.91, and combined with the historical probability analysis, the probability of "8" at this position is higher. After comprehensive multi-dimensional evaluation, the system replaces "3" with "8" to generate the corrected sequence.

[0068] Exemplarily, the system calls a preset vehicle identification format template. The standard format is usually set as "abbreviation of province + letter + 5-digit combination of numbers / letters".

[0069] Judge whether the inference result in the candidate character set meets the legality requirements of the position in the license plate sequence. The specific steps are as follows. The rule library stipulates that the second character of the license plate must be a letter (A-Z). If the candidate character is a number, it is automatically filtered. If all position constraints are met, it is judged as a legal character sequence. The legal character sequence is exhausted and arranged through an iterative comparison algorithm to obtain a set of character sequences to be matched.

[0070] Calculate the similarity score between each combination in the set of character sequences to be matched and the standard format. The system uses the edit distance algorithm to calculate the similarity between the candidate character and the recognition result. The similarity threshold is set to 0.85. When the similarity 0.91 calculated by the edit distance exceeds this threshold, the character is regarded as a valid candidate.

[0071] Fuse the similarity score and the confidence score weighted. The character sequence with the highest weighted score is determined as the optimal matching sequence.

[0072] Replace the characters of the optimal matching sequence with the original missing positions determined based on the damaged mask image to obtain the repaired complete character sequence.

[0073] The character sequence verification mechanism performs format verification on the repair result. If the repaired complete character sequence conforms to the preset national license plate coding standard, it is output as the final repaired character sequence.

[0074] Among them, the edit distance similarity is adopted: Where: are two character sequences to be compared; is the edit distance between the two sequences, that is, from transform to Minimum number of single-character edits required; Indicates the length of the sequence.

[0075] In step S18, the repaired character sequence is matched with the undamaged characters of the original image for consistency fusion verification. If the verification passes, the final complete vehicle identification information is output, including: Based on the repaired character sequence, standardized character outline information is obtained through pixel-level alignment processing; Spatial position matching is performed between the standardized character outline information and the remaining area of ​​the vehicle identification image. If the overlap between the character boundary and the boundary of the remaining area is greater than or equal to a preset threshold, the position matching is determined to be successful. A convolutional neural network is used to extract features from the character sequences that have successfully matched positions, generating a repaired character feature vector. The cosine similarity between the repaired character feature vector and the features of the remaining region is calculated to obtain a feature similarity score. Geometric consistency verification and illumination consistency analysis are performed on the character sequences that successfully match the positions. Geometric consistency scores are obtained by calculating the differences between character spacing and height, and illumination consistency scores are obtained by calculating the correlation coefficient of the brightness histogram. Based on the feature similarity score, geometric consistency score, illumination consistency score, and semantic consistency score obtained from comparison with the vehicle identification database, a weighted fusion algorithm is used to calculate the comprehensive confidence score. If the overall confidence score is greater than or equal to the preset verification threshold, the repaired character sequence is merged with the remaining area of ​​the vehicle identification image using an image fusion algorithm to output the final complete vehicle identification information.

[0076] It should be noted that, during the specific verification process, the system calculates the feature similarity score as follows: The standard deviation of character spacing is The difference in pixel and character height is Pixel, histogram correlation coefficient is The final calculated overall confidence score is: This improves the recognition accuracy from the original Upgraded to .

[0077] For example, character sequence data from the repair result is obtained, and standardized character contour information is obtained through pixel-level alignment. The standardized character contour information is spatially aligned with the remaining area, and the overlap between the character boundary and the remaining area boundary is calculated. If the overlap is greater than or equal to a preset position matching threshold (90%), the position matching is considered successful. A convolutional neural network is used to extract the edge, texture, and shape features of the successfully matched repaired characters, generating a high-dimensional 512-dimensional feature vector. This feature vector is compared with the features of the remaining part of the vehicle identification image, and a cosine similarity algorithm is used to calculate the feature similarity score.

[0078] A convolutional neural network is used to perform semantic verification on the character sequences that have successfully matched positions. By comparing them with standard templates in the vehicle identification database, and combining regular expression matching and license plate encoding rules, the semantic consistency of the character combinations is confirmed.

[0079] The geometric relationship between the repaired character and its surrounding characters is examined using a perspective transformation matrix. The standard deviation of character spacing and character height difference are calculated. If these geometric parameters are within the allowable threshold (3.0 pixels), geometric consistency is determined and converted into a geometric consistency score.

[0080] Brightness histograms of the repaired area and surrounding area are extracted, and the correlation is calculated using the histogram correlation coefficient formula to obtain an illumination consistency score. If the coefficient is close to 1, it indicates that the illumination conditions are well matched and the repair traces appear natural.

[0081] The fusion verification module comprehensively considers four dimensions: feature similarity, geometric consistency, illumination consistency, and semantic consistency, and uses a weighted fusion algorithm to calculate a comprehensive confidence score. If the comprehensive confidence score exceeds a set threshold of 0.85, the repair result is considered reliable. The verified character sequence is then merged pixel-level with the remaining areas using an image fusion algorithm to output the final complete vehicle identification information.

[0082] The cosine similarity formula is as follows: Used to calculate the feature vector of the repaired character With the original image feature vector Similarity between them: in: These are the components of the eigenvector; It is the dimension of the feature vector.

[0083] The semantic consistency scoring formula adopts: The standard deviation formula for character spacing is as follows: in, It corrects the standard deviation of the spacing between characters and surrounding characters; It is the preset allowable threshold for the standard deviation of the spacing.

[0084] The formula for character height difference is as follows: in, It corrects the height difference between the character and the surrounding characters; It is a preset threshold for allowing height differences.

[0085] The geometric consistency formula is as follows: in, The weights for character spacing score and character height score are respectively.

[0086] The formula for the histogram correlation coefficient is as follows: in: Represents grayscale level; It is a histogram Average value: .

[0087] The formula for scoring illumination consistency is as follows: Reference Figure 2 The second embodiment of this application provides a license plate intelligent recognition system based on image detection, including: The data acquisition and preprocessing module M11 first controls a high-resolution industrial camera to scan and acquire data from the target vehicle. After acquisition, the module applies a Gaussian filter to the original RGB image for noise reduction, with the filter kernel size set to [value missing]. This effectively removes sensor noise and environmental interference. Subsequently, the module calculates the mean and standard deviation of the RGB three channels of the image, completing the image standardization preprocessing. The generated image data is then fed into the contour extraction module.

[0088] The contour extraction module M12 receives preprocessed image data and uses the Canny edge detection algorithm to identify locations of abrupt pixel gradient changes. The algorithm sets dual threshold parameters to accurately identify the contour boundaries of vehicle markings. The module further performs connected component analysis on the detected edge regions, filtering those with areas larger than [a certain threshold]. The effective area of ​​pixels is determined, and the Douglas-Peucker algorithm is applied to approximate the boundary contour with polygons, finally outputting simplified boundary coordinate sequence data.

[0089] The damage detection module M13 analyzes the distance changes between adjacent boundary points in a boundary coordinate sequence. When a sudden change in distance between three consecutive points exceeds a preset threshold and the angle change is greater than a certain value, the module detects the damage. When the degree is reached, the module marks that location as a potential breakpoint. Simultaneously, the module calculates the rate of change of boundary curvature; if the radius of curvature is less than... Pixels or rate of change exceeding If the area is identified as an abnormally damaged area, the specific location information of the candidate damaged area can be determined.

[0090] The mask generation module M14, when the gap length of the discontinuities is greater than... pixels and gradient change exceeds When this occurs, the module is triggered to work. The module employs an improved seed filling algorithm, using the midpoint of discontinuous points as the seed, and sets the filling threshold to a grayscale difference of less than [a certain value]. The module generates the damaged region by expanding the 8-connected domain. Then, it binarizes the generated region and applies morphological closing operations to eliminate holes, generating the final damaged mask image.

[0091] The neighborhood feature extraction module M15 first performs morphological dilation on the damaged mask image to extend the neighborhood range outward. Each pixel is then used. Next, a bilinear interpolation algorithm is employed to overlay the mask onto the original image pixel by pixel, highlighting the boundaries of missing characters. The module uses a sliding window method to traverse the neighborhood, calculates the grayscale mean, variance, and gradient magnitude, and combines this with the local binary pattern histogram to construct a complete neighborhood feature vector.

[0092] The character inference module M16 has a built-in convolutional neural network model, which is input... The neighborhood feature vector of a pixel is output after processing with residual blocks. The model uses a softmax layer to output the probability distribution of each character category. If the confidence level is low, the module also adjusts the feature weights based on the character co-occurrence matrix. Finally, the module outputs an initial candidate character set based on probability ranking.

[0093] The character repair module M17 constrains candidate characters by combining a preset template of vehicle identification format. The module uses edit distance or Hamming distance algorithms to calculate the similarity between candidate characters and the recognition results. If the confidence level of a character at a certain position is low, the module uses a Bayesian probability model to analyze its historical frequency and, combined with format rules, determines the optimal matching character, generating the repaired character sequence.

[0094] The fusion verification and output module M18 extracts the feature vectors of the repaired characters and calculates their cosine similarity with the remaining regions. Simultaneously, the module performs geometric consistency verification and illumination consistency analysis. The module uses a weighted fusion algorithm to calculate a comprehensive confidence score; if the score exceeds... If the repair is successful, the final complete vehicle identification information with improved recognition accuracy will be output.

[0095] It should be noted that the image detection-based intelligent license plate recognition device provided in this application embodiment is used to execute all the process steps of the image detection-based intelligent license plate recognition method in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0096] This application also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an image detection-based intelligent license plate recognition program. When the processor executes the computer program, it implements the steps in the various image detection-based intelligent license plate recognition method embodiments described above, for example... Figure 1 The step S11 shown. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above-described device embodiments, such as the fusion verification and output module.

[0097] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0098] The electronic device may be a desktop computer, laptop, handheld computer, or smart tablet, etc. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0099] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting various parts of the electronic device through various interfaces and lines.

[0100] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0101] Wherein, if the modules / units integrated in the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0102] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0103] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application for those skilled in the art.

Claims

1. A method for intelligent license plate recognition based on image detection, characterized in that, include: Vehicle identification images are acquired using image acquisition equipment, and the vehicle identification images are preprocessed to obtain image data for damage detection. An edge detection algorithm is used to extract the contours of the image data to obtain the boundary coordinate sequence data of the vehicle identifier; The discontinuity of the boundary contour curve is calculated based on the boundary coordinate sequence data, and the candidate region for damage is determined based on the discontinuity. Calculate the feature parameters of the candidate damaged region. If the feature parameters of the candidate damaged region are greater than the preset damage feature threshold, then use an adaptive region filling algorithm to determine the specific range of damage in the candidate damaged region, and obtain a damage mask image based on the specific range. The vehicle identification image is overlaid with the damaged mask image to extract the neighborhood pixel information of the missing characters. The pixel information is then arranged in a fixed dimension to form a neighborhood feature vector. A convolutional neural network model is used to perform sequence inference on the neighborhood feature vectors to obtain a candidate character set; For the candidate character set and the preset national license plate coding rules, iterative matching is performed to obtain the optimal matching character combination, and the optimal matching character combination is used to fill the missing positions to obtain the repaired character sequence; The repaired character sequence is then fused with the undamaged characters of the original image for consistency verification. If the verification passes, the final complete vehicle identification information is output.

2. The intelligent license plate recognition method based on image detection according to claim 1, characterized in that, The process of acquiring vehicle identification images through an image acquisition device, preprocessing the vehicle identification images to obtain image data for damage detection includes: The vehicle's image is captured using an image acquisition device to obtain the original image data of the vehicle's identification. The original image data is subjected to noise removal and brightness and contrast adjustment to obtain a standardized vehicle identification image; The standardized vehicle identification image is processed using an edge detection algorithm to obtain the contour features of the vehicle identification. A normal state reference template for vehicle identification is established using the aforementioned contour features, and the pixel distribution pattern of the normal state reference template is recorded. The pixel grayscale value distribution of the current vehicle identification image is compared and analyzed at the pixel level with the pixel distribution pattern recorded in the normal state reference template to obtain the pixel grayscale value difference value. If the difference in pixel grayscale values ​​exceeds a preset damage detection threshold, it is preliminarily determined that there is damage, and image data for damage detection is obtained.

3. The intelligent license plate recognition method based on image detection according to claim 1, characterized in that, The step of using an edge detection algorithm to extract contours from the image data to obtain boundary coordinate sequence data of the vehicle identifier includes: Based on the image data used for damage detection, the Canny edge detection algorithm is used to identify the locations of pixel gradient abrupt changes, and the pixels corresponding to the abrupt changes are marked as edge pixels to obtain a set of edge pixel points; Based on the connection relationship between adjacent pixels in the edge pixel set, the complete contour path is extracted from the image data of the damage detection using the connected component analysis method. The pixels along the complete contour path are traversed and their coordinates mapped to represent the contour path as a series of ordered pixel coordinate sets, thereby obtaining the boundary coordinate sequence data of the vehicle identifier.

4. The intelligent license plate recognition method based on image detection according to claim 1, characterized in that, The step of calculating the discontinuity of the boundary contour curve based on the boundary coordinate sequence data, and determining the candidate damage region based on the discontinuity, includes: The adjacent boundary coordinates are obtained from the boundary coordinate sequence data, and the Euclidean distance between the adjacent boundary coordinates is calculated. If the Euclidean distance exceeds a preset discontinuity threshold, it is marked as a breakpoint. Based on the breakpoint, the location information of the potential damaged area is obtained. Based on the location information of the potential damaged area, the distribution characteristics of the breakpoints are statistically analyzed. If the distribution characteristics of the breakpoints meet the preset distribution characteristics, the location of the breakpoints is initially determined as a candidate damaged area.

5. The intelligent license plate recognition method based on image detection according to claim 1, characterized in that, The process involves calculating the feature parameters of the candidate damaged region. If the feature parameters of the candidate damaged region are greater than a preset damage feature threshold, an adaptive region filling algorithm is used to determine the specific extent of damage in the candidate damaged region. A damage mask image is then obtained based on the specific extent, including: If the breakpoint in the candidate damaged area is detected to meet the preset triggering condition, the edge tracking algorithm is used to identify the boundary position of the breakpoint and obtain the set of boundary points. Based on the connection relationship between adjacent points in the boundary point set, an adaptive region growth algorithm is used to divide the damaged area and determine the preliminary area range. For the initial region range, a smoothing filter operation is applied to smooth the region boundaries to obtain an optimized region shape; The optimized region shape is converted into a black and white mask format using binarization processing to obtain the initial mask image; If there are isolated noise pixels in the initial mask image, the noise is removed by median filtering to obtain a denoised mask image. For the denoised mask image, the integrity and connectivity of the damaged region are verified using connected component analysis, and the verification results are obtained. If the verification result meets the preset verification criteria, the denoised mask image is determined as the final damaged mask image.

6. The intelligent license plate recognition method based on image detection according to claim 1, characterized in that, The process of overlaying the vehicle identification image with the damaged mask image, extracting the neighborhood pixel information of the missing characters, and arranging the pixel information according to a fixed dimension to form a neighborhood feature vector includes: Obtain the pixel matrix data of the damaged mask image and the vehicle identification image, and determine the coordinate range of the overlapping area of ​​the two images by pixel-by-pixel traversal operation; Based on the coordinate range of the overlapping area, pixel value difference operation is used to identify the boundary contour of the missing character and obtain the coordinate range of the missing character area; Based on the coordinate range of the missing character region, calculate the gradient direction and intensity value of the complete pixels around the missing character region, and construct an initial feature descriptor based on the gradient direction and intensity value; If the dimension of the initial feature descriptor exceeds a preset dimension threshold, then the principal component analysis algorithm is applied to reduce the dimension of the initial feature descriptor to obtain an optimized feature representation. The optimized feature representation is then standardized and normalized to obtain the final neighborhood feature vector.

7. The intelligent license plate recognition method based on image detection according to claim 1, characterized in that, The step of using a convolutional neural network model to perform sequence inference on the neighborhood feature vectors to obtain a candidate character set includes: A convolutional neural network model is used to extract features from the neighborhood feature vector to obtain a high-dimensional feature representation. The high-dimensional feature representation is processed by a pre-trained sequence inference model, mapped to a character probability space, and the character category at the missing position is predicted. The prediction result is then determined as the initial character type of the missing character. If the confidence level of the initial character type is greater than or equal to the preset confidence threshold, then an initial candidate character set is generated based on the initial character type; If the confidence score of the initial character type is less than the preset confidence score threshold, then pattern matching and inference are performed again. According to the preset national license plate coding rules, characters that conform to the structural constraints in the coding rules are selected from the initial candidate character set to obtain an optimized candidate character set; The conditional random field model is used to perform sequence labeling and probability modeling on the optimized candidate character set to obtain the probability distribution of the character set; The probability distribution is sorted and truncated to obtain the highest probability character sequences as the final candidate character set.

8. The intelligent license plate recognition method based on image detection according to claim 1, characterized in that, The process of iteratively matching the candidate character set with the preset national license plate coding rules to obtain the optimal matching character combination, and then using the optimal matching character combination to fill the missing positions to obtain the repaired character sequence, includes: Call the preset vehicle identification format template, and determine whether the inference result in the candidate character set meets the character type requirements of the corresponding position in the format template. If it does, it is determined to be a valid character sequence. The legal character sequences are exhaustively enumerated and permuted using an iterative matching algorithm to obtain a set of character sequences to be matched, and the similarity score between each combination in the set of character sequences to be matched and the standard format is calculated. The similarity score and the confidence score are weighted and fused together to determine the character sequence with the highest weighted score as the optimal matching sequence; The characters of the optimal matching sequence are replaced with the original missing positions determined based on the damaged mask image to obtain the repaired complete character sequence; If the repaired complete character sequence conforms to the preset national license plate coding standard, it is taken as the final repaired character sequence.

9. The intelligent license plate recognition method based on image detection according to claim 1, characterized in that, The process involves performing consistency fusion verification between the repaired character sequence and the undamaged characters of the original image. If the verification passes, the final complete vehicle identification information is output, including: Based on the repaired character sequence, standardized character outline information is obtained through pixel-level alignment processing; Spatial position matching is performed between the standardized character outline information and the remaining area of ​​the vehicle identification image. If the overlap between the character boundary and the boundary of the remaining area is greater than or equal to a preset threshold, the position matching is determined to be successful. A convolutional neural network is used to extract features from the character sequences that have successfully matched positions, generating a repaired character feature vector. The cosine similarity between the repaired character feature vector and the features of the remaining region is calculated to obtain a feature similarity score. Geometric consistency verification and illumination consistency analysis are performed on the character sequences that successfully match the positions. Geometric consistency scores are obtained by calculating the differences between character spacing and height, and illumination consistency scores are obtained by calculating the correlation coefficient of the brightness histogram. Based on the feature similarity score, geometric consistency score, illumination consistency score, and semantic consistency score obtained from comparison with the vehicle identification database, a weighted fusion algorithm is used to calculate the comprehensive confidence score. If the overall confidence score is greater than or equal to the preset verification threshold, the repaired character sequence is merged with the remaining area of ​​the vehicle identification image using an image fusion algorithm to output the final complete vehicle identification information.

10. A license plate intelligent recognition system based on image detection, characterized in that, include: The data acquisition and preprocessing module is used to acquire vehicle identification images through image acquisition equipment, preprocess the vehicle identification images, and obtain image data for damage detection. The contour extraction module is used to extract the contours of the image data using an edge detection algorithm to obtain the boundary coordinate sequence data of the vehicle identifier; The damage detection module calculates the discontinuity of the boundary contour curve based on the boundary coordinate sequence data, and determines the candidate damage region based on the discontinuity; The mask generation module calculates the feature parameters of the candidate damaged region. If the feature parameters of the candidate damaged region are greater than the preset damage feature threshold, the adaptive region filling algorithm is used to determine the specific range of damage in the candidate damaged region, and a damaged mask image is obtained based on the specific range. The neighborhood feature extraction module overlays the vehicle identification image with the damaged mask image to extract the neighborhood pixel information of the missing characters, and arranges the pixel information according to a fixed dimension to form a neighborhood feature vector; the character inference module uses a convolutional neural network model to perform sequence inference on the neighborhood feature vector to obtain a candidate character set; The character repair module performs iterative matching on the candidate character set and the preset national license plate coding rules to obtain the optimal matching character combination, and uses the optimal matching character combination to fill the missing positions to obtain the repaired character sequence; The fusion verification and output module is used to perform consistency fusion verification between the repaired character sequence and the undamaged characters of the original image. If the verification is successful, the final complete vehicle identification information is output.

Images