Closed-loop license plate recognition method and system based on adaptive data set

Abstract

The invention relates to the technical field of license plate recognition, and discloses a closed-loop license plate recognition method and system based on a self-adaptive data set. The method includes the steps that S01, a license plate area is defined in an original image of a target vehicle; s02, segmenting the license plate area into character images; s03, marking a target character string from the license plate character string; s04, a matching quality score is calculated, and whether data set updating operation is triggered or not is judged; s05, updating the preset data set; s06, outputting license plate recognition information of the target vehicle; according to the invention, the linkage effect of comparison of the preset data set and updating of the preset data set can be realized, so that the license plate recognition system has the capabilities of dynamic learning and self-optimization, and the coverage range of the preset data set is ensured to be continuously expanded along with the passage of the timeline; the robustness and generalization ability of the license plate recognition system in coping with various interference factors are improved, and the adaptability and recognition precision of the license plate recognition system are enhanced.

Description

Technical Field

[0001] This invention relates to the field of license plate recognition technology, and more specifically, to a closed-loop license plate recognition method and system based on an adaptive dataset. Background Technology

[0002] License plate recognition systems are an important component of modern intelligent transportation systems. They can automatically extract specific character information from vehicle license plates and perform subsequent comparison processing. They are commonly used in the management of parking lots in various commercial plazas, enterprises, tourist attractions, etc., to accurately identify and effectively manage the license plates of vehicles entering and exiting the parking lot.

[0003] The patent application with publication number CN108091141A discloses a license plate recognition system. It connects the vehicle detection module, license plate detection module, and license plate recognition module based on an effective judgment mechanism and in combination with the first and second identification information in the communication file, instead of directly connecting the vehicle detection module, license plate detection module, and license plate recognition module together. Therefore, even if at least one of the vehicle detection module, license plate detection module, and license plate recognition module fails, the normal operation of other modules will not be affected by the failed module, which greatly improves the robustness of the license plate recognition system. Existing license plate recognition methods typically compare the recognized license plate string with a static, unchanging string in a pre-set dataset when identifying and comparing the character information in the license plate. When negative interference occurs during license plate recognition, such as complex lighting conditions or damaged license plates, the accuracy of the recognized license plate string will decrease. Furthermore, the static, unchanging pre-set dataset lacks a closed-loop optimization mechanism for dataset comparison and updating, making it impossible to perform autonomous verification and dataset optimization and updates for the license plate recognition results. This reduces the adaptability and accuracy of the license plate recognition process.

[0004] In view of this, the present invention proposes a closed-loop license plate recognition method and system based on an adaptive dataset to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned shortcomings of existing technologies and achieve the above objectives, this invention provides the following technical solution: a closed-loop license plate recognition method based on an adaptive dataset, applied to a license plate recognition server, comprising: S01: Mark the boundary turning points in the original image of the target vehicle, connect the boundary turning points to form the license plate area, and optimize the license plate area. S02: Mark the dividing lines in the optimized license plate area, divide the license plate area into character images by the dividing lines, and identify the characters and confidence scores of the character images by the character recognition model; S03: Based on the license plate structure rules, the recognized characters are combined into a license plate string, the reliability of the license plate string is dynamically verified, and the target string is marked from the verified license plate string; S04: Match the target string with the standard strings in the preset dataset, calculate the matching quality score of the target string, and determine whether to trigger the dataset update operation; if the dataset update operation is not triggered, proceed to S06; if the dataset update operation is triggered, proceed to S05. S05: Perform secondary analysis on the matching quality score to parse out the matching properties of the target string, and select an appropriate update mode to update the preset dataset based on the matching properties; S06: Output the license plate recognition information of the target vehicle and execute the working instructions adapted to the license plate recognition information.

[0006] Furthermore, the method for enclosing the license plate area is as follows: The original image of the target vehicle entering the license plate detection area is captured by a camera. The outer edge of the target vehicle in the original image is identified by computer vision technology, and the area inside the outer edge is recorded as the vehicle area. The image corresponding to the vehicle area is converted into a grayscale image. The grayscale image is denoised using a Gaussian filter. All pixels in the grayscale image are marked. The horizontal and vertical gradient components of the pixels are calculated based on the Canny operator. The horizontal and vertical gradient components of the pixels are then merged to calculate the edge intensity of the pixels. Pixels with edge strength greater than a preset strength threshold are recorded as edge pixels, resulting in A edge pixels. A circles of A pixels are drawn with each of the A edge pixels as the center and a radius of one-quarter of the standard license plate width. Count the number of edge pixels and the total number of pixels within the A pixel circles one by one. The pixel circle with the number of edge pixels greater than one-third of the total number of pixels is recorded as the target circle. The edge pixel corresponding to the center of the target circle is recorded as the license plate pixel, thus obtaining B license plate pixels. Draw a rectangular area on the grayscale image, and adjust the rectangular area along the horizontal and vertical directions until the rectangular area just surrounds all B license plate pixels to obtain the character area. Draw two diagonals of the character area. Extend the four corner points of the character area outward along the diagonal by the specified interval length to obtain four boundary turning points. Connect the four boundary turning points in sequence along the horizontal and vertical directions to enclose the area. The vehicle area located inside the enclosed area is recorded as the license plate area.

[0007] Furthermore, the character image segmentation method is as follows: Using the height and width of a single pixel as a standard, divide the license plate area into C rows horizontally and D columns vertically. Count the number of license plate pixels in each of the C rows one by one from top to bottom. Record the first and last rows in which the license plate pixel appears as the starting row and the ending row, and record the area between the starting row and the ending row as the segmented area. Count the number of license plate pixels in each of the D columns from left to right. Record the columns with 0 license plate pixels as the interval columns. Connect the positions of the interval columns that are in consecutive positions along the horizontal direction to obtain E interval lines. Draw a dividing line along the vertical direction, passing through the midpoint of E interval lines, and use this dividing line as the dividing standard to divide the region into sections. A character image.

[0008] Furthermore, the license plate structure rules are as follows: the license plate string must contain numbers, separators, letters, and numbers simultaneously; reliability includes high reliability and low reliability. The method for marking the target string is: After the first original image is captured, H dynamic license plate strings of the target vehicle in the license plate capture area are continuously collected at preset intervals. The first identified license plate string is recorded as the original license plate string. The number of dynamic license plate strings that overlap with the original license plate string is marked and recorded as the overlap value. When the overlap value is greater than the overlap threshold, the reliability of the original license plate string is high reliability, and the original license plate string is recorded as the target string; When the overlap value is less than or equal to the overlap threshold, the reliability of the original license plate string is low reliability. The number of overlaps among the H dynamic license plate strings is counted, and the dynamic license plate string corresponding to the maximum number of overlaps is recorded as the target string.

[0009] Furthermore, the matching quality score is calculated as follows: Within the same space, the target string is transformed into a semantic vector by converting it into a standard string in a pre-set dataset, resulting in a target vector and K standard vectors. The cosine similarity between the target vector and the K standard vectors is calculated one by one using the cosine similarity calculation formula, and the maximum value of the cosine similarity is recorded as the first quality value. The target string corresponding to The second quality value is obtained by summing the confidence levels and averaging them. The first and second quality values ​​are assigned corresponding weight coefficients and then added together to calculate the matching quality score.

[0010] Furthermore, the method for determining whether a dataset update operation is triggered is as follows: Compare the matching quality score of the target string with the defined lower limit of matching quality. When the matching quality score is lower than the lower limit of matching quality, a dataset update operation is triggered. If the matching quality score is greater than or equal to the lower limit of matching quality, it is determined that the dataset update operation will not be triggered.

[0011] Furthermore, matching properties include suspicious properties and unknown properties: The parsing method for matching properties is as follows: Compare the matching quality score of the target string with the calibrated lower limit of the property quality. When the matching quality score is greater than or equal to the lower limit of the specified property quality, the matching property of the target string is parsed as a suspicious property; When the matching quality score is less than the lower limit of the specified property quality, the matching property of the target string is parsed as an unknown property.

[0012] Furthermore, update methods include automatic updates and semi-automatic updates; The method for updating the pre-defined dataset is as follows: When the matching property is considered suspicious, the selected update mode is automatic update; Arrange the target strings in ascending order. The confidence levels are arranged sequentially, and the characters with the highest confidence levels (top two) are marked as suspicious characters. Two text boxes are created on the target string. Two suspicious characters are imported into the two text boxes respectively to generate a target string with suspicious comments. The target string is then imported into a preset dataset for automatic updates. When the matching property is unknown, the selected update mode is semi-automatic update; The license plate recognition server sends the target string to the manual review platform, where reviewers manually review the target string and determine the order of the characters to be recognized in the target string. A rule box is created on the target string, the order of the recognized characters is imported into the rule box, a target string with expanded rules is generated, and the target string is imported into a preset dataset for semi-automatic updating.

[0013] Furthermore, the license plate recognition information includes the license plate number and an update completion notification. When no dataset update operation is triggered, the license plate number information is specified; when a dataset update operation is triggered, an update completion message is specified. The working instructions include the return to standby instruction and the wait for identification instruction; When the license plate recognition information is the license plate number, execute the return to standby command; when the license plate recognition information is the update complete prompt message, execute the wait for recognition command.

[0014] A closed-loop license plate recognition system based on an adaptive dataset is applied to a license plate recognition server to implement a closed-loop license plate recognition method based on an adaptive dataset. The system includes a license plate region encirclement module, a character image segmentation module, a string tagging module, an update determination module, a dataset update module, and an information output module. The license plate area enclosing module is used to mark the boundary turning points in the original image of the target vehicle, connect the boundary turning points to enclose the license plate area, and optimize the license plate area. The character image segmentation module is used to mark the segmentation lines in the optimized license plate area, segment the license plate area into character images through the segmentation lines, and identify the recognized characters and confidence scores of the character images through the character recognition model; The string tagging module is used to combine recognized characters into a license plate string based on license plate structure rules, dynamically verify the reliability of the license plate string, and tag the target string from the verified license plate string; The update judgment module is used to match the target string with the standard strings in the preset dataset, calculate the matching quality score of the target string, and determine whether to trigger the dataset update operation. The dataset update module is used to perform secondary parsing of the matching quality score, parse out the matching properties of the target string, and select an appropriate update mode to update the preset dataset based on the matching properties. The information output module is used to output the license plate recognition information of the target vehicle and execute working instructions adapted to the license plate recognition information.

[0015] The technical advantages of the closed-loop license plate recognition method and system based on adaptive datasets proposed in this invention are as follows: (1): By marking dividing lines in the license plate area and dividing the license plate area into independent character images based on the dividing lines, the present invention can accurately divide the overall license plate area according to the specific character positions, thereby achieving the effect of breaking down the license plate area into parts, ensuring that each character in the license plate can maintain an independent and orderly state for recognition, thus avoiding the chaotic and disorderly phenomenon that is easy to occur when recognizing all characters in the license plate area as a whole, and also reducing the workload of license plate recognition operation.

[0016] (2): This invention calculates the matching quality score of the target string and, after parsing the matching quality score, determines whether to update the preset dataset. This enables the comparison between the target string and the standard string in the preset dataset, as well as the linkage effect of updating the standard string in the preset dataset. This allows the license plate recognition system to have the ability to learn dynamically and optimize itself, ensuring that the recognition accuracy of the license plate recognition system can directly benefit from the newly added sample data. It also ensures that the coverage of the preset dataset will continue to expand over time, thereby improving the robustness and generalization ability of the license plate recognition system in dealing with various interference factors, and enhancing the adaptability and recognition accuracy of the license plate recognition system. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a closed-loop license plate recognition method based on an adaptive dataset, provided in Embodiment 1 of the present invention. Figure 2 This is a schematic diagram of a closed-loop license plate recognition system based on an adaptive dataset, provided in Embodiment 2 of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: Please refer to Figure 1 As shown in this embodiment, a closed-loop license plate recognition method based on an adaptive dataset is applied to a license plate recognition server, including: S01: In license plate recognition mode, capture the original image of the target vehicle, mark the boundary turning points in the original image, connect the boundary turning points to form the license plate area, and optimize the license plate area. The license plate recognition status refers to the normal working state of the license plate recognition server, which is capable of performing a series of operations such as license plate recognition in real time. This ensures that the license plate recognition server can switch from its original dormant or standby state to a working state where it can randomly perform license plate recognition. When it is determined that the license plate recognition server is in license plate recognition mode, the license plate recognition server needs to receive a trigger signal that can trigger its normal operation.

[0020] In this embodiment, the trigger signal is a signal that can drive the license plate recognition server to switch from a dormant or standby state to a license plate recognition state; the trigger signal is usually detected by externally deployed sensors or other devices; For example, if the externally deployed sensor is a pressure sensor, when the vehicle drives into the detection area of ​​the pressure sensor, the license plate recognition server will receive the pressure signal and thus switch the working state to license plate recognition state; if the externally deployed sensor is radar, when the vehicle drives into the detection area of ​​the radar, the license plate recognition server will receive the reflected electromagnetic wave signal and thus switch the working state to license plate recognition state.

[0021] The original image is the image of the target vehicle directly captured by the camera device in license plate recognition mode, without any technical processing. At this time, the original image not only contains information belonging to the target vehicle itself, but also environmental information that does not belong to the target vehicle. In order to improve the targeting of subsequent license plate recognition operations and avoid interference from too much irrelevant image information, it is necessary to identify the specific areas covered by Chinese characters, symbols, numbers and letters on the license plate from the original image and record these areas as the license plate area. Since the license plate of a vehicle is actually a rectangular structure, and its shape in the original image is also a rectangular structure, when determining the license plate area from the original image, it is only necessary to determine the location of the four corner points of the license plate area and record the location of the four corner points of the rectangular structure as the boundary turning points. Specifically, the method for enclosing the license plate area is as follows: The original image of the target vehicle entering the license plate detection area is captured by a camera. The outer edge of the target vehicle in the original image is identified by computer vision technology, and the area inside the outer edge is recorded as the vehicle area. The image corresponding to the vehicle area is converted into a grayscale image. A Gaussian filter is used to denoise the grayscale image, and all pixels in the grayscale image are marked. The Gaussian filter can smooth the grayscale image while retaining some edge information, thereby removing Gaussian noise in the grayscale image. The gradient intensity of a pixel in the horizontal and vertical directions is calculated based on the Canny operator, and the horizontal gradient component and the vertical gradient component are obtained. The horizontal gradient component and the vertical gradient component of the same pixel are merged to calculate the edge intensity of the pixel. The edge intensity is used to integrate the gradient magnitude and edge direction of the pixel and serves as a numerical basis for determining whether the pixel is at an edge position. Pixels with edge strength greater than a preset strength threshold are designated as edge pixels, resulting in A edge pixels. Circles of A pixels are then drawn with each of these A edge pixels as its center and a radius equal to one-quarter of the width of a standard license plate. The preset strength threshold refers to the minimum edge strength when a pixel is designated as an edge pixel, providing a numerical basis for distinguishing edge pixels from ordinary pixels. Specifically, the preset strength threshold is obtained by averaging the minimum edge strength values ​​collected from a large number of historical pixel classifications. Count the number of edge pixels and the total number of pixels within the A pixel circles one by one. The pixel circle with the number of edge pixels greater than one-third of the total number of pixels is recorded as the target circle. The edge pixel corresponding to the center of the target circle is recorded as the license plate pixel, thus obtaining B license plate pixels. Draw a rectangular area on the grayscale image, and adjust the rectangular area along the horizontal and vertical directions until the rectangular area just surrounds all B license plate pixels to obtain the character area. Draw two diagonals of the character area. Extend the four corner points of the character area outward along the diagonal by the specified interval length to obtain four boundary turning points. Connect the four boundary turning points in sequence along the horizontal and vertical directions to enclose the area. The vehicle area located inside the enclosed area is recorded as the license plate area.

[0022] It should be noted that the specified interval length refers to the length of the character area extending outward from the corner point, which ensures that the character area can expand outward to form the license plate area. Specifically, the specified interval length can be set according to actual needs, and is usually less than one-quarter of the standard license plate width.

[0023] After obtaining the license plate area, no optimization processing is performed on the license plate area compared with the corresponding area of ​​the original image. This means that the real-time image quality of the original image directly reflects the image quality of the license plate area. When the original image is affected by adverse weather conditions or complex lighting during shooting, the image clarity may be poor, resulting in poor clarity of the subsequent license plate area. Therefore, it is necessary to optimize the recognized license plate area to improve its clarity. Specifically, when optimizing the license plate area, since the license plate area has already undergone grayscale conversion and Gaussian filtering for noise reduction, it is necessary to first correct the license plate area to ensure that it maintains a horizontal rectangular structure. Then, median filtering and bilateral filtering are applied to the corrected license plate area to remove different types of noise interference as much as possible. Finally, grayscale value expansion processing is applied to the denoised license plate area to increase the grayscale value of the license plate pixels, thereby highlighting the difference between the characters in the license plate and the corresponding pixels of the frame in the license plate.

[0024] S02: Mark the dividing lines in the optimized license plate area, divide the license plate area into character images by the dividing lines, and identify the character results of the character images by the character recognition model. The character results include the recognized characters and confidence scores. The dividing line is a baseline used to independently separate the Chinese characters, symbols, letters, and numbers in the license plate area. It provides a standard for independently separating two adjacent characters, so that each character in the license plate area can be separated independently. Specifically, the method for segmenting character images is as follows: Using the height and width of a single pixel as a standard, the license plate area is divided into C rows horizontally and D columns vertically. By dividing the license plate area into rows and columns, the license plate area can be effectively distinguished in both horizontal and vertical directions. This provides an intuitive and accurate statistical basis for the number of edge pixels at each position, avoiding random errors that exist when recognizing and counting the entire license plate area. Count the number of license plate pixels in each of the C rows one by one from top to bottom. Record the first and last rows in which the license plate pixel appears as the starting row and the ending row, and record the area between the starting row and the ending row as the segmented area. Count the number of license plate pixels in each of the D columns from left to right. Record the columns with 0 license plate pixels as the interval columns. Connect the positions of the interval columns that are in consecutive positions along the horizontal direction to obtain E interval lines. Draw a dividing line along the vertical direction, passing through the midpoint of E interval lines, and use this dividing line as the dividing standard to divide the region into sections. A character image.

[0025] It should be noted that each character image contains only one license plate pixel in a continuous state, so that a character image can only contain one type of character, which may be Chinese characters, letters, numbers, or symbols, with the symbol being the separator "." between letters and numbers; and the number of character images may be 8 or 9; for example, when the target vehicle's license plate is a Type 92 license plate or a license plate of certain special units, the number of character images is 8; when the target vehicle's license plate is a new energy vehicle license plate, the number of character images is 9.

[0026] After segmenting into individual character images, all character images can be imported one by one into a pre-trained character recognition model to identify the corresponding character in the character image. The character result can represent the specific character in the character image and the confidence level corresponding to the character in two dimensions, thereby realizing the recognition of the meaning of the specific character in the character image. Specifically, the character results include the identified character and the confidence level; the identified character refers to the specific meaning and content corresponding to the license plate pixels in the character image, including but not limited to Chinese characters, symbols, letters and numbers; the confidence level refers to the probability that the identified character in the character image has the correct meaning.

[0027] In this embodiment, the character recognition model is trained using a large number of historical character images of different types and the corresponding recognition characters and confidence scores as training data. The resulting machine learning model can recognize the corresponding recognition characters and confidence scores based on the character images, and can be used as a tool for automatically recognizing the specific meaning of characters in character images. At this time, the character recognition model can take character images as input and recognition characters and confidence scores as output, thereby achieving accurate character image recognition. It should be noted that the machine learning model can be either a Support Vector Machine (SVM) or a Convolutional Neural Network (CNN).

[0028] When recognizing characters in a character image, the characters are processed sequentially according to the segmentation order. The character images are numbered in ascending order, and then arranged sequentially from smallest to largest number. When a character image is input into a character recognition model, the output of the character recognition model is... The recognition results corresponding to each character image, and Character image and By binding each recognition result to a single one, accurate recognition of character images can be achieved.

[0029] S03: Based on the license plate structure rules, the recognition results are combined into a license plate string, the reliability of the license plate string is dynamically verified, and the target string is marked from the license plate string; After recognizing the characters and confidence level of the character image, the recognition results can be summarized and combined according to the license plate structure rules to generate the license plate string of the target vehicle. The license plate string is a string formed by arranging the recognized characters in the character image according to the combination method and structure of the license plate. When summarizing and combining the license plate strings, the summarization and combination should be based on the license plate structure rules to ensure that the summarized and combined license plate strings can be consistent with the standard license plate strings stored in the pre-set database. Specifically, the license plate structure rules are as follows: the license plate string must contain numbers, separators, letters, and numbers simultaneously.

[0030] When combining license plate strings, arrange them in ascending order of number. The characters corresponding to the character images are arranged horizontally and combined into a string. The characters before and after the separators in the string are recorded as the preceding and following characters, respectively, to generate the license plate string.

[0031] When the license plate string is obtained, it can only be used as the result of the license plate in the original image of the target vehicle at a certain moment. It cannot be used to dynamically identify and track the license plate of the target vehicle in the original image at different times over time. In order to ensure that the license plate string identified by the license plate recognition server can be consistent with the actual license plate of the target vehicle, it is necessary to dynamically track and verify the license plate string in the original image at multiple times to verify whether the identified license plate string is consistent with the standard string in the preset database. By comparing the license plate strings identified at different times with the standard string, the dynamic and continuous license plate strings of the target vehicle entering the license plate shooting area can be judged to determine the reliability of the license plate strings. The most reliable license plate string can then be verified as the basis for subsequent comparison and update operations. Specifically, reliability includes high reliability and low reliability; high reliability means that the license plate string is identified more often during subsequent dynamic tracking, while low reliability means that the license plate string is identified less often during subsequent dynamic tracking.

[0032] The target string refers to the license plate string that appears most frequently in the dynamic verification of the target vehicle within a subsequent continuous time period, which can be used as the best basis for subsequent license plate recognition.

[0033] The method for marking the target string is: After the first original image is captured, H original images of the target vehicle in the license plate capture area are continuously acquired at a preset interval, and H dynamic license plate strings are identified from the H original images. The preset interval is a numerical representation of the time interval between the capture of two adjacent original images, ensuring that the two adjacent original images remain independent on the timeline, thereby achieving a dynamic tracking capture effect along the timeline. For example, the preset interval can be 0.10S, 0.15S, etc.

[0034] The first identified license plate string is recorded as the original license plate string. The H dynamic license plate strings are analyzed one by one against the original license plate string for overlap, and the number of dynamic license plate strings that overlap with the original license plate string is marked and recorded as the overlap value. When the overlap value is greater than the overlap threshold, it indicates that the H dynamic license plate strings overlap with the original license plate string a significant number of times. In this case, the original license plate string is recognized more frequently, and its reliability is considered high. The original license plate string is then designated as the target string. The overlap threshold is a numerical criterion used to distinguish between high and low reliability, classifying the reliability of the original license plate string as either high or low. Specifically, the overlap threshold is set based on the number of dynamic license plate strings; for example, it can be set to 85% of the number of dynamic license plate strings. When the overlap value is less than or equal to the overlap threshold, it means that the number of times the H dynamic license plate strings overlap with the original license plate string is small. At this time, the original license plate string is recognized less often, so the reliability of the original license plate string is low. Count the number of overlapping license plate strings among H dynamic license plate strings, and denote the dynamic license plate string corresponding to the maximum number of overlapping strings as the target string.

[0035] It should be noted that when a license plate string is verified as highly reliable, it will maintain dynamic continuity in the subsequent timeline, thus providing a unique and accurate target string for subsequent processing. When a license plate string is verified as low reliable, it will not maintain dynamic continuity in the subsequent timeline, and the final target string will be determined based on the overlapping license plate strings that appear later.

[0036] S04: Match the target string with the standard strings in the preset dataset, determine the matching quality score of the target string, and determine whether to trigger the dataset update operation; The pre-set dataset refers to the data set in the license plate recognition server that stores a large number of different types of standard format license plate strings for input and collection, which can be used as the object for comparison and matching with the target string; The matching quality score is a score obtained by matching the target string with the standard strings in the preset dataset. It reflects the quality of the match and serves as a basis for determining whether the target string has been included in the preset dataset. Specifically, the matching quality score is calculated as follows: Within the same space, the target string is transformed into a semantic vector by converting it into a standard string in a pre-set dataset, resulting in a target vector and K standard vectors. The cosine similarity between the target vector and the K standard vectors is calculated one by one using the cosine similarity formula, and the maximum value of the cosine similarity is recorded as the first quality value. The similarity between the target vector and the standard vector can be evaluated by calculating the cosine of the angle between them in the same space. The closer the cosine similarity is to 1, the more consistent the directions of the target vector and the standard vector are, and the higher the similarity between the target string and the standard string corresponding to the target vector and the standard vector. The target string corresponding to The second quality value is obtained by summing the confidence levels and averaging them. The formula for calculating the second mass value is: ; In the formula, The second quality value, For the first One confidence level; The first and second quality values ​​are assigned corresponding weight coefficients and then added together to calculate the matching quality score. The formula for calculating the matching quality score is: ; In the formula, To match the quality score, The first quality value, , These are the weighting coefficients for the first and second quality values, respectively. and The sum of is 1. , All are greater than 0.

[0037] It should be noted that both the first quality value and the second quality value are dimensionless numerical values. By assigning different weighting coefficients to the first quality value and the second quality value, the degree of influence of the first quality value and the second quality value in the matching quality score can be indicated, thereby achieving an accurate calculation of the matching quality score.

[0038] After calculating the matching quality score, the matching quality score can be used as the result of matching the target string with the standard string in the preset dataset to determine whether the target string has been included in the preset dataset, and thus determine whether the dataset update operation needs to be triggered to update the preset dataset. Specifically, the dataset update operation refers to the measure of updating and optimizing the standard strings in the preset dataset when the target string has a low degree of matching overlap with the standard dataset in the preset dataset, thereby expanding the coverage of the standard strings in the preset dataset. The method for determining whether a dataset update operation has been triggered is as follows: Compare the matching quality score of the target string with the defined lower limit of matching quality. When the matching quality score is less than the lower limit of the matching quality, it means that the target string has a low degree of overlap with the standard string in the preset dataset, and the dataset update operation is triggered. When the matching quality score is greater than or equal to the lower limit of the matching quality, it means that the target string and the standard string in the preset dataset have a high degree of overlap, and it is determined that the dataset update operation will not be triggered.

[0039] It should be noted that the calibrated lower limit of matching quality refers to the minimum matching quality score when a dataset update operation is triggered, which can be used as the numerical basis for determining whether to trigger a dataset update operation. The calibrated lower limit of matching quality is obtained by collecting the minimum matching quality scores of a large number of historical dataset update operations and then averaging them.

[0040] S05: Perform secondary parsing on the matching quality score to parse out the matching properties of the target string, select the update mode of the preset dataset based on the matching properties, and update the preset dataset. When a dataset update operation is triggered, if the target string does not match the standard string in the preset dataset well, the preset dataset needs to be updated. When updating the pre-set dataset, it is necessary to perform a second parsing on the matching quality score between the target string and the standard string. This will reveal the matching properties of the target string. The matching properties at this point are used to specifically represent the license plate results presented when the target string matches the standard string, thus enabling diverse representations of different types and styles of license plate results. Specifically, the matching properties include suspicious properties and unknown properties; suspicious properties refer to the target string matching the standard string to a moderate degree, and the target string may be consistent with the standard string; unknown properties refer to the target string matching the standard string to a low degree, and the target string may be a newly emerging license plate style.

[0041] The parsing method for matching properties is as follows: Compare the matching quality score of the target string with the calibrated lower limit of the property quality. When the matching quality score is greater than or equal to the lower limit of the calibrated property quality, it indicates that the degree of overlap between the target string and the standard string in the preset dataset is moderate. At this time, the target string may have a consistent phenomenon, so the matching property of the target string is parsed as a suspicious property. When the matching quality score is less than the lower limit of the calibrated property quality, it indicates that the target string has a low degree of overlap with the standard string in the preset dataset. At this time, it is impossible for the target string to have a consistent phenomenon, so the matching property of the target string is parsed as an unknown property.

[0042] It should be noted that the calibrated lower limit of property quality refers to the minimum value of the matching quality score when the matching property is parsed as a suspicious property, which can be used as the data basis for determining suspicious or unknown properties. The calibrated lower limit of property quality is obtained by collecting a large number of historical minimum values ​​of matching quality scores when they were parsed as suspicious properties and then calculating their average value. In this embodiment, the calibrated lower limit of property quality is less than the calibrated lower limit of matching quality.

[0043] The update method is the specific way to update and supplement the standard strings in the pre-set dataset. The specific update method varies depending on the matching properties. Specifically, the update methods include automatic updates and semi-automatic updates; automatic updates refer to the pre-set dataset being updated automatically in real time, while semi-automatic updates refer to the pre-set dataset requiring manual updates.

[0044] After selecting a specific update method, the preset dataset can be updated to achieve the effect of automatic or semi-automatic updating of standard strings within the preset dataset. Specifically, the method for updating the pre-set dataset is as follows: When the matching property is considered suspicious, the selected update mode is automatic update; Arrange the target strings in ascending order. The confidence levels are arranged sequentially, and the characters with the highest confidence levels (top two) are marked as suspicious characters. Two text boxes are created on the target string. Two suspicious characters are imported into the two text boxes respectively to generate a target string with suspicious comments. The target string is then imported into a preset dataset for automatic updates. When the matching property is unknown, the selected update mode is semi-automatic update; The license plate recognition server sends the target string to the manual review platform, where reviewers manually review the target string and determine the order of the characters to be recognized in the target string. A rule box is created on the target string, and the order of the recognized characters is imported into the rule box to generate a target string with expanded rules. This target string is then imported into a pre-set dataset for semi-automatic updates. By generating expanded rules, an intuitive basis can be provided for the license plate rules and formats of the target string imported into the pre-set dataset, and a basis can be provided for subsequent license plate recognition comparison operations, facilitating subsequent license plate recognition comparison operations.

[0045] It should be noted that the specific operating methods for automatic and semi-automatic updates are different. Automatic updates can achieve real-time dynamic effects, while semi-automatic updates require manual review.

[0046] S06: Output the license plate recognition information of the target vehicle and execute the working instructions adapted to the license plate recognition information; License plate recognition information is the information obtained and output by the license plate recognition server after a series of recognitions on the license plate of the target vehicle. During the license plate recognition process, there are two ways: triggering or not triggering the dataset update operation. Therefore, the corresponding license plate recognition information needs to be formulated according to the specific method. Specifically, the license plate recognition information includes license plate number information and update completion prompt information; the license plate number information refers to the specific license plate number of the target vehicle obtained when no dataset update operation is triggered, and the update completion prompt information refers to the standard string update operation performed on the preset dataset when the dataset update operation is triggered. In this embodiment, when outputting the license plate recognition information of the target vehicle, if no dataset update operation is triggered, the license plate recognition server recognizes the license plate number of the target vehicle and outputs the license plate number information. When a dataset update operation is triggered, the license plate recognition server performs the update operation of the standard string in the preset dataset and outputs the update completion prompt message.

[0047] It should be noted that license plate recognition information can represent the license plate number of the target vehicle. For example, when the license plate of the target vehicle is a Type 92 license plate, the license plate recognition information of the target vehicle can be "Su C.0556D". For example, when the license plate of the target vehicle is a new energy vehicle license plate, the license plate recognition information of the target vehicle can be "Su C.D0556D".

[0048] After outputting the license plate number information or the update completion prompt message, the license plate recognition server can be controlled to execute different working instructions according to the specific information, so that different working instructions can correspond to different license plate number information and drive the license plate recognition server into different working states. Specifically, the working instructions include the return to standby instruction and the wait for identification instruction; In this embodiment, when executing the work instruction, if the license plate recognition information is the license plate number, it means that the license plate number of the target vehicle has been correctly recognized and the entire license plate recognition process has been successfully completed. At this time, the license plate recognition server can return to the initial state and execute the return to standby instruction. If the license plate recognition information is the update completion prompt, it means that the license plate number of the target vehicle has not been correctly recognized, but the preset dataset has completed the standard string update operation, and then the wait recognition instruction is executed.

[0049] It should be noted that when the return to standby command is executed, the license plate recognition server can return to the initial sleep or standby state, waiting to be woken up again to perform the license plate recognition operation; when the wait for recognition command is executed, the standard string in the preset dataset has already been fully expanded and updated, so the license plate recognition server can continue to wait for the next license plate recognition operation.

[0050] Example 2: Please refer to Figure 2 As shown, the parts not described in detail in this embodiment are described in Embodiment 1. A closed-loop license plate recognition system based on an adaptive dataset is provided and applied to a license plate recognition server to implement a closed-loop license plate recognition method based on an adaptive dataset. The system includes a license plate area encirclement module, a character image segmentation module, a string marking module, an update determination module, a dataset update module, and an information output module. The modules are connected to each other via wired or wireless networks. The license plate area enclosing module is used to mark the boundary turning points in the original image of the target vehicle, connect the boundary turning points to enclose the license plate area, and optimize the license plate area. The character image segmentation module is used to mark the segmentation lines in the optimized license plate area, segment the license plate area into character images through the segmentation lines, and identify the recognized characters and confidence scores of the character images through the character recognition model; The string tagging module is used to combine recognized characters into a license plate string based on license plate structure rules, dynamically verify the reliability of the license plate string, and tag the target string from the verified license plate string; The update judgment module is used to match the target string with the standard strings in the preset dataset, calculate the matching quality score of the target string, and determine whether to trigger the dataset update operation. The dataset update module is used to perform secondary parsing of the matching quality score, parse out the matching properties of the target string, and select an appropriate update mode to update the preset dataset based on the matching properties. The information output module is used to output the license plate recognition information of the target vehicle and execute working instructions adapted to the license plate recognition information.

[0051] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A closed-loop license plate recognition method based on an adaptive dataset, applied to a license plate recognition server, characterized in that, Comprise: S01: mark the boundary turning points in the original image of the target vehicle, connect the boundary turning points to enclose the license plate region, and optimize the license plate region; S02: mark the segmentation line in the optimized license plate region, divide the license plate region into character images through the segmentation line, and identify the recognition characters and confidence of the character images through the character recognition model; S03: based on the license plate structure rule, combine the recognition characters into a license plate string, dynamically verify the reliability of the license plate string, and mark the target string from the verified license plate string; S04: match the target string with the standard string in the preset data set, calculate the matching quality score of the target string, and determine whether to trigger the data set update operation; if not, execute S06; If the data set update operation is triggered, execute S05; S05: secondary analysis of the matching quality score, analyze the matching property of the target string, and update the preset data set according to the matching property; S06: output the license plate recognition information of the target vehicle, and execute the work instruction suitable for the license plate recognition information.

2. The closed loop license plate recognition method based on adaptive dataset according to claim 1, characterized in that, The surrounding method of the license plate region is: Take the original image of the target vehicle entering the license plate shooting area through the camera, identify the outer edge of the target vehicle in the original image through computer vision technology, and mark the area inside the outer edge as the vehicle area; Convert the image corresponding to the vehicle area to a grayscale image, denoise the grayscale image through a Gaussian filter, mark all pixel points in the grayscale image, calculate the horizontal gradient component and vertical gradient component of the pixel points based on the Canny operator, and then combine the horizontal gradient component and vertical gradient component of the pixel points to calculate the edge strength of the pixel points; Mark the pixel points with edge strength greater than the preset strength threshold as edge pixel points, obtain A edge pixel points, and draw A pixel circles with the A edge pixel points as the centers and one-fourth of the standard license plate width as the radii; Statistically count the number of edge pixel points in A pixel circles and the number of all pixel points, mark the pixel circle with the number of edge pixel points greater than one-third of the number of all pixel points as the target circle, and mark the edge pixel point corresponding to the center of the target circle as the license plate pixel point, and obtain B license plate pixel points; Draw a rectangular region on the grayscale image, adjust the rectangular region along the horizontal and vertical directions until the rectangular region just surrounds all B license plate pixel points, and obtain a character region; Draw two diagonal lines of the character region, extend the four corner points of the character region outward along the diagonal line direction by a marked interval length, obtain four boundary turning points, connect the four boundary turning points in turn along the horizontal and vertical directions, and enclose the vehicle area inside the enclosure as the license plate region.

3. The closed loop license plate recognition method based on adaptive dataset according to claim 2, characterized in that, The segmentation method of the character image is: Divide C rows and D columns in the license plate region along the horizontal and vertical directions respectively according to the height and width of a pixel point. In a top-down manner, the number of license plate pixel points in C rows is counted one by one, the row where the first license plate pixel point appears and the row where the last license plate pixel point appears are recorded as a starting row and an ending row, and the area between the starting row and the ending row is recorded as a segmentation area; In a left-to-right manner, the number of license plate pixel points in D columns is counted one by one, the column where the number of license plate pixel points is 0 is recorded as a spacing column, and after connecting the positions of the spacing columns in continuous positions in the horizontal direction, E spacing lines are obtained; In the vertical direction, a dividing line is drawn at the position of the midpoint of the E interval lines, and the dividing region is divided into character images using the dividing line as the dividing criterion.

4. The closed loop license plate recognition method based on adaptive dataset according to claim 3, characterized in that, The license plate structure rule is that the license plate string contains numbers, delimiters, letters and numbers at the same time; the reliability includes high reliability and low reliability; The marking method of the target string is: After the moment when the original image is first captured, H dynamic license plate strings of the target vehicle in the license plate shooting area are continuously collected at a preset interval time length; The first recognized license plate string is recorded as an original license plate string, and the number of dynamic license plate strings that coincide with the original license plate string is recorded as a coincidence value; When the coincidence value is greater than a coincidence threshold, the reliability of the original license plate string is high reliability, and the original license plate string is recorded as a target string; When the coincidence value is less than or equal to the coincidence threshold, the reliability of the original license plate string is low reliability, the number of coincidences in H dynamic license plate strings is counted, and the dynamic license plate string corresponding to the maximum number of coincidences is recorded as a target string.

5. The closed loop license plate recognition method based on adaptive dataset according to claim 4, characterized in that, The calculation method of the matching quality score is: In the same space, the target string and the standard string in the preset data set are converted into semantic vectors to obtain a target vector and K standard vectors; The cosine similarity between the target vector and the K standard vectors is calculated one by one through the cosine similarity calculation formula, and the maximum value of the cosine similarity is recorded as a first quality value; The target string corresponding to The second quality value is obtained by adding and averaging the confidence degrees. After the first quality value and the second quality value are respectively assigned to corresponding weight coefficients and added, the matching quality score is calculated.

6. The closed loop license plate recognition method based on adaptive dataset according to claim 5, wherein, The determination method of whether to trigger the data set update operation is: The matching quality score of the target string is compared with a calibrated matching quality lower limit value; When the matching quality score is less than the matching quality lower limit value, it is determined to trigger the data set update operation; When the matching quality score is greater than or equal to the matching quality lower limit value, it is determined not to trigger the data set update operation.

7. The method of claim 6, wherein the method further comprises: The matching property includes suspicious property and unknown property: The analysis method of the matching property is: The matching quality score of the target string is compared with a calibrated property quality lower limit value; When the matching quality score is greater than or equal to the calibrated property quality lower limit value, the matching property of the target string is analyzed as suspicious property; When the matching quality score is less than the calibrated property quality lower limit value, the matching property of the target string is analyzed as unknown property.

8. The closed loop license plate recognition method based on adaptive dataset according to claim 7, characterized in that, The update mode includes automatic update and semi-automatic update; The update method of the preset data set is: When the matching property is suspicious property, the selected update mode is automatic update; Arrange the target strings in ascending order. The confidence levels are arranged sequentially, and the characters with the highest confidence levels (top two) are marked as suspicious characters. Two text boxes are established on the target string, two suspicious characters are respectively imported into the two text boxes, a target string with suspicious remarks is generated, and the target string is imported into the preset data set for automatic update; When the matching property is an unknown property, the selected update mode is semi-automatic update; The license plate recognition server sends the target string to the manual review platform, and the reviewer manually reviews the target string to determine the arrangement order of the recognized characters in the target string; A rule box is established on the target string, the arrangement order of the recognized characters is imported into the rule box, a target string with extended rules is generated, and the target string is imported into the preset data set for semi-automatic update.

9. The closed loop license plate recognition method based on adaptive dataset according to claim 8, wherein, The license plate recognition information includes license plate number information and update completion prompt information; When the data set update operation is not triggered, the license plate number information is generated; when the data set update operation is triggered, the update completion prompt information is generated; The work instructions include a return standby instruction and a waiting recognition instruction; When the license plate recognition information is the license plate number information, the return standby instruction is executed; when the license plate recognition information is the update completion prompt information, the waiting recognition instruction is executed.

10. A closed-loop license plate recognition system based on adaptive dataset, applied to a license plate recognition server, for implementing the method of any one of claims 1-9, characterized in that, The system comprises a license plate region enclosing module, a character image segmentation module, a string marking module, an update determination module, a data set update module, and an information output module; The license plate region enclosing module is configured to mark boundary turning points in the original image of the target vehicle, connect the boundary turning points to enclose a license plate region, and perform optimization processing on the license plate region; The character image segmentation module is configured to mark a segmentation line in the license plate region after optimization processing, segment the license plate region into character images through the segmentation line, and identify recognized characters and confidence of the character images through a character recognition model; The string marking module is configured to combine the recognized characters into a license plate string based on a license plate structure rule, dynamically verify the reliability of the license plate string, and mark a target string from the verified license plate string; The update determination module is configured to match the target string with a standard string in a preset data set, calculate a matching quality score of the target string, and determine whether to trigger a data set update operation; The data set update module is configured to perform secondary analysis on the matching quality score, analyze a matching property of the target string, and select an appropriate update mode to update the preset data set according to the matching property; The information output module is configured to output license plate recognition information of the target vehicle and execute a work instruction suitable for the license plate recognition information.

Images