A vehicle vin code recognition method and system based on machine vision

Abstract

This invention discloses a vehicle VIN code recognition method and system based on machine vision, comprising: acquiring an original image of the vehicle; locating and geometrically correcting the original image to obtain a standardized image; obtaining initial recognition results and easily confused character positions based on the standardized image; generating a candidate character list for easily confused character positions based on the initial recognition results according to VIN code encoding rules; dividing the initial recognition results using optional check bits and constructing a VIN code encoding rule constraint satisfaction problem; performing hierarchical pruning based on the candidate character list through the constraint satisfaction problem; outputting candidate VIN recognition results based on the pruned list using a backtracking search algorithm; verifying the candidate VIN recognition results to obtain the VIN recognition result. This method utilizes standardized image geometric correction, temporal context feature modeling, and hierarchical constraint pruning to improve the accuracy and real-time performance of VIN code recognition, while also exhibiting good interpretability.

Description

Technical Field

[0001] This invention relates to the field of VIN code recognition, and more particularly to a vehicle VIN code recognition method and system based on machine vision. Background Technology

[0002] Vehicle Identification Number (VIN) serves as the unique identifier for motor vehicles and is widely used in scenarios such as vehicle annual inspection, registration, road checkpoint traceability, used car verification, and vehicle manufacturer production management. It is the core credential for the entire lifecycle management of motor vehicles. With the popularization of automated verification of traffic and motor vehicles, VIN recognition technology based on machine vision is gradually replacing manual data entry and has become the mainstream solution. Real-time image recognition captured by various industrial cameras and checkpoint capture devices has become a routine technical means for inspection stations and vehicle manufacturers.

[0003] Existing machine vision VIN recognition solutions mostly adopt single-character optical character recognition (OCR) mode, which has less processing of VIN encoding rules and sequence context features. In real-world shooting scenarios, they are easily affected by image tilt, blur, and character wear, resulting in low recognition accuracy. Some solutions improve the recognition rate by adding rule verification, but the candidate character screening logic is coarse. Coarse screening is prone to omissions, while fine screening is cumbersome in judging branches and the constraint pruning is inaccurate. This can easily lead to missed ground truths or recognition results that only conform to the format but do not correspond to real vehicles, resulting in fake compliant VIN codes. Meanwhile, the large amount of image information places higher demands on real-time performance and accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a vehicle VIN code recognition method and system based on machine vision.

[0005] To achieve the above objectives, the present invention is implemented according to the following technical solution: The first aspect of this invention provides a machine vision-based vehicle VIN code recognition method, comprising: Acquire the original image of the vehicle, perform localization and geometric correction on the original image to obtain a standardized image; The standardized image is subjected to image text recognition to obtain the initial recognition result of the VIN code and the character probability distribution characteristics. The positions of easily confused characters are calculated based on the character probability distribution characteristics. Based on the initial recognition results, a preset VIN code tag library is queried to obtain optional check bits and VIN code encoding rules. A candidate character list for easily confused character positions is generated according to the VIN code encoding rules. The initial recognition results are divided using the optional check bits. The vin code encoding rule constraint satisfaction problem is constructed based on the segmented word unit hierarchical structure. The hierarchical pruning is performed based on the constraint satisfaction problem according to the candidate character list. Based on the pruned list, a backtracking search algorithm is used to output candidate VIN identification results, which are then verified to obtain the final VIN identification result.

[0006] Furthermore, the method for obtaining the standardized graph includes: The original image is input into a pre-trained object detection network to locate the VIN code region and output the bounding box coordinates. Based on the bounding box coordinates, the region of interest (ROI) image is cropped from the original image. Gaussian filtering is applied to the ROI image for noise reduction. The coordinates of the four corner points of the VIN code region are extracted from the denoised image. Based on these four corner point coordinates and the coordinates of the preset target corner points, the homography matrix is ​​solved using the least squares method. The transformation formula for the homography matrix is ​​as follows: ; in The original coordinates of the region of interest. The coordinates are after transformation. Let be the perspective transformation matrix to be solved; The least-squares solution formula for the homography matrix is ​​as follows: ; in Let be the error function. For the first The x and y coordinates of the target corner points For the first A normalized coordinate, , ; Based on the homography matrix, perspective transformation is performed on the original region of interest image, and a standardized image is generated by bilinear interpolation resampling. In the standardized image, the characters are arranged horizontally, the character height occupies 60% to 80% of the image height, and the image aspect ratio is 4:1 to 5:1. The object detection network is selected from YOLOv8n, YOLOv8s, or Faster R-CNN.

[0007] Furthermore, the method for obtaining the position of the easily confused character includes: The standardized image is input into the backbone network of the object detection network to extract deep feature maps. Based on the character recognition branch of the object detection network, the deep visual feature maps are modeled in context through a temporal modeling layer to capture the sequential dependency and encoding rule association between VIN code characters. The modeled sequence features are mapped to the initial VIN code recognition result through a character classification layer, and the confidence probability distribution of all legal characters at each character position is output as the character probability distribution feature. The temporal modeling layer includes two layers of bidirectional long short-term memory network or Transformer encoder. The character recognition branch is parallel to the bounding box coordinate output branch. Based on the character probability distribution characteristics, the confidence probability values ​​of legal characters at each character position are squared and summed. The sum of squares is subtracted from the value 1 to obtain the Gini impurity index. The ratio of the second highest confidence probability value to the highest confidence probability value at each character position is used as the Top-2 concentration index. The Gini impurity index and concentration index of each character position are sorted in descending order. A first candidate set and a second candidate set are selected according to the preset number of positions. The result of the set operation between the first candidate set and the second candidate set is used as the easily confused character position. The set operation includes union and intersection.

[0008] Further, the method for obtaining the candidate character list includes: Based on the first three characters of the initial recognition result, a preset VIN code tag library is queried to obtain the corresponding manufacturer information, vehicle type identifier, check digit rule identifier, and VIN code encoding rule. Based on the check digit rule identifier, the check digit verification value is calculated according to the first to eighth and tenth to seventeenth characters of the initial recognition result. The calculated check digit verification value is compared with the ninth character of the initial recognition result. If they do not match, the check digit verification value is used as an optional check digit. If they match, the ninth character is used as the only optional check digit. The VIN code encoding rule includes prohibited character rules, location-related character set rules, year code rules, and vehicle feature compatibility rules. Based on the location of easily confused characters, the candidate character list is obtained by filtering according to the location-related character set rules. If the optional check bit belongs to the location of easily confused characters, the optional check bit is added to the candidate character list.

[0009] Furthermore, the method for obtaining the constraint satisfaction problem includes: If the optional check bit is a single character, then the 9th character of the initial recognition result is replaced. If the optional check bit is multiple characters, then the 9th character of the initial recognition result is replaced with a candidate character of the optional check bit, generating a preliminary corrected vin segment sequence. The vin segment sequence is greater than or equal to one set. The initially revised VIN segment sequence is divided into fixed segments, with the 17 characters divided into WMI segment, VDS segment, check segment, and VIS segment. The WMI segment, VDS segment, and VIS segment are respectively mapped to the first-level word, the second-level word, and the third-level word. The check segment is treated as an independent word. Each segment word is associated with the corresponding VIN code encoding rule. The WMI segment is the manufacturer identification segment from the 1st to the 3rd position, the VDS segment is the vehicle feature description segment from the 4th to the 8th position, and the VIS segment is the vehicle production instruction segment from the 10th to the 17th position. Each segment word is used as a basic variable, the segment word corresponding to the position of easily confused character is used as a variable to be optimized, and the segment word corresponding to the position of non-easily confused character is used as a fixed variable. The value range of the variable to be optimized is the candidate character list, and the value range of the fixed variable is the unique character at the corresponding position in the initial recognition result. Based on the VIN code encoding rules, a hierarchical constraint structure is constructed in a progressive manner to obtain the VIN code encoding rule constraint satisfaction problem. The constraint hierarchy includes a global constraint layer, a segmented local constraint layer, and a check bit constraint layer. The global constraint layer includes prohibited character rules and WMI segment manufacturer information. The segmented local constraint layer includes location-related character set rules and vehicle feature compatibility rules. The location-related character set rules include retaining only the valid year code in the 10th position of the VIS segment and only retaining numbers in the 12th to 17th positions. The vehicle feature compatibility rule is that the characters in the VDS segment match the vehicle type identifier corresponding to the WMI segment. The check bit constraint layer is that the check bit segment retains only the characters in the optional check bits, and the check bit verification value satisfies the modulo 11 check algorithm.

[0010] Furthermore, the method for obtaining the pruning list includes: Based on the candidate character list, query the manufacturer information for the first-level word corresponding to the variable to be optimized by satisfying the constraint problem, verify whether the current candidate character combination exists in the manufacturer information, and if not, remove the candidate character combination from the value range of the variable to be optimized and update the candidate character list. For the second-level word element, the current value of the variable to be optimized is verified to match the fixed variable of the first-level word element through the vehicle feature compatibility rule. For the third-level word element, the current value of the variable to be optimized is verified to conform to the year code rule and the number rule through the position related character set rule. If the verification fails, the candidate value that does not conform to the rule is removed from the value range of the corresponding variable to be optimized. The expected value of the check position is calculated for the variable to be optimized corresponding to the check position segment using the already assigned variables of the first-level word, second-level word, and third-level word. The value range of the variable to be optimized in the check position segment is intersected with the optional check position, and only the candidate values ​​that satisfy the modulo 11 check algorithm are retained. If the value range has no intersection with the optional check position, or the current assignment causes the check position verification to fail, backtracking is performed and the previous level is returned to reassign the value. The already assigned variables include fixed variables and the already assigned variable to be optimized. When assigning a value to the variable to be optimized, check the unassigned variables to be optimized related to the variable to be optimized, delete the candidate values ​​that conflict with the assignment from the relevant value range, and if the value range of the variable to be optimized becomes an empty set, backtrack and try the next candidate value of the variable to be optimized to obtain the pruning list.

[0011] Furthermore, the method for obtaining the VIN recognition result includes: Based on the pruned list, a backtracking search algorithm is used to traverse the remaining solution space. A composite heuristic strategy is then used to select the next variable to be optimized for priority assignment. The variable formula for the composite heuristic strategy is as follows: ; in Assign a value to the selected priority variable. This is the set of currently unassigned variables, obtained from the remaining variables to be optimized in the pruning list. For variables The number of constraints involved is obtained according to the VIN code encoding rules, which determine the total number of constraints involved in the problem. For variables The number of candidate characters in the current value range is obtained based on the size of the remaining range after the candidate character list has undergone hierarchical pruning. For the first Rank The confidence probability value of each legal character is obtained based on the character probability distribution characteristics. For the first The probability value of the character with the highest positional confidence. For the first The character probability value with the second highest positional confidence; If a candidate VIN is generated for each complete assignment during the backtracking search process, the VIN code is verified against the vehicle certificate database based on the candidate VIN recognition results. If two or more candidate VIN recognition results pass the verification, the candidate VIN recognition result with the highest confidence score from the OCR is taken as the VIN recognition result. If all candidate VIN recognition results fail the verification, manual review is performed to obtain the VIN recognition result.

[0012] A second aspect of the present invention provides a vehicle VIN code recognition system based on machine vision, comprising: Image acquisition module: used to acquire the original image of the vehicle, perform localization and geometric correction on the original image, and obtain a standardized image; Easily confused character extraction module: used to perform image text recognition on the standardized image, obtain the initial recognition result of the VIN code and the character probability distribution characteristics, and calculate the position of easily confused characters based on the character probability distribution characteristics; Candidate character list extraction module: used to query a preset VIN code tag library based on the initial recognition result, obtain optional check bits and VIN code encoding rules, and generate a candidate character list for easily confused character positions according to the VIN code encoding rules; Constraint Satisfaction Problem Module: Used to divide the initial recognition result using the optional check bits, construct the vin code encoding rule constraint satisfaction problem based on segmented word units, and perform hierarchical pruning based on the candidate character list through the constraint satisfaction problem; VIN code recognition module: Used to output candidate VIN recognition results based on the pruned list through a backtracking search algorithm, verify the candidate VIN recognition results, and obtain the VIN recognition result.

[0013] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects: This invention corrects the tilt, distortion, and blur of VIN images through standardized image preprocessing and bidirectional LSTM temporal context modeling, captures character sequence association features, reduces the error of isolated single character recognition, and improves the reliability of initial recognition results. It retains high-probability candidates through high-recall rules to avoid missing true values, eliminates illegal characters through three-level constraints of global, segmented, and check bits, reduces the subsequent search solution space, and reduces the computational cost of the algorithm. It optimizes the assignment logic of easily confused characters through composite heuristic backtracking search, improving search efficiency and reducing invalid traversals. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the steps of a machine vision-based vehicle VIN code recognition method in an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0016] Reference Figure 1 As shown, this invention provides a vehicle VIN code recognition method based on machine vision, comprising: Acquire the original image of the vehicle, perform localization and geometric correction on the original image to obtain a standardized image; In the actual evaluation, original images of the vehicle were captured at a resolution of 1920×1080 pixels in RGB888 format. The VIN code region in the original image was located in the lower left corner of the image. Due to the vehicle's parking angle and shooting perspective, the characters were tilted and slightly blurred by dust. The original image was input into a pre-trained YOLOv8n object detection network. The network output the bounding box coordinates of the VIN code region: x_min=220, y_min=480, x_max=1460, y_max=660. Based on the bounding box coordinates, a Region of Interest (ROI) image of the VIN code was cropped from the original image, with a size of 1240×180 pixels. The cropped ROI image was then subjected to Gaussian filtering for noise removal, with a Gaussian convolution kernel of 3×3 and a standard deviation of 1.2. This removed dust noise and motion blur noise from the image acquisition process while preserving the edge contour of the VIN character. The VIN code was then extracted from the denoised ROI image through edge detection and contour fitting. The coordinates of the four actual corner points of the region are: top left corner P1 (225, 485), top right corner P2 (1455, 500), bottom right corner P3 (1450, 655), and bottom left corner P4 (220, 650). The target corner point coordinates of the preset standard VIN image are: top left Q1 (0, 0), top right Q2 (1200, 0), bottom right Q3 (1200, 260), and bottom left Q4 (0, 260). Based on these four sets of actual corner points and target corner coordinates... The homography matrix was solved using the least squares method. The resulting homography matrix H is a 3×3 perspective transformation matrix, and the error function converges to 0.32. Based on the solved homography matrix H, the original ROI image was subjected to perspective transformation, and the image was resampled using bilinear interpolation to generate a standardized image. The standardized image has a size of 1200×260 pixels, an aspect ratio of 4.62:1, and a character size of 182 pixels, accounting for 70% of the total image height. The characters are arranged horizontally and neatly. The standardized image is subjected to image text recognition to obtain the initial recognition result of the VIN code and the character probability distribution characteristics. The positions of easily confused characters are calculated based on the character probability distribution characteristics. In the actual evaluation, standardized images were input into the backbone network of the YOLOv8n object detection network, where the backbone network is a C2f module. This extracted 16-channel deep visual feature maps. The deep feature maps were then input into a two-layer bidirectional LSTM temporal modeling layer to perform contextual association modeling on the sequence features of the vin character. The temporally modeled features were then input into a fully connected character classification layer, mapping to obtain an initial 17-bit vin code recognition result. The output included the confidence probability distribution of 33 classes of legal vin characters at each position. The 33 classes of legal vin characters are 0-9 and AZ, excluding I, O, and Q. The initial recognition result was LBV6B2108PSF08793. Based on the character probability distribution... The Gini impurity index and concentration index are calculated position by position for each feature. In the Gini impurity index, G=0.62 for the 9th position, G=0.58 for the 7th position, G=0.54 for the 12th position, and G≤0.15 for the remaining positions. In the Top-2 concentration index, T=0.90 for the 9th position, T=0.79 for the 7th position, T=0.69 for the 12th position, and T≤0.10 for the remaining positions. The two indices are sorted in descending order, and candidate sets are selected according to a preset number of positions (3). The first candidate set is the 9th, 7th, and 12th positions, and the second candidate set is also the 9th, 7th, and 12th positions. The two candidate sets are processed by union operation to obtain the easily confused character positions as the 7th, 9th, and 12th positions. Based on the initial recognition results, a preset VIN code tag library is queried to obtain optional check bits and VIN code encoding rules. A candidate character list for easily confused character positions is generated according to the VIN code encoding rules. In the actual evaluation, the LBV of the first three characters of the WMI manufacturer identification segment in the initial identification result is extracted. The manufacturer information, vehicle type identifier, check digit rule identifier, and VIN code encoding rule are obtained by querying the preset VIN code tag library. The LBV is the legally registered WMI code, the vehicle type identifier is passenger car, and the prohibited character rule in the VIN code encoding rule excludes I, O, and Q. In the position-related character set rule, the 10th character of the VIS segment is a legal year code, and characters 12-17 are pure numbers. In the year code rule, the 10th character P of the VIS corresponds to 2023. The VDS segment in the vehicle feature compatibility rule... The 4th-8th bits match the car coding specification. Based on the check bit rule identifier, the 1st-8th bits (LBV6B210) and the 10th-17th bits (PSF08793) of the initial recognition result are taken. The check bit verification value is calculated according to the VIN modulo 11 check algorithm. The weights of the 1st-8th and 10th-17th bits in the modulo 11 check weight are [8,7,6,5,4,3,2,10,9,8,7,6,5,4,3,2]. The weighted summation yields the character mapping value: L=12,B=2,V=22,6=6,B=2,2=2,1=1,0=0,P=15,S=19,F=6 Given 0=0, 8=8, 7=7, 9=9, 3=3, the weighted sum is 827. Modulo 11, 827%1=2, so the check bit verification value is 2. Comparing this with the initial recognition result (9th bit 8), they are inconsistent. Therefore, the selectable check bit set is {2}. Layered filtering is performed on easily confused character positions using VIN code encoding rules. The original recognized character 1 in the 7th bit has 7 as an easily confused candidate, belonging to bits 4-8 of the VDS segment. Matching the car feature compatibility rule, the VDS segment allows numbers plus letters, with no pure number restriction. Therefore, 1 and 7 are retained. The original recognized character 8 is the 9th check bit. The easily confused candidate is X, which belongs to a special segment of the check bit. Only characters in the optional check bit set are retained, so it is replaced with 2. The original recognized character of the 12th bit is 0. The easily confused candidate is 8, which belongs to the 12th bit of the VIS segment. It must meet the pure number rule and the corresponding confidence probability is high, so 0 and 8 are retained. The 7th bit of the candidate character list is {1,7}, which is based on the VDS segment vehicle feature compatibility rule and character probability distribution feature. The 9th bit is {2}, which is based on the modulo 11 check algorithm and optional check bit rule. The 12th bit is {0,8}, which is based on the VIS segment pure number rule and character probability distribution feature. The initial recognition results are divided using the optional check bits. The vin code encoding rule constraint satisfaction problem is constructed based on the segmented word unit hierarchical structure. The hierarchical pruning is performed based on the constraint satisfaction problem according to the candidate character list. In the actual evaluation, the 9th character of the initial recognition result was directly replaced by the optional check bit {2}, resulting in the corrected VIN code sequence LBV6B2102PSF08793. The 17 characters were then divided into fixed segments according to the ISO 3779 VIN code standard, defining segmented terms. The first-level terminology consisted of the 1st to 3rd positions of the WMI segment, with LBV as a fixed variable, a non-confusional position, and a unique value range. The second-level terminology consisted of the 4th to 8th positions of the VDS segment (6B210), with the 7th position being the variable to be optimized, taking a value range of {1,7}, and the remaining positions being fixed variables. The third-level terminology consisted of the 10th to 17th positions of the VIS segment (PSF08793), with the 12th position being... The variable to be optimized has a value range of {0, 8}, while the remaining positions are fixed variables. The 9th special term in the check bit is the variable to be optimized, with a value range of {2}. A constraint satisfaction problem CSP is constructed, where the fixed variables are the non-confusional positions of positions 1-6, 8, 10-11, and 13-17, with a value range corresponding to the characters in the initial recognition result. The variables to be optimized are the VDS term in position 7, the check bit term in position 9, and the VIS term in position 12, with corresponding domain sets of {1, 7}, {2}, and {0, 8}. The global constraint layer disables the characters I / O / Q. In the segmented local constraint layer, the VDS segment must match the car type identifier corresponding to WMI=LBV. In the VIS segment, the 10th position P is a valid year code, and the 12th-17th positions are pure numbers. In the check bit constraint layer, the 9th check bit must satisfy the modulo 11 check algorithm, with a value range that only retains the optional check bit {2}. Candidates are evaluated according to the hierarchical dependency order of WMI→VDS→VIS→check bit. The character list is progressively pruned. If there are no candidates that do not meet the constraints in the first-level, second-level, and third-level lexical units, the candidate is retained. When assigning a value to the 7th variable to be optimized, the 12th unassigned variable is checked simultaneously. The assigned value 1 does not conflict with the 12th candidate {0,8}, so the 12th candidate domain is retained. The assigned value 7 does not conflict with the 12th candidate {0,8}, so the 12th candidate domain is retained. There are no cases where the variable value domain is an empty set, so no backtracking is required, and the pruning is completed. In the actual evaluation, the corrected base vin sequence obtained from the original image of another vehicle is LSVNY2A49CT2683106. The 17 characters are divided into four levels of segmented words, with hierarchical and variable attributes bound. The first-level word (WMI manufacturing plant segment, bits 1-3) is LSV, a fixed variable, non-confusionally sensitive, with a unique value, and does not participate in optimization. The second-level word (VDS vehicle feature segment, bits 4-8) is NY2A4, with bit 5 being the variable to be optimized, candidate domain {Y,A,Q}. The third-level word (VIS production instruction segment, bits 10-...) The 17th bit is CT2683106, the 13th bit is the variable to be optimized, the candidate domain is {6,G}, the independent check bit word (the 9th bit) is 9, the variable to be optimized, the candidate domain is {4,X,9}. The candidate character list is progressively pruned according to the hierarchical dependency order of WMI→VDS→VIS→check bit. In each round, candidate characters that violate the corresponding constraints are removed, gradually shrinking the value range of the variable to be optimized. In the first round of global constraint pruning, all candidate domains of the variable to be optimized are traversed, filtering characters that violate the global prohibition rules. In the 5th candidate domain {Y,A,Q}, Q belongs to the current... The `vin` field is a banned character. The character `Q` is removed, narrowing the candidate field to `{Y,A}`. The 9th and 13th candidates do not contain globally banned characters and are not removed. The pruned intermediate candidate fields are: 5th position `{Y,A}`, 9th position `{4,X}`, and 13th position `{6,G}`. The second round of segmented local constraint pruning verifies the matching of candidate characters with segment attributes according to segment-specific rules, removing violations. The remaining candidates `Y` and `A` for the 5th position (VDS segment) both conform to the passenger car VDS coding standard and are not removed. The candidate `G` for the 13th position (VIS segment) is a letter, violating the VIS segment's pure numeric rule. After removing G, the candidate domain shrinks to {6}, resulting in the intermediate candidate domains being the 5th bit {Y,A}, the 9th bit {4,X}, and the 13th bit {6}. The third round of check bit constraint pruning is based on the fixed WMI, VDS, and VIS segment characters, recalculating the modulo 11 check value and retaining only the compliant check bits. The 9th candidate domain {4,X} has no intersection with the valid check bit 9, so 4 and X are removed, and the candidate domain is updated to the compliant value {9}. There are no cross-segment conflicts in the assignment of associated variables, so there is no need to trigger the backtracking process. The pruned list is obtained as the 5th bit {Y,A}, the 9th bit {9}, and the 13th bit {6}. Based on the pruned list, a backtracking search algorithm is used to output candidate VIN identification results, which are then verified to obtain the final VIN identification result.

[0017] In the actual evaluation, the pruned list is used as the solution space. Based on the pruned list, a backtracking search algorithm is used to traverse the remaining solution space. A composite heuristic strategy is used to select the next variable to be optimized for priority assignment. Among them, the 7th bit (VDS segment, denoted as x1), the 9th bit (check bit, denoted as x2), and the 12th bit (VIS segment, denoted as x3) are fixed variables, which are the remaining 14 characters, and their values ​​remain unchanged. The candidate field of x2 has only 1 value, and its constraint weight is the highest, so it is assigned the value 2 first. The candidate fields of x1 and x3 both have 2 values. They are sorted by Gini impurity, with x1 having higher priority than x3. The values ​​are assigned sequentially. After traversal, there are no assignment conflicts or empty value fields. No backtracking is required, and all legal combinations are directly generated. A total of 4 complete sets of 17 are generated. The candidate VIN sequences were 2×1×2 combinations with no illegal or redundant sequences. The VIN identification results were LBV6B212PSF08793, LBV6B212PSF88793, LBV6B212PSF88793, and LBV6B272PSF88793, with corresponding joint model confidence scores of 0.1373, 0.0948, 0.1082, and 0.0748, respectively. The joint confidence score is the product of the confidence scores of easily confused characters, and the confidence scores at fixed positions are all ≥0.92. The four candidate sequences were verified using the vehicle certificate database. Candidate 1 passed the verification, while candidates 2-4 failed the verification. The VIN identification result LBV6B2102PSF08793 was obtained.

[0018] In this embodiment, the method for obtaining the standardized graph includes: The original image is input into a pre-trained object detection network to locate the VIN code region and output the bounding box coordinates. Based on the bounding box coordinates, the region of interest (ROI) image is cropped from the original image. Gaussian filtering is applied to the ROI image for noise reduction. The coordinates of the four corner points of the VIN code region are extracted from the denoised image. Based on these four corner point coordinates and the coordinates of the preset target corner points, the homography matrix is ​​solved using the least squares method. The transformation formula for the homography matrix is ​​as follows: ; in The original coordinates of the region of interest. The coordinates are after transformation. Let be the perspective transformation matrix to be solved; The least-squares solution formula for the homography matrix is ​​as follows: ; in Let be the error function. For the first The x and y coordinates of the target corner points For the first A normalized coordinate, , ; Based on the homography matrix, perspective transformation is performed on the original region of interest image, and a standardized image is generated by bilinear interpolation resampling. In the standardized image, the characters are arranged horizontally, the character height occupies 60% to 80% of the image height, and the image aspect ratio is 4:1 to 5:1. The object detection network is selected from YOLOv8n, YOLOv8s, or Faster R-CNN.

[0019] In this embodiment, the method for obtaining the position of the easily confused character includes: The standardized image is input into the backbone network of the object detection network to extract deep feature maps. Based on the character recognition branch of the object detection network, the deep visual feature maps are modeled in context through a temporal modeling layer to capture the sequential dependency and encoding rule association between VIN code characters. The modeled sequence features are mapped to the initial VIN code recognition result through a character classification layer, and the confidence probability distribution of all legal characters at each character position is output as the character probability distribution feature. The temporal modeling layer includes two layers of bidirectional long short-term memory network or Transformer encoder. The character recognition branch is parallel to the bounding box coordinate output branch. Based on the character probability distribution characteristics, the confidence probability values ​​of legal characters at each character position are squared and summed. The sum of squares is subtracted from the value 1 to obtain the Gini impurity index. The ratio of the second highest confidence probability value to the highest confidence probability value at each character position is used as the Top-2 concentration index. The Gini impurity index and concentration index of each character position are sorted in descending order. A first candidate set and a second candidate set are selected according to the preset number of positions. The result of the set operation between the first candidate set and the second candidate set is used as the easily confused character position. The set operation includes union and intersection.

[0020] In this embodiment, the method for obtaining the candidate character list includes: Based on the first three characters of the initial recognition result, a preset VIN code tag library is queried to obtain the corresponding manufacturer information, vehicle type identifier, check digit rule identifier, and VIN code encoding rule. Based on the check digit rule identifier, the check digit verification value is calculated according to the first to eighth and tenth to seventeenth characters of the initial recognition result. The calculated check digit verification value is compared with the ninth character of the initial recognition result. If they do not match, the check digit verification value is used as an optional check digit. If they match, the ninth character is used as the only optional check digit. The VIN code encoding rule includes prohibited character rules, location-related character set rules, year code rules, and vehicle feature compatibility rules. Based on the location of easily confused characters, the candidate character list is obtained by filtering according to the location-related character set rules. If the optional check bit belongs to the location of easily confused characters, the optional check bit is added to the candidate character list.

[0021] In this embodiment, the method for obtaining the constraint satisfaction problem includes: If the optional check bit is a single character, then the 9th character of the initial recognition result is replaced. If the optional check bit is multiple characters, then the 9th character of the initial recognition result is replaced with a candidate character of the optional check bit, generating a preliminary corrected vin segment sequence. The vin segment sequence is greater than or equal to one set. The initially revised VIN segment sequence is divided into fixed segments, with the 17 characters divided into WMI segment, VDS segment, check segment, and VIS segment. The WMI segment, VDS segment, and VIS segment are respectively mapped to the first-level word, the second-level word, and the third-level word. The check segment is treated as an independent word. Each segment word is associated with the corresponding VIN code encoding rule. The WMI segment is the manufacturer identification segment from the 1st to the 3rd position, the VDS segment is the vehicle feature description segment from the 4th to the 8th position, and the VIS segment is the vehicle production instruction segment from the 10th to the 17th position. Each segment word is used as a basic variable, the segment word corresponding to the position of easily confused character is used as a variable to be optimized, and the segment word corresponding to the position of non-easily confused character is used as a fixed variable. The value range of the variable to be optimized is the candidate character list, and the value range of the fixed variable is the unique character at the corresponding position in the initial recognition result. Based on the VIN code encoding rules, a hierarchical constraint structure is constructed in a progressive manner to obtain the VIN code encoding rule constraint satisfaction problem. The constraint hierarchy includes a global constraint layer, a segmented local constraint layer, and a check bit constraint layer. The global constraint layer includes prohibited character rules and WMI segment manufacturer information. The segmented local constraint layer includes location-related character set rules and vehicle feature compatibility rules. The location-related character set rules include retaining only the valid year code in the 10th position of the VIS segment and only retaining numbers in the 12th to 17th positions. The vehicle feature compatibility rule is that the characters in the VDS segment match the vehicle type identifier corresponding to the WMI segment. The check bit constraint layer is that the check bit segment retains only the characters in the optional check bits, and the check bit verification value satisfies the modulo 11 check algorithm.

[0022] In this embodiment, the method for obtaining the pruning list includes: Based on the candidate character list, query the manufacturer information for the first-level word corresponding to the variable to be optimized by satisfying the constraint problem, verify whether the current candidate character combination exists in the manufacturer information, and if not, remove the candidate character combination from the value range of the variable to be optimized and update the candidate character list. For the second-level word element, the current value of the variable to be optimized is verified to match the fixed variable of the first-level word element through the vehicle feature compatibility rule. For the third-level word element, the current value of the variable to be optimized is verified to conform to the year code rule and the number rule through the position related character set rule. If the verification fails, the candidate value that does not conform to the rule is removed from the value range of the corresponding variable to be optimized. The expected value of the check position is calculated for the variable to be optimized corresponding to the check position segment using the already assigned variables of the first-level word, second-level word, and third-level word. The value range of the variable to be optimized in the check position segment is intersected with the optional check position, and only the candidate values ​​that satisfy the modulo 11 check algorithm are retained. If the value range has no intersection with the optional check position, or the current assignment causes the check position verification to fail, backtracking is performed and the previous level is returned to reassign the value. The already assigned variables include fixed variables and the already assigned variable to be optimized. When assigning a value to the variable to be optimized, check the unassigned variables to be optimized related to the variable to be optimized, delete the candidate values ​​that conflict with the assignment from the relevant value range, and if the value range of the variable to be optimized becomes an empty set, backtrack and try the next candidate value of the variable to be optimized to obtain the pruning list.

[0023] In this embodiment, the method for obtaining the VIN recognition result includes: Based on the pruned list, a backtracking search algorithm is used to traverse the remaining solution space. A composite heuristic strategy is then used to select the next variable to be optimized for priority assignment. The variable formula for the composite heuristic strategy is as follows: ; in Assign a value to the selected priority variable. This is the set of currently unassigned variables, obtained from the remaining variables to be optimized in the pruning list. For variables The number of constraints involved is obtained according to the VIN code encoding rules, which determine the total number of constraints involved in the problem. For variables The number of candidate characters in the current value range is obtained based on the size of the remaining range after the candidate character list has undergone hierarchical pruning. For the first Rank The confidence probability value of each legal character is obtained based on the character probability distribution characteristics. For the first The probability value of the character with the highest positional confidence. For the first The character probability value with the second highest positional confidence; If a candidate VIN is generated for each complete assignment during the backtracking search process, the VIN code is verified against the vehicle certificate database based on the candidate VIN recognition results. If two or more candidate VIN recognition results pass the verification, the candidate VIN recognition result with the highest confidence score from the OCR is taken as the VIN recognition result. If all candidate VIN recognition results fail the verification, manual review is performed to obtain the VIN recognition result.

[0024] A second aspect of the present invention also provides a vehicle VIN code recognition system based on machine vision, comprising: Image acquisition module: used to acquire the original image of the vehicle, perform localization and geometric correction on the original image, and obtain a standardized image; Easily confused character extraction module: used to perform image text recognition on the standardized image, obtain the initial recognition result of the VIN code and the character probability distribution characteristics, and calculate the position of easily confused characters based on the character probability distribution characteristics; Candidate character list extraction module: used to query a preset VIN code tag library based on the initial recognition result, obtain optional check bits and VIN code encoding rules, and generate a candidate character list for easily confused character positions according to the VIN code encoding rules; Constraint Satisfaction Problem Module: Used to divide the initial recognition result using the optional check bits, construct the vin code encoding rule constraint satisfaction problem based on segmented word units, and perform hierarchical pruning based on the candidate character list through the constraint satisfaction problem; VIN code recognition module: Used to output candidate VIN recognition results based on the pruned list through a backtracking search algorithm, verify the candidate VIN recognition results, and obtain the VIN recognition result.

[0025] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

Claims

1. A vehicle VIN code recognition method based on machine vision, characterized in that, Includes the following steps: Acquire the original image of the vehicle, perform localization and geometric correction on the original image to obtain a standardized image; The standardized image is subjected to image text recognition to obtain the initial recognition result of the VIN code and the character probability distribution characteristics. The positions of easily confused characters are calculated based on the character probability distribution characteristics. Based on the initial recognition results, a preset VIN code tag library is queried to obtain optional check bits and VIN code encoding rules. A candidate character list for easily confused character positions is generated according to the VIN code encoding rules. The initial recognition results are divided using the optional check bits. The vin code encoding rule constraint satisfaction problem is constructed based on the segmented word unit hierarchical structure. The hierarchical pruning is performed based on the constraint satisfaction problem according to the candidate character list. Based on the pruned list, a backtracking search algorithm is used to output candidate VIN identification results, which are then verified to obtain the final VIN identification result.

2. The vehicle VIN code recognition method based on machine vision according to claim 1, characterized in that, The method for obtaining the standardized graph includes: The original image is input into a pre-trained object detection network to locate the VIN code region and output the bounding box coordinates. Based on the bounding box coordinates, the region of interest (ROI) image is cropped from the original image. Gaussian filtering is applied to the ROI image for noise reduction. The coordinates of the four corner points of the VIN code region are extracted from the denoised image. Based on these four corner point coordinates and the coordinates of the preset target corner points, the homography matrix is ​​solved using the least squares method. The transformation formula for the homography matrix is ​​as follows: ； in The original coordinates of the region of interest. The coordinates are after transformation. Let be the perspective transformation matrix to be solved; The least-squares solution formula for the homography matrix is ​​as follows: ； in Let be the error function. For the first The x and y coordinates of the target corner points For the first A normalized coordinate, , ; Based on the homography matrix, perspective transformation is performed on the original region of interest image, and a standardized image is generated by bilinear interpolation resampling. In the standardized image, the characters are arranged horizontally, the character height occupies 60% to 80% of the image height, and the image aspect ratio is 4:1 to 5:

1. The object detection network is selected from YOLOv8n, YOLOv8s, or Faster R-CNN.

3. The vehicle VIN code recognition method based on machine vision according to claim 1, characterized in that, The method for obtaining the position of the easily confused character includes: The standardized image is input into the backbone network of the object detection network to extract deep feature maps. Based on the character recognition branch of the object detection network, the deep visual feature maps are modeled in context through a temporal modeling layer to capture the sequential dependency and encoding rule association between VIN code characters. The modeled sequence features are mapped to the initial VIN code recognition result through a character classification layer, and the confidence probability distribution of all legal characters at each character position is output as the character probability distribution feature. The temporal modeling layer includes two layers of bidirectional long short-term memory network or Transformer encoder. The character recognition branch is parallel to the bounding box coordinate output branch. Based on the character probability distribution characteristics, the confidence probability values ​​of legal characters at each character position are squared and summed. The sum of squares is subtracted from the value 1 to obtain the Gini impurity index. The ratio of the second highest confidence probability value to the highest confidence probability value at each character position is used as the Top-2 concentration index. The Gini impurity index and concentration index of each character position are sorted in descending order. A first candidate set and a second candidate set are selected according to the preset number of positions. The result of the set operation between the first candidate set and the second candidate set is used as the easily confused character position. The set operation includes union and intersection.

4. The vehicle VIN code recognition method based on machine vision according to claim 1, characterized in that, The method for obtaining the candidate character list includes: Based on the first three characters of the initial recognition result, a preset VIN code tag library is queried to obtain the corresponding manufacturer information, vehicle type identifier, check digit rule identifier, and VIN code encoding rule. Based on the check digit rule identifier, the check digit verification value is calculated according to the first to eighth and tenth to seventeenth characters of the initial recognition result. The calculated check digit verification value is compared with the ninth character of the initial recognition result. If they do not match, the check digit verification value is used as an optional check digit. If they match, the ninth character is used as the only optional check digit. The VIN code encoding rule includes prohibited character rules, location-related character set rules, year code rules, and vehicle feature compatibility rules. Based on the location of easily confused characters, the candidate character list is obtained by filtering according to the location-related character set rules. If the optional check bit belongs to the location of easily confused characters, the optional check bit is added to the candidate character list.

5. The vehicle VIN code recognition method based on machine vision according to claim 1, characterized in that, A method for obtaining the constraint satisfaction problem includes: If the optional check bit is a single character, then the 9th character of the initial recognition result is replaced. If the optional check bit is multiple characters, then the 9th character of the initial recognition result is replaced with a candidate character of the optional check bit, generating a preliminary corrected vin segment sequence. The vin segment sequence is greater than or equal to one set. The initially revised VIN segment sequence is divided into fixed segments, with the 17 characters divided into WMI segment, VDS segment, check segment, and VIS segment. The WMI segment, VDS segment, and VIS segment are respectively mapped to the first-level word, the second-level word, and the third-level word. The check segment is treated as an independent word. Each segment word is associated with the corresponding VIN code encoding rule. The WMI segment is the manufacturer identification segment from the 1st to the 3rd position, the VDS segment is the vehicle feature description segment from the 4th to the 8th position, and the VIS segment is the vehicle production instruction segment from the 10th to the 17th position. Each segment word is used as a basic variable, the segment word corresponding to the position of easily confused character is used as a variable to be optimized, and the segment word corresponding to the position of non-easily confused character is used as a fixed variable. The value range of the variable to be optimized is the candidate character list, and the value range of the fixed variable is the unique character at the corresponding position in the initial recognition result. Based on the VIN code encoding rules, a hierarchical constraint structure is constructed in a progressive manner to obtain the VIN code encoding rule constraint satisfaction problem. The constraint hierarchy includes a global constraint layer, a segmented local constraint layer, and a check bit constraint layer. The global constraint layer includes prohibited character rules and WMI segment manufacturer information. The segmented local constraint layer includes location-related character set rules and vehicle feature compatibility rules. The location-related character set rules include retaining only the valid year code in the 10th position of the VIS segment and only retaining numbers in the 12th to 17th positions. The vehicle feature compatibility rule is that the characters in the VDS segment match the vehicle type identifier corresponding to the WMI segment. The check bit constraint layer is that the check bit segment retains only the characters in the optional check bits, and the check bit verification value satisfies the modulo 11 check algorithm.

6. The vehicle VIN code recognition method based on machine vision according to claim 1, characterized in that, The method for obtaining the pruning list includes: Based on the candidate character list, query the manufacturer information for the first-level word corresponding to the variable to be optimized by satisfying the constraint problem, verify whether the current candidate character combination exists in the manufacturer information, and if not, remove the candidate character combination from the value range of the variable to be optimized and update the candidate character list. For the second-level word element, the current value of the variable to be optimized is verified to match the fixed variable of the first-level word element through the vehicle feature compatibility rule. For the third-level word element, the current value of the variable to be optimized is verified to conform to the year code rule and the number rule through the position related character set rule. If the verification fails, the candidate value that does not conform to the rule is removed from the value range of the corresponding variable to be optimized. The expected value of the check position is calculated for the variable to be optimized corresponding to the check position segment using the already assigned variables of the first-level word, second-level word, and third-level word. The value range of the variable to be optimized in the check position segment is intersected with the optional check position, and only the candidate values ​​that satisfy the modulo 11 check algorithm are retained. If the value range has no intersection with the optional check position, or the current assignment causes the check position verification to fail, backtracking is performed and the previous level is returned to reassign the value. The already assigned variables include fixed variables and the already assigned variable to be optimized. When assigning a value to the variable to be optimized, check the unassigned variables to be optimized related to the variable to be optimized, delete the candidate values ​​that conflict with the assignment from the relevant value range, and if the value range of the variable to be optimized becomes an empty set, backtrack and try the next candidate value of the variable to be optimized to obtain the pruning list.

7. The vehicle VIN code recognition method based on machine vision according to claim 1, characterized in that, The method for obtaining the VIN recognition result includes: Based on the pruned list, a backtracking search algorithm is used to traverse the remaining solution space. A composite heuristic strategy is then used to select the next variable to be optimized for priority assignment. The variable formula for the composite heuristic strategy is as follows: ; in Assign a value to the selected priority variable. This is the set of currently unassigned variables, obtained from the remaining variables to be optimized in the pruning list. For variables The number of constraints involved is obtained according to the VIN code encoding rules, which determine the total number of constraints involved in the problem. For variables The number of candidate characters in the current value range is obtained based on the size of the remaining range after the candidate character list has undergone hierarchical pruning. For the first Rank The confidence probability value of each legal character is obtained based on the character probability distribution characteristics. For the first The probability value of the character with the highest positional confidence. For the first The character probability value with the second highest positional confidence; If a candidate VIN is generated for each complete assignment during the backtracking search process, the VIN code is verified against the vehicle certificate database based on the candidate VIN recognition results. If two or more candidate VIN recognition results pass the verification, the candidate VIN recognition result with the highest confidence score from the OCR is taken as the VIN recognition result. If all candidate VIN recognition results fail the verification, manual review is performed to obtain the VIN recognition result.

8. A machine vision-based vehicle VIN code recognition system, used to execute the machine vision-based vehicle VIN code recognition method according to any one of claims 1 to 7, characterized in that, The system includes: Image acquisition module: used to acquire the original image of the vehicle, perform localization and geometric correction on the original image, and obtain a standardized image; Easily confused character extraction module: used to perform image text recognition on the standardized image, obtain the initial recognition result of VIN code and character probability distribution features, and calculate the position of easily confused characters based on the character probability distribution features; Candidate character list extraction module: used to query a preset VIN code tag library based on the initial recognition result, obtain optional check bits and VIN code encoding rules, and generate a candidate character list for easily confused character positions according to the VIN code encoding rules; Constraint Satisfaction Problem Module: Used to divide the initial recognition result using the optional check bits, construct the vin code encoding rule constraint satisfaction problem based on segmented word units, and perform hierarchical pruning based on the candidate character list through the constraint satisfaction problem; VIN code recognition module: Used to output candidate VIN recognition results based on the pruned list through a backtracking search algorithm, verify the candidate VIN recognition results, and obtain the VIN recognition result.

Images