VIN code recognition method and system based on YOLOv11 and PaddleOCR

Abstract

The present application relates to the technical field of computer vision, in particular to a VIN code recognition method and system based on YOLOv11 and PaddleOCR. First, the image is preprocessed by high lifting filter combined with Clahe algorithm to enhance the contrast of characters and background; a SA-YOLO lightweight detection model is constructed to optimize C3k2 module by CA attention, compress parameter quantity by depth separable convolution, and design improved EIoULoss with fusion of overlap suppression and center constraint term to accurately locate VIN character line; finally, character recognition is completed by PaddleOCRv4. By pre-processing the image first to enhance the contrast of the image, and using the YOLOv11n model improved by fusing CA attention mechanism and EIoULoss to obtain VIN code character line positioning information, and matching PaddleOCRv4 to complete character recognition, it not only adapts to the character characteristics of VIN code with multiple vehicle body colors, character changes, no occlusion and fixed aspect ratio, but also realizes positioning accuracy and recognition efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a VIN code recognition method and system based on YOLOv11 and PaddleOCR. Background Technology

[0002] The Vehicle Identification Number (VIN) serves as a unique legal identifier for a vehicle, encompassing crucial information such as manufacturer, year of manufacture, model, and engine code. It plays an irreplaceable role throughout the entire lifecycle of a car, including production, distribution, and after-sales service. Especially in the automotive manufacturing process, real-time and accurate VIN identification is a fundamental prerequisite for the production system to invoke assembly instructions and guide robots to complete customized assembly, directly determining production efficiency and assembly quality. While VINs inherently possess the characteristics of regular characters, no obstructions, and a fixed aspect ratio, traditional manual identification methods not only consume significant human resources but are also prone to misjudgment or incorrect input due to fatigue and subjective errors. This completely fails to meet the high-speed, high-precision production demands of the modern automotive industry. Therefore, the research and application of automatic VIN identification technology has become an industry necessity.

[0003] However, the actual application scenarios in automobile production workshops still present numerous technical challenges for automatic VIN code recognition: First, different vehicle colors (white, gray, black, etc.) result in significant differences in character and background contrast. Characters on white vehicles have clear contrast against the background, while characters on darker-colored vehicles have weak contrast, directly affecting the clarity of character recognition. Second, although VIN code character lines have the advantages of being regular, unobstructed, and having a fixed aspect ratio, some characters are fixed while others are variable, requiring adaptability of the recognition model. Third, industrial scenarios impose stringent standards on the real-time performance, lightweight design, and anti-interference capabilities of the recognition system, necessitating adaptation for edge device deployment while ensuring high accuracy. To address these issues, various VIN code recognition solutions have been proposed in related technical fields:

[0004] CN114241183A discloses a vehicle VIN code recognition method based on a CRNN network model, comprising: acquiring multiple VIN code images to form a VIN code database; marking difficult characters; inputting the character region images into the CRNN network model, storing the feature vectors of difficult characters in vector set I; storing the feature vectors of ordinary digit characters in vector set II; selecting hidden layer information belonging to vector set II and storing it as transition information; inputting vector set I back into the RNN, concatenating the transition information with the hidden layer information corresponding to the input of a single column of feature vectors; processing each feature vector in vector set I using the concatenated hidden layer information; updating the corresponding information to form a new probability matrix; iteratively optimizing a CRNN network model that meets the requirements; and using the trained CRNN network model to recognize vehicle VIN codes. This approach uses a CRNN (CNN+BiLSTM+CTC) deep learning model, achieving overall character recognition through temporal prediction, eliminating the need for a separate character segmentation step, and improving robustness to a certain extent by utilizing the sequential information of characters. However, it has two major flaws: First, it lacks an independent character line localization module and relies on temporal prediction to indirectly lock the character region, which is prone to localization errors when the character edges are blurred or the background is complex; Second, the preprocessing only uses simple operations such as grayscale transformation and scaling, which cannot solve the problem of character blurring in low-contrast scenes; In addition, the model performance is highly dependent on the number of samples. When the variable characters in the VIN code are significantly different from the training samples, the recognition accuracy will drop sharply, and no real-time indicators are explicitly mentioned, making it difficult to adapt to the high-speed industrial requirements.

[0005] CN109784322B discloses a VIN code recognition method, device, and medium based on image processing, applied in the field of image processing technology. The method includes: acquiring vehicle environment information of a target vehicle; determining target shooting parameters based on the vehicle environment information; using a shooting device and acquiring a VIN code image of the target vehicle according to the target shooting parameters; performing edge detection on the VIN code image to obtain an edge image after edge detection; performing morphological closing operations on the edge image to obtain a closed image; performing morphological opening operations on the closed image to obtain an open image; determining the minimum bounding rectangle of the VIN code region in the open image, and defining the region determined by the minimum bounding rectangle as the VIN code region of the target vehicle; and recognizing the VIN code region of the target vehicle to identify the VIN code. This type of scheme mainly relies on traditional visual techniques such as edge detection, morphological closing operations, and opening operations, achieving VIN code region localization through the "minimum bounding rectangle," and subsequently cooperating with traditional OCR technology to complete character recognition. Its core problem lies in its weak anti-interference ability and poor adaptability to low-contrast scenes. When the boundaries between characters and background are blurred, such as on a black car body, the positioning failure rate is extremely high. Moreover, the positioning accuracy depends on manual parameter adjustment, which easily includes redundant background, leading to increased subsequent recognition errors. At the same time, it has not been optimized for real-time performance in industrial scenarios, making it difficult to match the high-speed requirements of automobile production lines.

[0006] CN115937854A discloses a method, system, medium, device, and terminal for VIN code image character recognition. The method involves acquiring X-style VIN images and B-style VIN images; constructing a generative adversarial network (GAN) model using the X-style VIN image and a detection network model using the B-style VIN image; obtaining the detected character results using the detection network model; obtaining the character recognition result using VIN code character combination rules; determining whether the character is correctly recognized using the VIN code character ground truth values ​​from the X-style VIN image; if not, supplementing the training and returning to the generative network model and detection network model construction steps; if yes, outputting the result. This prior art partially addresses the style adaptation problem by constructing a GAN network to process VIN code images of specific styles and then combining the detection network with character rule judgments. However, this type of solution has significant limitations: GAN network training is unstable and prone to mode collapse, and generating high-quality images requires a lot of computing resources, resulting in excessive model complexity and long training cycles; although the inference stage has the theoretical advantage of single forward propagation, in actual deployment, it is limited by the model size, making it difficult to meet industrial needs in real time, and the multi-module coupled design increases the deployment difficulty and cannot flexibly adapt to the edge computing environment of automobile production.

[0007] CN118298438A discloses a multi-scenario VIN code recognition method and system based on deep learning and OCR technology. It utilizes a YOLO network model to train a VIN code localization model capable of locating the VIN code's position and identifying its category. This model can identify unhanged VIN codes, completely damaged or obscured VIN codes, and normal or partially obscured VIN codes. Secondly, it uses OCR technology to recognize characters on normal and partially obscured VIN codes, training them for binary classification. The method accurately distinguishes normal VIN codes based on the number of characters recognized. This technology recognizes N-codes, partially damaged VIN codes, and partially occluded VIN codes, achieving VIN code recognition across multiple scenarios. It also classifies and distinguishes occluded or damaged VIN code images. However, while existing technologies have attempted to use YOLO series models for VIN code localization, they are mostly based on the early YOLOv3 architecture, only classifying occluded and damaged scenarios. This process is cumbersome and lacks optimized localization accuracy. Furthermore, the recognition process does not incorporate industrial-grade high-performance OCR models, resulting in low overall recognition efficiency and failing to meet the real-time requirements of automotive production.

[0008] In addition, some existing technologies use traditional techniques such as four-point positioning to locate the VIN code area, and then complete the recognition through a preset recognition model and complex character rules post-processing. The main drawbacks are insufficient positioning accuracy and robustness: four-point positioning relies on image texture features, which is susceptible to angle shifts and noise interference, resulting in the positioning box not being able to completely cover the character line; moreover, the loss function and network structure are not optimized for VIN code recognition scenarios, resulting in problems such as center shift leading to character omission and false detection of overlapping boxes; at the same time, character post-processing relies on manually set filtering rules, which has poor robustness and is difficult to adapt to the VIN code character variation characteristics of different vehicle models.

[0009] In summary, existing VIN code recognition technologies generally suffer from poor adaptability to various vehicle colors, insufficient positioning accuracy, high computational resource consumption, and low recognition efficiency, making it difficult to simultaneously meet the comprehensive requirements of high precision, real-time performance, and lightweight design in automotive production scenarios. Therefore, developing a VIN code recognition method and system that can adapt to complex industrial environments and balance positioning accuracy with high recognition efficiency has become a pressing technical challenge in this field. Summary of the Invention

[0010] To address the shortcomings of existing technologies, a VIN code recognition method and system based on YOLOv11 and PaddleOCR is proposed. This method first preprocesses the image to enhance its contrast, and then uses a YOLOv11n model improved by fusing CA attention mechanism and EIoULoss to obtain the character line location information of the VIN code. Combined with PaddleOCRv4, the method completes character recognition. This approach is suitable for the characteristics of VIN codes, such as multiple vehicle body colors, character variations, no occlusion, and fixed aspect ratios, and achieves both accurate positioning and high recognition efficiency.

[0011] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is a VIN code recognition method based on YOLOv11 and PaddleOCR, comprising the following steps:

[0012] S1: Data preprocessing: A combination of high-boost filtering and Clahe is used to enhance the difference between the VIN code character line and the background from the image source, thereby improving image quality;

[0013] S2: Character line detection: Constructing the SA-YOLO model and making dual improvements to the lightweight YOLOv11n model:

[0014] a. Module Improvement: Based on the CoordinateAttention mechanism, the C3k2 module is improved to obtain the CA-C3k2 module, and depthwise separable convolutions are used to replace the ordinary convolutions in the network backbone.

[0015] b. Improved loss function: An improved EIoULoss is introduced, which integrates overlap suppression loss and center point distance strong constraint loss to achieve accurate bounding box regression;

[0016] S3: Character Recognition: PaddleOCRv4 is used to recognize the character lines after localization by the SA-YOLO model, and the VIN code result is output.

[0017] In the VIN code recognition method based on YOLOv11 and PaddleOCR described above, in S1, the formula for the high-lift filtering is: high-lift image = A × original image - (A-1) × blurred image, where A is the lifting coefficient and A≥1, and the blurred image is obtained by Gaussian blurring. The Clahe method enhances the details and textures through contrast limiting and eliminates block boundaries by combining bilinear interpolation, making the image transition smooth.

[0018] In the aforementioned VIN code recognition method based on YOLOv11 and PaddleOCR, in S2, the CA-C3k2 module supports dynamic switching: when parameter C3k=False, the basic Bottleneck module is used, and when C3k=True, it switches to the CA-C3k module. The core of the CA-C3k module is CA-Block, which adopts an asymmetric convolutional structure of kx1 and 1xk and incorporates a CoordinateAttention mechanism. The kx1 convolutional kernel is used to capture vertical features, and the 1xk convolutional kernel is used to extract horizontal information. The two work together to cover a receptive field comparable to that of the kxk symmetric convolutional kernel, and the parameter redundancy is significantly reduced.

[0019] The improved EIoULoss total loss function of the VIN code recognition method based on YOLOv11 and PaddleOCR described above is: ,in, To improve EIoU loss, To suppress the loss due to overlap, The loss is determined by strong constraints on the distance from the center point. , These are the training weights, with values ​​ranging from 0.1 to 0.2. , The value is limited to 0.1-0.2, which is the optimal range verified by 100 rounds of model training. When it is less than 0.1, the auxiliary loss has a weak effect and cannot suppress overlapping boxes and center offset; when it is greater than 0.2, the auxiliary loss accounts for too high a proportion and will interfere with the optimization direction of the core regression loss.

[0020] The improved EIoU loss formula for the VIN code recognition method based on YOLOv11 and PaddleOCR described above is as follows: ,in, To measure the overlap loss between the predicted bounding box and the ground truth bounding box, the ratio of the deviation between their intersection and union is calculated to assess the overall coverage matching degree of the bounding box. To calculate the center distance loss between the predicted bounding box and the ground truth bounding box, the Euclidean distance deviation between their center points in the image coordinate system is calculated, and the position offset of the bounding box is constrained.

[0021] , These are the width and height of the model's predicted bounding box, respectively;

[0022] , The width and height of the true bounding box of the VIN code character line are normalized instead of the "length and width of the circumscribed rectangle" in the traditional EIoU. The core purpose is to avoid the "scale dilution" problem caused by the circumscribed rectangle, and to more directly constrain the length and width deviation between the predicted box and the true bounding box, adapting to the "fixed aspect ratio" characteristic of the VIN code character line.

[0023] Parameter range:

[0024] It requires no additional hyperparameters and only relies on the inherent properties (position, size) of the predicted and ground truth boxes for calculation, requiring no manual adjustment.

[0025] Core constraint logic:

[0026] pass and This item forces the predicted bounding box to precisely match the dimensions of the actual bounding box; the smaller the error, the closer the loss value is to 0.

[0027] The overlap suppression loss formula for the aforementioned VIN code recognition method based on YOLOv11 and PaddleOCR is as follows: Where N is the number of predicted bounding boxes and M is the number of ground truth bounding boxes. The EIoU value between the i-th predicted bounding box and the j-th ground truth bounding box ranges from [0,1], with values ​​closer to 1 indicating a higher degree of matching; the exponential term... Penalty weights used to reinforce high-confidence prediction boxes: The higher the value, the closer the predicted bounding box is to the true bounding box. The smaller, The larger the value, the greater the final loss and the stronger the penalty.

[0028] Parameter range:

[0029] No additional hyperparameters are used; N is dynamically determined by the model inference results, and M is fixed at 1 (to adapt to single-character line scenarios).

[0030] Core constraint logic: When multiple predicted boxes match the same ground truth box, the total loss is increased by accumulating penalty terms, forcing the model to output only one high-confidence predicted box, with an overlapping box false detection rate of ≤0.2%.

[0031] The VIN code recognition method based on YOLOv11 and PaddleOCR described above uses the following formula for the strong constraint loss of the center point distance: ,in, The coordinates of the center point of the predicted bounding box are based on the image coordinate system (the top left corner is the origin, the x-axis is horizontal to the right, and the y-axis is vertically downward). The coordinates of the center point of the actual bounding box for the VIN code character line are used to avoid character leakage due to center offset of the predicted box; summation of absolute values. The sum of the offset distances between the predicted bounding box and the true bounding box in the x and y directions is calculated, which directly reflects the center point positioning deviation.

[0032] Parameter range:

[0033] No additional hyperparameters are required, and the range of coordinate values ​​matches the image resolution (the preferred image resolution in this invention is 640×480, so x∈[0,640], y∈[0,480]).

[0034] Core constraint logic: When the center point of the predicted bounding box is offset, the loss value increases linearly with the offset distance, forcing the model to accurately align with the center of the real bounding box and avoiding character leakage due to center offset (experimentally verified, the character leakage rate is ≤0.5%).

[0035] The aforementioned recognition system based on the YOLOv11 and PaddleOCR VIN code recognition method includes:

[0036] Preprocessing module: Used to perform the preprocessing operation of S1 and output the enhanced VIN code image;

[0037] SA-YOLO character line detection module: used to perform S2 character line detection operations and output precisely located character line regions;

[0038] PaddleOCR character recognition module: used to perform character recognition operations in S3 and output the final VIN code result;

[0039] The preprocessing module, SA-YOLO character line detection module, and PaddleOCR character recognition module are connected in series to achieve end-to-end recognition from image input to VIN code output.

[0040] The beneficial effects of the VIN code recognition method and system based on YOLOv11 and PaddleOCR in this invention are as follows: Addressing the characteristics of VIN codes during vehicle production, such as unclear imaging of characters with different vehicle body colors, some characters being fixed while others are variable, and the relatively fixed position of character lines, a VIN code recognition system based on YOLOv11 and PaddleOCR is designed. First, an SA-YOLO model is proposed, based on the CA attention mechanism, making the network more focused on spatial features. An improved loss function suppresses the original algorithm's detection of overlapping and occluded targets, making the model more suitable for the application scenario of this patent and achieving accurate detection of character lines. Second, PaddleOCR is used to realize character line recognition. Finally, a complete character recognition scheme is integrated, achieving complementarity in system accuracy and real-time performance.

[0041] By combining high-lift filtering with Clahe preprocessing, the problem of low contrast between character lines and background under different colored vehicle bodies (especially black vehicle bodies) is effectively solved. After processing, the character edge features are significantly enhanced, and the character outlines can be clearly distinguished, providing high-quality image input for subsequent detection and recognition, and greatly reducing the recognition error in low-contrast scenes.

[0042] The SA-YOLO model's modular design improves the model's ability to perceive character line space features through the CoordinateAttention mechanism. It also replaces ordinary convolution with depthwise separable convolution, which significantly reduces the number of model parameters and computational cost while ensuring detection accuracy, making the model more lightweight and suitable for deployment on industrial edge devices.

[0043] The improved EIoULoss fusion of overlap suppression loss and center point distance strong constraint loss significantly reduces the risk of overlapping prediction boxes for a single unoccluded character line, avoids the problem of character leakage caused by prediction box center offset, and greatly improves the accuracy of character line localization. The complete wrapping rate of the localized box to the character line reaches more than 98%.

[0044] The collaborative design of the SA-YOLO model and PaddleOCRv4 achieves efficient integration of VIN code localization and recognition. The total time from acquisition to output of a single image meets the real-time requirements of industry, with a high average recognition accuracy. It can effectively replace manual recognition, reduce human resource consumption, and improve automobile production efficiency.

[0045] All parameters are adapted to the industrial scene characteristics of "no occlusion, no overlap, and fixed aspect ratio" of VIN code character lines, avoiding parameter redundancy in complex scenarios in general object detection and ensuring a balance between lightweight model and high accuracy.

[0046] The system structure of this invention is simple, with strong compatibility between modules. It can be flexibly deployed to various industrial edge computing devices, adapting to the complex environment of automobile production workshops and the VIN code recognition requirements of different car models. It is robust and has broad industrial application value. Attached Figure Description

[0047] Figure 1 A flowchart illustrating the overall technical process of the VIN code recognition method based on YOLOv11 and PaddleOCR;

[0048] Figure 2 This is a schematic diagram of the internal structure of the CA-C3k2 module in the SA-YOLO model;

[0049] Figure 3 This is a block diagram of the overall structure of a VIN code recognition system based on YOLOv11 and PaddleOCR.

[0050] Figure 4 The images show a comparison of the preprocessing effects (a is the original image of the VIN code of the black vehicle body, b is the image after high-lift filtering, and c is the image after Clahe processing).

[0051] Figure 5 Comparison of localization performance of different models (a is the localization and recognition performance of PaddleOCRv4, b is the localization and recognition performance of YOLOv11n).

[0052] Figure 6 The images show the recognition effect for different vehicle body colors (a is a white vehicle body, b is a gray vehicle body, and c is a black vehicle body). Detailed Implementation

[0053] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0054] like Figure 1-6 As shown, a VIN code recognition method based on YOLOv11 and PaddleOCR includes the following steps:

[0055] S1: Data preprocessing: A combination of high-boost filtering and Clahe (limited contrast adaptive histogram equalization) is used to enhance the difference between VIN code character lines and the background from the image source, improve image quality, and provide high-quality input for subsequent character line detection.

[0056] S2: Character line detection: Construct an SA-YOLO model and perform dual optimizations on the YOLOv11n lightweight model, including module improvements and loss function improvements, to achieve accurate location of VIN code character lines.

[0057] a. Module Improvement: Based on the CoordinateAttention mechanism, the C3k2 module is improved to obtain the CA-C3k2 module, and depthwise separable convolutions are used to replace the ordinary convolutions in the network backbone.

[0058] The CA-C3k2 module supports dynamic switching: when C3k=False, the basic Bottleneck module is used to ensure lightweight operation of the model in simple scenarios; when C3k=True, it switches to the CA-C3k module to adapt to the character line detection requirements in complex scenarios. The core of the CA-C3k module is CA-Block, which adopts an asymmetric convolutional structure of kx1 and 1xk, and incorporates the CoordinateAttention mechanism. The kx1 convolutional kernel is used to capture vertical features, and the 1xk convolutional kernel is used to extract horizontal information. The two work together to cover a receptive field comparable to that of the kxk symmetric convolutional kernel, and the parameter redundancy is significantly reduced. This reduces the model's computational load while ensuring feature extraction capabilities. The CoordinateAttention mechanism can enhance the model's ability to perceive character line space features and model long-range dependencies, improving character line detection performance in low-contrast scenarios.

[0059] The specific implementation details of the CA-Block are as follows:

[0060] 1. Channel configuration: The number of input / output channels is consistent with the original C3k2 module of YOLOv11n (e.g., if the input is 64-dimensional, the output will be 64-dimensional), and the activation function is GELU;

[0061] 2. CA attention mechanism implementation steps: ① Compress the channel dimension to 1 / 16 of the original dimension through global average pooling; ② Perform x / y coordinate separation encoding on the compressed features (x-direction encoding: y-direction encoding: , W / H is the width and height of the feature map); ③ Attention weights are generated through two fully connected layers (hidden layer dimension = compressed dimension × 2); ④ The weights are multiplied element-wise with the original feature map to enhance the character line space features;

[0062] 3. Asymmetric convolution feature fusion: The output features of 3×1 and 1×3 convolutions are concatenated according to the channel dimension, and then reduced to the original number of channels by 1×1 convolution.

[0063] Depthwise separable convolutions are used to replace ordinary convolutions in the network backbone. Depthwise separable convolutions separate the spatial convolutions and channel convolutions of ordinary convolutions, further reducing the number of model parameters and computational cost without sacrificing feature extraction accuracy, thus ensuring the real-time operation of the model.

[0064] b. Improved loss function: An improved EIoULoss is introduced, which integrates overlap suppression loss and center point distance strong constraint loss to achieve accurate bounding box regression;

[0065] An improved EIoULoss is introduced to replace the CIoULoss of the original YOLOv11n model. The improved EIoULoss integrates the overlap suppression loss and the strong constraint loss of the center point distance to achieve accurate regression of the bounding box, which is suitable for the scene characteristics of regular VIN code character line positions, no occlusion and no overlap.

[0066] The improved EIoULoss total loss function is: ,in, To improve EIoU loss, To suppress the loss due to overlap, The loss is determined by strong constraints on the distance from the center point. , These are the training weights, with values ​​ranging from 0.1 to 0.2. , The value is limited to 0.1-0.2, which is the optimal range verified by 100 rounds of model training. When it is less than 0.1, the auxiliary loss has a weak effect and cannot suppress overlapping boxes and center offset; when it is greater than 0.2, the proportion of auxiliary loss is too high, which will interfere with the optimization direction of the core regression loss. The total loss function is only used as an auxiliary constraint and does not affect the priority of the core regression loss, ensuring the accuracy of bounding box regression.

[0067] In the total loss function, As the core loss, , The goal of setting the value to a small value (≤0.2) is to ensure that the auxiliary loss only plays a "correction and optimization" role, does not take priority over the core regression loss, and ensures that the accuracy of the bounding box regression is not disturbed.

[0068] The improved EIoU loss formula is as follows: ,in, To measure the overlap loss between the predicted bounding box and the ground truth bounding box, the ratio of the deviation between their intersection and union is calculated to assess the overall coverage matching degree of the bounding box. To calculate the center distance loss between the predicted bounding box and the ground truth bounding box, the Euclidean distance deviation between their center points in the image coordinate system is calculated, and the position offset of the bounding box is constrained.

[0069] , These are the width and height of the model's predicted bounding box, respectively;

[0070] , The width and height of the true bounding box of the VIN code character line are normalized instead of the "length and width of the circumscribed rectangle" in the traditional EIoU. The core purpose is to avoid the "scale dilution" problem caused by the circumscribed rectangle, and to more directly constrain the length and width deviation between the predicted box and the true bounding box, adapting to the "fixed aspect ratio" characteristic of the VIN code character line.

[0071] Parameter range:

[0072] It requires no additional hyperparameters and only relies on the inherent properties (position, size) of the predicted and ground truth boxes for calculation, requiring no manual adjustment.

[0073] Core constraint logic:

[0074] pass and This item forces the predicted bounding box to precisely match the dimensions of the actual bounding box; the smaller the error, the closer the loss value is to 0.

[0075] S3: Character Recognition: PaddleOCRv4 is used to recognize character lines located by the SA-YOLO model and output the VIN code result. PaddleOCRv4 is based on an advanced deep learning framework, extracts character features through CNN (Convolutional Neural Network), adopts a lightweight design and adaptive learning strategy, supports multilingual recognition, and achieves an accuracy rate of over 95% for Chinese and English recognition in general scenarios. Moreover, the character line recognition time per image is short, which can meet the real-time detection requirements of industrial production.

[0076] In S1, the formula for the high-lift filter is: High-lift image = A × Original image - (A-1) × Blurred image, where A is the lift coefficient and A≥1. When A=1, the original image is output; when A>1, a sharpening effect is produced, which can enhance the details of character edges. The blurred image is obtained through Gaussian blurring, which can smooth image noise while preserving the core outline of the characters. Combined with the high-lift filter, it achieves a balance between image sharpening and noise suppression. The Clahe filter enhances the details and textures through contrast limiting and eliminates block boundaries through bilinear interpolation, making the image transition smooth. By enhancing the details and textures of the image through the contrast limiting mechanism, interference caused by excessive background contrast enhancement is avoided. At the same time, the combination of bilinear interpolation technology eliminates obvious boundaries between image blocks, making the image transition smooth and further improving the distinction between characters and the background.

[0077] Combined processing logic: First, the structural information and contour features of the image are enhanced by high-boost filtering to make the character edges stand out initially. Then, Clahe is used to optimize the overall contrast of the image, taking into account both the highlighting of local details and the natural visual effect, effectively solving the problem of low contrast between characters and background under different colored car bodies (especially black car bodies).

[0078] The overlap suppression loss formula is as follows: Where N is the number of predicted bounding boxes and M is the number of ground truth bounding boxes. The EIoU value between the i-th predicted bounding box and the j-th ground truth bounding box ranges from [0,1], with values ​​closer to 1 indicating a higher degree of matching; the exponential term... Penalty weights used to reinforce high-confidence prediction boxes: The higher the value, the closer the predicted bounding box is to the true bounding box. The smaller, The larger the value, the greater the final loss and the stronger the penalty.

[0079] Parameter range:

[0080] No additional hyperparameters are used; N is dynamically determined by the model inference results, and M is fixed at 1 (to adapt to single-character line scenarios).

[0081] Core constraint logic: When multiple predicted boxes match the same ground truth box, the total loss is increased by accumulating penalty terms, forcing the model to output only one high-confidence predicted box, with an overlapping box false detection rate of ≤0.2%.

[0082] The formula for the strong constraint loss of the center point distance is: ,in, The coordinates of the center point of the prediction box. The coordinates of the center point of the ground truth bounding box are used to avoid character leakage caused by the center offset of the predicted bounding box. This loss function can strictly constrain the center position of the predicted bounding box, avoid character leakage caused by the center offset of the predicted bounding box, and ensure that the character line is completely wrapped within the localization box, providing a complete character region for subsequent character recognition.

[0083] The aforementioned recognition system based on the YOLOv11 and PaddleOCR VIN code recognition method includes:

[0084] Preprocessing module: Used to perform the preprocessing operation in S1 above, enhance the difference between the VIN code character line and the background by combining high-lift filtering and Clahe method, and output the enhanced VIN code image;

[0085] SA-YOLO character line detection module: used to perform the character line detection operation in S2 above. Through the SA-YOLO model improved by the module and loss function, it accurately locates the VIN code character line region and outputs the coordinate information of the location box.

[0086] PaddleOCR character recognition module: used to perform the character recognition operation in S3 above, receive the coordinates of the positioning box output by the SA-YOLO character line detection module, crop the corresponding character line area and perform character recognition, and output the final VIN code result;

[0087] The preprocessing module, SA-YOLO character line detection module, and PaddleOCR character recognition module are connected in series to form an end-to-end recognition process. From the acquisition and input of the VIN code image to the final output of the VIN code result, no manual intervention is required, achieving fully automated recognition.

[0088] Exception handling:

[0089] 1. Location confidence threshold: When the confidence of the SA-YOLO output bounding box is <0.7, the location is judged as a failure and the image is re-acquired (maximum of 2 retries, retry interval = 50ms).

[0090] 2. Feedback on suspicious results: When the confidence score of PaddleOCR recognition is <0.9, the data format pushed through the Profinet bus is "suspected VIN code result + original image URL + location box coordinates (x1,y1,x2,y2) + confidence score value";

[0091] 3. Manual review: The review interface automatically highlights characters with a confidence level of <0.9 and supports two operations: "Confirm Correct" and "Modify Error Character". The results are synchronized to the production system database in real time after modification.

[0092] Example 1

[0093] Data preprocessing implementation

[0094] 1. Data Acquisition: Collect VIN images of vehicles of different colors, such as white, gray, and black, in the automobile production workshop to construct a dataset containing 4,000 valid images. All images were taken on-site using an industrial camera (model Baslerac A2500-14uc, focal length 8mm, exposure time 10ms, frame rate 30FPS), covering different lighting conditions (natural light, workshop supplemental lighting, light intensity 500-800 lux) and different shooting angles (vertical shooting, ±5° tilt shooting, with ±5° tilt shooting images accounting for 30%). The resolution of all images was uniformly adjusted to 640×480 to ensure the scene diversity and representativeness of the dataset.

[0095] Dataset annotation standards: ① Location box annotation: strictly enclose the VIN code character line, with a margin of ≤2 pixels on all sides, and do not include redundant background; ② Character-level annotation: VIN code characters are uniformly annotated as uppercase (if the actual character is lowercase, it is uniformly converted to uppercase), and the annotation range covers the complete outline of the character, without cutting the character strokes.

[0096] Industrial camera installation requirements: ① Installation height: 50±5cm from the VIN code area of ​​the vehicle body; ② Shooting angle: perpendicular to the surface of the vehicle body, with a permissible deviation of ≤2°; ③ Installation position: located directly in front of the VIN code area, without obstructions, and avoiding direct sunlight that could cause glare.

[0097] 2. High-lift filtering: Set the lift factor A=1.5 (after multiple experiments, this parameter achieves the optimal balance between sharpening effect and noise suppression). Apply Gaussian blur to the original image using a 3×3 convolution kernel and a standard deviation σ=1.0 to obtain a blurred image. Then, generate a sharpened image according to the formula "High-lift image = 1.5 × original image - 0.5 × blurred image" to enhance character edge details while suppressing image noise.

[0098] 3. Clahe processing: Set the contrast limit parameter to 2.0 and the grid size to 8×8 to process the sharpened image; the contrast limit parameter controls the contrast enhancement of the background area to avoid excessive background interference; the grid size parameter is adapted to the block division of the 640×480 resolution image, and bilinear interpolation technology is used to eliminate block boundaries to obtain an enhanced image with smooth transition and high distinction between characters and background.

[0099] 4. Data partitioning and preprocessing validation: The 4000 preprocessed images were divided into a training set (3200 images) and a test set (800 images) at an 8:2 ratio for subsequent training and validation of the SA-YOLO model. The black car images in the test set were sampled for validation. The results showed that the character edge clarity was improved by more than 80% after combined preprocessing, and the contrast between characters and background was improved by more than 3 times, which met the requirements of subsequent detection.

[0100] Example 2

[0101] SA-YOLO Model Training and Deployment

[0102] 1. Model Structure Configuration: Based on the YOLOv11n open-source code, the model structure was modified within the PyTorch framework. The original C3k2 module was replaced with the CA-C3k2 module, and the parameter C3k=True was set to enable the CA-C3k module. All ordinary convolutions in the network backbone were replaced with depthwise separable convolutions, while keeping the number of convolution kernels unchanged to ensure no loss of feature extraction capability. The asymmetric convolution kernel sizes in CA-Block were set to 3×1 and 1×3 (to adapt to the aspect ratio characteristics of VIN character lines, balancing feature extraction accuracy and computational efficiency).

[0103] 2. Loss function configuration: In the model training configuration file, set the total loss function to [value].

[0104] ;in, =0.15、 =0.15 (Experiments have verified that this weight combination achieves the optimal balance between auxiliary constraints and core regression loss, suppressing overlapping boxes and center offset without affecting the accuracy of bounding box regression).

[0105] , , The calculation formula is as described in the technical solution;

[0106] 3. Training parameter settings: The Adam optimizer is used, with an initial learning rate of 0.001 and a learning rate decay strategy of cosine annealing (annealing period of 20 rounds, minimum learning rate of 0.0001); the batch size is set to 16 (to adapt to GPU memory capacity and balance training efficiency and stability); the training cycle is 100 rounds, and the model performance is verified on the test set every 10 rounds. An early stopping strategy is adopted (training is stopped if there is no improvement in mAP on the test set for 10 consecutive rounds), and the model weight file with the best performance is saved;

[0107] Training Supplementary Configuration: ① Initial Weights: Fine-tuned based on YOLOv11n official COCO pre-trained weights; ② Regularization Strategy: Weight decay coefficient weight_decay=0.0005, and a Dropout layer (dropout=0.1) added to CA-Block; ③ Early Stopping Threshold: If the mAP improvement on the test set is <0.1% for 10 consecutive rounds, it is considered no improvement; ④ Data Augmentation during Training (implemented based on the Albumentations library): a. Random horizontal flip (probability=0.5); b. Random angle rotation (±3°, probability=0.3, after rotation, the padding method is "constant padding", padding value=0); c. Brightness perturbation (±10%, probability=0.4); d. Contrast perturbation (±15%, probability=0.4); All augmentation operations are only applied to the training set, and no augmentation is performed on the test set.

[0108] 4. Model Deployment: The trained SA-YOLO optimal model is exported to ONNX format (compatible with mainstream industrial inference frameworks) and deployed to industrial edge computing devices (CPU: Intel Core i7-12700, GPU: NVIDIA RTX 3060, memory: 32GB). TensorRT inference engine is used for acceleration during deployment to optimize model inference speed, ultimately achieving a character line detection frame rate of over 30 FPS and a single frame detection time of ≤30ms, meeting industrial real-time requirements.

[0109] 5. Software environment configuration: ① Python 3.8; ② PyTorch 1.18.0; ③ TensorRT 8.6.1; ④ OpenCV 4.8.0; ⑤ NumPy 1.24.3; ⑥ Albumentations 1.3.0; ⑦ Dependencies are managed through requirements.txt to avoid version conflicts.

[0110] Example 3

[0111] System integration and performance testing

[0112] 1. PaddleOCRv4 Deployment: Download the PaddleOCRv4 pre-trained model for both Chinese and English provided by Baidu PaddlePaddle. Fine-tune it for 10 rounds based on 3200 training images from the dataset of this invention, with a learning rate of 0.0001 and a batch size of 8. Load it onto the aforementioned edge computing device and configure the inference parameters: batch size = 1, inference accuracy = FP32, inference engine = TensorRT, to ensure a balance between model inference speed and recognition accuracy. Character recognition post-processing logic: Filter erroneous results based on the VIN code "17 characters (digits 0-9 + letters AZ, excluding I / O / Q)" rule, and perform secondary verification on easily confused characters (0 / O, 1 / I).

[0113] PaddleOCRv4 fine-tuning configuration: ① Optimizer: Adam optimizer (β1=0.9, β2=0.999, ε=1e-8); ② Learning rate decay: cosine annealing strategy (annealing period=5 rounds, minimum learning rate=1e-5); ③ Loss function: CTC (Connectionist Temporal Classification) loss is used to adapt to continuous character line recognition scenarios; ④ Freeze strategy: freeze the first 8 layers of the PaddleOCRv4 backbone network during fine-tuning, and only train the head recognition layer.

[0114] 2. System Integration: The preprocessing module, SA-YOLO character line detection module, and PaddleOCR character recognition module are integrated using Python scripts. The specific process is as follows:

[0115] a. An industrial camera acquires VIN code images in real time at a frame rate of 30 FPS, with an image resolution of 640×480, and transmits them to an edge computing device via an industrial Ethernet network;

[0116] b. After receiving the image, the edge computing device calls the preprocessing module to perform high-boost filtering and Clahe combined processing according to the parameters of Example 1. The processing time is ≤30ms / frame.

[0117] c. The enhanced image is input into the SA-YOLO character line detection module, which outputs the coordinates of the character line location box (x1, y1, x2, y2), with a detection time of ≤30ms / frame;

[0118] Location result determination: If the confidence score of the location frame is <0.7, the location is determined to be a failure, and the image is re-acquired (maximum of 2 retries, retry interval = 50ms); if the location still fails after 2 consecutive retries, an equipment abnormality alarm is pushed to the production system.

[0119] d. Cropping the character line region according to the coordinates of the positioning box (cropping accuracy up to 1 pixel level), and inputting the cropped character line image into PaddleOCRv4 for character recognition, with a recognition time of ≤120ms / frame;

[0120] e.PaddleOCRv4 outputs VIN code results (including recognition confidence), saves them to the production system database via the industrial bus (Profinet), and feeds them back to the assembly robot to guide customized assembly operations;

[0121] f. Recognition Result Processing: If the recognition confidence level is ≥0.9, it is directly stored in the database; if the confidence level is <0.9, it is pushed to the manual review interface via the Profinet bus in the format "VIN code suspected result + original image URL + location box coordinates (x1,y1,x2,y2) + confidence value". The interface automatically highlights the low confidence character positions and supports "confirm correct" and "correct error character" operations. The results after modification are synchronized to the database in real time.

[0122] 3. System deployment software environment: ① PaddlePaddle 2.6.0; ② The versions of other dependent libraries are the same as in Example 2, and are managed uniformly through requirements.txt.

[0123] System performance testing: A one-week test was conducted in an automotive production workshop (temperature 20-28℃, humidity 40%-60%, no strong reflective interference light sources). The test sample included 10,000 VIN code images of three typical vehicle body colors: white, gray, and black. Specifically, there were 3,000 images of white vehicles, 3,000 images of gray vehicles, and 4,000 images of black vehicles. The test results are as follows:

[0124] a. Recognition accuracy: The overall average recognition accuracy reached 98.5%, with 99.2% for white vehicles, 98.8% for gray vehicles, and 97.6% for black vehicles, with no cases of misjudgment or missed recognition;

[0125] b. Real-time performance: The average total time from acquisition to output of a single image is 170ms, which is far below the 200ms threshold required by industrial production and meets the needs of high-speed production.

[0126] c. Positioning performance: The complete coverage rate of the character line positioning box reaches 99.3%, the false detection rate of overlapping boxes is ≤0.2%, and the character leakage rate caused by center offset is ≤0.5%;

[0127] d. Stability: It can run continuously for 100 hours without failure, and the model performance does not degrade significantly, making it suitable for long-term operation in complex workshop environments.

[0128] Model performance comparison and verification:

[0129] To verify the improved performance of the SA-YOLO model of this invention, mAP (mean accuracy), parameter count, and recall were used as the core performance indicators. The model was compared with the basic YOLOv11n model, the YOLOv11n_C model which only incorporates the CA attention mechanism, and the YOLOv11n_E model which only introduces the improved EIoULoss. The test set consisted of 800 preprocessed images from Example 1. The results are shown in the table below:

[0130] YOLOv11n - - 96.5 2.84 95.4 YOLOv11n_C √ - 97.4 2.18 96.7 YOLOv11n_E - √ 97.8 2.88 95.8 SA-YOLO √ √ 98.3 2.23 97.3

[0131] Note: mAP@0.5 calculation standard: referring to the COCO2017 dataset evaluation specification, IoU threshold=0.5, and the average accuracy is calculated using the 101-point interpolation method; only the detection results at the line level of VIN code characters are counted, and other irrelevant targets are not counted.

[0132] It can be seen from the above table:

[0133] After incorporating the CA attention mechanism, the number of parameters in the YOLOv11n_C model decreased from 2.84M to 2.18M (a reduction of 23.2%), and the recall rate increased from 95.4% to 96.7% (an increase of 1.3%), demonstrating that the CA attention mechanism effectively enhances the model's ability to detect character lines while reducing computational cost.

[0134] After introducing the improved EIoULoss model, the mAP of YOLOv11n_E model increased from 96.5% to 97.8% (an improvement of 1.3%), and the recall rate increased slightly to 95.8% (an improvement of 0.4%). This verifies that the improved EIoULoss improves the accuracy of bounding box regression and effectively suppresses false detections of overlapping boxes.

[0135] The SA-YOLO model, combined with the CA attention mechanism and the improved EIoULoss, achieves an mAP of 98.3% (1.8% higher than the basic model) and a recall of 97.3% (1.9% higher than the basic model), with only 2.23M parameters (21.5% less than the basic model). It achieves the optimal balance between detection accuracy and lightweight design, fully demonstrating the effectiveness of the model improvement scheme of this invention.

[0136] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A VIN code recognition method based on YOLOv11 and PaddleOCR, characterized in that: Includes the following steps: S1: Data preprocessing: A combination of high-boost filtering and Clahe is used to enhance the difference between the VIN code character line and the background from the image source, thereby improving image quality; S2: Character line detection: Constructing the SA-YOLO model and making dual improvements to the lightweight YOLOv11n model: a. Module Improvement: Based on the CoordinateAttention mechanism, the C3k2 module is improved to obtain the CA-C3k2 module, and depthwise separable convolutions are used to replace the ordinary convolutions in the network backbone. b. Improved loss function: An improved EIoULoss is introduced, which integrates overlap suppression loss and center point distance strong constraint loss to achieve accurate bounding box regression; S3: Character Recognition: PaddleOCRv4 is used to recognize the character lines after localization by the SA-YOLO model, and the VIN code result is output.

2. The VIN code recognition method based on YOLOv11 and PaddleOCR according to claim 1, characterized in that, In S1, the formula for the high-boost filtering is: High-boost image = A × Original image - (A-1) × Blur image, where A is the boost coefficient and A≥1, and the blur image is obtained by Gaussian blurring. The Clahe enhances the details and textures through contrast limiting and eliminates block boundaries by combining bilinear interpolation, making the image transition smooth.

3. The VIN code recognition method based on YOLOv11 and PaddleOCR according to claim 2, characterized in that, in S2, the CA-C3k2 module supports dynamic switching: when parameter C3k=False, the basic Bottleneck module is used, and when C3k=True, it is switched to the CA-C3k module; the core of the CA-C3k module is CA-Block, which adopts an asymmetric convolutional structure of kx1 and 1xk, and incorporates a CoordinateAttention mechanism; the kx1 convolutional kernel is used to capture vertical features, and the 1xk convolutional kernel is used to extract horizontal information.

4. The VIN code recognition method based on YOLOv11 and PaddleOCR according to claim 3, characterized in that, The improved total loss function of EIoULoss is: ,in, To improve EIoU loss, To suppress the loss through overlap, The loss is determined by strong constraints on the distance from the center point. , These are the training weights, with values ​​ranging from 0.1 to 0.

2.

5. The VIN code recognition method based on YOLOv11 and PaddleOCR according to claim 4, characterized in that, The improved EIoU loss formula is as follows: ,in, To measure the overlap loss between the predicted bounding box and the ground truth bounding box, the ratio of the deviation between their intersection and union is calculated to assess the overall coverage matching degree of the bounding box. To calculate the center distance loss between the predicted bounding box and the ground truth bounding box, the Euclidean distance deviation between their center points in the image coordinate system is calculated, and the position offset of the bounding box is constrained. , These are the width and height of the model's predicted bounding box, respectively; , This defines the width and height of the actual frame for the VIN character line.

6. The VIN code recognition method based on YOLOv11 and PaddleOCR according to claim 5, characterized in that, The overlap suppression loss formula is as follows: Where N is the number of predicted bounding boxes and M is the number of ground truth bounding boxes. The EIoU value between the i-th predicted bounding box and the j-th ground truth bounding box ranges from [0,1], with values ​​closer to 1 indicating a higher degree of matching; the exponential term... Penalty weights used to reinforce high-confidence prediction boxes: The higher, The smaller, The larger the value, the greater the final loss and the stronger the penalty. Parameter range: There are no additional hyperparameters; N is dynamically determined by the model inference results, and M is fixed at 1. When multiple predicted boxes match the same ground truth box, the total loss is increased by accumulating penalty terms, forcing the model to output only one high-confidence predicted box, with an overlapping box false detection rate of ≤0.2%.

7. The VIN code recognition method based on YOLOv11 and PaddleOCR according to claim 6, characterized in that, The formula for the strong constraint loss of the center point distance is: ,in, The coordinates of the center point of the predicted bounding box are based on the image coordinate system; The coordinates of the center point of the actual bounding box for the VIN code character line are used to avoid character leakage due to center offset of the predicted box; summation of absolute values. The sum of the offset distances between the predicted bounding box and the ground truth bounding box in the x and y directions is calculated, which directly reflects the center point positioning deviation. Parameter range: No additional hyperparameters; the coordinate value range matches the image resolution. When the center point of the predicted bounding box shifts, the loss value increases linearly with the shift distance, forcing the model to accurately align with the center of the true bounding box and avoiding character leakage due to center shift.

8. The VIN code recognition method based on YOLOv11 and PaddleOCR according to claim 7, characterized in that, include: Preprocessing module: Used to perform the preprocessing operation of S1 and output the enhanced VIN code image; SA-YOLO character line detection module: used to perform S2 character line detection operations and output precisely located character line regions; PaddleOCR character recognition module: used to perform character recognition operations in S3 and output the final VIN code result; The preprocessing module, SA-YOLO character line detection module, and PaddleOCR character recognition module are connected in series to achieve end-to-end recognition from image input to VIN code output.

Images