Deep learning-based vehicle body stamp VIN code character recognition method and medium

Abstract

This invention relates to a deep learning-based method and medium for recognizing VIN codes on vehicle body stamps. The constructed character recognition model includes: an input layer for receiving VIN code images; a feature extraction layer for performing multi-branch convolutional fusion and downsampling on the VIN code image to obtain a main feature map, and then performing edge enhancement and sequence modeling on the main feature map to obtain an enhanced feature map and character space features; a feature fusion layer for performing residual fusion on the main feature map, enhanced feature map, and character space features to obtain fused features; and an output layer for outputting VIN code characters based on the fused features. This provides a novel recognition technology solution that can adapt to complex working conditions, fully utilize the structural characteristics of VIN codes and the logical relationships between characters, and possesses high accuracy and high robustness.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and medium for recognizing VIN codes on vehicle body stamps based on deep learning. Background Technology

[0002] The Vehicle Identification Number (VIN) is a unique identifier for a vehicle, consisting of 17 characters that contains key information such as the vehicle's manufacturer, year of manufacture, model, body style and code, engine code, and assembly location. Automatic VIN recognition technology has significant application value in scenarios such as automotive production line inspection, vehicle repair, annual inspection, and traffic management. Currently, automatic VIN recognition mainly relies on the following three types of technical solutions:

[0003] I. Barcode / QR code recognition technology

[0004] In existing technologies, some solutions use barcodes or QR codes affixed to the vehicle frame as VIN information carriers, which are then identified using optical scanning equipment. However, this type of technology has the following significant drawbacks:

[0005] (1) Strong carrier dependence and easy failure: Barcodes / QR codes need to be additionally affixed to the surface of the vehicle frame. During automobile production, assembly, warehousing, transportation and long-term use, the carrier is easily damaged by oil pollution and physical scratches. Studies have shown that when the damaged area is ≥30%, it cannot be identified; or the carrier may fall off due to poor adhesion, directly causing identification failure. Such carrier failure problems cannot meet the requirements of "continuous production line inspection" for zero-interruption identification in automobile production, and it is also difficult to adapt to the stable identification needs in long-term vehicle use (such as maintenance and annual inspection).

[0006] (2) Weak anti-interference capability of recognition algorithms: Existing barcode / QR code recognition relies heavily on traditional machine vision algorithms (such as edge detection and template matching) or general convolutional neural network (CNN) models. Traditional algorithms suffer from a sharp drop in stripe / dot matrix feature extraction accuracy under conditions such as fluctuating lighting in production workshops (such as alternating light and dark on the assembly line) and surface stains covering the carrier (coverage area ≥15%). Furthermore, the general CNN model has not been specifically optimized for the structural characteristics of barcodes / QR codes, such as "fixed stripe spacing and regular dot matrix arrangement" and the shortcoming of "local damage equals overall failure". When faced with local wear on the carrier (such as corner damage), the recognition accuracy drops sharply, making it difficult to meet the stringent requirement of "recognition accuracy ≥99%" in automobile manufacturing.

[0007] II. Steel Seal VIN Code Recognition Technology Based on Traditional Machine Vision

[0008] For VIN codes directly engraved on vehicle frames, existing technologies mainly employ traditional image processing solutions based on template matching and edge detection. These technologies have stringent requirements regarding the imaging environment: when there is uneven lighting (such as strong or weak outdoor light), rust / paint coverage on the stamp surface, or shooting angle deviations (±5° or more), the stability of character feature extraction drops sharply, and the recognition success rate decreases significantly. This makes them unsuitable for complex real-world production applications, especially failing to meet the rigid requirements for high recognition accuracy in the manufacturing process.

[0009] III. Deep Learning-Based Steel Seal VIN Code Recognition Technology

[0010] In recent years, some solutions have attempted to use deep learning models based on general-purpose CNNs for VIN code recognition on steel stamps. However, existing deep learning solutions have significant limitations: the training datasets lack diversity (mostly ideal steel stamp images under standard lighting and without wear), failing to fully consider non-ideal conditions such as character deformation, uneven spacing, local missing parts, and poor contrast in real-world applications; more importantly, the VIN code is not a simple character sequence, but its 17 characters have strict logical relationships (e.g., the first 1-3 characters are the world manufacturer identification code, the 4th-8th characters are the vehicle feature code, and the 9th character is the check digit), and existing models fail to fully utilize this logical correlation information between characters; furthermore, CNNs are limited by their local receptive field and lack the ability to correlate distant feature information. These factors result in poor generalization ability of existing deep learning models for non-ideal steel stamps, and the recognition accuracy is insufficient to meet the ultra-high precision requirements of practical applications.

[0011] In summary, existing VIN code recognition technologies have shortcomings in terms of carrier stability, environmental adaptability, feature extraction robustness, and utilization of character logical associations. There is an urgent need for a new recognition technology solution that can adapt to complex working conditions, fully utilize the structural characteristics of VIN codes and the logical associations between characters, and possess high accuracy and high robustness. Summary of the Invention

[0012] To address at least one of the aforementioned technical problems and overcome the shortcomings of existing VIN code recognition technologies in terms of carrier stability, environmental adaptability, feature extraction robustness, and utilization of character logical associations, this invention provides a deep learning-based method and medium for recognizing VIN codes on vehicle body stamps.

[0013] Firstly, this application provides a deep learning-based method for recognizing VIN codes on vehicle body stamps, including:

[0014] Collect several historical VIN code images and label the corresponding VIN code characters to construct a training dataset;

[0015] Based on the training dataset, a character recognition model is constructed and trained that takes the image of the stamped VIN code as input and the VIN code character as output, including:

[0016] The input layer is used to receive the image of the stamped VIN code;

[0017] The feature extraction layer is used to perform multi-branch convolutional fusion and downsampling on the stamped VIN code image to obtain the main feature map. Then, edge feature extraction and sequence modeling are performed on the main feature map to obtain the enhanced feature map and character space features.

[0018] The feature fusion layer is used to perform residual fusion of the main feature map, the enhanced feature map, and the character space features to obtain fused features;

[0019] The output layer is used to output VIN code characters based on the fusion features;

[0020] The current VIN code image is captured and input into the trained character recognition model to obtain the current VIN code character.

[0021] Furthermore, the steps of collecting several historical VIN code images and labeling them with the corresponding VIN code characters to construct a training dataset also include:

[0022] Collect several historical VIN code images and label the corresponding VIN code characters;

[0023] Each historical stamped VIN code image is converted to the HSV color space, channel separation and illumination adaptive preprocessing are performed, and then restored to grayscale to obtain an illumination adaptive enhanced image.

[0024] A contrast-enhanced image is obtained by performing linear pixel value transformation and pixel value boundary cropping on the illumination adaptive enhancement image.

[0025] The contrast-enhanced image is randomly horizontally flipped and noise is injected to obtain the enhanced stamped VIN code image, and a training dataset is constructed based on the enhanced stamped VIN code image and its corresponding VIN code character.

[0026] Further, the steps to obtain the illumination-adaptive enhanced image include:

[0027] The stamped VIN code image is converted into a single-channel grayscale image, then copied into a pseudo-RGB image with the same three-channel pixel values, and converted to the HSV color space to separate the hue channel, saturation channel and brightness channel.

[0028] Obtain the original hue value of the hue channel, the original luminance value of the luminance channel, and the original saturation value of the saturation channel;

[0029] The original hue value is randomly shifted within the prior color bias tolerance range to obtain the corrected hue channel; the original saturation value is randomly scaled according to the first scaling factor range to obtain the corrected saturation channel; the original luminance value is randomly scaled according to the second scaling factor range to obtain the corrected luminance channel.

[0030] The hue channel, saturation channel, and luminance channel are combined to obtain the adjusted HSV image, which is then converted back to RGB space and restored to a single-channel grayscale image to obtain the illumination-adaptive enhanced image.

[0031] Furthermore, the steps of performing pixel value linear transformation and boundary cropping on the illumination-adaptive enhanced image to obtain a contrast-enhanced image include:

[0032] The pixel values ​​of the illumination-adaptive enhancement image are mapped to a priori discrete integer range to obtain an integer pixel image;

[0033] Obtain the average grayscale value of the character region and the background region in the integer pixel image, and determine whether the difference between the two is less than the grayscale threshold. If yes, use the first contrast factor as the contrast adjustment parameter; otherwise, ignore the contrast adjustment parameter. Obtain and determine whether the maximum grayscale value of the integer pixel image is greater than the brightness threshold. If yes, use the second contrast factor as the contrast adjustment parameter; otherwise, ignore the contrast adjustment parameter. The first contrast factor is greater than the second contrast factor.

[0034] Obtain the average grayscale value of the integer pixel image and determine whether it is less than the first brightness threshold. If so, obtain the first brightness offset value as the brightness adjustment parameter. Determine whether the average grayscale value is greater than the second brightness threshold and obtain the second brightness offset value as the brightness adjustment parameter. The first brightness offset value is greater than 0, and the second brightness offset value is less than 0.

[0035] A linear transformation is performed on the integer pixel image based on the contrast adjustment parameters and the brightness adjustment parameters to obtain a linearly adjusted image;

[0036] Obtain the pixel values ​​of the linearly adjusted image and crop the pixel value boundaries according to the prior discrete integer interval. Map the cropped pixel values ​​to the normalized interval to obtain the contrast-enhanced image.

[0037] Furthermore, the feature extraction layer includes an encoder, as well as edge enhancement branches and LSTM branches arranged side-by-side with their inputs connected to the encoder's output;

[0038] The encoder is used to perform step-by-step multi-branch convolutional fusion and downsampling on the stamped VIN code image to obtain the main feature map;

[0039] The edge enhancement branch is used to extract edge features and adaptively weight the main feature map to obtain the enhanced feature map;

[0040] The LSTM branch is used to perform dimensionality transformation and sequence modeling on the main feature map to obtain character space features.

[0041] Furthermore, the encoder includes a downsampling module and a feature fusion residual block connected in sequence;

[0042] The downsampling module includes several cascaded downsampling elements; each downsampling element includes a residual block and a pooling component connected in sequence. Each residual block is used to perform multi-branch convolutional fusion on the input data, and each pooling component is used to downsample the input data to obtain downsampling features with different receptive fields.

[0043] The feature fusion residual block is used to perform multi-branch convolution fusion on downsampled features from different receptive fields to obtain the main feature map.

[0044] Furthermore, the edge enhancement branch includes Canny edge units and edge attention units connected in sequence;

[0045] Canny edge units are used to perform Sobel convolution on the main feature map to extract the gradient magnitude map;

[0046] The edge attention unit is used to map the gradient magnitude map to the channel dimension of the main feature map to generate adaptive attention weights. The gradient magnitude map and the main feature map are then weighted and fused according to the adaptive attention weights to obtain the enhanced feature map.

[0047] Furthermore, the LSTM branch includes a series of sequentially connected convolutional projection units, dimension transformation units, semantic constraint units, and sequence-to-space units;

[0048] The convolutional projection unit is used to perform convolution operations on the main feature map, projecting the main feature map to a dimension that matches the input of the LSTM branch, thus obtaining the projected feature map.

[0049] The dimension transformation unit is used to flatten the spatial dimension of the projected feature map according to the prior character width direction to obtain the character sequence;

[0050] The semantic constraint unit is used to model the character sequence using a bidirectional LSTM to capture the forward and backward contextual relationships of the characters, resulting in a character sequence with semantic constraints.

[0051] The sequence-to-space unit is used to reshape a semantically constrained character sequence into a feature map that matches the spatial dimension of the main feature map, thus obtaining character spatial features.

[0052] Furthermore, the feature fusion layer includes a residual fusion module and a global pooling module connected in sequence;

[0053] The residual fusion module is used to add the residuals of the main feature map, the enhanced feature map, and the character space features to obtain the fused features;

[0054] The global pooling module is used to perform global average pooling on the fused features to obtain the pooled fused features.

[0055] Secondly, this application also provides a computer storage medium storing executable program code; the executable program code is used to execute the deep learning-based vehicle body stamp VIN code character recognition method described in any one of the first aspects.

[0056] This invention provides a deep learning-based method and medium for recognizing VIN codes on vehicle body stamps. It involves collecting several historical VIN code images and labeling them with corresponding VIN code characters to construct a training dataset. Based on this dataset, a character recognition model is constructed and trained, taking the VIN code images as input and the VIN code characters as output. The model includes: an input layer for receiving VIN code images; a feature extraction layer for performing multi-branch convolutional fusion and downsampling on the VIN code images to obtain a main feature map, followed by edge feature extraction and sequence modeling to obtain an enhanced feature map and character space features; a feature fusion layer for residual fusion of the main feature map, enhanced feature map, and character space features to obtain fused features; and an output layer for outputting the VIN code characters based on the fused features. The current VIN code image is collected and input into the trained character recognition model to obtain the current VIN code character. This provides a novel recognition technology solution that can adapt to complex working conditions, fully utilize the structural characteristics of VIN codes and the logical relationships between characters, and possesses high accuracy and robustness. Attached Figure Description

[0057] Figure 1 This is a flowchart of a deep learning-based method for recognizing VIN codes on vehicle body stamps, according to an embodiment of the present invention.

[0058] Figure 2 This is a flowchart illustrating the construction of the training dataset according to an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the character recognition model according to an embodiment of the present invention;

[0060] Figure 4 This is another structural diagram of the character recognition model according to an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0062] It should be noted that if the embodiments of the present invention involve directional indications, such as up, down, left, right, front, back, etc., these directional indications are only used to explain the relative positional relationships and movement of the components in a specific posture. If the specific posture changes, the directional indications will also change accordingly. Furthermore, if the embodiments of the present invention involve descriptions such as "first," "second," "S1," "S2," "step one," "step two," etc., these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance, or implicitly indicating the number of technical features indicated or the execution order of the method. Those skilled in the art will understand that anything that does not violate the inventive concept should be included within the scope of protection of the present invention.

[0063] like Figure 1 As shown, this invention provides a deep learning-based method for recognizing VIN codes on vehicle body stamps, comprising:

[0064] S1: Collect several historical VIN code images and label the corresponding VIN code characters to construct a training dataset;

[0065] Specifically, it is possible, but not limited to, collecting several images of the vehicle body surface and cropping historical steel stamp VIN code images based on the prior ROI region, and then labeling the corresponding VIN code characters to construct a training dataset, providing a data foundation for subsequent training steps.

[0066] Preferably, since the steel stamp image dataset may have insufficient scene coverage during the acquisition process (e.g., only containing ideal lighting and no wear samples), leading to poor model generalization ability, it is advisable to generate diverse training samples by simulating common non-ideal conditions in industrial scenarios such as lighting fluctuations, dirt interference, and shooting deviations, so that the model can learn robust character features. Preferably, the input of this step is the original historical steel stamp area image (an RGB image obtained by resizing the vehicle body surface image to a size and structure suitable for the model input), and the output is a standardized 128×128 single-channel grayscale image (adapted to the model input size).

[0067] Preferably, the steps for collecting several historical VIN code images and labeling them with the corresponding VIN code characters to construct the training dataset are as follows: Figure 2 As shown, it also includes:

[0068] S11: Collect several historical VIN code images and label the corresponding VIN code characters;

[0069] Specifically, several historical VIN code images can be collected and labeled with the corresponding VIN code characters. The accuracy of the VIN code characters is ensured by manual labeling, providing reliable labels for supervised learning. This covers steel stamp samples of different car models, years, and wear levels, thereby improving the model's generalization ability.

[0070] S12: Convert each historical stamped VIN code image to the HSV color space, perform channel separation and illumination adaptive preprocessing, and then restore it to grayscale to obtain the illumination adaptive enhanced image;

[0071] Specifically, the brightness and chromaticity information can be decoupled using the HSV color space to achieve independent control of lighting factors. By independently controlling the three channels of hue, saturation, and brightness, the problems of uneven lighting and color shift can be solved, and then the image can be restored to grayscale to maintain consistency with the format of the model input.

[0072] Preferably, the step of obtaining an illumination-adaptive enhanced image includes:

[0073] S121: Convert the stamped VIN code image into a single-channel grayscale image, then copy it into a pseudo-RGB image with the same three-channel pixel values, and convert it to the HSV color space to separate the hue channel, saturation channel and brightness channel.

[0074] S122: Obtain the original hue value of the hue channel, the original luminance value of the luminance channel, and the original saturation value of the saturation channel;

[0075] Specifically, the stamped VIN code image can be converted into a single-channel grayscale image using a weighted average (e.g., R: 0.299, G: 0.587, B: 0.114) to retain the brightness information sensitive to the human eye and remove color redundancy. Then, the single-channel grayscale image is converted into a three-channel pseudo-RGB image through channel duplication, providing an adaptation format for HSV space conversion and ensuring compatibility with standard HSV conversion libraries. Independent hue, saturation, and luminance channels are obtained from the HSV space, achieving decoupled control of illumination and chromaticity factors. The original values ​​of the three channels are then obtained separately, providing a data basis for subsequent independent adjustments. The distribution information of the original values ​​is retained, making it easy to set the adjustment range according to prior constraints. Mutual interference between channels is avoided, ensuring the independence and controllability of each channel adjustment.

[0076] For example, the original stamped image (RGB format) can be converted into a single-channel grayscale image (achieved by weighted averaging of the three RGB channels, with weighting coefficients of R: 0.299, G: 0.587, B: 0.114), preserving the grayscale features of the character edges; then, the single-channel grayscale image is converted into a 3-channel pseudo-RGB image (with identical pixel values ​​in all three channels) through channel duplication, providing a suitable format for HSV space conversion; then, the pseudo-RGB image is converted to the HSV color space, separating the three channels representing hue (H), saturation (S), and brightness (V), and constrained adjustments are made according to the set rules.

[0077] S123: Randomly shift the original hue value within the prior color bias tolerance range to obtain the corrected hue channel; randomly scale the original saturation value according to the first scaling factor range to obtain the corrected saturation channel; randomly scale the original luminance value according to the second scaling factor range to obtain the corrected luminance channel.

[0078] Specifically, the following methods can be employed: First, the original hue value can be randomly shifted within the prior color bias tolerance range to introduce hue interference into the training data, providing a robust foundation for color bias in subsequent model learning. Second, the original saturation value can be randomly scaled according to a first scaling factor range to introduce saturation interference into the training data, providing a robust foundation for saturation deviation in subsequent model learning and reducing feature loss due to fading or reflection during subsequent recognition. Third, the original brightness value can be randomly scaled according to a second scaling factor range to introduce brightness interference into the training data, providing a robust foundation for brightness deviation in subsequent model learning, avoiding excessive influence of single lighting conditions on the enhancement results during subsequent recognition, and reducing the probability of local missing values.

[0079] For example, the original hue value of the hue channel can be randomly adjusted within ±5° (corresponding to a HSV space value range of 0.01~0.03) to add hue interference to the training data, thereby avoiding confusion between characters and background colors due to ambient light color bias during subsequent model recognition (such as a steel stamp image under red workshop lights), and preventing character feature distortion caused by over-adjustment.

[0080] For example, the original saturation value of the saturation channel can be randomly scaled by a factor of 0.8 to 1.2, so that the model can enhance the color contrast for faded stamps (low saturation) and reduce color interference for reflective stamps (high saturation) in the subsequent recognition process, ensuring that the grayscale gradient features of the character edges can be recognized.

[0081] For example, the original brightness value of the brightness channel can be randomly scaled by a factor of 0.7 to 1.3 to simulate low light (V value increased by 1.1 to 1.3 times) and strong light overexposure (V value decreased by 0.7 to 0.9 times) scenarios, so that the model can solve the problem of local character loss caused by uneven lighting (such as the edge of the stamp being blurred due to shadow) in the subsequent recognition process.

[0082] S124: Combine the corrected hue channel, corrected saturation channel, and corrected luminance channel to obtain the adjusted HSV image, convert it back to RGB space, and then restore it to a single-channel grayscale image to obtain an illumination-adaptive enhanced image.

[0083] Specifically, the hue channel, saturation channel, and luminance channel can be calibrated by combination to obtain an adjusted HSV image, which is then converted back to RGB space and restored to a single-channel grayscale image to obtain an illumination-adaptive enhanced image. This provides a standardized color space representation, facilitating subsequent format restoration and model input.

[0084] In this embodiment, the integrity of the color space is maintained by combining the corrected three channels into an adjusted HSV image; conversion back to RGB space ensures standard compatibility of the image format; finally, it is restored to a single-channel grayscale image to ensure that the output image retains the original grayscale distribution characteristics of the stamped characters, while adapting to the single-channel input requirements of the subsequent encoder; a format closed loop (RGB→grayscale→pseudoRGB→HSV→RGB→grayscale) is formed to ensure the reversibility and consistency of the enhancement process.

[0085] For example, the adjusted HSV image can be converted back to RGB space and then restored to a single-channel image through grayscale processing to ensure that the output image still retains the original grayscale distribution characteristics of the stamped characters, while adapting to the single-channel input requirements of subsequent models.

[0086] S13: Perform linear transformation of pixel values ​​and cropping of pixel value boundaries on the illumination adaptive enhancement image to obtain a contrast-enhanced image;

[0087] Specifically, the image can be linearly transformed for pixel values ​​and cropped at the boundary. The linear transformation stretches the range of pixel values, enhances the contrast of local textures, and prevents feature breakage caused by pixel overflow, ensuring that the enhanced sample still conforms to the distribution characteristics of the real stamped image.

[0088] Preferably, the step of performing pixel value linear transformation and boundary cropping on the illumination-adaptive enhanced image to obtain a contrast-enhanced image includes:

[0089] S131: Map the pixel values ​​of the illumination-adaptive enhancement image to a priori discrete integer range to obtain an integer pixel image;

[0090] Specifically, the pixel values ​​of the illumination-adaptive enhanced image can be obtained and mapped to a priori discrete integer range to avoid the loss of floating-point operation precision, ensure the accuracy of character edge grayscale values, and obtain an integer pixel image.

[0091] For example, the pixel values ​​of the grayscale image can be optionally restored from the normalized range of [0,1] to the prior discrete integer range of [0,255] to avoid precision loss caused by decimal operations.

[0092] S132: Obtain the average grayscale value of the character region and the background region in the integer pixel image, and determine whether the difference between the two is less than the grayscale threshold. If yes, use the first contrast factor as the contrast adjustment parameter; otherwise, ignore the contrast adjustment parameter. Obtain and determine whether the maximum grayscale value of the integer pixel image is greater than the brightness threshold. If yes, use the second contrast factor as the contrast adjustment parameter; otherwise, ignore the contrast adjustment parameter. The first contrast factor is greater than the second contrast factor.

[0093] Specifically, the average grayscale values ​​of the character region and the background region in the integer pixel image can be obtained in advance by those skilled in the art, and it can be determined whether the difference between the two is less than the grayscale threshold. If so, it is determined that the contrast is too low. Therefore, a high factor of the first contrast factor can be used to stretch the grayscale range to enhance the distinction between the character and the background. The overexposure situation is determined by the maximum grayscale value. When the maximum grayscale value is greater than the brightness threshold, it is determined to be strong light overexposure. A low factor of the second contrast factor is used to avoid the loss of details in the bright area. Adaptive enhancement is achieved through a dynamic parameter selection mechanism. The optimal strategy is automatically selected according to the image quality to avoid under-enhancement or over-enhancement caused by fixed parameters.

[0094] For example, the grayscale threshold can be selected as 30, the brightness threshold can be selected as 200, the first contrast factor can be selected as (1.2, 1.5), and the second contrast factor can be selected as (0.7, 1.0). When the difference between the two is less than the grayscale threshold, the value of the first contrast factor is inversely proportional to the difference. When the maximum grayscale value of the integer pixel image is greater than the brightness threshold, the value of the second contrast factor is inversely proportional to the maximum grayscale value.

[0095] For example, the contrast factor (contrast_factor) has a value of (0.7, 1.5). When the overall image of the stamp is blurred (e.g., rust causing a grayscale difference of <30 between the characters and the background), the first contrast factor (1.2, 1.5) is used to stretch the grayscale range. When there are local overly bright areas in the image (e.g., reflective spots), the second contrast factor (0.7, 1.0) is used to avoid losing details in bright areas. When there is neither an overall blurry stamp image nor local overly bright areas, the contrast factor can be ignored, or the value range of the contrast factor can be [1.0, 1.2].

[0096] More specifically, since the area where the stamped VIN code is located is small, the difference between the average gray values ​​of the character area and the background area is less than the gray value threshold and the maximum gray value of the integer pixel image is greater than the brightness threshold will not occur at the same time. Therefore, there is no need to consider the case where the first contrast factor and the second contrast factor are selected as contrast adjustment parameters at the same time.

[0097] S133: Obtain the average grayscale value of the integer pixel image and determine whether it is less than the first brightness threshold. If so, obtain the first brightness offset value as the brightness adjustment parameter. Determine whether the average grayscale value is greater than the second brightness threshold and obtain the second brightness offset value as the brightness adjustment parameter. The first brightness offset value is greater than 0, and the second brightness offset value is less than 0.

[0098] Specifically, the overall brightness state can be determined by the average grayscale value. When the average grayscale value is less than the first brightness threshold, it is determined to be a low-light, dark state, and the overall brightness is improved by positive offset of the first brightness offset value. When the average grayscale value is greater than the second brightness threshold, it is determined to be a strong light, overexposed state, and the bright area is suppressed by negative offset of the second brightness offset value. The bidirectional brightness adjustment covers low-light and overexposed scenes, solving the problem of partial character loss caused by uneven lighting.

[0099] For example, the first brightness threshold can be selected as 50, the second brightness threshold can be selected as 200, the first brightness offset value can be selected as 10 to 20, and the second brightness offset value can be selected as -20 to -10. When the average grayscale value is less than the first brightness threshold, the value of the first brightness offset value is inversely proportional to the average grayscale value. When the average grayscale value is greater than the second brightness threshold, the value of the second brightness offset value is inversely proportional to the average grayscale value.

[0100] For example, the brightness shift value (brightness_shift) has a range of (-20, +20). For dark stamped areas in low light (average grayscale value < 50), a positive shift of (+10, +20) is added to improve the overall brightness. For overexposed stamped areas in strong light (average grayscale value > 200), a negative shift of (-20, -10) is added to suppress bright areas. When there is neither low light nor strong light, the brightness shift value can be ignored, or the contrast factor can be set to a range of [-10, +10].

[0101] S134: Perform a linear transformation on the integer pixel image based on the contrast adjustment parameters and brightness adjustment parameters to obtain a linear adjustment image;

[0102] Specifically, a linear transformation can be performed on the integer pixel image based on the contrast adjustment parameters and the brightness adjustment parameters to obtain a linear adjustment image. By coordinating the adjustment of the contrast factor and the brightness offset value, the grayscale range and the brightness reference are stretched simultaneously to achieve joint optimization of contrast and brightness. The dynamically selected parameters ensure that the transformation intensity matches the image quality and avoid feature distortion caused by excessive transformation.

[0103] For example, an integer pixel image can be linearly adjusted using the formula adjusted_pixel = original_pixel × contrast_factor + brightness_shift, where adjusted_pixel is the linearly adjusted image, original_pixel is the integer pixel image, contrast_factor is the contrast factor, and brightness_shift is the brightness shift value.

[0104] S135: Obtain the pixel values ​​of the linear adjustment map and crop the pixel value boundaries according to the prior discrete integer interval. Map the cropped pixel values ​​to the normalized interval to obtain the contrast-enhanced image.

[0105] Specifically, the adjusted pixel values ​​can be cropped to the prior discrete integer range (optionally [0,255]) to avoid character feature breakage caused by exceeding the grayscale range (such as excessive enhancement causing character edge pixel values ​​to overflow); and then re-normalized to the range of [0,1] to ensure consistency with the model input format. No additional preprocessing is required. A numerical safety closed loop is formed through boundary cropping and normalization to ensure the reasonable distribution of the enhanced samples.

[0106] S14: Randomly flip the contrast-enhanced image horizontally and inject noise to obtain the enhanced stamped VIN code image, and construct the training dataset based on the enhanced stamped VIN code image and its corresponding VIN code character.

[0107] Specifically, the contrast-enhanced image can be randomly horizontally flipped to simulate shooting angle deviation, while maintaining the horizontal arrangement of characters to avoid logical errors caused by vertical flipping, resulting in an enhanced stamped VIN code image. This adds angle interference to the training data, providing a robust foundation for angle deviation in subsequent model learning and reducing the decrease in recognition accuracy caused by shooting angle deviation in the subsequent recognition process. Then, noise is generated according to a set standard deviation and superimposed on the flipped image to simulate sensor noise and surface reflective particles. At the same time, the standard deviation intensity is controlled to prevent the masking of key texture features, resulting in an enhanced stamped VIN code image. This provides a noise robust foundation for subsequent model learning and reduces the decrease in recognition accuracy caused by decreased shooting clarity or environmental noise in the subsequent recognition process.

[0108] For example, the grayscale image can be horizontally flipped with a 50% probability to simulate the left-right angle deviation of the camera during shooting (such as the horizontal tilt when a worker holds the equipment to shoot), while vertical flipping is prohibited (because the stamped characters are arranged horizontally, vertical flipping will cause the characters to be inverted, causing the model to learn incorrect features).

[0109] For example, Gaussian noise can be randomly generated with a standard deviation of 0.01 to 0.03 and superimposed on the grayscale image to simulate the granular interference caused by camera sensor noise and reflection from the steel stamp surface in industrial scenarios, thereby enhancing the model's tolerance to subtle noise. The upper limit of the noise standard deviation is controlled within 0.03 to avoid noise obscuring the key textures of character edges (such as the corner features of the numbers "0" and "6").

[0110] More specifically, strict parameter constraints can be set for all enhancement operations to ensure that the enhanced samples still conform to the distribution characteristics of the real embossed image:

[0111] After adjusting brightness and contrast, the average grayscale difference between the character area and the background should be kept above 20 (determined by pre-calculating the grayscale difference distribution of the original samples) to avoid over-enhancement that could cause character features to disappear.

[0112] The enhanced images are uniformly resized to 128×128 pixels using bicubic interpolation to ensure that the image size input to the subsequent improved ResUNet model is consistent, reducing the interference of scale changes on feature extraction.

[0113] Through the above enhancement process, the scene coverage of the original stamp image dataset can be improved by several times (including non-ideal scenes such as uneven lighting, noise interference, and low contrast), providing a rich and realistic sample basis for subsequent model training and significantly improving the model's recognition robustness in real industrial scenarios.

[0114] S2: Based on the training dataset, construct and train a character recognition model that takes the stamped VIN code image as input and the VIN code characters as output. Figure 3 , Figure 4 As shown, it includes:

[0115] The input layer is used to receive the image of the stamped VIN code;

[0116] The feature extraction layer is used to perform multi-branch convolutional fusion and downsampling on the stamped VIN code image to obtain the main feature map. Then, edge feature extraction and sequence modeling are performed on the main feature map to obtain the enhanced feature map and character space features.

[0117] The feature fusion layer is used to perform residual fusion of the main feature map, the enhanced feature map, and the character space features to obtain fused features, and to classify VIN code characters based on the fused features;

[0118] The output layer is used to output VIN code characters;

[0119] Specifically, the input layer can receive the stamped VIN code image to provide a data foundation for subsequent steps. Then, the feature extraction layer extracts the main feature map, edge feature map, and character space features from the stamped VIN code image, thereby capturing multi-scale semantic information, explicit edge contour information, and character contextual information, improving the completeness and robustness of feature representation. Next, the feature fusion layer performs residual fusion on the main feature map, enhanced feature map, and character space features to obtain fused features. By fusing complementary information from multi-dimensional features, the system achieves an organic unity between spatial detail enhancement and sequence semantic constraints, avoiding excessive influence of a single feature branch on the final result and reducing the probability of misidentification. Then, classification prediction is performed based on the fused features to obtain the VIN code character. Finally, the output layer outputs the VIN code character to provide accurate character recognition results, facilitating automated information management.

[0120] Specifically, the following details the processing steps of the character recognition model, including its specific structure and training methods. It is important to note that only preferred embodiments of model construction and training are provided, and these are not intended to limit the scope of the invention. The key to this invention lies in the hierarchical feature extraction of stamped VIN code images and the backbone-branch fusion of multi-source heterogeneous features. All technical solutions based on this inventive concept should be within the protection scope of this invention.

[0121] In a preferred embodiment, the feature extraction layer includes an encoder, and an edge enhancement branch and an LSTM branch arranged side by side with their inputs connected to the output of the encoder;

[0122] The encoder includes a downsampling module and a feature fusion residual block connected in sequence;

[0123] The downsampling module includes several cascaded downsampling elements; each downsampling element includes a residual block and a pooling component connected in sequence. The residual block is used to perform multi-branch convolutional fusion on the input data, and each pooling component is used to downsample the input data to obtain downsampling features with different receptive fields.

[0124] The feature fusion residual block is used to perform multi-branch convolutional fusion of downsampled features from different receptive fields to obtain the main feature map. Specifically, the VIN code image can be downsampled and fused stepwise through multi-branch convolutional fusion using a downsampling module. The receptive field is gradually expanded through a cascaded structure, while preserving multi-scale detail information and avoiding detail loss caused by large-scale downsampling at one time. The hierarchical richness of features is maintained through stepwise processing, providing rich hierarchical features for the subsequent feature fusion residual block. Then, the feature fusion residual block performs multi-branch convolutional fusion of downsampled features from different receptive fields. By capturing and fusing features from multiple scales, from details to the global scale, the feature map is obtained, avoiding feature loss caused by a single scale. For example, the encoder may adopt an improved ResUnet structure, which includes several cascaded downsampling modules (DownB) and residual blocks (ResB). The DownB module consists of ResB residual blocks and max pooling layers. The ResB residual blocks fuse different receptive field features through multi-branch convolution (Branch0-Branch4) to gradually expand the representation ability of character details and global semantics, and output the main feature map. The number of downsampling modules is preferably 4.

[0125] The edge enhancement branch is used to extract edges and adaptively weight the main feature map to obtain an enhanced feature map;

[0126] The LSTM branch is used to perform dimensionality transformation and sequence modeling on the main feature map to obtain character space features.

[0127] This embodiment provides specific implementations of the main feature map, enhanced feature map, and character space features. The specific selection of these three types of features precisely corresponds to the core difficulties in recognizing the vehicle body stamp VIN code:

[0128] Main Feature Map: The VIN code characters on the steel stamp exhibit multi-scale morphological changes in complex industrial scenarios (differences in the thickness of character strokes, overall arrangement structure, and global contextual relationships). The hierarchical receptive field of the convolutional neural network can capture spatial hierarchical features such as stroke details, character structure, and sequence layout layer by layer. By using hierarchical multi-branch convolutional fusion and downsampling to extract the main feature map, a hierarchical representation of the steel stamp characters can be established from the semantic dimension, effectively improving the completeness and robustness of feature expression.

[0129] Enhanced Feature Map: Due to rust, oil stains, and uneven lighting on the surface of the stamp, the character edges are blurred. Traditional convolution is insufficient in perceiving low-frequency edges, while the character edge contour is the key basis for distinguishing similar characters (such as "0" and "O", "1" and "I"). By explicitly extracting the gradient magnitude through Sobel convolution on the main feature map, and then using adaptive attention weighted fusion, the edge detail response can be enhanced while maintaining semantic information. It is more sensitive to recognizing worn and low-contrast stamps than simply relying on the main feature map, and avoids character misclassification caused by missing edge information.

[0130] Character space features: VIN code characters have strong order constraints (such as the fixed rule of the manufacturer code in the 1st-3rd digits and the year code in the 10th digit), but convolutional operations lack the ability to model long-distance sequence dependencies; by converting the main feature map into a character sequence and performing bidirectional LSTM modeling, the forward and backward contextual associations between characters can be captured. This is more effective than single-character classification in correcting misidentification of isolated characters (such as inferring the ambiguous "F" through the context of "LFV"), and avoids logical errors caused by local visual ambiguity.

[0131] Therefore, the extraction and specific selection of these three types of features can provide a comprehensive feature foundation for subsequent steps. The main feature map provides multi-scale semantic representation, the enhanced feature map provides explicit edge contours, and the character sequence provides sequential logical constraints. The three form a "semantic-edge-logic" closed loop, thereby covering the multi-dimensional technical requirements of vehicle body stamp VIN code recognition and effectively improving the recognition accuracy in complex industrial scenarios.

[0132] In a preferred embodiment, the edge enhancement branch includes a Canny edge unit and an edge attention unit connected in sequence:

[0133] Canny edge units are used to perform Sobel convolution on the main feature map to extract the gradient magnitude map;

[0134] The edge attention unit is used to map the gradient magnitude map to the channel dimension of the main feature map to generate adaptive attention weights. The gradient magnitude map and the main feature map are then weighted and fused according to the adaptive attention weights to obtain the enhanced feature map.

[0135] Specifically, a custom Sobel operator convolution can be performed on the main feature map using Canny edge units to extract gradient magnitude maps. By explicitly calculating gradients, the boundary information between characters and the background can be captured, which makes up for the shortcomings of implicit edge learning in convolutional neural networks. Then, the gradient magnitude maps are mapped to the main feature channel dimension through edge attention units to generate adaptive attention weights. The edge fusion intensity is dynamically adjusted in a data-driven manner to suppress edge detection artifacts (such as misjudging edges in rusted areas) and enhance the true character contours, resulting in an enhanced feature map.

[0136] In another preferred embodiment, the edge enhancement branch consists of a Canny edge unit and an edge attention unit. The Canny edge unit extracts the gradient magnitude map of the main feature map through convolution with a custom Sobel operator, explicitly capturing the character edge contours. The edge attention unit maps the gradient magnitude map to the main feature channel dimension, generates adaptive attention weights, and weights and fuses the edge information into the main feature map to suppress noise and enhance edge details.

[0137] In a preferred embodiment, the LSTM branch includes a convolutional projection unit, a dimension transformation unit, a semantic constraint unit, and a sequence-to-space unit connected in sequence.

[0138] The convolutional projection unit is used to perform convolution operations on the main feature map, projecting the main feature map to a dimension that matches the LSTM input to obtain the projected feature map.

[0139] The dimension transformation unit is used to flatten the spatial dimension of the projected feature map according to the prior character width direction to obtain the character sequence;

[0140] The semantic constraint unit is used to model the character sequence using a bidirectional LSTM to capture the forward and backward contextual relationships of the characters, resulting in a character sequence with semantic constraints.

[0141] The sequence-to-space unit is used to reshape a semantically constrained character sequence into a feature map that matches the spatial dimension of the main feature map, thus obtaining character spatial features.

[0142] Specifically, the main feature map can be convolved using a convolutional projection unit to project it onto a dimension that matches the LSTM input, resulting in a projected feature map. Dimensional adaptation is achieved by changing the number of channels while maintaining the integrity of the spatial structure, avoiding the destruction of positional information caused by direct flattening. Then, a dimension transformation unit flattens the spatial dimension of the projected feature map according to the prior character width direction (the horizontal arrangement direction of VIN code characters), converting the two-dimensional spatial features into a one-dimensional sequence arranged in the character reading order, providing a temporal structure for LSTM processing. Finally, a semantic constraint unit performs bidirectional LSTM modeling on the character sequence, capturing historical character information through a forward LSTM and future character information through a backward LSTM, and performing implicit logic verification using the fixed format rules of VIN code to obtain a character sequence with semantic constraints. Finally, a sequence-to-space unit reshapes the semantically constrained character sequence into a feature map that matches the spatial dimension of the main feature map, obtaining character spatial features for subsequent feature fusion.

[0143] In another preferred embodiment, the LSTM branch first projects the main feature map to a dimension matching the LSTM input through a 1×1 convolution, then flattens the spatial dimension to convert it into a character sequence (the length of which is the number of pixels in the character width direction), inputs it into a bidirectional LSTM (BiLSTM) to capture the forward and backward contextual associations of the characters, strengthens the semantic constraints of the VIN code character order (such as the logical order of manufacturer code and year information), and obtains a character sequence with semantic constraints. Finally, it reshapes the features to obtain the character space features.

[0144] In a preferred embodiment, the feature fusion layer includes a residual fusion module and a global pooling module connected in sequence:

[0145] The residual fusion module is used to add the residuals of the main feature map, the enhanced feature map, and the character space features to obtain the fused features;

[0146] The global pooling module is used to perform global average pooling on the fused features to obtain the pooled fused features.

[0147] Specifically, the residual fusion module can be used to add the residuals of the main feature map, the enhanced feature map, and the character space features. By complementing the information from multiple sources, the effective features of each branch are preserved, avoiding the prediction error caused by a single feature. At the same time, gradient multi-path is constructed to promote training convergence. Then, the fused features are globally averaged and pooled by the global pooling module to obtain the pooled fused features. By compressing the spatial dimension, the number of parameters in the fully connected layer is reduced, enhancing the positional robustness and preventing overfitting.

[0148] In a preferred embodiment, the output layer includes a fully connected module;

[0149] The fully connected module is used to convert the pooled fused features into several one-dimensional vectors and map all one-dimensional vectors to each character position. Then, the output of the fully connected layer is converted into a probability distribution to obtain the predicted probability of the character category corresponding to each character position. The character category with the highest classification probability is taken as the character corresponding to each character position, thus obtaining the VIN code character.

[0150] Specifically, the pooled fusion features can be classified using a fully connected module. The pooled fusion features can be converted into several one-dimensional vectors and mapped to 17 character positions. Then, the TaylorSoftmax function can be used to convert the output of the fully connected layer into a probability distribution to obtain the predicted probability of the character category corresponding to each character position. The character category with the highest classification probability is taken as the character corresponding to each character position to obtain the VIN code character and complete the recognition step.

[0151] In another preferred embodiment, the feature fusion layer first achieves multi-source information complementarity by adding the main feature map, enhanced feature map and character space features through residual addition. The fused features are then compressed in space dimension by global average pooling and then input to the fully connected module of the output layer. The fully connected module outputs the classification probability of 17 character positions (standard length of VIN code) and 36 types of characters (number + letter) at each position, thus completing the accurate recognition of VIN code characters.

[0152] In this embodiment, a deep learning-based method for recognizing VIN codes on vehicle body stamps is presented, the core of which is:

[0153] 1. The model uses an image of the stamped VIN code as input and sequentially performs multi-branch convolutional fusion, downsampling, edge enhancement, and sequence modeling on the image to obtain the main feature map, enhanced feature map, and character space features, which are then fused and predicted. This significantly improves the accuracy, robustness, and generalization ability of VIN code recognition, while also possessing core technical advantages such as complex scene adaptation, character logic error correction, and industrial-grade real-time processing. Specific effects include:

[0154] To overcome the limitations of single feature branches: The main feature map is extracted through progressive multi-branch convolutional fusion and downsampling by the encoder, which can capture multi-scale semantic information of stamped characters (such as stroke details, character structure, and global arrangement), but is sensitive to edge blurring and noise interference; the edge feature map is explicitly extracted through a custom Sobel operator, which can enhance the boundary distinction between characters and background, but lacks high-level semantic understanding; the character spatial features are modeled through bidirectional LSTM, which can capture the logical order constraints between VIN code characters (such as the fixed rules of manufacturer code and year position). After the fusion of the three source features, the feature dimensions are more comprehensive, and the full-chain association of "spatial details-edge contours-sequence logic" can be realized, reducing the missed or false judgments caused by single features and reducing recognition errors. Through feature complementarity, it can effectively distinguish between samples with "weary edges but compliant logic" and "clear edges but abnormal sequence", avoiding over-reliance on single feature fluctuations (such as edge detection failure caused by local reflection), while reducing the recognition missed judgments caused by single feature normality (such as clear edges but character logic errors).

[0155] Achieving robust recognition in complex industrial scenarios: Capturing key recognition signals from low-quality images. Actual acquisition scenarios for vehicle body VIN codes present complex interferences such as uneven lighting, surface corrosion, oil stains, and shooting angle deviations. These factors exhibit differentiated anomalies in different feature branches (e.g., uneven lighting leads to grayscale distortion in the main feature map, corrosion increases noise in the edge feature map, and angle deviation causes spatial misalignment of the character sequence). Feature fusion can identify complementary signals under these interferences, compensating for failures in single-branch features through other branches, providing technical assurance for highly reliable recognition in industrial settings.

[0156] 2. First, extract the main feature map, then extract the enhanced feature map and character space features from the main feature map, and finally fuse the main feature map, enhanced feature map, and character space features to obtain the fused feature. This feature processing paradigm of "main branch extraction - parallel branching - multi-source fusion" has unique technical effects compared with single-branch linear processing or simple parallel fusion, including clear feature hierarchy, full utilization of information, accurate complementarity, and low redundancy. Specifically, it includes:

[0157] A hierarchical feature extraction system is established to enhance feature reuse and gradient flow: the main feature map serves as a shared backbone feature, extracted once by the encoder and then input separately into the edge enhancement branch and the LSTM branch. This avoids computational redundancy caused by the two branches repeatedly extracting low-level features, while ensuring that the enhancement feature map and character space features are constructed based on the same semantic space, maintaining feature consistency. This hierarchical structure also alleviates the gradient vanishing problem in deep networks—gradients from the edge enhancement and LSTM branches can be backpropagated to the encoder through the main feature map, forming multi-path gradient flow, accelerating model convergence and improving training stability.

[0158] To achieve parallel complementarity between edge details and sequence logic and reduce feature coupling: Enhanced feature maps and character space features are extracted in parallel from the main feature map, allowing spatial edge information and sequence logic information to be optimized independently at the same semantic level. This avoids error accumulation caused by sequential processing (e.g., when edge enhancement is performed before sequence modeling, edge noise can interfere with sequence learning). Two branches perform specific optimizations for different characteristics of the main feature map: the edge enhancement branch explicitly strengthens local gradients through the Sobel operator, while the LSTM branch captures long-distance dependencies through bidirectional modeling. The two branches do not interfere with each other and have complementary gains. Finally, a fusion layer is used to achieve joint optimization of "spatial accuracy + semantic coherence".

[0159] Accurately fusing multi-source heterogeneous features enables full-chain "visual-logical" recognition modeling: the fusion layer adds the residuals of the main feature map (basic semantics), enhanced feature map (edge ​​details), and character space features (logical constraints), forming complementary multi-source information. This fusion mechanism can accurately establish a recognition chain of "original features → edge enhancement → sequence verification": the main feature map provides a complete visual representation, the enhanced feature map corrects the feature response of blurred edge areas, and the character space features suppress abnormal predictions that do not conform to VIN code rules. Compared to directly fusing different modalities of the original input, this "trunk-branch" structure can significantly improve the correlation and interpretability between features and VIN code characters.

[0160] To reduce feature redundancy and noise interference and improve feature quality: the main feature map, enhanced feature map, and character space features originate from the same backbone and undergo differentiated processing, resulting in independent noise types (convolutional noise in the main feature map, edge detection artifacts in the enhanced feature map, and sequence modeling errors in the character space features). Multi-source fusion can suppress noise from a single branch through information complementarity: when a region in the main feature map produces false textures due to corrosion, the enhanced feature map can reduce the weight of that region through edge consistency detection; when the character space features produce logical biases due to insufficient long-distance dependence, the spatial distribution of the main feature map can provide location priors for correction. This noise independence makes the signal-to-noise ratio of the fused features significantly higher than that of single-branch features.

[0161] Enhanced model interpretability and industrial debuggability: The hierarchical structure makes the source of features clear and traceable, allowing field engineers to pinpoint and identify the specific error: if the error originates from the main feature map, it suggests optimizing the encoder or improving image acquisition quality; if the error originates from the enhanced feature map, it suggests adjusting edge detection parameters; if the error originates from character space features, it suggests verifying the LSTM's sequence modeling capabilities or VIN code logic rules. This interpretability makes model debugging and troubleshooting in industrial settings more targeted, avoiding the difficulties of maintaining "black box" models.

[0162] Enhancing model flexibility and deployment adaptability: The trunk-branch structure supports flexible model pruning: In scenarios with limited computing resources (such as edge computing devices), basic recognition can be performed using only the main feature map; in scenarios with high accuracy requirements, enhanced branches can be enabled to strengthen edge quality; in scenarios with strict logic verification requirements, LSTM branches can be enabled for rule constraints. Compared to end-to-end single-branch models, this modular structure is more adaptable to deployment conditions in different industrial settings, and a dynamic trade-off between accuracy and efficiency can be achieved through branch switching.

[0163] S3: Collect the current stamped VIN code image and input it into the trained character recognition model to obtain the current VIN code character.

[0164] Specifically, it is optional, but not limited to, acquiring images of the VIN code stamped on the current vehicle body and inputting them into the trained character recognition model to obtain the VIN code character corresponding to the current vehicle body.

[0165] Compared with existing technologies, this invention addresses the pain points of traditional VIN code recognition methods through technological innovation, with the following specific benefits:

[0166] 1. Significantly improved anti-interference capability in complex industrial scenarios, high recognition accuracy, and able to meet actual high-precision recognition needs.

[0167] Traditional barcode / QR code recognition relies on the integrity of the carrier and is susceptible to interference from lighting and stains (failure occurs when the coverage area is ≥30%). Traditional machine vision methods are sensitive to surface corrosion, paint coverage, or shooting angle deviations (±5° or more) of the stamped VIN code. This invention utilizes the synergistic effect of edge enhancement branches and improved ResUnet multi-scale feature extraction: the edge enhancement module (Canny edge unit + edge attention unit) explicitly extracts the edge contour of the stamped VIN, and enhances edge details through an adaptive attention fusion mechanism, suppressing feature blurring caused by lighting fluctuations and surface corrosion; based on the Inception architecture in the ResUnet encoder, multi-branch convolution fuses features from different receptive fields (single-branch 3×3 convolution with a 3×3 receptive field, and multi-branch stacking correspondingly expands the receptive field), enhancing the ability to represent complex textures such as character deformation and local missing parts.

[0168] 2. Make full use of character logical associations to reduce the error rate of illegal character combinations.

[0169] Traditional deep learning models (such as general CNNs) focus only on local features, ignoring the logical relationships between VIN code characters (such as the order constraints of manufacturer codes and year information). This invention addresses this problem by using a bidirectional LSTM sequence modeling branch: the high-dimensional features output by the ResUnet encoder are flattened into a spatial sequence (the length of which is the number of pixels in the character width direction), and input into the bidirectional LSTM to capture forward and backward contextual information, making full use of the logical relationships between characters inherent in the VIN code.

[0170] 3. It has strong generalization ability, adapts to diverse industrial data, and meets the long-term stable needs of production lines (traditional methods rely on highly diverse data for training).

[0171] Traditional deep learning models suffer from poor generalization ability to non-standard scenes (such as low-contrast, blurry images) due to the lack of diversity in training data (mostly ideal images). This invention reduces data dependence through a multi-feature fusion strategy and residual connection design: the original ResB features preserve global semantics, edge enhancement features supplement details, and LSTM features provide sequence constraints. These three complement each other to reduce dependence on a single feature; residual connections alleviate gradient vanishing in deep networks and improve the model's ability to fit complex samples.

[0172] 4. High real-time performance, suitable for high-speed detection in production lines.

[0173] This invention optimizes inference efficiency through a lightweight architecture design: the encoder adopts ResUnet residual connections and multi-scale feature sharing to reduce redundant computation; edge enhancement and LSTM branching are computed in parallel to avoid serial latency. It can be seamlessly integrated into industrial edge devices, supporting continuous inspection with "zero interruption" on the production line.

[0174] On the other hand, the present invention also provides a computer storage medium storing executable program code; the executable program code is used to execute any of the above-mentioned deep learning-based vehicle body stamp VIN code character recognition methods.

[0175] On the other hand, the present invention also provides a terminal device, including a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute any of the above-mentioned deep learning-based vehicle body stamp VIN code character recognition methods.

[0176] For example, the program code can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the program code in the terminal device.

[0177] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the terminal device may also include input / output devices, network access devices, buses, etc.

[0178] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0179] The memory can be an internal storage unit of the terminal device, such as a hard drive or RAM. The memory can also be an external storage device of the terminal device, such as a plug-in hard drive, SmartMediaCard (SMC), Secure Digital (SD) card, or FlashCard. Furthermore, the memory can include both internal and external storage units of the terminal device. The memory is used to store the program code and other programs and data required by the terminal device. The memory can also be used to temporarily store data that has been output or will be output.

[0180] The aforementioned computer storage medium and terminal device are created based on the aforementioned deep learning-based vehicle body stamp VIN code character recognition method. Their technical functions and beneficial effects will not be elaborated here. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for recognizing VIN codes on vehicle body stamps based on deep learning, characterized in that, include: Collect several historical VIN code images and label the corresponding VIN code characters to construct a training dataset; Based on the training dataset, a character recognition model is constructed and trained that takes the image of the stamped VIN code as input and the VIN code character as output, including: The input layer is used to receive the image of the stamped VIN code; The feature extraction layer includes an encoder, and parallel edge enhancement branches and LSTM branches whose inputs are connected to the encoder's output. The encoder performs multi-branch convolutional fusion and downsampling on the stamped VIN code image to obtain the main feature map, including sequentially connected downsampling modules and feature fusion residual blocks. The downsampling modules include several cascaded downsampling elements; each downsampling element includes sequentially connected residual blocks and pooling components. Each residual block performs multi-branch convolutional fusion on the input data, and each pooling component downsamples the input data to obtain downsampled features with different receptive fields. The feature fusion residual blocks perform multi-branch convolutional fusion on the downsampled features with different receptive fields to obtain the main feature map. The edge enhancement branch extracts edge features from the main feature map and adaptively weights them to obtain an enhanced feature map. The LSTM branch converts the main feature map into a character sequence and performs bidirectional LSTM modeling to obtain a semantically constrained character sequence, which is then reshaped to obtain character space features. The feature fusion layer is used to perform residual fusion of the main feature map, the enhanced feature map, and the character space features to obtain fused features; The output layer is used to output VIN code characters based on the fusion features; The current VIN code image is captured and input into the trained character recognition model to obtain the current VIN code character.

2. The method according to claim 1, characterized in that, The steps of collecting several historical VIN code images and labeling the corresponding VIN code characters to construct a training dataset also include: Collect several historical VIN code images and label the corresponding VIN code characters; Each historical stamped VIN code image is converted to the HSV color space, channel separation and illumination adaptive preprocessing are performed, and then restored to grayscale to obtain an illumination adaptive enhanced image. A contrast-enhanced image is obtained by performing linear pixel value transformation and pixel value boundary cropping on the illumination adaptive enhancement image. The contrast-enhanced image is randomly horizontally flipped and noise is injected to obtain the enhanced stamped VIN code image, and a training dataset is constructed based on the enhanced stamped VIN code image and its corresponding VIN code character.

3. The method according to claim 2, characterized in that, The steps to obtain an illumination-adaptive enhanced image include: The stamped VIN code image is converted into a single-channel grayscale image, then copied into a pseudo-RGB image with the same three-channel pixel values, and converted to the HSV color space to separate the hue channel, saturation channel and brightness channel. Obtain the original hue value of the hue channel, the original luminance value of the luminance channel, and the original saturation value of the saturation channel; The original hue value is randomly shifted within the prior color bias tolerance range to obtain the corrected hue channel; the original saturation value is randomly scaled according to the first scaling factor range to obtain the corrected saturation channel; the original luminance value is randomly scaled according to the second scaling factor range to obtain the corrected luminance channel. The hue channel, saturation channel, and luminance channel are combined to obtain the adjusted HSV image, which is then converted back to RGB space and restored to a single-channel grayscale image to obtain the illumination-adaptive enhanced image.

4. The method according to claim 2, characterized in that, The steps for performing linear pixel value transformation and boundary cropping on an illumination-adaptive enhanced image to obtain a contrast-enhanced image include: The pixel values ​​of the illumination-adaptive enhancement image are mapped to a priori discrete integer range to obtain an integer pixel image; Obtain the average grayscale value of the character region and the background region in the integer pixel image, and determine whether the difference between the two is less than the grayscale threshold. If yes, use the first contrast factor as the contrast adjustment parameter; otherwise, ignore the contrast adjustment parameter. Obtain and determine whether the maximum grayscale value of the integer pixel image is greater than the brightness threshold. If yes, use the second contrast factor as the contrast adjustment parameter; otherwise, ignore the contrast adjustment parameter. The first contrast factor is greater than the second contrast factor. Obtain the average grayscale value of the integer pixel image and determine whether it is less than the first brightness threshold. If so, obtain the first brightness offset value as the brightness adjustment parameter. Determine whether the average grayscale value is greater than the second brightness threshold and obtain the second brightness offset value as the brightness adjustment parameter. The first brightness offset value is greater than 0, and the second brightness offset value is less than 0. A linear transformation is performed on the integer pixel image based on the contrast adjustment parameters and the brightness adjustment parameters to obtain a linearly adjusted image; Obtain the pixel values ​​of the linearly adjusted image and crop the pixel value boundaries according to the prior discrete integer interval. Map the cropped pixel values ​​to the normalized interval to obtain the contrast-enhanced image.

5. The method according to claim 1, characterized in that, The edge enhancement branch includes Canny edge units and edge attention units connected in sequence; Canny edge units are used to perform Sobel convolution on the main feature map to extract the gradient magnitude map; The edge attention unit is used to map the gradient magnitude map to the channel dimension of the main feature map to generate adaptive attention weights. The gradient magnitude map and the main feature map are then weighted and fused according to the adaptive attention weights to obtain the enhanced feature map.

6. The method according to claim 1, characterized in that, The LSTM branch includes a series of sequentially connected convolutional projection units, dimension transformation units, semantic constraint units, and sequence-to-space units; The convolutional projection unit is used to perform convolution operations on the main feature map, projecting the main feature map to a dimension that matches the input of the LSTM branch, thus obtaining the projected feature map. The dimension transformation unit is used to flatten the spatial dimension of the projected feature map according to the prior character width direction to obtain the character sequence; The semantic constraint unit is used to model the character sequence using a bidirectional LSTM to capture the forward and backward contextual relationships of the characters, resulting in a character sequence with semantic constraints. The sequence-to-space unit is used to reshape a semantically constrained character sequence into a feature map that matches the spatial dimension of the main feature map, thus obtaining character spatial features.

7. The method according to any one of claims 1 to 6, characterized in that, The feature fusion layer includes a residual fusion module and a global pooling module connected in sequence; The residual fusion module is used to add the residuals of the main feature map, the enhanced feature map, and the character space features to obtain the fused features; The global pooling module is used to perform global average pooling on the fused features to obtain the pooled fused features.

8. A computer storage medium, characterized in that, The system stores executable program code; the executable program code is used to execute the deep learning-based vehicle body stamp VIN code character recognition method according to any one of claims 1 to 7.

Images