Tire imprint character dynamic recognition method based on a cooperative network

Abstract

The application provides a tire character dynamic identification method based on a cooperative network, first, a YOLO-V5s network is used to identify a tire in a captured image, and a picture is cut according to the outline of a detection frame; the tire image obtained through cutting is divided into four parts, an ENet is used to locate a character area in the part, and a noise area with too small area is filtered out; based on a region growing algorithm, a complete character area is extracted; a circumscribed rectangle is made for an irregular mask area, and four segmented images are spliced; the area of the character area after splicing is calculated and sorted, and a non-target character area is filtered out; after making a circumscribed rectangle again, the character is in a normal position through image rotation and projection development, and the character is straightened; a YOLO-V5s network is used to identify the character in the image block. The application has high stability, low cost, wide application scene, can dynamically realize identification of a specified character part, and filters out redundant information.

Description

Technical Field

[0001] This invention belongs to the field of image processing and relates to a method for dynamic recognition of tire embossed characters based on cooperative networks. Background Technology

[0002] In recent years, my country's land transportation has flourished, with provincial and national highways experiencing peak traffic volumes from heavy and medium-sized trucks, leading to frequent accidents involving overloaded and oversized vehicles. Based on the principles of physics, tires deform during operation, and the deformation characteristics can be used to infer the stress on the tires, determining information such as vehicle load and tire lifespan. However, due to differences in tire size, material, and model, the deformation produced by tires under the same stress conditions varies. Therefore, it is essential to identify the key information in the tire's embossed characters to obtain basic manufacturing information. Tire embossed characters mainly include tire brand, tire specifications, technical parameters, production date and certification marks, as well as other special markings. Some of this information is irrelevant to the tire's performance and is redundant during actual inspection. Furthermore, the character arrangement varies significantly between different tires, placing high demands on OCR algorithms.

[0003] At present, there are the following limitations in the process of imprinted character recognition: (1) There are few methods for character recognition and detection on the tire sidewall, most of which are done manually. This not only requires the tire to be stationary, but also results in low detection accuracy; (2) Most machine vision-based character recognition methods also require the image to be stationary and cannot recognize tilted characters. The image orientation needs to be manually adjusted, which is not suitable for recognizing tire sidewall characters arranged in a circle; (3) It cannot filter out character areas that are unrelated to the tire's performance, resulting in poor positioning accuracy, slow recognition speed, and low recognition efficiency.

[0004] Therefore, there is an urgent need to provide an algorithm with high accuracy that can automatically identify tire appearance characters and detect dynamic tires, in order to overcome the above-mentioned shortcomings. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic recognition method for tire imprint characters based on a collaborative network, thereby overcoming the technical bottlenecks of existing tire character recognition devices, such as the inability to recognize static tires, the need for manual direction adjustment, and low accuracy. To solve the above technical problems, the technical solution adopted by this invention is as follows:

[0006] A method for dynamic recognition of tire imprint characters based on cooperative networks includes the following steps:

[0007] Step 1: Create a tire image based on the YOLO-V5 network and the original image containing the tire;

[0008] Step 2: Create a dataset for character localization based on tire images;

[0009] Step 3: Based on the ENet network model and the character localization dataset, obtain the weights of the ENet network model and load the weights into the ENet network model to generate the trained ENet network model.

[0010] Step 4: Based on the trained ENet network model, perform character localization on the tire image to be recognized, and output the segmented character localization map:

[0011] Step 4A: Crop the tire image to be identified into 4 images along the vertical and horizontal centerlines;

[0012] Step 4B: Input the four images into the ENet network model trained in Step 3 to locate the characters in the tire image;

[0013] Step 4C: Use the logical target filtering instruction to select character regions with an area greater than 2000 pixels for each image;

[0014] Step 4D: For the selected character region, use the region growing algorithm and the cv2.minAreaRect function to output the character location map corresponding to each image;

[0015] Step 5: Perform character morphology processing on the four segmented character positioning images and the four original tire images obtained in Step 4A corresponding to the four character positioning images, so as to output a complete character positioning image and tire image corresponding to each tire image in Step 4A; wherein, the character positioning image contains the four largest character regions.

[0016] Step 6: Based on the character region and the tire image generated in Step 5A, perform an image rotation operation to output a forward rectangular stitched image;

[0017] Step 7: Output the straightened character area in the positive direction: Based on the image of the positive rectangle stitching, perform a fan-shaped character expansion operation to make the curved characters vertical, so as to improve the efficiency and accuracy of character recognition.

[0018] Step 8: Input the straightened character region into the trained YOLO-V5 network to output identifiers arranged in the order shown in the tire image.

[0019] Preferably, step 1 specifically includes:

[0020] Step 1 A, Image Capture: Use a high-speed camera combined with supplemental lighting to capture raw images of the vehicle, including the tires, while it is in motion;

[0021] Step 1B, Tire Positioning: The original images acquired in Step 1A are used to identify and position the tires using the YOLO-V5s network;

[0022] Step 1 C, Image Cropping: Cropping is performed based on the detection box to obtain the tire image.

[0023] Preferably, in step 2, the dataset for character localization includes a test set, a training set, and a validation set.

[0024] Preferably, in step 3, the ENet network model training process specifically includes:

[0025] Step 3A: Based on the training set in Step 2, perform segmentation training on the ENet network model, and then validate the ENet network model based on the validation set in Step 2.

[0026] Step 3B: When the verification result reaches the set value, load the weights generated after training the ENet network model into the ENet network model to generate the trained ENet network model.

[0027] Preferably, step 8, the YOLO-V5 network training process specifically includes:

[0028] Step 8A: Straighten the character regions corresponding to multiple original images to create a character recognition dataset, which includes a test set, a training set, and a validation set.

[0029] Step 8B: Based on the training set in Step 8A, train the YOLO-V5s network for character feature recognition, and then validate the YOLO-V5s network based on the validation set in Step 8A.

[0030] Step 8C: When the verification result reaches the set value, load the weights generated after training the YOLO-V5s network into the YOLO-V5s network to generate the trained YOLO-V5s network.

[0031] Compared with the prior art, the advantages of the present invention are:

[0032] 1) It has high detection stability, low detection cost, and is suitable for multiple scenarios. It can comprehensively and effectively realize the tire character positioning and tire character information recognition of tire sidewall characters.

[0033] 2) By training the ENet network, the effective character parts can be located, and interference from redundant information on the tire can be filtered out.

[0034] 3) By filtering out small character areas, some noise interference is effectively eliminated.

[0035] 4) By cropping the image into four equally sized regions for training and recognition, detection accuracy was improved. Data augmentation was used to reduce the number of original images required and shorten the model training time. Attached Figure Description

[0036] Figure 1 This is a flowchart of the present invention;

[0037] Figure 2 This is a schematic diagram illustrating the process and function of each module of the present invention;

[0038] Figure 3 An illustration of the effect of YOLO-V5 network identifying and locating tires;

[0039] Figure 4 This is a high-resolution tire sidewall image obtained after recognition and cropping.

[0040] Figure 5 This is a schematic diagram of the ENet network model.

[0041] Figure 6 The image consists of a mask and a tire image that has undergone cropping, segmentation, character recognition, and preliminary logical target filtering.

[0042] Figure 7 Grayscale images for locating characters, filtering logical targets, and enclosing rectangles for the ENet network.

[0043] Figure 8 The rectangular mask image obtained from the stitching and filtering operations;

[0044] Figure 9 Rotate the rectangular mask area to the correct position of the tire image;

[0045] Figure 10 A diagram of a rectangular text area rotated to its correct position;

[0046] Figure 11 This is a diagram showing the effect of character projection transformation;

[0047] Figure 12 This is a screenshot showing the character recognition results based on the YOLO-V5 network. Implementation

[0048] The following will describe in more detail the method for dynamic recognition of tire embossed characters based on a collaborative network according to the present invention, with reference to the schematic diagrams illustrating preferred embodiments of the invention. It should be understood that those skilled in the art can modify the invention described herein while still achieving its advantageous effects. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the invention.

[0049] like Figure 1 and Figure 2 As shown, a method for dynamic recognition of tire imprint characters based on cooperative networks includes the following steps:

[0050] Step 1: Create a tire image based on the YOLO-V5 network and the original image containing the tire.

[0051] Step 1 A, Image Capture: Use a high-speed, high-resolution camera combined with supplemental lighting to capture raw images of the moving vehicle, including the tires. The high-speed, high-resolution camera ensures that the character area has more than 2000 pixels.

[0052] Specifically, when it is necessary to inspect the tire deformation or lifespan of a vehicle in motion, high-speed cameras and supplementary lighting equipment can be installed at toll booths or on both sides of single-lane roads to capture images (raw images) of the lower half of the vehicle, including clear and bright tires, in dimly lit environments.

[0053] Step 1B, Tire Positioning: The original image acquired in Step 1A is used to identify and position the tires using a YOLO-V5s network, specifically as follows: Figure 3 As shown. Specifically, when creating the sample dataset used to build the YOLO-V5 network, 430 vehicle photos should be selected. The target tires in the photos should be manually bounded using labelimg software, and the labels should be stored in XML format to create the sample dataset required for training.

[0054] The YOLO-V5s network was used for learning, testing, and validating tire features, achieving a learning accuracy of over 90%.

[0055] Finally, the image to be detected is input into the validated YOLO-V5s network; such as Figure 5 As shown.

[0056] Step 1 C, Image Cropping: Cropping is performed based on the detection bounding box to obtain the tire image. Based on the detection bounding box ( Figure 3 Cropping out the tire image and saving it, at which point the center of the tire and the center of the image are basically coincident, as shown in the example below. Figure 4 As shown.

[0057] Step 2: Based on tire images, a dataset for character localization is created, which is an existing technology.

[0058] The dataset for character localization includes a test set, a training set, and a validation set.

[0059] The dataset includes the tire images obtained in step 1C, the augmented images obtained in step 2B, and label files (two types: those corresponding to the tire images and those corresponding to the augmented images).

[0060] Dataset: Includes color images (tire) + grayscale images (label files), belonging to existing techniques involved in deep learning.

[0061] The specific process is as follows:

[0062] Step 2A: Create the label file;

[0063] For the 430 tire images acquired, LabelMe software was used to label the text regions containing the tire's large-size markings, small-size markings, location markings, inflation pressure, and load index. For example... Figure 10 As shown, (a) to (d) sequentially display the text areas containing the small-sized marker, inflation pressure and load index, site marker, and large-sized marker.

[0064] The preferred calibration process is as follows: starting from the upper left of the text area, proceed clockwise along the text outline, passing through the lower right of the text area, and outline the target with a closed polygon.

[0065] Four different text region labels are created for each image. Since the label files generated by labelme are in JSON format, the JSON label files need to be converted into grayscale PNG images and stored in another folder.

[0066] In the grayscale image generated by the labelme software, the background pixel grayscale is 0, and the pixel grayscale of the text area is 1, 2, 3, and 4 respectively.

[0067] Step 2B: Dataset augmentation.

[0068] In step 2A, the grayscale image and its corresponding color image are cropped along the vertical and horizontal central axes, respectively. Each image is divided into 4 parts, resulting in 1720 samples. The grayscale image and its corresponding color image that do not contain the target text area are deleted, leaving 1271 samples.

[0069] Four methods were selected from horizontal flipping, vertical flipping, Gaussian blur, affine transformation, random deformation, cropping, and scaling to increase the sample size to 5 times the original size, resulting in 6355 data color images and 6355 corresponding label grayscale images (label files).

[0070] Step 2C: Divide the dataset: Randomly divide the (color images and corresponding label files) obtained in Step 2B into a training set, a validation set, and a test set in an optimal ratio of 8:1:1. Store the image names in the training set in train.txt, the image names in the validation set in val.txt, and the image names in the test set in test. Generate a sample dataset for character localization.

[0071] Step 3: Based on the ENet network model and the character localization dataset, obtain the weights of the ENet network model and load the weights into the ENet network model to generate the trained ENet network model.

[0072] The ENet network model training process specifically includes:

[0073] Step 3A: Based on the training set in Step 2, perform segmentation training on the ENet network model, and then validate the ENet network model based on the validation set in Step 2.

[0074] The ENet network first converts the image to a predetermined size of 1024×1024. Then, it extracts image features and removes visual redundancy through convolution, pooling, and stitching operations in the initial module. Next, it performs downsampling operations through the bottleneck module to ultimately achieve pixel classification, as detailed below. Figure 5 As shown.

[0075] Step 3B: When the verification result reaches the set value, load the weights generated after training the ENet network model into the ENet network model to generate the trained ENet network model.

[0076] With an optimal learning rate of 0.0005 and a batch size of 2, after 300 rounds of training and validation, the mean IoU ratio of the validation set reaches 56%, at which point the network training can be considered complete.

[0077] Step 4: Based on the trained ENet network model, perform character localization on the tire image to be identified and output the segmented character localization map.

[0078] Step 4A: Cut the tire image to be identified into 4 images along the vertical and horizontal centerlines.

[0079] Step 4B: Input the four images into the ENet network model trained in Step 3 to locate the characters in the tire image.

[0080] Step 4C: Use the logical target filtering instruction to select character regions with an area greater than 2000 pixels for each image;

[0081] Step 4D: For the selected character region, use the region growing algorithm and the cv2.minAreaRect function to output the character location map corresponding to each image; for example... Figure 6 As shown in Figure (d).

[0082] Specifically, combining region growing algorithms yields more complete character regions, such as... Figure 6 (The image shows a tire) with a text area.

[0083] The `cv2.minAreaRect` function is used to find the minimum area bounding rectangle of the character region. The specific processes of character positioning, logical target filtering, region growing, and rectangle bounding are as follows: Figure 7 As shown.

[0084] Figure 7 and Figure 6 Correspondence, that is Figure 7 In the middle, Figures (a-1) to (a-3) correspond to Figure 6 The middle figure (a) and so on.

[0085] Figure 7 In each image, 1 represents character positioning, 1-2 represent the logical target filtering and region growing process, and 2-3 represent the rectangle bounding operation. Specifically, Figure 7 In the diagram, Figure (a-1) corresponds to character positioning, Figures (a-1) to (a-2) represent the process of logical target filtering and region growth, and Figures (a-2) to (a-3) represent rectangle enclosing operations.

[0086] Figure 7 The classification of other illustrations is the same as above, and so on.

[0087] The ENet network used in this embodiment can still recognize and locate multi-shaped, curved, and inverted characters, demonstrating high robustness.

[0088] Step 5: Perform character morphology processing on the four segmented character positioning images and the four original tire images obtained in Step 4A corresponding to the four character positioning images, so as to output a complete character positioning image and tire image corresponding to each tire image in Step 4A.

[0089] Specifically, the four segmented character positioning images and the four images corresponding to the four character positioning images are processed to make the character positioning areas in the tire image and the grayscale image correspond to each other.

[0090] The purpose of splitting and then splicing is to connect target characters that may have been cut apart; splitting makes it easier to locate characters.

[0091] Step 5 A1, Character Concatenation: Combine the four regions (4 images) segmented in Step 4. Figure 6 The four location images obtained from the location are stitched together to form a composite image (four character location images). Figure 7 This allows the separated characters to be rejoined. For example... Figure 8 As shown in Figure (a).

[0092] Step 5 A2, Filtering Operation: Calculate and sort the areas of the spliced ​​character regions. To avoid missed detections, perform logical target filtering again, taking the four regions with the largest areas and drawing them directly in the original image (marking these four character regions in the original tire image using code). The rectangular mask in the corresponding grayscale image of the original image is as follows: Figure 8 As shown in Figure (b).

[0093] Based on the character location map and the tire image, the area of ​​the stitched character regions is calculated and sorted; logical target filtering is then performed again, and finally the top four character regions by area are output, such as... Figure 8 As shown in Figure (b).

[0094] Step 6: Based on the character region and the generated tire image from Step 5, perform an image rotation operation (existing technology) to output a positively oriented rectangular stitched image, such as... Figure 9 and Figure 10 As shown.

[0095] Specifically, the angle between the line connecting the center of the character region to the center of the entire image (character positioning map) and the vertical central axis of the image (character positioning map) is calculated, and the target character region is rotated to its correct position. That is, for... Figure 8 In the middle, the color image (tire image) corresponding to Figure (b) is rotated.

[0096] The preferred method for rotating the aforementioned character image (existing technology) is as follows:

[0097] Calculate the center of the rectangular mask region, the center of the original image, and the center of the top edge of the original image in step 5A2. Calculate the angle between the line connecting the center point and the center of the mask region, and the angle between the line connecting the center point of the mask region and the top edge center. Straighten the tilted target mask region by rotating it in the appropriate direction and angle, such as... Figure 9 As shown. The rectangle that has been straightened is cropped and saved, and the resulting area is used as a new mask area, specifically as follows: Figure 10 As shown. Among them, Figure 10 That is Figure 9 In the middle, (d) is the result of cropping the mask portion.

[0098] Step 7: Output the straightened character area in the positive direction: For the image stitched with positive rectangles, perform another rectangle circumvention operation and a fan-shaped character expansion operation to straighten the curved characters, thereby improving the efficiency and accuracy of character recognition. Specifically, as follows... Figure 11 As shown. Among them, Figure 11 and Figure 10 One-to-one correspondence.

[0099] Step 8: Input the straightened character region from the positive direction into the trained YOLO-V5 network to output an identifier arranged in the correct character order in the tire image. That is, arrange the characters in the correct order on the tire, such as "LOAD" as "LOAD".

[0100] The YOLO-V5 network training process specifically includes:

[0101] Step 8A: Straighten the character regions corresponding to multiple original images to create a character recognition dataset, which includes a test set, a training set, and a validation set.

[0102] The process of creating the character recognition dataset involves using labelimg to create labels for 1376 rectangular character region images obtained from 430 high-resolution original tire images through character localization, morphological processing, and cropping. The process is similar to step 2A.

[0103] The tags "DOT", "PRESSURE", and "LOAD" can be used to create tags for entire words, while numbers and other characters can be tagged separately. The tags are '2', '0', '5', ' / ', 'R', '1', '6', '9', 'V', 'DOT', 'G', 'P', 'E', 'K', 'H', 'MAX', 'LOAD', 'KG', '3', 'LBS', 'AT', 'KPA', 'PSI', 'PRESSURE', 'F', 'D', 'A', 'L', 'B', '4', '7', 'Z', 'Y', 'T', 'N', 'X', '8', 'W', 'C', 'COLD', 'U', and 'J'. The 48 categories, including 'INFLATION', 'I', 'O', 'M', 'S', and 'Q', are stored in XML format.

[0104] Step 8B: Based on the training set in Step 8A, train the YOLO-V5s network for character feature recognition, and then validate the YOLO-V5s network based on the validation set in Step 8A.

[0105] Specifically, the dataset is converted to a size of 640*640. It is preferable to use the YOLO-V5s network and set the ratio of the dataset to 8 (training set): 1 (validation set): 1 (test set). When training character features, it is preferable to set the hyperparameter to batch_size=16, the learning rate to 0.00001, and perform 150 rounds of training. The learning accuracy can reach 98.456%.

[0106] Step 8C: When the verification result reaches the set value, load the weights generated after training the YOLO-V5s network into the YOLO-V5s network to generate the trained YOLO-V5s network.

[0107] The preferred parameter settings are as follows: confidence threshold is 0.25, intersection-over-union (IoU) threshold is 0.45, resulting in sequentially arranged identifiers. The specific recognition effect is as follows: Figure 12 As shown.

[0108] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A dynamic identification method for tire imprint character based on a cooperative network, characterized in that, The method comprises the following steps: Step 1, based on the YOLO-V5 network and the original image containing the tire, a tire image is made; Step 2, based on the tire image, a data set for character positioning is made; Step 3, based on the ENet network model and the data set for character positioning, the weight of the ENet network model is obtained, and the weight is loaded into the ENet network model to generate a trained ENet network model; The ENet network model training process specifically comprises: Step 3A, based on the training set in step 2, the ENet network model is trained, and then based on the verification set in step 2, the ENet network model is verified; Step 3B, when the verification result reaches the set value, the weight generated after the ENet network model is trained is loaded into the ENet network model to generate a trained ENet network model; Step 4, based on the trained ENet network model, the character positioning of the tire image to be recognized is carried out, and the segmented character positioning map is output: Step 4A, the tire image to be recognized is cut along the vertical and horizontal center axis into 4 pictures; Step 4B, the 4 pictures are respectively input into the ENet network model trained in step 3 to realize the positioning of the characters of the tire image; Step 4C, a logical target filtering instruction is adopted, and for each picture, a character region with an area greater than 2000 pixels is selected; Step 4D, for the selected character region, a region growing algorithm and a cv2.minAreaRect function are used to output the character positioning map corresponding to each picture; Step 5, the 4 segmented character positioning maps and the 4 original tire images obtained in step 4A corresponding to the 4 character positioning maps are respectively subjected to character shape processing to output a complete character positioning map corresponding to each tire image in step 4A and a tire image; wherein the character positioning map comprises the top four character regions in terms of area; Step 6, based on the character region and the tire image generated in step 5A, an image rotation operation is performed to output a forward rectangular splicing image; Step 7, output the straightened character region in the forward direction: based on the forward rectangular splicing image, a fan-shaped character expansion operation is performed to make the curved characters vertical, so as to improve the efficiency and accuracy of character recognition; Step 8, input the straightened character region in the forward direction into the trained YOLO-V5 network to output identifiers arranged in the order of the tire image; The YOLO-V5 network training process specifically comprises: Step 8A, the straightened character regions corresponding to the plurality of original images are made into a character recognition data set, and the character recognition data set comprises a test set, a training set and a verification set; Step 8B, based on the training set in step 8A, the YOLO-V5s network is trained for character feature recognition, and then based on the verification set in step 8A, the YOLO-V5s network is verified; Step 8C, when the verification result reaches the set value, the weight generated after the YOLO-V5s network is trained is loaded into the YOLO-V5s network to generate a trained YOLO-V5s network.

2. The method of claim 1, wherein, Step 1 specifically comprises: Step 1 A, image shooting: using a high-pixel high-speed camera combined with a light supplementing facility to capture original images of the tire in motion; Step 1 B, tire positioning: using a YOLO-V5s network to identify and position the tire in the original images collected in step 1 A; Step 1 C, image cropping: cropping according to the detection frame to obtain the tire image.

3. The method of claim 1, wherein the method further comprises: In step 2, the character positioning dataset includes a test set, a training set, and a validation set.

Images