Tire mold character detection method and system fusing semantic segmentation and template matching

Abstract

The present application relates to a tire mold character detection method fusing semantic segmentation and template matching, based on the characteristics that tire mold characters have different sizes, types and reflective surfaces, a visual defect detection method and system fusing semantic segmentation and image matching are designed, which uses many corresponding algorithms for recognition, including image semantic segmentation, image matching, character recognition and detection, etc., and finally compares the string length and single character content of the recognized mold string with the CAD string, outputs the comparison result, realizes the detection of tire mold character defects by machine instead of manual work in the field of machine vision, on the one hand, solves the problems of high false detection rate, high cost and long time consumption of manual detection method; on the other hand, solves the problems of poor segmentation effect of characters on the image by traditional image processing method, and low recognition rate and low recognition efficiency in the subsequent character detection process.

Description

Technical Field

[0001] The present invention relates to the technical field related to performance defect detection, and in particular to a tire mold character detection method and system integrating semantic segmentation and template matching. Background Art

[0002] Tire mold markings refer to the characters used in the molds used to manufacture tires. These characters are generally used to mark information such as the tire's specifications, brand, and production date. Common tire mold markings include numbers, letters, and symbols, and are engraved on the tire's sidewalls or treads.

[0003] The traditional method of tire mold character detection is, on the one hand, through manual detection, which has problems such as high false detection rate, high cost, and long time consumption; on the other hand, it is detected through traditional image processing methods. Due to the characteristics of tire mold character images, the difference between them and the characters on the CAD image is too large. The effect of using traditional image processing methods to segment the characters on the image is very poor, and the subsequent character detection process has problems such as low recognition rate and low recognition efficiency. Summary of the Invention

[0004] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a tire mold character detection method and system that integrates semantic segmentation and template matching.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] Specifically, a tire mold character detection method integrating semantic segmentation and template matching is proposed, including the following:

[0007] Step 110: Obtain a mold image and its corresponding CAD image;

[0008] Step 120: Preprocess the mold image and the CAD image to obtain a mold flat image and a CAD flat image respectively;

[0009] Step 130: Process the mold flat image using a trained improved U-Net neural network to output a binary image after semantic segmentation;

[0010] Step 140: Pre-process the segmented binary image before matching to obtain a large character image and a small character image, and count the number of large characters in the large character image as Num1, and count the number of small characters in the small character image as Num2;

[0011] Step 150: When both Num1 and Num2 are 0, it is directly determined that there is no character defect in the mold image; when Num1+Num2 is less than 2, no image matching is performed; when Num1+Num2 is not less than 2, if Num2 ≤ 5, the binary image is used to perform image matching with the CAD flat image; if Num2 > 5, the small character image is used to perform image matching with the CAD flat image;

[0012] Step 160: Based on the image matching result of step 150, a character segmentation operation is performed on the mold flat image to obtain a first segmentation result, and a character segmentation operation is performed on the CAD flat image to obtain a second segmentation result;

[0013] Step 170: Using a trained character library model, the first segmentation result is recognized to obtain a mold character string, and the second segmentation result is recognized to obtain a CAD character string.

[0014] Step 180 : Compare the mold character string with the CAD character string by comparing the character string length and the single character content to determine character defects, and output the comparison result.

[0015] Further, specifically, the pre-processing operation in step 120 includes:

[0016] The mold image or CAD image is subjected to grayscale processing, image filtering processing, peripheral contour extraction and polar coordinate transformation in sequence to obtain a corresponding flat image.

[0017] Further, specific, improved U-Net neural network includes,

[0018] Based on the traditional U-net neural network, in its decoding part, the number of channels in each layer is reduced to half of the original number of channels, the upsampling convolution layer is removed, and the direct upsampling is selected and spliced ​​with the encoding part; all valid convolutions are replaced with the same convolution, so that subsequent cropping is not required to achieve equal-size splicing.

[0019] Furthermore, specifically, the improved U-Net neural network is trained in the following way:

[0020] Use the labelme control to label all the mold flat images used as samples as label images;

[0021] The flat mold images and their corresponding label images were deformed, contrasted, and adjusted for brightness to augment the dataset. All the enhanced flat mold images and label images were used as the training data set. The network model was trained in PyCharm. The training function used the binary cross entropy loss function and the RMSprop optimizer algorithm. The initial learning rate (lr) was set to 1e-3, and ReduceLROnPlateau was used to adjust the learning rate. An early stopping function was set, and training was stopped when the average training loss for 10 consecutive rounds was less than 2e-5.

[0022] After the model training is completed, the trained improved U-Net neural network is obtained.

[0023] Furthermore, specifically, the pre-matching pre-processing in step 140 includes:

[0024] Based on the different pixel height ranges of large and small characters, the binary image is divided into a large character image and a small character image with 180 pixels as the dividing line.

[0025] Furthermore, specifically, the image matching method in step 150 includes:

[0026] Using a sliding window matching strategy, a shape-based template matching algorithm is used to perform a first matching on all the segmented binary images and the CAD flat image to obtain a first matching result;

[0027] Performing a second match on the first template matching result so that each row of CAD character text finds the corresponding mold character text, achieving a one-to-one match between the corresponding positions of each binary image and the CAD flat image. The second match involves cropping the matched CAD character area, extracting the region of interest, and completing the cropping of broken characters.

[0028] The sliding window matching strategy is to limit the sliding window to move one step to the right after each successful match. At the same time, to prevent the matching area from crossing the boundary, a judgment is made every time the sliding window moves one step to the right. When the vertical coordinate of the right boundary of the window is greater than the vertical coordinate of the right boundary of the CAD flat image, the sliding window moves to the head position of the CAD image and then continues to move to the right to complete the match.

[0029] Furthermore, the method also includes, after obtaining the first segmentation result and the second segmentation result, normalizing the characters in the first segmentation result and the second segmentation result, so as to adjust characters of different sizes to a uniform size.

[0030] Furthermore, specifically, the trained character library model is trained in the following way:

[0031] Build an MLP classifier;

[0032] A character library is established, wherein the character library includes 26 uppercase English letters, 26 lowercase English letters, 10 numbers from 0 to 9, and 9 symbols, and the character library is trained by the MLP classifier to obtain a trained character library model.

[0033] Furthermore, specifically, the mold character string and the CAD character string are compared in terms of string length and single character content to determine character defects, and the comparison results are output, including:

[0034] Count the length of the recognized CAD string, recorded as len1, and count the length of the recognized mold string, recorded as len2;

[0035] If len1=len2, the corresponding CAD character string is compared with the mold character string character by character. The mold character that is different from the CAD character is the misprinted character.

[0036] If len1 <len2,以模具字符串作为被搜索对象,从CAD字符串的第一个字符开始,对模具字符串开始搜索,每次的搜索步长为1,当搜索到相同字符时,则开始下一个CAD字符的搜索,下一个字符的搜索从上一个字符搜索得到相同模具字符的后一个模具字符开始,如果上一个CAD字符未搜索到相同字符,则下一个CAD字符应从上一个CAD字符的最后搜索模具字符的后一个模具字符开始,每个CAD字符的搜索长度=模具字符串未被搜索的字符长度-CAD字符串未开始搜索的字符长度+1,在搜索长度范围内若该CAD字符没有搜索到对应模具字符,则判定该搜索范围内的第一个模具字符为错印字符,待所有CAD字符都搜索完毕后,剩下未被搜索的模具字符则为多印字符;

[0037] If len1>len2, the CAD string is taken as the search object, and the CAD string is searched starting from the first mold character of the mold string. When the corresponding CAD character is not found in the search range, the character is a misprinted character. After all characters are searched, the CAD characters that have not been searched are missed characters.

[0038] The present invention also proposes a tire mold character detection system that integrates semantic segmentation and template matching, including the following:

[0039] Data acquisition module, used to acquire mold images and their corresponding CAD images;

[0040] A preprocessing module, configured to preprocess the mold image and the CAD image to obtain a mold flat image and a CAD flat image respectively;

[0041] A semantic segmentation module, configured to process the mold flat image using a trained improved U-Net neural network to output a binary image after semantic segmentation;

[0042] a pre-matching pre-processing module for performing pre-matching pre-processing on the segmented binary image to obtain a large character image and a small character image, and counting the number of large characters in the large character image as Num1, and counting the number of small characters in the small character image as Num2;

[0043] The image matching module is used to make the following judgments:

[0044] When Num1 and Num2 are both 0, it is directly judged that there is no character defect in the mold image. When Num1+Num2 is less than 2, no image matching is performed. When Num1+Num2 is not less than 2, if Num2≤5, the binary image is used to perform image matching with the CAD flat image. If Num2>5, the small character image is used to perform image matching with the CAD flat image.

[0045] a character segmentation module configured to perform a character segmentation operation on the mold flat image based on the image matching result of the image matching module to obtain a first segmentation result, and to perform a character segmentation operation on the CAD flat image to obtain a second segmentation result;

[0046] A character recognition module, configured to use a trained character library model to respectively recognize the first segmentation result to obtain a mold character string and recognize the second segmentation result to obtain a CAD character string;

[0047] The result output module is used to compare the mold character string with the CAD character string by comparing the string length and the content of a single character to determine character defects and output the comparison results.

[0048] The beneficial effects of the present invention are:

[0049] The present invention proposes a tire mold character detection method that integrates semantic segmentation and template matching. Based on the characteristics that tire mold characters have different sizes and types and the surface is reflective, a visual defect detection method and system that integrates semantic segmentation and image matching are designed. This method utilizes many corresponding algorithms, including image semantic segmentation, image matching, character recognition and detection, etc., and realizes the use of machines to replace manual labor to complete the detection of tire mold character defects in the field of machine vision. On the one hand, it solves the problems of high false detection rate, high cost, and long time consumption existing in manual detection methods; on the other hand, it solves the problems that traditional image processing methods have poor effect in segmenting characters on their images, and the subsequent character detection process has low recognition rate and low recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and other features of the present disclosure will become more apparent through a detailed description of the embodiments shown in conjunction with the accompanying drawings. The same reference numerals in the drawings of the present disclosure represent the same or similar elements. Obviously, the drawings described below are only some embodiments of the present disclosure. It is possible for a person skilled in the art to derive other drawings based on these drawings without inventive effort. In the drawings:

[0051] Figure 1 Shown is a flow chart of a tire mold character detection method integrating semantic segmentation and template matching according to the present invention;

[0052] Figure 2 FIG2 is an algorithm principle diagram of the tire mold character detection method integrating semantic segmentation and template matching according to the present invention;

[0053] Figure 3 Shown is an original mold image in one embodiment;

[0054] Figure 4 Shown is a mold flat image corresponding to the original mold image in one embodiment;

[0055] Figure 5 Shown is an original CAD image corresponding to the original mold image in one embodiment;

[0056] Figure 6 Shown is a CAD flattened image corresponding to the original CAD image in one embodiment;

[0057] Figure 7 FIG2 is a schematic diagram showing the structural principle of the depthwise separable convolution used in the tire mold character detection method integrating semantic segmentation and template matching of the present invention;

[0058] Figure 8FIG2 is a schematic diagram showing the structural principle of the improved U-Net neural network in the tire mold character detection method integrating semantic segmentation and template matching of the present invention;

[0059] Figure 9 Shown is a flat image of a mold and its corresponding label image in one embodiment;

[0060] Figure 10 Shown are diagrams of a mold flat image and semantic segmentation effects of different models in one embodiment;

[0061] Figure 11 Shown is a schematic diagram of a situation where an error occurs during matching in one embodiment;

[0062] Figure 12 FIG2 is a schematic diagram showing the principle of the sliding window matching strategy used in the tire mold character detection method integrating semantic segmentation and template matching of the present invention;

[0063] Figure 13 The figure shows the effect after the first matching in one embodiment;

[0064] Figure 14 Shown is a CAD flat image and its corresponding cropped rendering in one embodiment;

[0065] Figure 15 The figure shows an original image of a character string and its segmentation effect in one embodiment;

[0066] Figure 16 FIG2 is a flowchart of character defect determination when len1<len2 in one embodiment;

[0067] Figure 17 Shown is a mold image, a CAD image, and a proofreading result diagram after applying the tire mold character detection method integrating semantic segmentation and template matching of the present invention in one embodiment. DETAILED DESCRIPTION

[0068] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other unless there is a conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0069] Reference Figure 1 as well as Figure 2 In Example 1, the present invention proposes a tire mold character detection method that integrates semantic segmentation and template matching, including the following:

[0070] Step 110: Obtain a mold image and its corresponding CAD image;

[0071] Step 120: Preprocess the mold image and the CAD image to obtain a mold flat image and a CAD flat image respectively;

[0072] Step 130: Process the mold flat image using a trained improved U-Net neural network to output a binary image after semantic segmentation;

[0073] Step 140: Pre-process the segmented binary image before matching to obtain a large character image and a small character image, and count the number of large characters in the large character image as Num1, and count the number of small characters in the small character image as Num2;

[0074] Step 150: When both Num1 and Num2 are 0, it is directly determined that there is no character defect in the mold image; when Num1+Num2 is less than 2, no image matching is performed; when Num1+Num2 is not less than 2, if Num2 ≤ 5, the binary image is used to perform image matching with the CAD flat image; if Num2 > 5, the small character image is used to perform image matching with the CAD flat image;

[0075] Step 160: Based on the image matching result of step 150, a character segmentation operation is performed on the mold flat image to obtain a first segmentation result, and a character segmentation operation is performed on the CAD flat image to obtain a second segmentation result;

[0076] Step 170: Using a trained character library model, the first segmentation result is recognized to obtain a mold character string, and the second segmentation result is recognized to obtain a CAD character string.

[0077] Step 180 : Compare the mold character string with the CAD character string by comparing the character string length and the single character content to determine character defects, and output the comparison result.

[0078] In this embodiment 1, based on the characteristics of tire mold characters of different sizes and types and reflective surface, a visual defect detection method and system that integrates semantic segmentation and image matching are designed. This method utilizes many corresponding algorithms, including image semantic segmentation, image matching, character recognition and detection, etc., to achieve the purpose of replacing manual labor with machines to complete the detection of tire mold character defects in the field of machine vision. On the one hand, it solves the problems of high false detection rate, high cost, and long time consumption existing in manual detection methods; on the other hand, it solves the problems of poor effect of traditional image processing methods in segmenting characters on their images, as well as low recognition rate and low recognition efficiency in the subsequent character detection process.

[0079] As a preferred embodiment of the present invention, specifically, the pre-processing operation in step 120 includes:

[0080] The mold image or CAD image is subjected to grayscale processing, image filtering processing, peripheral contour extraction and polar coordinate transformation in sequence to obtain a corresponding flat image.

[0081] In this preferred embodiment, 1. the original three-dimensional image data of the mold is reduced to one-dimensional data through grayscale processing, thereby reducing the image data size and shortening the time of subsequent image processing; 2. the impact of noise on the image is reduced through image filtering algorithm; 3. the mold annular image and the CAD annular image to be matched are converted into flat images by extracting the outer circle contour and polar coordinate transformation, so that the characters in the image are arranged horizontally.

[0082] As a preferred embodiment of the present invention, specifically, the improved U-Net neural network includes:

[0083] Based on the traditional U-net neural network, in its decoding part, the number of channels in each layer is reduced to half of the original number of channels, the upsampling convolution layer is removed, and the direct upsampling is selected and spliced ​​with the encoding part; all valid convolutions are replaced with the same convolution, so that subsequent cropping is not required to achieve equal-size splicing.

[0084] In this preferred embodiment, in order to make the neural network more portable and better meet the requirements of high efficiency, the present invention improves the traditional U-Net neural network and adopts depthwise separable convolution (as shown below): Figure 7 As shown in the figure, the traditional convolution structure after upsampling in the traditional U-Net neural network is replaced by the improved U-Net neural network. This combination greatly reduces the number of parameters and calculations, and greatly improves the operation speed of the model. At the same time, compared with other neural network segmentation methods, the improved U-Net neural network segmentation method has a good segmentation effect (as shown in the figure). Figure 10 As shown in the figure), it has high efficiency and strong generalization, and can make the segmented binary image convenient for subsequent image matching and character recognition and proofreading processes, thereby greatly reducing the difficulty of character recognition and character proofreading.

[0085] As a preferred embodiment of the present invention, specifically, the improved U-Net neural network is trained in the following manner:

[0086] Use the labelme control to label all the mold flat images used as samples as label images;

[0087] The flat mold images and their corresponding label images were deformed, contrasted, and adjusted for brightness to augment the dataset. All the enhanced flat mold images and label images were used as the training data set. The network model was trained in PyCharm. The training function used the binary cross entropy loss function and the RMSprop optimizer algorithm. The initial learning rate (lr) was set to 1e-3, and ReduceLROnPlateau was used to adjust the learning rate. An early stopping function was set, and training was stopped when the average training loss for 10 consecutive rounds was less than 2e-5.

[0088] After the model training is completed, the trained improved U-Net neural network is obtained.

[0089] As a preferred embodiment of the present invention, specifically, the pre-matching preprocessing in step 140 includes dividing the binary image into a large character image and a small character image based on the different pixel height ranges of large and small characters and taking 180 pixels as the dividing line.

[0090] As a preferred embodiment of the present invention, specifically, the image matching method in step 150 includes:

[0091] Using a sliding window matching strategy, a shape-based template matching algorithm is used to perform a first matching on all the segmented binary images and the CAD flat image to obtain a first matching result;

[0092] Performing a second match on the first template matching result so that each row of CAD character text finds the corresponding mold character text, achieving a one-to-one match between the corresponding positions of each binary image and the CAD flat image. The second match involves cropping the matched CAD character area, extracting the region of interest, and completing the cropping of broken characters.

[0093] The sliding window matching strategy is to limit the sliding window to move one step to the right after each successful match. At the same time, to prevent the matching area from crossing the boundary, a judgment is made every time the sliding window moves one step to the right. When the vertical coordinate of the right boundary of the window is greater than the vertical coordinate of the right boundary of the CAD flat image, the sliding window moves to the head position of the CAD image and then continues to move to the right to complete the match.

[0094] In this preferred embodiment, a shape-based template matching algorithm is used to perform a global search within the mold image to find the area most similar to the CAD image. This algorithm accurately matches and locates the semantically segmented mold binary image with the flat CAD image. After the first matching step, a secondary matching algorithm is used to crop the CAD area located by the first matching step. The text is then segmented by line, and each line of CAD characters is then matched again against the mold image, ensuring that each line of CAD characters accurately matches the corresponding mold character.

[0095] As a preferred embodiment of the present invention, the method further includes, after obtaining the first segmentation result and the second segmentation result, normalizing the characters in the first segmentation result and the second segmentation result, adjusting characters of different sizes to a uniform size.

[0096] As a preferred embodiment of the present invention, specifically, the trained character library model is trained in the following manner:

[0097] Build an MLP classifier;

[0098] A character library is established, wherein the character library includes 26 uppercase English letters, 26 lowercase English letters, 10 numbers from 0 to 9, and 9 symbols, and the character library is trained by the MLP classifier to obtain a trained character library model.

[0099] As a preferred embodiment of the present invention, specifically, the mold character string and the CAD character string are compared in terms of character length and single character content to determine character defects, and the comparison results are output, including:

[0100] Count the length of the recognized CAD string, recorded as len1, and count the length of the recognized mold string, recorded as len2;

[0101] If len1=len2, the corresponding CAD character string is compared with the mold character string character by character. The mold character that is different from the CAD character is the misprinted character.

[0102] If len1 <len2,以模具字符串作为被搜索对象,从CAD字符串的第一个字符开始,对模具字符串开始搜索,每次的搜索步长为1,当搜索到相同字符时,则开始下一个CAD字符的搜索,下一个字符的搜索从上一个字符搜索得到相同模具字符的后一个模具字符开始,如果上一个CAD字符未搜索到相同字符,则下一个CAD字符应从上一个CAD字符的最后搜索模具字符的后一个模具字符开始,每个CAD字符的搜索长度=模具字符串未被搜索的字符长度-CAD字符串未开始搜索的字符长度+1,在搜索长度范围内若该CAD字符没有搜索到对应模具字符,则判定该搜索范围内的第一个模具字符为错印字符,待所有CAD字符都搜索完毕后,剩下未被搜索的模具字符则为多印字符;

[0103] If len1>len2, the CAD string is taken as the search object, and the CAD string is searched starting from the first mold character of the mold string. When the corresponding CAD character is not found in the search range, the character is a misprinted character. After all characters are searched, the CAD characters that have not been searched are missed characters.

[0104] In this preferred embodiment, the algorithm idea adopted is to compare the lengths of the CAD character string and the mold character string, and there are three different situations: 1. If the lengths are equal, they are compared character by character, and the mold characters that are different from the CAD characters are misprinted characters; 2. If the mold character string is long, the mold character string is used as the search object, and the CAD characters are searched from the mold characters, and a character-by-character proofreading is performed to find out whether the mold character string has misprinted or overprinted characters; 3. If the CAD character string is long, the CAD character string is used as the search object, and the mold characters are searched from the CAD characters, and a character-by-character proofreading is performed to find out whether there are misprinted or missing characters in the mold character string.

[0105] The following is a complete implementation process integrating all preferred implementation methods:

[0106] 1) The system starts running, according to the system process Figure 2 As shown, first read the original image of the tire mold (such as Figure 3 as shown) and CAD images (as Figure 5 shown).

[0107] 2) According to the specific situation of the image, the inner and outer diameter ratio in the polar coordinate transformation algorithm can be modified (the default setting is 0.75) so that the converted flat image can fully display the effective area of ​​the character.

[0108] 3) After setting the inner-outer diameter ratio parameter, run the image preprocessing module algorithm code to complete the image preprocessing operation of the mold image to be corrected, and output the mold flat image as shown in Figure 3 . Figure 4 . Figure 5 . Figure 6 . .

[0109] 4) Use the pytorch machine learning framework to build an improved U-Net neural network (as shown in Figure 8 ): Improve the traditional U-Net neural network, reduce the number of channels of each layer to half of the original number of channels, remove the convolution layer after upsampling, and select direct upsampling and splicing with the encoding part; Replace all valid convolutions with same convolutions so that subsequent equal size splicing does not require cropping.

[0110] 5) Use the labelme control to label all mold flat images as label images (as shown in Figure 9 ).

[0111] 6) Perform data augmentation on flat images and corresponding label images through deformation adjustment, contrast and brightness adjustment to expand the data set. All enhanced flat images and label images are used as data training sets to train the neural network model in pycharm. The binary cross-entropy loss function is used in the training function, the RMSprop optimizer algorithm is used, the initial learning rate (lr) is set to 1e-3, and ReduceLROnPlateau is used to adjust the learning rate. Set the early stop function to stop training when the average training loss changes by less than 2e-5 for 10 consecutive rounds.

[0112] 7) After model training, run the segmentation code to import the mold image to be segmented as a test set into the model for prediction and export the binary image after semantic segmentation (as shown in Figure 10 ).

[0113] 8) Preprocessing purpose before matching: If the entire character image is used for matching directly, it will cause a certain displacement of small characters in the mold image and small characters in the CAD image in the vertical direction (as shown in Figure 11 ).

[0114] 9) Determine whether there are characters in the binary image. When (Num1+Num2)<2, no image matching is performed; otherwise, another judgment is performed. If Num2<5, the entire character image is used for matching; when Num2>5, only the small character image is used for matching.

[0115] 10) Sliding window matching strategy: After each successful match, the matching area of ​​the next image is limited. Figure 12 As shown in the figure, the red box indicates the position of the current image after a successful match, and the blue box indicates the sliding window, which is the matching area of ​​the next image. After each match, the sliding window moves 0.5H to the right. At the same time, to prevent the matching area from crossing the boundary, a judgment is made every time the sliding window moves one step to the right. When the vertical coordinate of the right boundary of the window is greater than the vertical coordinate of the right boundary of the CAD image, the sliding window moves to the head position of the CAD image and then continues to move to the right to complete the match.

[0116] 11) According to the above sliding window matching strategy, run the shape-based template matching algorithm to perform the first template matching on all the mold binary images and CAD flat images of each mold. The first matching effect is as follows: Figure 13 shown.

[0117] 12) Perform a secondary match based on the first match, run the setting code of the secondary match rule, crop the matched CAD character area, extract the area of ​​interest, and complete the cropping of the broken characters (such as Figure 14 As shown), each line of CAD character text can find the corresponding mold character text, and each mold drawing of each mold can be matched with the corresponding position of the CAD drawing.

[0118] 13) Run the character segmentation algorithm to perform character segmentation operations on each character in each line of the mold image and the CAD area that matches it (such as Figure 15 shown).

[0119] 14) Perform normalization on the characters segmented from the mold flat drawing and CAD drawing, and adjust characters of different sizes to a uniform size.

[0120] 15) Build an MLP (Multi-layer Perceptron) classifier.

[0121] 16) A character library is established based on the characters segmented from the mold binary image and the CAD image (including 26 uppercase English letters, 26 lowercase English letters, 10 numbers, and 9 symbols, where the 9 symbols refer to "Ψ", "&", "(", ")", "*", " / ", ":", "^", and "+") that may be used in tire molds, and the model is trained using the constructed classifier.

[0122] 17) Run the character recognition code to use the trained character library model to recognize each segmented character, and complete the recognition of each line of character strings in the mold flat map and its matching CAD area one by one.

[0123] 18) The recognized CAD character string and the mold character string are collated. First, the length of the recognized CAD character string is counted as len1; the length of the mold character string is counted as len2. The comparison of the character lengths of the two can be divided into three cases: (1) len1 = len2; (2) len1<len2;(3)len1> len2.

[0124] 19) Principle of proofreading algorithm: The above three situations are processed with different proofreading algorithms. If len1=len2, the corresponding CAD character string is compared with the mold character string character by character. The mold character that is different from the CAD character is the misprinted character.

[0125] 20) If len1 <len2,如 Figure 16 As shown, the mold string is used as the search object, starting from the first character of the CAD string, and the mold string is searched. The search step is 1 each time. When the same character is found, the search for the next CAD character begins. The search for the next character should start from the character after the previous character search that found the same mold character. If the previous CAD character does not find the same character, the next character should start from the character after the last searched character of the previous character. It should be noted that the search length of each CAD character = the length of the unsearched characters in the mold string - the length of the characters in the CAD string that have not been searched + 1. If the CAD character does not find the corresponding mold character within the search length range, the first mold character in the search range is determined to be a misprinted character. After all CAD characters have been searched, the remaining unsearched mold characters are considered to be overprinted characters.

[0126] 21) If len1>len2, the CAD string is used as the search object, and the search for the CAD string begins from the first character of the mold string. When the corresponding CAD character of the mold character is not found within the search range, the character is considered a misprinted character. After all characters are searched, the unsearched CAD character is considered a missed character.

[0127] 22) Run the character proofreading algorithm program to distinguish misprints, omissions and overprints with different colored characters. The character defect detection effect is as follows: Figure 17 At this point, the system operation is completed.

[0128] The present invention also proposes a tire mold character detection system that integrates semantic segmentation and template matching, including the following:

[0129] a data acquisition module configured to acquire a mold image and a CAD image corresponding to the mold image;

[0130] a preprocessing module configured to respectively preprocess the mold image and the CAD image to obtain a mold flat image and a CAD flat image;

[0131] a semantic segmentation module configured to process the mold flat image by using a trained improved U-Net neural network to output a binary image after semantic segmentation;

[0132] a pre-matching preprocessing module configured to perform pre-matching preprocessing on the segmented binary image to obtain a large character image and a small character image, count the number of large characters in the large character image as Num1, and count the number of small characters in the small character image as Num2;

[0133] an image matching module configured to perform the following judgment,

[0134] when Num1 and Num2 are both 0, it is directly judged that the mold image does not have a character defect, when Num1+Num2 is less than 2, no image matching is performed, when Num1+Num2 is not less than 2, if Num2≤5, the binary image and the CAD flat image are used for image matching, and if Num2>5, the small character image and the CAD flat image are used for image matching;

[0135] a character segmentation module configured to perform a character segmentation operation on the mold flat image based on the image matching result of the image matching module to obtain a first segmentation result, and perform a character segmentation operation on the CAD flat image to obtain a second segmentation result;

[0136] a character recognition module configured to recognize the first segmentation result by using a trained character library model to obtain a mold character string, and recognize the second segmentation result to obtain a CAD character string;

[0137] a result output module configured to compare the mold character string and the CAD character string in terms of string length and single character content to determine a character defect, and output a comparison result.

[0138] The modules described as separate components can or can not be physically separate, and the components displayed as modules can or can not be physical modules, i.e., they can be located in one place or distributed on multiple network modules. According to actual needs, some or all of the modules can be selected to achieve the purpose of the scheme in the embodiment.

[0139] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0140] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0141] Although the present invention has been described in considerable detail and with particularity with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be construed as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively encompassing the intended scope of the invention. In addition, the invention has been described above in terms of embodiments foreseen by the inventors for the purpose of providing a useful description, and those insubstantial modifications of the invention that are not currently foreseen may still represent equivalent modifications of the invention.

[0142] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. As long as the technical effects of the present invention are achieved by the same means, they shall fall within the scope of protection of the present invention. Within the scope of protection of the present invention, various modifications and variations of the technical solutions and / or implementation methods may be made.

Claims

1. A tire mold character detection method integrating semantic segmentation and template matching, characterized in that: These include: Step 110: Obtain a mold image and its corresponding CAD image; Step 120: Preprocess the mold image and the CAD image to obtain a mold flat image and a CAD flat image respectively; Step 130: Process the mold flat image using a trained improved U-Net neural network to output a binary image after semantic segmentation; Step 140: Pre-process the segmented binary image before matching to obtain a large character image and a small character image, and count the number of large characters in the large character image as Num1, and count the number of small characters in the small character image as Num2; Step 150: When both Num1 and Num2 are 0, it is directly determined that there is no character defect in the mold image; when Num1+Num2 is less than 2, no image matching is performed; when Num1+Num2 is not less than 2, if Num2 ≤ 5, the binary image is used to perform image matching with the CAD flat image; if Num2 > 5, the small character image is used to perform image matching with the CAD flat image; Step 160: Based on the image matching result of step 150, a character segmentation operation is performed on the mold flat image to obtain a first segmentation result, and a character segmentation operation is performed on the CAD flat image to obtain a second segmentation result; Step 170: Using a trained character library model, the first segmentation result is recognized to obtain a mold character string, and the second segmentation result is recognized to obtain a CAD character string. Step 180 : Compare the mold character string with the CAD character string by comparing the character string length and the single character content to determine character defects, and output the comparison result.

2. The tire mold character detection method integrating semantic segmentation and template matching according to claim 1, characterized in that: Specifically, the pre-processing operation in step 120 includes: The mold image or CAD image is subjected to grayscale processing, image filtering processing, peripheral contour extraction and polar coordinate transformation in sequence to obtain a corresponding flat image.

3. The tire mold character detection method integrating semantic segmentation and template matching according to claim 1, characterized in that: Specifically, the improved U-Net neural network includes: Based on the traditional U-net neural network, in its decoding part, the number of channels in each layer is reduced to half of the original number of channels, the upsampling convolution layer is removed, and the direct upsampling is selected and spliced ​​with the encoding part; all valid convolutions are replaced with the same convolution, so that subsequent cropping is not required to achieve equal-size splicing.

4. The tire mold character detection method integrating semantic segmentation and template matching according to claim 3 is characterized in that: Specifically, the improved U-Net neural network is trained in the following way: Use the labelme control to label all the mold flat images used as samples as label images; The flat mold images and their corresponding label images were deformed, contrasted, and adjusted for brightness to augment the dataset. All the enhanced flat mold images and label images were used as the training data set. The network model was trained in PyCharm. The training function used the binary cross entropy loss function and the RMSprop optimizer algorithm. The initial learning rate (lr) was set to 1e-3, and ReduceLROnPlateau was used to adjust the learning rate. An early stopping function was set, and training was stopped when the average training loss for 10 consecutive rounds was less than 2e-5. After the model training is completed, the trained improved U-Net neural network is obtained.

5. The tire mold character detection method integrating semantic segmentation and template matching according to claim 1, characterized in that: Specifically, the pre-matching preprocessing in step 140 includes: Based on the different pixel height ranges of large and small characters, the binary image is divided into a large character image and a small character image with 180 pixels as the dividing line.

6. The tire mold character detection method integrating semantic segmentation and template matching according to claim 1, characterized in that: Specifically, the image matching method in step 150 is: include, Using a sliding window matching strategy, a shape-based template matching algorithm is used to perform a first matching on all the segmented binary images and the CAD flat image to obtain a first matching result; Performing a second match on the first matching result so that each line of CAD character text finds the corresponding mold character text, achieving a one-to-one match between the corresponding positions of each binary image and the CAD flat image. The second match involves cropping the matched CAD character area, extracting the region of interest, and completing the cropping of broken characters. The sliding window matching strategy is to limit the sliding window to move one step to the right after each successful match. At the same time, to prevent the matching area from crossing the boundary, a judgment is made every time the sliding window moves one step to the right. When the vertical coordinate of the right boundary of the window is greater than the vertical coordinate of the right boundary of the CAD flat image, the sliding window moves to the head position of the CAD image and then continues to move to the right to complete the match.

7. The tire mold character detection method integrating semantic segmentation and template matching according to claim 1, characterized in that: The method further includes, after obtaining the first segmentation result and the second segmentation result, performing normalization processing on the characters in the first segmentation result and the second segmentation result, so as to adjust characters of different sizes to a uniform size.

8. The tire mold character detection method integrating semantic segmentation and template matching according to claim 1, characterized in that: Specifically, the trained character library model is trained in the following way: Build an MLP classifier; A character library is established, wherein the character library includes 26 uppercase English letters, 26 lowercase English letters, 10 numbers from 0 to 9, and 9 symbols, and the character library is trained by the MLP classifier to obtain a trained character library model.

9. The tire mold character detection method integrating semantic segmentation and template matching according to claim 1, characterized in that: Specifically, the mold character string and the CAD character string are compared in terms of string length and single character content to determine character defects, and the comparison results are output, including: Count the length of the recognized CAD string, recorded as len1, and count the length of the recognized mold string, recorded as len2; If len1=len2, the corresponding CAD character string is compared with the mold character string character by character. The mold character that is different from the CAD character is the misprinted character. If len1 < len2, take the die string as the object to be searched. Starting from the first character of the CAD string, search for the die string. The search step size is 1 each time. When the same character is found, start searching for the next CAD character. The search for the next character starts from the next die character after the die character that is the same as the previous CAD character. If no same character is found for the previous CAD character, the search for the next CAD character should start from the next die character after the last searched die character of the previous CAD character. The search length for each CAD character = the length of the un-searched characters in the die string - the length of the un-started searched characters in the CAD string + 1. If no corresponding die character is found for this CAD character within the search length range, determine the first die character within this search length range as the misprinted character. After all CAD characters have been searched, the remaining un-searched die characters are the overprinted characters; If len1 > len2, take the CAD string as the object to be searched. Starting from the first die character of the die string, search for the CAD string. When no corresponding CAD character is found for the die character within the search length range, this character is the misprinted character. After all characters have been searched, the un-searched CAD characters are the missing printed characters.

10. A tire mold character detection system integrating semantic segmentation and template matching, characterized by: The following are included: A data acquisition module for acquiring a die image and its corresponding CAD image; A preprocessing module for preprocessing the die image and the CAD image respectively to obtain a die flat image and a CAD flat image; A semantic segmentation module for processing the die flat image through a trained improved U-Net neural network to output a binary image after semantic segmentation; A preprocessing module before matching for preprocessing the segmented binary image before matching to obtain a large character image and a small character image, and counting the number of large characters in the large character image as Num1, and counting the number of small characters in the small character image as Num2; An image matching module for making the following judgments, When both Num1 and Num2 are 0, directly determine that there are no character defects in the die image. When Num1 + Num2 is less than 2, no image matching is performed. When Num1 + Num2 is not less than 2, if Num2 ≤ 5, perform image matching using the binary image and the CAD flat image. If Num2 > 5, perform image matching using the small character image and the CAD flat image; A character segmentation module for performing character segmentation operations on the die flat image based on the image matching result of the image matching module to obtain a first segmentation result, and performing character segmentation operations on the CAD flat image to obtain a second segmentation result; A character recognition module for respectively recognizing the first segmentation result through a trained character library model to obtain a die string, and recognizing the second segmentation result to obtain a CAD string; A result output module for comparing the die string and the CAD string in terms of string length and single character content to determine character defects and output the comparison result.

Images