A method and system for detecting rubber tube defects based on deep learning

Abstract

The present invention provides a method and system for detecting rubber tube defects based on deep learning, including: calculating an error based on a binary Gaussian heat map and image features output by a model; updating the model based on the error; calculating a correlation coefficient between the verified image features and their corresponding heat maps using the updated model; repeating the above process until the maximum number of loops is reached; obtaining an optimal model according to the correlation coefficient; using the optimal model to detect an image of a rubber tube to be measured and outputting image features of the rubber tube to be measured; performing comparative analysis on the image features of the rubber tube to be measured to obtain the defect position, type, and confidence level, and visually outputting the defective part. The method and system for detecting rubber tube defects based on deep learning in the embodiments of the present invention realize the automatic detection of rubber tube surface defects, have high accuracy, and greatly improve the quality inspection efficiency of vehicle rubber tubes.

Description

Technical Field

[0001] The present invention relates to the field of digital images, and in particular, to a method and system for detecting rubber tube defects based on deep learning. Background Art

[0002] As one of the important materials in vehicle engineering, rubber tubes play a key role in the connection of various important components. Therefore, it is particularly important to ensure the quality compliance of vehicle rubber tubes. As an industrial product, during the production process, due to process reasons, process reasons, improper operation of workers and other problems, it is inevitable that surface defects of rubber tubes will appear. Finding these defects, removing the corresponding products, and preventing them from entering the vehicle manufacturing stage are one of the important tasks in the quality inspection process of vehicle rubber tubes.

[0003] The detection of rubber tube surface defects has the following difficulties: the sizes are different, relative to the size of the rubber tube itself, some defects are clear and large, which are obvious, while some are very small. Even for the same type of defect, there are also large differences in size; there is also a lot of water stains, dust, etc. on the surface of the rubber tube. These stains have similar characteristics to some defects, but cannot be detected as defects; between different types of defects, there are also similar and indistinguishable characteristics, which requires the detection method to have strong classification capabilities.

[0004] The currently used solutions can basically be summarized into three directions. One is manual inspection. This method has high detection accuracy, but high detection costs, low work efficiency, certain harm to the human body, and high employee training costs; the second is an algorithm based on traditional digital image processing technology. Compared with manual inspection, although the efficiency is improved and the detection cost is reduced, the accuracy is much lower than that of manual detection. It can only locate some defects and cannot determine the category, and manual recheck is still required later; the third is an algorithm based on deep learning networks. Compared with traditional algorithms, the accuracy is greatly improved, and the defect category can be determined. Compared with manual work, within the acceptable range of accuracy, the work efficiency is greatly improved, and the training cost is greatly reduced. However, the current general classification model pre-trained based on the ImageNet dataset or the general object detection and segmentation model trained based on the COCO dataset cannot well adapt to the current scenario, and tiny defects cannot be well detected by object detection and segmentation.

[0005] "Online Defect Detection Device and Method for Rubber Hoses" (CN107154039A) proposed a solution based on traditional computer digital image processing technology. By detecting abnormal changes in the gray values within the ROI region of the image, it locates possible defect areas. However, its algorithm is only limited to the abnormal changes in the surface gray scale of the object to be detected. Not only is the logic too simple, resulting in low accuracy, but it also cannot detect minor defects and cannot classify the detected defects. Moreover, this algorithm requires a large number of empirical parameter settings, which inevitably leads to poor generalization ability.

[0006] "Automobile Rubber Hose Mandrel Quality Evaluation Method Based on BP Neural Network" (CN108133261A) proposed a solution based on BP neural network. Its algorithm designed a very simple BP neural network to classify the defects on the object to be detected. However, its algorithm is too simple to provide defect location information and cannot detect minor defects.

[0007] The defects of the existing technologies are summarized as follows:

[0008] 1) "Online Defect Detection Device and Method for Rubber Hoses" (CN107154039A) is limited by traditional computer digital image processing technology, with low detection accuracy, unable to detect minor defects, unable to classify the detected defects, requiring a large number of empirical parameters to be set manually during use, having poor generalization ability, and still requiring manual re-inspection later, which limits the overall efficiency.

[0009] 2) "Automobile Rubber Hose Mandrel Quality Evaluation Method Based on BP Neural Network" (CN108133261A) designed a simple BP neural network to classify the defects on the surface of the object to be detected, but it cannot provide defect location information and lacks the ability to detect minor defects. Summary of the Invention

[0010] Aiming at the defects in the existing technologies, the purpose of the present invention is to provide a deep learning-based rubber hose defect detection method and system.

[0011] According to one aspect of the present invention, there is provided a deep learning-based rubber hose defect detection method, including:

[0012] Calculating an error based on a binary Gaussian heat map and the image features output by the model;

[0013] Updating the model based on the error;

[0014] Calculating the correlation coefficient between the verification image features and their corresponding heat maps using the updated model;

[0015] Repeating the above process until the maximum number of loops is reached;

[0016] An optimal model is obtained according to the correlation coefficient;

[0017] The optimal model is used to detect the image of the rubber hose to be tested, and the features of the image of the rubber hose to be tested are output;

[0018] The features of the image of the rubber hose to be tested are compared and analyzed to obtain the defect position, type and confidence level, and the defective part is visually output.

[0019] Preferably, the calculation of the error based on the binary Gaussian heat map and the image features output by the model includes:

[0020] The input training image and relevant information are obtained, and the relevant information includes the positions, lengths and widths, and defect types of all defects on the image;

[0021] The training image is preprocessed;

[0022] A binary Gaussian heat map of the rubber hose defect is drawn according to the training image and the relevant information;

[0023] The model is used to extract the image features of the preprocessed training image;

[0024] The error between the image features and the binary Gaussian heat map is calculated.

[0025] Preferably, the preprocessing includes:

[0026] Data augmentation processing is performed on the input image;

[0027] The input image and the relevant information are processed into a standardized data format;

[0028] Preferably, the error acquisition process includes:

[0029] For a certain type of rubber hose defect actually existing in the image, the relative error between classes of the results of this type and the results of other types is calculated, and this error is maximized;

[0030] The Dice loss function between the heat map and the output result of the model is calculated, and this error is minimized;

[0031] The regression loss between the heat map and the output result of the model is calculated, summed, and normalized by the defect area to balance the loss weights of large-area and small-area defects;

[0032] The regression loss between the heat map and the output result of the model is calculated, averaged, and normalized according to the number of defect categories to increase the loss weight of large-area defects;

[0033] The loss functions calculated in the above 4 small steps are weighted and summed.

[0034] Preferably, the model adopts a general convolutional neural network model.

[0035] Preferably, calculating the correlation coefficient between the verification image features and their corresponding heatmaps using the updated model includes:

[0036] Calculating the error of each training image, and obtaining N errors for N training images;

[0037] Dividing N training images into y groups, with x images in each group, N = x * y;

[0038] Averaging the errors of x images in a group for updating the model;

[0039] After updating y times, then using the results of all verification images to calculate the Pearson correlation coefficient;

[0040] Obtaining the optimal model according to the correlation coefficient includes:

[0041] Assuming the maximum number of loops is M, then there are M Pearson correlation coefficients;

[0042] Selecting the one with the largest Pearson correlation coefficient from the M Pearson correlation coefficients as the optimal model.

[0043] Preferably, performing a comparative analysis on the detection results to obtain the defect location, type, and confidence level, and visualizing the defective part, including: comparing the highest confidence level of each category in the model output result with the threshold, and if the highest confidence level is greater than the preset threshold, determining that this section of the rubber hose has this type of defect;

[0044] Drawing the detected defects on the original image as a heatmap and visually displaying the defects as the final result.

[0045] According to the second aspect of the present invention, there is provided a rubber hose defect detection system based on deep learning, including:

[0046] An error module, which calculates the error based on the training image features output by the model and the binary Gaussian heatmap of the training image;

[0047] An optimization and update module, which updates the model based on the error;

[0048] A correlation coefficient evaluation module, which calculates the correlation coefficient between the verification image features and their heatmaps using the updated model;

[0049] A loop module, which repeats the above process until the maximum number of loops is reached;

[0050] An optimal module, which obtains the optimal model according to the correlation coefficient;

[0051] An application module that uses the optimal model to detect the image of the rubber hose to be tested and outputs the detection result;

[0052] A result module that conducts comparative analysis on the detection result, obtains the defect location, type, and confidence level, and visualizes the defective part.

[0053] According to the third aspect of the present invention, there is provided a terminal, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, it can be used to execute the above method or run the above system.

[0054] According to the fourth aspect of the present invention, there is provided a computer-readable storage medium with a computer program stored thereon, wherein when the program is executed by a processor, it can be used to execute the above method or run the above system.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The method and system for detecting rubber hose defects based on deep learning in the embodiments of the present invention achieve automatic detection of rubber hose surface defects with high accuracy, greatly improving the quality inspection efficiency of vehicle rubber hoses.

[0057] The method and system for detecting rubber hose defects based on deep learning in the embodiments of the present invention use a neural network to process digital images, which have stronger network fitting ability compared to the existing technologies using neural networks; they can not only give the detailed categories of defects but also the position information of the defects. Moreover, the superior performance of the convolutional neural network itself guarantees the accuracy. At the same time, the running speed of the convolutional neural network itself is not slow, and a processing speed of 5 frames per second can fully meet the real-time requirement of detection.

[0058] The method and system for detecting rubber hose defects based on deep learning in the embodiments of the present invention use a heat map for model training. The image features output by the model are at the pixel level, and the error calculation for each image is also at the pixel level. At the same time, there is a corresponding small target error function, enabling the model to focus on small target learning and having the ability to detect minor defects.

[0059] The method and system for detecting rubber hose defects based on deep learning in the embodiments of the present invention can be used in other similar scenarios with only simple parameter adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] By reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings, other features, objects, and advantages of the present invention will become more apparent:

[0061] Figure 1Flow chart of the rubber hose defect detection method based on deep learning in an embodiment of the present invention;

[0062] Figure 2 Flow chart of the rubber hose defect detection method based on deep learning in a preferred embodiment of the present invention;

[0063] Figure 3 Schematic structural diagram of the rubber hose defect detection system based on deep learning in a preferred embodiment of the present invention. Detailed implementation manners

[0064] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made. These all belong to the protection scope of the present invention.

[0065] Refer to Figure 1 , the present invention provides an embodiment, a rubber hose defect detection method based on deep learning, and the process is as follows:

[0066] S1, calculate the error based on the training image features and the training image binary Gaussian heat map output by the model;

[0067] S2, update the model based on the error;

[0068] S3, calculate the correlation coefficient between the verification image features and its heat map using the updated model;

[0069] S4, repeat the above process until the maximum number of loops is reached;

[0070] S5, obtain the optimal model according to the correlation coefficient;

[0071] S6, use the optimal model to detect the image of the rubber hose to be tested and output the detection result;

[0072] S7, perform comparative analysis on the detection result, obtain the defect position, type and confidence level, and visualize and warn the defect part.

[0073] This embodiment realizes the automatic detection of the surface defects of the rubber hose, with high accuracy. Workers only need to screen according to the alarm information, which greatly improves the quality inspection efficiency of the automotive rubber hose;

[0074] Refer to Figure 2 , in a preferred embodiment of the present invention, when implementing S1, the specific process includes:

[0075] S101, read the input rubber hose image and related information;

[0076] S102. Preprocess the input image in S101;

[0077] S103. Draw a binary Gaussian heat map of the hose defect based on the input image and related information in S101; its size is the same as the image feature size output by the model;

[0078] S104. Send the preprocessed image in S102 into the model to extract image features;

[0079] S105. Calculate the error between the image features output by the model in S104 and the heat map obtained in S103.

[0080] In a preferred embodiment, implement S102, and its specific process is as follows:

[0081] S1021. Perform necessary data augmentation processing on the input image;

[0082] S1022. Process the input image and related information into a standardized data format.

[0083] In a preferred embodiment, implement S105, and its specific process is as follows:

[0084] S1051. Based on the image-related information, obtain the defects existing in the image; for a certain type of rubber hose defect actually existing in the image, calculate the inter-class relative error between the results of this type and the results of other types, and maximize this error;

[0085] S1052. Calculate the Dice loss function between the heat map and the image features output by the model, and minimize this error;

[0086] S1053. Calculate the regression loss between the heat map and the model output result, sum it, and normalize it according to the defect pixel area (this area is obtained through the heat map. When drawing the heat map, the pixel values at positions without defects are all 0. So the defect area = the number of pixel points with pixel values > 0 on the heat map) to balance the loss weights of large-area and small-area defects;

[0087] S1054. Calculate the regression loss between the heat map and the model output result, take the average, and normalize it according to the number of defect categories to increase the loss weight of large-area defects;

[0088] S1055. Weightedly sum the loss functions calculated in the above 4 small steps. Here, the weight is a simple positive integer, which is multiplied by the loss term and will be adjusted according to the model training effect and is not fixed.

[0089] In the above embodiments, multiple errors are utilized to optimize the model from multiple perspectives; it expands the class interval at the feature level; reduces the gap between the image features and the heat map; balances the error weights of large and small area defects; and the obtained model has strong generalization ability.

[0090] See Figure 2 , in a preferred embodiment of the present invention, in step S3, the correlation coefficient between the verified image features and their heat map is calculated using the updated model, that is: after all the training images are calculated, the Pearson correlation coefficient between the image features and the heat map output by the model for all the verified images is calculated. Specifically, the error of each training image is calculated, and N errors are obtained for N training images;

[0091] The N training images are divided into y groups, with x images in each group, N = x * y;

[0092] The errors of the x images in a group are averaged and used to update the model;

[0093] After updating y times, the results of all the verified images are used to calculate the Pearson correlation coefficient;

[0094] See Figure 2 , in a preferred embodiment of the present invention, in steps S4 and S5, the optimal model is selected according to the Pearson correlation coefficient obtained in step S3 of the above embodiment. Specifically, assuming the maximum number of loops is M, then there are M Pearson correlation coefficients;

[0095] The one with the largest Pearson correlation coefficient is selected from the M Pearson correlation coefficients as the optimal model.

[0096] See Figure 2 , in a preferred embodiment of the present invention, in step S6, the specific process is as follows:

[0097] S601, read the image of the rubber tube with unknown defect conditions;

[0098] S602, perform normalization processing on the input image;

[0099] S603, send the preprocessed image into the model to extract image features.

[0100] In a preferred embodiment of the present invention, in step S7, the specific process is as follows:

[0101] S701, compare the highest confidence of each category in the model output result with the threshold. If the highest confidence is greater than the preset threshold, it is determined that this section of the rubber tube has the defect of this category;

[0102] S702, draw the detected defect in the original image as a heat map and visually display the defect as the final result.

[0103] The output result of the model in this embodiment is called "image feature", which itself contains the confidence of each defect category and its position. Subsequently, the image feature is enlarged to the same size as the image, and the area on the image feature where the confidence of a certain category is higher than the threshold corresponds to the area with the defect of this category on the image. The detected defects are visually feedback, which is intuitive and convenient for correction.

[0104] Based on the same technical concept, the present invention also provides a rubber tube defect detection system based on deep learning, which includes:

[0105] An error module, which calculates the error based on the training image feature output by the model and the training image binary Gaussian heat map;

[0106] An optimization and update module, which updates the model based on the error;

[0107] A correlation evaluation module, which calculates the correlation coefficient between the verified image feature and its heat map by using the updated model;

[0108] A loop module, which repeats the above process until the maximum number of loops is reached;

[0109] An optimal module, which obtains the optimal model according to the correlation coefficient;

[0110] An application module, which uses the optimal model to detect the image of the rubber tube to be measured and outputs the detection result;

[0111] A result module, which conducts a comparative analysis on the detection result, obtains the defect position, type and confidence, and visualizes the defect part.

[0112] See Figure 3 , in a preferred embodiment, the error module includes an input module, an image preprocessing module, a weight parameter module, and an error calculation module;

[0113] See Figure 3 , in a preferred embodiment, the application module includes an input module, a normalization processing module, and a weight parameter module;

[0114] See Figure 3 , in a preferred embodiment, the result module includes a defect output module and a result visualization module.

[0115] In the above examples of the present invention, the specific implementation techniques of each module can refer to the corresponding steps of the rubber tube defect detection method based on deep learning in the above embodiments, which will not be elaborated here.

[0116] Based on the same inventive concept, in other embodiments of the present invention, there is provided a terminal, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it can be used to execute the above method, or run the above system.

[0117] Based on the same inventive concept, in other embodiments of the present invention, there is provided a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it can be used to execute the above method, or run the above system.

[0118] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various deformations or modifications within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. A method for detecting rubber tube defects based on deep learning, characterized in that, Including: Calculating the error based on the binary Gaussian heat map and the image features output by the model; Updating the model based on the error; Calculating the correlation coefficient between the verified image features and their corresponding heat maps using the updated model; Repeating the above process until the maximum number of loops is reached; Obtaining the optimal model according to the correlation coefficient; Using the optimal model to detect the image of the rubber hose to be tested and outputting the image features of the rubber hose to be tested; Performing comparative analysis on the image features of the rubber hose to be tested to obtain the defect position, type, and confidence level, and visually outputting the defective part.

2. The method for detecting rubber tube defects based on deep learning according to claim 1, wherein, The calculating the error based on the binary Gaussian heat map and the image features output by the model includes: Obtaining the input training image and relevant information, where the relevant information includes the positions, lengths, widths, and defect types of all defects on the image; Preprocessing the training image; Drawing the binary Gaussian heat map of the rubber hose defect according to the training image and relevant information; Using the model to extract the image features of the preprocessed training image; Calculating the error between the image features and the binary Gaussian heat map.

3. The method for detecting rubber tube defects based on deep learning according to claim 2, wherein, The preprocessing includes: Performing data augmentation processing on the input image; Processing the input image and relevant information into a standardized data format.

4. A method for detecting rubber tube defects based on deep learning according to claim 1, characterized in that The model uses a general convolutional neural network model.

5. The method for detecting rubber tube defects based on deep learning according to claim 1, wherein, The calculating the correlation coefficient between the verified image features and their corresponding heat maps using the updated model includes: Calculating the error of each training image, and obtaining N errors for N training images; Divide N training images into y groups, with x images in each group. ; Averaging the errors of x images in a group for updating the model; After updating y times, using the results of all verified images to calculate the Pearson correlation coefficient; The obtaining the optimal model according to the correlation coefficient includes: Assuming the maximum number of loops is M, then there are M Pearson correlation coefficients; Selecting the one with the largest Pearson correlation coefficient from the M Pearson correlation coefficients as the optimal model.

6. A method for detecting rubber tube defects based on deep learning according to claim 1, characterized in that The performing comparative analysis on the image features of the rubber hose to be tested to obtain the defect position, type, and confidence level, and visually outputting the defective part includes: Comparing the highest confidence level of each defect category in the image features output by the model with the threshold. If the highest confidence level is greater than the preset threshold, it is determined that this section of the rubber hose has this type of defect, and the position of this type of defect is obtained at the same time; Drawing the heat map of the defect on the original image to visually display the defect as the final result.

7. A rubber tube defect detection system based on deep learning, characterized in that, Including: Error module, which calculates the error based on the training image features output by the model and the binary Gaussian heat map of the training image; Optimization and update module, which updates the model based on the error; Correlation coefficient evaluation module, which calculates the correlation coefficient between the verified image features and their heat maps using the updated model; Loop module, which repeats the above process until the maximum number of loops is reached; Optimal module, which obtains the optimal model according to the correlation coefficient; Application module, which uses the optimal model to detect the image of the rubber hose to be tested and outputs the detection result; Result module, which performs comparative analysis on the detection result to obtain the defect position, type, and confidence level, and visualizes the defective part.

8. A terminal, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to execute the method described in any one of claims 1-6, or to run the system described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it can be used to execute the method described in any one of claims 1-6, or to run the system described in claim 7.

Images