Vision-based surface defect detection method and detection system

Abstract

The invention provides a vision-based surface defect detection method and detection system, and relates to the technical field of data processing, and the method comprises the steps of data acquisition and processing, data addition, modeling, model training and detection and the like. The system comprises a data acquisition and processing module, a data adding module, a modeling module, a model training module and a detection module, new defect data is generated by adopting a generative adversarial network, a detection model is trained by adopting the new defect data and original historical defect data, and the accuracy of bearing surface defect detection is improved.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a vision-based surface defect detection method and detection system. Background Art

[0002] In industrial production, bearings are key components, and their quality and performance are directly related to the operating efficiency and safety of the entire equipment. However, due to factors such as harsh working environment and long-term wear, bearings often have various defects, such as cracks, peeling, rust, etc. These defects will not only reduce the service life of the bearing, but may also cause equipment failure and even lead to serious safety accidents.

[0003] With the rapid development of computer vision and deep learning technology, bearing defect detection methods based on computer vision and deep learning have gradually emerged. This method collects bearing data and uses deep learning algorithms to analyze and process the collected bearing data, thereby realizing automatic detection and identification of bearing defects. However, in practical applications, high-quality bearing defect data with clear labels is often very scarce, which limits the training effect and practical application value of deep learning models. Summary of the invention

[0004] In order to increase the amount of bearing defect data with clear labels and improve the training effect of deep learning models, the present application provides a vision-based surface defect detection method and detection system.

[0005] In the first aspect, the present application provides a method for detecting surface defects based on vision, which adopts the following technical solution: A surface defect detection method based on vision comprises the following steps: Data collection and processing: Collect historical defect data and bearing data to be processed, and build defect labels based on the historical defect data; Increase data: including network construction, data generation, data judgment, optimization and data integration; Network construction: establishing a generative adversarial network, wherein the generative adversarial network includes a generator and two discriminators, and the two discriminators are of different categories; Generate data: Based on historical defect data and defect labels, use the generator to generate new defect data; Data judgment: Input the new defect data into the two discriminators respectively, and obtain the outputs of the two discriminators respectively. If the output results of the two discriminators are consistent and the same as the expected defect labels, the data integration step is performed; if not, the optimization step is performed; Optimization: Adjust the parameters of the generative adversarial network and execute the steps of generating data until the preset optimization stop conditions are met; Data integration: new defect data and historical defect data are used as new historical defect data; Modeling: Building a detection model; Model training: Use new historical defect data to train the detection model to obtain the trained detection model; Detection: The bearing data to be processed is used as the input of the trained detection model to obtain the first detection result.

[0006] By adopting the above technical solution, this application improves the accuracy of data and the consistency of labels by collecting historical defect data and constructing defect labels, and uses a generative adversarial network to generate new defect data, effectively solving the problem of data scarcity, while improving the diversity and authenticity of the generated defect data, thereby improving the richness and quality of the data set. By using two different categories of discriminators to verify the generated defect data, the quality of the new defect data meets the expected defect labels, providing high-quality data support for the model training step. Through the optimization step, the parameters of the generative adversarial network are continuously adjusted, so that the generated data is more in line with actual needs, the efficiency and accuracy of data generation are improved, and the efficiency and accuracy of model training are improved.

[0007] Optionally, in the step of data collection and processing, a bearing image corresponding to the historical defect data is also collected, the detection model includes: an image segmentation sub-model and a CNN sub-model, and the step of model training also includes: a first training, a second training, a third training and a first judgment; First training: input the bearing image corresponding to the historical defect data into the image segmentation sub-model, and output n sub-images of the target image area; Second training: input the i-th sub-image into the m hidden layers of the CNN sub-model respectively, and obtain m outputs of the hidden layers of the CNN sub-model; Third training: input the m outputs of the hidden layer into the output layer of the CNN sub-model to obtain the second detection result; First judgment: determine whether the second detection result is the same as the defect label. If so, execute the detection step. If not, take the i+1th sub-image as the new i-th sub-image and re-execute the first training step until the preset stop judgment condition is met.

[0008] By adopting the above technical solution, the CNN sub-model extracts features of multiple features of the sub-image through multiple hidden layers, and each hidden layer can learn feature information at different levels. In the first judgment stage, if the second detection result is inconsistent with the defect label, it means that there may be a problem with the training sample. At this time, the detection model will automatically select the next sub-image for retraining. This adaptive learning mechanism helps the detection model to continuously optimize its own performance. Through continuous iterative training, the detection model can gradually learn more accurate and stable detection rules and improve the robustness of the detection. The present application also combines the image segmentation sub-model and the CNN sub-model, uses the image segmentation sub-model to segment the image, and uses the CNN sub-model to extract features, so that more feature information can be learned in a shorter time, thereby improving training efficiency.

[0009] Optionally, after performing the first training step and before performing the second training step, the method further includes: Image processing: grayscale the n sub-images to obtain n grayscale sub-images; First extraction: obtain the gray value of the jth pixel in the i-th grayscale sub-image, recorded as the first gray value; obtain the gray values ​​of eight pixels adjacent to the jth pixel, recorded as the second gray value; Second extraction: obtaining the maximum and minimum values ​​of the second grayscale value; The third judgment: judging whether the first gray value is greater than the minimum value and less than the maximum value, if so, executing the first replacement step; if not, executing the marking step; First replacement: taking the j+1th pixel point as the new jth pixel point, and executing the first extraction step until the preset first replacement stop condition is met, and executing the second replacement step; Marking: Mark the j-th pixel as a noise point, replace the gray value of the j-th pixel with the average gray value of the eight adjacent pixels, and perform the first replacement step; Second replacement: taking the i+1th grayscale sub-image as the new ith grayscale sub-image, and executing the first extraction step until a preset second replacement stop condition is met, and executing the second training step.

[0010] By adopting the above technical solution, grayscale processing can reduce the color information of the image and retain only the brightness information, thereby reducing the complexity of image processing. By obtaining the grayscale value of the pixel and the grayscale value of its adjacent pixel, basic data can be provided for subsequent noise detection and replacement. The calculation of the maximum and minimum values ​​in the second extraction step helps to determine the range of the first grayscale value. By judging whether the first grayscale value falls between the maximum and minimum values, it is judged whether the pixel is a noise point. If so, it is marked and the grayscale value is replaced to reduce the influence of noise points on defect judgment.

[0011] Optionally, after executing the third judgment step and before executing the marking step, the method further includes: Fourth judgment: judging whether the first gray value is less than the minimum value, if so, executing the first calculation step; if not, executing the second calculation step; First calculation: calculate the difference between the first gray value and the minimum value, recorded as the first difference; Second calculation: calculate the difference between the first gray value and the maximum value, and record it as the second difference; Fifth judgment: judge whether the first difference or the second difference is greater than a preset difference threshold, if so, execute the marking step; if not, execute the first replacement step.

[0012] By adopting the above technical solution, the first calculation and the second calculation respectively calculate the difference between the first gray value and the minimum and maximum values. These differences reflect the degree of deviation of the gray value of the pixel relative to the gray value of its neighborhood. By comparing the relationship between the difference and the preset difference threshold, the noise point can be more accurately identified, that is, when the first difference or the second difference is greater than the preset difference threshold, the pixel is a noise point and the marking step needs to be performed, which helps to reduce the impact of noise on subsequent processing. When the first difference or the second difference is not greater than the preset difference threshold, it means that the difference is within an acceptable range. Then, the first replacement step is performed to avoid unnecessary processing of non-noise points.

[0013] Optionally, after performing the first training step and before performing the second training step, the method further includes: Establish a sub-image set: integrate n sub-images into a sub-image set; The third extraction: using laser radar to obtain the point cloud data corresponding to each pixel in each sub-image, and integrating all the point cloud data of a sub-image into a point cloud data set; Fourth extraction: obtain the normal vector of each point cloud point in the point cloud data set; Sixth judgment: judge whether any two normal vectors are parallel to each other. If so, execute the deletion step; if not, execute the traversal step; Deletion: deleting the sub-image corresponding to the point cloud data set in the sub-image set; Traversal: Traverse all sub-images in the sub-image set, and repeat the steps from the third extraction to the sixth judgment until the preset traversal stop condition is met.

[0014] By adopting the above technical solution, the laser radar is used to obtain the point cloud data corresponding to each pixel in the sub-image, and all the point cloud data of a sub-image are integrated into a point cloud data set, which can more accurately reflect the three-dimensional shape and structure of the current bearing. Obtaining the normal vector of each point cloud point in the point cloud data set is helpful to analyze the surface characteristics of the bearing. When there are no defects on the bearing surface, the normal vectors should be parallel to each other. When there are defects on the bearing surface, the normal vectors are not parallel to each other. Therefore, by judging whether the normal vectors are parallel to each other, it is possible to identify whether there are defects on the surface. Through the deletion step, the sub-images without defects are deleted to reduce resource usage.

[0015] Optionally, after executing the sixth judgment step and before executing the traversal step, the method further includes: Fifth extraction: obtain the kth normal vector and the k+1th normal vector with an included angle, obtain the point cloud point corresponding to the kth normal vector, and obtain the point cloud point corresponding to the k+1th normal vector; The third calculation: calculate the curvature at two point cloud points respectively, and obtain the point cloud point with the smallest curvature, which is recorded as the first point cloud point; Sixth extraction: obtaining a pixel point corresponding to the first point cloud point, recording it as the first pixel point, and taking the first pixel point as the seed point; Fourth calculation: performing a nearest neighbor search based on the seed point, calculating the angle between the normal vector of the seed point and the normal vector of the neighboring point, and obtaining the first data; Seventh judgment: judging whether the first data is smaller than a preset angle threshold, if so, executing the traversal step; if not, executing the classification step; Classification: Classify neighbor points and seed points into the same defect label; Third replacement: use the k+1th normal vector as the new kth normal vector, and perform the fifth extraction step.

[0016] By adopting the above technical solution, the present application helps to identify the defect location in the point cloud data by extracting the normal vector with an angle and its corresponding point cloud point, and calculating the curvature at two point cloud points, so as to identify the relatively flat or curved area in the point cloud data, and then select the point cloud point with the smallest curvature as the seed point, and use the pixel point corresponding to the point cloud point with the smallest curvature as the seed point, which helps to locate the defect edge more accurately in the nearest neighbor search. Performing a nearest neighbor search based on the seed point and calculating the angle between the seed point and the normal vector of the neighbor point helps to identify the boundary of the defect area. If the first data is less than the preset angle threshold, the neighbor point and the seed point are classified as the same defect label, which helps to uniformly process the defect area in subsequent processing and improves the accuracy and efficiency of defect identification.

[0017] Optionally, after executing the fourth calculation step and before executing the seventh judgment step, the method further includes: Fifth calculation: calculate the curvature of neighboring points; Eighth judgment: judging whether the curvature of the seed point is smaller than the curvature of the neighboring point, if so, executing the seventh judgment; if not, executing the fourth replacement step; Fourth replacement: taking the neighbor point as a new seed point and performing the fourth calculation step.

[0018] By adopting the above technical solution, by calculating the curvature of the neighboring points, important information about the local shape of the neighboring points is provided. The curvature information helps to distinguish between defective areas and non-defective areas, as well as different parts within the defective area. By comparing the curvature of the seed point and the neighboring points, it can be further confirmed whether the seed point is located at the center or key position of the defective area. When the curvature of the seed point is not less than the curvature of the neighboring points, the neighboring points are used as new seed points, which helps to gradually approach the edge position of the defective area.

[0019] Optionally, after executing the sixth judgment step and before executing the traversal step, the method further includes: Seventh extraction: based on the sixth judgment step, extracting the point cloud points corresponding to the two normal vectors to obtain two point cloud points, and integrating the two point cloud points into a point cloud point pair; Ninth judgment: judge whether the point cloud point pair is located at the edge of the bearing. If so, execute the deletion step; if not, execute the traversal step.

[0020] By adopting the above technical solution, the point cloud points corresponding to the two normal vectors are extracted as point cloud point pairs, which helps to focus on specific feature points in the point cloud data that may be related to the bearing edge. By judging whether the point cloud point pair is located at the edge of the bearing, the sub-images related to the bearing edge can be further screened and removed, which helps to reduce the complexity of subsequent processing and improve accuracy.

[0021] Optionally, after executing the ninth judgment step and before executing the deletion step, the method further includes: The eighth extraction: Based on the statistical analysis method, obtain the change law of the angle between the normal vectors at the edge of the bearing; Ninth extraction: obtaining the angle between the two normal vectors corresponding to the point cloud point pair, recorded as the first angle; Tenth judgment: judging whether the first angle conforms to the change rule, if so, executing the deletion step; if not, executing the recording step; Record: A defect is marked at the point cloud point pair.

[0022] By adopting the above technical solution, by statistically analyzing the changing law of the angle between the normal vectors at the edge of the bearing, the geometric characteristics of the bearing edge can be more deeply understood. The angle between the normal vectors corresponding to the point cloud point pair is obtained, and the normal vector is directly associated with the geometric characteristics of the currently processed point cloud point pair or sub-image set. By judging whether the first angle conforms to the changing law obtained by statistical analysis, it can be further confirmed whether the currently processed point cloud point pair or sub-image set is located at the edge of the bearing. If so, it is judged that the first angle is caused by the edge curve. If not, it indicates that the point cloud point pair is located in the defect area.

[0023] In a second aspect, the present invention provides a surface defect detection system based on vision, which adopts the following technical solution: A vision-based surface defect detection system, comprising: The data collection and processing module is used to collect historical defect data and bearing data to be processed; it is also used to construct defect labels based on historical defect data; Add data modules, including network construction unit, data generation unit, data judgment unit, optimization unit and data integration unit; A network building unit, used to build a generative adversarial network, wherein the generative adversarial network includes a generator and two discriminators, and the two discriminators are of different categories; A data generation unit, used to generate new defect data using a generator based on historical defect data and defect labels; The data judgment unit is used to input the new defect data into the two discriminators respectively, and obtain the outputs of the two discriminators respectively. If the output results of the two discriminators are consistent and the same as the expected defect labels, the data integration unit is triggered; otherwise, the optimization unit is triggered; An optimization unit, used to adjust the parameters of the generative adversarial network; A data integration unit, used for taking the new defect data and the historical defect data as new historical defect data; Modeling module, used to build detection models; A model training module is used to train the detection model using new historical defect data to obtain a trained detection model; The detection module is used to use the bearing data to be processed as the input of the trained detection model to obtain a first detection result.

[0024] By adopting the above-mentioned technical solution, this application comprehensively collects and processes historical defect data through the data acquisition and processing module to construct accurate defect labels; adds a data module to use the generative adversarial network to efficiently generate diverse and realistic defect data and corresponding bearing images, and ensures data quality through precise data judgment and optimization iteration, and integrates the new defect data with the original historical defect data into a new rich data set, providing a solid foundation for the modeling module. The detection model trained by the model training module can realize fast and accurate detection of bearing defects, significantly improving detection efficiency and accuracy.

[0025] In summary, the present application includes at least one of the following beneficial technical effects: 1. This application ensures the accuracy of data and the consistency of labels by collecting historical defect data and constructing defect labels. It uses a generative adversarial network to generate new defect data, effectively solving the problem of data scarcity, while improving the diversity and authenticity of the generated defect data, thereby improving the richness and quality of the data set. By using two different categories of discriminators to verify the generated defect data, the quality of the new defect data meets the expected defect labels, providing high-quality data support for the model training step. Through the optimization step, the parameters of the generative adversarial network are continuously adjusted to make the generated data more in line with actual needs, improve the efficiency and accuracy of data generation, and thus improve the efficiency and accuracy of model training.

[0026] 2. Use LiDAR to obtain the point cloud data corresponding to each pixel in the sub-image, and integrate all the point cloud data of a sub-image into a point cloud data set, which can more accurately reflect the three-dimensional shape and structure of the current bearing. Obtaining the normal vector of each point cloud point in the point cloud data set is helpful to analyze the surface characteristics of the bearing. When there are no defects on the bearing surface, the normal vectors should be parallel to each other. When there are defects on the bearing surface, the normal vectors are not parallel to each other. Therefore, by judging whether the normal vectors are parallel to each other, it is possible to identify whether there are defects on the surface. Through the deletion step, the sub-images without defects are deleted to reduce resource usage.

[0027] 3. Extracting the point cloud points corresponding to the two normal vectors as point cloud pairs helps to focus on specific feature points in the point cloud data that may be related to the bearing edge. By determining whether the point cloud pair is located at the edge of the bearing, the sub-images related to the bearing edge can be further screened and removed, which helps to reduce the complexity of subsequent processing and improve accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of Example 1 of the present application; Figure 2 is a flow chart of Example 2 of the present application; Figure 3 is a flow chart of Example 3 of the present application; Figure 4 This is a flow chart of Example 4 of the present application. DETAILED DESCRIPTION

[0029] The following combination Figures 1 to 4 This application is described in further detail.

[0030] Embodiment 1: This embodiment discloses a surface defect detection method based on vision, referring to Figure 1 The detection method includes: S1 data collection and processing, S2 data addition, S3 modeling, S4 model training and S5 detection. First, the historical defect data of the bearing and the bearing data to be processed are collected, then defect labels are constructed based on the historical defect data, then new defect data is generated by using a generative adversarial network, then a detection model is established, then the detection model is trained, and then the image to be processed is input into the trained detection model to obtain the detection result. This embodiment includes the following steps: S1 data collection and processing, collecting historical defect data, bearing images corresponding to historical defect data and bearing data to be processed.

[0031] Historical defect data includes various defect types and degrees of bearings and related maintenance records.

[0032] The historical defect data is collated and analyzed to extract key defect features, and based on the key defect features, defect labels such as "crack" and "wear" are added to the corresponding bearing images.

[0033] S2 adds data, including S21 network construction, S22 data generation, S23 data judgment, S24 optimization and S25 data integration.

[0034] S21 builds a network, i.e., establishes a generative adversarial network, wherein the generative adversarial network includes a generator and two discriminators, and the two discriminators are of different categories.

[0035] The task of the generator is to generate realistic defect data and defective bearing images, while the two discriminators are responsible for verifying the quality of these new defect data. The two discriminators are of different categories, indicating that they judge the generated data from different perspectives. For example, one discriminator focuses on the overall structure and texture of the image, while the other pays more attention to the details and local features of the image. This generative adversarial network helps to improve the similarity between the generated data and the real data.

[0036] S22 generates data, based on the historical defect data and defect labels, using a generator to generate new defect data and defect bearing images under various defect labels. The new defect data and defect bearing images are similar in features to the original historical defect data and the images corresponding to the historical defect data, but not completely the same, thereby increasing the diversity and richness of the data set, helping the detection model to learn more feature information, and improving the accuracy and generalization ability of detection.

[0037] S23 data judgment, input the new defect data and defective bearing image into two discriminators respectively, and obtain the output of the two discriminators respectively. If the output results of the two discriminators are consistent and the same as the expected defect labels, it means that the generated data quality is high, and then execute S25 data integration; if not, execute S24 optimization.

[0038] S24 optimization, according to the judgment result in S23 data judgment, adjusts the parameters of the generative adversarial network (for example, adjusts the structure of the generator), and executes S22 to generate data until the preset optimization stop condition is met. The optimization stop condition includes: reaching the preset number of executions of S22 to generate data, the quality of the new defect data reaches a certain threshold, etc.

[0039] S25 data integration, taking the new defect data and the collected historical defect data as new historical defect data; taking the new defect bearing image and the bearing image corresponding to the historical defect data as the bearing image corresponding to the new historical defect data.

[0040] S3 is a modeling step for establishing a detection model, wherein the detection model includes an image segmentation sub-model and a CNN sub-model, wherein the output of the image segmentation sub-model is connected to the input of the CNN sub-model.

[0041] The image segmentation sub-model is responsible for segmenting the input bearing image into multiple sub-images in order to focus on key image areas, which helps to reduce the amount of computation and improve the accuracy of detection.

[0042] The CNN sub-model is responsible for extracting deep features in the image. The CNN sub-model can process complex image data and output the judgment results of defects.

[0043] S4 model training uses new historical defect data to train the detection model to obtain the trained detection model.

[0044] The S4 model training also includes S41 first training, S42 second training, S43 third training and S44 first judgment.

[0045] S41 is the first training, in which the bearing image corresponding to the historical defect data is input into the image segmentation sub-model, and n sub-images of the target image area are output.

[0046] S42 is the second training, in which the i-th sub-image is input into the m hidden layers of the CNN sub-model respectively. Each hidden layer of the CNN sub-model processes a feature (such as color, texture) of the sub-image and matches it with the defect label to obtain m outputs of the hidden layer of the CNN sub-model.

[0047] S43 is the third training, in which the m outputs of the hidden layer are used as inputs of the output layer of the CNN sub-model. The output layer of the CNN sub-model processes the m outputs and outputs a second detection result.

[0048] S44 is the first judgment, judging whether the second detection result is the same as the defect label, and then executing S5 detection. If not, the i+1th sub-image is used as the new i-th sub-image, and the first training of S41 is re-executed until the preset stop judgment condition is met, and the preset stop judgment condition is i+1>n.

[0049] S5 detection, using the bearing data to be processed as the input of the trained detection model to obtain the first detection result.

[0050] Taking bearing wear detection based on computer vision as an example, the solution of this embodiment is as follows: 1. Data collection: Extract various types of bearing defects (such as cracks, wear, corrosion, etc.), defect levels (minor, moderate, severe) and related maintenance records that have occurred in the past few years from the company's historical record system.

[0051] From the company's image database, find the bearing image corresponding to the above historical defect data.

[0052] On the production line, industrial CDD cameras are used to capture images of bearings in real time.

[0053] Data processing: Sort out the collected historical defect data and extract key features related to bearing wear, such as wear area, wear depth, wear shape, etc. Based on the extracted key features, label the corresponding bearing image with a defect. For wear defects, the label can be marked as "wear".

[0054] 2.1. Network construction: A generative adversarial network is established, wherein the generative adversarial network includes a generator and two discriminators, and the two discriminators are of different categories.

[0055] 2.2. Generate data: Based on historical defect data and defect labels, use the generator to generate new defect data.

[0056] 2.3. Data judgment: Input the new defect data into the two discriminators respectively, and obtain the outputs of the two discriminators respectively. If the output results of the two discriminators are consistent and the same as the expected defect labels, perform the data integration step; if not, perform the optimization step.

[0057] 2.4. Optimization: Adjust the parameters of the generative adversarial network and execute the steps of generating data until the preset optimization stop conditions are met.

[0058] 2.5. Data integration: The new defect data and the collected historical defect data are used as new historical defect data.

[0059] 3. Modeling: Establish a detection model, which includes: an image segmentation sub-model and a CNN sub-model.

[0060] 4. Model training: Use new historical defect data to train the detection model to obtain the trained detection model.

[0061] 4.1. First training: Input the bearing image corresponding to the historical defect data into the image segmentation sub-model and output n sub-images of the target image area.

[0062] 4.2. Second training: Each sub-image is input into the m hidden layers of the CNN sub-model. Each hidden layer processes a feature of the sub-image and matches it with the defect label. After processing by m hidden layers, m outputs of the hidden layer of the CNN sub-model are obtained. .

[0063] 4.3. Third training: m outputs of the hidden layer As the input of the output layer of the CNN sub-model, the output layer processes the m outputs and outputs the second detection result.

[0064] ; Wherein, Y is the probability corresponding to the second detection result; is the weight of the i-th output; is the ith output.

[0065] 4.4. First judgment: judge whether the second detection result is the same as the defect label.

[0066] If they are the same, the detection step is executed.

[0067] If they are different, the i+1th sub-image is used as the new i-th sub-image, and the first training step is executed again until a preset stop judgment condition (i+1>n) is met.

[0068] Preset stop judgment condition: When all sub-images have been processed (i+1>n), stop training. In other embodiments, other stop conditions may also be set, such as reaching a certain number of iterations, loss function convergence, etc.

[0069] 5. Detection: The bearing data to be processed captured in real time from the production line is input into the trained detection model. The detection model processes the input data and outputs the first detection result.

[0070] In this embodiment, S1 data acquisition is used to collect historical defect data, related bearing images and images to be processed; historical defect data is sorted and analyzed, key features are extracted, and defect labels are marked for bearing images; S2 data is added to generate new data through a generative adversarial network; S3 modeling is used to build a detection model including an image segmentation sub-model and a CNN sub-model, the former segments the image to reduce the amount of calculation, and the latter extracts deep features and determines the defect type; S4 model training is used to train and judge the detection model multiple times using new defect data and the original historical defect data until the stopping condition is met to obtain a trained detection model; S5 detection is used to input the bearing image to be processed into the trained detection model to obtain a first detection result.

[0071] Example 2: Reference Figure 2 The difference between this embodiment and the first embodiment is that after executing the first training in S41 and before executing the second training in S42, the present embodiment further includes: S61 image processing, gray-scaling the n sub-images (using a component method, a maximum value method, an average value method, a weighted average method, etc. to convert a color image into a gray-scale image), and obtaining n gray-scale sub-images.

[0072] Component method: Select one of the R, G, and B components of each pixel in the color sub-image as the grayscale value. For example, if the red information in the sub-image is more important, the R component can be selected as the grayscale value.

[0073] Maximum method: Select the maximum value of the R, G, and B components of each pixel in the color image as the grayscale value.

[0074] Average value method: Calculate the average value of the R, G, and B components of each pixel in the color image and use it as the grayscale value.

[0075] Weighted average method: According to the sensitivity of the human visual system to different colors, different weights are assigned to the R, G, and B components, and then the weighted average is calculated as the grayscale value.

[0076] In the above method, the value corresponding to the R component is (255, 0, 0), that is, the maximum intensity of the channel corresponding to the R component is 255, and the intensities of the channels corresponding to the G component and the channels corresponding to the B component are both 0. The value corresponding to the G component is (0, 255, 0), that is, the maximum intensity of the channel corresponding to the G component is 255, and the intensities of the channels corresponding to the R component and the channels corresponding to the B component are both 0. The value corresponding to the B component is (0, 0, 255), that is, the maximum intensity of the channel corresponding to the B component is 255, and the intensities of the channels corresponding to the G component and the channels corresponding to the R component are both 0, and the value of each channel varies between 0 and 255.

[0077] S62: First extraction, obtaining the grayscale value of the j-th pixel in the i-th grayscale sub-image, recorded as the first grayscale value; obtaining the grayscale values ​​of eight pixels adjacent to the j-th pixel, recorded as the second grayscale value.

[0078] S63: second extraction, obtaining the maximum value and the minimum value of the second grayscale value.

[0079] S64 is the third judgment, judging whether the first grayscale value is greater than the minimum value and less than the maximum value. If so, it means that the j-th pixel point is connected with the eight adjacent pixels, and the first replacement of S65 is executed; if not, it means that the j-th pixel point is not connected with the eight adjacent pixels, and the j-th pixel point may be a noise point, which requires further judgment, that is, executing the fourth judgment of S641.

[0080] S641 is the fourth judgment, which determines whether the grayscale value (first grayscale value) of the j-th pixel is less than the minimum value. If so, it means that the j-th pixel is darker than the darkest pixel among its adjacent pixels, which may be caused by noise or error, and the first calculation of S642 needs to be executed; if not, the second calculation of S643 is executed.

[0081] S642 is a first calculation, which calculates the difference between the grayscale value of the j-th pixel and the minimum value, and records it as the first difference.

[0082] S643 is a second calculation, which is to calculate the difference between the grayscale value of the j-th pixel and the maximum value, and record it as the second difference.

[0083] S644 is the fifth judgment, judging whether the first difference or the second difference is greater than the preset difference threshold. If so, it means that the grayscale value of the j-th pixel point is too different from the grayscale value of the adjacent pixel point, and the point is a noise point or an abnormal point, and S66 is executed for marking; if not, it means that although the grayscale value of the j-th pixel point is different from that of the adjacent pixel point, the difference is within the acceptable error range, and then S65 is executed for the first replacement.

[0084] S65 is the first replacement, taking the j+1th pixel point as the new jth pixel point, and executing S62 the first extraction, until the preset first replacement stop condition is met, and then executing S67 the second replacement, wherein the first replacement stop condition includes: the jth pixel point is a pixel point on the image boundary or the preset number of executions of S62 the first extraction is reached.

[0085] S66 marks the j-th pixel as a noise point, replaces the grayscale value of the j-th pixel with the average grayscale value of the eight adjacent pixels, removes the noise point by replacing the grayscale value, and keeps the j-th pixel connected with the eight adjacent pixels, and then executes the first replacement in S65.

[0086] S67 second replacement, taking the i+1th grayscale sub-image as the new i-th grayscale sub-image, and executing S62 first extraction, until the preset second replacement stop condition is met, and then executing S42 second training, the second replacement stop condition is that all sub-images are processed or the preset number of executions of S62 first extraction is reached.

[0087] This embodiment grayscales all sub-images, compares the grayscale values ​​of pixels in each grayscale sub-image, and if the grayscale value of the pixel is within the grayscale value range of its eight adjacent pixels, iterative replacement is performed until a stop condition is met; if it is not within the range, it is further determined whether the error is within an acceptable range. If it is unacceptable, the pixel is marked as a noise point and replaced with the average grayscale value of adjacent pixels. This process is then iterated between all grayscale sub-images until all sub-images are processed.

[0088] Example 3: Reference Figure 3 The difference between this embodiment and the first embodiment is that after executing the first training in S41 and before executing the second training in S42, the present embodiment further includes: S701 creates a sub-image set, integrating all sub-images into a sub-image set.

[0089] S702 is the third extraction, in which a laser radar is used to scan each pixel of each sub-image, obtain point cloud data corresponding to each pixel, and integrate all point cloud data of a sub-image into a point cloud data set.

[0090] S703 fourth extraction, for each point cloud point in the point cloud data set, calculate its normal vector. The normal vector is a vector perpendicular to the point cloud surface and pointing to the outside of the bearing, which can provide information about the direction of the point cloud surface.

[0091] S704 is the sixth judgment, checking the parallelism between all normal vectors in the point cloud data set, and judging whether any two normal vectors are parallel. If so, it indicates that these point cloud points are located on the same plane, indicating that the surface of the sub-image corresponding to the point cloud data set is smooth and has no defects, and then S715 deletion is executed; if not, it indicates that the surface of the sub-image corresponding to the point cloud data set is not smooth and may have defects, and it is necessary to execute S705 fifth extraction.

[0092] S705 is the fifth extraction, obtaining the kth normal vector and the k+1th normal vector with an angle, obtaining the point cloud point corresponding to the kth normal vector, and obtaining the point cloud point corresponding to the k+1th normal vector. These two point cloud points may be located at the edge or defect of the object.

[0093] S706 The third calculation is to calculate the curvature at two point cloud points respectively, obtain the point cloud point with the smallest curvature, and mark it as the first point cloud point. The point with smaller curvature is located in a smoother area, that is, at the edge of the defect area, and the point with larger curvature is located at the defect.

[0094] S707 Sixth extraction: obtaining a pixel point corresponding to the first point cloud point, recording it as a first pixel point, and taking the first pixel point as a seed point.

[0095] S708 fourth calculation: performing nearest neighbor search based on the seed point, and calculating the angle between the normal vector of the seed point and the normal vector of the neighboring point, and recording the value of the angle as the first data.

[0096] S709 Fifth calculation: calculating the curvature of the neighboring points.

[0097] S710 is the eighth judgment, judging whether the curvature of the seed point is smaller than the curvature of the neighboring points. If so, it indicates that the seed point is closer to the edge of the defect area than the neighboring points, and then the seventh judgment S712 is executed; if not, it indicates that the neighboring points are closer to the edge of the defect area than the seed point, and it is necessary to execute the fourth replacement S711 until the curvature of the seed point is 0, then the edge pixel points of the defect area can be obtained, and after obtaining all the pixel points, the accurate defect area can be obtained.

[0098] S711 is the fourth replacement, taking the neighbor point as a new seed point, and performing S708 the fourth calculation.

[0099] S712 is the seventh judgment, judging whether the first data is less than the preset angle threshold. If so, it indicates that the angle is caused by an error and there is no defect here, and then S716 traversal is executed; if not, it indicates that the seed point and the neighboring point are in the same defect area, and S713 classification needs to be executed.

[0100] S713 classification, classifies neighbor points and seed points into the same defect label.

[0101] S714 is the third replacement, recording the point cloud point corresponding to the k-th normal vector in the fifth extraction of S705 as the edge point of the defect area, and taking the k+1-th normal vector as the new k-th normal vector, and executing the fifth extraction of S705 until all edge points of the defect area are extracted, and then executing the traversal of S716.

[0102] After the third replacement in S714 is performed, if it is found that multiple defective regions are nested with each other, the largest defective region is selected as the real defective region, and the defective regions inside it are deleted.

[0103] S715: deleting the sub-image corresponding to the point cloud data set from the sub-image set.

[0104] S716 traverses all sub-images in the sub-image set, and repeatedly executes S702 third extraction to S704 sixth judgment until a preset traversal stop condition is met, wherein the traversal stop condition is that all sub-images in the sub-image set have been processed.

[0105] This embodiment establishes a sub-image set, extracts the point cloud data and normal vector of each sub-image, screens sub-images that may contain defects by judging the parallelism of the normal vector, then calculates the curvature of the point cloud points, performs a nearest neighbor search based on the curvature and the angle between the normal vectors, classifies similar points as the same defect label, and finally traverses all sub-images to complete defect detection. If the normal vectors are parallel, it is considered to be defect-free and the corresponding sub-image is deleted; if the parallel condition is not met, a series of calculation and judgment steps are used to identify and mark the defect area, thereby improving the accuracy of dividing the defect area.

[0106] Example 4: Reference Figure 4 The difference between this embodiment and the third embodiment is that after executing the sixth judgment in S704 and before executing the traversal in S716, the following is further included: S81 seventh extraction, based on S704 sixth judgment, when there is an angle between the two normal vectors, extract the point cloud points corresponding to the two normal vectors, obtain two point cloud points, and mark the two point cloud points together as a point cloud point pair.

[0107] S82 is the ninth judgment, judging whether the point cloud point pair is located at the edge of the bearing. If so, it indicates that the angle between the two normal vectors may be caused by the edge of the bearing, and it is necessary to execute S83, the eighth extraction; if not, it indicates that the point cloud point pair is in the defective area, and then execute S716 traversal.

[0108] S83 eighth extraction: if the point cloud point pair is located at the edge of the bearing, then based on the statistical analysis method, obtain the change law of the angle between the normal vectors at the edge of the bearing.

[0109] The changing trend of the normal vector at the edge of a normal and defect-free bearing is the same. This application is based on a statistical analysis method to statistically analyze the changing law, and determines whether there is a surface defect at the edge of the bearing by judging whether the first angle conforms to the said changing law.

[0110] S84 is a ninth extraction, obtaining the angle between the two normal vectors corresponding to the point cloud point pair, recorded as the first angle.

[0111] S85 is the tenth judgment, judging whether the first angle conforms to the said change rule. If so, it means that the point cloud point pair (i.e., the points at the edge) conforms to the expected geometric features, and the point cloud point pair is considered normal, and then S715 is executed for deletion; if not, it means that the point cloud point pair has surface defects, and S86 is executed for recording.

[0112] S86 records: a defect is marked at the point cloud point pair.

[0113] Taking the detection of bearing edge surface defects as an example, this embodiment includes the following steps: 8.1. Seventh extraction: The two normal vectors are: , corresponding to point A in the point cloud; , corresponding to point cloud point B; point cloud point A and point cloud point B are jointly marked as a point cloud point pair.

[0114] 8.2. Ninth judgment: By obtaining the pixel point corresponding to the point cloud point A / point cloud point B, and then obtaining the position of the pixel point, it is determined that the point cloud point pair is located at the edge of the bearing.

[0115] 8.3. The eighth extraction: Through statistical analysis, the variation law of the angle between the normal vectors at the edge of the current bearing is: the angle between two adjacent normal vectors at the edge bend differs by 5°.

[0116] 8.4. Ninth extraction: Calculate the angle between two normal vectors: ; ; ; .

[0117] 8.5. The tenth judgment: 39.29°>5°, so the first angle does not conform to the change law, and the point cloud point pair is in the defect area.

[0118] 8.6. Record: Mark the existence of defects at the point pairs in the point cloud.

[0119] When this embodiment finds that there is an angle between two normal vectors, it will extract the corresponding two point cloud points and mark them as a point cloud pair. Afterwards, it is determined whether these point cloud pairs are located at the edge of the bearing. If so, the angle may be caused by the geometric shape of the bearing edge; if not, it may be in a defective area and needs to continue traversing. Afterwards, the change pattern of the normal vector angle at the edge of the bearing is obtained through statistical analysis. Afterwards, the angle (first angle) between the two normal vectors corresponding to the point cloud pair is calculated. Afterwards, it is determined whether this first angle conforms to the previously statistical change pattern. If it conforms, the point cloud pair is considered normal and deleted; if it does not conform, the point cloud pair is marked as defective. This embodiment refines the normal vectors with angles, reducing the error of identifying the normal vector angles at the edge as defects.

[0120] Embodiment 5: This embodiment discloses a surface defect detection system based on vision, and the detection system comprises: The data acquisition and processing module is used to collect historical defect data and bearing data to be processed. This module is also used to build defect labels based on the historical defect data collected by the data acquisition module. These labels will be used as the target output when training the detection model to guide the learning process of the detection model.

[0121] Defect labels include key information such as defect type, location, and size, which will be extracted from historical defect data and organized into a format suitable for model training.

[0122] Add data modules, including network construction unit, data generation unit, data judgment unit, optimization unit and data integration unit.

[0123] The network building unit is used to build a generative adversarial network, wherein the generative adversarial network includes a generator and two discriminators, and the two discriminators are of different categories.

[0124] The data generation unit is used to generate new defect data based on historical defect data and defect labels using the generator constructed in the network construction unit.

[0125] The data judgment unit is used to input new defect data into the two discriminators respectively, and obtain the outputs of the two discriminators respectively. If the output results of the two discriminators are consistent and the same as the expected defect labels, it means that the generated data quality is high and meets the characteristics of the real data, and the modeling module can be triggered; otherwise, the optimization unit is triggered.

[0126] The optimization unit is used to adjust the parameters of the generative adversarial network to improve the quality of the generated data.

[0127] The data integration unit is used to integrate new defect data and collected historical defect data into new historical defect data.

[0128] Modeling module, used to build detection models.

[0129] The model training module is used to train the detection model using historical defect data and obtain the trained detection model.

[0130] The detection module is used to use the bearing data to be processed as the input of the trained detection model to obtain defect detection results.

[0131] The data acquisition and processing module of this embodiment is used to collect historical defect data and construct defect labels based on historical data. The data addition module is used to generate new defect data using a generative adversarial network. The modeling module constructs a detection model including an image segmentation sub-model and a CNN sub-model. The model training module trains the model through the first training unit, the second training unit, the third training unit and the judgment unit until the output result matches the defect label. Finally, the detection module uses the trained model to perform defect detection on the bearing image to be processed, thereby realizing automatic and accurate identification of bearing defects.

[0132] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A surface defect detection method based on vision, characterized in that: include: Data collection and processing: Collect historical defect data and bearing data to be processed, and build defect labels based on the historical defect data; Increase data: including network construction, data generation, data judgment, optimization and data integration; Network construction: establishing a generative adversarial network, wherein the generative adversarial network includes a generator and two discriminators, and the two discriminators are of different categories; Generate data: Based on historical defect data and defect labels, use the generator to generate new defect data; Data judgment: Input the new defect data into the two discriminators respectively, and obtain the outputs of the two discriminators respectively. If the output results of the two discriminators are consistent and the same as the expected defect labels, the data integration step is performed; if not, the optimization step is performed; Optimization: Adjust the parameters of the generative adversarial network and execute the steps of generating data until the preset optimization stop conditions are met; Data integration: new defect data and historical defect data are used as new historical defect data; Modeling: Building a detection model; Model training: Use new historical defect data to train the detection model to obtain the trained detection model; Detection: The bearing data to be processed is used as the input of the trained detection model to obtain the first detection result.

2. The visual-based surface defect detection method according to claim 1, characterized in that: In the step of data collection and processing, a bearing image corresponding to the historical defect data is also collected, the detection model includes: an image segmentation sub-model and a CNN sub-model, and the step of model training also includes: a first training, a second training, a third training and a first judgment; First training: input the bearing image corresponding to the historical defect data into the image segmentation sub-model, and output n sub-images of the target image area; Second training: input the i-th sub-image into the m hidden layers of the CNN sub-model respectively, and obtain m outputs of the hidden layers of the CNN sub-model; Third training: input the m outputs of the hidden layer into the output layer of the CNN sub-model to obtain the second detection result; First judgment: determine whether the second detection result is the same as the defect label. If so, execute the detection step. If not, take the i+1th sub-image as the new i-th sub-image and re-execute the first training step until the preset stop judgment condition is met.

3. The visual-based surface defect detection method according to claim 2, characterized in that: After performing the first training step and before performing the second training step, the method further includes: Image processing: grayscale the n sub-images to obtain n grayscale sub-images; First extraction: obtain the gray value of the jth pixel in the i-th grayscale sub-image, recorded as the first gray value; obtain the gray values ​​of eight pixels adjacent to the jth pixel, recorded as the second gray value; Second extraction: obtaining the maximum and minimum values ​​of the second grayscale value; The third judgment: judging whether the first gray value is greater than the minimum value and less than the maximum value, if so, executing the first replacement step; if not, executing the marking step; First replacement: taking the j+1th pixel point as the new jth pixel point, and executing the first extraction step until the preset first replacement stop condition is met, and executing the second replacement step; Marking: Mark the j-th pixel as a noise point, replace the gray value of the j-th pixel with the average gray value of the eight adjacent pixels, and perform the first replacement step; Second replacement: taking the i+1th grayscale sub-image as the new ith grayscale sub-image, and executing the first extraction step until a preset second replacement stop condition is met, and executing the second training step.

4. The visual-based surface defect detection method according to claim 3, characterized in that: After executing the third judgment step and before executing the marking step, the method further includes: Fourth judgment: judging whether the first gray value is less than the minimum value, if so, executing the first calculation step; if not, executing the second calculation step; First calculation: calculate the difference between the first gray value and the minimum value, recorded as the first difference; Second calculation: calculate the difference between the first gray value and the maximum value, and record it as the second difference; Fifth judgment: judge whether the first difference or the second difference is greater than a preset difference threshold, if so, execute the marking step; if not, execute the first replacement step.

5. The method for detecting surface defects based on vision according to claim 2, characterized in that: After performing the first training step and before performing the second training step, the method further includes: Establish a sub-image set: integrate n sub-images into a sub-image set; The third extraction: using laser radar to obtain the point cloud data corresponding to each pixel in each sub-image, and integrating all the point cloud data of a sub-image into a point cloud data set; Fourth extraction: obtain the normal vector of each point cloud point in the point cloud data set; Sixth judgment: judge whether any two normal vectors are parallel to each other. If so, execute the deletion step; if not, execute the traversal step; Deletion: deleting the sub-image corresponding to the point cloud data set in the sub-image set; Traversal: Traverse all sub-images in the sub-image set, and repeat the steps from the third extraction to the sixth judgment until the preset traversal stop condition is met.

6. The visual-based surface defect detection method according to claim 5, characterized in that: After executing the sixth judgment step and before executing the traversal step, the method further includes: Fifth extraction: obtain the kth normal vector and the k+1th normal vector with an included angle, obtain the point cloud point corresponding to the kth normal vector, and obtain the point cloud point corresponding to the k+1th normal vector; The third calculation: calculate the curvature at two point cloud points respectively, and obtain the point cloud point with the smallest curvature, which is recorded as the first point cloud point; Sixth extraction: obtaining a pixel point corresponding to the first point cloud point, recording it as the first pixel point, and taking the first pixel point as the seed point; Fourth calculation: performing a nearest neighbor search based on the seed point, calculating the angle between the normal vector of the seed point and the normal vector of the neighboring point, and obtaining the first data; Seventh judgment: judging whether the first data is smaller than a preset angle threshold, if so, executing the traversal step; if not, executing the classification step; Classification: Classify neighbor points and seed points into the same defect label; Third replacement: use the k+1th normal vector as the new kth normal vector, and perform the fifth extraction step.

7. The method for detecting surface defects based on vision according to claim 6, characterized in that: After executing the fourth calculation step and before executing the seventh judgment step, the method further includes: Fifth calculation: calculate the curvature of neighboring points; Eighth judgment: judging whether the curvature of the seed point is smaller than the curvature of the neighboring point, if so, executing the seventh judgment; if not, executing the fourth replacement step; Fourth replacement: taking the neighbor point as a new seed point and performing the fourth calculation step.

8. The visual-based surface defect detection method according to any one of claims 5 to 7, characterized in that: After executing the sixth judgment step and before executing the traversal step, the method further includes: Seventh extraction: based on the sixth judgment step, extracting the point cloud points corresponding to the two normal vectors to obtain two point cloud points, and integrating the two point cloud points into a point cloud point pair; Ninth judgment: judge whether the point cloud point pair is located at the edge of the bearing. If so, execute the deletion step; if not, execute the traversal step.

9. The visual-based surface defect detection method according to claim 8, characterized in that: After executing the ninth judgment step and before executing the deletion step, the method further includes: The eighth extraction: Based on the statistical analysis method, obtain the change law of the angle between the normal vectors at the edge of the bearing; Ninth extraction: obtaining the angle between the two normal vectors corresponding to the point cloud point pair, recorded as the first angle; Tenth judgment: judging whether the first angle conforms to the change rule, if so, executing the deletion step; if not, executing the recording step; Record: A defect is marked at the point cloud point pair.

10. A vision-based surface defect detection system, the system being applicable to the method according to any one of claims 1 to 9, characterized in that: include: Data collection and processing module, used to collect historical defect data and bearing data to be processed; It is also used to build defect labels based on historical defect data; Add data modules, including network construction unit, data generation unit, data judgment unit, optimization unit and data integration unit; A network building unit, used to build a generative adversarial network, wherein the generative adversarial network includes a generator and two discriminators, and the two discriminators are of different categories; A data generation unit, used to generate new defect data using a generator based on historical defect data and defect labels; The data judgment unit is used to input new defect data into the two discriminators respectively, and obtain the outputs of the two discriminators respectively. If the output results of the two discriminators are consistent and the same as the expected defect labels, the modeling module is triggered; Otherwise, the optimization unit is triggered; An optimization unit, used to adjust the parameters of the generative adversarial network; A data integration unit, used for taking the new defect data and the historical defect data as new historical defect data; Modeling module, used to build detection models; A model training module is used to train the detection model using new historical defect data to obtain a trained detection model; The detection module is used to use the bearing data to be processed as the input of the trained detection model to obtain a first detection result.

Images