A method, system, device, and medium for detecting flaws on a surface of a workpiece

Abstract

The application discloses a workpiece surface defect detection method, system, device and medium, wherein the method acquires a multi-view data set of a workpiece surface to be detected; and inputs the multi-view data set into a trained defect detection model to obtain a target defect detection result; wherein the trained defect detection model is obtained through the following steps: acquiring a multi-view image training set; performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set; inputting the multi-view image vector set into an initialized defect detection model for multi-level feature capturing to obtain a target multi-view feature map, wherein the feature channel number and the feature resolution of the target multi-view feature map are negatively correlated; and performing parameter updating on the initialized defect detection model according to the target multi-view feature map to obtain the trained defect detection model. The method can effectively improve the accuracy of defect detection on the workpiece surface. The application relates to the technical field of computer vision.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, system, device and medium for detecting defects on the surface of a workpiece. Background Technology

[0002] Industrial defect detection refers to the use of machine vision technology to inspect the surface of industrial products in order to identify and locate surface defects that do not meet quality requirements.

[0003] Currently, traditional industrial defect detection methods are usually based on computer vision technology to detect defects on the surface of industrial products from a single perspective. This method is easily affected by complex environments (such as complex lighting factors, environmental shadows, and reflection interference), resulting in poor accuracy in defect detection.

[0004] Therefore, the problems existing in the current technology still need to be solved and optimized. Summary of the Invention

[0005] To address at least one of the aforementioned technical problems, this application provides a method, system, device, and medium for detecting defects on the surface of a workpiece, wherein the method can effectively improve the accuracy of defect detection on the surface of a workpiece.

[0006] According to a first aspect of this application, a method for detecting defects on the surface of a workpiece is provided, comprising:

[0007] Obtain a multi-view dataset of the surface of the workpiece to be inspected;

[0008] The multi-view dataset is input into the trained defect detection model to detect surface defects and obtain the target defect detection result.

[0009] The trained defect detection model is obtained through the following steps:

[0010] Obtain a multi-view image training set;

[0011] The multi-view image training set is subjected to vector embedding transformation to obtain a multi-view image vector set;

[0012] The multi-view image vector set is input into the initialized defect detection model for multi-level feature capture to obtain the target multi-view feature map. The number of feature channels and feature resolution of the target multi-view feature map are negatively correlated.

[0013] Based on the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model.

[0014] In some embodiments, the vector embedding transformation of the multi-view image training set to obtain a multi-view image vector set includes:

[0015] The multi-view image training set is segmented to obtain a multi-view image segmentation set;

[0016] Linear projection embedding is performed on the multi-view image segmentation set to obtain a multi-view image embedding set;

[0017] The multi-view image embedding set is positionally encoded to obtain the multi-view image vector set.

[0018] In some embodiments, the initialized defect detection model includes a plurality of cascaded multi-view visual transformation modules, each of which is used to perform the following steps:

[0019] Obtain the input features and the first attention parameters of the input features, wherein the input features are a multi-view image vector set or a multi-view intermediate feature map output by the previous multi-view visual conversion module;

[0020] The first attention parameter is downsampled by the interval window parameter to obtain the second attention parameter, and the tensor resolution corresponding to the second attention parameter is smaller than the tensor resolution corresponding to the first attention parameter.

[0021] The second attention parameter is expanded by attention channels to obtain a third attention parameter, wherein the number of channels corresponding to the third attention parameter is greater than the number of channels corresponding to the second attention parameter;

[0022] Based on the third attention parameter, feature attention is calculated on the input features to obtain the multi-view intermediate feature map output by the current multi-view visual conversion module.

[0023] In some embodiments, the step of performing feature attention calculation on the input features based on the third attention parameter to obtain a multi-view intermediate feature map includes:

[0024] Attention calculations are performed on the third attention parameter to obtain the first intermediate attention feature;

[0025] Based on the input features, the first intermediate attention features are summed to obtain the second intermediate attention features;

[0026] The second intermediate attention feature is normalized to obtain the multi-view intermediate feature map.

[0027] In some embodiments, updating the parameters of the initialized defect detection model based on the target multi-view feature map to obtain the trained defect detection model includes:

[0028] Obtain the view dimension features and channel dimension features of the target multi-view feature map;

[0029] Based on the viewpoint dimension features, the channel dimension features are merged to obtain the first dimension features;

[0030] The first dimension feature is reduced in feature space to obtain the second dimension feature;

[0031] Perform feature defect detection on the second dimension feature to obtain the original defect detection result;

[0032] Based on the original defect detection results, the initialized defect detection model is trained to obtain the trained defect detection model.

[0033] In some embodiments, the method further includes:

[0034] Obtain several multi-view intermediate feature maps output by the multi-view visual conversion modules;

[0035] Viewpoint attention weight analysis is performed on all the multi-view intermediate feature maps to obtain the multi-view attention weight corresponding to each of the multi-view intermediate feature maps. Each multi-view attention weight includes several single-view attention weights, and each single-view attention weight corresponds to a different viewpoint dimension.

[0036] Perform single-view attention summation and sorting on all the single-view attention weights to obtain several target view dimensions.

[0037] Based on all the target perspective dimensions, the multi-view dataset is updated to obtain the updated multi-view dataset.

[0038] In some embodiments, the step of performing single-view attention summation and sorting on all the single-view attention weights to obtain several target view dimensions includes:

[0039] All the single-view attention weights are filtered by viewpoint to obtain several single-view attention datasets. Each single-view attention dataset includes several single-view attention weights with the same viewpoint dimension.

[0040] The attention weights of all the single-view attention datasets are summed to obtain the statistical attention value corresponding to each single-view attention dataset.

[0041] All the statistical attention values ​​are sorted by size and filtered to obtain several target attention values. The target attention value is the statistical attention value that ranks first among all the statistical attention values.

[0042] Based on all the target attention values, several target view dimensions are obtained, and each target view dimension corresponds to one target attention value.

[0043] According to a second aspect of this application, a defect detection system for a workpiece surface is provided, comprising:

[0044] The first processing unit is used to acquire a multi-view dataset of the surface of the workpiece to be inspected.

[0045] The second processing unit is used to input the multi-view dataset into the trained defect detection model to perform surface defect detection and obtain the target defect detection result.

[0046] The trained defect detection model is obtained through the following steps:

[0047] Obtain a multi-view image training set;

[0048] The multi-view image training set is subjected to vector embedding transformation to obtain a multi-view image vector set;

[0049] The multi-view image vector set is input into the initialized defect detection model for multi-level feature capture to obtain the target multi-view feature map. The number of feature channels and feature resolution of the target multi-view feature map are negatively correlated.

[0050] Based on the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model.

[0051] According to a third aspect of this application, a computer device is provided, comprising:

[0052] At least one processor;

[0053] At least one memory for storing at least one program;

[0054] When the at least one program is executed by the at least one processor, the at least one processor performs the method as described in any of the above aspects.

[0055] According to a fourth aspect of this application, a computer-readable storage medium is provided, wherein a processor-executable program is stored, which, when executed by the processor, is used to implement the method as described in any of the preceding aspects.

[0056] The beneficial effects of the technical solutions provided in this application are:

[0057] This application provides a method, system, device, and medium for detecting defects on the surface of a workpiece. The method involves acquiring a multi-view dataset of the workpiece surface to be inspected; inputting the multi-view dataset into a trained defect detection model for surface defect detection to obtain a target defect detection result; wherein the trained defect detection model is obtained through the following steps: acquiring a multi-view image training set; performing vector embedding transformation on the multi-view image training set to obtain a multi-view image vector set; inputting the multi-view image vector set into an initialized defect detection model for multi-level feature capture to obtain a target multi-view feature map, wherein the number of feature channels and feature resolution of the target multi-view feature map are negatively correlated; and updating the parameters of the initialized defect detection model based on the target multi-view feature map to obtain the trained defect detection model. This method performs multi-level feature steps on the multi-view image vector set through the defect detection model, obtaining a target multi-view feature map where the number of feature channels and feature resolution change negatively with the level of feature capture. It can fully acquire feature representations of global and local information of the workpiece surface from multiple perspectives, effectively suppressing the influence of complex environments on defect detection, thereby significantly improving the accuracy of defect detection. Attached Figure Description

[0058] Figure 1 A schematic flowchart illustrating a method for detecting defects on the surface of a workpiece, provided in an embodiment of this application;

[0059] Figure 2 A schematic diagram illustrating the training process of a defect detection model provided in an embodiment of this application;

[0060] Figure 3 A detailed flowchart of step S140 provided for an embodiment of this application;

[0061] Figure 4 A flowchart illustrating a multi-view vision conversion module provided in an embodiment of this application;

[0062] Figure 5 A detailed flowchart of step B4 provided for an embodiment of this application;

[0063] Figure 6 A detailed flowchart of step S160 provided for an embodiment of this application;

[0064] Figure 7 This is a schematic flowchart of one optional method for detecting defects on the surface of a workpiece, provided in an embodiment of this application.

[0065] Figure 8 A detailed flowchart of step D3 provided for an embodiment of this application;

[0066] Figure 9A schematic diagram of the framework of a workpiece surface defect detection system provided in an embodiment of this application;

[0067] Figure 10 This is a structural block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0068] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0069] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0071] Currently, traditional industrial defect detection methods are usually based on computer vision technology to detect defects on the surface of industrial products from a single perspective. This method is easily affected by complex environments (such as complex lighting factors, environmental shadows, and reflection interference). For example, for the detection of defects on 2D smooth surfaces, metal surfaces or high-gloss materials may produce strong reflections when lighting conditions change. These reflections can easily cover up surface defects, resulting in poor accuracy of defect detection.

[0072] In addition, for the method of detecting defects on the surface of industrial products through a single view, the lighting method provided during defect detection is relatively fixed. This method lacks the flexibility of defect detection and often cannot adapt to different detection needs and scenarios. It has poor adaptability and flexibility and high application cost.

[0073] In view of this, embodiments of this application provide a method, system, device, and medium for detecting defects on the surface of a workpiece. This method employs a defect detection model to perform multi-level feature steps on a multi-view image vector set. Specifically, each multi-view visual transformation module downsamples and expands the input features, resulting in a negative correlation between the number of feature channels and the feature resolution of the final target multi-view feature map and the level of feature capture. This allows for the full acquisition of feature representations of the global and local information of the workpiece surface from multiple perspectives, effectively suppressing the influence of complex environments on defect detection and significantly improving the accuracy of defect detection. Furthermore, this method performs single-view attention summation and sorting on all single-view attention weights to obtain several image perspectives (i.e., target perspective dimensions) most relevant to the defect features of the workpiece surface. The multi-view dataset is then updated using these target perspective dimensions, reducing the computational burden on the workpiece surface during subsequent defect detection, improving overall detection efficiency, and lowering required computational resources and costs.

[0074] This application provides a method, system, device, and medium for detecting defects on the surface of a workpiece. The specific details can be illustrated through the following embodiments. First, a method for detecting defects on the surface of a workpiece is described in this application.

[0075] The workpiece surface defect detection method provided in this application can be applied to workpiece quality inspection scenarios. In these scenarios, workpiece quality inspection service providers can use the method provided in this application to detect surface defects in the workpiece to be inspected, thereby achieving quality inspection of the workpiece surface and effectively improving the accuracy of defect detection.

[0076] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0077] Reference Figure 1 , Figure 1This is an optional flowchart of a method for detecting defects on the surface of a workpiece provided in an embodiment of this application, which may include, but is not limited to, steps S110 to S120.

[0078] Step S110: Obtain a multi-view dataset of the surface of the workpiece to be inspected;

[0079] Step S120: Input the multi-view dataset into the trained defect detection model to perform surface defect detection and obtain the target defect detection result;

[0080] In this embodiment of the application, the surface of the workpiece to be inspected can be the surface of the industrial product to be inspected, and the multi-view dataset can be an image dataset of the industrial product acquired by a camera device from different view dimensions. The multi-view dataset includes image data from several different view dimensions.

[0081] Understandably, after obtaining the multi-view dataset, it can be input into a trained defect detection model. The defect detection model can then extract feature representations of the multi-view dataset under different view dimensions to achieve surface defect detection and obtain the target defect detection result.

[0082] Among them, reference Figure 2 The trained defect detection model is obtained through the following steps:

[0083] Step S130: Obtain a multi-view image training set;

[0084] In this embodiment of the application, the multi-view image training set can be obtained by collecting historical image data from the industrial product production environment. Specifically, for a certain image data in the multi-view image training set, it can be obtained by taking pictures of the workpiece surface from different shooting angles using an RGB industrial camera. The other image data are obtained in the same way, thus obtaining the multi-view image training set.

[0085] Understandably, after obtaining the original multi-view image training set, each image data in the multi-view image training set can be preprocessed. Specifically, discrete wavelet transform can be performed on each image data to eliminate complex texture noise in the image data, highlight the location of defects on the workpiece surface, and obtain an updated multi-view image training set.

[0086] Step S140: Perform vector embedding transformation on the multi-view image training set to obtain a multi-view image vector set;

[0087] In this embodiment, vector embedding transformation is used to convert each image data in the multi-view image training set into the input format of the defect detection model. Specifically, it can be to convert the multi-view image training set into the corresponding embedding vector to obtain the multi-view image vector set.

[0088] Reference Figure 3 In some embodiments, step S140, performing vector embedding transformation on the multi-view image training set to obtain a multi-view image vector set, includes:

[0089] A1. Perform image segmentation on the multi-view image training set to obtain a multi-view image segmentation set;

[0090] A2. Perform linear projection embedding on the multi-view image segmentation set to obtain a multi-view image embedding set;

[0091] A3. Perform position encoding on the multi-view image embedding set to obtain the multi-view image vector set.

[0092] In this embodiment, step A1 may involve performing image segmentation on each image data in the multi-view image training set to obtain multiple segmentation slices corresponding to each image data, and determining all segmentation slices with different view dimensions as a multi-view image segmentation set; step A2 may involve embedding each segmentation slice into a three-dimensional vector space (M×N×D) through linear projection to obtain a multi-view image embedding set, where M is the number of view dimensions, N is the number of segmentation slices corresponding to each image data, and D is the number of dimensions of the segmentation embedding.

[0093] It is understandable that step A3 can first obtain the position code and camera pose information corresponding to the linear projection of each image in the multi-view image embedding set, and then add the matrix of the linear projection of the image corresponding to the multi-view image embedding set through the ADD operator to obtain the multi-view image vector set.

[0094] Step S150: Input the multi-view image vector set into the initialized defect detection model for multi-level feature capture to obtain the target multi-view feature map. The number of feature channels and feature resolution of the target multi-view feature map are negatively correlated.

[0095] In this embodiment, the defect detection model can be built based on the Transformer deep learning model architecture. The defect detection model can combine multi-view information and multi-scale information to perform defect detection. The multi-view information can provide the defect detection model with the defect performance of the workpiece surface under different lighting and viewing angle dimensions. The multi-scale information can enhance the sensitivity of the defect detection model to defects of different sizes.

[0096] It is understandable that step S150 may involve inputting a multi-view image vector set into an initialized defect detection model for multi-level feature capture to obtain a target multi-view feature map. The number of feature channels and feature resolution of the target multi-view feature map are negatively correlated, specifically, the number of feature channels and feature resolution of the target multi-view feature map change negatively correlated with the number of feature capture levels.

[0097] Reference Figure 4 In some embodiments, the initialized defect detection model includes several cascaded multi-view visual transformation modules, each of which is used to perform the following steps:

[0098] B1. Obtain the input features and the first attention parameters of the input features, wherein the input features are a multi-view image vector set or a multi-view intermediate feature map output by the previous multi-view visual conversion module;

[0099] B2. The first attention parameter is downsampled by the interval window parameter to obtain the second attention parameter, and the tensor resolution corresponding to the second attention parameter is smaller than the tensor resolution corresponding to the first attention parameter.

[0100] B3. Expand the attention channels of the second attention parameter to obtain a third attention parameter, wherein the number of channels corresponding to the third attention parameter is greater than the number of channels corresponding to the second attention parameter;

[0101] In this embodiment, the defect detection model includes several cascaded multi-view visual transformation modules. The "several" can refer to one or more modules. The intermediate multi-view feature map output by the previous multi-view visual transformation module serves as the input feature of the next multi-view visual transformation module. Specifically, if the current multi-view visual transformation module is the first module in the cascade, its input feature can be a set of multi-view image vectors; or, if the current multi-view visual transformation module is the third module in the cascade, its input feature can be the intermediate multi-view feature map output by the second module. Furthermore, the intermediate multi-view feature map output by the last module in the cascade can be the target multi-view feature map in this embodiment.

[0102] Understandably, the first attention parameter of the input feature can be the query vector, key vector, and value vector corresponding to the input feature. Step B2 can involve tensor reduction of the query vector, key vector, and value vector in the first attention parameter, specifically by reducing the tensor resolution through interval window downsampling, thereby reducing the overall length of the input sequence. Specifically, the query vector, key vector, and value vector can be downsampled using an interval parameter. For example, if the interval parameter is 2, the query vector, key vector, and value vector can be sampled every two sampling pixels, while the missed sampling pixels are saved as other data channels, thus obtaining the second attention parameter. The tensor resolution corresponding to the second attention parameter is smaller than the tensor resolution corresponding to the first attention parameter.

[0103] It should be noted that step B3 can use a multi-head attention mechanism to expand the channels of the query vector, key vector and value vector in the second attention parameter, thereby obtaining a third attention parameter with more channels. Each channel of the third attention parameter corresponds to an image feature at a resolution.

[0104] B4. Based on the third attention parameter, perform feature attention calculation on the input features to obtain the multi-view intermediate feature map output by the current multi-view visual conversion module.

[0105] Reference Figure 5 Further, step B4, calculating feature attention on the input features based on the third attention parameter to obtain a multi-view intermediate feature map, includes:

[0106] B41. Perform attention calculation on the third attention parameter to obtain the first intermediate attention feature;

[0107] B42. Based on the input features, the first intermediate attention features are summed to obtain the second intermediate attention features;

[0108] B43. Normalize the second intermediate attention feature to obtain the multi-view intermediate feature map.

[0109] In this embodiment of the application, the attention calculation in step B41 can first be the dot product of the query vector in the third attention parameter with all key vectors to obtain the corresponding dot product result; then, the dot product result is converted into attention weights in the range of [0, 1] by a normalization function (such as the SoftMax function), and the attention weights are weighted and summed with the value vector to obtain the attention feature representation (i.e. the first intermediate attention feature) corresponding to the input feature.

[0110] Understandably, after obtaining the first intermediate attention feature, the first intermediate attention feature and the input feature can be added together to prevent the loss of the initial feature, thereby obtaining the second intermediate attention feature. Then, the second intermediate attention feature is normalized to eliminate the difference in the units between features, thereby accelerating the convergence speed of the defect detection model, and thus obtaining the multi-view intermediate feature map output by the current multi-view visual conversion module.

[0111] S160. Based on the target multi-view feature map, update the parameters of the initialized defect detection model to obtain the trained defect detection model.

[0112] In this embodiment of the application, step S160 may first involve inputting the target multi-view feature map into a classifier for classification and detection, thereby obtaining the classification and detection result corresponding to the multi-view image training set. Then, based on the classification and detection result, the defect detection model is updated with parameters through the backpropagation algorithm to obtain the trained defect detection model.

[0113] Reference Figure 6 In some embodiments, step S160, updating the parameters of the initialized defect detection model based on the target multi-view feature map to obtain the trained defect detection model, includes:

[0114] C1. Obtain the view dimension features and channel dimension features of the target multi-view feature map;

[0115] C2. Based on the perspective dimension features, the channel dimension features are merged to obtain the first dimension features;

[0116] C3. Reduce the feature space of the first dimension feature to obtain the second dimension feature;

[0117] C4. Perform feature defect detection on the second dimension feature to obtain the original defect detection result;

[0118] C5. Based on the original defect detection results, train the initialized defect detection model to obtain the trained defect detection model.

[0119] In this embodiment, step C1 may decompose the target multi-view feature map to obtain visual dimension features, channel dimension features and spatial dimension features; step C2 may merge and splice the visual dimension features and channel dimension features from different viewpoints to obtain the first dimension feature; then, the first dimension feature is averaged in the spatial dimension (such as height and width) to reduce the input feature map corresponding to the first dimension feature in the spatial dimension to a single value to obtain the second dimension feature.

[0120] Understandably, after obtaining the second-dimensional features, these features can be input into a classifier, specifically a multilayer perceptron head, to perform multi-classification, thereby obtaining the original defect detection results. Furthermore, before the defect detection model is deployed, it needs to be trained to adjust its internal parameters and achieve better detection performance. Specifically, during model training, a batch of multi-view images can be acquired. Each multi-view image includes multi-view image data of the workpiece surface and the corresponding workpiece surface defect label. Then, each multi-view image and its corresponding label can be used as a set of training data. The model's input data is the multi-view images, and the model predicts from these images, outputting the original defect detection results. After obtaining the original defect detection results output by the model, the accuracy of the model's predictions can be evaluated based on the original defect detection results and the corresponding workpiece surface defect labels, thereby updating the model's parameters.

[0121] Specifically, for machine learning models, the accuracy of the model's prediction results can be measured by a loss function. The loss function is defined on a single training data point and measures the prediction error of that data point. Specifically, the loss value of that training data point is determined by the label of that individual training data point and the model's detection result for that data. However, in actual training, a training dataset contains many data points. Therefore, a cost function is generally used to measure the overall error of the training dataset. The cost function is defined on the entire training dataset and calculates the average prediction error of all training data points, providing a better measure of the model's prediction performance. For general machine learning models, the aforementioned cost function, plus a regularization term to measure model complexity, can serve as the training objective function. Based on this objective function, the loss value of the entire training dataset can be calculated. Many types of loss functions are commonly used, such as 0-1 loss, squared loss, absolute loss, logarithmic loss, and cross-entropy loss, which can all be used as loss functions for machine learning models. These will not be elaborated upon here. In this embodiment, any one of these loss functions can be selected to determine the training loss value, such as the cross-entropy loss function. Based on the training loss value, the backpropagation algorithm is used to update the model parameters. After several iterations, a trained defect detection model can be obtained. The specific number of iterations can be preset, or training can be considered complete when the accuracy requirement is met on the test set.

[0122] Reference Figure 7 In some embodiments, the method further includes:

[0123] D1. Obtain several multi-view intermediate feature maps output by the multi-view visual conversion modules;

[0124] D2. Perform view attention weight analysis on all the multi-view intermediate feature maps to obtain the multi-view attention weight corresponding to each of the multi-view intermediate feature maps. Each of the multi-view attention weights includes several single-view attention weights, and each single-view attention weight corresponds to a different view dimension.

[0125] In this embodiment, an attention mechanism for multi-view filtering can be inserted at the end of each multi-view visual transformation module. Specifically, for a multi-view intermediate feature map output by a certain multi-view visual transformation module, the attention mechanism for multi-view filtering can first perform global max pooling and global average pooling operations on the multi-view intermediate feature map in each view dimension, calculate the maximum eigenvalue and average eigenvalue of the multi-view intermediate feature map in each view dimension, and concatenate them into a maximum eigenvector and an average eigenvector, respectively. Then, the maximum eigenvector and the average eigenvector are input into a shared fully connected layer for learning the attention weights for each view, and the Sigmoid activation function is applied to produce multi-view attention weights corresponding to the multi-view intermediate feature map. The multi-view attention weights include several single-view attention weights. The same applies to the multi-view intermediate feature maps output by other multi-view visual transformation modules, which can be easily deduced by analogy. This application will not elaborate further here.

[0126] D3. Perform single-view attention summation and sorting on all the single-view attention weights to obtain several target view dimensions.

[0127] Reference Figure 8 Furthermore, in step D3, the single-view attention weights are summed, statistically analyzed, and sorted to obtain several target view dimensions, including:

[0128] D31. Perform perspective filtering on all the single-view attention weights to obtain several single-view attention datasets. Each single-view attention dataset includes several single-view attention weights with the same perspective dimension.

[0129] D32. Perform attention weight summation and statistical analysis on all the single-view attention datasets to obtain the statistical attention value corresponding to each single-view attention dataset.

[0130] D33. Sort and filter all the statistical attention values ​​by size to obtain several target attention values. The target attention value is the statistical attention value that ranks first among all the statistical attention values.

[0131] D34. Based on all the target attention values, several target view dimensions are obtained, and each target view dimension corresponds to one target attention value.

[0132] In this embodiment of the application, step D31 can be based on the view dimension as a filtering condition to filter single-view attention weights with the same view dimension into the same single-view attention dataset; then, for a certain single-view attention dataset, all single-view attention weights in the single-view attention dataset can be summed to obtain the statistical attention value corresponding to the single-view attention dataset, and the same applies to other single-view attention datasets.

[0133] It is understandable that after obtaining all statistical attention values, they can be sorted by size to obtain several target attention values. Specifically, in the embodiments of this application, the top three statistical attention values ​​in the size sort can be used as target attention values, and then the view dimension corresponding to each target attention can be determined as the target view dimension.

[0134] D4. Based on all the target perspective dimensions, update the data perspective of the multi-view dataset to obtain the updated multi-view dataset.

[0135] In this embodiment of the application, step D4 can be based on the target view dimension to update the data view of the multi-view dataset, and obtain the updated multi-view dataset. The view dimension corresponding to the updated multi-view dataset is the target view dimension, so that the subsequent input of the defect detection model is the image data corresponding to the target view dimension. It can dynamically select the most suitable view for the workpiece to be inspected, ensuring that the subsequent defect detection can achieve the best effect, while also reducing the required computing resources and reducing costs.

[0136] Figure 9 A schematic diagram of the framework of a workpiece surface defect detection system provided in this application embodiment includes:

[0137] The first processing unit 810 is used to acquire a multi-view dataset of the surface of the workpiece to be inspected.

[0138] The second processing unit 820 is used to input the multi-view dataset into the trained defect detection model to perform surface defect detection and obtain the target defect detection result.

[0139] The trained defect detection model is obtained through the following steps:

[0140] Obtain a multi-view image training set;

[0141] The multi-view image training set is subjected to vector embedding transformation to obtain a multi-view image vector set;

[0142] The multi-view image vector set is input into the initialized defect detection model for multi-level feature capture to obtain the target multi-view feature map. The number of feature channels and feature resolution of the target multi-view feature map are negatively correlated.

[0143] Based on the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model.

[0144] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0145] Figure 10 A schematic diagram of the structure of a computer device provided in this application embodiment includes:

[0146] At least one processor 980;

[0147] At least one memory 920 is used to store at least one program;

[0148] When the at least one program is executed by the at least one processor 980, the at least one processor 980 performs the method as described in the foregoing embodiments.

[0149] This application also provides a computer-readable storage medium storing a processor-executable program, which, when executed by the processor 980, is used to implement the methods described in the foregoing embodiments.

[0150] Figure 10 Specifically, computer equipment can be either a user terminal or a server.

[0151] This application uses a computer device as a user terminal as an example, as detailed below:

[0152] like Figure 10 As shown, the computer device 900 may include an RF (Radio Frequency) circuit 910, a memory 920 including one or more computer-readable storage media, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a WiFi module 970, a processor 980 including one or more processing cores, and a power supply 990, among other components. Those skilled in the art will understand that... Figure 10 The device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0153] The RF circuit 910 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and hands it over to one or more processors 980 for processing; additionally, it transmits uplink data to the base station. Typically, the RF circuit 910 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, etc. Furthermore, the RF circuit 910 can also communicate wirelessly with networks and other devices. Wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.

[0154] The memory 920 can be used to store software programs and modules. The processor 980 executes various functional applications and data processing by running the software programs and modules stored in the memory 920. The memory 920 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the device 900 (such as audio data, telephone directory, etc.). In addition, the memory 920 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 920 may also include a memory controller to provide access to the memory 920 for the processor 980 and the input unit 930. Although Figure 10 The RF circuit 910 is shown, but it is understood that it is not a necessary component of the device 900 and can be omitted as needed without changing the nature of the invention.

[0155] The input unit 930 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, the input unit 930 may include a touch-sensitive surface 932 and other input devices 931. The touch-sensitive surface 932, also known as a touch display screen or touchpad, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface 932), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface 932 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 980, and can receive and execute commands from the processor 980. In addition, the touch-sensitive surface 932 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 932, the input unit 930 may also include other input devices 931. Specifically, other input devices 931 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0156] Display unit 940 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of device 900. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Display unit 940 may include display panel 941, optionally configured as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, touch-sensitive surface 932 may cover display panel 941. When touch-sensitive surface 932 detects a touch operation on or near it, it transmits the information to processor 980 to determine the type of touch event. Subsequently, processor 980 provides corresponding visual output on display panel 941 according to the type of touch event. Although in Figure 10 In this embodiment, the touch-sensitive surface 932 and the display panel 941 are implemented as two separate components to realize input and output functions. However, in some embodiments, the touch-sensitive surface 932 and the display panel 941 can be integrated to realize input and output functions.

[0157] The computer device 900 may also include at least one sensor 950, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 941 according to the ambient light level, and the proximity sensor can turn off the display panel 941 and / or backlight when the device 900 is moved to the ear. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometers, taps), etc. Other sensors that the device 900 may be equipped with, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0158] Audio circuitry 960, speaker 961, and microphone 962 provide an audio interface between the user and device 900. Audio circuitry 960 converts received audio data into electrical signals, which are then transmitted to speaker 961, where they are converted into sound signals for output. Conversely, microphone 962 converts collected sound signals into electrical signals, which are received by audio circuitry 960, converted back into audio data, and then processed by processor 980 before being transmitted via RF circuitry 910 to another control device, or output to memory 920 for further processing. Audio circuitry 960 may also include an earphone jack to facilitate communication between peripheral headphones and device 900.

[0159] Device 900 can transmit information with the wireless transmission module set up on the battle equipment via WiFi module 970.

[0160] The processor 980 is the control center of the device 900. It connects various parts of the control device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 920, and by calling data stored in the memory 920, it performs various functions of the device 900 and processes data, thereby providing overall monitoring of the control device. Optionally, the processor 980 may include one or more processing cores; optionally, the processor 980 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 980.

[0161] The device 900 also includes a power supply 990 (such as a battery) that supplies power to the various components. Preferably, the power supply can be logically connected to the processor 980 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 990 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0162] Although not shown, device 900 may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0163] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the methods described in the foregoing embodiments.

[0164] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0165] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0166] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0169] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0170] The step numbers in the above method embodiments are set only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0171] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for detecting defects on the surface of a workpiece, characterized in that, include: Obtain a multi-view dataset of the surface of the workpiece to be inspected; The multi-view dataset is input into the trained defect detection model to detect surface defects and obtain the target defect detection result. The trained defect detection model is obtained through the following steps: Obtain a multi-view image training set; The multi-view image training set is subjected to vector embedding transformation to obtain a multi-view image vector set; The multi-view image vector set is input into the initialized defect detection model for multi-level feature capture to obtain the target multi-view feature map. The number of feature channels and feature resolution of the target multi-view feature map are negatively correlated. Based on the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model; The initialized defect detection model includes several cascaded multi-view vision conversion modules, each of which is used to perform the following steps: Obtain the input features and the first attention parameters of the input features, wherein the input features are a multi-view image vector set or a multi-view intermediate feature map output by the previous multi-view visual conversion module; The first attention parameter is downsampled by the interval window parameter to obtain the second attention parameter, and the tensor resolution corresponding to the second attention parameter is smaller than the tensor resolution corresponding to the first attention parameter. The second attention parameter is expanded by attention channels to obtain a third attention parameter, wherein the number of channels corresponding to the third attention parameter is greater than the number of channels corresponding to the second attention parameter; Based on the third attention parameter, feature attention is calculated on the input features to obtain the multi-view intermediate feature map output by the current multi-view visual conversion module.

2. The defect detection method according to claim 1, characterized in that, The step of performing vector embedding transformation on the multi-view image training set to obtain a multi-view image vector set includes: The multi-view image training set is segmented to obtain a multi-view image segmentation set; Linear projection embedding is performed on the multi-view image segmentation set to obtain a multi-view image embedding set; The multi-view image embedding set is positionally encoded to obtain the multi-view image vector set.

3. The defect detection method according to claim 1, characterized in that, The step of performing feature attention calculation on the input features based on the third attention parameter to obtain a multi-view intermediate feature map includes: Attention calculations are performed on the third attention parameter to obtain the first intermediate attention feature; Based on the input features, the first intermediate attention features are summed to obtain the second intermediate attention features; The second intermediate attention feature is normalized to obtain the multi-view intermediate feature map.

4. The defect detection method according to claim 1, characterized in that, The step of updating the parameters of the initialized defect detection model based on the target multi-view feature map to obtain the trained defect detection model includes: Obtain the view dimension features and channel dimension features of the target multi-view feature map; Based on the viewpoint dimension features, the channel dimension features are merged to obtain the first dimension features; The first dimension feature is reduced in feature space to obtain the second dimension feature; Perform feature defect detection on the second dimension feature to obtain the original defect detection result; Based on the original defect detection results, the initialized defect detection model is trained to obtain the trained defect detection model.

5. The defect detection method according to claim 1, characterized in that, The method further includes: Obtain several multi-view intermediate feature maps output by the multi-view visual conversion modules; Viewpoint attention weight analysis is performed on all the multi-view intermediate feature maps to obtain the multi-view attention weight corresponding to each of the multi-view intermediate feature maps. Each multi-view attention weight includes several single-view attention weights, and each single-view attention weight corresponds to a different viewpoint dimension. Perform single-view attention summation and sorting on all the single-view attention weights to obtain several target view dimensions. Based on all the target perspective dimensions, the multi-view dataset is updated to obtain the updated multi-view dataset.

6. The defect detection method according to claim 5, characterized in that, The process of summing, sorting, and filtering all the single-view attention weights yields several target view dimensions, including: All the single-view attention weights are filtered by viewpoint to obtain several single-view attention datasets. Each single-view attention dataset includes several single-view attention weights with the same viewpoint dimension. The attention weights of all the single-view attention datasets are summed to obtain the statistical attention value corresponding to each single-view attention dataset. All the statistical attention values ​​are sorted by size and filtered to obtain several target attention values. The target attention value is the statistical attention value that ranks first among all the statistical attention values. Based on all the target attention values, several target view dimensions are obtained, and each target view dimension corresponds to one target attention value.

7. A defect detection system for the surface of a workpiece, characterized in that, include: The first processing unit is used to acquire a multi-view dataset of the surface of the workpiece to be inspected. The second processing unit is used to input the multi-view dataset into the trained defect detection model to perform surface defect detection and obtain the target defect detection result. The trained defect detection model is obtained through the following steps: Obtain a multi-view image training set; The multi-view image training set is subjected to vector embedding transformation to obtain a multi-view image vector set; The multi-view image vector set is input into the initialized defect detection model for multi-level feature capture to obtain the target multi-view feature map. The number of feature channels and feature resolution of the target multi-view feature map are negatively correlated. Based on the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model; The initialized defect detection model includes several cascaded multi-view vision conversion modules, each of which is used to perform the following steps: Obtain the input features and the first attention parameters of the input features, wherein the input features are a multi-view image vector set or a multi-view intermediate feature map output by the previous multi-view visual conversion module; The first attention parameter is downsampled by the interval window parameter to obtain the second attention parameter, and the tensor resolution corresponding to the second attention parameter is smaller than the tensor resolution corresponding to the first attention parameter. The second attention parameter is expanded by attention channels to obtain a third attention parameter, wherein the number of channels corresponding to the third attention parameter is greater than the number of channels corresponding to the second attention parameter; Based on the third attention parameter, feature attention is calculated on the input features to obtain the multi-view intermediate feature map output by the current multi-view visual conversion module.

8. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images