Workpiece surface flaw detection method, system, equipment and medium

Abstract

The invention discloses a flaw detection method, system and device for a workpiece surface and a medium, and the method comprises the steps: obtaining a multi-view data set of the surface of a to-be-detected workpiece; inputting the multi-view data set into a trained flaw detection model to obtain a target flaw detection result; wherein the trained flaw detection model is obtained through the following steps: obtaining a multi-view image training set; performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set; the multi-view image vector set is input into an initialized flaw detection model for multi-level feature capture, a target multi-view feature map is obtained, and the number of feature channels of the target multi-view feature map is in negative correlation with the feature resolution; and according to the target multi-view feature map, performing parameter updating on the initialized flaw detection model to obtain a trained flaw detection model. The method can effectively improve the accuracy of flaw detection on the surface of the workpiece. The invention relates to the technical field of computer vision.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, system, device and medium for detecting defects on the surface of a workpiece. Background Art

[0002] Industrial defect detection refers to the inspection of the surface of industrial products through machine vision technology to identify and locate surface defects that do not meet quality requirements.

[0003] At present, traditional industrial defect detection methods are usually based on computer vision technology to perform defect detection on the surface of industrial products from a single perspective. This method is easily affected by complex environments (such as complex lighting factors, environmental shadows, reflection interference, etc.), and the accuracy of defect detection is poor.

[0004] Therefore, the problems existing in the prior art still need to be solved and optimized. Summary of the invention

[0005] In order to solve at least one of the above-mentioned technical problems, the present application provides a method, system, device and medium for flaw detection on the surface of a workpiece, wherein the method can effectively improve the accuracy of flaw detection on the surface of the workpiece.

[0006] According to a first aspect of the present application, a method for detecting defects on a workpiece surface is provided, comprising:

[0007] Acquire a multi-view data set of the surface of the workpiece to be inspected;

[0008] Inputting the multi-view data set into a trained defect detection model to perform surface defect detection to obtain a target defect detection result;

[0009] The trained defect detection model is obtained by the following steps:

[0010] Obtain a multi-view image training set;

[0011] Performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set;

[0012] Inputting the multi-view image vector set into an initialized defect detection model for multi-level feature capture to obtain a target multi-view feature map, wherein the number of feature channels and feature resolution of the target multi-view feature map are negatively correlated;

[0013] According to the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model.

[0014] In some embodiments, performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set includes:

[0015] Performing image segmentation on the multi-view image training set to obtain a multi-view image segmentation set;

[0016] Performing linear projection embedding on the multi-view image segmentation set to obtain a multi-view image embedding set;

[0017] Position encoding is performed on the multi-view image embedding set to obtain the multi-view image vector set.

[0018] In some embodiments, the initialized defect detection model includes a plurality of cascaded multi-view visual conversion modules, each of which is used to perform the following steps:

[0019] Obtaining an input feature and a first attention parameter of the input feature, wherein the input feature is a multi-view image vector set or a multi-view intermediate feature map output by a previous multi-view visual conversion module;

[0020] Downsampling the first attention parameter by an interval window parameter to obtain a second attention parameter, wherein a tensor resolution corresponding to the second attention parameter is smaller than a tensor resolution corresponding to the first attention parameter;

[0021] Performing attention channel expansion on the second attention parameter to obtain a third attention parameter, wherein the number of channels corresponding to the third attention parameter is greater than the number of channels corresponding to the second attention parameter;

[0022] According to the third attention parameter, feature attention calculation is performed on the input feature to obtain a multi-view intermediate feature map output by the current multi-view visual conversion module.

[0023] In some embodiments, performing feature attention calculation on the input feature according to the third attention parameter to obtain a multi-view intermediate feature map includes:

[0024] Performing attention calculation on the third attention parameter to obtain a first intermediate attention feature;

[0025] According to the input feature, adding the first intermediate attention feature to obtain a second intermediate attention feature;

[0026] The second intermediate attention feature is normalized to obtain the multi-view intermediate feature map.

[0027] In some embodiments, the updating of parameters of the initialized defect detection model according to the target multi-view feature map to obtain the trained defect detection model includes:

[0028] Obtaining the view dimension features and channel dimension features of the target multi-view feature map;

[0029] According to the viewing angle dimension feature, the channel dimension feature is combined to obtain a first dimension feature;

[0030] Performing feature space reduction on the first dimensional features to obtain second dimensional features;

[0031] Performing feature defect detection on the second dimensional feature to obtain an original defect detection result;

[0032] According to the original defect detection result, the initialized defect detection model is trained to obtain the trained defect detection model.

[0033] In some embodiments, the method further comprises:

[0034] Obtaining a plurality of multi-view intermediate feature maps output by the multi-view visual conversion modules;

[0035] Performing a perspective attention weight analysis on all the multi-perspective intermediate feature maps to obtain a multi-perspective attention weight corresponding to each of the multi-perspective intermediate feature maps, wherein each of the multi-perspective attention weights includes a plurality of single-perspective attention weights, and each single-perspective attention weight corresponds to a different perspective dimension;

[0036] Performing single-view attention summing statistics and sorting screening on all the single-view attention weights to obtain a plurality of target view dimensions;

[0037] According to all the target perspective dimensions, the multi-perspective dataset is updated to obtain an updated multi-perspective dataset.

[0038] In some embodiments, the single-view attention weights are summed and counted and sorted to obtain several target view dimensions, including:

[0039] Performing perspective screening on all the single-view attention weights to obtain a plurality of single-view attention data sets, each of the single-view attention data sets including a plurality of single-view attention weights with the same perspective dimension;

[0040] Performing attention weight summing statistics on all the single-view attention data sets to obtain a statistical attention value corresponding to each single-view attention data set;

[0041] Sorting and screening all the statistical attention values ​​to obtain a plurality of target attention values, wherein the target attention value is a statistical attention value that is ranked higher than all the statistical attention values;

[0042] According to all the target attention values, a plurality of target viewing angle dimensions are obtained, and each target viewing angle dimension corresponds to one target attention value.

[0043] According to a second aspect of the present application, a system for detecting defects on a workpiece surface is provided, comprising:

[0044] A first processing unit is used to obtain a multi-view data set of the surface of the workpiece to be inspected;

[0045] A second processing unit is used to input the multi-view data set into a trained defect detection model to perform surface defect detection and obtain a target defect detection result;

[0046] The trained defect detection model is obtained by the following steps:

[0047] Obtain a multi-view image training set;

[0048] Performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set;

[0049] Inputting the multi-view image vector set into an initialized defect detection model for multi-level feature capture to obtain a target multi-view feature map, wherein the number of feature channels and feature resolution of the target multi-view feature map are negatively correlated;

[0050] According to the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model.

[0051] According to a third aspect of the present application, a computer device is provided, comprising:

[0052] at least one processor;

[0053] at least one memory for storing at least one program;

[0054] When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of the above aspects.

[0055] According to a fourth aspect of the present application, a computer-readable storage medium is provided, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the method described in any of the above aspects.

[0056] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0057] The present application provides a method, system, device and medium for flaw detection on the surface of a workpiece, wherein the method obtains a multi-view data set of the surface of the workpiece to be detected; inputs the multi-view data set into a trained flaw detection model to perform surface flaw detection, and obtains a target flaw detection result; wherein the trained flaw detection model is obtained by the following steps: obtaining a multi-view image training set; performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set; inputting the multi-view image vector set into an initialized flaw detection model to perform multi-level feature capture, and obtain a target multi-view feature map, wherein the number of feature channels and the feature resolution of the target multi-view feature map are negatively correlated; according to the target multi-view feature map, the parameters of the initialized flaw detection model are updated to obtain the trained flaw detection model. The method performs multi-level feature steps on the multi-view image vector set through the flaw detection model, and obtains a target multi-view feature map in which the number of feature channels and the feature resolution change negatively with the level of feature capture, which can fully obtain the feature representation of the global and local information of the workpiece surface under multiple perspectives, and can effectively suppress the influence of complex environment on flaw detection, thereby significantly improving the accuracy of flaw detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a flow chart of a method for detecting defects on a workpiece surface provided in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of a training process of a defect detection model provided in an embodiment of the present application;

[0060] Figure 3 A detailed flow chart of step S140 provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of a flow chart of a multi-view visual conversion module provided in an embodiment of the present application;

[0062] Figure 5 A detailed flow chart of step B4 provided in an embodiment of the present application;

[0063] Figure 6 A detailed flow chart of step S160 provided in an embodiment of the present application;

[0064] Figure 7 A schematic diagram of an optional flow chart of a method for detecting defects on a workpiece surface provided in an embodiment of the present application;

[0065] Figure 8 A detailed flow chart of step D3 provided in an embodiment of the present application;

[0066] Fig. 9A schematic diagram of a system for detecting defects on a workpiece surface provided in an embodiment of the present application;

[0067] Fig.10 A structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0069] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0071] At present, traditional industrial defect detection methods are usually based on computer vision technology to perform defect detection on the surface of industrial products from a single perspective. This method is easily affected by complex environments (such as complex lighting factors, environmental shadows, reflection interference, etc.). For example, for 2D smooth surface defect detection, metal surfaces or high-gloss materials may produce strong reflections when lighting conditions change. This reflection can easily cover up surface defects, resulting in poor accuracy of defect detection.

[0072] In addition, for the method of defect detection on the surface of industrial products through a single perspective, the lighting method provided during defect detection is relatively rigid. This method lacks flexibility in defect detection and often cannot adapt to different detection needs and scenarios. It has poor adaptability and flexibility and high application costs.

[0073] In view of this, the embodiment of the present application provides a method, system, device and medium for defect detection on the surface of a workpiece, wherein the method performs multi-level feature steps on a multi-view image vector set through a defect detection model, and specifically downsamples and expands the input features through each multi-view visual conversion module, which can make the number of feature channels and feature resolution of the target multi-view feature map obtained in the end negatively correlated with the level of feature capture, and can fully obtain the feature representation of the global and local information of the workpiece surface under multiple perspectives, and can effectively suppress the influence of complex environments on defect detection, thereby significantly improving the accuracy of defect detection. In addition, the method also obtains several image perspectives (i.e., target perspective dimensions) that are most relevant to the defect features of the workpiece surface by summing and sorting all single-view attention weights, and then updates the multi-view data set through the obtained target perspective dimensions, which can reduce the computational burden of the workpiece surface in the subsequent defect detection process, improve the overall detection efficiency of defect detection, and reduce the required computing resources and costs.

[0074] A method, system, device and medium for detecting defects on the surface of a workpiece provided in an embodiment of the present application can be specifically described through the following embodiments. First, a method for detecting defects on the surface of a workpiece in an embodiment of the present application is described.

[0075] The method for detecting defects on the surface of a workpiece provided in the embodiment of the present application can be applied to the application scenario of workpiece quality inspection. In the application scenario of workpiece quality inspection, the user of the workpiece quality inspection service can perform surface defect detection on the workpiece to be inspected by the method provided in the embodiment of the present application, thereby realizing the quality inspection of the workpiece surface, which can effectively improve the accuracy of defect detection.

[0076] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0077] Reference Figure 1 , Figure 1It is an optional flow chart of a method for detecting defects on a workpiece surface provided in an embodiment of the present application, which may include but is not limited to steps S110 to S120.

[0078] Step S110, obtaining a multi-view data set of the surface of the workpiece to be inspected;

[0079] Step S120, inputting the multi-view data set into a trained defect detection model to perform surface defect detection to obtain a target defect detection result;

[0080] In an embodiment of the present application, the surface of the workpiece to be inspected may be the surface of an industrial product to be inspected, and the multi-perspective data set may be an image data set of the industrial product at different perspective dimensions acquired by a camera device, and the multi-perspective data set includes image data of several different perspective dimensions.

[0081] It can be understood that after obtaining the multi-view dataset, the multi-view dataset can be input into a trained defect detection model, and the defect detection model can be used to extract the feature representation of the multi-view dataset in different view dimensions, so as to realize surface defect detection and obtain the target defect detection result.

[0082] Among them, refer to Figure 2 , the trained defect detection model is obtained by the following steps:

[0083] Step S130, obtaining a multi-view image training set;

[0084] In the embodiment of the present application, the multi-view image training set can be obtained by collecting historical image data in the industrial product production environment. Specifically, for a certain image data in the multi-view image training set, it can be obtained by shooting the workpiece surface at different shooting angles using an RGB industrial camera, and the rest of the image data is similar, thereby obtaining the multi-view image training set.

[0085] It can be understood that after obtaining the original multi-view image training set, each image data in the multi-view image training set can be preprocessed. Specifically, each image data can be subjected to discrete wavelet transform to eliminate the complex texture noise in the image data, highlight the defect position on the workpiece surface, and obtain an updated multi-view image training set.

[0086] Step S140, performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set;

[0087] In an embodiment of the present application, the vector embedding conversion is used to convert each image data in the multi-view image training set into the input format of the defect detection model, which can specifically be to convert the multi-view image training set into a corresponding embedding vector to obtain a multi-view image vector set.

[0088] Reference Figure 3 In some embodiments, the step S140 of performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set includes:

[0089] A1. performing image segmentation on the multi-view image training set to obtain a multi-view image segmentation set;

[0090] A2. Perform linear projection embedding on the multi-view image segmentation set to obtain a multi-view image embedding set;

[0091] A3. Position encoding is performed on the multi-view image embedding set to obtain the multi-view image vector set.

[0092] In an embodiment of the present application, step A1 may be to perform image segmentation on each image data in a multi-view image training set to obtain multiple segmentation slices corresponding to each image data, and determine the segmentation slices of all different view dimensions as a multi-view image segmentation set; step A2 may be to embed each segmentation slice into a three-dimensional vector space (M×N×D) through linear projection to obtain a multi-view image embedding set, wherein M is the number of view dimensions, N is the number of segmentation slices corresponding to each image data, and D is the number of dimensions of the segmentation embedding.

[0093] It can be understood that step A3 can first obtain the position code and camera posture information corresponding to the linear projection of each image in the multi-view image embedding set, and then perform matrix addition on the linear projection of the image corresponding to the multi-view image embedding set through the ADD operator to obtain the multi-view image vector set.

[0094] Step S150, inputting the multi-view image vector set into an initialized defect detection model to perform multi-level feature capture, and obtaining a target multi-view feature map, wherein the number of feature channels and feature resolution of the target multi-view feature map are negatively correlated;

[0095] In an embodiment of the present application, a defect detection model can be established based on the Transformer deep learning model architecture. The defect detection model can combine multi-perspective information and multi-scale information for defect detection. The multi-perspective information can provide the defect detection model with the defect performance of the workpiece surface under different lighting and perspective dimensions. The multi-scale information can enhance the sensitivity of the defect detection model to defects of different sizes.

[0096] It can be understood that step S150 can be to input the multi-view image vector set into the initialized defect detection model for multi-level feature capture, so as to obtain a target multi-view feature map, and the number of feature channels and feature resolution of the target multi-view feature map are negatively correlated, specifically, the number of feature channels and feature resolution of the target multi-view feature map change negatively correlated with the level of feature capture.

[0097] Reference Figure 4 In some embodiments, the initialized defect detection model includes a plurality of cascaded multi-view visual conversion modules, each of which is used to perform the following steps:

[0098] B1. Obtain input features and first attention parameters of the input features, wherein the input features are a multi-view image vector set or a multi-view intermediate feature map output by a previous multi-view visual conversion module;

[0099] B2. Downsampling the first attention parameter by an interval window parameter to obtain a second attention parameter, wherein the tensor resolution corresponding to the second attention parameter is smaller than the tensor resolution corresponding to the first attention parameter;

[0100] B3. performing attention channel expansion on the second attention parameter to obtain a third attention parameter, wherein the number of channels corresponding to the third attention parameter is greater than the number of channels corresponding to the second attention parameter;

[0101] In an embodiment of the present application, the defect detection model includes a plurality of cascaded multi-view visual conversion modules, the number referred to by the plurality may be one or more, and the multi-view intermediate feature map output by the previous multi-view visual conversion module serves as the input feature of the next multi-view visual conversion module. Specifically, if the current multi-view visual conversion module is the first multi-view visual conversion module in the cascade relationship, its input feature may be a multi-view image vector set; or, if the current multi-view visual conversion module is the third multi-view visual conversion module in the cascade relationship, its input feature may be a multi-view intermediate feature map output by the second multi-view visual conversion module. In addition, the multi-view intermediate feature map output by the last multi-view visual conversion module in the cascade relationship may be the target multi-view feature map in the embodiment of the present application.

[0102] It can be understood that the first attention parameter of the input feature can be the query vector, key vector and value vector corresponding to the input feature; step B2 can be the query vector, key vector and value vector in the first attention parameter, respectively, and the resolution of the tensor is reduced by downsampling the interval window, thereby reducing the overall sequence length of the input. Specifically, the query vector, key vector and value vector can be downsampled by the interval parameter. For example, if the interval parameter is 2, the query vector, key vector and value vector can be sampled every two sampled pixels, and the missed sampled pixels can be saved as other data channels to obtain the second attention parameter. The tensor resolution corresponding to the second attention parameter is smaller than the tensor resolution corresponding to the first attention parameter.

[0103] It should be noted that step B3 may be to use a multi-head attention mechanism to perform channel expansion on the query vector, key vector and value vector in the second attention parameter, so as to obtain a third attention parameter corresponding to a larger number of channels, where each channel corresponding to the third attention parameter corresponds to an image feature at a resolution.

[0104] B4. Perform feature attention calculation on the input feature according to the third attention parameter to obtain a multi-view intermediate feature map output by the current multi-view visual conversion module.

[0105] Reference Figure 5 Further, the step B4, performing feature attention calculation on the input feature according to the third attention parameter to obtain a multi-view intermediate feature map, includes:

[0106] B41, performing attention calculation on the third attention parameter to obtain a first intermediate attention feature;

[0107] B42. Adding the first intermediate attention feature according to the input feature to obtain a second intermediate attention feature;

[0108] B43. Perform feature normalization on the second intermediate attention feature to obtain the multi-view intermediate feature map.

[0109] In an embodiment of the present application, the attention calculation in step B41 can first be the dot product of the query vector in the third attention parameter and all key vectors to obtain the corresponding dot product result; then, the dot product result is converted into an attention weight in the range of [0, 1] through a normalization function (such as a SoftMax function), and the attention weight is weighted and summed with the value vector to obtain the attention feature representation corresponding to the input feature (i.e., the first intermediate attention feature).

[0110] It can be understood that after obtaining the first intermediate attention feature, the first intermediate attention feature and the input feature can be added to prevent the initial feature from being lost, so as to obtain the second intermediate attention feature; then, the second intermediate attention feature is normalized to eliminate the dimensional difference between the features and accelerate the convergence speed of the defect detection model, so as to obtain the multi-view intermediate feature map output by the current multi-view visual conversion module.

[0111] S160 . Update the parameters of the initialized defect detection model according to the target multi-view feature map to obtain the trained defect detection model.

[0112] In an embodiment of the present application, step S160 may first be to input the target multi-view feature map into a classifier for classification detection, so as to obtain a classification detection result corresponding to the multi-view image training set, and then based on the classification detection result, update the parameters of the defect detection model through the back propagation algorithm to obtain a trained defect detection model.

[0113] Reference Figure 6 In some embodiments, the step S160, updating the parameters of the initialized defect detection model according to the target multi-view feature map to obtain the trained defect detection model, includes:

[0114] C1. Obtain the view dimension features and channel dimension features of the target multi-view feature map;

[0115] C2. According to the viewing angle dimension feature, the channel dimension feature is merged to obtain a first dimension feature;

[0116] C3, reducing the feature space of the first dimensional features to obtain second dimensional features;

[0117] C4. Perform feature defect detection on the second dimensional feature to obtain an original defect detection result;

[0118] C5. Performing model training on the initialized defect detection model according to the original defect detection result to obtain the trained defect detection model.

[0119] In an embodiment of the present application, step C1 may be to decompose the target multi-view feature map to obtain visual dimension features, channel dimension features and spatial dimension features; step C2 may be to merge and splice the visual dimension features and channel dimension features of different viewpoints to obtain the first dimension features; then, the first dimension features are average pooled in the spatial dimensions (such as height and width) to reduce the input feature map corresponding to the first dimension features in the spatial dimensions to a single value to obtain the second dimension features.

[0120] It is understandable that after obtaining the second dimension feature, the second dimension feature can be input into the classifier, which can be a multi-layer perceptron head for multi-classification, so as to obtain the original defect detection result; in addition, before the defect detection model is put into use, it needs to be trained to adjust its internal parameters to achieve a better detection effect. Specifically, when training the model, a batch of multi-view images can be obtained, wherein each multi-view image includes multi-view image data of the workpiece surface, and the workpiece surface defect label corresponding to the multi-view image is also obtained. Then, each multi-view image and its corresponding label can be used as a set of training data, the input data of the model is the multi-view image, and the multi-view image is predicted by the model, and the output data of the model is the original defect detection result. After obtaining the original defect detection result output by the model, the accuracy of the model prediction can be evaluated according to the original defect detection result and the corresponding workpiece surface defect label, so as to update the parameters of the model.

[0121] Specifically, for machine learning models, the accuracy of model prediction results can be measured by a loss function (LossFunction), which is defined on a single training data and is used to measure the prediction error of a training data. Specifically, the loss value of the training data is determined by the label of a single training data and the detection result of the model on the training data. In actual training, a training data set has a lot of training data, so a cost function (CostFunction) is generally used to measure the overall error of the training data set. The cost function is defined on the entire training data set and is used to calculate the average value of the prediction error of all training data, which can better measure the prediction effect of the model. For general machine learning models, based on the aforementioned cost function, plus a regularization term that measures the complexity of the model, it can be used as the objective function of the training, and the loss value of the entire training data set can be calculated based on the objective function. There are many types of commonly used loss functions, such as 0-1 loss function, square loss function, absolute loss function, logarithmic loss function, cross entropy loss function, etc., which can all be used as loss functions of machine learning models, which will not be described one by one here. In the embodiment of the present application, one loss function can be selected to determine the loss value of the training, such as the cross entropy loss function. Based on the training loss value, the back propagation algorithm is used to update the parameters of the model. After several iterations, a trained defect detection model can be obtained. The specific number of iterations can be set in advance, or the training is considered complete when the test set meets the accuracy requirements.

[0122] Reference Figure 7 In some embodiments, the method further comprises:

[0123] D1. Obtaining a plurality of multi-view intermediate feature maps output by the multi-view visual conversion module;

[0124] D2. Performing a perspective attention weight analysis on all the multi-perspective intermediate feature maps to obtain a multi-perspective attention weight corresponding to each of the multi-perspective intermediate feature maps, wherein each of the multi-perspective attention weights includes a plurality of single-perspective attention weights, and each single-perspective attention weight corresponds to a different perspective dimension;

[0125] In an embodiment of the present application, an attention mechanism for multi-view screening can be inserted at the end of each multi-view visual conversion module. Specifically, for a multi-view intermediate feature map output by a multi-view visual conversion module, the attention mechanism for multi-view screening can first perform global large pooling and global average pooling operations on the multi-view intermediate feature map in each view dimension, calculate the maximum eigenvalue and average eigenvalue of the multi-view intermediate feature map in each view dimension, and splice them into a maximum eigenvector and an average eigenvector, respectively; then, the maximum eigenvector and the average eigenvector are input into a shared fully connected layer for learning the attention weight of each view, and a Sigmoid activation function is applied to generate a multi-view attention weight corresponding to the multi-view intermediate feature map, wherein the multi-view attention weight includes several single-view attention weights. The multi-view intermediate feature maps output by the remaining multi-view visual conversion modules are similarly derived by simple analogy, and the present application will not elaborate on them here.

[0126] D3, performing single-view attention sum statistics and sorting screening on all the single-view attention weights to obtain several target view dimensions;

[0127] Reference Figure 8 Further, in step D3, all the single-view attention weights are summed and sorted to obtain several target perspective dimensions, including:

[0128] D31, performing perspective screening on all the single-view attention weights to obtain a plurality of single-view attention data sets, each of the single-view attention data sets including a plurality of single-view attention weights with the same perspective dimension;

[0129] D32, performing attention weight summing statistics on all the single-view attention data sets to obtain a statistical attention value corresponding to each single-view attention data set;

[0130] D33, sorting and screening all the statistical attention values ​​to obtain a plurality of target attention values, wherein the target attention value is a statistical attention value that is ranked higher than all the statistical attention values;

[0131] D34. Based on all the target attention values, several target viewing angle dimensions are obtained, and each target viewing angle dimension corresponds to one target attention value.

[0132] In an embodiment of the present application, step D31 can be based on the screening condition of the viewing angle dimension, and filter the single-view attention weights of the same viewing angle dimension to the same single-view attention data set; then, for a certain single-view attention data set, all the single-view attention weights in the single-view attention data set can be summed and counted to obtain the statistical attention value corresponding to the single-view attention data set, and the same is true for the other single-view attention data sets.

[0133] It can be understood that after obtaining all the statistical attention values, all the statistical attention values ​​can be sorted by size to obtain several target attention values. Specifically, in an embodiment of the present application, the first three statistical attention values ​​in the size sorting can be used as target attention values, and then the perspective dimension corresponding to each target attention can be determined as the target perspective dimension.

[0134] D4. According to all the target perspective dimensions, update the multi-perspective dataset to obtain an updated multi-perspective dataset.

[0135] In an embodiment of the present application, step D4 can be to update the data perspective of the multi-perspective dataset based on the target perspective dimension to obtain an updated multi-perspective dataset. The perspective dimension corresponding to the updated multi-perspective dataset is the target perspective dimension, so that the subsequent input of the defect detection model is the image data corresponding to the target perspective dimension, which can dynamically select the perspective that best suits the workpiece to be inspected, while ensuring that subsequent defect detection can achieve the best effect, it can also reduce the required computing resources and reduce costs.

[0136] Fig. 9 A schematic diagram of a workpiece surface defect detection system provided in an embodiment of the present application includes:

[0137] The first processing unit 810 is used to obtain a multi-view data set of the surface of the workpiece to be inspected;

[0138] The second processing unit 820 is used to input the multi-view data set into the trained defect detection model to perform surface defect detection and obtain a target defect detection result;

[0139] The trained defect detection model is obtained by the following steps:

[0140] Obtain a multi-view image training set;

[0141] Performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set;

[0142] Inputting the multi-view image vector set into an initialized defect detection model for multi-level feature capture to obtain a target multi-view feature map, wherein the number of feature channels and feature resolution of the target multi-view feature map are negatively correlated;

[0143] According to the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model.

[0144] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0145] Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of the present application includes:

[0146] at least one processor 980;

[0147] At least one memory 920, used to store at least one program;

[0148] When the at least one program is executed by the at least one processor 980, the at least one processor 980 implements the methods described in the foregoing embodiments.

[0149] An embodiment of the present application also provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor 980, it is used to implement the methods described in the above embodiments.

[0150] Fig.10 Specifically, the computer device may be a user terminal or a server.

[0151] The present application embodiment takes the computer device as a user terminal as an example, and the details are as follows:

[0152] like Fig.10 As shown, the computer device 900 may include an RF (Radio Frequency) circuit 910, a memory 920 including one or more computer-readable storage media, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a WiFi module 970, a processor 980 including one or more processing cores, and a power supply 990. Those skilled in the art will appreciate that Fig.10 The device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0153] The RF circuit 910 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is handed over to one or more processors 980 for processing; in addition, the data related to the uplink is sent to the base station. Generally, the RF circuit 910 includes but is not limited to an antenna, at least one amplifier, a tuner, one or more oscillators, a user identity module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, etc. In addition, the RF circuit 910 can also communicate with the network and other devices through wireless communication. Wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.

[0154] The memory 920 can be used to store software programs and modules. The processor 980 executes various functional applications and data processing by running the software programs and modules stored in the memory 920. The memory 920 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the device 900 (such as audio data, a phone book, etc.), etc. In addition, the memory 920 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 920 may also include a memory controller to provide the processor 980 and the input unit 930 with access to the memory 920. Although Fig.10 The RF circuit 910 is shown, but it is understandable that it is not an essential component of the device 900 and can be omitted as required without changing the essence of the invention.

[0155] The input unit 930 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit 930 may include a touch-sensitive surface 932 and other input devices 931. The touch-sensitive surface 932, also known as a touch display screen or a touch pad, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch-sensitive surface 932), and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface 932 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 980, and can receive and execute commands sent by the processor 980. In addition, the touch-sensitive surface 932 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch-sensitive surface 932, the input unit 930 may further include other input devices 931. Specifically, the other input devices 931 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0156] The display unit 940 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the device 900, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 940 may include a display panel 941. Optionally, the display panel 941 may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 932 may be covered on the display panel 941. When the touch-sensitive surface 932 detects a touch operation on or near it, it is transmitted to the processor 980 to determine the type of touch event. Subsequently, the processor 980 provides a corresponding visual output on the display panel 941 according to the type of touch event. Although in Fig.10 In the embodiment, the touch-sensitive surface 932 and the display panel 941 are implemented as two independent components to implement input and output functions, but in some embodiments, the touch-sensitive surface 932 and the display panel 941 can be integrated to implement input and output functions.

[0157] The computer device 900 may also include at least one sensor 950, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 941 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 941 and / or the backlight when the device 900 is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can also be configured in the device 900, they will not be repeated here.

[0158] The audio circuit 960, the speaker 961, and the microphone 962 can provide an audio interface between the user and the device 900. The audio circuit 960 can transmit the electrical signal converted from the received audio data to the speaker 961, which is converted into a sound signal for output; on the other hand, the microphone 962 converts the collected sound signal into an electrical signal, which is received by the audio circuit 960 and converted into audio data, and then the audio data is processed by the output processor 980 and sent to another control device through the RF circuit 910, or the audio data is output to the memory 920 for further processing. The audio circuit 960 may also include an earplug jack to provide communication between an external headset and the device 900.

[0159] The device 900 can transmit information with the wireless transmission module set on the fighting device through the WiFi module 970.

[0160] The processor 980 is the control center of the device 900. It uses various interfaces and lines to connect various parts of the entire control device. It executes various functions of the device 900 and processes data by running or executing software programs and / or modules stored in the memory 920, and calling data stored in the memory 920, so as to monitor the control device as a whole. Optionally, the processor 980 may include one or more processing cores; optionally, the processor 980 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 980.

[0161] The device 900 also includes a power supply 990 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 980 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 990 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0162] Although not shown, the device 900 may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0163] An embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the method described in the above embodiments.

[0164] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0165] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0166] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0169] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0170] The step numbers in the above method embodiment are only provided for the convenience of explanation and description, and no limitation is imposed on the order of the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0171] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the described embodiments. Technical personnel familiar with the field may make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for detecting defects on a workpiece surface, characterized in that: include: Acquire a multi-view data set of the surface of the workpiece to be inspected; Inputting the multi-view data set into a trained defect detection model to perform surface defect detection to obtain a target defect detection result; The trained defect detection model is obtained by the following steps: Obtain a multi-view image training set; Performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set; Inputting the multi-view image vector set into an initialized defect detection model for multi-level feature capture to obtain a target multi-view feature map, wherein the number of feature channels and feature resolution of the target multi-view feature map are negatively correlated; According to the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model.

2. The defect detection method according to claim 1, characterized in that: The step of performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set includes: Performing image segmentation on the multi-view image training set to obtain a multi-view image segmentation set; Performing linear projection embedding on the multi-view image segmentation set to obtain a multi-view image embedding set; Position encoding is performed on the multi-view image embedding set to obtain the multi-view image vector set.

3. The defect detection method according to claim 1, characterized in that: The initialized defect detection model includes a plurality of cascaded multi-view visual conversion modules, each of which is used to perform the following steps: Obtaining an input feature and a first attention parameter of the input feature, wherein the input feature is a multi-view image vector set or a multi-view intermediate feature map output by a previous multi-view visual conversion module; Downsampling the first attention parameter by an interval window parameter to obtain a second attention parameter, wherein a tensor resolution corresponding to the second attention parameter is smaller than a tensor resolution corresponding to the first attention parameter; Performing attention channel expansion on the second attention parameter to obtain a third attention parameter, wherein the number of channels corresponding to the third attention parameter is greater than the number of channels corresponding to the second attention parameter; According to the third attention parameter, feature attention calculation is performed on the input feature to obtain a multi-view intermediate feature map output by the current multi-view visual conversion module.

4. The defect detection method according to claim 3, characterized in that: The step of performing feature attention calculation on the input feature according to the third attention parameter to obtain a multi-view intermediate feature map includes: Performing attention calculation on the third attention parameter to obtain a first intermediate attention feature; According to the input feature, adding the first intermediate attention feature to obtain a second intermediate attention feature; The second intermediate attention feature is normalized to obtain the multi-view intermediate feature map.

5. The defect detection method according to claim 1, characterized in that: The step of updating the parameters of the initialized defect detection model according to the target multi-view feature map to obtain the trained defect detection model includes: Obtaining the view dimension features and channel dimension features of the target multi-view feature map; According to the viewing angle dimension feature, the channel dimension feature is combined to obtain a first dimension feature; Performing feature space reduction on the first dimensional features to obtain second dimensional features; Performing feature defect detection on the second dimensional feature to obtain an original defect detection result; According to the original defect detection result, the initialized defect detection model is trained to obtain the trained defect detection model.

6. The defect detection method according to claim 3, characterized in that: The method further comprises: Obtaining a plurality of multi-view intermediate feature maps output by the multi-view visual conversion modules; Performing a perspective attention weight analysis on all the multi-perspective intermediate feature maps to obtain a multi-perspective attention weight corresponding to each of the multi-perspective intermediate feature maps, wherein each of the multi-perspective attention weights includes a plurality of single-perspective attention weights, and each single-perspective attention weight corresponds to a different perspective dimension; Performing single-view attention summing statistics and sorting screening on all the single-view attention weights to obtain a plurality of target view dimensions; According to all the target perspective dimensions, the multi-perspective dataset is updated to obtain an updated multi-perspective dataset.

7. The defect detection method according to claim 6, characterized in that: The single-view attention summing statistics and sorting and screening are performed on all the single-view attention weights to obtain several target view dimensions, including: Performing perspective screening on all the single-view attention weights to obtain a plurality of single-view attention data sets, each of the single-view attention data sets including a plurality of single-view attention weights with the same perspective dimension; Performing attention weight summing statistics on all the single-view attention data sets to obtain a statistical attention value corresponding to each single-view attention data set; Sorting and screening all the statistical attention values ​​to obtain a plurality of target attention values, wherein the target attention value is a statistical attention value that is ranked higher than all the statistical attention values; According to all the target attention values, a plurality of target viewing angle dimensions are obtained, and each target viewing angle dimension corresponds to one target attention value.

8. A system for detecting defects on the surface of a workpiece, characterized in that: include: A first processing unit is used to obtain a multi-view data set of the surface of the workpiece to be inspected; A second processing unit is used to input the multi-view data set into a trained defect detection model to perform surface defect detection and obtain a target defect detection result; The trained defect detection model is obtained by the following steps: Obtain a multi-view image training set; Performing vector embedding conversion on the multi-view image training set to obtain a multi-view image vector set; Inputting the multi-view image vector set into an initialized defect detection model for multi-level feature capture to obtain a target multi-view feature map, wherein the number of feature channels and feature resolution of the target multi-view feature map are negatively correlated; According to the target multi-view feature map, the parameters of the initialized defect detection model are updated to obtain the trained defect detection model.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images