Object defect detection method and device based on double-flow characteristics

Abstract

The invention provides an object defect detection method and device based on double-flow characteristics. The method comprises the following steps: establishing a corresponding relation between color data and shape data of a detection piece and a defect condition of the detection piece through an artificial intelligence model; wherein color features are extracted from the color data by the first processing layer; extracting shape features from the shape data by a second processing layer; determining a corresponding relation through the fusion layer according to the color features and the shape features; collecting image data of the target detection piece; wherein the image data comprises actual color data and actual shape data; and determining an actual defect condition corresponding to the target detection piece according to the actual color data and the actual shape data through the corresponding relationship. The object defect detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of detection technologies, and particularly to an object defect detection method and device based on dual-stream characteristics. Background Art

[0002] During the manufacturing process of industrial parts, terminal detection usually relies on manual labor or simple automated detection equipment. When faced with complex graphics and diverse parts, traditional data models often fail to complete the detection tasks quickly and accurately, resulting in low production efficiency and increased costs. Summary of the Invention

[0003] In view of the above problems, the present application is proposed to provide an object defect detection method and device based on dual-stream characteristics that overcome or at least partially solve the above problems, including: An object defect detection method based on dual-stream characteristics, the method involving an artificial intelligence model; the method is used to detect defects of a target detection part through the artificial intelligence model; the artificial intelligence model includes: a first processing layer, a second processing layer, and a fusion layer; wherein, the first processing layer and the second processing layer are independent of each other and are respectively used to process different data streams, and the output results of each are transmitted to the fusion layer; The method includes: Establish a correspondence relationship between the color data and shape data of the detection part and the defect situation of the detection part through the artificial intelligence model; wherein, color features are extracted from the color data through the first processing layer; shape features are extracted from the shape data through the second processing layer; the correspondence relationship is determined by the fusion layer based on the color features and the shape features; Collect image data of the target detection part; wherein, the image data includes actual color data and actual shape data; Determine the actual defect situation corresponding to the target detection part based on the actual color data and the actual shape data through the correspondence relationship.

[0004] Further, the training data set includes a color characteristic training data set and a shape characteristic training data set; The step of establishing a correspondence relationship between the color data and shape data of the detection part and the defect situation of the detection part through the artificial intelligence model includes: Input the training data set into the artificial intelligence model for training; Wherein, color training prediction data is determined by the first processing layer based on the color characteristic training data set; Shape training prediction data is determined by the second processing layer based on the shape characteristic training data set; Determine the training prediction result according to the color training prediction data and the shape training prediction data through the fusion layer; Determine the training comparison result according to the training prediction result and a preset true value, and update the artificial intelligence model according to the training comparison result.

[0005] Further, the method involves the BCEwithLogits loss function; the training prediction result includes a predicted logit value; The step of determining the training comparison result according to the training prediction result and a preset true value, and updating the artificial intelligence model according to the training comparison result includes: Compare the predicted logit value and the true value through the BCEwithLogits loss function to determine the training comparison result; Update the artificial intelligence model according to the training comparison result.

[0006] Further, the color feature includes color weight data; The step of extracting the color feature from the color data includes: Calculate the color Euclidean distance according to the color data and a preset target color data: Determine the color weight data according to the color Euclidean distance and the maximum possible distance in the color space.

[0007] Further, the shape feature includes shape weight data; The step of extracting the shape feature from the shape data includes: Calculate the shape cosine similarity according to the shape data and the target shape data; Calculate the shape weight data according to the shape cosine similarity.

[0008] Further, the step of determining the corresponding relationship according to the color feature and the shape feature includes: Determine the overall weight data according to the color feature and the shape feature; Determine the defect situation according to the overall weight data, and establish the corresponding relationship.

[0009] Further, the method involves a laser scanning device; the target detection part includes at least 1 production part; the method is used to sequentially collect data of the production parts through the laser scanning device arranged on the production line when the production line produces parts; The step of collecting the image data of the target detection part includes: Scan the production parts through the laser scanning device to determine the actual color data and the actual shape data.

[0010] An object defect detection device based on dual-stream characteristics, the device relates to an artificial intelligence model; the device is used to detect the defects of a target detection part through the artificial intelligence model; the artificial intelligence model includes: a first processing layer, a second processing layer and a fusion layer; wherein, the first processing layer and the second processing layer are independent of each other, and are respectively used to process different data streams, and the output results of each are transmitted to the fusion layer; The device includes: A correspondence relationship establishment module, which is used to establish the correspondence relationship between the color data and shape data of the detection part and the defect situation of the detection part through the artificial intelligence model; wherein, color features are extracted from the color data through the first processing layer; shape features are extracted from the shape data through the second processing layer; the correspondence relationship is determined by the fusion layer according to the color features and the shape features; An image data acquisition module, which is used to acquire the image data of the target detection part; wherein, the image data includes actual color data and actual shape data; An actual defect situation determination module, which is used to determine the actual defect situation corresponding to the target detection part according to the actual color data and the actual shape data through the correspondence relationship.

[0011] A computer device includes a processor, a memory, and a computer program stored on the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the object defect detection method based on dual-stream characteristics as described in any embodiment of the present application.

[0012] A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the object defect detection method based on dual-stream characteristics as described in any one of claims 1 to 7.

[0013] The present application has the following advantages: In an embodiment of the present application, in view of the fact that traditional data models often cannot complete detection tasks quickly and accurately, resulting in low production efficiency and increased costs, the present application provides a solution of "inputting color data and shape data into independent first and second processing layers respectively for feature extraction, and then processing the color features and shape features simultaneously by a fusion layer to identify defects of a detection piece to establish a corresponding relationship, and performing data collection and defect detection on a target detection piece through the corresponding relationship", specifically: establishing a corresponding relationship between the color data and shape data of a detection piece and the defect condition of the detection piece through the artificial intelligence model; wherein, extracting color features from the color data through the first processing layer; extracting shape features from the shape data through the second processing layer; determining the corresponding relationship through the fusion layer according to the color features and the shape features; collecting image data of the target detection piece; wherein, the image data includes actual color data and actual shape data; determining the actual defect condition corresponding to the target detection piece according to the actual color data and the actual shape data through the corresponding relationship. The present application improves the efficiency and accuracy of object defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the present application, the drawings required for the description of the present application will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0015] Figure 1 FIG. is a flowchart of steps of a method for detecting object defects based on dual-stream characteristics provided by an embodiment of the present application; Figure 2 FIG. is a comparison relationship diagram of color data and shape data of a target detection piece in an embodiment of the present application; Figure 3 FIG. is a schematic structural diagram of a dual-stream Alexnet model in an embodiment of the present application; Figure 4 FIG. is a structural block diagram of a device for detecting object defects based on dual-stream characteristics provided by an embodiment of the present application; Figure 5 FIG. is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] To make the objectives, features, and advantages of this application more obvious and understandable, the following provides a more detailed description of this application in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of this application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts fall within the scope of protection of this application.

[0017] Through the analysis of the prior art, the inventors found that: the fundamental reason why existing models cannot quickly and accurately complete the detection task is that existing models usually process color information and shape information in the same image simultaneously, or separately extract features of color and shape based on the same planar image information and then process them separately. The above may lead to a decrease in detection accuracy in complex scenarios and is prone to detection errors due to the neglect of a certain feature.

[0018] Based on the above systematic analysis, the core technical concept of the present invention is to separately input color data and shape data into independent first and second processing layers for feature extraction, and then simultaneously process color features and shape features by a fusion layer to identify defects of the detection piece to establish a corresponding relationship, and collect data and detect defects of the target detection piece through the corresponding relationship.

[0019] Referring to Figure 1 , there is shown a method for detecting object defects based on dual-stream characteristics provided by an embodiment of this application. The method relates to an artificial intelligence model; the method is used to detect defects of a target detection piece through the artificial intelligence model; the artificial intelligence model includes: a first processing layer, a second processing layer, and a fusion layer; wherein, the first processing layer and the second processing layer are independent of each other and are respectively used to process different data streams, and the output results of each are transmitted to the fusion layer; The method includes: S110. Establish a corresponding relationship between the color data and shape data of the detection piece and the defect situation of the detection piece through the artificial intelligence model; wherein, extract color features from the color data through the first processing layer; extract shape features from the shape data through the second processing layer; determine the corresponding relationship according to the color features and the shape features through the fusion layer; S120. Collect image data of the target detection piece; wherein, the image data includes actual color data and actual shape data; S130. Determine the actual defect situation corresponding to the target detection piece according to the actual color data and the actual shape data through the corresponding relationship.

[0020] In the embodiments of the present application, aiming at the problem that traditional data models often cannot complete detection tasks quickly and accurately, resulting in low production efficiency and increased costs, the present application provides a solution of "inputting color data and shape data into independent first and second processing layers respectively for feature extraction, and then using a fusion layer to process color features and shape features simultaneously to identify defects of a detected object to establish a corresponding relationship, and collecting data and detecting defects of a target detected object through the corresponding relationship", specifically: establishing a corresponding relationship between the color data and shape data of the detected object and the defect situation of the detected object through the artificial intelligence model; wherein, extracting color features from the color data through the first processing layer; extracting shape features from the shape data through the second processing layer; determining the corresponding relationship through the fusion layer according to the color features and the shape features; collecting image data of the target detected object; wherein, the image data includes actual color data and actual shape data; determining the actual defect situation corresponding to the target detected object through the corresponding relationship according to the actual color data and the actual shape data. The present application improves the efficiency and accuracy of object defect detection.

[0021] Next, a method for detecting object defects based on the dual-stream characteristic in this exemplary embodiment will be further described.

[0022] It should be noted that the artificial intelligence model may include a dual-stream Alexnet model. The two streams in the Alexnet dual-stream model work independently and in parallel. One stream analyzes the color details of the scanned item, and the other stream analyzes the shape parameters of the scanned item, thereby improving the accuracy of laser scanning for complex graphics. Compared with manual inspection one by one, it is faster and has a lower error rate. The present application analyzes the color and shape of the scanned item through the dual-stream model, significantly improving the detection speed and accuracy, and solving the problems of slow detection speed and abnormal color detection of traditional data models.

[0023] The first processing layer and the second processing layer respectively correspond to two streams in the Alexnet two-stream model. The first processing layer and the second processing layer each receive input data and independently extract relevant features. The first processing layer can be used to extract color information in an image. Using an input image with RGB channels, through the processing of multiple convolutional layers, pooling layers, and activation functions, features related to color in the image are gradually extracted. The second processing layer can be used to extract shape and structural features in the image, and can perform steps similar to those of the first processing layer, but the difference is that the second processing layer is used to extract features related to shape, while the first processing layer is used to extract features related to color. After the convolutional processing of the first processing layer and the second processing layer respectively, these two streams will perform feature fusion in the fully connected layer of the fusion layer. The fused feature vector contains comprehensive information from color and shape, thus representing features more comprehensively and accurately.

[0024] As an example, Figure 2 shows the comparison relationship between the color data and the shape data of the target detection part (i.e., the original part), Figure 3 shows the structure of the two-stream Alexnet model. The first processing layer may include Conv-1, Conv-2, and Conv-3, the fusion layer may include FC-4 and FC-5, the color data may be input to Conv-1 of the first processing layer, and the shape data may be input to Conv-1 of the second processing layer. Among them, Conv-1 includes a convolutional layer r + ReLU; both Conv-2 and Conv-3 include a convolutional layer r + ReLU and a pooling layer; FC-4 and FC-5 include a fully connected layer + ReLU; the data after full connection is classified and output by the pooling layer.

[0025] In an embodiment of the present application, the specific process before the step S110 of "establishing the correspondence relationship between the color data and the shape data of the detection part and the defect situation of the detection part through the artificial intelligence model" can be further described in combination with the following description.

[0026] As described in the following steps, the training data set is input into the artificial intelligence model for training.

[0027] It should be noted that before inputting the training data set into the artificial intelligence model, it is necessary to first deploy a two-stream Alexnet model. The specific steps may include: based on the two-stream characteristics of Alexnet, setting one stream to analyze the color details of the scanned item, and the other stream to analyze the shape parameters of the scanned item. That is, the first processing layer includes a convolutional neural network for processing color information, and the second processing layer includes a convolutional neural network for processing shape information. The outputs of the two streams are fused, and the fused features are finally classified or regressed through the fusion layer to define the data model structure of the model.

[0028] The training data set can be divided into three data sets: the training set is used for training the model, generally accounting for 70%-80% of the total data; the validation set is used to adjust the model parameters and select the best model, generally accounting for 10%-15% of the total data; the test set is used to evaluate the performance of the final model, generally accounting for 10%-15% of the total data.

[0029] In a specific embodiment of the present application, the step of "inputting the training data set into the artificial intelligence model for training" may specifically include: Generate a database containing part images. 1000 images can be imported, and the size of each image can be 256x256 pixels to complete data collection; among them, the data set can contain 3 different colors and 16 different shapes, and each image is labeled with shape and color information; the images can be divided into two parts: the shape data stream and the color data stream; Analyze based on the two-stream characteristics of Alexnet; among them, for the first stream, the color channel of the input image is passed to this part; for the second stream, the input image undergoes morphological processing to highlight the shape features and then is passed to this part.

[0030] Train colors and shapes; among them, colors and shapes are uniformly analyzed and learned from 700 samples, and the model simulates part features under multiple color and shape combinations; Introduce the BCEWithLogits loss function; among them, by maximizing the consistency between the model prediction and the actual label through the loss function, the classification error is reduced, and the model can effectively distinguish different color and shape combinations, thereby improving the detection accuracy; After importing the sample model, the model will analyze 150 sample images. The prediction parameters obtained through the loss function are compared with the scanning parameters of the image sample, and through the least squares method inside the model, the best fitting parameter combination of these 150 image samples is extracted, and its system parameters are slightly modified to minimize the difference between the model output and the actual detection result, and the parameters of the model are calibrated to optimize the model performance; Import the remaining 150 image samples in the database and perform detection using the parameter combinations in the database to ensure that the accuracy rate on the test set reaches 95% or above. If the accuracy rate cannot reach 95% or above, the validation set will randomly select 150 image samples from the training set again, and the model will retrain, validate, and test the image samples until the accuracy rate reaches 95% or above. That is, after the model can accurately detect the color and shape abnormalities of the parts, the model can be put into use.

[0031] It should be noted that during the detection process, each part image scanned on the production line will be input into the trained two-stream model; the model will analyze the color and shape features separately. If it is found through the loss function that the detection accuracy rate of this training round is higher than the accuracy rate of the currently used parameter combination, the model will automatically update the parameter combination to achieve self-optimization of the model.

[0032] After testing, during the process of being put into use, a total of 1000 parts are scanned in one training round. In this training round, the model scans a total of 32 defective parts. After manual inspection, there are a total of 35 defective parts in this training round. Among them, the model misses 4 defective parts, and 1 part is misdetected because the welding color of the part is too different from the welding color of the conventional part. The total number of non-defective parts = total number of parts - actual number of defective parts = 1000 - 35 = 965 pieces. The number of correctly identified non-defective parts = actual number of correctly detected defective parts + number of correctly identified non-defective parts - number of misdetected parts by the model = 27 + 964 = 990 pieces. The overall accuracy rate is: (total number of correctly detected by the model) / (total number of scanned parts) = 990 / 1000 = 99%. It can meet the actual use.

[0033] In an embodiment of the present application, the training data set includes a color characteristic training data set and a shape characteristic training data set; the specific process of "inputting the training data set into the artificial intelligence model for training" can be further described in combination with the following description.

[0034] As described in the following steps, the first processing layer determines color training prediction data based on the color characteristic training data set; The second processing layer determines shape training prediction data based on the shape characteristic training data set; The fusion layer determines a training prediction result based on the color training prediction data and the shape training prediction data; Determine a training comparison result based on the training prediction result and a preset true value, and update the artificial intelligence model based on the training comparison result.

[0035] It should be noted that the training dataset can collect color and shape data of various parts through an information collection system, and use the Alexnet two-stream model to train and test the collected data.

[0036] In a specific implementation, the specific steps for collecting the training dataset may include: Deploy a laser scanning device and its related sensors to obtain laser scanning data containing color and shape parameters, and label the data to ensure that the color and shape information of each data point is accurate; The system sets the data types to be collected, and uses an online information collection system to determine all the part data types required by the system, as well as relevant information, including part materials, colors, and shapes; The system processes missing values and abnormal data to ensure data quality, standardizes or normalizes the selected parts through color and shape data, and divides them into a training set, a validation set, and a test set; among them, the training set is used to train the model and adjust the corresponding parameters of all parts; the validation set is used to evaluate the performance of the model and determine the final performance of the model to maximize the accuracy; the test set is used to test the accuracy of the model.

[0037] It should be noted that when using the test set, it may happen that the accuracy and parameters are not accurate enough to meet the standards for use. To ensure its accuracy and sustainable optimization, the loss function will be used to apply the function to the three datasets, compare the predicted values of all the data with the predicted values currently determined and applied by the model, and optimize the parameters until the optimal settings are found.

[0038] In an embodiment of the present application, the method involves the BCEwithLogits loss function; the training prediction result includes the predicted logit value; the specific process of "determining the training comparison result based on the training prediction result and the preset true value, and updating the artificial intelligence model based on the training comparison result" can be further described in combination with the following description.

[0039] As described in the following steps, the training comparison result is determined by comparing the predicted logit value and the true value through the BCEwithLogits loss function; Update the artificial intelligence model based on the training comparison result.

[0040] It should be noted that by defining the BCEwithLogits loss function, the loss function is used to evaluate the accuracy of the model prediction. Specifically, it measures the gap between the model prediction value and the true value, and the optimization algorithm trains the model by minimizing the loss function, making the prediction result of the model closer to the true value.

[0041] In a specific embodiment of the present application, the training comparison result is determined by comparing the predicted logit value and the true value through the BCEwithLogits loss function; wherein, the expression of the BCEwithLogits loss function is:

[0042] where N is the number of samples in the training data set; C is the number of categories in the training data set; represents the true value of the j-th label of the i-th sample in the training data set; represents the predicted logit value of the j-th label of the i-th sample in the training data set; represents the sigmoid function applied to the logit and the calculation formula of is:

[0043] The artificial intelligence model is updated according to the training comparison result. By using the BCEwithLogits loss function to evaluate the detection result, the model is adjusted according to the feedback of the loss function to optimize the detection accuracy and ensure its stable and efficient operation.

[0044] It should be noted that the BCEwithLogits loss function combines the binary cross-entropy loss and the activation function, combines the Sigmoid layer and the binary cross-entropy loss in one formula, simplifies the derivative calculation in the optimization process, and the BCEwithLogits loss function is used to measure the difference between the predicted probability and the true label, as well as calculate the matching degree between the predicted value and the true value for each sample and each label. The Sigmoid function is used to convert the logit value into a probability value.

[0045] In an embodiment of the present application, the color feature includes color weight data; the specific process of "extracting color features from the color data" in step S110 can be further described in combination with the following description.

[0046] As described in the following steps, the color Euclidean distance is calculated based on the color data and the preset target color data: The color weight data is determined based on the color Euclidean distance and the maximum possible distance in the color space.

[0047] It should be noted that in the dual-stream model, the color information flow uses the RGB model to extract color-related features in the image. The RGB model defines any color by specifying the intensities of three primary colors: red (R), green (G), and blue (B). The value of each color component is usually between 0 and 255 (in 8-bit representation), where: R represents the intensity of the red component, G represents the intensity of the green component, and B represents the intensity of the blue component. The difference between the target color and the actual color can be judged by defining the Euclidean distance between them.

[0048] In a specific embodiment of the present application, the first processing layer is used to define colors through RGB color components.

[0049] As an example, taking the feature extraction of actual color data as an example, the specific process of "extracting color features from the color data" described in step S110 is further described.

[0050] The definition of the actual color data is as follows:

[0051] The definition of the target color data is as follows:

[0052] Wherein, represents the red component of the actual color data, represents the green component of the actual color data, represents the blue component of the actual color data;

[0053] The steps of extracting color features from the actual color data include: Calculating the color Euclidean distance according to the actual color data and the preset target color data through the following formula:

[0054] Wherein, represents the color Euclidean distance; Calculating the color weight data according to the color Euclidean distance and the maximum possible distance in the color space through the following formula:

[0055] Wherein, represents the maximum possible distance in the color space.

[0056] It should be noted that the above steps compare the Euclidean distance with the maximum possible color difference to normalize the Euclidean distance. The maximum possible distance in the color space The value range can be from (0, 0, 0) to (255, 255, 255). By calculating the color weight data based on the ratio between the Euclidean distance of colors and the maximum possible distance in the color space, it can be ensured that when the color difference is the largest (i.e., extreme cases such as black and white), the color weight data will be close to 0; when the colors match exactly, the weight is equal to 0.5; if the color weight data deviates too much from 0.5, the color interval will not pass the review.

[0057] In an embodiment of the present application, the shape feature includes shape weight data; the specific process of "extracting shape features from the shape data" in step S110 can be further described in combination with the following description.

[0058] As described in the following steps, calculate the shape cosine similarity based on the shape data and the target shape data; Calculate the shape weight data based on the shape cosine similarity.

[0059] It should be noted that the shape information flow measures the similarity between shapes using cosine similarity, and represents shapes through vectors.

[0060] In a specific embodiment of the present application, the second processing layer is used to define shapes through vectors; the shape feature includes shape weight data.

[0061] As an example, taking the feature extraction of actual shape data as an example, the specific process of "extracting shape features from the shape data" in step S110 is further described.

[0062] The steps of extracting shape features from actual shape data include: Calculate the shape cosine similarity based on the actual shape data and the target shape data through the following formula:

[0063] Where, represents the actual shape data; represents the target shape data, represents the shape cosine similarity; Calculate the shape weight data based on the shape cosine similarity through the following formula:

[0064] Where, represents the shape weight data.

[0065] It should be noted that the value of cosine similarity is between -1 and 1, and its numerical meaning is as follows: when the cosine similarity is equal to 1, it means the shapes are exactly the same; when the cosine similarity is equal to -1, it means the shapes are exactly opposite; when the cosine similarity is equal to 0, it means the shapes have no correlation. When the shapes are exactly matched, the cosine similarity is 1, and the calculated is 0.5; when the shape difference is large, the cosine similarity value is close to 0 or negative, and the calculated will be far from 0.5.

[0066] It should be understood that the steps of extracting color features from color data can be mutually referred to the steps of extracting color features from actual color data; the steps of extracting shape features from shape data can be mutually referred to the steps of extracting shape features from actual shape data.

[0067] In an embodiment of the present application, the specific process of "determining the corresponding relationship according to the color feature and the shape feature" described in step S110 can be further described in combination with the following description.

[0068] As described in the following steps, determine the overall weight data according to the color feature and the shape feature; Determine the defect situation according to the overall weight data, and establish the corresponding relationship.

[0069] It should be noted that the overall weight data is the detection result of the two-stream model. The detection result of the two-stream model is comprehensively analyzed through a fully connected layer to mark all possible defective parts of the parts, so as to determine the defect situation and establish the corresponding relationship.

[0070] In a specific embodiment of the present application, the color feature includes color weight data; the shape feature includes shape weight data; The step of determining the corresponding relationship according to the color feature and the shape feature includes: Determine the overall weight data according to the color weight data and the shape weight data through the following formula:

[0071] Wherein, represents the overall weight data, represents the color weight data, represents the shape weight data; Determine the defect situation according to the overall weight data, and establish the corresponding relationship.

[0072] It should be noted that after the first processing layer and the second processing layer process their respective part information, feature information fusion is performed in the fully connected layer of the fusion layer, and the fused feature vector is the overall weight data. , the overall weight data contains comprehensive information from color and shape.

[0073] According to the overall weight data, the defect situation can be determined by the following judgment criteria: If is 1, it means the part is completely qualified and there are no defects; If or any value in is not 0.5, it means the part is unqualified and there are defects.

[0074] In a specific embodiment of the present application, it further includes: using the BCEwithLogits loss function to evaluate the detection result, adjusting the model according to the feedback of the loss function, and optimizing the detection accuracy to ensure its stable and efficient operation. Specifically, it may include: Determine the total data volume of each scan, defined as the batch size, and each complete scan and export of the result is one training epoch; The part parameters obtained through the integrated data analysis of the training set are prediction values, and for the data obtained in each subsequent training epoch, if the parameters are optimized, the optimized parameters of this epoch are the prediction values of the next training epoch. If there is no optimization, that is, the parameters are the same or the parameters decrease due to too many part defects in this epoch, then the test values of this epoch are continued to be used in the next epoch; In each training epoch, import the real-time scan data into the model. The parameters obtained through the analysis of this data become the real values. Use the loss function to calculate the error between the prediction value and the real value, evaluate the detection result, optimize the algorithm, and update the model weights; Evaluate the model performance of each training epoch, monitor the loss rate and accuracy rate, record the implementation process of this round of detection, determine the model improvement line of defense, update the model, and improve the accuracy rate.

[0075] In the embodiments of the present application, by separating the processing of color and shape, the present application can more accurately identify and distinguish subtle defects; the application of the dual-stream model enables the analysis of color and shape to be carried out in parallel, significantly reducing the detection time; compared with the traditional method of analyzing each feature one by one, this parallel processing significantly improves the overall detection efficiency; the detection results of color and shape are comprehensively analyzed through the fully connected layer, and any abnormality will significantly affect the final detection result, thereby reducing the probability of false detection or missed detection; compared with the traditional single-channel model, it can more effectively avoid detection errors caused by the neglect of a certain feature; by using the BCEWithLogits loss function, it can effectively handle multi-label classification problems and adapt to the detection requirements of different types and specifications of parts, which is particularly important in industrial manufacturing and can handle complex and changeable actual detection scenarios; the present application is applicable to the terminal laser detection field in industrial manufacturing, solves the problems of slow speed and high error rate of traditional manual inspection, and can handle complex part shape and color detection tasks, and has good practical application value.

[0076] In an embodiment of the present application, the method involves a laser scanning device; the target detection part includes at least 1 production part; the method is used to sequentially collect data of the production parts through the laser scanning device arranged on the production line when the production line produces parts; the specific process of "collecting the image data of the target detection part" in step S120 can be further described in combination with the following description.

[0077] As described in the following steps, the actual color data and actual shape data are determined by scanning the production parts through the laser scanning device.

[0078] It should be noted that by collecting data of the production parts in real time, the real-time monitoring of the defects of the production parts is realized.

[0079] In a specific implementation, the specific steps of real-time monitoring may include: Integrate the trained model into the detection system on the production line, input the part data on the production line into the model in real time, use the model to detect the parts in real time, compare the part parameters in the database with the corresponding part parameters on the production line, and identify potential defects; The detection results of the Alexnet dual-stream model are uniformly output, all the exported data are comprehensively analyzed, and the abnormal parts and the possible defective parts are marked.

[0080] The above is the description of the method embodiment of the present application. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment.

[0081] Refer to Figure 4, which shows an object defect detection device based on the dual-stream characteristic provided by an embodiment of the present application. The device relates to an artificial intelligence model; the device is used to detect defects of a target detection part through the artificial intelligence model; the artificial intelligence model includes: a first processing layer, a second processing layer, and a fusion layer; wherein, the first processing layer and the second processing layer are independent of each other, and are respectively used to process different data streams, and the output results of each are transmitted to the fusion layer; The device includes: A correspondence establishing module 410, configured to establish a correspondence between the color data and shape data of the detection part and the defect condition of the detection part through the artificial intelligence model; wherein, color features are extracted from the color data through the first processing layer; shape features are extracted from the shape data through the second processing layer; the correspondence is determined by the fusion layer according to the color features and the shape features; An image data acquisition module 420, configured to acquire image data of the target detection part; wherein, the image data includes actual color data and actual shape data; An actual defect condition determining module 430, configured to determine the actual defect condition corresponding to the target detection part according to the actual color data and the actual shape data through the correspondence.

[0082] In an embodiment of the present application, the correspondence establishing module 410 includes: A training module, configured to input a training data set into the artificial intelligence model for training.

[0083] In an embodiment of the present application, the training data set includes a color characteristic training data set and a shape characteristic training data set; the training module includes: A color training prediction data determining sub-module, configured to determine color training prediction data according to the color characteristic training data set through the first processing layer; A shape training prediction data determining sub-module, configured to determine shape training prediction data according to the shape characteristic training data set through the second processing layer; A training prediction result determining sub-module, configured to determine a training prediction result according to the color training prediction data and the shape training prediction data through the fusion layer; A model updating sub-module, configured to determine a training comparison result according to the training prediction result and a preset true value, and update the artificial intelligence model according to the training comparison result.

[0084] In an embodiment of the present application, the device relates to a BCEwithLogits loss function; the training prediction result includes a predicted logit value; the model updating sub-module includes: A training comparison result determination component for determining a training comparison result by comparing a predicted logit value and a true value using a BCEwithLogits loss function; A model update component for updating the artificial intelligence model according to the training comparison result.

[0085] In an embodiment of the present application, the color feature includes color weight data; the correspondence establishment module 410 includes: A color Euclidean distance calculation sub-module for calculating a color Euclidean distance based on the color data and preset target color data: A color weight data determination sub-module for determining color weight data based on the color Euclidean distance and the maximum possible distance in the color space.

[0086] In an embodiment of the present application, the shape feature includes shape weight data; the correspondence establishment module 410 includes: A shape cosine similarity calculation sub-module for calculating a shape cosine similarity based on the shape data and the target shape data; A shape weight data calculation sub-module for calculating the shape weight data based on the shape cosine similarity.

[0087] In an embodiment of the present application, the correspondence establishment module 410 includes: An overall weight data determination sub-module for determining overall weight data based on the color feature and the shape feature; A correspondence establishment sub-module for determining the defect situation based on the overall weight data and establishing the correspondence.

[0088] In an embodiment of the present application, the device relates to a laser scanning device; the target detection part includes at least 1 production part; the device is used to sequentially collect data on the production parts through the laser scanning device arranged on the production line when the production line produces parts; the image data collection module 420 includes: A laser scanning sub-module for scanning the production parts through the laser scanning device to determine actual color data and actual shape data.

[0089] Refer to Figure 5 , which shows a structural block diagram of a computer device provided by an embodiment of the present application. The computer device 12 is suitable for implementing the embodiments of the present invention and may specifically include the following: The computer device 12 is presented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processing units 16, a system memory 28, and a bus 18 that connects different system components (including the system memory 28 and the processing unit 16). The computer device 12 may be a device attached to the bus.

[0090] The bus 18 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0091] The computer device 12 typically includes a variety of computer system-readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0092] The system memory 28 may include computer system-readable media in the form of volatile memory, such as RAM 30 (Random Access Memory) and / or a cache 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 may be used for reading and writing on a non-removable, non-volatile magnetic medium (commonly referred to as a "hard disk drive"). Although Figure 5 not shown in the figure, a disk drive for reading and writing on a removable non-volatile disk (such as a "floppy disk"), and an optical disk drive for reading and writing on a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 through one or more data media interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the embodiments of the present invention.

[0093] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the system memory 28. Such program modules 42 include - but are not limited to - an operating system, one or more application programs, other program modules, and program data, and the implementation of a network environment may be included in each or some combination of these examples. The program modules 42 generally perform the functions and / or methods in the embodiments described in the present invention.

[0094] The computer device 12 can also communicate with one or more external devices 14 (such as a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a user to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through the I / O interface 22 (Input / Output interface). Moreover, the computer device 12 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network (such as the Internet)) through the network adapter 20. As Figure 5 shown, the network adapter 20 communicates with other modules of the computer device 12 through the bus 18. It should be understood that although Figure 5 not shown in the figure, other hardware and / or software modules can be used in combination with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0095] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the... method provided by any embodiment of the present invention.

[0096] That is, when the program is executed by the processor, it realizes: establishing a correspondence relationship between the color data and shape data of the detection piece and the defect situation of the detection piece through the artificial intelligence model; wherein, extracting color features from the color data through the first processing layer; extracting shape features from the shape data through the second processing layer; determining the correspondence relationship according to the color features and the shape features through the fusion layer; collecting image data of the target detection piece; wherein, the image data includes actual color data and actual shape data; determining the actual defect situation corresponding to the target detection piece according to the actual color data and the actual shape data through the correspondence relationship.

[0097] The computer device 12 is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present invention.

[0098] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it realizes the... method provided by any embodiment of the present application.

[0099] That is, when the program is executed by a processor, it realizes: establishing a correspondence between the color data and shape data of a detection piece and the defect condition of the detection piece through the artificial intelligence model; wherein, extracting color features from the color data through the first processing layer; extracting shape features from the shape data through the second processing layer; determining the correspondence according to the color features and the shape features through the fusion layer; collecting image data of the target detection piece; wherein, the image data includes actual color data and actual shape data; determining the actual defect condition corresponding to the target detection piece according to the actual color data and the actual shape data through the correspondence.

[0100] The computer storage medium may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program may be used by or in combination with an instruction execution system, apparatus, or device.

[0101] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including - but not limited to - an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0102] The program code contained on the computer-readable medium may be transmitted by any appropriate medium, including - but not limited to - wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0103] Computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a LAN or WAN, or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0104] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0105] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the element.

[0106] The above provides a detailed introduction to a method and apparatus for detecting object defects based on dual-stream characteristics. In this article, specific examples are used to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for object defect detection based on dual-stream characteristics, characterized in that: The method relates to an artificial intelligence model; the method is used to detect defects of a target inspection part through the artificial intelligence model; the artificial intelligence model includes: a first processing layer, a second processing layer and a fusion layer; wherein the first processing layer and the second processing layer are independent of each other and are respectively used to process different data streams, and the output results of each are transmitted to the fusion layer; The method comprises: The corresponding relationship between the color data and shape data of the inspection part and the defect condition of the inspection part is established through the artificial intelligence model; wherein the color feature is extracted from the color data through the first processing layer; the shape feature is extracted from the shape data through the second processing layer; and the corresponding relationship is determined according to the color feature and the shape feature through the fusion layer; Collecting image data of the target detection part; wherein the image data includes actual color data and actual shape data; The actual defect situation corresponding to the target inspection part is determined according to the actual color data and the actual shape data through the corresponding relationship.

2. The object defect detection method based on dual-stream characteristics according to claim 1 is characterized in that: The training data set includes a color characteristic training data set and a shape characteristic training data set; The step of establishing a corresponding relationship between the color data and shape data of the inspection piece and the defect condition of the inspection piece by using the artificial intelligence model comprises: Inputting the training data set into the artificial intelligence model for training; wherein, the color training prediction data is determined by the first processing layer according to the color characteristic training data set; Determining shape training prediction data based on the shape characteristic training data set through the second processing layer; Determine a training prediction result according to the color training prediction data and the shape training prediction data through the fusion layer; A training comparison result is determined based on the training prediction result and a preset true value, and the artificial intelligence model is updated based on the training comparison result.

3. The object defect detection method based on dual-stream characteristics according to claim 2 is characterized in that: The method involves a BCEwithLogits loss function; the training prediction result includes a predicted logit value; The step of determining the training comparison result based on the training prediction result and the preset true value, and updating the artificial intelligence model based on the training comparison result includes: The training comparison result is determined by comparing the predicted logit value and the true value through the BCEwithLogits loss function; The artificial intelligence model is updated according to the training comparison result.

4. The object defect detection method based on dual-stream characteristics according to claim 1, characterized in that: The color features include color weight data; The step of extracting color features from the color data comprises: The color Euclidean distance is calculated based on the color data and the preset target color data: Color weight data is determined based on the color Euclidean distance and the maximum possible distance in the color space.

5. The object defect detection method based on dual-stream characteristics according to claim 1, characterized in that: The shape features include shape weight data; The step of extracting shape features from the shape data comprises: Calculating shape cosine similarity based on the shape data and the target shape data; The shape weight data is calculated according to the shape cosine similarity.

6. The object defect detection method based on dual-flow characteristics according to any one of claims 1 and 4-5, characterized in that: The step of determining the corresponding relationship according to the color feature and the shape feature comprises: Determining overall weight data according to the color feature and the shape feature; The defect situation is determined according to the overall weight data, and the corresponding relationship is established.

7. The object defect detection method based on dual-stream characteristics according to claim 1, characterized in that: The method involves a laser scanning device; the target detection part includes at least one production part; the method is used to collect data of the production parts in sequence by using the laser scanning device arranged on the production line when the production line produces the parts; The step of collecting image data of the target detection part includes: The production part is scanned by the laser scanning device to determine the actual color data and the actual shape data.

8. An object defect detection device based on dual-flow characteristics, characterized in that: The device relates to an artificial intelligence model; the device is used to detect defects of a target inspection part through the artificial intelligence model; The artificial intelligence model includes: a first processing layer, a second processing layer and a fusion layer; wherein the first processing layer and the second processing layer are independent of each other and are used to process different data streams respectively, and their respective output results are transmitted to the fusion layer; The device comprises: A correspondence establishing module, used to establish a correspondence between the color data and shape data of the inspection part and the defect condition of the inspection part through the artificial intelligence model; wherein the color feature is extracted from the color data through the first processing layer; the shape feature is extracted from the shape data through the second processing layer; and the correspondence is determined according to the color feature and the shape feature through the fusion layer; An image data acquisition module, used to acquire image data of the target detection component; wherein the image data includes actual color data and actual shape data; An actual defect condition determination module is used to determine the actual defect condition corresponding to the target inspection part according to the actual color data and the actual shape data through the corresponding relationship.

9. A computer device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the object defect detection method based on dual-stream characteristics as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the object defect detection method based on dual-stream characteristics as described in any one of claims 1 to 7 is implemented.

Images