Defect detection method and device, computer equipment and storage medium

Abstract

The invention provides a flaw detection method and device, computer equipment and a storage medium. The method comprises the following steps: mapping all second feature data of a workpiece training sample to nodes corresponding to a graph structure; according to a preset random walk rule, carrying out walk in the mapped graph structure to generate a node sequence; obtaining a first vector representation of a plurality of target workpiece samples in the node sequence; determining the same feature data between any two target workpiece samples in the plurality of target workpiece samples, and enabling the minimum mean square error of the same feature data to be minimum so as to adjust a plurality of second vector representations in each target workpiece sample; forming a new first vector representation of the target workpiece sample by using the plurality of adjusted second vector representations, and performing multi-round iteration to obtain a workpiece flaw detection model; and determining whether the to-be-detected workpiece sample has flaws or not through the workpiece flaw detection model. The workpiece flaw detection accuracy can be greatly improved.

Description

Technical Field

[0001] This patent relates to the field of model training technology, and specifically to a defect detection method, device, computer equipment and storage medium. Background Art

[0002] In the industrial manufacturing process, the detection of surface defects of workpieces is an important part of ensuring product quality.

[0003] As the manufacturing industry's demand for automation and high-precision inspection increases, defect detection methods based on computer vision and machine learning are gradually becoming mainstream. However, at present, defect detection methods based on computer vision and machine learning can only perform defect detection based on a single industrial data of the workpiece; however, in actual implementation scenarios, the complexity and diversity of industrial data may lead to a significant reduction in the detection accuracy of defect detection methods. Summary of the invention

[0004] In view of this, this patent proposes a defect detection method, device, computer equipment and storage medium to solve the problem that the detection accuracy of workpiece defect detection in related technologies is greatly reduced.

[0005] The first embodiment of the present invention provides a defect detection method, the method comprising: Acquire a plurality of first feature data of a workpiece training sample, each of the first feature data includes a plurality of second feature data, and the plurality of second feature data of each first feature data are feature data of a plurality of types of the corresponding first feature data; According to the correlation of all the second feature data, all the second feature data of the workpiece training sample are mapped to the nodes corresponding to the graph structure, and the connection lines of the nodes in the graph structure represent the relationship of the second feature data connected to each other; Acquire a workpiece sample set, wherein the workpiece sample set includes a plurality of workpiece samples related to the first feature data and the second feature data; For any one of the artifact sample sets, walk in the mapped graph structure according to a preset random walk rule to generate a node sequence of the artifact sample; the node sequence includes a plurality of nodes arranged in a preset order, the plurality of nodes correspond to a plurality of target artifact samples one by one, a first target artifact sample among the plurality of target artifact samples is associated with the artifact sample, and a previous target artifact sample among the plurality of target artifact samples is associated with a next target artifact sample; Acquire a first vector representation of a plurality of target workpiece samples in the node sequence; the first vector representation of each target workpiece sample includes a plurality of feature data of the target workpiece sample; Determine the same feature data between any two target workpiece samples among the plurality of target workpiece samples to adjust a plurality of second vector representations in each target workpiece sample; each second vector representation refers to a vector representation of corresponding feature data of a corresponding target workpiece sample; The adjusted multiple second vector representations are used to form a new first vector representation of the target workpiece sample, and multiple rounds of iterations are performed to obtain a workpiece defect detection model; The workpiece defect detection model is used to determine whether the workpiece sample to be detected has defects.

[0006] The embodiment of the present patent can fully utilize the advantages of multi-source heterogeneous data and greatly improve the accuracy of industrial workpiece defect detection by mapping all second feature data of the workpiece training sample to the nodes corresponding to the graph structure; preferably, for any workpiece sample in the workpiece sample set, a walk is performed in the mapped graph structure according to a preset random walk rule to generate a node sequence of the workpiece sample, and the node sequence is used to train the model to be trained, eliminating the defect that the traditional method can only obtain a single type of variable node vector representation, making full use of the correlation between the nodes in the workpiece collection data, and effectively improving the defect detection effect of the algorithm.

[0007] In the embodiment of this patent, the method further includes: Acquire a plurality of first workpiece test samples and a defect label of each first workpiece test sample; any defect label is used to indicate whether the corresponding first workpiece test sample has a defect; For any first workpiece test sample among the plurality of first workpiece test samples, screening out at least one second workpiece test sample related to the first workpiece test sample from a historical sample database; The workpiece defect detection model is verified through each first workpiece test sample or each second workpiece test sample.

[0008] In an embodiment of the present invention, a historical sample database includes a plurality of historical artifact samples and a plurality of first feature data of each historical artifact sample; for any one of the plurality of historical artifact samples, at least one second artifact test sample related to the first artifact test sample is screened out from the historical sample database, including: If at least one first feature data among the plurality of first feature data of the historical artifact sample is the same as the first artifact test sample, taking the historical artifact sample as a first target test sample; If at least one first feature data among the plurality of first feature data of the historical artifact sample is the same as the first target test sample, taking the historical artifact sample as a second target test sample; Based on a plurality of first target test samples and a plurality of second target test samples, at least one second workpiece test sample related to the first workpiece test sample is obtained.

[0009] The embodiment of this patent selectively selects relevant samples from historical samples as test samples, thereby improving the computational efficiency of the workpiece defect detection model and reducing the complexity of the algorithm.

[0010] In the embodiment of the present invention, the workpiece defect detection model is verified by each first workpiece test sample or each second workpiece test sample, including: Inputting a plurality of second feature data of each first workpiece test sample into the workpiece defect detection model, and outputting a first defect detection result; Inputting a plurality of second feature data of each second workpiece test sample into the workpiece defect detection model, and outputting a second defect detection result; If the first defect detection result is inconsistent with the defect label of the first workpiece test sample, or if the second defect detection result is inconsistent with the defect label of the second workpiece test sample, the workpiece defect detection model is optimized.

[0011] The embodiment of this patent can determine whether the detection result of the model is accurate by inputting the characteristic data of the test sample into the model and comparing it with the defect label. If inconsistency is found, that is, the model fails to correctly identify the defect, the parameters and structure can be adjusted through model optimization to improve the accuracy of detection; model optimization can not only be adjusted for the current test sample, but also improve the generalization ability of the model to unknown data through adjustment, so that it can accurately detect defects when facing new workpiece samples in the future, thereby improving product quality and production efficiency.

[0012] In the embodiment of the present patent, the plurality of first feature data include image data, process data, log data and sensor data; The second characteristic data corresponding to the image data include at least one of a high-resolution image, a multi-spectral image, a high-speed camera image, a real-time video image and a three-dimensional image; the second characteristic data corresponding to the process data include at least one of temperature, humidity, pressure and production line speed; the second characteristic data corresponding to the log data include at least one of an operation log and a production log; the second characteristic data corresponding to the sensor data include at least one of vibration sensor data and acoustic sensor data.

[0013] In the embodiment of the present invention, walking in the mapped graph structure according to a preset random walk rule to generate a node sequence of the workpiece sample includes: Determine a target workpiece sample associated with the workpiece sample in a workpiece sample database according to a preset random walk rule; Taking the target workpiece sample as a new workpiece sample, and executing the step of determining a target workpiece sample associated with the workpiece sample in a workpiece sample database according to a preset random walk rule until a plurality of target workpiece samples are obtained; The node sequence is generated according to the obtained multiple target workpiece samples.

[0014] In an embodiment of the present invention, a target workpiece sample associated with the workpiece sample is determined in a workpiece sample database according to a preset random walk rule, including: If at least one feature data among the plurality of feature data of the workpiece sample is the same as feature data of at least one workpiece sample in the workpiece sample database, the at least one workpiece sample is used as the target workpiece sample; If the at least one workpiece sample includes two or more workpiece samples, one workpiece sample is randomly selected from the two or more workpiece samples as the target workpiece sample.

[0015] The second embodiment of the present invention provides a defect detection device, comprising: A data acquisition module, used to acquire a plurality of first feature data of a workpiece training sample, each of the first feature data includes a plurality of second feature data, and the plurality of second feature data of each first feature data are feature data of a plurality of types of the corresponding first feature data; A data mapping module, used to map all the second feature data of the workpiece training sample to the nodes corresponding to the graph structure according to the correlation of all the second feature data, wherein the connection lines of the nodes in the graph structure represent the relationship of the second feature data connected to each other; A sample set acquisition module, used for acquiring a workpiece sample set, wherein the workpiece sample set includes a plurality of workpiece samples related to the first feature data and the second feature data; a node sequence generation module, for generating a node sequence of any one of the artifact samples in the artifact sample set by walking in the mapped graph structure according to a preset random walk rule; the node sequence comprises a plurality of nodes arranged in a preset order, the plurality of nodes correspond to a plurality of target artifact samples one by one, the first target artifact sample in the plurality of target artifact samples is associated with the artifact sample, and the previous target artifact sample in the plurality of target artifact samples is associated with the next target artifact sample; A first vector representation acquisition module is used to acquire first vector representations of a plurality of target workpiece samples in the node sequence; the first vector representation of each target workpiece sample includes a plurality of feature data of the target workpiece sample; a second vector representation determination module, configured to determine the same feature data between any two target workpiece samples among the plurality of target workpiece samples, so as to adjust a plurality of second vector representations in each target workpiece sample; each second vector representation refers to a vector representation of the corresponding feature data of the corresponding target workpiece sample; A model training module, used to form a new first vector representation of a target workpiece sample by combining the adjusted multiple second vector representations, and perform multiple rounds of iterations to obtain a workpiece defect detection model; The detection module is used to determine whether the workpiece sample to be detected has defects through the workpiece defect detection model.

[0016] An embodiment of the third aspect of this patent provides an electronic device, which includes a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method described in the first aspect above by executing the computer instructions.

[0017] The embodiment of the fourth aspect of this patent provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the method described in the first aspect above.

[0018] Additional aspects and advantages of this patent will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of this patent.

[0019] The embodiments of this patent have the following technical effects: 1. A two-layer iterative dynamic graph representation learning method is proposed. It innovatively learns the vector representation of a single workpiece and simultaneously iteratively updates the node vector representation of different types of workpiece data to build a two-layer iterative neural network model. This algorithm can directly obtain the node vector representation of a single workpiece, eliminating the defect that traditional methods can only obtain node vector representations of a single type of variable, making full use of the correlation of nodes in the workpiece acquisition data, and effectively improving the defect detection effect of the algorithm; 2. Aiming at workpiece defect detection, we innovatively designed a random walk rule based on a single workpiece, and obtained the random walk route as the training text for subsequent graph representation learning; 3. Construct a double-layer loss function representation learning model. The internal nodes use the mean square error to construct the target loss function and adjust the vector representation of the nodes. The external artifacts use the training text of the random walk route to obtain the vector representation of a single artifact. 4. Since there are many historical samples for defect detection, this patent innovatively considers selecting relevant samples from historical samples to build a dynamic heterogeneous network representation learning database when new samples are acquired, thereby improving computing efficiency and reducing algorithm complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiment below. The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present patent. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 A schematic diagram of a defect detection method provided in an embodiment of the present invention is shown; Figure 2 A schematic diagram of a process for classifying feature data provided by an embodiment of the present invention is shown; Figure 3 A schematic diagram of an outer word2vec algorithm provided by an embodiment of the present patent is shown; Figure 4 A schematic diagram of inner node level optimization provided by an embodiment of the present patent is shown; Figure 5 A schematic diagram of test workpiece sample screening provided by an embodiment of the present patent is shown; Figure 6 A schematic diagram of a process flow of a workpiece defect detection method provided by an embodiment of the present invention is shown; Figure 7 A schematic diagram showing the workpiece defect detection effect provided by an embodiment of the present patent; Figure 8 A schematic diagram of the structure of a defect detection device provided by an embodiment of the present invention is shown; Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown; Fig.10 A schematic diagram of a storage medium provided by an embodiment of the present patent is shown. DETAILED DESCRIPTION

[0021] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0022] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this patent should have the common meanings understood by technicians in the field to which this patent belongs.

[0023] The following is an explanation of the technical scenarios involved in the embodiments of this patent.

[0024] In the industrial manufacturing process, the detection of surface defects of workpieces is an important part of ensuring product quality. Traditional detection methods mainly rely on manual inspection, which is inefficient and easily affected by human factors. With the increasing demand for automation and high-precision detection in the manufacturing industry, defect detection methods based on computer vision and machine learning have gradually become a research hotspot. However, the complexity and diversity of industrial data have brought challenges to automated detection. Surface defect detection of workpieces involves multi-source heterogeneous data, which includes both unstructured data (such as image data) and structured data (such as sensor data and process data of production records).

[0025] In order to effectively fuse multi-source heterogeneous data, graph representation learning methods are widely used. Graph representation learning constructs a graph model to represent different types of nodes (such as image features, sensor readings, process parameters, production logs, etc.) as vectors. These node vectors not only contain their own data features, but also express the relationship between nodes through the graph structure. Through technologies such as graph neural networks (GNN), these heterogeneous nodes can be effectively learned and fused in high-dimensional space, thereby improving the accuracy and robustness of defect detection. Methods based on graph representation learning can fully utilize the advantages of multi-source heterogeneous data and provide more accurate and intelligent solutions for industrial workpiece defect detection.

[0026] At present, there have been some research results on graph representation learning, but they are basically applied in the fields of paper classification, and there is no method for workpiece defect detection. In addition, traditional graph representation learning methods are based on the node level. The embodiments of this patent are aimed at defect detection and need to obtain vector representations based on the integrity of industrial data. At the same time, some of the existing dynamic heterogeneous network representation learning methods are not based on random walk algorithms, and therefore are not suitable for application in the field of defect detection involving multi-source heterogeneous data. The main problem solved by this patent is to design random walk rules suitable for overall workpiece defect detection for the data collected from the workpiece, construct a two-layer dynamic heterogeneous network representation learning model, iterate the vector weights of the case while updating the vector representation of a single node, obtain the vector representation of the overall parameters of the workpiece, and realize the defect detection and identification of the workpiece.

[0027] According to an embodiment of the present patent, an embodiment of a defect detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] In this embodiment, a defect detection method is provided. Figure 1 is a flow chart of a defect detection method according to an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps: Step S101, obtaining a plurality of first feature data of a workpiece training sample.

[0029] In the embodiment of the present patent, each first feature data includes multiple second feature data, and the multiple second feature data of each first feature data are feature data of multiple types of the corresponding first feature data.

[0030] In some specific embodiments, the first feature data includes but is not limited to image data, process data, log data, sensor data, and workpiece data, such as Figure 2 shown.

[0031] In some specific embodiments, the second characteristic data corresponding to the image data includes at least one of a high-resolution image, a multi-spectral image, a high-speed camera image, a real-time video image, and a three-dimensional image; the second characteristic data corresponding to the process data includes at least one of temperature, humidity, pressure, and production line speed; the second characteristic data corresponding to the log data includes at least one of an operation log and a production log; the second characteristic data corresponding to the sensor data includes at least one of vibration sensor data and acoustic sensor data; the second characteristic data corresponding to the workpiece data includes at least one of a workpiece number and a material composition.

[0032] Step S102: Mapping all the second feature data of the workpiece training sample to nodes corresponding to the graph structure according to the correlation of all the second feature data.

[0033] In an embodiment of the present patent, the mapped graph structure includes multiple nodes and connections between the nodes; wherein each node represents corresponding second feature data, and the connection between each node represents the relationship between the second feature data connected to each other.

[0034] Step S103, obtaining a workpiece sample set.

[0035] In an embodiment of the present patent, the acquired workpiece sample set includes a plurality of workpiece samples associated with the first feature data and the second feature data.

[0036] Step S104 : for any one of the artifact samples in the artifact sample set, walk in the mapped graph structure according to a preset random walk rule to generate a node sequence of the artifact sample.

[0037] In the embodiment of the present invention, the node sequence includes a plurality of nodes arranged in a preset order, the plurality of nodes correspond to a plurality of target workpiece samples one by one, the first target workpiece sample in the plurality of target workpiece samples is associated with the workpiece sample, and the previous target workpiece sample in the plurality of target workpiece samples is associated with the next target workpiece sample. For example: the node sequence is A→B→C→D, wherein the target workpiece sample A is associated with the target workpiece sample B, the target workpiece sample B is associated with the target workpiece sample C, and the target workpiece sample C is associated with the target workpiece sample D.

[0038] In some specific embodiments, the above step S104 includes steps S1041 to S1043: Step S1041 : determining a target workpiece sample associated with the workpiece sample in a workpiece sample database according to a preset random walk rule.

[0039] In some specific embodiments, the following steps S10411-S10412 may be performed to determine a target workpiece sample associated with the workpiece sample in the workpiece sample database according to a preset random walk rule: Step S10411: if at least one feature data among the plurality of feature data of the workpiece sample is the same as feature data of at least one workpiece sample in the workpiece sample database, the at least one workpiece sample is used as the target workpiece sample.

[0040] Step S10412: If the at least one workpiece sample includes two or more workpiece samples, randomly select one workpiece sample from the two or more workpiece samples as the target workpiece sample.

[0041] In the embodiment of this patent, the above steps S10411-S10412 are described by way of example: The workpiece sample database contains multiple workpiece samples, each of which has multiple feature data (such as image data, process data, log data, and sensor data); for any workpiece sample A in the workpiece sample set, if the workpiece sample A has at least one identical feature data with a workpiece sample B in the workpiece sample database, then the workpiece sample B in the workpiece sample database can be used as a target workpiece sample; if the workpiece sample A has at least one identical feature data with workpiece samples B, C, and D in the workpiece sample database, then one of the workpiece samples B, C, and D can be randomly selected as a target workpiece sample for walking; Among them, the transition probability of the random walk step i is defined as follows:

[0042] in, represents the workpiece sample node of the i-th step, Indicates that the artifact sample node type is The number of neighbor nodes, is the random walk rule.

[0043] Step S1042 , taking the target workpiece sample as a new workpiece sample, and executing the step of determining a target workpiece sample associated with the workpiece sample in a workpiece sample database according to a preset random walk rule, until a plurality of target workpiece samples are obtained.

[0044] Step S1043: generating the node sequence according to the obtained multiple target workpiece samples.

[0045] The above steps S1041 to S1043 are explained by the following specific examples: When artifact sample A and artifact sample B in the artifact sample database have at least one identical feature data, artifact sample B is used as the first node in the node sequence; then determine whether artifact sample B has the same feature data as other artifact samples in the artifact sample database. If there are other artifact samples C, D and E that have the same feature data as B, then randomly select another artifact sample as the second node in the node sequence, and then find the remaining nodes in the node sequence in the above manner, and stop when no artifact sample with the same feature data is found.

[0046] Step S105 , obtaining first vector representations of a plurality of target workpiece samples in the node sequence.

[0047] In an embodiment of the present patent, the first vector representation of each target workpiece sample includes multiple feature data of the target workpiece sample.

[0048] In some specific embodiments, the first vector representation of each workpiece sample can be obtained based on the word2vec method, such as Figure 3 As shown. Each artifact sample consists of 4 nodes. The main goal of the solution is to maximize the conditional probability of neighbor samples given the artifact sample. The reason is that these artifact training samples are obtained by random walks and are correlated. Therefore, the maximum likelihood function can be used for optimization. The specific calculation formula is as follows:

[0049] in Refers to the workpiece sample v No. t types of neighbor samples, It is the softmax function, and the calculation formula is as follows.

[0050]

[0051] in Refers to the workpiece sample v The vector representation of .

[0052] At this point, the first vector representation of each workpiece sample as a whole in one iteration process has been obtained.

[0053] Step S106 , determining the same feature data between any two target workpiece samples among the multiple target workpiece samples to adjust multiple second vector representations in each target workpiece sample; each second vector representation refers to a vector representation of corresponding feature data of a corresponding target workpiece sample.

[0054] In the embodiment of the present invention, since the same nodes exist in different workpiece samples, the vector representations of the same nodes obtained in different workpiece samples should be the same. Therefore, the mean square error of the same nodes of different workpiece samples should be minimized during the iteration process. The loss function is as follows:

[0055] in, n It is variable, depending on the number of different nodes in different artifact samples. Since each artifact sample contains 5 nodes, the target loss function needs to consider 5 different nodes in total, and the number of repetitions of each node is different. y 1 is the vector representation of each node appearing in the first artifact sample, which is a vector, y i Each node appears in i The vector representation of the workpiece samples is also a vector, such as Figure 4 shown.

[0056] Step S107 , the adjusted multiple second vector representations are used to form a new first vector representation of the target workpiece sample, and multiple rounds of iterations are performed to obtain a workpiece defect detection model.

[0057] Specifically, after the adjusted multiple second vector representations are combined into a new first vector representation of the target workpiece sample, it is necessary to return to step S104 for a new round of iteration. In this way, reverse iterations are continuously performed to reduce the gradient of the loss function of the workpiece defect detection model.

[0058] Step S108: determining whether the workpiece sample to be inspected has defects by using the workpiece defect detection model.

[0059] In the embodiment of the present patent, according to the vector representation of each workpiece sample, a normal workpiece sample training model is used to calculate whether the Hotelling T2 statistic result of the test sample exceeds the normal threshold. If it exceeds the normal threshold, it is determined that the test sample has defects, so as to realize defect detection of the workpiece sample.

[0060] In some specific embodiments, the method further comprises steps b1 to b3: Step b1, obtaining a plurality of first workpiece test samples and a defect label of each first workpiece test sample.

[0061] Specifically, any defect label is used to indicate whether the corresponding first workpiece test sample has a defect; the collected multiple first workpiece test samples include a portion of normal workpiece samples and a portion of defective workpiece samples, the normal workpiece samples are marked with a first defect label to indicate that the workpiece sample is normal; the defective workpiece samples are marked with a second defect label to indicate that the workpiece sample has a defect. The defect here can be understood as: unqualified parts of the workpiece, such as: size deviation, shape deviation, cracks, dents, burrs, spots on the workpiece surface, etc.

[0062] Step b2: for any first workpiece test sample among the plurality of first workpiece test samples, screen out at least one second workpiece test sample related to the first workpiece test sample from a historical sample database.

[0063] In an embodiment of the present patent, the historical sample database includes multiple historical artifact samples and multiple first feature data of each historical artifact sample.

[0064] In some specific embodiments, the above step b2 includes steps b21 to b23: Step b21: If at least one first feature data among the plurality of first feature data of the historical artifact sample is the same as the first artifact test sample, then the historical artifact sample is used as a first target test sample.

[0065] In the embodiment of the present patent, both the historical workpiece sample and the first workpiece test sample include a plurality of first feature data, such as image data, process data, log data, and sensor data; if any one of the four pieces of information is the same between the historical workpiece sample and the first workpiece test sample, it is determined that the historical workpiece sample and the first workpiece test sample are related, and therefore, the historical workpiece sample can be used as the first target test sample, for example Figure 5 "Related first-category artifacts" in.

[0066] Step b22: If at least one first feature data among the plurality of first feature data of the historical artifact sample is the same as the first target test sample, then the historical artifact sample is used as a second target test sample.

[0067] In the embodiment of the present invention, the above method can be used to determine whether the historical artifact sample is used as the second target test sample, that is, the four information of the historical artifact sample and the first target test sample are compared. If any one of the four information is the same, it is determined that the historical artifact sample and the first target test sample are related, and the historical artifact sample can be used as the second target test sample. For example Figure 5 "Related second category artifacts" in.

[0068] Step b23: obtaining at least one second workpiece test sample related to the first workpiece test sample based on the plurality of first target test samples and the plurality of second target test samples.

[0069] Specifically, after obtaining the first target test sample and the second target test sample, these two types of samples can be integrated as the second workpiece test sample related to the first workpiece test sample, for example Figure 5 Dynamic Artifact Selection in .

[0070] Step b3: verifying the workpiece defect detection model through each first workpiece test sample or each second workpiece test sample.

[0071] In the embodiments of this patent, the detection effect of the workpiece defect detection model can be verified by any one or more of the following methods: Inputting a plurality of second feature data of each first workpiece test sample into the workpiece defect detection model, and outputting a first defect detection result; Inputting a plurality of second feature data of each second workpiece test sample into the workpiece defect detection model, and outputting a second defect detection result; If the first defect detection result is inconsistent with the defect label of the first workpiece test sample, or if the second defect detection result is inconsistent with the defect label of the second workpiece test sample, the workpiece defect detection model is optimized.

[0072] The embodiment of the present invention also provides a specific embodiment to illustrate the detection of workpiece defects, for example Figure 6 As shown: The effect of the algorithm was verified by collecting data from real workpieces. Data from 1,200 workpieces were collected, of which 1,000 were manually marked as intact workpieces and the remaining 200 were marked as defective workpieces. 800 normal samples were used as training matrices, and 200 normal samples and 200 defective samples were used as test matrices. In this patent, each workpiece uses four features as workpiece nodes: images taken by industrial cameras, workpiece production logs, sensor parameters of the production process, and process setting parameters.

[0073] The implementation process is divided into several steps, the specific process is as follows; Step 1: Random walk path design The defect detection of workpieces generally collects data from industrial camera image data, text log records, production process parameters, production processing sensor parameters, etc., including high-resolution images, high-speed camera shooting parameters, event text recorded by operators, log files generated by machines during the production process, temperature and humidity detected by sensors, production equipment pressure values, set production speeds, workpiece size specifications, etc. Figure 2 As shown. This patent uses four variables: workpiece image, log file, sensor temperature and humidity, and set production speed to detect workpiece defects. Based on expert knowledge and considering that workpiece defects may have clustering characteristics, the random walk rule is designed as workpiece image-log file-sensor temperature and humidity-set production speed-sensor temperature and humidity-log file-workpiece image.

[0074] Step 2: Obtain training and test artifacts The training artifacts are 800 samples labeled as normal; The test artifact acquisition is to first select the historical samples related to the newly collected samples as candidate first-class samples, and similarly select the historical samples related to the candidate first-class samples as candidate second-class samples. The two types of samples are integrated as a database for subsequent dynamic sample updates, namely the test artifact, such as Figure 5 shown.

[0075] Step 3: Obtain training text The workpiece defect detection method based on graph representation learning requires the construction of training text to obtain vector representation. This patent uses a single workpiece sample as a whole according to the random walk path rule, and searches for the next sample that can be connected in the workpiece sample database according to the random walk rule. Case 1 High-resolution image--Case 2 Log--Case 3 Temperature--Case 4 Humidity--Case 5 Production speed--Case 6 Humidity--Case 7 Temperature--Case 8 Log--Case 9 High-resolution image. It should be noted at this point that if there are multiple random walk samples that can be selected for the next step of the random walk, one of them is randomly selected for the walk, and the transition probability of the i-th step of the random walk is defined as:

[0076] in, Indicates i The workpiece sample node of the step, Indicates that the artifact sample node type is The number of neighbor nodes, is the random walk rule.

[0077] Step 4: Graph Representation Learning Model Construction Based on the random walk routes obtained above, the vector representation of each workpiece sample is obtained based on the word2vec method, as follows: Figure 3 As shown. Each artifact sample consists of 4 nodes. The main goal of the solution is to maximize the conditional probability of neighbor samples given the artifact sample. The reason is that these training texts are obtained by random walks and are relevant. They can be optimized using the maximum likelihood function. The specific calculation formula is as follows:

[0078] in refers to the t-th type of neighbor sample of the artifact sample v, It is the softmax function, and the calculation formula is as follows.

[0079]

[0080] in Refers to the workpiece sample v The vector representation of .

[0081] At this point, the vector representation of each workpiece sample as a whole has been obtained during one iteration. Since the same nodes exist in different samples, the vector representations of the same nodes in different workpiece samples should be the same. Therefore, the mean square error of the same nodes in different workpiece samples should be minimized during the iteration process. The loss function is as follows:

[0082] in, n It is variable, depending on the number of different nodes in different artifact samples. Since each artifact sample contains 5 nodes, the target loss function needs to consider 5 different nodes in total, and the number of repetitions of each node is different. y 1 is the vector representation of each node appearing in the first artifact sample, which is a vector, yi Each node appears in i The vector representation of the workpiece samples is also a vector, such as Figure 4 shown.

[0083] After using the above formula to reduce the loss function, adjust the vector representation of a single node in each overall sample, and return to step 4 to continue using the skip-gram in the Word2vec method to obtain the vector representation of the node at the overall level of the artifact sample.

[0084] Step 5: Workpiece defect detection The training samples and test samples obtained in step 2 are represented according to the node vector of each workpiece sample obtained in steps 3 and 4. The model is trained using normal workpiece samples to calculate whether the Hotelling T2 statistic result of the test sample exceeds the normal threshold, thereby realizing defect detection of workpiece samples.

[0085] In order to verify the effect of the double-layer iterative graph representation learning workpiece sample defect detection algorithm proposed in this patent, the defect detection effect evaluation algorithm is used to finally obtain the effect schematic diagram, as shown in the figure below: Figure 7 shown.

[0086] Among them, the designed hyperparameters can be set as follows: 1) Node vector representation dimension: 64 2) Number of negative samples: 5 3) Gradient descent learning rate: 0.01 4) Random walk path length: 30 5) Number of random walks per node: 10 6) Window size: 7 7) Number of iterations: 5.

[0087] The embodiments of this patent have the following technical effects: 1. A two-layer iterative dynamic graph representation learning method is proposed. It innovatively learns the vector representation of a single workpiece and simultaneously iteratively updates the node vector representation of different types of workpiece data to build a two-layer iterative neural network model. This algorithm can directly obtain the node vector representation of a single workpiece, eliminating the defect that traditional methods can only obtain node vector representations of a single type of variable, making full use of the correlation of nodes in the workpiece acquisition data, and effectively improving the defect detection effect of the algorithm; 2. Aiming at workpiece defect detection, we innovatively designed a random walk rule based on a single workpiece, and obtained the random walk route as the training text for subsequent graph representation learning; 3. Construct a double-layer loss function representation learning model. The internal nodes use the mean square error to construct the target loss function and adjust the vector representation of the nodes. The external artifacts use the training text of the random walk route to obtain the vector representation of a single artifact. 4. Since there are many historical samples for defect detection, this patent innovatively considers selecting relevant samples from historical samples to build a dynamic heterogeneous network representation learning database when new samples are acquired, thereby improving computing efficiency and reducing algorithm complexity.

[0088] Corresponding to the implementation of the above defect detection method, the embodiment of the present invention also provides a defect detection device for performing the above Figures 1 to 7 The defect detection method described in any of the embodiments shown in the figure. Figure 8 As shown, the defect detection device comprises: A data acquisition module, used to acquire a plurality of first feature data of a workpiece training sample, each of the first feature data includes a plurality of second feature data, and the plurality of second feature data of each first feature data are feature data of a plurality of types of the corresponding first feature data; A data mapping module, used to map all the second feature data of the workpiece training sample to the nodes corresponding to the graph structure according to the correlation of all the second feature data, wherein the connection lines of the nodes in the graph structure represent the relationship of the second feature data connected to each other; A sample set acquisition module, used for acquiring a workpiece sample set, wherein the workpiece sample set includes a plurality of workpiece samples related to the first feature data and the second feature data; a node sequence generation module, for generating a node sequence of any one of the artifact samples in the artifact sample set by walking in the mapped graph structure according to a preset random walk rule; the node sequence comprises a plurality of nodes arranged in a preset order, the plurality of nodes correspond to a plurality of target artifact samples one by one, the first target artifact sample in the plurality of target artifact samples is associated with the artifact sample, and the previous target artifact sample in the plurality of target artifact samples is associated with the next target artifact sample; A first vector representation acquisition module is used to acquire first vector representations of a plurality of target workpiece samples in the node sequence; the first vector representation of each target workpiece sample includes a plurality of feature data of the target workpiece sample; a second vector representation determination module, configured to determine the same feature data between any two target workpiece samples among the plurality of target workpiece samples, so as to adjust a plurality of second vector representations in each target workpiece sample; each second vector representation refers to a vector representation of the corresponding feature data of the corresponding target workpiece sample; A model training module, used to form a new first vector representation of a target workpiece sample by combining the adjusted multiple second vector representations, and perform multiple rounds of iterations to obtain a workpiece defect detection model; The detection module is used to determine whether the workpiece sample to be detected has defects through the workpiece defect detection model.

[0089] Optionally, the device further comprises: A test sample acquisition module is used to acquire a plurality of first workpiece test samples and a defect label of each first workpiece test sample; any defect label is used to indicate whether the corresponding first workpiece test sample has a defect; a related sample screening module, configured to screen out at least one second artifact test sample related to any first artifact test sample among the plurality of first artifact test samples from a historical sample database; The model verification module is used to verify the workpiece defect detection model through each first workpiece test sample or each second workpiece test sample.

[0090] Optionally, the related sample screening module is also used to use the historical artifact sample as a first target test sample if at least one of the multiple first feature data of the historical artifact sample is the same as the first artifact test sample; use the historical artifact sample as a second target test sample if at least one of the multiple first feature data of the historical artifact sample is the same as the first target test sample; and obtain at least one second artifact test sample related to the first artifact test sample based on the multiple first target test samples and the multiple second target test samples.

[0091] Optionally, the model verification module is also used to: input multiple second feature data of each first workpiece test sample into the workpiece defect detection model, and output a first defect detection result; input multiple second feature data of each second workpiece test sample into the workpiece defect detection model, and output a second defect detection result; if the first defect detection result is inconsistent with the defect label of the first workpiece test sample, or the second defect detection result is inconsistent with the defect label of the second workpiece test sample, then optimize the workpiece defect detection model.

[0092] Optionally, the node sequence generation module is also used to: determine a target artifact sample associated with the artifact sample in the artifact sample database according to a preset random walk rule; use the target artifact sample as a new artifact sample, and execute the step of determining a target artifact sample associated with the artifact sample in the artifact sample database according to a preset random walk rule until multiple target artifact samples are obtained; and generate the node sequence based on the multiple target artifact samples obtained.

[0093] Optionally, the node sequence generation module is also used to: if at least one feature data among the multiple feature data of the artifact sample is the same as the feature data of at least one artifact sample in the artifact sample database, then the at least one artifact sample is used as the target artifact sample; if the at least one artifact sample includes two or more artifact samples, then randomly select one artifact sample from the two or more artifact samples as the target artifact sample.

[0094] The defect detection device provided in the above-mentioned embodiment of this patent and the defect detection method provided in the embodiment of this patent are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0095] This patent embodiment also provides an electronic device to perform the above-mentioned defect detection method. Please refer to Fig. 9 , which shows a schematic diagram of an electronic device provided by some embodiments of this patent. Fig. 9 As shown, the electronic device 9 includes: a processor 900, a memory 901, a bus 902 and a communication interface 903, wherein the processor 900, the communication interface 903 and the memory 901 are connected via the bus 902; the memory 901 stores a computer program that can be run on the processor 900, and the processor 900 executes the aforementioned Figures 1 to 7 The defect detection method provided by any of the illustrated embodiments.

[0096] The memory 901 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 903 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0097] The bus 902 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 901 is used to store programs. After receiving the execution instruction, the processor 900 executes the program. Figures 1 to 7 The defect detection method disclosed in any of the illustrated embodiments may be applied to the processor 900 or implemented by the processor 900 .

[0098] The processor 900 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 900 or the instruction in the form of software. The above processor 900 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of this patent can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of this patent may be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 901, and the processor 900 reads the information in the memory 901 and completes the steps of the above method in combination with its hardware.

[0099] The electronic device provided by the embodiment of this patent and the defect detection method provided by the embodiment of this patent are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented by them.

[0100] This patent embodiment also provides a computer-readable storage medium corresponding to the defect detection method provided in the above embodiment, please refer to Fig.10 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the defect detection method provided by any of the aforementioned embodiments will be executed.

[0101] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0102] The computer-readable storage medium provided by the above-mentioned embodiment of this patent and the defect detection method provided by the embodiment of this patent are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0103] It should be noted that: In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present invention can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0104] Similarly, it should be understood that in order to streamline the patent and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the patent, the various features of the patent are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following schematic diagram: the claimed patent requires more features than the features explicitly recited in each claim. Rather, as reflected in the claims below, the inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the patent.

[0105] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of this patent and form different embodiments. For example, in the following claims, any one of the claimed embodiments may be used in any combination.

[0106] The above is only a preferred specific implementation of this patent, but the protection scope of this patent is not limited to this. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed in this patent should be covered within the protection scope of this patent. Therefore, the protection scope of this patent should be based on the protection scope of the claims.

Claims

1. A defect detection method, characterized in that: The method comprises: Acquire a plurality of first feature data of a workpiece training sample, each of the first feature data includes a plurality of second feature data, and the plurality of second feature data of each first feature data are feature data of a plurality of types of the corresponding first feature data; According to the correlation of all the second feature data, all the second feature data of the workpiece training sample are mapped to the nodes corresponding to the graph structure, and the connection lines of the nodes in the graph structure represent the relationship of the second feature data connected to each other; Acquire a workpiece sample set, wherein the workpiece sample set includes a plurality of workpiece samples related to the first feature data and the second feature data; For any one of the artifact sample sets, walk in the mapped graph structure according to a preset random walk rule to generate a node sequence of the artifact sample; the node sequence includes a plurality of nodes arranged in a preset order, the plurality of nodes correspond to a plurality of target artifact samples one by one, the first target artifact sample among the plurality of target artifact samples is associated with the artifact sample, and the previous target artifact sample among the plurality of target artifact samples is associated with the next target artifact sample; Acquire a first vector representation of a plurality of target workpiece samples in the node sequence; the first vector representation of each target workpiece sample includes a plurality of feature data of the target workpiece sample; Determine the same feature data between any two target workpiece samples among the plurality of target workpiece samples, and minimize the minimum mean square error of the same feature data, so as to adjust a plurality of second vector representations in each target workpiece sample; each second vector representation refers to a vector representation of the corresponding feature data of the corresponding target workpiece sample; The adjusted multiple second vector representations are used to form a new first vector representation of the target workpiece sample, and multiple rounds of iterations are performed to obtain a workpiece defect detection model; The workpiece defect detection model is used to determine whether the workpiece sample to be detected has defects.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a plurality of first workpiece test samples and a defect label of each first workpiece test sample; any defect label is used to indicate whether the corresponding first workpiece test sample has a defect; For any first workpiece test sample among the plurality of first workpiece test samples, screening out at least one second workpiece test sample related to the first workpiece test sample from a historical sample database; The workpiece defect detection model is verified through each first workpiece test sample or each second workpiece test sample.

3. The method according to claim 2, characterized in that The historical sample database includes a plurality of historical artifact samples and a plurality of first feature data of each historical artifact sample; for any one of the plurality of historical artifact samples, at least one second artifact test sample related to the first artifact test sample is screened out from the historical sample database, comprising: If at least one first feature data among the plurality of first feature data of the historical artifact sample is the same as the first artifact test sample, taking the historical artifact sample as a first target test sample; If at least one first feature data among the plurality of first feature data of the historical artifact sample is the same as the first target test sample, taking the historical artifact sample as a second target test sample; Based on a plurality of first target test samples and a plurality of second target test samples, at least one second workpiece test sample related to the first workpiece test sample is obtained.

4. The method according to claim 2 or 3, characterized in that: The workpiece defect detection model is verified by each first workpiece test sample or each second workpiece test sample, including: Inputting a plurality of second feature data of each first workpiece test sample into the workpiece defect detection model, and outputting a first defect detection result; Inputting a plurality of second feature data of each second workpiece test sample into the workpiece defect detection model, and outputting a second defect detection result; If the first defect detection result is inconsistent with the defect label of the first workpiece test sample, or if the second defect detection result is inconsistent with the defect label of the second workpiece test sample, the workpiece defect detection model is optimized.

5. The method according to claim 1 or 2, characterized in that: The plurality of first feature data include image data, process data, log data and sensor data; The second characteristic data corresponding to the image data include at least one of a high-resolution image, a multi-spectral image, a high-speed camera image, a real-time video image and a three-dimensional image; the second characteristic data corresponding to the process data include at least one of temperature, humidity, pressure and production line speed; the second characteristic data corresponding to the log data include at least one of an operation log and a production log; the second characteristic data corresponding to the sensor data include at least one of vibration sensor data and acoustic sensor data.

6. The method according to claim 1 or 2, characterized in that: Walking in the mapped graph structure according to a preset random walk rule to generate a node sequence of the workpiece sample includes: Determine a target workpiece sample associated with the workpiece sample in a workpiece sample database according to a preset random walk rule; Taking the target workpiece sample as a new workpiece sample, and executing the step of determining a target workpiece sample associated with the workpiece sample in a workpiece sample database according to a preset random walk rule until a plurality of target workpiece samples are obtained; The node sequence is generated according to the obtained multiple target workpiece samples.

7. The method according to claim 6, characterized in that Determining a target workpiece sample associated with the workpiece sample in a workpiece sample database according to a preset random walk rule includes: If at least one feature data among the plurality of feature data of the workpiece sample is the same as feature data of at least one workpiece sample in the workpiece sample database, the at least one workpiece sample is used as the target workpiece sample; If the at least one workpiece sample includes two or more workpiece samples, one workpiece sample is randomly selected from the two or more workpiece samples as the target workpiece sample.

8. A defect detection device, characterized in that: The device comprises: A data acquisition module, used to acquire a plurality of first feature data of a workpiece training sample, each of the first feature data includes a plurality of second feature data, and the plurality of second feature data of each first feature data are feature data of a plurality of types of the corresponding first feature data; A data mapping module, used to map all the second feature data of the workpiece training sample to the nodes corresponding to the graph structure according to the correlation of all the second feature data, wherein the connection lines of the nodes in the graph structure represent the relationship of the second feature data connected to each other; A sample set acquisition module, used for acquiring a workpiece sample set, wherein the workpiece sample set includes a plurality of workpiece samples related to the first feature data and the second feature data; a node sequence generation module, for generating a node sequence of any one of the artifact samples in the artifact sample set by walking in the mapped graph structure according to a preset random walk rule; the node sequence comprises a plurality of nodes arranged in a preset order, the plurality of nodes correspond to a plurality of target artifact samples one by one, the first target artifact sample in the plurality of target artifact samples is associated with the artifact sample, and the previous target artifact sample in the plurality of target artifact samples is associated with the next target artifact sample; A first vector representation acquisition module is used to acquire first vector representations of a plurality of target workpiece samples in the node sequence; the first vector representation of each target workpiece sample includes a plurality of feature data of the target workpiece sample; a second vector representation determination module, configured to determine the same feature data between any two target workpiece samples among the plurality of target workpiece samples, so as to adjust a plurality of second vector representations in each target workpiece sample; each second vector representation refers to a vector representation of the corresponding feature data of the corresponding target workpiece sample; A model training module, used to form a new first vector representation of a target workpiece sample by combining the adjusted multiple second vector representations, and perform multiple rounds of iterations to obtain a workpiece defect detection model; The detection module is used to determine whether the workpiece sample to be detected has defects through the workpiece defect detection model.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images