Part defect detection method and system

Abstract

The invention relates to a part defect detection method and system, and relates to the technical field of part defect detection. Comprising the steps of classifying and sorting a part processing library by using a part information sorting module, and constructing a generative adversarial network by using a sample image expansion module, generating a defect sample image to expand a defect sample database, constructing an image defect recognition model by utilizing an image recognition module to perform defect recognition, and performing defect state scoring and grade division operation on the part by utilizing a defect grade determination module. The defect sample database is expanded by constructing the generative adversarial network, data guarantee is provided for subsequent recognition, and recognition efficiency and accuracy are improved; besides, based on the score of the related defect of each feature surface, the state of the part is evaluated on the whole, and a data basis is provided for the subsequent defect grade and degree of the part, so that the part is reasonably defined, the treatment pertinence of the defective part is improved, the invalid loss is reduced, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of part defect detection, and particularly to a part defect detection method and system. Background Art

[0002] In the current industrial environment, part defect detection faces many challenges. For example, there is a wide variety of parts with different shapes, and the forms of defects are also diverse, such as cracks, pores, corrosion, wear, etc. These defects will not only affect the performance of the parts, but may even lead to the failure of the entire equipment. Therefore, how to quickly and accurately detect these defects has become the key to improving product quality and production efficiency.

[0003] Traditional part detection methods often rely on manual inspection or simple measuring tools. These methods are not only inefficient but also error-prone, and cannot meet the needs of large-scale and high-precision production. With the rapid development of computer vision and artificial intelligence technologies, image-based part defect detection technologies have gradually emerged. However, due to the imperfect defect samples and single detection logic, the defect detection accuracy of parts is often not high, and it is limited to the evaluation of local defects of parts, lacking the evaluation of overall defects of parts, resulting in the direct scrapping of parts that have defects but can still be used, increasing costs. For this reason, we propose a part defect detection method and system. Summary of the Invention

[0004] The purpose of the present invention is to provide a part defect detection method and system to solve the above problems.

[0005] The present invention can be realized through the following technical solutions: A part defect detection method, comprising the following steps:

[0006] Step 1: Classify based on the structure, material, and functional characteristics of the parts, and thus mark each feature surface of the same type of parts and determine common defect types;

[0007] Step 2: Construct a generative adversarial network combined with an attention mechanism to generate defect image samples, thereby expanding the defect sample database of various parts, and marking the defect positions of the defect sample images in the defect sample database and setting defect type labels;

[0008] Step 3: Construct an image defect recognition model based on a convolutional neural network and use this model to perform defect recognition on the actual shot part images. This model takes the defect sample images in the defect sample database as input and outputs the part type, defect position, and corresponding defect type;

[0009] Step 4: Conduct real-time detection, generate part detection serial numbers based on time series coding for subsequent screening, and construct a defect identification form. Each row of the form is set with multiple characterization elements, including: part type, batch number, presence or absence of defects, number of defects, defect location, and defect type;

[0010] Step 5: Manually review and confirm the defect identification form, import the mis-identified images into the generative adversarial network to generate similar image samples, and train and update the image defect recognition model;

[0011] Step 6: Classify the defects of similar parts in the same batch in the defect identification form, obtain the defect status score of the part based on parameters such as the part feature surface, defect type, number of defects, and area, and classify the parts into different defect levels according to the defect status score. Define the defect degree of the part according to different levels, and take repair or recycling measures according to the defect degree.

[0012] A further technical improvement of the present invention is that the generative adversarial network includes a generator and a discriminator that oppose each other. The objective function of the discriminator is expressed as:

[0013]

[0014] Among them, V(G,D) is the value function, which enables the generator and the discriminator to conduct a minimax game. E is the expected distribution. G and D respectively represent the discriminator and the generator. x represents the sampled real image sample. D(~) is the output of the discriminator. p d (x) represents the real data distribution, z represents the random noise, and p z (z) represents the noise distribution, and G(z) represents the noise image sample output by the generator;

[0015] The objective function of the generator is expressed as:

[0016] In summary, the objective function of the generative adversarial network satisfies:

[0017]

[0018] When the input is a real image sample x, the expected output of the discriminant function D(x) is 1. When the input is the noise image generated by the generator, the expected output of the discriminant function D(G(z)) is 0. When training the generator, the discriminator does not participate in the update, and the expected output of the discriminant function D(G(z)) is 1. Finally, it reaches the Nash equilibrium state, and for both real image samples and noise image samples, the prediction probability is one-half.

[0019] A further technical improvement of the present invention lies in: introducing a spatial - position - based attention mechanism into the generator to construct a spatial attention module; the convolutional layer of the generator extracts image feature elements from the input real - sample image data, and fuses the extracted image feature elements into the part background image based on the part standard background to obtain a fused image, and then passes it to the spatial attention module. The spatial attention module assigns weights to pixel positions to obtain a fused sample image with fused position weights, that is, the final defective sample image.

[0020] A further technical improvement of the present invention lies in: a 1x1 convolutional layer is set in the spatial attention module to map the fused image to a new feature space and another convolutional layer is used to generate a spatial attention weight matrix. The spatial size of the weight matrix is the same as that of the fused image, and each element in the weight matrix corresponds to a pixel position. Subsequently, normalization processing is performed, and the normalized weight matrix is multiplied by the fused image to weight each position in the fused image to obtain a weighted fused sample image.

[0021] A further technical improvement of the present invention lies in: the specific steps for obtaining the defect - state score of a part include:

[0022] Set importance - level tag values for each feature surface of the part;

[0023] Combined with feature surfaces of different importance levels and the defect types on these feature surfaces, set corresponding influence factors;

[0024] Calculate the comprehensive scores of all defect types on each feature surface of a certain part respectively and sum them up to obtain the defect - state score of the corresponding part.

[0025] A further technical improvement of the present invention lies in: the calculation formula for the defect - state score is specifically:

[0026]

[0027] where, δ i,k represents the ratio of the total defective area existing in part feature surface k to the overall area of this part feature surface, L k represents the importance - level tag value of part feature surface k; σ i,k represents the influence factor of defect type i in the current part feature surface k; N i represents the number of defect type i in the current feature surface; S i represents the area of each defect type i in the current feature surface; P represents the part - feature - surface number; Q represents the defect - type number.

[0028] A further technical improvement of the present invention lies in: defining multiple defect level scoring thresholds, scoring according to the defect status, and classifying defective parts at the corresponding scoring thresholds into the corresponding defect levels, thereby determining the defect levels of the parts; specifically repairing the parts defined as defective parts and substandard parts according to the defect levels, and recycling and regenerating the parts defined as scrap parts.

[0029] A part defect detection system, comprising:

[0030] A part information sorting module, used to classify and sort the part processing library in the factory, and determine the characteristic surfaces and corresponding defect types of a type of part based on the part shape structure characteristics, material, and function;

[0031] A sample image expansion module, constructing a generative adversarial network integrated with a spatial position attention mechanism to generate defective sample images to expand the defective sample database;

[0032] An image recognition module, constructing an image defect recognition model to recognize defects in the actual part images obtained by machine vision, and obtaining the part type, defect location, and defect type of the defective parts;

[0033] A defect level determination module, constructing a defect recognition form and performing manual confirmation, then scoring and classifying the defect status of the same type of parts in the same batch in the determined form, and defining the defect degree of the parts based on the defect level, and taking different treatment measures according to different defect degrees.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. By constructing a generative adversarial network, the present invention generates defective sample images similar to the real sample images that can pass off as real based on a small number of real defective sample images, thereby expanding the entire defective sample database, providing data support for the construction and training of the subsequent image defect recognition model, and improving the accuracy and efficiency of the entire part defect recognition.

[0036] 2. By introducing a spatial position attention mechanism into the generative adversarial network, the present invention enhances the attention to the working surface and the reference surface, so that the defect generation details of the characteristic surfaces with a greater impact on the part are more accurate, grasps the key points that have a greater impact on defect recognition, and is conducive to improving the accuracy of the defect status scoring of the overall part in the subsequent process;

[0037] 3. The present invention obtains the defect status score of the entire part based on the scores of the relevant defects of each feature surface, providing objective data support for the evaluation of the part defect status, evaluating the status of the part as a whole, providing a data basis for the subsequent defect level and degree of the part, defining the part more reasonably, and then taking different treatment measures, improving the pertinence of part treatment, reducing ineffective losses, and lowering costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] For the convenience of those skilled in the art to understand, the present invention will be further described below with reference to the accompanying drawings.

[0039] Figure 1 It is a schematic flowchart of the method of the present invention;

[0040] Figure 2 It is a system block diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following will be described in detail with reference to the accompanying drawings and preferred embodiments regarding the specific embodiments, structures, features, and effects of the present invention.

[0042] Please refer to Figure 1 As shown, a method for detecting part defects specifically includes the following steps:

[0043] Step 1: Classify the parts and confirm the working surface and defect type

[0044] For the part processing library in the factory, the parts are classified based on the part structure characteristics, material characteristics, and functional characteristics. The structure characteristics and functional characteristics determine the working surface of the part, and the material characteristics determine the different types of defects;

[0045] Mark the feature surfaces of the classified parts of the same type and determine the common defect types of this type.

[0046] Step 2: Construct and expand the defect sample database for various parts

[0047] During the production of various parts, the defect image samples of the parts are relatively lacking, and the data of normal samples and defect samples are unbalanced, which affects the accuracy of subsequent defect recognition; it is necessary to expand the database when the number of defect samples is insufficient;

[0048] This application uses an unsupervised neural network model to construct a generative adversarial network and introduces an attention mechanism, enabling the production adversarial network to focus on the key defect areas of the working surface and the reference surface, improving the ability to extract small target features in the image, and avoiding the loss of key information;

[0049] The generative adversarial network consists of two adversarial sub-networks, namely the generator and the discriminator. Among them, the goal of the generator is to generate fake image samples similar to those in the dataset by learning the features of real samples in the dataset. The discriminator, on the other hand, trains the generator to continuously generate more realistic fake samples by judging real and fake samples.

[0050] When the discriminator can no longer successfully judge whether an image comes from the generator, it is considered that the generator has learned from the training data to map noise to the distribution of real data, that is, the generator can generate images that are indistinguishable from real ones. At this time, an equilibrium is reached between the generator and the discriminator.

[0051] The objective function of the discriminator is expressed as: m D axV(G,D)=E x～pd(x) [ln(D(x))]+E z～pz(z) [ln(1 - D(G(z)))];

[0052] Among them, V(G,D) is the value function, which enables the generator and the discriminator to conduct a minimax game. E is the expected distribution. G and D represent the discriminator and the generator respectively. x represents the sampled real image samples. D(~) is the output of the discriminator. p d (x) represents the real data distribution. z represents random noise. p z (z) represents the noise distribution. G(z) represents the noise image samples output by the generator;

[0053] The objective function of the generator is expressed as:

[0054] In summary, the objective function of the generative adversarial network satisfies:

[0055]

[0056] When training the discriminator, when the input is a real image sample x, the expected output of the discriminant function D(x) is 1. When the input is a noise image generated by the generator, the expected output of the discriminant function D(G(z)) is 0. When training the generator, the discriminator does not participate in the update. In order to make the value of the objective function as small as possible, the expected output of the discriminant function D(G(z)) is 1. Until reaching the Nash equilibrium state in game theory, at this time, whether for real image samples or noise image samples, their prediction probabilities are both one-half.

[0057] In the generator of the above generative adversarial network, an attention mechanism based on spatial position is introduced: the generator extracts the image feature elements of the real sample image, combines the superimposed spatial positions for weighted averaging, so that the generator focuses more on the generation of defective parts;

[0058] The convolutional layer of the generator extracts image feature elements from the input real sample image data, fuses the extracted image feature elements into the part background image based on the part standard background to obtain a fused image, and then passes it to the spatial attention module;

[0059] A 1x1 convolutional layer is set in the spatial attention module to map the fused image to a new feature space and another convolutional layer is used to generate a spatial attention weight matrix. The spatial size of this weight matrix is the same as that of the fused image, and each element in the weight matrix corresponds to a pixel position. Subsequently, a Sigmoid activation function is used for normalization processing to compress the weight values between 0 and 1. At this time, the obtained weight values are equivalent to the attention degree of the generator to that position;

[0060] Multiply the normalized weight matrix by the fused image to weight each position in the fused image, and obtain the weighted fused sample image, that is, the fake image;

[0061] Use a generative adversarial network to generate defective sample images to expand the defective sample database, and mark the defective positions of the defective sample images and set defective type labels.

[0062] Step 3: Construct an image defect recognition model based on the defective sample library

[0063] Taking the defective sample images as input and the part type, defective position and corresponding defective type as output, construct an image defect recognition model. Divide the defective sample images in the defective sample database into a training set and a test set according to a ratio of 7:3, and train the image defect recognition model to obtain an image defect recognition model based on a convolutional neural network.

[0064] Step 4: Obtain a real-shot part image and input it into the image defect recognition model for recognition

[0065] Use a high-definition camera to photograph the surface of the part to obtain a part real-scene image, and perform preprocessing operations such as cropping and denoising on the part real-scene image to obtain an image to be inspected;

[0066] The image defect recognition model identifies the type of the detected part and whether there are defects. When there are defects, mark the defective position and the defective type;

[0067] During the detection process, generate a part detection serial number based on the production time sequence coding and assign it to the corresponding part. After performing image defect recognition based on the real-shot part image, obtain a defect recognition form arranged based on the part detection serial number. Each row of the form is set with multiple characterization elements, including: part type, batch number, whether there are defects, number of defects, defective position, defective type, etc.

[0068] Step 5: Confirm the defective parts based on the defect identification form and classify the defect levels

[0069] Under normal circumstances, the vast majority of parts are non-defective parts, and the existence of defects in parts is a small probability event. Therefore, the defective parts in the form can be confirmed manually; after manual confirmation, if it is found that the image defect recognition model has misidentifications (such as incorrect correspondence of defect types and positions), the misidentified images are added to the generative adversarial network to generate a large number of similar image samples, and the image defect recognition model is retrained;

[0070] In the confirmed defect identification form, classify the defect levels of the same type of parts in the same batch:

[0071] First, add importance level labels to each feature surface of this type of part. The importance levels of the part feature surfaces are sorted from high to low in the order of the force-bearing working surface, the reference positioning surface, the auxiliary working surface, the auxiliary positioning surface, and the contour surface, and importance level label values are set respectively;

[0072] Secondly, for the defect types occurring on each type of part feature surface, set corresponding influence factors based on expert experience. For example, if there is a breakage defect on the force-bearing working surface, the value of the influence factor is relatively large; if there is color dirt on the force-bearing working surface, the value of the influence factor is relatively small, and the influence factor takes values between 0 and 1;

[0073] Then, combine the number and size of the defects on the opposite feature surface of the part to comprehensively evaluate its defect status score DSscore. The defect status score is the sum of the comprehensive scores of all defect types existing on each feature surface. The calculation formula is:

[0074]

[0075] Among them, δ i,k represents the ratio of the total defect area size existing in the part feature surface k to the overall area of this part feature surface, L k represents the importance level label value of the part feature surface k; σ i,k represents the influence factor of the defect type i in the current part feature surface k; N i represents the number of the defect type i in the current feature surface; S i represents the area of each defect type i in the current feature surface; P represents the feature surface number of this part; Q represents the defect type number.

[0076] Finally, set multiple defect level score thresholds. According to the defect status score, classify the defective parts at the corresponding score thresholds into the corresponding defect levels, so as to determine the defect levels of the parts;

[0077] Step 6. Mark and sort according to the defect levels of defective parts. Define the corresponding parts as defective parts, substandard parts, and scrap parts according to the defect levels. Among them, defective parts and substandard parts retain the basic functions of the parts and can be used continuously after targeted repair. Scrap parts cannot perform basic functions and are recycled for waste regeneration.

[0078] As Figure 2 shown, the above-mentioned part defect detection method completes the execution work process based on a part defect detection system. The system includes:

[0079] A part information sorting module, which is used to classify and sort the part processing library inside the factory, classify according to the part shape structure characteristics, material, and function, so as to determine the characteristic surface of the same type of parts. The material characteristics can determine which types of defects will occur in the corresponding parts;

[0080] A defect sample database, which is used to store the defect sample images of parts, and the defect types and defect positions that appear on the characteristic surface of the parts are marked on the defect sample images;

[0081] A sample image expansion module, which constructs a generative adversarial network and introduces a spatial position-based attention mechanism into the generator of the generative adversarial network, so as to improve the attention to the working surface and the positioning surface when generating defect sample images;

[0082] An image recognition module, which constructs an image defect recognition model. The image defect recognition model takes the defect sample image as the input, and the part type, defect position, and corresponding defect type as the output. The training set and test set are constructed using the defect sample database to train the model. An image acquisition unit and an image preprocessing unit are also set in the image recognition module. The image acquisition unit specifically uses an alarm camera to take pictures of the part surface to obtain the actual scene image of the part, and performs cropping and denoising operations through the image preprocessing unit to obtain the image to be inspected. Subsequently, the image to be inspected is used as the image defect recognition model for defect recognition;

[0083] A defect level determination module, which constructs a defect recognition form, manually determines the defects in the form one by one, and then divides the defect levels of the same type of parts in the same batch in the determined form. Based on parameters such as the part characteristic surface, defect type, defect quantity, and area, the defect status score of the part is obtained, and the part is divided into different defect levels according to the defect status score. Subsequently, the part is defined as a defective part, a substandard part, or a scrap part based on the defect level, and corresponding treatment measures are taken for different defective parts.

[0084] The above are only the preferred embodiments of the present invention and do not impose any formal restrictions on the present invention. Although the present invention has been disclosed above in the preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments by using the disclosed technical content within the scope of the technical solution of the present invention. However, as long as it does not depart from the content of the technical solution of the present invention, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for detecting part defects, characterized in that: The steps include: Step 1: Classify parts based on their structure, material and functional characteristics, so as to mark the characteristic surfaces of similar parts and determine the common defect types; Step 2: Build a generative adversarial network combined with an attention mechanism to generate defect image samples, thereby expanding the defect sample database of various parts, and mark the defect locations of the defect sample images in the defect sample database and set defect type labels; Step 3: Construct an image defect recognition model based on a convolutional neural network and use the model to identify defects in real-shot part images. The model takes defect sample images in the defect sample database as input and takes part type, defect location and corresponding defect type as output; Step 4: Perform real-time inspection, generate a part inspection serial number based on time series coding for the parts to facilitate subsequent screening, and build a defect identification form. Each row of the form sets multiple characterization elements, including: part type, batch, defect, defect quantity, defect location, and defect type; Step 5: Manually review and confirm the defect recognition form, import the misidentified image into the generative adversarial network to generate similar image samples to train and update the image defect recognition model; Step 6. Classify the defect levels of similar parts in the same batch in the defect identification form, obtain the defect status score of the parts based on parameters such as part feature surface, defect type, defect quantity and area, and classify the parts into different defect levels according to the defect status score, define the degree of part defects according to different levels, and take repair or recycling measures according to the degree of defects.

2. A method for detecting part defects according to claim 1, characterized in that: The generative adversarial network includes a generator and a discriminator that compete with each other. The objective function of the discriminator is expressed as: D axV(G,D)=E x～pd(x) [ln(D(x))]+E z～pz(z) [ln(1-D(G(z)))]; Among them, V(G,D) is the value function, which makes the generator and the discriminator play a minimax game, E is the expected distribution, G, D represent the discriminator and the generator respectively, x represents the real image sample, D(~) is the discriminator output, p d (x) represents the real data distribution, z represents random noise, and p z (z) represents the noise distribution, G(z) represents the noise image sample output by the generator; The objective function of the generator is expressed as: In summary, the objective function of the generative adversarial network satisfies: When the input is a real image sample x, the expected output of the discriminant function D(x) is 1. When the input is a noise image generated by the generator, the expected output of the discriminant function D(G(z)) is 0. When training the generator, the discriminator does not participate in the update, and the expected output of the discriminant function D(G(z)) is 1. Finally, the Nash equilibrium state is reached, and the prediction probability for both real image samples and noise image samples is one-half.

3. A part defect detection method according to claim 2, characterized in that: An attention mechanism based on spatial position is introduced into the generator to construct a spatial attention module; the convolution layer of the generator extracts image feature elements from the input real sample image data, and the extracted image feature elements are fused with the part background image based on the part standard background to obtain a fused image, which is then passed to the spatial attention module. The spatial attention module assigns weights to pixel positions to obtain a fused sample image with fused position weights, i.e., the final defect sample image.

4. A parts defect detection method according to claim 3, characterized in that: A 1x1 convolution layer is set in the spatial attention module to map the fused image to a new feature space and another convolution layer is used to generate a spatial attention weight matrix. The spatial size of the weight matrix is ​​the same as the size of the fused image. Each element in the weight matrix corresponds to a pixel position. Then normalization is performed and the normalized weight matrix is ​​multiplied by the fused image to weight each position in the fused image to obtain a weighted fused sample image.

5. A part defect detection method according to claim 1, characterized in that: The specific steps of obtaining the defect status score of the part include: Set importance label values ​​for each feature surface of the part; Combine the feature surfaces of different importance and the defect types on the feature surfaces to set the corresponding influencing factors; The comprehensive scores of all defect types on each feature surface of a part are calculated separately and summed up to obtain the defect status score of the corresponding part.

6. A method for detecting part defects according to claim 5, characterized in that: The calculation formula of the defect status score is specifically: Among them, δ i,k It indicates the ratio of the total defect area in the feature surface k of the part to the overall area of ​​the feature surface of the part. L k Indicates the importance label value of the feature surface k of the part; σ i,k Indicates the impact factor of defect type i in the current part feature surface k; N i Indicates the number of defect type i in the current feature surface; S i It represents the area of ​​each defect type i in the current feature surface; P represents the feature surface number of the part; Q represents the defect type number.

7. A method for detecting part defects according to claim 5, characterized in that: Define multiple defect grade scoring thresholds, and classify defective parts within the corresponding scoring thresholds into corresponding defect grades according to the defect status scores, thereby determining the defect grade of the parts; carry out targeted repairs on parts defined as defective parts or inferior parts according to the defect grade, and recycle parts defined as scrap parts.

8. A system for running the part defect detection method according to any one of claims 1 to 7, characterized in that: include: The parts information sorting module is used to classify and sort the parts processing library in the factory, and determine the characteristic surfaces and corresponding defect types of a type of parts based on the shape and structure characteristics, material and function of the parts; The sample image expansion module builds a generative adversarial network that integrates the spatial position attention mechanism to generate defect sample images; Image recognition module, builds an image defect recognition model to identify defects in the real scene model of the part obtained by machine vision, and obtains the part type, defect location and defect type of the defective part; The defect level determination module constructs a defect identification form and performs manual confirmation. It then scores and grades the defect status of similar parts in the same batch in the determined form, defines the degree of defects of the parts based on the defect level, and takes different treatment measures according to different defect levels.

Images