Intelligent self-learning visual inspection system

Abstract

The invention relates to an intelligent self-learning visual inspection system which is composed of an acquisition system, a data engine, an iteration engine, a correction system, a detection system and an NG / OK judgment system, and the acquisition system is used for being responsible for image acquisition of a workpiece to be detected and transmitting a picture detected by a light source to the data engine or the correction system; the data engine is used for storing the data transmitted by the acquisition system and intelligently marking the data to form formatted data required by model iteration; and the detection system is used for sending a detection request to the iteration engine after detecting the picture transmitted by the correction system, and the iteration engine calls the current optimal model for reasoning after receiving the request of the detection system and returns a reasoning result to the detection system. According to the invention, an intelligent identification model in the aspect of computer vision is utilized to automatically discover internal defects of the to-be-detected product and calculate the size of the defects as a reference basis for product waste judgment.

Description

Technical field:

[0001] The invention belongs to the technical field of visual inspection, and in particular relates to an intelligent self-learning visual inspection system. Background technology:

[0002] In modern manufacturing, product quality control is an important manifestation of corporate competitiveness. With the increasing complexity of product design and the diversification of material types, traditional optical inspection methods have been unable to meet the needs of high precision and high efficiency. These methods usually rely on visible light or other simple light sources, which limits their ability in deep defect detection and material analysis. In addition, the manual inspection process is not only inefficient, but also prone to human errors, resulting in potential quality problems not being discovered in time.

[0003] X-ray technology, with its excellent penetration and sensitivity to different materials, has gradually been applied to industrial inspection fields such as aerospace, electronic products, and automotive parts. It can penetrate deep into the material and reveal defects such as pores, cracks, and inclusions that are difficult to detect through surface observation. This makes X-ray an indispensable quality assurance tool in high-end manufacturing. With the rise of industry and intelligent manufacturing, how to combine X-ray technology with automated and intelligent inspection systems has become a new trend in the development of the industry.

[0004] In the existing technology, X-ray defect inspection still needs to rely on manual settings and operations. This not only reduces the efficiency of detection, but also increases the workload of operators, limiting the improvement of overall production efficiency. In addition, the image processing technology lags behind. Although some systems have introduced image processing technology, most of them are based on traditional algorithms and lack the support of deep learning and artificial intelligence technology. This leads to insufficient flexibility and accuracy of the system in the automatic identification and classification of complex defects, making it difficult to meet the needs of high-precision detection. At the same time, the degree of integration is insufficient. The current X-ray inspection system is often independent and difficult to be effectively integrated with other equipment and systems on the production line. This dispersion leads to poor data flow, inability to achieve real-time monitoring and feedback, and affects the optimization of the production process. In addition, there is also a high cost problem. The high-precision X-ray detection equipment and its related maintenance costs are high, which makes many small and medium-sized enterprises reluctant to invest, which limits the popularization of new technologies and affects the progress of the entire industry. In addition, the universality of the algorithm is not strong enough. The algorithm model optimized on product A will have a performance decline when applied to product B. As a result, application manufacturers must do some additional tuning and adaptation work when switching products between different production lines, which greatly limits the convenience of use and generates related costs for enterprises. In addition, the lack of self-learning ability does not give full play to the role of big data in promoting algorithm performance improvement and optimization iteration, resulting in a huge waste of data resources. The above problems need to be solved urgently. Summary of the invention:

[0005] The technical problem to be solved by the present invention is to provide an intelligent self-learning visual inspection system, which uses an intelligent recognition model in computer vision to automatically discover internal defects of the product to be inspected and calculate the size of the defect as a reference for product rejection.

[0006] The technical solution of the present invention is to provide an intelligent self-learning visual inspection system, which consists of an acquisition system, a data engine, an iteration engine, a correction system, a detection system and an NG / OK judgment system, wherein:

[0007] The acquisition system is responsible for collecting images of the workpiece to be tested and transmitting the images detected by the light source to the data engine or the correction system;

[0008] The data engine is used to store and intelligently mark the data transmitted by the acquisition system to form the formatted data required for model iteration; the data engine includes three modules: intelligent marking system, data management system and image information summary;

[0009] The detection system is used to send a detection request to the iteration engine after detecting the image transmitted by the correction system. After receiving the request from the detection system, the iteration engine calls the current optimal model for inference and returns the inference result to the detection system.

[0010] The acquisition system is used to perform HDR conversion on the transmitted images and map them to the dynamic range that the human eye can distinguish;

[0011] The intelligent marking system includes a conditional generative deep learning model, a conditional image generation deep learning model, a standard normal distribution sampler and a defect morphology sampler; the standard normal distribution sampler is the input of the conditional generative deep learning model, and is used to generate a defect-free and variable workpiece image under the constraint of the current input workpiece image; the conditional image generation deep learning model takes the flawless image and defect mask of the workpiece as input, and takes the type of defect as conditional input to generate a workpiece image with defects; the defect morphology sampler is used to generate a defect mask, and together with the output of the conditional generative deep learning model, forms a workpiece image with a defect mask; and the image is used as the input of the conditional image generation deep model.

[0012] As a preferred option, the workflow of the intelligent marking system is as follows:

[0013] S10: Read the image x uploaded from the workstation by the acquisition system in real time;

[0014] S20: Perform a downsampling operation on the image, and record the downsampled image as x according to the sampling number n. n ;

[0015] S30: sampling s~N(0,1), where N represents normal distribution;

[0016] S40: Sampling y~N(μ θ (s|x 4 ), σ), where μ θ (s|x 4 ) represents a conditional generative neural network model;

[0017] S50: sampling s~N(0,1);

[0018] S60: sampling m~N(π θ (s),σ), where π θ (s) represents the parameterized function in the defect morphology sampler, which can be a neural network or a self-defined elementary function; the sampled m is binarized, and the processed result is still recorded as m;

[0019] S60: According to formula y m =y*m generates a normal workpiece image with a mask;

[0020] S70: defect type sampling s, where s is a text description of the defect type of the workpiece to be tested;

[0021] S80: Sampling x d ~N(ρ θ (y m |s),σ), where ρ θ (y m |s) represents a conditional generative graph network;

[0022] S90:(x,x d , m, s) as a formatted data and enter the data management system;

[0023] Among them, s represents the output of the sampler, y represents the normal workpiece image, m represents the defect mask, x represents the original image of the workpiece on the production line, and x d represents the defect image that meets the feature of workpiece x, and s is the defect type.

[0024] Preferably, the input of the conditional generative deep learning model obeys a normal distribution with a mean of 0 and a standard deviation of 1; the condition is a downsampled image of a local image of the workpiece; and the number of downsampling times is adjustable, ranging from 2 to 8 times.

[0025] Preferably, the defect morphology sampler is a defect morphology distribution generated using a deep learning model as a parameterized tool, or a uniform sampling from a large number of defect morphology sample libraries.

[0026] As a preferred embodiment, the iteration engine is composed of a neural network for semantic classification and a cost function for training the neural network. The iteration engine updates the network parameters in real time according to the network cost function, so that the value of the cost function decreases continuously; the neural network is a discriminant network, denoted as: P θ (y|x); the distribution of x is consistent with the data distribution on the production line; the distribution of y is consistent with the distribution of the defect morphology sampler in the data engine, and θ is the parameter of the neural network; the discriminant network can be any one that has been disclosed so far; the evaluation function of the neural network can be written as follows:

[0027]

[0028] where y * Represents the statistical distribution of the discriminative neural network output; y comes from the sampling of defects by the data engine and obeys the distribution of real defects.

[0029] Preferably, the correction system includes a tone mapping function, a smoothing function, a weighted fusion module, a noise estimation module and an estimation module of a fusion parameter α, wherein:

[0030] The tone mapping function is recorded as:

[0031] x out =T(x in )

[0032] where x in Indicates the input 16-bit tiff format image, with a value range of 0 to 65535; x out Represents the output of tone mapping, with a value range of 0 to 1; tone mapping is a tone mapping function or a neural network;

[0033] The smoothing function is used to smooth the output x of the tone mapping function out Smoothing is performed and x blur Record the smoothed output; the smoothing function can be Gaussian blur, mean blur, or other publicly available smoothing functions;

[0034] Weighted fusion module, the formula of weighted fusion is:

[0035] x final =(1+α)x out +αx blur

[0036] Among them, α is the fusion coefficient, x final is the output of the correction system;

[0037] The noise standard deviation estimated by the noise estimation module is σ noise ;

[0038] The estimation formula of the estimation module of the fusion parameter α is as follows:

[0039]

[0040] α in the formula 0 , c, β and σ 0 It can be adjusted according to the image conditions of the acquisition system. In the estimation formula of this module, α_0=2.0, c=20.0, β=1000, σ_0=0.0001.

[0041] Furthermore, the light source is any one of x-ray light, visible light, ultrasonic wave or millimeter wave.

[0042] Compared with the prior art, the present invention has the following advantages:

[0043] The present invention provides an intelligent self-learning visual inspection system, which uses an intelligent recognition model in computer vision to automatically discover internal defects of products to be inspected and calculate the size of the defects as a reference for product rejection. The system has an automatic learning function, high detection efficiency, and can be applied to a variety of light sources. In terms of automatic recognition and classification of complex defects, the system has higher flexibility and accuracy to meet the needs of high-precision detection. It can realize real-time monitoring and feedback, which is convenient for the optimization of the production process. The algorithm is highly universal and can adapt to product switching of different production lines. Description of the drawings:

[0044] Figure 1 It is a schematic diagram of the process of the present invention. Specific implementation method:

[0045] The present invention will be further described below with reference to the accompanying drawings:

[0046] An intelligent self-learning visual inspection system, such as Figure 1 As shown, the system is composed of a collection system, a data engine, an iteration engine, a correction system, a detection system and an NG / OK judgment system. In this embodiment, x-ray light is used as a light source for detailed description.

[0047] The acquisition system is responsible for collecting x-ray images of the workpiece and transmitting the x-ray detected images to the data engine or correction system. The technical details of the acquisition system are common practices in this field and will not be described in detail. It is necessary to introduce the requirements of this detection system for the formatted data of the acquisition system. The formatted data required by the correction system are: image, station number, and the parameters of the six-degree-of-freedom robot corresponding to the station number. The specific parameters include xc, yc, zc, w1, w2, w3; among which xc, yc, zc represent the position of the robot, and w1, w2, w3 represent the posture of the robot.

[0048] The data engine is responsible for storing and intelligently marking the data transmitted by the collection system to form the formatted data required for model iteration.

[0049] In this embodiment, the data engine includes three parts. The first part is an intelligent marking system. The intelligent marking system provided in the present invention is one of the key features that distinguish the present invention from data modules of other detection systems. The second part is a data management system, which is responsible for storing different types of data, and the storage format meets the requirements of the iterative system. The third part is image information summary, which functions to format the data description of the input original workpiece image, the generated defect image, and the defect mask image according to the defect type sampled by the defect sampler. The formatted data description is the basis for the data management system to store files; it is the basis for the iterative system to read training materials.

[0050] Specifically, the intelligent marking system includes:

[0051] A conditional generative deep learning model; the input of the conditional generative deep learning model obeys a normal distribution with a mean of 0 and a standard deviation of 1; the condition is a downsampled image of a local image of a workpiece; the number of downsampling times is adjustable, generally 4 to 6 times; the number of downsampling times in this embodiment is 4 times;

[0052] A conditional image generation deep learning model; the conditional image generation deep learning model takes the flawless image and defect mask of the workpiece as input; takes the type of defect as conditional input; and generates an image of the workpiece with defects;

[0053] A standard normal distribution sampler, which is the input of the conditional generation deep learning model. Its function is to generate defect-free and variable workpiece images under the constraints of the current input workpiece image;

[0054] A defect morphology sampler; its function is to generate a defect mask and, together with the output of a conditional generative deep learning model, form a workpiece image with a defect mask; the image is used as the input of a conditional image generation deep model; the defect morphology sampler can be a defect morphology distribution generated by a deep learning model as a parameterized tool, or it can be a uniform sampling from a large number of defect morphology sample libraries.

[0055] The workflow of this intelligent marking system is shown in the arrow of the system block diagram:

[0056] S10: Read the image x uploaded from the workstation by the acquisition system in real time;

[0057] S20: Perform a downsampling operation on the image, and record the downsampled image as x according to the sampling number 4. 4Although the system block diagram shows that the number of sampling times is 4, it can actually be set flexibly to 2, 6, 8, etc. The validity of its authorization is not affected by the change of the number of sampling times.

[0058] S30: sampling s~N(0,1), where N represents normal distribution;

[0059] S40: Sampling y~N(μ θ (s|x 4 ), σ), where μ θ (s|x 4 ) represents a conditional generative neural network model;

[0060] S50: sampling s~N(0,1);

[0061] S60: sampling m~N(π θ (s),σ), where π θ (s) represents the parameterized function in the defect morphology sampler, which can be a neural network or a self-defined elementary function; the sampled m is binarized, and the processed result is still recorded as m;

[0062] S60: According to formula y m =y*m generates a normal workpiece image with a mask;

[0063] S70: defect type sampling s, where s is a text description of the defect type of the workpiece to be tested;

[0064] S80: Sampling x d ~N(ρ θ (y m |s),σ), where ρ θ (y m |s) represents a conditional generative graph network;

[0065] S90:(x,x d , m, s) as a formatted data and enter the data management system;

[0066] Among them, each parameter is a random variable, s represents the output of the sampler, y represents the normal workpiece image, m represents the defect mask, x represents the original image of the workpiece on the production line, and x d represents the defect image that meets the feature of workpiece x, and s is the defect type.

[0067] At this point, the workflow of the intelligent marking system is complete. The marking system ensures that the data distribution provided to the iteration engine is completely consistent with the production line data distribution; ensures that the defect types of the data provided to the iteration engine are rich and varied, which is conducive to the training and optimization of the model; and ensures that the data provided to the iteration engine are correctly marked and the defect boundaries are accurate and clear.

[0068] The data management system is responsible for storing the formatted data output by the intelligent marking system according to the requirements of the iteration engine, which is a common method in this field and will not be elaborated in detail. The storage format provided by the present invention is:

[0069] S1. Workpiece location

[0070] S1.1 Workpiece Image

[0071] S1.2 Semantic labels of artifact images.

[0072] The iterative engine of this embodiment is composed of a neural network for semantic classification and a cost function for training the neural network. The iterative engine updates the network parameters in real time according to the network cost function, so that the value of the cost function continues to decrease. The updating method can be a random gradient descent method, or other currently disclosed methods, which are well-known technologies and will not be described in detail.

[0073] The so-called neural network for semantic judgment is a discriminant network, denoted as: P θ (y|x); where x comes from the data generated by the data engine, and its distribution is consistent with the data distribution on the production line; the distribution of y is consistent with the distribution of the defect morphology sampler in the data engine, and θ is the parameter of the neural network; the discriminant network can be any one that has been disclosed so far;

[0074] The so-called evaluation function of the neural network can be written as follows:

[0075]

[0076] where y * Represents the statistical distribution of the discriminative neural network output; y comes from the sampling of defects by the data engine and obeys the distribution of real defects.

[0077] After the detection system detects the image transmitted by the correction system, it sends a detection request to the iteration engine. After receiving the request from the detection system, the iteration engine calls the current optimal model for reasoning and returns the reasoning result to the detection system. The model reasoning involved in this module is a well-known technology and will not be elaborated on here.

[0078] The correction system is also called HDR system in some occasions. The full name of HDR is High Dynamic Range, which is a mapping or transformation. Its function is to transform the image transmitted by the acquisition system into HDR and map it to the dynamic range that the human eye can distinguish. It is essentially a visualization technology. HDR for x-ray light has its own special features. The present invention provides a new HDR method that is different from the existing public information. The specific technical features are given here:

[0079] In this embodiment, the correction system includes a tone mapping function, which is expressed as:

[0080] x out =T(x in )

[0081] where x in Indicates the input 16-bit tiff format image, with a value range of 0 to 65535; x out Represents the output of tone mapping, with a value range of 0 to 1; tone mapping can be any of the currently available technologies, either a traditional tone mapping function or a neural network;

[0082] The correction system also includes a smoothing function, which is the output x of the tone mapping function out For the sake of recording, use x blur Record the smoothed output; the smoothing function can be Gaussian blur, mean blur, or other publicly available smoothing functions;

[0083] At the same time, the correction system also includes a weighted fusion module and a noise estimation module. The formula of the weighted fusion is:

[0084] x final =(1+α)x out +αx blur

[0085] Where α is the fusion coefficient, and its value method will be given later; x final is the output of the correction system;

[0086] The noise estimation method of the noise module can adopt any of the currently available methods, and the estimated noise standard deviation is σ noise ;

[0087] In addition, the correction system also includes an estimation method for the fusion parameter α, and the estimation formula is given here:

[0088]

[0089] α in the formula 0 , c, β and σ 0 It can be flexibly adjusted according to the image conditions of the acquisition system; a set of parameters for the implementation of the present invention is given here: α 0 =2.0, c=20.0, β=1000,σ 0 =0.0001.

[0090] The NG / OK determination system is also a common method in this field and will not be described in detail.

[0091] The intelligent self-learning visual inspection system provided by the present invention utilizes the intelligent recognition model of computer vision to automatically discover the internal defects of the product to be inspected and calculate the size of the defects as a reference for product rejection. It has an automatic learning function, high detection efficiency, and can be applied to a variety of light sources. In terms of automatic identification and classification of complex defects, the system has higher flexibility and accuracy, meeting the needs of high-precision detection. It can realize real-time monitoring and feedback, which is convenient for the optimization of the production process. In addition, the algorithm is universal and robust, and can adapt to product switching of different production lines.

[0092] It should be noted that, although the present embodiment uses x-ray as the light source, the authorized protection scope of the present invention cannot be limited to the technical feature field using x-ray as the light source, nor can it be limited to the application field using x-ray as the light source. All technical fields and application fields that are similar to the system block diagram and the work flow, formulas, and module functions described in the description of the present invention are within the scope of protection required by the present invention; such as defect detection or poor appearance detection implemented by using X-ray, visible light, ultrasound, millimeter waves, etc. as light sources; such as application fields using X-ray, visible light, ultrasound, millimeter waves, etc. as light sources for defect detection or poor appearance detection; as long as the technical implementation matches the technical features given in the present invention, it is protected by the present invention.

Claims

1. An intelligent self-learning visual inspection system, characterized in that: The system consists of an acquisition system, a data engine, an iteration engine, a correction system, a detection system, and an NG / OK judgment system. The acquisition system is responsible for collecting images of the workpiece to be tested and transmitting the images detected by the light source to the data engine or the correction system; The data engine is used to store and intelligently mark the data transmitted by the acquisition system to form the formatted data required for model iteration; the data engine includes three modules: intelligent marking system, data management system and image information summary; The detection system is used to send a detection request to the iteration engine after detecting the image transmitted by the correction system. After receiving the request from the detection system, the iteration engine calls the current optimal model for inference and returns the inference result to the detection system. The acquisition system is used to perform HDR conversion on the transmitted images and map them to the dynamic range that the human eye can distinguish; The intelligent marking system includes a conditional generative deep learning model, a conditional image generation deep learning model, a standard normal distribution sampler and a defect morphology sampler; the standard normal distribution sampler is the input of the conditional generative deep learning model, and is used to generate a defect-free and variable workpiece image under the constraint of the current input workpiece image; the conditional image generation deep learning model takes the flawless image and defect mask of the workpiece as input, and takes the type of defect as conditional input to generate a workpiece image with defects; the defect morphology sampler is used to generate a defect mask, and together with the output of the conditional generative deep learning model, forms a workpiece image with a defect mask; and the image is used as the input of the conditional image generation deep model.

2. The intelligent self-learning visual inspection system according to claim 1, characterized in that: The working process of the intelligent marking system is as follows: S10: Read the image x uploaded from the workstation by the acquisition system in real time; S20: Perform a downsampling operation on the image, and record the downsampled image as x according to the sampling number n. n ; S30: sampling s~N(0,1), where N represents normal distribution; S40: Sampling y~N(μ θ (s|x4),σ), where μ θ (s|x4) represents the conditional generative neural network model; S50: sampling s~N(0,1); S60: sampling m~N(π θ (s),σ), where π θ (s) represents the parameterized function in the defect morphology sampler, and performs binarization processing on the sampled m, and the processed result is still recorded as m; S60: According to formula y m =y*m generates a normal workpiece image with a mask; S70: defect type sampling s, where s is a text description of the defect type of the workpiece to be tested; S80: Sampling x d ~N(ρ θ (y m |s),σ), where ρ θ (y m |s) represents a conditional generative graph network; S90:(x,x d , m, s) as a formatted data and enter the data management system; Among them, s represents the output of the sampler, y represents the normal workpiece image, m represents the defect mask, x represents the original image of the workpiece, and x d represents the defect image that meets the feature of workpiece x, and s is the defect type.

3. The intelligent self-learning visual inspection system according to claim 1, characterized in that: The input of the conditional generative deep learning model obeys a normal distribution with a mean of 0 and a standard deviation of 1; the condition is a downsampled image of a local image of the workpiece; the number of downsampling times is adjustable, ranging from 2 to 8 times.

4. The intelligent self-learning visual inspection system according to claim 1, characterized in that: The defect shape sampler is a defect shape distribution generated by a deep learning model as a parameterized tool, or a uniform sampling from a defect shape sample library.

5. The intelligent self-learning visual inspection system according to claim 2, characterized in that: The iteration engine consists of a neural network for semantic classification and a cost function for training the neural network. The iteration engine updates the network parameters in real time according to the network cost function. The neural network is a discriminant network, denoted as: P θ (y|x); the distribution of x is consistent with the data distribution on the production line; the distribution of y is consistent with the distribution of the defect morphology sampler in the data engine, and θ is the parameter of the neural network; the discriminant network can be any one that has been disclosed so far; the evaluation function of the neural network can be written as follows: where y * Represents the statistical distribution of the discriminative neural network output; y comes from the sampling of defects by the data engine and obeys the distribution of real defects.

6. The intelligent self-learning visual inspection system according to claim 1, characterized in that: The correction system includes a tone mapping function, a smoothing function, a weighted fusion module, a noise estimation module and an estimation module of a fusion parameter α, wherein: The tone mapping function is recorded as: x out =T(x in ) where x in Indicates the input 16-bit tiff format image, with a value range of 0 to 65535; x out Represents the output of tone mapping, with a value range of 0 to 1; tone mapping is a tone mapping function or a neural network; The smoothing function is used to smooth the output x of the tone mapping function out Smoothing is performed and x blur Record the smoothed output; Weighted fusion module, the formula of weighted fusion is: x final =(1+α)x out +αx blur Among them, α is the fusion coefficient, x final is the output of the correction system; The noise standard deviation estimated by the noise estimation module is σ noise ; The estimation formula of the estimation module of the fusion parameter α is as follows: α0, c, β and σ0 in the formula can be adjusted according to the image conditions of the acquisition system.

7. The intelligent self-learning visual inspection system according to claim 6, characterized in that: In the estimation formula of the estimation module of the fusion parameter α, α_0=2.0, c=20.0, β=1000, σ_0=0.0001.

8. The intelligent self-learning visual inspection system according to claim 1, characterized in that: The light source is any one of x-ray light, visible light, ultrasonic wave or millimeter wave.

Images