An intelligent self-learning visual inspection system

Abstract

The present invention relates to an intelligent self-learning visual inspection system, which is composed of an acquisition system, a data engine, an iteration engine, a correction system, a detection system, and an NG / OK judgment system. The acquisition system is responsible for collecting images of the workpiece to be tested and transmitting the images detected by the light source to the data engine or the correction system; the data engine is used to store the data transmitted by the acquisition system and intelligently mark it to form the formatted data required for model iteration; the detection system is used to send a detection request to the iteration engine after detecting the image transmitted by the correction system. After receiving the request from the detection system, the iteration engine calls the current optimal model for inference and returns the inference result to the detection system. The present invention utilizes an intelligent recognition model in computer vision to automatically discover internal defects of the product to be tested and calculate the defect size as a reference for product rejection.

Description

Technical field:

[0001] The present invention belongs to the technical field of visual inspection, and in particular relates to an intelligent self-learning visual inspection system. Background technology:

[0002] In modern manufacturing, product quality control is a crucial element of a company's competitiveness. With the increasing complexity of product designs and the diversification of material types, traditional optical inspection methods are no longer able to meet the demands for high precision and efficiency. These methods typically rely on visible light or other simple light sources, limiting their capabilities for deep defect detection and material analysis. Furthermore, manual inspection processes are not only inefficient but also prone to human error, leading to potential quality issues not being discovered in a timely manner.

[0003] X-ray technology, with its exceptional penetrating power and sensitivity to diverse materials, is increasingly being applied to industrial inspection applications such as aerospace, electronics, and automotive parts. Its ability to penetrate deep into materials reveals defects such as pores, cracks, and inclusions that are difficult to detect through surface observation. This has made X-ray an indispensable quality assurance tool in high-end manufacturing. With the rise of Industry 4.0 and smart manufacturing, integrating X-ray technology with automated and intelligent inspection systems has become a new industry trend.

[0004] Existing X-ray defect inspection still relies on manual setup and operation. This not only reduces inspection efficiency but also increases the workload of operators, limiting overall production efficiency. Furthermore, image processing technology lags behind. While some systems have incorporated image processing technology, most rely on traditional algorithms and lack the support of deep learning and artificial intelligence technologies. This results in insufficient flexibility and accuracy in the automatic identification and classification of complex defects, making it difficult to meet the demands of high-precision inspection. Furthermore, integration is insufficient. Current X-ray inspection systems are often standalone, making them difficult to effectively integrate with other equipment and systems on the production line. This fragmented nature hinders data flow, preventing real-time monitoring and feedback, and hindering production process optimization. Furthermore, high costs are a concern. High-precision X-ray inspection equipment and its associated maintenance costs are prohibitive, making investment prohibitive for many small and medium-sized enterprises. This hinders the widespread adoption of new technologies and the overall industry progress. Furthermore, the algorithm's generalizability is insufficient. An algorithm model optimized for product A will experience performance degradation when applied to product B. This means that application manufacturers must perform additional tuning and adaptation work when switching products between different production lines. This significantly limits ease of use and incurs costs for businesses. Furthermore, the lack of self-learning capabilities prevents big data from leveraging its role in improving algorithm performance and optimizing iterations, resulting in a significant waste of data resources. All of these issues need to be addressed urgently. Summary of the invention:

[0005] The technical problem to be solved by the present invention is to provide an intelligent self-learning visual inspection system, which uses an intelligent recognition model in computer vision to automatically detect internal defects of the product to be inspected and calculate the size of the defects as a reference for product rejection.

[0006] The technical solution of the present invention is to provide an intelligent self-learning visual inspection system, which consists of an acquisition system, a data engine, an iteration engine, a correction system, a detection system and an NG / OK judgment system, wherein:

[0007] The acquisition system is responsible for collecting images of the workpiece to be tested and transmitting the detected images to the data engine or correction system;

[0008] The data engine is used to store and intelligently mark the data transmitted by the acquisition system to form the formatted data required for model iteration; the data engine includes three modules: intelligent marking system, data management system, and image information aggregation;

[0009] After detecting the image passed by the correction system, the detection system sends a detection request to the iteration engine. After receiving the request from the detection system, the iteration engine calls the current optimal model for inference and returns the inference result to the detection system.

[0010] The acquisition system is used to perform HDR conversion on the transmitted images, mapping them to the dynamic range that the human eye can distinguish;

[0011] The intelligent marking system includes a conditional generative deep learning model, a conditional image generation deep learning model, a standard normal distribution sampler and a defect morphology sampler; the standard normal distribution sampler is the input of the conditional generative deep learning model, and is used to generate a defect-free and variable workpiece image under the constraints of the current input workpiece image; the conditional image generation deep learning model takes the flawless image and defect mask of the workpiece as input, and uses the defect type as conditional input to generate a workpiece image with defects; the defect morphology sampler is used to generate a defect mask, and together with the output of the conditional generative deep learning model, forms a workpiece image with a defect mask; and this image is used as the input of the conditional image generation deep model.

[0012] As a preferred option, the workflow of the intelligent marking system is as follows:

[0013] S10: Read the image x uploaded from the workstation by the acquisition system in real time;

[0014] S20: Perform downsampling on the image, and record the downsampled image as x according to the number of sampling times n. n ;

[0015] S30: sampling s~N(0,1), where N represents normal distribution;

[0016] S40: Sampling y~N(μ θ (s|x4),σ), where μ θ (s|x4) represents the conditional generative neural network model;

[0017] S50: sampling s~N(0,1);

[0018] S60: sampling m~N(π θ (s),σ), where π θ (s) represents the parameterized function in the defect morphology sampler, which can be a neural network or a self-defined elementary function; the sampled m is binarized, and the processed result is still recorded as m;

[0019] S70: According to formula y m =y*m generates a normal workpiece image with a mask;

[0020] S80: defect type sample s, where s is a text description of the defect type of the workpiece to be tested;

[0021] S90: Sampling x d ~N(ρ θ (y m |s),σ), where ρ θ (y m |s) represents a conditional generative graph network;

[0022] S100:(x,x d , m, s) as a formatted data and enter the data management system;

[0023] Among them, y represents the normal workpiece image, m represents the defect mask, x represents the original image of the workpiece on the production line, and x d represents the defect image that meets the feature of workpiece x, and s is the defect type.

[0024] Preferably, the input of the conditional generative deep learning model obeys a normal distribution with a mean of 0 and a standard deviation of 1; the condition is a downsampled image of a local image of the workpiece; and the number of downsampling times is adjustable, ranging from 2 to 8 times.

[0025] Preferably, the defect morphology sampler is a defect morphology distribution generated by using a deep learning model as a parameterized tool, or a uniform sampling from a large number of defect morphology sample libraries.

[0026] As a preferred embodiment, the iterative engine is composed of a neural network for semantic classification and a cost function for training the neural network. The iterative engine updates the network parameters in real time according to the network cost function, so that the value of the cost function continues to decrease; the neural network is a discriminant network, denoted as: P θ (y|x); where the distribution of x is consistent with the data distribution on the production line; the distribution of y is consistent with the distribution of the defect morphology sampler in the data engine, and θ is the parameter of the neural network; the discriminant network can be any of the currently available ones; the evaluation function of the neural network can be written as follows:

[0027]

[0028] where y * Represents the statistical distribution of the discriminative neural network output; y comes from the sampling of defects by the data engine and obeys the distribution of real defects.

[0029] Preferably, the correction system includes a tone mapping function, a smoothing function, a weighted fusion module, a noise estimation module and an estimation module of a fusion parameter α, wherein:

[0030] The tone mapping function is recorded as:

[0031] x out =T(x in )

[0032] where x in Indicates the input 16-bit tiff format image, with a value range of 0 to 65535; x out Represents the output of tone mapping, with a value range of 0 to 1; tone mapping is a tone mapping function or a neural network;

[0033] The smoothing function is used to smooth the output x of the tone mapping function out Smoothing is performed and x blur Record the smoothed output; the smoothing function can be Gaussian blur, mean blur, or other publicly available smoothing functions;

[0034] Weighted fusion module, the weighted fusion formula is:

[0035] x final =(1+ɑ)x out +ɑx blur

[0036] Among them, ɑ is the fusion coefficient, x final is the output of the correction system;

[0037] The noise standard deviation estimated by the noise estimation module is σ noise ;

[0038] The estimation formula of the estimation module of the fusion parameter α is as follows:

[0039]

[0040] The α0, c, β, and σ0 in the formula can be adjusted according to the image conditions of the acquisition system. In the estimation formula of this module, α0 = 2.0, c = 20.0, β = 1000, and σ0 = 0.0001.

[0041] Furthermore, the light source is any one of X-ray light, visible light, ultrasonic wave or millimeter wave.

[0042] Compared with the prior art, the present invention has the following advantages:

[0043] The present invention provides an intelligent self-learning visual inspection system, which uses intelligent recognition models in computer vision to automatically discover internal defects of products to be inspected and calculate the size of the defects as a reference for product rejection. The system has an automatic learning function, high detection efficiency, and can be applied to a variety of light sources. In terms of automatic identification and classification of complex defects, the system has higher flexibility and accuracy, meeting the needs of high-precision detection. It can realize real-time monitoring and feedback, facilitating the optimization of the production process. In addition, the algorithm is universal and robust, and can adapt to product switching on different production lines. Description of the drawings:

[0044] Figure 1 It is a schematic diagram of the process of the present invention. Specific implementation method:

[0045] The present invention will be further described below with reference to the accompanying drawings:

[0046] An intelligent self-learning visual inspection system, such as Figure 1 As shown, the system is composed of an acquisition system, a data engine, an iteration engine, a correction system, a detection system and an NG / OK judgment system. In this embodiment, X-ray light is used as a light source for detailed description.

[0047] The acquisition system is responsible for capturing X-ray images of the workpiece and transmitting these images to the data engine or correction system. The technical details of the acquisition system are common practice in this field and are not detailed here. The inspection system's requirements for formatted data from the acquisition system are discussed here. The correction system requires the following formatted data: the image, the workstation number, and the corresponding six-degree-of-freedom robotic arm parameters. Specifically, the parameters include xc, yc, zc, w1, w2, and w3; xc, yc, and zc represent the robotic arm's position, and w1, w2, and w3 represent the robotic arm's posture.

[0048] The data engine is responsible for storing and intelligently marking the data transmitted from the acquisition system to form the formatted data required for model iteration.

[0049] In this embodiment, the data engine includes three parts. The first part is the intelligent marking system. The intelligent marking system provided in the present invention is one of the key features that distinguishes the present invention from other detection system data modules; the second part is the data management system, which is responsible for storing different types of data, and the storage format meets the requirements of the iterative system; the third part is the image information summary, which functions to format the data description of the input original workpiece image, the generated defect image, and the defect mask image according to the defect type sampled by the defect sampler; the formatted data description is the basis for the data management system to store files; it is the basis for the iterative system to read training materials.

[0050] Specifically, the intelligent marking system includes:

[0051] A conditional generative deep learning model; the input of the conditional generative deep learning model obeys a normal distribution with a mean of 0 and a standard deviation of 1; the condition is a downsampled image of a local image of a workpiece; the number of downsampling times is adjustable, generally 4 to 6 times; the number of downsampling times in this embodiment is 4 times;

[0052] A conditional image generation deep learning model; the conditional image generation deep learning model takes the defect-free image of the workpiece and the defect mask as input; takes the defect type as a conditional input; and generates an image of the workpiece with the defect;

[0053] A standard normal distribution sampler, which is the input of the conditional generative deep learning model and is used to generate defect-free and variable workpiece images under the constraints of the current input workpiece image;

[0054] A defect shape sampler; its function is to generate a defect mask and, together with the output of the conditional generative deep learning model, form a workpiece image with the defect mask; this image serves as the input of the conditional image generation deep model; the defect shape sampler can be a defect shape distribution generated by using the deep learning model as a parameterization tool, or it can be a uniform sampling from a large number of defect shape sample libraries.

[0055] The workflow of this intelligent marking system is shown in the arrows of the system block diagram:

[0056] S10: Read the image x uploaded from the workstation by the acquisition system in real time;

[0057] S20: Downsample the image. According to the sampling number 4, the downsampled image is recorded as x4. Although the system block diagram shows that the sampling number is 4, it can be flexibly set to 2, 6, 8, etc. The validity of the authorization is not affected by the change of the sampling number.

[0058] S30: sampling s~N(0,1), where N represents normal distribution;

[0059] S40: Sampling y~N(μ θ (s|x4),σ), where μ θ (s|x4) represents the conditional generative neural network model;

[0060] S50: sampling s~N(0,1);

[0061] S60: sampling m~N(π θ (s),σ), where π θ (s) represents the parameterized function in the defect morphology sampler, which can be a neural network or a self-defined elementary function; the sampled m is binarized, and the processed result is still recorded as m;

[0062] S70: According to formula y m =y*m generates a normal workpiece image with a mask;

[0063] S80: defect type sample s, where s is a text description of the defect type of the workpiece to be tested;

[0064] S90: Sampling x d ~N(ρ θ (y m |s),σ), where ρ θ (y m |s) represents a conditional generative graph network;

[0065] S100:(x,x d , m, s) as a formatted data and enter the data management system;

[0066] Among them, each parameter is a random variable, y represents the normal workpiece image, m represents the defect mask, x represents the original image of the workpiece on the production line, and x d represents the defect image that meets the feature of workpiece x, and s is the defect type.

[0067] This completes the workflow of the intelligent marking system. This marking system ensures that the data distribution provided to the iteration engine is completely consistent with the production line data distribution; that the defect types provided to the iteration engine are rich and varied, which is conducive to model training and optimization; and that the data provided to the iteration engine is correctly labeled, with clear and accurate defect boundaries.

[0068] The data management system is responsible for storing the formatted data output by the intelligent marking system according to the requirements of the iteration engine. This is a common method in this field and will not be elaborated in detail. The storage format provided by the present invention is:

[0069] S1. Workpiece location

[0070] S1.1 Workpiece Image

[0071] S1.2 Semantic labels for artifact images.

[0072] The iterative engine of this embodiment consists of a neural network for semantic classification and a cost function for training the neural network. The iterative engine updates network parameters in real time based on the network cost function, causing the value of the cost function to continuously decrease. The update method can be random gradient descent or other currently available methods. This is a well-known technique and will not be further described.

[0073] The so-called neural network for semantic judgment is a discriminant network, denoted as: P θ (y|x); where x comes from the data generated by the data engine, and its distribution is consistent with the data distribution on the production line; the distribution of y is consistent with the distribution of the defect morphology sampler in the data engine, and θ is the parameter of the neural network; the discriminant network can be any of the currently available ones;

[0074] The so-called evaluation function of the neural network can be written as follows:

[0075]

[0076] where y * Represents the statistical distribution of the discriminative neural network output; y comes from the sampling of defects by the data engine and obeys the distribution of real defects.

[0077] After the detection system detects the image transmitted by the correction system, it sends a detection request to the iteration engine. After receiving the request from the detection system, the iteration engine calls the current optimal model for inference and returns the inference result to the detection system. The model inference involved in this module is a well-known technology and will not be described in detail.

[0078] The correction system, sometimes also referred to as an HDR system, stands for High Dynamic Range. HDR is a mapping or transformation that transforms images transmitted from the acquisition system into a dynamic range discernible by the human eye. Essentially, it is a visualization technology. HDR for X-rays has its own unique characteristics. This invention provides a novel HDR method that differs from existing publicly available information. Specific technical features are provided below:

[0079] In this embodiment, the correction system includes a tone mapping function, which is expressed as:

[0080] x out =T(x in )

[0081] where x in Indicates the input 16-bit tiff format image, with a value range of 0 to 65535; x out The output of the tone mapping, ranging from 0 to 1. The tone mapping can be any of the currently available technologies, either a traditional tone mapping function or a neural network.

[0082] The correction system also includes a smoothing function, which is the output x of the tone mapping function out For the sake of recording, use x blur Record the smoothed output; the smoothing function can be Gaussian blur, mean blur, or other publicly available smoothing functions;

[0083] At the same time, the correction system also includes a weighted fusion module and a noise estimation module. The formula of the weighted fusion is:

[0084] x final =(1+α)x out +αx blur

[0085] Where α is the fusion coefficient, and its value method will be given later; x final is the output of the correction system;

[0086] The noise estimation method of the noise module can adopt any of the currently available methods, and the estimated noise standard deviation is σ noise ;

[0087] In addition, the correction system also includes an estimation method for the fusion parameter ɑ, and the estimation formula is given here:

[0088]

[0089] The ɑ0, c, β and σ0 in the formula can be flexibly adjusted according to the image conditions of the acquisition system; a set of parameters for the implementation of the present invention is given here: ɑ0 = 2.0, c = 20.0, β = 1000, σ0 = 0.0001.

[0090] The NG / OK determination system is also a common method in this field and will not be described in detail.

[0091] The intelligent self-learning visual inspection system proposed by the present invention uses the intelligent recognition model of computer vision to automatically discover the internal defects of the product to be inspected and calculate the size of the defects as a reference for product rejection. It has an automatic learning function, high detection efficiency, and can be applied to a variety of light sources. In terms of automatic identification and classification of complex defects, the system has higher flexibility and accuracy, meeting the needs of high-precision detection. It can realize real-time monitoring and feedback, facilitating the optimization of the production process. In addition, the algorithm is universal and robust, and can adapt to product switching on different production lines.

[0092] It should be noted that, although this embodiment uses X-ray as the light source, the authorized protection scope of the present invention cannot be limited to the technical feature field using X-ray as the light source, nor can it be limited to the application field using X-ray as the light source. All technical fields and application fields that are similar to the system block diagram and the work flow, formulas, and module functions described in the description of the present invention are within the scope of protection required by the present invention; such as defect detection or poor appearance detection implemented by using X-ray, visible light, ultrasound, millimeter wave, etc. as light sources; such as application fields using X-ray, visible light, ultrasound, millimeter wave, etc. as light sources for defect detection or poor appearance detection; as long as the technical implementation matches the technical features given in the present invention, it is protected by the present invention.

Claims

1. An intelligent self-learning visual inspection system, characterized by: The system consists of an acquisition system, a data engine, an iteration engine, a correction system, a detection system, and an NG / OK judgment system. The acquisition system is responsible for collecting images of the workpiece to be tested and transmitting the detected images to the data engine or correction system; The data engine is used to store and intelligently mark the data transmitted by the acquisition system to form the formatted data required for model iteration; the data engine includes three modules: intelligent marking system, data management system, and image information aggregation; After detecting the image passed by the correction system, the detection system sends a detection request to the iteration engine. After receiving the request from the detection system, the iteration engine calls the current optimal model for inference and returns the inference result to the detection system. The acquisition system is used to perform HDR conversion on the transmitted images, mapping them to the dynamic range that the human eye can distinguish; The intelligent marking system includes a conditional generative deep learning model, a conditional image generation deep learning model, a standard normal distribution sampler, and a defect morphology sampler; the standard normal distribution sampler is the input of the conditional generative deep learning model, and is used to generate a defect-free and variable workpiece image under the constraints of the current input workpiece image; the conditional image generation deep learning model takes the flawless image and defect mask of the workpiece as input, and uses the defect type as a conditional input to generate a workpiece image with defects; the defect morphology sampler is used to generate a defect mask, and together with the output of the conditional generative deep learning model, forms a workpiece image with a defect mask; and this image is used as the input of the conditional image generation deep model; The correction system includes a tone mapping function, a smoothing function, a weighted fusion module, a noise estimation module and an estimation module of a fusion parameter α, wherein: The tone mapping function is recorded as: x out =T(x in ) where x in Indicates the input 16-bit tiff format image, with a value range of 0 to 65535; x out Represents the output of tone mapping, with a value range of 0 to 1; tone mapping is a tone mapping function or a neural network; The smoothing function is used to smooth the output x of the tone mapping function out Smoothing is performed and x blur Record the smoothed output; Weighted fusion module, the weighted fusion formula is: x final =(1+α)x out +αx blur Among them, α is the fusion coefficient, x final is the output of the correction system; The noise standard deviation estimated by the noise estimation module is σ noise ; The estimation formula of the estimation module of the fusion parameter α is as follows: α0, c, β and σ0 in the formula can be adjusted according to the image conditions of the acquisition system.

2. The intelligent self-learning visual inspection system according to claim 1, characterized in that: The workflow of the intelligent marking system is as follows: S10: Read the image x uploaded from the workstation by the acquisition system in real time; S20: Downsample the image and record the downsampled image as x according to the number of sampling times n. n : S30: sampling s~N(0,1), where N represents normal distribution; S40: Sampling y~N(μ θ (s|x4),σ), where μ θ (s|x4) represents the conditional generative neural network model; S50: sampling s~N(0,1); S60: Sample m~N(π θ (s),σ), where π θ (s) represents the parameterized function in the defect morphology sampler, which performs binary processing on the sampled m, and the processed result is still recorded as m; S70: According to formula y m =y*m generates a normal workpiece image with a mask; S80: Defect type sampling s, where s is a text description of the defect type of the workpiece to be tested; S90: Sampling x d ~N(ρ θ (y m |s),σ), where ρ θ (y m |s) represents a conditional generative graph network; S100: (x, x d , m, s) as a formatted data and enter the data management system; Among them, y represents the normal workpiece image, m represents the defect mask, x represents the original image of the workpiece, and x d represents the defect image that meets the feature of workpiece x, and s is the defect type.

3. The intelligent self-learning visual inspection system according to claim 1, characterized in that: The input of the conditional generative deep learning model follows a normal distribution with a mean of 0 and a standard deviation of 1. The condition is a downsampled image of a local image of the workpiece. The number of downsampling times is adjustable, ranging from 2 to 8 times.

4. The intelligent self-learning visual inspection system according to claim 1, characterized in that: The defect shape sampler is a defect shape distribution generated by a deep learning model as a parameterized tool, or a uniform sampling from a defect shape sample library.

5. The intelligent self-learning visual inspection system according to claim 2, characterized in that: The iterative engine consists of a neural network for semantic classification and a cost function for training the neural network. The iterative engine updates the network parameters in real time according to the network cost function. The neural network is a discriminant network, denoted as: P θ (y|x); where the distribution of x is consistent with the data distribution on the production line; the distribution of y is consistent with the distribution of the defect morphology sampler in the data engine, and θ is the parameter of the neural network; the discriminant network can be any of the currently available ones; the evaluation function of the neural network can be written as follows: where y * Represents the statistical distribution of the discriminative neural network output; y comes from the sampling of defects by the data engine and obeys the distribution of real defects.

6. The intelligent self-learning visual inspection system according to claim 1, characterized in that: In the estimation formula of the estimation module of the fusion parameter α, α0=2.0, c=20.0, β=1000, σ0=0.0001.

7. The intelligent self-learning visual inspection system according to claim 1, characterized in that: The light source is any one of X-ray light, visible light, ultrasonic wave or millimeter wave.

Images