A defect detection method and device based on snapshot compression imaging

Abstract

The present application relates to the field of defect detection technology, and more specifically, to a defect detection method and device based on snapshot compression imaging, which can solve the problem that the image resolution becomes lower as the acquisition frequency and imaging speed of the camera increase. The defect detection method includes: during the product transmission process, continuously acquiring a first image of the product at a first frequency, and imaging the first image on a mask structure; the mask structure rotating at a uniform speed sequentially modulates the intensity of the first image acquired at the first frequency to obtain a second image; and reconstructing the second image to obtain a third image, the third image corresponding to the first image continuously acquired at the first frequency, and the third image is used to determine the defects of the product.

Description

Technical Field

[0001] The present application relates to the field of high-speed imaging technology, and in particular to a defect detection method and device based on snapshot compression imaging. Background Art

[0002] With the advancement of modern technology, products tend to be more precise and miniaturized, so the inspection of product defects and flaws has become more stringent. Visual inspection technology has been applied to many processing and production processes to replace manual inspection due to its advantages such as high efficiency, stability, and easier observation of tiny products than the human eye.

[0003] In the implementation of some visual inspection technologies applied to defect detection, the sample moves at a constant speed in a certain direction on a single-axis motion table, and a strip light illuminates one row of the strip samples. The reflected light is collected by the line scan lens and the camera to form an image. In the case of higher requirements for production speed and efficiency, we need to improve the inspection quality and efficiency. In order to improve the inspection efficiency, we need to speed up the movement speed of the product to increase the inspection speed.

[0004] However, as the product's movement speed increases, we need to increase the camera's acquisition frequency and imaging speed accordingly. However, in this scenario, the image resolution decreases as the imaging speed increases. Summary of the invention

[0005] In order to solve the problem that the image resolution becomes lower as the camera's acquisition frequency and imaging speed increase when inspecting products, the present application provides a defect detection method and device based on snapshot compression imaging:

[0006] According to one aspect of an embodiment of the present application, a defect detection method based on snapshot compression imaging is provided, the method comprising:

[0007] During the product conveying process, continuously acquiring a first image of the product at a first frequency, wherein the first image includes an illumination area of ​​the product by a bar light source, and the first image is imaged on a mask structure;

[0008] The uniformly rotating mask structure sequentially modulates the intensity of the first image acquired at a first frequency to obtain a second image, wherein the second image is formed by compressing a plurality of the first images after intensity modulation, and the mask structure includes a plurality of modulation modes;

[0009] The second image is reconstructed to obtain a third image, where the third image corresponds to the first image continuously acquired at a first frequency, the third image is a theoretical lossless restoration image of the first image, and the third image is used to determine defects in the product.

[0010] In some embodiments, in the step of sequentially performing intensity modulation on the first image acquired at the first frequency by the uniformly rotating mask structure, the method further comprises:

[0011] The imaging position on the mask structure corresponding to the first image is the first position. When the mask structure rotates at a uniform speed, the modulation pattern passed by the first position at that moment performs intensity modulation on the first image imaged on the mask structure at that moment.

[0012] In some embodiments, in the step of obtaining a second image by sequentially performing intensity modulation on the first image acquired at a first frequency by the uniformly rotating mask structure, the method further comprises:

[0013] Receiving a first image of the product continuously acquired at a first frequency, and sequentially performing intensity modulation on the first image through a uniformly rotating mask structure, wherein a plurality of modulation modes are arranged on the mask structure, and in a uniformly rotating state, the plurality of modulation modes sequentially perform intensity modulation on the first image;

[0014] The first image after intensity modulated on the mask structure is collected, and the first image is compressed by a line array camera to obtain a second image. The line array camera is used to receive the collected first image after intensity modulated on the mask structure, and compress multiple modulated first images so that the multiple first images form one image, which is the second image.

[0015] In some embodiments, in the step of receiving the first image of the product continuously acquired at the first frequency, and sequentially performing intensity modulation on the first image by means of a mask structure rotating at a uniform speed, the method further comprises:

[0016] The mask structure is a photolithography disk used for intensity modulation of the first image. A plurality of patterns for modulation are etched on the mask structure, and the plurality of patterns for intensity modulation are respectively arranged on a plurality of radii of the mask structure.

[0017] In some embodiments, in the step of acquiring the first image after intensity modulation on the mask structure and compressing the first image through a line array camera to obtain a second image, the method further comprises:

[0018] In the process of intensity modulating the first image, continuously acquiring a modulated image on the mask structure, the modulated image including the first image after intensity modulating, the modulated image being imaged on the line array camera;

[0019] The linear array camera sequentially combines the continuously acquired modulated images to obtain the second image, and the number of the modulated images included in the second image is determined by the acquisition frequency and exposure time of the lens.

[0020] In some embodiments, in the step of receiving the first image of the product continuously acquired at the first frequency, and sequentially performing intensity modulation on the first image by means of a mask structure rotating at a uniform speed, the method further comprises:

[0021] The rotation speed of the mask structure is related to the first frequency. As the mask structure rotates at a uniform speed, the multiple patterns for modulation respectively etched on multiple radii respectively perform intensity modulatory signals on the first image of the product that is continuously acquired. When the first image of the product is continuously acquired, the mask structure can provide different modulation modes for the first image through the multiple patterns for modulation respectively etched on multiple radii.

[0022] In some embodiments, during the product conveying process, a first image of the product is continuously acquired at a first frequency, the first image includes an illumination area of ​​the product by a bar light source, and the first image is imaged in a mask structure step, and the method further includes:

[0023] The imaging light reflected by the product after being illuminated by the strip light source is collected at a first frequency through a first lens, wherein the light is a line on the product and is used to form an image on a mask structure, and the first frequency is a fixed frequency, and the first frequency is determined by the transmission speed of the product and the hardware conditions of the first lens;

[0024] The collected light is propagated to the mask structure, imaged on the mask structure, and the first image is obtained. The light is continuously acquired at a first frequency and transmitted to the mask structure to form an image on the mask structure.

[0025] In some embodiments, in the step of reconstructing the second image to obtain a third image, the method further comprises:

[0026] An approximate image of the second image is calculated by an iterative algorithm to gradually approach the first image, and a third image is obtained after multiple iterations. In the iterative algorithm, image denoising is performed by a neural network.

[0027] In some embodiments, during the product conveying process, a first image of the product is continuously acquired at a first frequency, the first image includes an illumination area of ​​the product by a bar light source, and the first image is imaged in a mask structure step, and the method further includes:

[0028] The strip light source is arranged on one side of a conveyor belt conveying the product, and the area illuminated by the strip light source on the conveyor belt is a first area, the first area is fixedly arranged, and the third image is used to determine defects of the product corresponding to the first area.

[0029] According to another aspect of the embodiments of the present application, a defect detection device based on snapshot compression imaging is provided, which is applied to any of the above methods. The device includes:

[0030] A strip light source, which is arranged on one side of a conveyor belt for conveying products, and is arranged toward the conveyor belt, and is used to provide lighting for products passing on the conveyor belt;

[0031] A first lens, which is disposed on a side of the conveyor belt where the strip light source is disposed, and is used to collect reflected light from an area on the product illuminated by the strip light source, and to image the reflected light on a mask structure to obtain a first image;

[0032] A mask structure, wherein the mask structure is arranged in the imaging direction of the first lens, a plurality of modulation modes are arranged on the mask structure, and the mask structure is connected to a rotating motor, wherein the rotating motor is used to drive the mask structure to rotate at a constant speed, and the mask structure is used to perform intensity modulation on a plurality of the first images imaged on the mask structure through the imaging lens;

[0033] A line array camera is used to receive the modulated image on the mask structure collected by the second lens, and compress a plurality of modulated images to obtain a second image.

[0034] The beneficial effects of the present application are as follows: by acquiring a first image, it is possible to capture an image of the area illuminated by a strip light source when the product passes through a first area; further, by intensity modulating and compressing the first image, a second image for reconstruction can be acquired; further, through the second image, a third image to be output is determined, and the third image is used to determine defects in the product, thereby overcoming the problem of reduced resolution after increasing the acquisition frequency and imaging speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0036] Figure 1 A schematic diagram of a defect detection method in an embodiment of the present application is shown;

[0037] Figure 2 A schematic diagram of a process of pre-modulating and compressing a first image in another embodiment of the present application is shown;

[0038] Figure 3 A schematic diagram of a process of compressing a modulated first image in another embodiment of the present application is shown;

[0039] Figure 4 A schematic diagram of a process for acquiring a first image in another embodiment of the present application is shown;

[0040] Figure 5 A schematic diagram showing a process of reconstructing a second image in another embodiment of the present application is shown;

[0041] Figure 6 A schematic diagram of the structure of a defect detection device provided by an embodiment of the present application is shown;

[0042] Figure 7 shows a schematic diagram of the structure of line scan detection;

[0043] Figure 8 A schematic diagram showing normal exposure and delayed exposure of a line scan camera in the prior art is shown;

[0044] Fig. 9 A schematic diagram of a curve showing resolution and sampling frequency is shown;

[0045] Fig.10 A schematic diagram showing the effect of reconstructing a compressed video frame;

[0046] Fig.11 A schematic diagram of the process of compressing a motion scene is shown;

[0047] Fig.12 A schematic diagram showing the process of compressing n modulated images into a single image;

[0048] Fig.13 A schematic diagram showing the process of reconstructing n modulated images from a single image is shown;

[0049] Fig.14 A schematic diagram of the process of image reconstruction using an iterative algorithm is shown;

[0050] Fig.15 A schematic diagram showing the process of image reconstruction using a neural network.

[0051] Description of the drawings: 100, strip light source; 200, first lens; 300, reflector; 400, mask structure; 500, second lens; 600, linear array camera. DETAILED DESCRIPTION

[0052] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0053] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0054] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0055] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0056] With the advancement of modern technology, products tend to be more precise and miniaturized, so the inspection of product defects and flaws has become more stringent. Visual inspection technology has been used in many processing and production processes to replace manual inspection due to its advantages of high efficiency, stability, and easier observation of tiny products than the human eye. However, in the field of industrial machine vision, how to improve the speed of detection while ensuring detection accuracy has always been a pain point and difficulty, especially for two-dimensional single-axis motion detection represented by printing and packaging detection. The speed of its single-axis operation directly determines the production speed and efficiency. How to perform high-resolution imaging of fast-moving products is a key and difficult problem that needs to be solved urgently.

[0057] Figure 7 shows a schematic diagram of the structure of line scan detection, Figure 8 A schematic diagram showing normal exposure and delayed exposure of a line scan camera in the prior art is shown.

[0058] Usually when testing samples, such as Figure 7As shown in the figure, the sample to be tested moves at a constant speed along a certain direction on a single-axis motion stage. The strip light illuminates one row of the strip sample, and the reflected light is collected by the line scan lens and camera to form an image. The line scan camera only receives a single row of images each time, and exposes them at regular intervals to form the second row. During the interval between exposures, the sample still keeps moving. If the camera does not expose for a long time after one exposure, a pattern similar to a trailing will be formed, such as Figure 8 As shown, the sample moves from top to bottom, and the camera collects one line of image each time. The lower half is the image during normal exposure, which can form a normal collected image. When the refresh stops, the image will stay at the single line of the last collection and repeat continuously, forming a tail in the direction of movement. With the development of society, the requirements for quality and efficiency are constantly increasing, and the speed of detection also determines the production speed and efficiency. Therefore, increasing the movement speed of the product during detection to increase the detection speed is the most common practice.

[0059] However, when inspecting products, as the product transmission speed increases, the camera's acquisition frequency also needs to be increased. However, after the imaging speed is increased, the image resolution will become lower.

[0060] In existing technical solutions, a simple stacking hardware method is usually used to solve the contradiction between imaging speed and resolution in the direction of motion. By placing twice as many light sources, lenses + camera detection systems in the direction of motion, the imaging resolution in the direction of motion can be increased by a corresponding factor.

[0061] Fig. 9 A schematic diagram of a curve of resolution and sampling frequency is shown.

[0062] However, simply stacking hardware devices or replacing high-performance cameras will increase costs exponentially. At the same time, too many devices increase the overall complexity and difficulty of assembly and adjustment of the assembly line equipment. In addition, hardware devices take up space and cannot be stacked infinitely, which leads to bottlenecks in improving the effect. In fact, Figure 8 As shown, the contradiction between resolution and sampling speed exists in a space-bandwidth product curve in theory. Without changing the optical design, the limitation of this curve cannot be broken, which does not help improve the overall detection efficiency.

[0063] Therefore, in response to the above problems, the present application proposes a defect detection method and device based on snapshot compression imaging, which can use the lossless recovery characteristics of snapshot compression imaging, introduce it into the field of defect detection of assembly line products, break through the current technical bottleneck, and achieve an increase in detection speed.

[0064] Fig.10 A schematic diagram showing the effect of reconstructing compressed video frames.

[0065] like Fig.10As shown, Compressed Sensing is a computational imaging technology based on the restoration of a single-frame original image, which is used to improve spatial and temporal resolution or capture video and hyperspectral images. Snapshot compression imaging (SCI) is a method that compresses a multi-frame video sequence image into a single-frame 2D image through modulation methods such as masks, and restores it through subsequent algorithms. To be precise, SCI compresses the target high-dimensional image data into a two-dimensional image, and reconstructs the high-dimensional data using an algorithm. At present, SCI is mainly used for the reconstruction of three-dimensional data (x, y, z), the restoration of video data (x, y, t), and the acquisition of hyperspectral images (x, y, λ). In the present invention, SCI is used to restore video data, such as Fig.10 As shown, a compressed image is restored to multiple original images.

[0066] Fig.11 FIG. 4 shows a schematic diagram of the process of compressing a motion scene. Fig.12 The schematic diagram shows the process of compressing n modulated images into a single image. Fig.13 The figure shows a schematic diagram of the process of reconstructing n modulated images from a single image.

[0067] The compression process of video data is as follows: Fig.11 As shown in the figure, the light emitted by the moving scene is collected by the objective lens, and then the wavefront intensity is modulated by the optical modulator. The modulated image is then imaged onto the camera (CCD or CMOS) by the relay lens. The camera maintains a long exposure time, so each captured image corresponds to the sum of several modulated intensity images. Assuming that the camera captures one image for every n images modulated, the compression ratio of the system is n:1, as Fig.12 shown.

[0068] After the camera collects m compressed images, since each image is compressed by n modulated images, at most m·n images can be restored losslessly to form a video frame. This process is opposite to the image compression / modulation process and is called the image demodulation process. The demodulation process requires the use of an inherent compressed sensing algorithm. Its basic process is as follows: Fig.13 shown.

[0069] The method in some embodiments of the present application can be used to solve the technical problem of low image resolution in the moving direction when the sample moves at high speed, and can further improve the detection rate, reduce the over-detection rate, and reduce material costs and structural system complexity.

[0070] Specifically, it is possible to solve defect detection problems through a single workstation; increase the running speed of samples while maintaining resolution; maintain the performance and cost of the optoelectronic device itself, and break through the space-bandwidth product curve from a system design perspective.

[0071] Combine the following Figure 1 - Figure 4 The high-speed imaging method provided by the present application is described.

[0072] Figure 1 A schematic flow chart of a defect detection method in an embodiment of the present application is shown, and the method can be executed by a defect detection device based on snapshot compression imaging.

[0073] like Figure 1 As shown, the method comprises the following steps:

[0074] Step 110: During the product conveying process, continuously acquire a first image of the product at a first frequency.

[0075] The first image includes the lighting area of ​​the product by the strip light source, and the first image is imaged on the mask structure. During the transmission of the product, the light reflected by the product is collected by the lens, and the first image is imaged on the mask structure.

[0076] Step 120: The uniformly rotating mask structure sequentially performs intensity modulation on the first image acquired at the first frequency to obtain a second image.

[0077] Among them, the second image is formed by compressing multiple first images after intensity modulation, the mask structure includes multiple modulation modes, the multiple modulation modes are distributed in the mask structure according to a certain rule, and the intensity of the acquired first image is modulated in a certain order as the mask structure rotates at a uniform speed. Intensity modulation refers to laser oscillation in which the intensity of the optical carrier varies with the modulation mode.

[0078] Step 130: Reconstruct the second image to obtain a third image.

[0079] Among them, the third image corresponds to the first image continuously acquired at the first frequency. The third image is a theoretical lossless restoration image of the first image. The third image is used to determine defects in the product. Reconstruction refers to restoring the image compressed according to a certain pattern to the original image according to a subsequent algorithm. With multiple iterations of the algorithm, the calculation result is compared with the actual compression result obtained by actual acquisition. When the pixel difference between the two is small enough, the calculation result can be considered to be the real image, that is, the third image.

[0080] It can be seen that by acquiring the first image, it is possible to acquire the image of the product in the area illuminated by the strip light source; further, by intensity modulating and compressing the first image, a second image for reconstruction can be acquired; further, through the second image, the third image to be output is determined, and the third image is used to determine defects in the product, thereby overcoming the problem of reduced resolution after increasing the acquisition frequency.

[0081] In some embodiments, in the step of sequentially performing intensity modulation on the first image acquired at the first frequency by the uniformly rotating mask structure, the method further comprises:

[0082] The imaging position on the mask structure corresponding to the first image is the first position. When the mask structure rotates at a uniform speed, the modulation pattern passed by the first position at that moment performs intensity modulation on the first image imaged on the mask structure at that moment.

[0083] Among them, the acquired first image will be projected on a fixed position on the mask structure, and this position is the first position. As the mask structure rotates at a uniform speed, each part of the mask structure will pass through the first position. Therefore, at each moment, the modulation mode set for the part of the mask structure passing through the first position at that moment will perform intensity modulation on the first image imaged at the first position at that moment, the modulation mode.

[0084] It can be seen that the uniformly rotating mask structure can conveniently and quickly realize accurate dynamic modulation when the product is in high-speed motion. Every time t passes, a total of n lines of sample images are collected, which are compressed into one line of pixels by n different patterns on the mask structure, realizing image acquisition with a compression ratio of n:1.

[0085] Figure 2 A schematic diagram of a process of pre-modulating and compressing a first image in another embodiment of the present application is shown.

[0086] In some embodiments, Figure 2 As shown, in the step of obtaining a second image by sequentially modulating the intensity of the first image acquired at the first frequency with the mask structure rotating at a uniform speed, the method further includes:

[0087] Step 210: receiving a first image of a product continuously acquired at a first frequency, and sequentially performing intensity modulation on the first image through a mask structure rotating at a uniform speed.

[0088] Wherein, a plurality of modulation modes are arranged on the mask structure, and in a state of uniform rotation, the plurality of modulation modes perform intensity modulation on the first image in turn.

[0089] Step 220: Acquire the intensity-modulated first image on the mask structure, and compress the first image through a line array camera to obtain a second image.

[0090] The linear array camera is used to receive the intensity-modulated first image on the acquired mask structure, and compress a plurality of modulated first images to form one image from the plurality of first images, which is the second image.

[0091] It can be seen that since the sample is moving at high speed, different modulation modes need to be applied to different frames according to the principle of snapshot compression imaging. Therefore, the mask structure needs to keep rotating at a high speed and uniform speed along with the high-speed uniform motion of the sample to be tested, and different modulation modes are applied to the adjacent images. The mask structure is set in a disk form, which is convenient for setting different modulation modes on multiple radii, and for intensity modulation of the first image by rotation. At every moment t, the imaging lens collects a total of n rows of sample images, which are intensity modulated by n different modes on the mask structure, i.e., n radii, and are finally compressed into a row of pixels in the linear array camera, thereby realizing image acquisition with a compression ratio of n:1.

[0092] In some embodiments, in the step of receiving a first image of a product continuously acquired at a first frequency and sequentially performing intensity modulation on the first image by a mask structure rotating at a uniform speed, the method further comprises:

[0093] The mask structure is a photolithography disk used for intensity modulation of the first image. A plurality of patterns for modulation are etched on the mask structure, and the plurality of patterns for intensity modulation are respectively arranged on a plurality of radii of the mask structure.

[0094] Wherein, a plurality of patterns for intensity modulation are arranged on the mask structure at equal distances and all coincide with the radius of the mask structure.

[0095] It can be seen that since there are countless points on the disk, multiple patterns for intensity modulation are set on the mask structure at equal distances and coincide with the radius of the mask structure, so that dynamic intensity modulation can be better performed when the mask structure rotates at a uniform speed.

[0096] Figure 3 A schematic diagram of a process of compressing a modulated first image in another embodiment of the present application is shown.

[0097] In some embodiments, Figure 3 As shown, in the step of acquiring the first image after intensity modulation on the mask structure and compressing the first image through a linear array camera to obtain a second image, the method further includes:

[0098] Step 310: During the process of intensity modulating the first image, continuously acquiring the modulated image on the mask structure.

[0099] The modulated image includes a first image after intensity modulation, and the modulated image is imaged on a linear array camera.

[0100] Step 320: The linear array camera sequentially combines the modulated images continuously acquired to obtain a second image.

[0101] The number of modulated images contained in the second image is determined by the acquisition frequency and exposure time of the lens.

[0102] It can be seen that the received modulated first image can be imaged by the linear array camera to make it a row in the second image. After the modulated images on the mask structure are continuously acquired and imaged on the linear array camera in sequence, they will be merged into a complete image, namely the second image.

[0103] In some embodiments, in the step of receiving a first image of a product continuously acquired at a first frequency and modulating the intensity of the first image by a mask structure rotating at a uniform speed, the method further comprises:

[0104] The rotation speed of the mask structure is related to the first frequency. As the mask structure rotates at a uniform speed, the multiple modulation patterns etched on the multiple radii respectively perform intensity modulatory modulation on the first image of the product that is continuously acquired. When the first image of the product is continuously acquired, the mask structure can provide different modulation modes for the first image through the multiple modulation patterns etched on the multiple radii.

[0105] Among them, the first frequency is related to the speed of collecting the first image through the lens. The mask structure needs to perform intensity modulation on the first image imaged thereon, so its rotation speed is related to the first frequency. At the same time, the first frequency is related to the transmission speed of the lens itself and the product. Only in this way can the mask structure provide different modulation modes for the first image by means of multiple modulation patterns etched on multiple radii while continuously acquiring the first image of the product.

[0106] It can be seen that by controlling the rotation speed of the mask structure to make it related to the first frequency and the conveying speed of the product, multiple modulation modes can be used more effectively to modulate the intensity of the first image of the continuously acquired product to perform accurate dynamic modulation.

[0107] Figure 4 A schematic diagram of a process for acquiring a first image in another embodiment of the present application is shown.

[0108] In some embodiments, Figure 4 As shown, during the product conveying process, a first image of the product is continuously acquired at a first frequency, the first image includes an illumination area of ​​the product by a strip light source, and the first image is imaged in a mask structure step. The method further includes:

[0109] Step 410: Using a first lens, collect imaging light reflected by the product after being illuminated by a bar light source at a first frequency.

[0110] Among them, the imaging light is the light reflected by a line illuminated by the strip light source on the product, and the imaging light is used to perform imaging on the mask structure. The first frequency is a fixed frequency, and the first frequency is determined by the transmission speed of the product and the hardware conditions of the first lens. The first lens refers to the imaging lens, which is used to collect light and perform imaging.

[0111] Step 420: propagate the collected light to the mask structure, image the mask structure, and obtain a first image.

[0112] Among them, light is continuously acquired at a first frequency and transmitted to the mask structure to form an image on the mask structure. Imaging refers to the real image projected on the mask structure after the light is collected by the lens and refracted, diffracted or propagated in a straight line through a small hole.

[0113] It can be seen that the imaging light reflected by a line illuminated by a strip light source on the product is collected using a lens at a fixed first frequency, and the imaging light is propagated to the mask structure for imaging. The structure is simple, the collection effect is good, and a better first image can be obtained.

[0114] Fig.14 A schematic diagram showing the process of image reconstruction using an iterative algorithm is shown. Fig.15 A schematic diagram showing the process of image reconstruction using a neural network.

[0115] like Fig.14 As shown in the figure, the traditional demodulation algorithm of compressed sensing images is mainly based on iterative algorithms combined with regularization methods. Iterative algorithms represented by GAP-TV and DeSCI take the known modulation mode as prior knowledge, starting from the compressed image obtained, and gradually approaching the real image through different mathematical iterative algorithms. In each iteration, the calculation result is compressed again and compared with the real compression result obtained by actual acquisition. When the pixel difference between the two is small enough, the calculation result can be considered to be the real image. The traditional iterative algorithm has a complete mathematical model and theoretical derivation, and has a high confidence level, so it is favored by many users. However, the traditional iterative algorithm requires multiple rounds of iterations and is slow. It often takes tens of minutes or even longer to calculate an image, which is completely unbearable for pipeline application scenarios.

[0116] like Fig.15As shown, in recent years, with the gradual rise of deep learning methods, using neural networks to accelerate the reconstruction process has become a new hot idea. The use of end-to-end networks (E2E) can accurately reconstruct blurred compressed images, but it also has its own problems: for each sample to be tested, the network needs to be trained separately, otherwise the reconstruction effect will be greatly reduced.

[0117] Figure 5 A schematic diagram of a process of reconstructing a second image in another embodiment of the present application is shown.

[0118] In some embodiments, Figure 5 As shown, in the step of reconstructing the second image to obtain the third image, the method further includes:

[0119] An approximate image of the second image is calculated through an iterative algorithm to gradually approach the first image, and a third image is obtained after multiple iterations. In the iterative algorithm, image denoising is performed through a neural network.

[0120] Among them, the iterative algorithm, also known as the rolling algorithm, refers to a process of constantly using the old value of a variable to recursively deduce a new value. The iterative algorithm is a basic method for solving problems with computers. It uses the characteristics of computers that are fast and suitable for repetitive operations to allow the computer to repeatedly execute a set of instructions (or certain steps). Each time this set of instructions (or these steps) is executed, a new value of the variable is derived from its original value. The iterative method is divided into precise iteration and approximate iteration. A neural network refers to an algorithmic mathematical model that imitates the behavioral characteristics of animal neural networks and performs distributed parallel information processing. This network relies on the complexity of the system to adjust the interconnected relationships between a large number of internal nodes to achieve the purpose of processing information.

[0121] It can be seen that in some embodiments of the present application, the compressed image restoration and reconstruction process can be performed after the compressed image is obtained. The present invention combines the neural network in the deep learning method with the iterative framework in the traditional iterative algorithm. Its basic framework process still uses iterative calculation, but in each iterative operation process, the most time-consuming image denoising process will be completed by the neural network. The neural denoising network used here does not need to be retrained. There are a large number of denoising networks that have been trained on the Internet. They can be directly applied in the iterative framework to achieve image denoising.

[0122] This plug-and-play method avoids the shortcomings of traditional iterative methods that take too long, and also solves the problems of multiple long-term training of E2E networks and the lack of a complete mathematical model. The calculation time for a single image is compressed from minutes to milliseconds, making it possible to combine it with pipeline detection to form a complete, high-speed detection method based on snapshot compression imaging.

[0123] In some embodiments, during the product conveying process, a first image of the product is continuously acquired at a first frequency, the first image includes an illumination area of ​​the product by a bar light source, and the first image is imaged in a mask structure step, and the method further includes:

[0124] The strip light source is arranged on one side of a conveyor belt for conveying products. The area illuminated by the strip light source on the conveyor belt is a first area. The first area is fixedly arranged. The third image is used to determine defects of the product corresponding to the first area.

[0125] Among them, the installation position of the strip light source is fixed, and its lighting direction is also fixed. Therefore, the fixed lighting area of ​​the strip light source on the conveyor belt is the first area. When the product passes through, part of its upper position is illuminated by the strip light source, and the reflected light is collected by the lens to form a first image for subsequent processing, and finally a third image for determining defects is obtained. This third image is used to determine the defects of the first area when the product passes through and is illuminated by the strip light source, that is, reading English.

[0126] It can be seen that by fixing the strip light source on one side of the conveyor belt that transports the product, a fixed first area can be obtained. When the product passes through, the first image of the product corresponding to the first area can be clearly captured, and the corresponding third image can be obtained after processing, which is used to determine the defects of the product corresponding to the first area.

[0127] Combine the following Figure 6 The defect detection device based on snapshot compression imaging provided in the present application is described. The defect detection device based on snapshot compression imaging described below and the defect detection method based on snapshot compression imaging described above can refer to each other.

[0128] Figure 6 FIG. 1 shows a schematic diagram of the structure of a defect detection device provided by an embodiment of the present application. Figure 6 As shown, the defect detection device based on snapshot compression imaging includes:

[0129] The bar-shaped light source 100 is disposed on one side of a conveyor belt for conveying products, and is disposed toward the conveyor belt, and is used to provide lighting for the products passing on the conveyor belt.

[0130] The first lens 200 is disposed on one side of the conveyor belt where the bar light source 100 is disposed, and is used to collect reflected light from the area illuminated by the bar light source on the product, and image the reflected light on the mask structure 400 to obtain a first image.

[0131] The mask structure 400 is arranged in the imaging direction of the first lens 200. A plurality of modulation modes are arranged on the mask structure 400. The mask structure 400 is connected to a rotating motor. The rotating motor is used to drive the mask structure 400 to rotate at a uniform speed. The mask structure 400 is used to perform intensity modulation on a plurality of first images imaged on the mask structure 400 through the imaging lens.

[0132] The line array camera 600 is used to receive the modulated image on the mask structure 400 collected by the second lens 500, and compress a plurality of modulated images to obtain a second image.

[0133] The controller is used to receive the second image transmitted by the linear array camera 600, and reconstruct the second image to obtain multiple third images. The third image corresponds to the first image continuously acquired at the first frequency. The third image is a theoretically restored image of the first image. The third image is used to determine defects in the product.

[0134] Among them, the distance from the first lens 200 to the product is the working distance of the first lens 200; the distance from the first lens 200 to the mask structure 400 is the image distance of the first lens 200; the distance from the second lens 500 to the mask structure 400 is the working distance of the second lens 500; the distance from the second lens 500 to the line array camera 600 is the image distance of the second lens 500, and the setting angles can make the light vertically incident on the surfaces of each component. A reflector 300 can also be set between the first lens 200 and the mask structure 400. When the on-site conditions limit the placement position, the light can be refracted by the reflector 300 to facilitate light propagation.

[0135] Some embodiments of the present application are developed based on line scanning scenarios. The sample moves at a uniform high speed in a single direction on the translation stage. The intensity modulation of the mask structure 400 and the camera acquisition are performed in the form of a single row of images each time, and finally a complete two-dimensional image is formed in the direction of movement. Therefore, the scene applicable to the present invention is a one-dimensional scene. The illumination light source is a strip line scanning light source, which illuminates a line on the product. After the image on the line is illuminated, the reflected light is collected by the first lens 200 and imaged onto the mask structure 400. The mask structure 400 is a photolithography disk etched with a pattern for modulation. After each line on the sample is intensity modulated by each radius on the mask structure 400, it is imaged onto the line array camera 600 by the second lens 500.

[0136] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion in some embodiments is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A defect detection method based on snapshot compression imaging, characterized in that: The method comprises: During the product conveying process, continuously acquiring a first image of the product at a first frequency, wherein the first image includes an illumination area of ​​the product by a bar light source, and the first image is imaged on a mask structure; receiving a first image of the product continuously acquired at a first frequency, and sequentially performing intensity modulation on the first image through a uniformly rotating mask structure, wherein the rotation speed of the mask structure is related to the first frequency, and as the mask structure rotates at a uniform speed, a plurality of modulation patterns respectively etched on a plurality of radii respectively perform intensity modulation on the first image of the product continuously acquired, so that when the first image of the product is continuously acquired, the mask structure can provide different modulation modes for the first image through the plurality of modulation patterns respectively etched on the plurality of radii; Acquire the first image after intensity modulated on the mask structure, and compress the first image through a line array camera to obtain a second image, wherein the line array camera is used to receive the acquired first image after intensity modulated on the mask structure, and compress a plurality of modulated first images so that the plurality of first images form one image, which is the second image; The second image is reconstructed, and an approximate image of the second image is calculated through an iterative algorithm to gradually approach the first image. After multiple iterations, a third image is obtained. In the iterative algorithm, image denoising is performed through a neural network. The third image corresponds to the first image continuously acquired at a first frequency, and the third image is a theoretical lossless restoration image of the first image. The third image is used to determine defects in the product.

2. A defect detection method based on snapshot compression imaging as claimed in claim 1, characterized in that: In the step of sequentially performing intensity modulation on the first image acquired at the first frequency by the uniformly rotating mask structure, the method further comprises: The imaging position on the mask structure corresponding to the first image is a first position. When the mask structure rotates at a uniform speed, the modulation pattern passed by the first position performs intensity modulation on the first image imaged on the mask structure.

3. A defect detection method based on snapshot compression imaging as claimed in claim 1, characterized in that: In the step of receiving the first image of the product continuously acquired at the first frequency, and sequentially performing intensity modulation on the first image by means of a mask structure rotating at a uniform speed, the method further comprises: The mask structure is a photolithography disk used for intensity modulation of the first image. A plurality of patterns for modulation are etched on the mask structure, and the plurality of patterns for intensity modulation are respectively arranged on a plurality of radii of the mask structure.

4. A defect detection method based on snapshot compression imaging as claimed in claim 1, characterized in that: In the step of acquiring the first image after intensity modulation on the mask structure and compressing the first image by a line array camera to obtain a second image, the method further comprises: In the process of intensity modulating the first image, continuously acquiring a modulated image on the mask structure, the modulated image including the first image after intensity modulating, the modulated image being imaged on the line array camera; The linear array camera sequentially combines the continuously acquired modulated images to obtain the second image, and the number of the modulated images included in the second image is determined by the acquisition frequency and exposure time of the lens.

5. A defect detection method based on snapshot compression imaging as claimed in claim 1, characterized in that: During the product conveying process, a first image of the product is continuously acquired at a first frequency, the first image includes an illumination area of ​​the product by a bar light source, and the first image is imaged in a mask structure step. The method further includes: The imaging light reflected by the product after being illuminated by the strip light source is collected at a first frequency through a first lens, the light is a line on the product, and the light is used to form an image on the mask structure, the first frequency is a fixed frequency, and the first frequency is determined by the transmission speed of the product and the hardware conditions of the first lens; The collected light is propagated to the mask structure, imaged on the mask structure, and the first image is obtained. The light is continuously acquired at a first frequency and transmitted to the mask structure to form an image on the mask structure.

6. A defect detection method based on snapshot compression imaging as claimed in claim 1, characterized in that: During the product conveying process, a first image of the product is continuously acquired at a first frequency, the first image includes an illumination area of ​​the product by a bar light source, and the first image is imaged in a mask structure step. The method further includes: The strip light source is arranged on one side of a conveyor belt conveying the product, and the area illuminated by the strip light source on the conveyor belt is a first area, the first area is fixedly arranged, and the third image is used to determine defects of the product corresponding to the first area.

7. A defect detection device based on snapshot compression imaging, characterized in that: The method applied to any one of claims 1 to 6, wherein the device comprises: A strip light source, which is arranged on one side of a conveyor belt for conveying products, and is arranged toward the conveyor belt, and is used to provide lighting for products passing on the conveyor belt; A first lens, which is disposed on a side of the conveyor belt where the strip light source is disposed, and is used to collect reflected light from an area on the product illuminated by the strip light source, and to image the reflected light on a mask structure to obtain a first image; A mask structure, wherein the mask structure is arranged in the imaging direction of the first lens, a plurality of modulation modes are arranged on the mask structure, and the mask structure is connected to a rotating motor, the rotating motor is used to drive the mask structure to rotate at a constant speed, and the mask structure is used to perform intensity modulation on a plurality of first images imaged on the mask structure through the imaging lens; A line array camera is used to receive the modulated image on the mask structure collected by the second lens, and compress a plurality of modulated images to obtain a second image.

Images