Meta-imaging based industrial inspection system, method, electronic device, and storage medium

Abstract

The application discloses an industrial detection system and method based on meta-imaging, an electronic device and a storage medium. The system comprises a collection module configured to collect four-dimensional scanning light field information of an industrial product; a reconstruction module configured to reconstruct the four-dimensional scanning light field information into a three-dimensional structure of the industrial product by using a meta-imaging method; and a detection module configured to detect defects in the three-dimensional structure of the industrial product to obtain a defect detection result of the industrial product. Thus, the three-dimensional high-resolution light field information of the industrial product can be quickly obtained, and the three-dimensional fine structure of the industrial product can be accurately reconstructed for defect detection, so that the three-dimensional defect detection of the surface of the industrial product can be realized quickly, accurately and in real time. Therefore, the problems of slow speed, low accuracy and high cost of the existing industrial detection technology are solved.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to an industrial inspection system, method, electronic device and storage medium based on meta-imaging. Background Technology

[0002] In industrial production, limitations and deficiencies in existing technology and working conditions can easily affect the quality of finished products. Therefore, to ensure a high pass rate and reliable quality, surface defect detection is essential. Surface defects are the most direct manifestation of compromised product quality.

[0003] "Defect" can generally be understood as a missing part, defect, or area compared to a normal sample. Surface defect detection refers to detecting scratches, defects, foreign object obstruction, color contamination, holes, and other defects on the surface of a sample, thereby obtaining a series of relevant information such as the type, outline, location, and size of the defects on the surface of the sample being tested.

[0004] In related technologies, manual defect detection is generally used, but this method is inefficient; the detection results are easily affected by subjective human factors and cannot meet the requirements of real-time detection. It has been gradually replaced by other methods. Existing camera-based industrial inspection methods generally use structured light imaging or zoom scanning. Both require active light source illumination. The difference is that the former acquires multiple images and reconstructs them using computational photography methods, while the latter is a zoom scanning method with a slower detection speed, making it difficult to meet the real-time requirements of industrial production processes, and thus urgently needs to be addressed. Summary of the Invention

[0005] This application provides an industrial inspection system, method, electronic device, and storage medium based on meta-imaging to solve the problems of slow speed, low accuracy, and high cost of existing industrial inspection technologies.

[0006] The first aspect of this application provides an industrial inspection system based on meta-imaging, comprising: an acquisition module for acquiring four-dimensional scanning light field information of an industrial product; a reconstruction module for reconstructing the three-dimensional structure of the industrial product using a meta-imaging method; and a detection module for detecting defects in the three-dimensional structure of the industrial product and obtaining defect detection results of the industrial product.

[0007] Optionally, in one embodiment of this application, the acquisition module includes: an optical lens for acquiring the light signal of the industrial product; a microlens array for imaging the light signal; a scanning element for controlling the scanning action of the microlens array; and a sensor for receiving the imaging result of the microlens array to obtain the four-dimensional scanning light field information of the industrial product.

[0008] Optionally, in one embodiment of this application, it further includes: a correction module for globally correcting optical aberrations in the four-dimensional scanning light field information using geometric optics.

[0009] Optionally, in one embodiment of this application, detecting defects in the three-dimensional structure of the industrial product includes: detecting the defect category, defect outline, defect location, and defect size of the three-dimensional structure of the industrial product.

[0010] A second aspect of this application provides an industrial inspection method based on meta-imaging, comprising the following steps: acquiring four-dimensional scanning light field information of an industrial product; using a meta-imaging method to reconstruct the three-dimensional structure of the industrial product from the four-dimensional scanning light field information; detecting defects in the three-dimensional structure of the industrial product, and obtaining defect detection results of the industrial product.

[0011] Optionally, in one embodiment of this application, the acquisition of four-dimensional scanning light field information of industrial products includes: acquiring four-dimensional scanning light field information of industrial products using a meta-imaging system including an optical lens, a microlens array, and a sensor.

[0012] Optionally, in one embodiment of this application, after acquiring the four-dimensional scanning light field information of the industrial product, the method further includes: globally correcting the optical aberrations in the four-dimensional scanning light field information using geometric optics.

[0013] Optionally, in one embodiment of this application, detecting defects in the three-dimensional structure of the industrial product includes: detecting the defect category, defect outline, defect location, and defect size of the three-dimensional structure of the industrial product.

[0014] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the meta-imaging-based industrial inspection method as described in the above embodiments.

[0015] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to perform the meta-imaging-based industrial inspection method as described in the above embodiments.

[0016] Therefore, the embodiments of this application have the following beneficial effects:

[0017] The embodiments of this application acquire four-dimensional scanning light field information of industrial products, and use meta-imaging methods to reconstruct the three-dimensional structure of the industrial products from the four-dimensional scanning light field information to obtain the three-dimensional structure of the industrial products. Defects in the three-dimensional structure of the industrial products are then detected, and defect detection results are obtained. This allows for the rapid acquisition of high-resolution three-dimensional light field information of the industrial products and the accurate reconstruction of their fine three-dimensional structure for defect detection, achieving rapid, accurate, and real-time three-dimensional defect detection on the surface of industrial products. Therefore, it solves the technical problems of existing industrial inspection technologies, such as slow speed, low accuracy, and high cost.

[0018] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0020] Figure 1 This is a block diagram illustrating an industrial inspection system based on meta-imaging according to an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of a meta-imaging structure for industrial inspection provided according to an embodiment of this application;

[0022] Figure 3 This is a flowchart of an industrial inspection method based on meta-imaging provided according to an embodiment of this application;

[0023] Figure 4 A schematic diagram of the structure of the electronic device provided in the application embodiment.

[0024] Explanation of reference numerals in the attached diagram: Acquisition module-100, Reconstruction module-200, Detection module-300, Memory-401, Processor-402, Communication interface-403. Detailed Implementation

[0025] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0026] The following description, with reference to the accompanying drawings, outlines an industrial inspection method, apparatus, electronic device, and storage medium based on meta-imaging according to embodiments of this application. Addressing the problems mentioned in the background section, this application provides an industrial inspection system based on meta-imaging. First, an optical lens images the industrial product onto a microlens array. Then, the microlens array performs spatial scanning sampling of the image of the industrial product, which is subsequently received by a sensor. Finally, the acquired high-dimensional, high-resolution light field information is calculated and reconstructed to obtain the three-dimensional fine structure of the industrial product, completing the industrial inspection process. Embodiments of this application offer fast detection speed, high accuracy, and real-time industrial inspection capabilities. This solves the problems of slow speed, low accuracy, and high cost associated with existing industrial inspection technologies.

[0027] Specifically, Figure 1 This is a block diagram of an industrial inspection system based on meta-imaging, according to an embodiment of this application.

[0028] like Figure 1 As shown, the meta-imaging-based industrial inspection system 10 includes: an acquisition module 100, a reconstruction module 200, and a detection module 300.

[0029] The acquisition module 100 is used to acquire four-dimensional scanning light field information of the industrial product. The reconstruction module 200 is used to reconstruct the three-dimensional structure of the industrial product from the four-dimensional scanning light field information using a meta-imaging method. The detection module 300 is used to detect defects in the three-dimensional structure of the industrial product and obtain the defect detection results.

[0030] It should be noted that in the industrial production process, acquiring three-dimensional images of products facilitates comprehensive, multi-angle defect detection, thereby ensuring product quality. In the embodiments of this application, a meta-imaging method can be used to quickly obtain high-resolution three-dimensional light field information of industrial products through industrial inspection equipment based on a light field camera. The specific acquisition process is described in detail below. Furthermore, those skilled in the art can also choose appropriate methods based on the actual situation, such as using binocular stereo vision or laser triangulation to acquire three-dimensional image information of the product.

[0031] Optionally, in one embodiment of this application, the acquisition module 100 includes: an optical lens for acquiring light signals from an industrial product; a microlens array for imaging the light signals; a scanning element for controlling the scanning action of the microlens array; and a sensor for receiving the imaging results of the microlens array to obtain four-dimensional scanning light field information of the industrial product.

[0032] Specifically, the industrial testing equipment described in the embodiments of this application comprises an optical lens, a microlens array, a scanning element, and an image sensor, such as... Figure 2As shown. The optical lens (group) is used for primary recording of the light signal of the target scene; the microlens array is a two-dimensional array composed of multiple microlens units, used for secondary optical encoding of the optical signal recorded by the optical lens (group); the scanning platform is used to control the lens array to perform two-dimensional spatial scanning, thereby achieving high-precision resolution of the target scene information; the image sensor is used to receive high-dimensional, high-resolution light field information of industrial products. The pupil surface of the lens and the photosensitive surface of the image sensor are conjugate with respect to the microlens array; in other words, the lens projects onto the image sensor through each microlens unit to form a small microlens sub-image.

[0033] Optionally, in one embodiment of this application, the meta-imaging-based industrial inspection system 10 further includes: a correction module for globally correcting optical aberrations in the four-dimensional scanning light field information using geometric optics.

[0034] It is understandable that although the above process can obtain high-dimensional and high-resolution light field information of industrial products, due to the inconsistency between the results obtained by non-paraxial ray tracing and paraxial ray tracing during the imaging process of the light field camera, optical aberrations such as field curvature, astigmatism, and distortion are inevitable.

[0035] Therefore, in the embodiments of this application, in order to accurately estimate the system's optical aberrations, a pupil segmentation method can be used. First, wavefront estimation and correction processes are combined in a multi-dimensional decoupling approach, and the acquired data is processed by algorithms such as registration. Second, digital phase modulation is achieved by translating the light propagation trajectories of different phase space frequency components. Finally, the Wigner data corresponding to different phase space frequencies are calibrated, and the aberration distribution is simulated using the Zernike function to reconstruct the pupil function of the imaging system, thereby correcting the error influence of system optical aberrations on the acquisition results.

[0036] In actual implementation, technicians can also perform global correction of optical aberrations in the four-dimensional scanning light field information by appropriately changing the lens combination in the optical lens group, reducing the field of view, or using distortion correction algorithms, without making specific restrictions.

[0037] After obtaining the high-dimensional, high-resolution optical field information of the corrected industrial product, further, in the embodiments of this application, the high-dimensional, high-resolution spatial information can be used as an invariant spatial constraint, and different spatial point spread functions can be used to update and iterate different phase space frequency components. The four-dimensional phase space point spread function is expressed as follows:

[0038]

[0039] According to Passevar's theorem:

[0040]

[0041] It should be noted that during the iteration of the spatial deconvolution model, the high-resolution spatial frequency components are updated step by step through backpropagation to update the scene information. The weights updated each time correspond to the total energy of the spread function of the corresponding point in the experimental setup. Based on the traditional Richard-Lucy deconvolution architecture, information from different spatial frequencies is used to simultaneously model and acquire images. Finally, the different spatial frequency distributions are digitally adaptively fused to achieve four-dimensional scanning light field information reconstruction, thereby obtaining the three-dimensional fine structure of industrial products.

[0042] Optionally, in one embodiment of this application, detecting defects in the three-dimensional structure of an industrial product includes: detecting the defect category, defect outline, defect location, and defect size of the three-dimensional structure of the industrial product.

[0043] After obtaining the detailed three-dimensional structure of an industrial product, those skilled in the art can inspect the product from three aspects: texture features, color features, and shape features, depending on the intended use of the surface features. Specific inspection methods include, but are not limited to, traditional machine vision surface defect detection methods and deep learning-based methods. Deep learning methods include supervised, unsupervised, and weakly supervised methods. In practice, technicians can select the optimal inspection method based on different product types, without specific limitations.

[0044] The industrial inspection system based on meta-imaging proposed in this application collects four-dimensional scanning light field information of industrial products, uses meta-imaging to reconstruct the three-dimensional structure of the industrial products, and then detects defects in the three-dimensional structure of the industrial products to obtain defect detection results. This allows for the rapid acquisition of three-dimensional high-resolution light field information of industrial products and accurate reconstruction of the three-dimensional fine structure of industrial products for defect detection. It enables rapid, accurate, and real-time three-dimensional defect detection of industrial product surfaces, which is expected to promote the development of the industrial inspection field and has significant economic and social benefits.

[0045] Secondly, the industrial inspection method based on meta-imaging proposed according to the embodiments of this application is described with reference to the accompanying drawings.

[0046] Figure 3 This is a flowchart illustrating an industrial inspection method based on meta-imaging, provided as an embodiment of this application.

[0047] like Figure 3 As shown, this meta-imaging-based industrial inspection method includes the following steps:

[0048] In step S101, the four-dimensional scanning light field information of the industrial product is acquired.

[0049] Optionally, in one embodiment of this application, acquiring four-dimensional scanning light field information of industrial products includes: acquiring four-dimensional scanning light field information of industrial products using a meta-imaging system including an optical lens, a microlens array, and a sensor.

[0050] Optionally, in one embodiment of this application, after acquiring the four-dimensional scanning light field information of the industrial product, the method further includes: using geometric optics to globally correct optical aberrations in the four-dimensional scanning light field information.

[0051] In step S102, the three-dimensional structure of the industrial product is obtained by reconstructing the four-dimensional scanning light field information using the meta-imaging method.

[0052] In step S103, defects in the three-dimensional structure of the industrial product are detected to obtain the defect detection results of the industrial product.

[0053] Optionally, in one embodiment of this application, detecting defects in the three-dimensional structure of an industrial product includes: detecting the defect category, defect outline, defect location, and defect size of the three-dimensional structure of the industrial product.

[0054] It should be noted that the foregoing explanation of the embodiment of the meta-imaging-based industrial inspection system also applies to the meta-imaging-based industrial inspection method of this embodiment, and will not be repeated here.

[0055] The industrial inspection method based on meta-imaging proposed in this application collects four-dimensional scanning light field information of industrial products, and uses meta-imaging to reconstruct the three-dimensional structure of the industrial products by image reconstruction. Then, it detects defects in the three-dimensional structure of the industrial products and obtains the defect detection results. This method can quickly obtain three-dimensional high-resolution light field information of industrial products and accurately reconstruct the three-dimensional fine structure of industrial products for defect detection. It can achieve rapid, accurate and real-time three-dimensional defect detection on the surface of industrial products, which is expected to promote the development of the industrial inspection field and has great economic and social benefits.

[0056] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0057] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0058] When the processor 402 executes the program, it implements the industrial inspection method based on meta-imaging provided in the above embodiments.

[0059] Furthermore, electronic devices also include:

[0060] Communication interface 403 is used for communication between memory 401 and processor 402.

[0061] The memory 401 is used to store computer programs that can run on the processor 402.

[0062] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0063] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0064] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0065] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0066] This embodiment also provides a computer-readable storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the above-described meta-imaging-based industrial inspection method.

[0067] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0068] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0069] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0070] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0071] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A meta-imaging based industrial inspection system, characterized in that, The method comprises the following steps: acquiring four-dimensional scanning light field information of an industrial product; The acquisition module comprises: an optical lens for acquiring light signals of the industrial product; a microlens array for imaging the light signals; a scanning device for controlling the microlens array to perform spatial two-dimensional scanning; a sensor for receiving imaging results of the microlens array to obtain the four-dimensional scanning light field information of the industrial product; a correction module for correcting optical aberrations in the four-dimensional scanning light field information by using geometric optics, wherein the correction module performs registration algorithm processing on the acquired data by combining wavefront estimation and correction processes in a multi-dimensional decoupling mode through a pupil segmentation method, realizes digital phase modulation by shifting light propagation trajectories of different phase space frequency components, calibrates Wigner data corresponding to different phase space frequency components, simulates the distribution of aberrations by using Zernike functions, reconstructs a pupil function of an imaging system, and further corrects the error influence of system optical aberrations on the acquisition results; a reconstruction module for reconstructing an image of the four-dimensional scanning light field information by using a meta-imaging method to obtain a three-dimensional structure of the industrial product, wherein the reconstruction module uses different spatial point spread functions to update different phase space frequency components by taking high-dimensional high-resolution spatial information as an invariant spatial constraint, and simultaneously models the collected image by using information of different spatial frequencies on the basis of a traditional Richard-Lucy deconvolution architecture, and digitally adaptively fuses different spatial frequency distributions, so as to realize four-dimensional scanning light field information reconstruction and obtain a three-dimensional fine structure of the industrial product; a detection module for detecting defects in the three-dimensional structure of the industrial product to obtain a defect detection result of the industrial product.

2. The system of claim 1, wherein, The detection of defects in the three-dimensional structure of the industrial product comprises: detecting a defect category, a defect contour, a defect position, and a defect size of the three-dimensional structure of the industrial product.

3. A method of meta-imaging based industrial inspection, the system of any of claims 1-2 being used to perform the method, characterized by, The method comprises the following steps: acquiring four-dimensional scanning light field information of an industrial product; reconstructing an image of the four-dimensional scanning light field information by using a meta-imaging method to obtain a three-dimensional structure of the industrial product; detecting defects in the three-dimensional structure of the industrial product to obtain a defect detection result of the industrial product.

4. The method of claim 3, wherein, The acquisition of the four-dimensional scanning light field information of the industrial product comprises: acquiring the four-dimensional scanning light field information of the industrial product by using a meta-imaging system comprising an optical lens, a microlens array, and a sensor.

5. The method of claim 3, wherein, After the acquisition of the four-dimensional scanning light field information of the industrial product, the method further comprises: correcting optical aberrations in the four-dimensional scanning light field information by using geometric optics.

6. The method of claim 3, wherein, The detection of defects in the three-dimensional structure of the industrial product comprises: detecting a defect category, a defect contour, a defect position, and a defect size of the three-dimensional structure of the industrial product.

7. An electronic device, comprising: The method comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the meta-imaging-based industrial detection method according to any one of claims 3-6.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor for implementing the meta-imaging based industrial detection method as claimed in any one of claims 3-6.

Images