Multi-view machine vision system based on two-dimensional grating and use method

Abstract

The invention discloses a multi-view machine vision system based on a two-dimensional grating and a use method, and the system comprises an optical imaging module which is used for converging light field information of a target; the two-dimensional grating is used for enabling the converged light field to form a multi-stage diffraction image on an image plane; the plurality of image acquisition modules are used for recording the intensities of different diffraction order light fields of the two-dimensional grating to obtain a plurality of images with parallax information; and the image analysis and processing module is used for processing the acquired image to obtain parallax information of the target. According to the multi-view machine vision system, multi-view vision information with parallax can be obtained only through one imaging system, angle information of a target is obtained while the spatial resolution of the system is not reduced, the problem that the spatial resolution and the angle resolution of a traditional multi-view vision system are mutually restricted is solved, the two-dimensional grating machining technology is simple, and the machining cost is low. The system structure is simplified, the reliability is improved, and the cost is reduced.

Description

Technical Field

[0001] This invention belongs to the fields of optical system imaging, optical measurement and computer, and specifically relates to a multi-view machine vision system based on a two-dimensional grating and its usage method. Background Technology

[0002] Multi-view machine vision systems are computer vision technologies that utilize multiple cameras working collaboratively to infer distance and depth information by observing the positional changes (parallax) of the same object in an image from different viewpoints. Currently, the technological bottleneck restricting the application of multi-view machine vision systems is the mutual constraint between spatial resolution and angular resolution. The single aperture of a multi-view machine vision system determines the spatial resolution of light field detection, while the number of apertures determines the angular resolution. According to Shannon's sampling theorem, for an imaging system—a signal system constrained in both spatial and frequency domains—the spatial bandwidth product is proportional to the product of its field of view area and its spatial frequency domain area. Optical systems with a large spatial bandwidth product possess higher degrees of freedom and greater data throughput. For multi-view machine vision systems, a large spatial bandwidth product means a greater number of apertures and larger aperture sizes, leading to a significant increase in system weight and cost. When the spatial bandwidth product is constant, increasing the number of apertures inevitably leads to a decrease in aperture size. This mutual constraint between spatial resolution and angular resolution limits the application of multi-view machine vision systems in complex scenarios such as long-distance, small-target situations.

[0003] Currently, scholars are improving multi-view machine vision systems through various methods. For example, coherent synthesis technology is incorporated into multi-view machine vision systems, which improves the spatial resolution of images without affecting angular resolution. However, the application of this method is limited by the complexity, difficulty, and high cost of coherent synthesis technology. Another method involves differentiated design of the imaging system to acquire a high spatial resolution image and multiple low-resolution parallax images. Then, using this high spatial resolution image as a reference, a computer fusion algorithm is used to improve the resolution of the low-resolution parallax images, achieving super-resolution imaging. However, this high-resolution image is obtained from a large aperture. The higher the resolution, the larger the aperture. With a fixed total receiving surface, this reduces the number of camera arrays, leading to a decrease in angular resolution. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a multi-view machine vision system and its usage method based on a two-dimensional grating. The two-dimensional grating forms multiple images on the image plane from the received light field, and the resolution of each image is consistent with the resolution of a single image without a grating, thus solving the problem of mutual constraint between spatial resolution and angular resolution in multi-view machine vision systems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a multi-view machine vision system based on a two-dimensional grating, the system comprising: an optical imaging module, a two-dimensional grating, multiple image acquisition modules, and an image analysis and processing module;

[0007] The optical imaging module includes a light source and a main lens; the light source is used to emit light towards the target; the main lens is used to converge the light field information of the target;

[0008] The two-dimensional grating is placed close to the main lens or etched on the main lens; the two-dimensional grating is used to form a multi-level diffraction image on the image plane by the light field converged by the main lens.

[0009] The multiple image acquisition modules are used to synchronously acquire the multi-level diffraction images;

[0010] The image analysis and processing module is used to process and extract data from the multi-level diffraction images acquired by the image acquisition module to obtain a high-resolution target image and the disparity information of the target.

[0011] Secondly, the present invention provides a method for using a multi-view machine vision system based on a two-dimensional grating, applied to the aforementioned multi-view machine vision system, the method comprising:

[0012] The light source in the optical imaging module emits light towards the target, and the main lens in the optical imaging module converges the light field information of the target.

[0013] A two-dimensional grating is used to converge the light field of the main lens to form a multi-level diffraction image on the image plane;

[0014] The multi-level diffraction images are acquired synchronously using multiple image acquisition modules;

[0015] The image analysis and processing module is used to process and extract data from the multi-level diffraction images acquired by the image acquisition module to obtain a high-resolution target image and the disparity information of the target.

[0016] The beneficial effects of this invention are as follows:

[0017] (1) Compared with traditional multi-view machine vision systems, the present invention solves the problem of mutual constraint between spatial resolution and angular resolution.

[0018] (2) Compared with multi-view machine vision systems that use coherent synthesis technology, this invention not only does not increase the complexity and technical difficulty of the system, but also simplifies the structure of the system and reduces the system cost.

[0019] (3) The present invention uses diffraction images as images of each sub-aperture, which makes the relative positions between images more stable and avoids problems such as parameter changes caused by system movement and the need for secondary calibration.

[0020] (4) The present invention does not require sub-apertures, and the parallax image is a diffraction image of a two-dimensional grating, thus avoiding the influence of aberrations and adjustment between sub-apertures on the parallax image. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of the multi-view machine vision system based on a two-dimensional grating according to the present invention; wherein, the reference numerals are: 1-optical imaging module; 2-two-dimensional grating; 3-image acquisition module; 4-image analysis and processing module.

[0022] Figure 2 This is a structural diagram of a multi-view machine vision system based on a two-dimensional grating according to an embodiment of the present invention; wherein, the reference numerals are: a-resolution plate; b-main lens; c-two-dimensional grating; d-photodetector; e-computer.

[0023] Figure 3 This is a schematic diagram of two-dimensional grating diffraction according to the present invention.

[0024] Figure 4 This is an experimental diagram of the point spread function of the multi-view machine vision system based on a two-dimensional grating according to the present invention.

[0025] Figure 5 The figure shows the experimental results of the multi-view machine vision system based on two-dimensional gratings of the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0027] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0028] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0029] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0030] Furthermore, to better illustrate the present invention, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that the present invention can be practiced without certain specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order to highlight the spirit of the invention.

[0031] The key challenge currently hindering the performance improvement of multi-view machine vision systems is the mutual constraint between spatial resolution and angular resolution. In recent years, with the rapid development and application of aperture coding and deep learning technologies, spatial and angular resolutions have been improved to some extent, given a fixed spatial bandwidth product. While theoretically, coherent combining technology could completely solve the problem of this mutual constraint, coherent combining is complex and difficult to apply to multi-view machine vision systems. With the widespread application of micro-nano technology, two-dimensional grating fabrication technology has become very mature, achieving precision down to tens of nanometers, and is relatively inexpensive. Multi-view machine vision systems based on two-dimensional gratings can simultaneously acquire multiple parallax images with the same resolution as those without gratings on the image plane. Therefore, this invention proposes a multi-view machine vision system based on two-dimensional gratings.

[0032] Figure 1 A schematic diagram of the structure of the multi-view machine vision system based on a two-dimensional grating according to the present invention is shown, as follows: Figure 1 As shown, the system includes an optical imaging module 1, a two-dimensional grating 2, multiple image acquisition modules 3, and an image analysis and processing module 4.

[0033] The optical imaging module 1 includes a light source and a main lens; the light source is used to emit light towards the target; the main lens is used to converge the light field information of the target.

[0034] For example, a complete optical imaging module 1 can be designed according to actual application requirements, and core parameters such as object distance, focal length, field of view size, and camera target size can be optimized and matched to make its imaging resolution meet the requirements of actual application.

[0035] The two-dimensional grating 2 is placed close to the main lens or etched on the main lens; the two-dimensional grating 2 is used to form a multi-level diffraction image on the image plane by focusing the light field of the main lens.

[0036] Among them, the grid spacing and grating constant of the two-dimensional grating 2 determine the spacing between diffraction images. Diffraction images of the same order have the same brightness, while diffraction images of different orders have different brightness; the central diffraction image has the highest brightness.

[0037] For example, the material and processing dimensions of the two-dimensional grating 2 can be selected by comparing the baseline length requirements and channel number requirements of a multi-view machine vision system under the same conditions, based on actual application needs and scenarios, thus determining the type and grating constant of the two-dimensional grating 2. The grating type and grating constant are matched with the optical imaging module 1 and the multiple image acquisition modules 3.

[0038] For example, the types of two-dimensional grating 2 include planar two-dimensional gratings and curved two-dimensional gratings.

[0039] For example, MATLAB can be used to simulate the optical imaging module 1 after adding the two-dimensional grating 2 to determine the tolerance of processing errors.

[0040] For example, the two-dimensional grating 2 can be etched onto a separate material or directly onto the main lens.

[0041] Multiple image acquisition modules 3 are used to synchronously acquire multi-level diffraction images formed by the image plane.

[0042] Among them, the parallax image acquired by the multi-view machine vision system should be the same diffraction order image of the two-dimensional grating 2.

[0043] For example, multiple photodetectors can be used to simultaneously acquire multi-level diffraction images, with each image acquisition module 3 corresponding to one photodetector; or, a large photodetector can be used to achieve simultaneous acquisition of multi-level diffraction images through region segmentation, with each image acquisition module 3 corresponding to one region of the photodetector.

[0044] As an example, photodetectors can be implemented using charge-coupled device (CCD) arrays.

[0045] For example, the photosensitive area, device size, and number of devices of each image acquisition module 3 are matched with the two-dimensional grating 2.

[0046] The image analysis and processing module 4 is used to process and extract data from the multi-level diffraction images acquired by the image acquisition module 3 to obtain high-resolution target images and parallax information of the target.

[0047] For example, the image analysis and processing module 4 can be implemented by a computer.

[0048] For example, the image analysis and processing module 4 performs filtering processing on the multi-level diffraction images acquired by the multiple image acquisition modules 3, and obtains high-resolution target images and parallax information of the target through methods such as image fusion.

[0049] For example, in the case of insufficient light, the image analysis and processing module 4 uses the central diffraction image acquired by the image acquisition module 3 as a reference image and uses computer algorithms to compensate for the reduced image resolution caused by insufficient diffraction level brightness.

[0050] For example, the light source emits monochromatic light towards the target; when the light emitted by the light source is non-monochromatic or natural light, a narrowband filter is added in front of the main lens or image acquisition module 3 to achieve the effect of monochromatic light.

[0051] This invention relates to a multi-view machine vision system based on a two-dimensional grating. By using a two-dimensional grating, the light field acquired by a single optical imaging system is projected onto the image plane to form multiple images of the same resolution with parallax information (i.e., multi-level diffraction images). Through image analysis and processing, high-resolution target images and target parallax information can be obtained, improving the spatial resolution of the multi-view machine vision. This multi-view machine vision system requires only one optical imaging system to obtain multi-view visual information with parallax, acquiring the target's angular information without reducing the system's spatial resolution, thus solving the problem of the mutual constraint between spatial and angular resolution in traditional multi-view machine vision systems. Furthermore, this multi-view machine vision system can perform the tasks of multiple imaging systems with a single optical imaging system. The two-dimensional grating fabrication process is simple, simplifying the experimental system, improving reliability, and reducing costs.

[0052] The multi-view machine vision system based on two-dimensional gratings of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0053] Figure 2 This diagram illustrates the structure of a multi-view machine vision system based on a two-dimensional grating according to an embodiment of the present invention. Figure 2 As shown, the system includes a resolution plate a (corresponding to the target in a specific application), a main lens b, a two-dimensional grating c, a photodetector d, and a computer e. The photodetector d can be a CCD array.

[0054] The target is illuminated with monochromatic light. If the light is non-monochromatic or natural, a narrow-band filter is added to the front of the main lens b or photodetector d to achieve monochromatic light. The main lens b receives and converges the monochromatic light scattered from the target, and generates multiple images of different orders, with the same resolution but different brightness, on the image plane through diffraction by the two-dimensional grating c. Photodetectors d are placed at different positions on the image plane to collect the intensity information of these diffracted images. The specific number and placement of photodetectors are determined according to actual needs. The computer e processes and analyzes the images collected by the photodetector d to obtain the required detection information (including high-resolution target images and target parallax information).

[0055] Figure 3A schematic diagram illustrating the diffraction process of light by the two-dimensional grating of the present invention is shown, as follows: Figure 3 As shown, monochromatic parallel light, after being modulated by a converging lens (i.e., the primary lens) and a two-dimensional grating, will produce multiple diffraction spots. The intensity of the diffraction spots on the image plane, i.e., the distribution of the point spread function, is given by formula (1):

[0056] (1)

[0057] in, This represents the distribution of the point spread function; , These represent the length and width of the two-dimensional grating, respectively. , These represent the number of grids in the length and width of the two-dimensional grating, respectively. These are sinusoidal basis functions; For frequency domain coordinates, , , and Let these represent the coordinates of any point on the object plane and the corresponding coordinates on the image plane, respectively. It is the magnification of the image. and These represent the object distance and image distance, respectively. Indicates wave number, , Indicates wavelength.

[0058] It should be noted that, for the sake of convenience, Figure 3 Only the diffraction order along the lateral direction is marked (i.e. Figure 3 The center point, positive and negative first order, positive and negative second order, positive and negative third order (the vertical and intersecting diffraction orders are not labeled).

[0059] Figure 4 The simulation results of the point spread function theory are shown, where, Figure 4 In this context, (a) represents a point on the object plane, (b) represents the imaging system of a two-dimensional grating, (c) represents the multi-point spread function on the image plane, and (d) represents the width of the point spread function and the distance between the point spread functions. The theory is described by the above formula (1), where any two symmetrical images can be used as parallax images. For a multi-view machine vision system, the image plane coordinates of any point on the object plane in the binocular vision's left and right cameras can be expressed as:

[0060] (2)

[0061] in, and It is on the surface of the object. The coordinates are the coordinates of the two image points corresponding to the left and right camera image planes; It is the distance between the left and right cameras, i.e., the baseline length; This represents the focal length of the lens. The coordinates of any point on the object surface can be derived from formula (1). The coordinates of any two symmetrical points on the image plane satisfy the formula:

[0062] (3)

[0063] in, The image order of diffraction; The width of the grating slits. Indicates the grating width. Indicates the number of grid cells; This represents the wavelength. Therefore, the equivalent baseline length is:

[0064] (4)

[0065] Because current grating fabrication techniques enable grating markings to reach several thousand lines per millimeter, that is... and The equivalent baseline length of this multi-view machine vision system based on diffraction gratings can reach the scale of several focal lengths.

[0066] Figure 5 The experimental results of the multi-view machine vision system based on two-dimensional gratings of the present invention are shown in the figure. The two-dimensional grating structure in the experiment is as follows. Figure 3 As shown in the two-dimensional grating, the grating grid is evenly spaced with a spacing of 20 micrometers. The diameter of the main lens is 2.4 centimeters, the focal length is 30 centimeters, and the illumination light is a 650-nanometer laser source. Among these, Figure 5 In the image, (a) is the image acquired without a grating; (b) is the central diffraction image acquired with a grating; (c) is the transverse positive first-order diffraction image acquired with a grating; (d) is the transverse negative first-order diffraction image acquired with a grating; (e) is the longitudinal positive first-order diffraction image acquired with a grating; and (f) is the longitudinal negative first-order diffraction image acquired with a grating. The parallax information of the target can be extracted from the acquired diffraction images.

[0067] This invention also proposes a method for using a multi-view machine vision system based on a two-dimensional grating, applied to the multi-view machine vision system based on a two-dimensional grating of this invention, the method comprising:

[0068] The light source in the optical imaging module emits light towards the target, and the main lens in the optical imaging module converges the light field information of the target.

[0069] A two-dimensional grating is used to converge the light field of the main lens to form a multi-level diffraction image on the image plane;

[0070] The multi-level diffraction images are acquired synchronously using multiple image acquisition modules;

[0071] The image analysis and processing module is used to process and extract data from the multi-level diffraction images acquired by the image acquisition module to obtain a high-resolution target image and the disparity information of the target.

[0072] The specific implementation steps of the method of the present invention can be referred to the description of the above system embodiments, and will not be repeated here.

[0073] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0074] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. The above descriptions are exemplary and not exhaustive. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-view machine vision system based on two-dimensional gratings, characterized in that, The system comprises an optical imaging module, a two-dimensional grating, a plurality of image acquisition modules, and an image analysis and processing module; The optical imaging module comprises a light source and a main lens; the light source is used to emit light to a target; and the main lens is used to converge light field information of the target; The two-dimensional grating is placed close to the main lens or is etched on the main lens; and the two-dimensional grating is used to form a plurality of levels of diffraction images on an image plane from the light field converged by the main lens; The plurality of image acquisition modules are used to synchronously acquire the plurality of levels of diffraction images; The image analysis and processing module is used to process and extract data from the plurality of levels of diffraction images acquired by the image acquisition modules, so as to obtain a high-resolution target image and parallax information of the target.

2. The system of claim 1, wherein, The optical imaging module is designed according to actual requirements, and core parameters of the optical imaging module are optimized and matched, so that the imaging resolution of the optical imaging module reaches the actual application requirements; the core parameters include object distance, focal length, field of view size, and camera target surface size.

3. The system of claim 1, wherein, Parameters of the two-dimensional grating are determined according to baseline length requirements and channel number requirements of the multi-view machine vision system; the parameters include material, processing size, type, and grating constant.

4. The system of claim 3, wherein, After the parameters of the two-dimensional grating are determined, the optical imaging module with the two-dimensional grating is simulated by using matlab, so as to determine the tolerance of processing error.

5. The system of claim 1, wherein, The plurality of image acquisition modules are used to synchronously acquire the plurality of levels of diffraction images by using a plurality of photoelectric detectors; each image acquisition module corresponds to one photoelectric detector. Alternatively, one photoelectric detector is used to synchronously acquire the plurality of levels of diffraction images by using region segmentation; each image acquisition module corresponds to one region of the photoelectric detector.

6. The system of claim 1, wherein, The photosensitive area, device size, and device number of each image acquisition module are matched with the two-dimensional grating.

7. The system of claim 1, wherein, The image analysis and processing module is used to filter the plurality of levels of diffraction images acquired by the image acquisition modules, and the high-resolution target image and the parallax information are obtained by using an image fusion method.

8. The system of claim 7, wherein, In the case of insufficient light, the image analysis and processing module uses the central diffraction image acquired by the image acquisition module as a reference image, and uses a computer algorithm to compensate for the problem of reduced image resolution caused by insufficient diffraction level brightness.

9. A method of using a multi-view machine vision system based on two-dimensional gratings, applied to the system of any of claims 1-8, characterized in that, The method comprises: a light source in an optical imaging module is used to emit light to a target, and a main lens in the optical imaging module is used to converge light field information of the target; a two-dimensional grating is used to form a plurality of levels of diffraction images on an image plane from the light field converged by the main lens; a plurality of image acquisition modules are used to synchronously acquire the plurality of levels of diffraction images; an image analysis and processing module is used to process and extract data from the plurality of levels of diffraction images acquired by the image acquisition modules, so as to obtain a high-resolution target image and parallax information of the target.

Images