Space point positioning method and system based on lens-free imaging system

Abstract

The invention provides a spatial point positioning method and system based on a lensless imaging system, and the method comprises the steps: generating a mask pattern with a position sensing characteristic: enabling each sub-region of the pattern to uniquely correspond to the displacement direction and amplitude of a light source through optimizing the shape and arrangement mode of the mask pattern, the position sensing characteristic realizes decoupling of displacement direction and amplitude through unique mapping of a local pattern structure; calculating the sub-pixel displacement of the light spot image: based on the modulation result of the mask pattern on the light source, collecting two adjacent frames of light spot images, analyzing a cross-power spectrum through a phase correlation method, and fitting the sub-pixel displacement; mapping the physical position change of the light source: converting the sub-pixel displacement into coordinate change of the light source in a three-dimensional space according to calibration parameters and a geometric model; wherein the geometric model comprises compensation calculation of the thickness and the refractive index of the mask plate.

Description

Technical Field

[0001] The present invention belongs to the technical fields of lensless imaging, spatial point positioning, etc., and particularly relates to a spatial point positioning method and system based on a lensless imaging system. Background Art

[0002] In modern high-precision engineering and scientific research, target positioning technology is a key means to achieve precise operation, improve efficiency and reduce errors. Target positioning technology is widely used in industrial automation, robot navigation, aerospace, medical equipment and other fields, and its core lies in determining the spatial position of the target object in real time and accurately. At present, target positioning technology mainly includes four categories: mechanical, imaging, electromagnetic and optical positioning. Among them, the optical positioning system is widely used because of its high precision, large positioning space and good environmental adaptability. However, existing optical positioning systems mostly rely on complex optical instruments and cumbersome measurement steps, which are not only costly, but also have many limitations in practical applications, such as demanding measurement environments, and the measurement accuracy is easily affected by external interference, etc., making it difficult to meet the requirements of rapid, efficient and low-cost measurement of the target position.

[0003] In recent years, lensless imaging technology has received wide attention due to its compact, lightweight and cost-effective characteristics. This technology does not rely on traditional optical lenses, but realizes imaging through the cooperation of a mask plate and an image sensor. The lensless imaging system can achieve high-precision measurement of the target position by using the specific design of the mask plate and image processing algorithms. For example, a lensless light source marking positioning system realizes high-precision measurement of the light source position through the aliasing effect of a grating mask and an image sensor. In addition, lensless imaging technology also has strong adaptability to ambient light and can maintain high measurement accuracy under complex lighting conditions.

[0004] In the field of target positioning, the application prospect of lensless imaging technology is broad. Its compact size and low-cost design make it particularly suitable for scenarios with strict requirements on equipment volume and cost. For example, a lensless camera can be integrated onto the target object, and by analyzing the imaging information of the light source after passing through the mask plate, the position of the target object can be measured in real time. This method not only simplifies the measurement process, but also significantly reduces the cost and complexity of the system.

[0005] In addition, lensless imaging technology also shows great potential in multi-view imaging and 3D reconstruction. For example, through the joint processing of multi-view lensless images, high-precision 3D scene reconstruction can be achieved, providing richer visual information for target positioning. The progress of this technology provides new ideas and methods for the development of target positioning systems, and is expected to promote further innovation and application of related technologies.

[0006] In summary, the application of lensless imaging technology in target localization has significant advantages, especially in improving measurement accuracy, reducing costs, and enhancing environmental adaptability. However, currently, this technology is still in the development stage and requires further optimization and verification to meet the strict requirements of practical applications. Summary of the Invention

[0007] In view of this, aiming at the defects and deficiencies of the existing technology, the purpose of the present invention is to provide a spatial point localization method and system based on a lensless imaging system, which can improve the accuracy and measurement range of the spatial point localization system and reduce the volume and cost of the localization device.

[0008] The technical solution specifically adopted by the present invention to solve its technical problems is as follows:

[0009] A spatial point localization method based on a lensless imaging system includes the following steps:

[0010] Generate a mask pattern with position perception characteristics: By optimizing the shape and arrangement of the mask pattern, each sub-region of the pattern uniquely corresponds to the displacement direction and amplitude of the light source, and the position perception characteristics are realized by the uniqueness mapping of the local pattern structure to decouple the displacement direction and amplitude;

[0011] Calculate the sub-pixel displacement of the light spot image: Based on the modulation result of the mask pattern on the light source, collect two adjacent frames of light spot images, analyze the cross-power spectrum by the phase correlation method, and fit the sub-pixel displacement amount;

[0012] Map the physical position change of the light source: According to the calibration parameters and the geometric model, convert the sub-pixel displacement into the coordinate change of the light source in three-dimensional space;

[0013] Among them, the geometric model includes the compensation calculation of the mask thickness and refractive index.

[0014] Furthermore, the position perception characteristics are realized by the De Bruijn sequence, and the specific method for generating the mask pattern is: Based on the De Bruijn sequence generation rule, construct an encoding sequence, and generate a two-dimensional pattern covering the mask plate through cyclic displacement and splicing.

[0015] Furthermore, the rule for optimizing the arrangement of the mask pattern is: By adjusting the relative position relationship between the pattern units, the imaging error caused by the optical proximity effect is reduced.

[0016] Furthermore, the specific implementation of the phase correlation method includes: performing a fast Fourier transform on two frames of light spot images, calculating the normalized cross-power spectrum and then performing an inverse transform, and fitting the sub-pixel coordinates of the peak value of the impulse response through a Gaussian surface.

[0017] Further, the calibration parameters are obtained by moving the light source to a calibration point with known coordinates, establishing a linear mapping relationship between the pixel displacement of the light spot and the true displacement, and calculating the refraction offset according to the thickness and refractive index of the mask;

[0018] The geometric model is to solve the light source position by combining the similar triangle model of the punctuation displacement and the shadow displacement with the calibration parameters.

[0019] Further, when calculating the sub-pixel displacement of the light spot image, deblurring processing is performed on the light spot image, and the size and standard deviation of the Gaussian deconvolution kernel are adjusted according to the degree of blurring.

[0020] And, a spatial point positioning system based on a lensless imaging system, comprising:

[0021] A marked light source assembly for generating a controllable light spot;

[0022] An amplitude-type binary mask, the pattern of which is a two-dimensional De Bruijn sequence;

[0023] An image sensor fixed behind the mask to collect the light spot image;

[0024] A processor configured to execute the steps of the method described above.

[0025] Further, the amplitude-type binary mask is prepared by a lithography technique, and the pattern is a two-dimensional De Bruijn sequence.

[0026] Further, the system includes a three-dimensional moving platform, and the marked light source assembly is fixed on the three-dimensional moving platform.

[0027] Further, the processor includes:

[0028] An imaging model module for constructing an imaging model according to the positional relationship between the light source, the mask and the light spot;

[0029] A projection pattern processing module for processing the light spot image and calculating the sub-pixel displacement;

[0030] A spatial position calculation module for mapping the pixel displacement to the three-dimensional coordinates of the light source.

[0031] Compared with the prior art, the beneficial effects of the present invention and its preferred solutions at least include:

[0032] System miniaturization and low cost: By eliminating the traditional optical lens assembly and using a lithography technique to prepare the mask, the hardware complexity and volume are significantly reduced;

[0033] High-precision displacement mapping: Based on the mask pattern design of the De Bruijn sequence, each sub-region uniquely corresponds to the displacement direction and amplitude of the light source, and combined with the sub-pixel level displacement calculation of the phase correlation method, high positioning accuracy is achieved;

[0034] Anti-environmental interference ability: By optimizing the mask pattern arrangement, the optical proximity effect is suppressed, and the imaging error is reduced.

[0035] Robustness to non-ideal factors: The thickness and refractive index compensation calculations of the mask are introduced into the geometric model to improve the measurement stability under complex working conditions.

[0036] Calibration simplification and scalability: By establishing a linear mapping relationship between pixels and physical displacements through calibration parameters, the system deployment difficulty is reduced, and multi-scene applications are supported. Brief Description of the Drawings

[0037] The present invention will be further described in detail below in conjunction with the drawings and specific embodiments:

[0038] Figure 1 It is a flowchart for constructing the solution of the embodiment of the present invention;

[0039] Figure 2 It is a flowchart of the system operation of the embodiment of the present invention;

[0040] Figure 3 It is a schematic diagram of the system of the embodiment of the present invention. Detailed Description of the Invention

[0041] To make the features and advantages of this patent more obvious and understandable, specific embodiments are given below and described in detail as follows:

[0042] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0044] As Figures 1 - 3 shown, the embodiment of the present invention provides a detection solution for obtaining the lateral symmetry of the back based on a point cloud registration method and a three-dimensional surface reconstruction algorithm, and its construction process includes the following steps:

[0045] Step S1: Build a lensless imaging system and place a marker light source;

[0046] Step S2: Establish a projection imaging model;

[0047] Step S3: Design a mask pattern according to the position sensing principle;

[0048] Step S4: Shoot the spot motion pattern and perform preprocessing;

[0049] Step S5: Calculate the sub-pixel displacement of the image and infer the actual position of the light source according to the imaging model.

[0050] As a preferred solution of this embodiment, step S1 is specifically as follows:

[0051] Select a high-brightness LED as the marking light source, use a three-dimensional moving platform as the carrier for the light source marking movement, attach the marking light source to the moving platform, and associate to realize the association between the light source and the moving object. Fix the image sensor on the bracket to ensure its stable and horizontal position, and avoid imaging errors caused by vibration or tilt. Place an amplitude binary mask in front of the sensor, whose function is to modulate the light intensity distribution of the light source so as to form a specific spot pattern on the sensor. Move the light source to ensure that the image sensor can capture the spot pattern generated by the light source irradiating the mask plate.

[0052] As a preferred solution of this embodiment, step S2 is specifically as follows:

[0053] Move the measured target, and record the position change of the shadow generated by the light source during the movement of the target on the imaging plane. According to the principle that light travels in a straight line, build a visual cone, establish the linear relationship between the corresponding points of the light source, the mask, and the imaging pattern, and construct an imaging model. Define a three-dimensional Cartesian coordinate system, including the light source, the mask, and the imaging pattern. Among them, the mask plane is located in the z = 0 plane with coordinates (u, v, 0); the image sensor plane is parallel to the mask and is located in the z = d plane with coordinates (u′, v′, d); the position coordinates of the point light source are (x, y, z). Based on the straight-line propagation hypothesis, establish the light equation:

[0054]

[0055] where (u, v) are the coordinates of the mask, and (u′, v′) are the coordinates of the image sensor.

[0056] The sensor image I(u′, v′) is determined by the mask transmittance T(u, v):

[0057]

[0058] To simplify the calculation, the projection change can be expressed as a matrix operation in homogeneous coordinates:

[0059]

[0060] As a preferred solution of this embodiment, step S3 is specifically as follows:

[0061] The mask pattern design module is used to design patterns with position-sensing characteristics. These patterns can help the system estimate the displacement of the marked light source through their unique structures and distributions. By optimizing the design of the mask pattern, the positioning accuracy and imaging quality of the system can be significantly improved. High-quality mask patterns can ensure the accurate estimation of the light source position, thereby enhancing the performance of the entire lensless positioning system. In the embodiments of the present invention, the preferably adopted position-sensing pattern is a two-dimensional De Bruijn sequence. First, the parameters of the two-dimensional De Bruijn sequence need to be determined according to the requirements of the imaging system. These mainly include the order n of the sequence and the size k of the character set. In a lensless imaging system, the size k of the character set usually corresponds to the different gray levels or transparency levels that can be used in the mask pattern, while the order n determines the complexity and coverage of the sequence. For example, for a two-dimensional De Bruijn sequence with k = 2 (binary), the mask pattern can consist of two states: transparent and opaque; for the case of k > 2, more complex patterns can be achieved through gray-level modulation. The generation of the two-dimensional De Bruijn sequence is based on the one-dimensional sequence. Each node in the De Bruijn graph represents a subsequence of length n - 1, and each edge represents the transition from one subsequence to another. Find a path in the De Bruijn graph such that each edge is visited exactly once. This path is the one-dimensional De Bruijn sequence. Then, through displacement and splicing operations on the one-dimensional sequence, each row or column of the matrix is randomly displaced, or multiple matrices are spliced together to generate the two-dimensional De Bruijn sequence. According to the designed pattern, a corresponding binary mask plate is generated using lithography technology.

[0062] As a preferred solution of this embodiment, step S4 is specifically:

[0063] Start the lensless imaging system, turn on the light source, and let it form a spot pattern on the sensor through the mask. Use a high-precision stepper motor to move the light source. During the process of moving the light source, observe the movement of the spot and use an image sensor to collect the spot pattern. Collect multiple spot pattern images to obtain information about the spot at different positions and states. Since the imaging process is equivalent to a low-pass filter, the collected spot pattern will be blurred to varying degrees according to the position of the light source. Therefore, denoise the collected images. According to the noise situation of the spot, use Gaussian blur kernel deconvolution to shoot the shadow pattern, select an appropriate Gaussian kernel size and standard deviation to obtain a clear pattern, compare the images before and after processing to ensure that the noise is effectively suppressed, and at the same time, the shape and position information of the spot are not significantly affected. If the contrast of the spot pattern is low, it may cause difficulties in subsequent processing. To improve the recognizability of the image, histogram equalization or contrast enhancement algorithms can be used. Histogram equalization adjusts the gray distribution of the image to make it more uniform, thereby improving the overall contrast. The contrast enhancement algorithm adjusts the brightness and contrast parameters of the image to highlight the difference between the spot and the background. Since there is significant noise at the image edges, crop and align the image to remove the invalid areas at the image edges and ensure that the coordinate systems of all images are consistent.

[0064] As a preferred solution of this embodiment, step S5 is specifically as follows:

[0065] The spatial position calculation module is used to convert the calculated pixel-level displacement into a real displacement. To achieve the conversion from pixel displacement to real displacement, first, the imaging system needs to be calibrated to determine the internal parameters of the system and the correspondence between pixels and real distances. The calibration is carried out in three steps: 1. Select calibration points: Select several calibration points with known positions within the field of view of the imaging system. These calibration points should be evenly distributed to cover the entire working range of the imaging system; 2. Record pixel displacement: Place the light source at each calibration point in turn, and use the image sensor to record the corresponding spot pattern. Calculate the pixel displacement of the spot on the sensor through an image processing algorithm. 3. Establish the correspondence: According to the real positions of the calibration points and the pixel displacements recorded on the sensor, establish the correspondence between pixels and real distances through fitting. By moving the position of the light source in multiple directions, the internal parameters of the imaging system are calibrated to obtain the pixel-millimeter correspondence of the measurement system. The displacement of the spot pattern is calculated by the phase correlation method. Select two consecutive frames of patterns, denoted as the reference pattern Img ref and Img cur . Perform a fast Fourier transform on the two pictures and calculate the cross-power spectrum:

[0066]

[0067] The inverse Fourier transform is performed on the obtained cross-power spectrum to obtain the impulse response, and the peak coordinates thereof correspond to the movement of the pattern. The sub-pixel displacement (Δu′, Δv′) is obtained by Gaussian fitting.

[0068] According to the geometric model, the punctuation displacement and the shadow displacement satisfy the similar triangle model, and the light source position is inversely solved by combining the calibration parameters:

[0069]

[0070] The depth z of the light source is solved according to the depth relationship of multiple groups of known displacements or phases:

[0071]

[0072] According to the design of the above embodiments, the overall architecture design of the system of the embodiments of the present invention is obtained. The hardware components of the system include:

[0073] Marker light source component: A high-brightness LED light source is fixed on a three-dimensional moving platform for generating a controllable light spot;

[0074] Mask: An amplitude-type binary mask with a pattern of a two-dimensional De Bruijn sequence, prepared by lithography technology;

[0075] Image sensor: Fixed on a rigid bracket and placed parallel to the mask;

[0076] Calibration device: Includes a calibration plate with known spatial coordinates;

[0077] The algorithm module includes:

[0078] Imaging module: Constructs the physical mapping relationship of the light source-mask-light spot, including the ideal model (light ray propagating in a straight line) and the non-ideal model (mask thickness, refraction compensation);

[0079] Mask design module: Generates a two-dimensional De Bruijn sequence and optimizes the pattern arrangement to suppress the optical proximity effect;

[0080] Pattern processing module: Performs Gaussian deconvolution deblurring and phase correlation method to calculate the sub-pixel displacement;

[0081] Positioning algorithm module: Maps the pixel displacement to the real position through the system calibration parameters and the similar triangle model.

[0082] Among them, the imaging module uses light-emitting diodes to illuminate the photolithography mask to generate a motion trajectory; the mask design module uses position sensing patterns to involve mask shapes; the image processing module analyzes the blur of the image and performs preprocessing; the positioning algorithm module analyzes the displacement between different frame patterns through Fourier transform, and analyzes the displacement results of the light source through the inverse imaging process. The present invention innovatively proposes to use a lensless imaging system as a positioning carrier, and with its unique imaging principle, it achieves a high-precision and high-stability positioning effect. This technology is applied to light mark positioning at a lower cost and higher controllability, which can bring new breakthroughs in light source positioning.

[0083] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0084] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

[0085] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive various other forms of a spatial point positioning method and system based on a lensless imaging system under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.

Claims

1. A spatial point positioning method based on a lensless imaging system, characterized in that: The following steps are involved: Generate a mask pattern with position-awareness characteristics: by optimizing the shape and arrangement of the mask pattern, each sub-region of the pattern uniquely corresponds to the displacement direction and amplitude of the light source, and the position-awareness characteristics realize the decoupling of the displacement direction and amplitude through the uniqueness mapping of the local pattern structure; Calculating the sub-pixel displacement of the spot image: based on the modulation result of the mask pattern on the light source, collecting two adjacent frames of spot images, analyzing the cross power spectrum by phase correlation method, and fitting the sub-pixel displacement; Mapping the physical position change of the light source: converting the sub-pixel displacement into the coordinate change of the light source in three-dimensional space according to the calibration parameters and the geometric model; The geometric model includes compensation calculations for mask thickness and refractive index.

2. The spatial point positioning method based on a lensless imaging system according to claim 1, characterized in that: The position sensing feature is realized by a De Bruijn sequence, and the specific method for generating the mask pattern is: constructing a coding sequence based on a De Bruijn sequence generation rule, and generating a two-dimensional pattern covering the mask plate by cyclic shifting and splicing.

3. The spatial point positioning method based on a lensless imaging system according to claim 1, characterized in that: The rule for optimizing the mask pattern arrangement is to reduce the imaging error caused by the optical proximity effect by adjusting the relative position relationship between the pattern units.

4. The spatial point positioning method based on a lensless imaging system according to claim 1, characterized in that: The specific implementation of the phase correlation method includes: performing fast Fourier transform on two frames of spot images, calculating the normalized cross-power spectrum and then performing inverse transform, and fitting the sub-pixel coordinates of the impulse response peak through Gaussian surface.

5. The spatial point positioning method based on a lensless imaging system according to claim 1, characterized in that: The calibration parameters are established by moving the light source to a calibration point with known coordinates, establishing a linear mapping relationship between the spot pixel displacement and the real displacement, and calculating the refraction offset according to the mask thickness and refractive index; The geometric model is that the punctuation point displacement and the shadow displacement satisfy the similar triangle model combined with the calibration parameters to inversely solve the light source position.

6. The spatial point positioning method based on a lensless imaging system according to claim 1, characterized in that: When calculating the sub-pixel displacement of the spot image, a deblurring process is performed on the spot image, and the size and standard deviation of the Gaussian deconvolution kernel are adjusted according to the degree of blur.

7. A spatial point positioning system based on a lensless imaging system, characterized in that: include: A marking light source component is used to generate a controllable light spot; Amplitude-type binary mask, whose pattern is a two-dimensional De Bruijn sequence; An image sensor is fixed behind the mask to collect a light spot image; A processor configured to execute the steps of any method described in claims 1-6.

8. The spatial point positioning system based on a lensless imaging system according to claim 7, characterized in that: The amplitude-type binary mask is prepared by photolithography, and its pattern is a two-dimensional De Bruijn sequence.

9. The spatial point positioning system based on a lensless imaging system according to claim 7, characterized in that: The system comprises a three-dimensional mobile platform, and the marking light source assembly is fixed on the three-dimensional mobile platform.

10. The spatial point positioning system based on a lensless imaging system according to claim 7, characterized in that: The processor comprises: An imaging model module is used to construct an imaging model according to the positional relationship between the light source, the mask and the light spot; A projection pattern processing module, used to process the spot image and calculate the sub-pixel displacement; The spatial position estimation module is used to map pixel displacement into three-dimensional coordinates of the light source.

Images