A lens-free light source marking positioning system and calibration method thereof

Abstract

The present invention discloses a lensless light source marking positioning system and a calibration method thereof, comprising: arranging the light source on an object to be measured at a distance, photographing the light source through an image sensor, obtaining an aliasing image of an aliasing pattern due to aliasing between the frequency of a grating mask and the sampling frequency of the image sensor, obtaining an aliasing image function based on the aliasing image, initially labeling the period of the aliasing image, constructing an optimization function based on the initial label and the aliasing image function, calculating the optimization function through a least squares method, obtaining an actual period and a camera intrinsic parameter to implement intrinsic parameter calibration, and obtaining the position of the light source based on the actual period and the camera intrinsic parameter.

Description

Technical Field

[0001] The present invention belongs to the technical field of light source calibration, and in particular relates to a lens-free light source marking positioning system and a calibration method thereof. Background Art

[0002] Existing methods for light source marker localization mostly use reflective or luminous elements, such as reflective balls and point light sources, to mark and track targets. Most methods use these elements to form structured patterns within images, allowing them to extract patterns with sub-pixel accuracy. This results in the upper bound of the implicit localization resolution being locked at sub-pixel resolution, severely restricting the development of related fields in object pose perception.

[0003] There are currently two main methods to improve positioning resolution:

[0004] 1. Use the neighborhood information of pixel points to improve the sub-pixel accuracy of image points. This method increases the density of distinguishable points in the image.

[0005] 2. Use multiple cameras to capture the target simultaneously and calculate the target position by integrating multi-view information. Multi-view fusion is essentially the superposition of image points from different perspectives.

[0006] Although these methods can theoretically improve spatial resolution infinitely, due to limited calibration accuracy and edge effects, their effects on improving visual resolution are not very significant. Summary of the Invention

[0007] In order to solve the above technical problems, the present invention proposes a lens-less light source marking positioning system and a calibration method thereof to solve the problems existing in the above-mentioned prior art.

[0008] To achieve the above-mentioned objectives, the present invention provides a lens-free light source marking positioning system, comprising: a light source, a grating mask and an image sensor; wherein the light source is at a distance from the object to be measured, the grating mask is arranged between the light source and the image sensor, and the frequency of the grating mask and the sampling frequency of the image sensor are aliased.

[0009] In order to better achieve the above technical objectives, the present invention also provides a calibration method based on the above system, including:

[0010] The light source is set on the object to be measured at the distance, and the light source is photographed by an image sensor. Due to aliasing between the frequency of the grating mask and the sampling frequency of the image sensor, an aliasing image of the aliasing pattern is obtained. According to the aliasing image, an aliasing image function is obtained, and the period of the aliasing image is initially labeled. An optimization function is constructed based on the initial label and the aliasing image function. The optimization function is calculated by the least squares method to obtain the actual period label and the camera intrinsic parameters to realize intrinsic parameter calibration. The position of the light source is obtained based on the actual period and the camera intrinsic parameters.

[0011] Optionally, the aliased image function S A for:

[0012]

[0013] Where F is the period of 2π, represents the unwrapped phase of the aliasing pattern, w A The circular frequency of the aliasing pattern, x represents the pixel position in the image plane, Indicates the initial phase of the projected pattern.

[0014] Optionally, the circular frequency w of the aliasing pattern A for:

[0015]

[0016] Among them, d L is the distance from the light source to the mask, h is the distance from the image plane to the mask, w M is the mask frequency, w e is the mask frequency w M and the difference between the CCD sampling frequency.

[0017] Optionally, the aliased image function I is:

[0018]

[0019] Where F is the period of 2π, u represents the pixel coordinates of the point on the aliasing pattern, ω A is the pixel circular frequency, is the phase of the aliasing pattern, where is the initial phase of the aliasing pattern,

[0020] Optionally, the pixel circular frequency is:

[0021] ω A =w A T S =2πT S / T A =2π / τ A

[0022] Among them, T A represents the period of the aliasing pattern, T S represents the CCD sampling period, τ A is the aliasing pattern S A pixel period.

[0023] Optionally, the optimization function is:

[0024]

[0025] Among them, ω n represents the circular frequency of the aliasing pattern at each light source position when the light source position is moved at equal intervals, ω e represents the camera intrinsic parameter error caused by the installation error, u1 represents the initial position of the pixel coordinate of the light source, Δu represents the pixel coordinate moving step of the light source, represents the initial phase at each light source position, ψ M Indicates the actual phase.

[0026] Optionally, the process of initially labeling the periods of the aliased image includes:

[0027] Any phase zero point in the mask image is marked as a control point of the aliased image, that is, the tracking projection image S p The m-th phase zero point of , where m is the period number. Compared with the prior art, the present invention has the following advantages and technical effects:

[0028] This paper develops a lensless light source marking positioning system and its calibration method. Using a lensless camera (CCD sensor) in conjunction with a mask, a light-emitting element marks the target object, avoiding ambient light bands so that the light emitted by the target point dominates the imaging process. Object motion is detected by measuring the displacement of a point light source, and Moiré acceleration is introduced to amplify target motion and ensure accurate positioning results. The position of the point light source is reversed by analyzing the motion of the Moiré pattern, and the system's internal parameters are calibrated.

[0029] The present invention utilizes the aliasing of the sampling frequency and the mask projection frequency to form an interference pattern on the imaging surface, generating a Moore's acceleration phenomenon associated with the target displacement, thereby amplifying the target's motion. Based on the property that the unfolded phase at the foot of the light source marker remains unchanged, the light source marker can be positioned in reverse, and the system internal parameters can be solved through an optimization algorithm. This positioning process is innovative and is not constrained by the limitations of traditional spatial resolution. The principle of generating the Moore's acceleration phenomenon is also innovative, requiring only the target point as the projection light source, without the need to strictly control the entire workspace, and with high positioning accuracy, it is suitable for most applications of visual positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0031] Figure 1 A schematic diagram of an apparatus for applying the present invention according to an embodiment of the present invention;

[0032] Figure 2 This is a diagram showing the relationship between the mask, light source, and CCD image plane in an embodiment of the present invention;

[0033] Figure 3 The relationship between the projection moving distance and the moiré moving distance according to an embodiment of the present invention;

[0034] Figure 4 This is a flow chart of a positioning method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] The present invention develops a lensless light source marking positioning system and its calibration method. This method uses the sampling frequency of a lensless camera (CCD sensor) to alias the projection frequency of a single mask to produce Moore acceleration. A light-emitting element is used to mark the target object, avoiding the ambient light band so that the light emitted by the target point dominates the imaging process. By analyzing the motion of the Moore fringes, the point light source can be reversely positioned and the internal parameters of the lensless system can be calibrated. The present invention only requires the target point as the projection light source, eliminating the need for strict control of the entire workspace, and has high positioning accuracy, making it suitable for most applications of visual positioning.

[0038] The present invention develops a lens-free light source marking positioning system and a calibration method thereof.

[0039] A lensless camera (CCD sensor) is used in conjunction with a single-piece amplitude mask. A light-emitting element is used to mark the target object, avoiding ambient light bands so that the light emitted by the target point dominates the imaging process. Object motion is detected by measuring the displacement of a point light source, and Moiré acceleration is introduced to amplify the target motion and ensure the accuracy of the positioning results. The position of the point light source is reversed by analyzing the motion of the Moiré pattern, and the internal parameters of the system are calibrated.

[0040] The specific method is as follows:

[0041] like Figure 1 As shown, a black and white stripe cycle (T M ) mask. Light from the target point passes through the holes in the mask, projecting an array of bright spots on the imaging surface. The aliasing of the mask's projection and the CCD image plane's sampling frequency creates a moiré pattern on the imaging surface, leading to a phenomenon called moiré acceleration, manifesting as an interference waveform composed of rapidly changing phases and varying light and dark fringes.

[0042] like Figure 2 As shown, the origin of the camera coordinate system is defined at the endpoint of the image plane, the x-axis is the image plane, and the z-axis is the image plane. L Axis is the depth from the light source to the image plane, d L is the distance from the light source to the mask, and h is the distance from the image plane to the mask. The circular frequency of the mask, that is, the mask frequency, is w M According to the triangle similarity principle, the projection pattern S of the mask on the image plane can be calculated. P The circular frequency w P :

[0043]

[0044] The camera imaging is approximated as S P The impulse sampling of CCD sampling is w S The ideal aliasing pattern S obtained by sampling A The circular frequency w A 'for:

[0045]

[0046] Where n indicates that the camera samples at n pixels, and n>1 means that the camera performs interval sampling.

[0047] Since there is an error in the machining accuracy, let w e is the mask frequency w M and CCD sampling frequency w S The difference between

[0048] w S =w e +w M

[0049] If w is satisfied P ∈(0.5,1)w S , taking the error into consideration, the aliasing pattern S obtained by sampling can be obtained A The circular frequency w A :

[0050]

[0051] The mask pattern is designed to be black and white stripes, and its waveform is a periodic function, so it can be written as:

[0052]

[0053] Where F is the period of 2π, is the unwrapped phase of the mask, is the phase of the boundary at the endpoint of the image plane, that is, the initial phase. M represents the mask pattern, x M Indicates the position of a point on the mask pattern.

[0054] Again using the triangle similarity principle, we have the following relationship:

[0055] z L x M =d L x+hx L

[0056] Among them, x M Indicates the position of the mask pattern point, x L Indicates the position of the point light source, and x represents point x on the mask pattern mask M The location of the projection.

[0057] Bring in The projection pattern S can be obtained P Function:

[0058]

[0059] in, represents the unwrapped phase of the projected pattern, Indicates the initial phase of the projected pattern.

[0060] At the same time, at the image sampling position nT S Projection pattern S P With aliasing pattern S A The points are equal, that is

[0061] S P (nT S )=S A (nT S )

[0062] T S Indicates the CCD sampling period, and n indicates the sampling period number.

[0063] Available Therefore, the aliasing pattern function S A It can be written as:

[0064]

[0065] in, represents the unwrapped phase of the aliasing pattern, is the initial phase of the aliasing pattern.

[0066] The combined phase of the aliasing pattern can also be written as For S A At the foot of the light source, x L The unwrapped phase, that is, x = x L When , we can get:

[0067]

[0068] Because of w e As the internal parameter of the system, it can be clearly seen that during the movement of the light source, the unfolded phase of the perpendicular foot position remains unchanged, so the reverse positioning of the light source marker can be achieved.

[0069] Here, the data obtained by the camera is an aliasing pattern S A The image I needs to be converted into pixels, pixel circle frequency ω A for:

[0070] ω A =w A T S =2πT S / T A =2π / τ A

[0071] Among them, T A represents the period of the aliasing pattern, T S Indicates the CCD sampling period.

[0072] where τ A is the aliasing pattern S A The pixel period of the image I function can be written as:

[0073]

[0074] Where u represents the pixel coordinates of a point on the aliasing pattern.

[0075] Positioning method:

[0076] Based on the property that the unfolded phase at the foot of the light source marker is invariant, the aliased image S on the image I can be estimated. A The pixel period τ A Then the combined phase in the positioning image I is However, in tracking the phase, the periodic confusion problem needs to be solved.

[0077] Solve the cycle confusion problem by using cycle labels:

[0078] 1. In the mask image S M Mark any phase zero point on the image, making it a control point that can be located in the image I, that is, tracking the projected image S p The mth phase zero point of ;

[0079] 2. When positioning, calculate the sub-pixel position u of the control point c =n+k, where n is the control point on the right side of the nth pixel and the period coefficient k∈[0,1);

[0080] 3. Based on the aliasing pattern S on image I A The pixel period τ A ;

[0081] 4. Find the sub-pixel distance u in the image c -kτ A The nearest aliasing pattern S A The phase zero point is marked with nm and extends left and right.

[0082] Positioning phase The pixel coordinates u of the light source can be calculated using the period L ,

[0083] u L =u / (1-τ A / τ e );

[0084] Among them, u represents the combined phase The pixel position of the pixel, τ e Indicates the pixel period of the system intrinsic parameter.

[0085] However, in this positioning method, the value of m is unknown and is affected by assembly and is usually difficult to control. As a result, the calculated phase difference is 2mπ from the actual phase. m can be calculated through internal calibration.

[0086] Internal reference calibration:

[0087] Based on the property of the unwrapped phase invariance at the foot of the light source marker, it is converted into an image.

[0088]

[0089] in, Represents the initial phase of the aliased image, ω e is the installation error, the difference between the sampling frequency and the mask frequency w e With relevant connections, specific e =w e T S, is one of the internal parameters of the system

[0090] By moving the light source at equal intervals, N light source positions are collected. Since the phase difference between the positioning method and the actual phase is 2mπ, we can get:

[0091]

[0092] Among them, ω n ω represents the position of each light source A , u1 represents the initial pixel coordinate position of the light source, Δu represents the pixel coordinate moving step of the light source, Indicates that each light source position is the initial phase on the image, The optimization algorithm is used to calibrate the unknowns: cycle number, closing phase, installation error, i.e. internal parameters.

[0093] Consider the optimization problem and write the formula as

[0094]

[0095] Since there is a nonlinear term (ω n -ω e )(u1+nΔu), the alternating optimization method can be used.

[0096] Because ω e Does not change with the movement of the light source, so first assume that ω e Known, optimize other parameters.

[0097]

[0098] Among them, a n =ω n -ω e is the circular frequency of the aliasing pattern at each light source position after error removal, ψ M Indicates the actual phase.

[0099] Find the first-order derivative of the optimization object and find the extreme point

[0100]

[0101]

[0102] Converted into matrix form, we can get:

[0103]

[0104] The inverse solution can be used to obtain the initial pixel coordinate position u1 of the light source in the least squares optimal sense, and the pixel coordinate movement step Δ of the light source u , the actual phase ψ M,because So the obtained ψ M The integer number of cycles in the middle can be removed to find the phase zero point number corresponding to the control point, that is, the cycle number m, and the phase

[0105] Optimize ω e

[0106]

[0107] Among them, c n Indicates auxiliary parameters, b n Indicates the pixel position of the current light source.

[0108] Derivative the optimization object and find the extreme point

[0109]

[0110] Transformation can be obtained

[0111]

[0112] The most meaningful ω under the least squares method can be obtained e

[0113] Example

[0114] The present invention proposes a novel lens-free light source positioning system for vision-based spatial point coordinate estimation, referred to as visual positioning. The system consists of a point light source, a grating mask, and an image sensor. The light source is the target to be positioned. Similar to a fiducial marker, the light source can be attached to an object of interest for tracking. By positioning the light source, the position and displacement of the object of interest can be indirectly determined. The frequency of the mask pattern and the sampling frequency of the image sensor are aliased to produce moiré patterns. The tiny displacement of the point light source converts the low-frequency macroscopic changes of the micromoiré fringes into micro-moiré fringes. By analyzing the changes in the moiré fringes, the position of the light source can be inferred. Due to the amplification effect of the moiré fringes, this point-to-pattern mapping relationship can exceed the upper limit of the spatial resolution of a one-dimensional camera.

[0115] Analysis of the changes in moiré fringes

[0116] Due to the amplification effect of the moiré effect, period confusion can occur arbitrarily based on the changes between the captured images. Therefore, it is first necessary to improve the period confusion by labeling the period of the moiré pattern. This label only represents the difference between the periods to distinguish the phase differences, but this label differs from the actual period by a fixed value.

[0117] For which w e The frequency difference between the mask pattern and the sampling frequency of the image sensor is also a system internal parameter and needs to be calculated through calibration.

[0118] The real period and the system internal parameters are calibrated by the optimization algorithm based on the least squares algorithm. Optimization algorithm: First, based on the property that the unfolded phase at the foot of the light source marker is constant, by using this property, the corresponding equation can be obtained for the equally spaced displacement light source. As long as the equation is greater than the number of unknowns, the unknowns can be solved. Then, the optimization algorithm is used to calibrate the unknowns under the optimal result.

[0119] Finally, a calibration result or positioning result is obtained. The solution has a final result output and can solve specific technical problems.

[0120] The advantages and uses of this product.

[0121] This paper develops a lensless light source marking positioning system and its calibration method. Using a lensless camera (CCD sensor) in conjunction with a mask, a light-emitting element is used to mark the target object, avoiding the ambient light band so that the light emitted by the target point dominates the imaging process. The object's motion is detected by measuring the displacement of the point light source and introducing Moore acceleration to amplify the target motion and ensure the accuracy of the positioning result. By inversely analyzing the motion of the Moore pattern, the initial pixel coordinate position u1 of the light source and the pixel coordinate movement step length Δ of the light source are obtained. u The position of the point light source is deduced through the triangle similarity principle and the internal parameter control point of the system corresponds to the phase zero point, that is, the period number m, and the phase Installation error ω e Calibration is performed. The present invention utilizes the aliasing of the sampling frequency and the mask projection frequency to form an interference pattern on the imaging surface, generating a Moore's acceleration phenomenon associated with the target displacement, thereby amplifying the target movement. Based on the property that the unfolded phase at the vertical foot of the light source marker remains unchanged, the light source marker positioning can be achieved in reverse, and the system internal parameters are solved by the optimization algorithm. This positioning process is innovative and is not constrained by the limitations of traditional spatial resolution. The principle of generating the Moore's acceleration phenomenon is also innovative. It only requires the target point as a projection light source, and there is no need to strictly control the entire workspace. The positioning accuracy is high, and it is suitable for most applications of visual positioning.

[0122] The process or method of using the product.

[0123] 1. Place a black and white stripe cycle (T M ) mask with control points, the points can be observed on the image plane, and the point light source is fixed on the object to be measured. The light source is moved and the moiré pattern at the corresponding position is captured by the CCD.

[0124] 2. Generate the corresponding grayscale change curve based on the image taken by the camera

[0125] 3. Perform periodic labeling and calculate the internal parameters to achieve the positioning and calibration of the light source marker.

[0126] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A calibration method for a lensless light source marking positioning system, characterized in that: The lensless light source marking positioning system comprises: a light source, a grating mask and an image sensor; wherein the light source is arranged on the object to be measured, the grating mask is arranged between the light source and the image sensor, and the frequency of the grating mask and the sampling frequency of the image sensor are aliased; The calibration method based on the above positioning system includes: The light source is disposed on the object to be measured, and the light source is photographed by an image sensor. Due to aliasing between the frequency of the grating mask and the sampling frequency of the image sensor, an aliased image of an aliased pattern is obtained. An aliased image function is obtained based on the aliased image, and a period of the aliased image is initially labeled. An optimization function is constructed based on the initial label and the aliased image function. The optimization function is calculated by a least squares method to obtain an actual period label m and a camera intrinsic parameter to implement intrinsic parameter calibration, and the position of the light source is obtained based on the actual period and the camera intrinsic parameter. The optimization function is: Among them, ω n Represents the circular frequency ω of the pixel at each light source position when the light source position is moved at equal intervals A ,ω e represents the camera intrinsic parameter error caused by the installation error, u1 represents the initial position of the pixel coordinate of the light source, Δu represents the pixel coordinate moving step of the light source, represents the initial phase at each light source position, ψ M represents the actual phase, and N represents the number of light source positions; is the phase of the boundary at the endpoint of the image plane, that is, the initial phase; The process of initially labeling the periods of the aliased image includes: Mark any phase zero point in the mask image as the control point of the aliased image, that is, track the projection image S p The mth phase zero point of , where m is the actual cycle number.

2. The method according to claim 1, characterized in that Circular frequency w of the aliasing pattern A for: Where h is the distance from the image plane to the mask, w M is the mask frequency, w e is the mask frequency w M The difference between the sampling frequency of the CCD and the sampling frequency of the CCD; L =h+d L , d L is the distance from the light source to the mask.

3. The method according to claim 2, characterized in that The pixel circular frequency is: oh A =w A T S =2πT S / T A =2π / τ A Among them, T A represents the period of the aliasing pattern, T S represents the CCD sampling period, τ A is the aliasing pattern S A pixel period.

Images