Lens-free light source mark positioning system and calibration method thereof
The lensless light source marking positioning system uses frequency aliasing of the grating mask and image sensor to generate molar acceleration, which solves the problem of insufficient positioning resolution in the light source marking positioning technology, and realizes high-precision light source positioning and system internal parameter calibration. in most visual positioning applications.
Patent Information
- Application Number
- CN202410997051.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-07-24
AI Technical Summary
The positioning resolution of existing light source marking positioning technology is limited by subpixels, making it difficult to significantly improve visual resolution, and there are problems such as edge effects.
The lensless light source marking positioning system is adopted, through the combination of light source, grating mask and image sensor, the aliasing of sampling frequency and mask projection frequency generates molar acceleration phenomenon, amplifies the target motion, and reversely deduces the light source position through the analysis of molar patterns. and system internal reference.
It realizes high-precision light source positioning, breaks through the limitations of traditional spatial resolution, is suitable for most visual positioning applications, and does not require strict control of the workspace.
Smart Images

Figure CN119027491A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of light source calibration, and in particular relates to a lens-free light source marking positioning system and a calibration method thereof. Background Art
[0002] Most existing light source marker positioning methods use reflective or luminous elements to mark and track targets, such as reflective balls and point light sources. Most methods use reflective or luminous elements to form structured graphics in images to extract patterns with sub-pixel accuracy. This will cause the upper limit of the resolution of the implicit positioning to be locked at the sub-pixel positioning resolution, which seriously restricts the development of related fields in target posture perception.
[0003] There are currently two main methods to improve positioning resolution:
[0004] 1. Use the neighborhood information of pixels to improve the sub-pixel accuracy of image points. This method increases the density of distinguishable points in the image.
[0005] 2. Use multiple cameras to capture the target at the same time and calculate the target position by integrating multi-view information. Multi-view fusion is essentially the superposition of image points from different perspectives.
[0006] Although these methods can theoretically improve spatial resolution infinitely, due to limited calibration accuracy and edge effects, their effects on improving visual resolution are not very significant. Summary of the invention
[0007] In order to solve the above technical problems, the present invention proposes a lens-free light source marking positioning system and a calibration method thereof to solve the above problems existing in the prior art.
[0008] To achieve the above-mentioned purpose, the present invention provides a lens-free light source marking positioning system, comprising: a light source, a grating mask and an image sensor; wherein the light source is at a distance from the object to be measured, the grating mask is arranged between the light source and the image sensor, and the frequency of the grating mask and the sampling frequency of the image sensor are aliased.
[0009] In order to better achieve the above technical objectives, the present invention also provides a calibration method based on the above system, including:
[0010] The light source is set on the object to be measured at the distance, and the light source is photographed by an image sensor. Due to aliasing between the frequency of the grating mask and the sampling frequency of the image sensor, an aliasing image of the aliasing pattern is obtained. According to the aliasing image, an aliasing image function is obtained, and the period of the aliasing image is initially labeled. An optimization function is constructed according to the initial label and the aliasing image function. The optimization function is calculated by the least squares method to obtain the actual period label and the camera intrinsic parameters to realize intrinsic parameter calibration, and the position of the light source is obtained based on the actual period and the camera intrinsic parameters.
[0011] Optionally, the aliased image function S A for:
[0012]
[0013] Where F is the period of 2π, represents the unwrapped phase of the aliasing pattern, w A The circular frequency of the aliasing pattern, x represents the pixel position in the image plane, Indicates the initial phase of the projected pattern.
[0014] Optionally, the circular frequency w of the aliasing pattern A for:
[0015]
[0016] Among them, d L is the distance from the light source to the mask, h is the distance from the image plane to the mask, and w M is the mask frequency, w e is the mask frequency w M The difference between the sampling frequency of the CCD and the sampling frequency of the CCD.
[0017] Optionally, the aliased image function I is:
[0018]
[0019] Where F is the period of 2π, u represents the pixel coordinates of the point on the aliasing pattern, ω A is the pixel circular frequency, is the phase of the aliasing pattern, where is the initial phase of the aliasing pattern,
[0020] Optionally, the pixel circle frequency is:
[0021] ω A =w A T S =2πT S / T A =2π / τ A
[0022] Among them, T A represents the period of the aliasing pattern, T S represents the CCD sampling period, τ A is the aliasing pattern S A The pixel period.
[0023] Optionally, the optimization function is:
[0024]
[0025] Among them, ω n represents the circular frequency of the aliasing pattern at each light source position when the light source position is moved at equal intervals, ω e represents the camera internal parameter error caused by installation error, u 1 represents the initial pixel coordinate position of the light source, Δu represents the pixel coordinate moving step of the light source, represents the initial phase at each light source position, ψ M Indicates the actual phase.
[0026] Optionally, the process of initially labeling the period of the aliased image includes:
[0027] Any phase zero point in the mask image is marked as a control point of the aliased image, that is, the tracking projection image S p The mth phase zero point of , where m is the period number. Compared with the prior art, the present invention has the following advantages and technical effects:
[0028] The present invention develops a lensless light source marking positioning system and its calibration method, which uses a lensless camera (CCD sensor) in conjunction with a mask plate, uses a light-emitting element to mark the target object, avoids the ambient light band and allows the light emitted by the target point to dominate the imaging process. The object movement is detected by measuring the displacement of the point light source, and the moiré acceleration is introduced to amplify the target movement and ensure the accuracy of the positioning result. The position of the point light source is reversely deduced by analyzing the motion of the moiré pattern and the internal parameters of the system are calibrated.
[0029] The present invention utilizes the aliasing of the sampling frequency and the mask projection frequency to form an interference pattern on the imaging surface, generating a Moore acceleration phenomenon associated with the target displacement, thereby amplifying the movement of the target. Based on the property that the unfolded phase at the foot of the light source mark is invariant, the light source mark positioning can be reversed, and the system internal parameters are solved by the optimization algorithm. The positioning process is innovative and is not constrained by the limitations of traditional spatial resolution. The principle of generating the Moore acceleration phenomenon is also innovative, requiring only the target point as a projection light source, without the need to strictly control the entire workspace, and with high positioning accuracy, it is suitable for most applications of visual positioning. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0031] Figure 1 A schematic diagram of a device for applying the present invention according to an embodiment of the present invention;
[0032] Figure 2 This is a diagram showing the relationship between the mask, the light source and the CCD image plane in an embodiment of the present invention;
[0033] Figure 3 is the relationship between the projection moving distance and the moiré moving distance in an embodiment of the present invention;
[0034] Figure 4 The figure is a flow chart of a positioning method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0036] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0037] The present invention develops a lensless light source marking positioning system and a calibration method thereof. The method uses the sampling frequency of a lensless camera (CCD sensor) and the projection frequency of a single mask to alias to produce moiré acceleration, uses a light-emitting element to mark the target object, avoids the ambient light band and allows the light emitted by the target point to dominate the imaging process. The point light source can be reversely positioned by analyzing the motion of the moiré fringes, and the internal parameters of the lensless system can be calibrated. The present invention only requires the target point as a projection light source, does not need to strictly control the entire workspace, and has high positioning accuracy, and is suitable for most applications of visual positioning.
[0038] The present invention develops a lens-free light source marking positioning system and a calibration method thereof.
[0039] A lensless camera (CCD sensor) is used in conjunction with a single amplitude mask, and a light-emitting element is used to mark the target object, avoiding the ambient light band so that the light emitted by the target point dominates the imaging process. The object movement is detected by measuring the displacement of the point light source, and Moore acceleration is introduced to amplify the target movement and ensure the accuracy of the positioning result. The position of the point light source is reversely deduced through the motion analysis of the Moore pattern and the internal parameters of the system are calibrated.
[0040] The specific method is as follows:
[0041] like Figure 1 As shown, a black and white stripe cycle (T M ) mask. The light emitted by the target point will pass through the holes in the mask and project a bright spot array on the imaging surface. The projection of the mask and the sampling frequency of the CCD image plane are aliased, and a moiré pattern will be generated on the imaging surface, resulting in a moiré acceleration phenomenon, which is manifested as an interference waveform with a fast phase change composed of the difference in light and dark fringes.
[0042] like Figure 2 As shown, the origin of the camera coordinate system is defined at the end point of the image plane, the x-axis is the image plane, and the z-axis is the image plane. L Axis is the depth from the light source to the image plane, d L is the distance from the light source to the mask, and h is the distance from the image plane to the mask. The circular frequency of the mask, i.e. the mask frequency, is w M According to the triangle similarity principle, the projection pattern S of the mask on the image plane can be calculated. P The circular frequency w P :
[0043]
[0044] The camera imaging is approximated as S P The impulse sampling of CCD sampling, the circular frequency of CDD sampling is w S The ideal aliasing pattern S obtained by sampling A The circular frequency w A 'for:
[0045]
[0046] Where n means that the camera samples n pixels, and n>1 means that the camera performs interval sampling.
[0047] Since there is an error in machining accuracy, let w e is the mask frequency w M and CCD sampling frequency w S The difference between
[0048] w S =w e +w M
[0049] If w is satisfied P ∈(0.5,1)w S , taking the error into account, the aliasing pattern S obtained by sampling can be obtained A The circular frequency w A :
[0050]
[0051] The mask pattern is designed as black and white stripes, and its waveform is a periodic function, so it can be recorded as:
[0052]
[0053] Where F is the period of 2π, is the unwrapped phase of the mask, is the phase of the boundary at the endpoint of the image plane, that is, the initial phase. M represents the mask pattern, x M Indicates the location of a point on the mask pattern.
[0054] Again, using the triangle similarity principle, we have the following relationship:
[0055] z L x M =d L x+hx L
[0056] Among them, x M Indicates the position of the mask pattern point, x L Indicates the position of the point light source, and x indicates point x on the mask pattern mask M The location of the projection.
[0057] Bring in The projection pattern S can be obtained P Function:
[0058]
[0059] in, represents the unwrapped phase of the projected pattern, Indicates the initial phase of the projected pattern.
[0060] At the same time, at the image plane sampling position nT S Projection pattern S P With aliasing pattern S A The points are equal, that is
[0061] S P (nT S )=S A (nT S )
[0062] T S represents the CCD sampling period, and n represents the sampling period.
[0063] Available Therefore, the aliasing pattern function S AIt can be written as:
[0064]
[0065] in, represents the unwrapped phase of the aliasing pattern, is the initial phase of the aliasing pattern.
[0066] The combined phase of the aliasing pattern can also be written as For S A At the position x of the foot of the light source L The unwrapped phase, that is, x = x L When , we can get:
[0067]
[0068] Because of w e As the internal parameter of the system, it can be clearly seen that during the movement of the light source, the unfolded phase of the perpendicular foot position remains unchanged, so the reverse positioning of the light source marker can be achieved.
[0069] Here, the data acquired by the camera is an aliasing pattern S A The image I needs to be converted into pixels, pixel circular frequency ω A for:
[0070] ω A =w A T S =2πT S / T A =2π / τ A
[0071] Among them, T A represents the period of the aliasing pattern, T S Indicates the CCD sampling period.
[0072] where τ A is the aliasing pattern S A The pixel period of the image I function can be written as:
[0073]
[0074] Where u represents the pixel coordinates of a point on the aliasing pattern.
[0075] Positioning method:
[0076] Based on the property of the unwrapped phase invariance at the foot of the light source marker, the aliased image S on the image I can be estimated A The pixel period τ A Then the combined phase in the localization image I is However, when tracking the phase, the periodic confusion problem needs to be solved.
[0077] Solve the period confusion problem by period labeling:
[0078] 1. In the mask image S M Mark any phase zero point on the image, making it a control point that can be located in image I, that is, tracking the projected image S p The mth phase zero point of ;
[0079] 2. When positioning, calculate the sub-pixel position u of the control point c =n+k, n is the control point on the right side of the nth pixel, and the periodic coefficient k∈[0,1);
[0080] 3. Based on the aliasing pattern S on image I A The pixel period τ A ;
[0081] 4. Find the sub-pixel distance u in the image c -kτ A The nearest aliasing pattern S A The phase zero point is marked with nm and extends to the left and right.
[0082] Positioning phase The pixel coordinates u of the foot of the light source can be obtained by using the period L ,
[0083] u L =u / (1-τ A / τ e );
[0084] Among them, u represents the combined phase The pixel position of the pixel, τ e Represents the pixel period of the system intrinsic parameter.
[0085] However, in this positioning method, the value of m is unknown, and it is affected by the assembly and is usually difficult to control, so the calculated phase difference is 2mπ from the actual phase. m can be obtained through internal calibration.
[0086] Internal reference calibration:
[0087] Based on the property of the unwrapped phase invariance at the foot of the light source marker, it is converted into an image:
[0088]
[0089] in, Represents the initial phase of the aliased image, ω e is the installation error, the difference between the sampling frequency and the mask frequency w e With relevant links, specifice =w e T S , is one of the internal parameters of the system
[0090] By moving the light source at equal intervals, N light source positions are collected. Since the phase difference between the positioning method and the actual phase is 2mπ, we can get:
[0091]
[0092] Among them, ω n Represents ω at each light source position A ,u 1 represents the initial pixel coordinate position of the light source, Δu represents the pixel coordinate moving step of the light source, Indicates that each light source position is the initial phase on the image, The optimization algorithm is used to calibrate the unknowns: cycle number, phase, installation error, i.e. internal parameters.
[0093] Consider the optimization problem and write the formula as
[0094]
[0095] Since there is a nonlinear term (ω n -ω e )(u 1 +nΔu), an alternating optimization method can be used.
[0096] Because ω e Does not change with the movement of the light source, so first assume that ω e Known, optimize other parameters.
[0097]
[0098] Among them, a n =ω n -ω e is the circular frequency of the aliasing pattern at each light source position after error removal, ψ M Indicates the actual phase.
[0099] Take the first-order derivative of the optimization object and find the extreme point
[0100]
[0101]
[0102] Converted into matrix form, we get:
[0103]
[0104] The inverse solution can be used to obtain the initial pixel coordinate position u of the light source in the least squares optimal sense. 1 , the pixel coordinate movement step of the light source Δ u , the actual phase ψ M ,because So the obtained ψ M The integer number of cycles in the middle can be removed to find the phase zero point number corresponding to the control point, that is, the cycle number m, and the phase
[0105] Optimization e
[0106]
[0107] Among them, c n represents auxiliary parameters, b n Indicates the pixel position of the current light source.
[0108] Derivative the optimization object to find the extreme point
[0109]
[0110] Transformation can be obtained
[0111]
[0112] The least squares most meaningful ω can be obtained e
[0113] Example
[0114] Based on the spatial point coordinate estimation based on vision, referred to as visual positioning, the present invention proposes a novel lens-free light source positioning system. The system consists of a point light source, a grating mask and an image sensor. The light source is the target to be positioned. Similar to the reference mark, the light source can be attached to the object of interest for tracking, and the positioning and displacement measurement of the object of interest can be indirectly determined by positioning the light source. The frequency of the mask pattern and the sampling frequency of the image sensor are aliased to produce moiré. The tiny displacement of the point light source will convert the low-frequency macroscopic changes of the micro-moiré fringes. By analyzing the changes in the moiré fringes, the position of the light source can be inferred. Due to the amplification effect of the moiré fringes, this point-to-pattern mapping relationship can exceed the upper limit of the spatial resolution of the one-dimensional camera.
[0115] Analysis of the changes in moiré fringes
[0116] Due to the amplification effect of the moiré effect, there may be arbitrary period confusion based only on the changes between the taken pictures. Therefore, it is necessary to first improve the period confusion and label the period of the moiré pattern. The label only indicates the difference between the periods to distinguish the phase differences, but the label differs from the actual period by a fixed value.
[0117] For which w e The frequency difference between the mask pattern and the sampling frequency of the image sensor is also a system internal parameter and needs to be found through calibration.
[0118] The real period and the system internal parameters are calibrated by the optimization algorithm based on the least squares algorithm. Optimization algorithm: First, based on the property that the unfolded phase at the foot of the light source marker is invariant, by using this property, the corresponding equation can be obtained by displacing the light source at equal intervals. As long as the equation is greater than the number of unknowns, the unknowns can be found. Then, the optimization algorithm is used to calibrate the unknowns under the optimal result.
[0119] Finally, a calibration result or positioning result is obtained. The solution has a final result output and can solve specific technical problems.
[0120] Advantages and uses of this product.
[0121] The present invention has developed a lensless light source marking positioning system and its calibration method, which uses a lensless camera (CCD sensor) in conjunction with a mask plate, uses a light-emitting element to mark the target object, avoids the ambient light band and allows the light emitted by the target point to dominate the imaging process. The object movement is detected by measuring the displacement of the point light source, and the moiré acceleration is introduced to amplify the target movement and ensure the accuracy of the positioning result. The pixel coordinate initial position u of the light source is obtained by inversely analyzing the motion of the moiré pattern. 1 , the pixel coordinate movement step of the light source Δ u The position of the point light source is deduced through the triangle similarity principle and the internal parameter control point of the system corresponds to the phase zero point, that is, the period number m. Installation errorω e Calibration is performed. The present invention utilizes the aliasing of the sampling frequency and the mask projection frequency to form an interference pattern on the imaging surface, generating a Moore's acceleration phenomenon associated with the target displacement, thereby amplifying the target's movement. Based on the property that the unfolded phase at the foot of the light source mark remains unchanged, the light source mark positioning can be reversed, and the system internal parameters are solved by an optimization algorithm. The positioning process is innovative and is not constrained by the limitations of traditional spatial resolution. The principle of generating the Moore's acceleration phenomenon is also innovative, requiring only the target point as a projection light source, without the need to strictly control the entire workspace, and with high positioning accuracy, it is suitable for most applications of visual positioning.
[0122] The process or manner in which a product is used.
[0123] 1. Place a black and white stripe cycle (T M ) mask and set control points, you can observe the distinguished points on the image plane, fix the point light source on the object to be measured. Move the light source and use CCD to capture the moiré pattern at the corresponding position.
[0124] 2. Generate the corresponding grayscale change curve based on the image taken by the camera
[0125] 3. Perform periodic labeling and calculate the internal parameters to achieve the positioning and calibration of the light source marker.
[0126] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A lensless light source marking positioning system, characterized in that: include: Light source, grating mask and image sensor; The light source is at a distance from the object to be measured, and the grating mask is arranged between the light source and the image sensor, wherein the frequency of the grating mask and the sampling frequency of the image sensor are aliased.
2. The calibration method of the positioning system according to claim 1, characterized in that: include: The light source is set on the object to be measured at the distance, and the light source is photographed by an image sensor. Due to aliasing between the frequency of the grating mask and the sampling frequency of the image sensor, an aliasing image of the aliasing pattern is obtained. According to the aliasing image, an aliasing image function is obtained, and the period of the aliasing image is initially labeled. An optimization function is constructed according to the initial label and the aliasing image function. The optimization function is calculated by the least squares method to obtain the actual period label and the camera intrinsic parameters to realize intrinsic parameter calibration, and the position of the light source is obtained based on the actual period and the camera intrinsic parameters.
3. The method according to claim 2, characterized in that The aliased image function S A for: Among them, F is the period of 2π, represents the unwrapped phase of the aliasing pattern, w A The circular frequency of the aliasing pattern, x represents the pixel position in the image plane, Indicates the initial phase of the projected pattern.
4. The method according to claim 2, characterized in that: The circular frequency w of the aliasing pattern A for: Among them, d L is the distance from the light source to the mask, h is the distance from the image plane to the mask, and w M is the mask frequency, w e is the mask frequency w M and the difference between the CCD sampling frequency.
5. The method according to claim 4, characterized in that The aliased image function I is: Where F is the period of 2π, u represents the pixel coordinates of the point on the aliasing pattern, ω A is the pixel circular frequency, is the phase of the aliasing pattern, where is the initial phase of the aliasing pattern, 6. The method according to claim 5, characterized in that The pixel circular frequency is: oh A =w A T S =2πT S / T A =2π / τ A Among them, T A represents the period of the aliasing pattern, T S represents the CCD sampling period, τ A is the aliasing pattern S A The pixel period.
7. The method according to claim 1, characterized in that The optimization function is: Among them, ω n represents the circular frequency of the aliasing pattern at each light source position when the light source position is moved at equal intervals, ω e represents the camera internal parameter error caused by the installation error, u1 represents the initial position of the pixel coordinate of the light source, Δu represents the moving step of the pixel coordinate of the light source, represents the initial phase at each light source position, ψ M Indicates the actual phase.
8. The method according to claim 2, characterized in that: The process of initially labeling the period of the aliased image includes: Any phase zero point in the mask image is marked as a control point of the aliased image, that is, the tracking projection image S p The mth phase zero point of , where m is the cycle number.
Citation Information
Patent Citations
Image point tracing-based object raster image phase shift method phase measurement error correction method
CN110230994A
Mask base plate, mask plate and photoetching equipment
CN115185153A
Orthogonal multi-projection aliasing image non-separation structured light three-dimensional measurement method
CN116429014A
Embedded Moore acceleration light source mark translation measurement method
CN118565352A
Semiconductor device and optical device using the same
EP1396891A2
Cited By
Point positioning method based on 3-D Moire effect
CN119826690A
