Point positioning method based on 3-D Moire effect

Abstract

The invention provides a point positioning method based on a 3-D Moire effect, which adopts a point positioning system based on the 3-D Moire effect, carries out accurate spatial point positioning by utilizing the 3-D Moire effect brought by shadows generated by series masks fixed in front of a naked camera sensor, and carries out illumination by a beacon marked on a target point. The beacon is a target point needing to be positioned; according to the method, a positioning spectrum method is adopted to deduce a geometrical relationship between a beacon and a Moire pattern; a positioning model of the positioning spectrum method is considered to be a pinhole model with a periodic expansion range, and the frequency of a mask pattern determines an analogy focal length; the method is easy to implement, the algorithm is visual and simple, and the positioning resolution and precision are about one order of magnitude higher than those of a reference model (including a pinhole model and a single mask model).

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision and precision measurement, and in particular to a point positioning method based on 3-D Moiré effect. Background Art

[0002] Vision-based estimation of spatial point coordinates, or point localization, is the cornerstone of motion measurement in many fields, including industry, medicine, and entertainment. The number of spatial points that a localization method can identify is called resolution, which is an important indicator of the method's performance and is scale-independent.

[0003] Most point localization methods rely on the pinhole camera model, in which spatial points are projected along a line through the optical center. This introduces an upper bound: the resolution of a pinhole camera-based method will not be greater than the number of image points it can resolve. Within this limit, there are two ways to improve. First, by adding markers. The target is marked with a marker such as a cross, circle, or ball. Localizing the projected target with the help of markers can use more information to resolve uncertainty. Second, by using multiple views. The target is observed from different angles, the projected target in each view is located independently, and the results are then fused to minimize the reprojection error. Although both methods can improve the resolution with sub-pixel accuracy, they are still limited by upper bounds. Specifically, their upper bound is the product of the number of pixels and the square of the sub-pixel resolution. Compared with the full potential of the overall value of the image, that is, the number of intensity levels raised to the power of the number of pixels, there is still a lot of room for improvement in resolution. Therefore, the potential of point localization is still largely untapped.

[0004] The upper bound is derived from the pinhole camera model used for imaging. If imaging is not required, it is possible to break through this upper bound and fully tap the potential of spatial point positioning. For this reason, positioning methods based on the 3-D Moiré effect have attracted widespread attention. This method aims to exploit the sensitivity of the 3-DMoiré pattern generated by the shadow of the tandem mask to its mask displacement. This is the basis of grating rulers and a milestone in the field of optical metrology. However, there are many problems in previous work. These methods lack a clear positioning model. The geometric relationship between the Moiré pattern and the measured value is established by enumeration or machine learning. They are only superior to the pinhole method in terms of view angle estimation, not in point positioning. Therefore, the present invention utilizes the 3-D Moiré effect caused by the shadow of the tandem mask fixed in front of the bare camera sensor, while being illuminated by a beacon marked on the target point. In addition, a new method called positioning spectrum is proposed, based on which the positioning model of the proposed method can be derived in an easier way than the existing 3-DMoiré method. Summary of the invention

[0005] The present invention proposes a point positioning method based on the 3-DMoiré effect, which is a spatial point positioning method with low cost, simple production, simple modeling and easy use. The method of the present invention embeds a complex 3-DMoiré effect into the interior of a camera, and a user only needs to fix a beacon on a target, which is easier than using a feature map, an imaging medium or a 3-DMoiré marker. The beacon can be in various forms, such as a point light source, a laser, a single-mode optical fiber, etc. The positioning model of the present invention is clear and concise, and can be interpreted as a pinhole model with a periodic expansion range, in which the frequency of the mask pattern determines the analog focal length, which facilitates the use of established projective geometry to design positioning and calibration algorithms, thereby making these algorithms simple and intuitive. In addition, the positioning resolution and accuracy of the present invention are both about an order of magnitude higher than those of the benchmark model (including the pinhole model and the single mask model).

[0006] The present invention adopts the following technical solutions.

[0007] A point positioning method based on 3-DMoiré effect, the method adopts a point positioning system based on 3-DMoiré effect, which uses the 3-DMoiré effect brought by the shadow produced by the series mask fixed in front of the naked camera sensor to accurately locate the spatial point, and is illuminated by a beacon marked on the target point; the beacon is the target point to be located; the method adopts a positioning spectrum method to derive the geometric relationship between the beacon and the Moiré pattern; the positioning model of the positioning spectrum method is regarded as a pinhole model with a periodic expansion range, and the frequency of the mask pattern determines the analog focal length;

[0008] The steps include:

[0009] Step 1: Obtain the Moiré image S required for calibration through the point positioning system;

[0010] Step 2: Use Moiré image to perform phase unwrapping calibration to obtain unwrapped phases m and n;

[0011] Step 3, calculate the image point h in the image;

[0012] Step 4: Use h to calibrate the system internal parameters to obtain the system internal parameters f, a, b;

[0013] Step 5: Position the spatial point.

[0014] The beacon may be in the form of a point light source, a laser, or a single-mode optical fiber.

[0015] The positioning spectrum method includes a calibration algorithm for phase unwrapping, which is used to accurately locate the phase ambiguity of the decoded periodic pattern; the positioning model of the positioning spectrum method includes a calibration algorithm, which is used to obtain the internal parameters of the system for accurate positioning, specifically calibrating the phase unwrapping of the 3-D Moiré through projective geometry, thereby calibrating the system internal parameters;

[0016] The calibration algorithm includes three steps: image processing, quadratic surface fitting estimation of periodic phase and least squares method;

[0017] In the image processing step, the Moiré image is low-pass filtered to smooth the image to reduce signal noise, and the image is histogram equalized to increase the image contrast to improve the image signal-to-noise ratio;

[0018] In the step of estimating the periodic phase by quadratic surface fitting, a complete cycle of the 3-D Moiré signal is regarded as a quadratic surface. The sub-pixel position of the peak position of the signal in the image is extracted by quadratic surface fitting, and the period u, v of the Moiré and the corresponding phase are estimated. and ψ;

[0019] In the least squares step, the basic model for solving periodic ambiguity is derived through projective geometry and positioning spectrum. In this model, the unknowns of the unwrapped phase that need to be solved are fixed and the number does not change, and they are solved by the least squares method.

[0020] The pinhole model with a periodic expansion range uses a checkerboard calibration algorithm to calibrate the system. Specifically, based on the camera imaging principle, calibration images are taken at different angles, and the internal and external parameters of the system are calculated using the feature points in the image to improve accuracy and expand the scope of application.

[0021] When taking an image, the beacon is moved in space, and the moving trajectory is a perfect checkerboard array, which replaces the checkerboard calibration plate of the traditional checkerboard calibration algorithm.

[0022] The point positioning system includes a bare camera sensor and two masks etched on opposite sides of a quartz glass substrate; the quartz glass substrate is pasted on the sensor cover glass, the mask and the bare camera sensor are parallel to each other, the thickness of the cover glass and the substrate are h and s respectively; the transmittance of the mask is recorded as T i , where i = 1, 2, representing the front and back sides of the mask, respectively;

[0023] T i Designed as a checkerboard pattern, the simplified form is T i (x) = F(Φ i (x)), where F is a two-dimensional cosine function with a period of (1, 1);

[0024] Align the mask's origin and orientation and define the camera origin O at the perpendicular projection of its origin C , thus obtaining the unwrapped phase of the mask, which can be expressed as:

[0025] Φ1(x)=(u T x,v T x) T

[0026] Φ2(x)=(1+α)Φ1(x)

[0027] where (1+α)∈[1,2] represents the frequency scaling of the two masks; u and v are not necessarily aligned with the X and Y axes;

[0028] Etch a control point at the origin of T1, denoted as g = (0, 0, h + s) T , used to resolve the periodic blurring of the moiré pattern;

[0029] The image coordinate system O is defined at the upper left of the sensor I , in O C -(0, 0, t) T A virtual coordinate system O is defined at G ; This coordinate system is used to describe the system as a pinhole camera coordinate system; C , O I and O G The X and Y axes are parallel to the edge of the sensor;

[0030] Beacon in O C The coordinates in are expressed as P = (p T , z) T , where p represents the coordinate in the plane, z represents the depth; the distance from P to the front mask is recorded as d=zhs;

[0031] T i The projection pattern on the sensor is projected by the beacon and is denoted as The composite image T and Moiré pattern S are expressed as

[0032]

[0033] Where LP{..} is a low-pass filter, and the image of the Moiré pattern is I=Img(S), where Img{..} involves sampling, quantization, and blurring operations;

[0034] The positioning model is expressed as S(x) = F(Φ(x)) and

[0035] where β is inversely proportional to the period length of S; φ and Φ are the initial phases in the interval [0, 1).

[0036] The optical path connecting P and p In the positioning spectrum on The projection of is an orthogonal transformation and is not affected by refraction; then The projection spectrum on is S ∞ (x) = LP{T1(x)T2(x)}. According to the product and identity, Φ ∞ (x) = Φ2(x)-Φ1(x); therefore, Φ ∞ (x) = αΦ1(x); and because F(Φ(p)) = F(Φ ∞ (p)), so there is,

[0037]

[0038] (β-α)v T p+(ψ-n)=0

[0039] Define h as the (m, n) peak of S, that is

[0040]

[0041] βv T h+ψ=n

[0042] And combine it with the above formula. We have,

[0043]

[0044] The in-plane coordinate p of the beacon is bound to a characteristic peak of the moiré pattern S; h is regarded as the "image point" of p;

[0045] The calibration algorithm of phase unwrapping is as follows: The direct relationship between light and pattern is established through the control point g:

[0046]

[0047] Also because ,have:

[0048]

[0049] (β-λ * )v T c+(ψ-n c )=0,

[0050] where λ * =(1+α)(1-λ) is an internal parameter obtained by calibration; period m c and n c Solution,

[0051] Get

[0052] Where RoundP..} represents the rounding function; m c and n c are integers, β, λ * ,u,v,c, and ψ are values ​​that do not require high precision;

[0053] The calibration algorithm of the positioning model is as follows: what is read from I is O J Pixel coordinates in the coordinate system and The relationship between them is Where w is the pixel size, (a, b) T O C The pixel coordinates in the coordinate system. Therefore,

[0054]

[0055] This positioning model is equivalent to an optical center at (0, 0, γ*) T , the pinhole model with a focal length of f is regarded as a virtual camera fixed on the system, and its coordinate system is O G =O C -(0, 0, γ*) T , projective geometry techniques are used to calibrate the ghost camera and solve P through PnP or binocular triangulation.

[0056] In point positioning systems, such as Figure 1 As shown, a photolithography glass plate (mask) with masks on both sides is attached to the bare camera sensor (sensor in the figure); the side close to the bare camera sensor is marked as T1, and the other side is marked as T2; the beacon P is projected on the sensor to generate Therefore, the Moiré image S is generated by the Moiré effect; the movement of S can be observed from the movement of the beacon P, and the spatial position of the beacon P can be located based on the movement of S, so P is also called the target point;

[0057] In the positioning spectrum method, the hovering theorem accompanying the positioning spectrum is used; specifically: Figure 3 In , a mask-based visual measurement system is used, which regards the mask as N transmission planes. When there are two mask plates, the transmittance function is recorded as , where F is a two-dimensional cosine function;

[0058] Each transmission plane is divided into points (p, p z ) T =(p x , p y , p z ) T The transmittance projection of the projection center on the XY plane is recorded as The comprehensive transmittance T and the moiré pattern S formed thereon are:

[0059]

[0060] As shown in the figure, given a point q on S, the light path from P to q is recorded as Place P along Push to infinity, that is, P ∞ =(p,∞) T , that is, the projective transformation becomes a parallel transformation; the transformed transmittance projection is recorded as Assuming that the optical path from P to q remains unchanged, we have T and There is the following relationship

[0061]

[0062] Then, assuming that the light path nearby The same is true under projective transformation and parallel transformation.

[0063]

[0064] in, It is called the positioning spectrum, q is called the hovering point, the transmittance of the hovering point is called the hovering value, and the above relationship is called the hovering theorem.

[0065] In step 2, the Moiré image is used for phase unwrapping calibration to obtain the unwrapped phases m and n, as Figure 4 As shown, the following steps are included:

[0066] Step A1: Low-pass filter the Moiré image to smooth the image and reduce the signal noise, and perform histogram equalization on the image to improve the image contrast and signal-to-noise ratio, and detect the image coordinates c of the control point; Step A2: Use quadratic surface fitting to extract the sub-pixel position of the signal peak in the image, and estimate the Moiré period u, v and the corresponding phase and ψ;

[0067] Step A3: Calibrate λ by least squares method * , in order to calibrate λ * Attach a beacon to the one-dimensional grating ruler and move the beacon to capture a series of have:

[0068]

[0069] (β j -λ * )v T c j +(ψj -n j )=0

[0070] If m j and n j The absolute value of cannot be determined, so the image is taken continuously to monitor its increase or decrease; therefore, there is m j =m0+Δm j and n j =n0+Δn j , where m0 and n0 are unknown, Δm j and Δn j known;

[0071] In matrix form, we have,

[0072]

[0073] When there are sufficient samples, λ* is solved by the least squares algorithm.

[0074] Step A4: Using the formula Perform the least squares method to solve for m and n.

[0075] The pinhole model with periodic expansion range is as follows: Figure 6 As shown, the periodic expansion is Figure 7 as shown; Figure 6 As shown, it is regarded as a virtual camera fixed on the system, which can be expressed as

[0076] O G =O C -(0, 0, γ * ) T ,

[0077] The virtual imaging plane is on the virtual camera sensor plane O G Compared with the actual system camera sensor plane O C Between the virtual imaging plane and O G The distance is called the focal length f of the pinhole model; then, the projective geometry technique can be used to calibrate the virtual camera and solve P through PnP or binocular triangulation; specifically:

[0078] Given a WxW sensor and the moving range of P (beacon), in order to optimize the localization resolution, the camera in the pinhole model needs to cover this range through its field of view, such as Figure 7 As shown in part a of FIG. 3-D Moiré model, a large f is concentrated in a small range.

[0079] like Figure 8 As shown in the figure, the peak point of the blue dot Moiré pattern is

[0080] If the obtained Moiré pattern is periodic, its period is extended by solving the phase ambiguity, and then its extended range is used to cover other ranges, see Figure 7 Part b of the resolution limit can improve R = (Wex / W) 2 , where Wex is the side length of the extended range.

[0081] In view of the shortcomings in the prior art, the present invention provides a spatial point positioning method that is low in cost, simple to make, and simple to model and easy to use. The method embeds the complex 3-D Moiré effect into the camera. The user only needs to fix the beacon on the target, which is easier than using a feature map, an imaging medium, or a 3-DMoiré marker. The beacon can be in various forms, such as a point light source, a laser, a single-mode optical fiber, etc. Most of the strategies mentioned in previous work lack a clearly defined positioning model, while the positioning model of the present invention is clear and concise. It can be interpreted as a pinhole model with a periodic expansion range, in which the frequency of the mask pattern determines the analog focal length. This facilitates the design of positioning and calibration algorithms using established projective geometry, making these algorithms simple and intuitive. In addition, the positioning resolution and accuracy of the present invention are about one order of magnitude higher than the benchmark model (including the pinhole model and the single mask model).

[0082] The present invention provides a point positioning system based on the 3-D Moiré effect, including: proposing a new method called positioning spectrum, based on which the basic model of the present invention can be obtained more concisely and clearly; a phase unwrapping calibration algorithm for decoding the phase ambiguity of periodic patterns for precise positioning; and a positioning model calibration algorithm for obtaining the internal parameters of the system for precise positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0084] Attached Figure 1 is a schematic structural diagram of a point positioning method based on a 3-D Moiré effect provided in an embodiment of the present invention;

[0085] Attached Figure 2 is a schematic flow chart of a point positioning method based on a 3-D Moiré effect provided in an embodiment of the present invention;

[0086] Attached Figure 3 is a schematic diagram of a mask-based visual measurement system provided and described in an embodiment of the present invention;

[0087] Attached Figure 4 is a schematic diagram of a process of unwrapping phase calibration provided by an embodiment of the present invention;

[0088] Attached Figure 5 It is a schematic diagram of a flow chart of a calibration algorithm of a positioning model provided in an embodiment of the present invention;

[0089] Attached Figure 6 It is a schematic diagram of a pinhole model provided by an embodiment of the present invention;

[0090] Attached Figure 7 is a schematic diagram of a pinhole model with a periodic expansion range provided by an embodiment of the present invention;

[0091] Attached Figure 8 It is a schematic diagram of periodically expanding the Moiré pattern obtained by the system provided in the embodiment of the present invention. DETAILED DESCRIPTION

[0092] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present invention. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiment of the present disclosure. However, it is obvious that one or more embodiments may also be implemented without these specific details. In addition, in the following description, the description of known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present disclosure.

[0093] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0094] Moreover, the shapes and sizes of the components in the figures do not reflect the real size and proportion, but only illustrate the contents of the embodiments of the present disclosure. In addition, in the claims, any reference symbols between brackets shall not be constructed as limiting the claims.

[0095] In addition, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of the present invention.

[0096] As shown in the figure, a point positioning method based on the 3-D Moiré effect is used in the method. The point positioning system based on the 3-D Moiré effect is used to accurately position the space point by using the 3-DMoiré effect caused by the shadow generated by the series mask fixed in front of the naked camera sensor, and illuminated by the beacon marked on the target point; the beacon is the target point to be positioned; the method uses the positioning spectrum method to derive the geometric relationship between the beacon and the Moiré pattern; the positioning model of the positioning spectrum method is regarded as a pinhole model with a periodic expansion range, and the frequency of the mask pattern determines the analog focal length;

[0097] The steps include:

[0098] Step 1: Obtain the Moiré image S required for calibration through the point positioning system;

[0099] Step 2: Use Moiré image to perform phase unwrapping calibration to obtain unwrapped phases m and n;

[0100] Step 3, calculate the image point h in the image;

[0101] Step 4: Use h to calibrate the system internal parameters to obtain the system internal parameters f, a, b;

[0102] Step 5: Position the spatial point.

[0103] The beacon may be in the form of a point light source, a laser, or a single-mode optical fiber.

[0104] The positioning spectrum method includes a calibration algorithm for phase unwrapping, which is used to accurately locate the phase ambiguity of the decoded periodic pattern; the positioning model of the positioning spectrum method includes a calibration algorithm, which is used to obtain the internal parameters of the system for accurate positioning, specifically calibrating the phase unwrapping of the 3-D Moiré through projective geometry, thereby calibrating the system internal parameters;

[0105] The calibration algorithm includes three steps: image processing, quadratic surface fitting estimation of periodic phase and least squares method;

[0106] In the image processing step, the Moiré image is low-pass filtered to smooth the image to reduce signal noise, and the image is histogram equalized to increase the image contrast to improve the image signal-to-noise ratio;

[0107] In the step of estimating the periodic phase by quadratic surface fitting, a complete cycle of the 3-D Moiré signal is regarded as a quadratic surface. The sub-pixel position of the peak position of the signal in the image is extracted by quadratic surface fitting, and the period u, v of the Moiré and the corresponding phase are estimated. and ψ;

[0108] In the least squares step, the basic model for solving periodic ambiguity is derived through projective geometry and positioning spectrum. In this model, the unknowns of the unwrapped phase that need to be solved are fixed and the number does not change, and they are solved by the least squares method.

[0109] The pinhole model with a periodic expansion range uses a checkerboard calibration algorithm to calibrate the system. Specifically, based on the camera imaging principle, calibration images are taken at different angles, and the internal and external parameters of the system are calculated using the feature points in the image to improve accuracy and expand the scope of application.

[0110] When taking an image, the beacon is moved in space, and the moving trajectory is a perfect checkerboard array, which replaces the checkerboard calibration plate of the traditional checkerboard calibration algorithm.

[0111] The point positioning system includes a bare camera sensor and two masks etched on opposite sides of a quartz glass substrate; the quartz glass substrate is pasted on the sensor cover glass, the mask and the bare camera sensor are parallel to each other, the thickness of the cover glass and the substrate are h and s respectively; the transmittance of the mask is recorded as T i , where i = 1, 2, representing the front and back sides of the mask, respectively;

[0112] T i Designed as a checkerboard pattern, the simplified form is T i (x) = F(Φ i (x)), where F is a two-dimensional cosine function with a period of (1, 1);

[0113] Align the mask's origin and orientation and define the camera origin O at the perpendicular projection of its origin C , thus obtaining the unwrapped phase of the mask, which can be expressed as:

[0114] Φ1(x)=(u T x,v T x) T

[0115] Φ2(x)=(1+α)Φ1(x)

[0116] Where (1+α)∈[1,2] represents the frequency scaling of the two masks; u and v are not necessarily aligned with the X and Y axes; a control point is etched at the origin of T1, denoted as g=(0,0,h+s) T , used to resolve the periodic blurring of the moiré pattern;

[0117] The image coordinate system O is defined at the upper left of the sensor I , in O C -(0, 0, t) T A virtual coordinate system O is defined at G; This coordinate system is used to describe the system as a pinhole camera coordinate system; C , O I and O G The X and Y axes are parallel to the edge of the sensor;

[0118] Beacon in O C The coordinates in are expressed as P = (p T , z) T , where p represents the coordinate in the plane, z represents the depth; the distance from P to the front mask is recorded as d=zhs;

[0119] T i The projection pattern on the sensor is projected by the beacon and is denoted as The composite image T and Moiré pattern S are expressed as

[0120]

[0121] Where LP{..} is a low-pass filter, and the image of the Moiré pattern is I=Img(S), where Img{..} involves sampling, quantization, and blurring operations;

[0122] The positioning model is expressed as S(x) = F(Φ(x)) and

[0123] where β is inversely proportional to the period length of S; φ and ψ are the initial phases in the interval [0, 1).

[0124] The optical path connecting P and p In the positioning spectrum on The projection of is an orthogonal transformation and is not affected by refraction; then The projection spectrum on is S ∞ (x) = LP{T1(x)T2(x)}. According to the product and identity, Φ ∞ (x) = Φ2(x)-Φ1(x); therefore, Φ ∞ (x) = αΦ1(x); and because F(Φ(p)) = F(Φ ∞ (p)), so there is,

[0125]

[0126] (β-α)v T p+(ψ-n)=0

[0127] Define h as the (m, n) peak of S, that is

[0128]

[0129] βv T h+ψ=n

[0130] And combine it with the above formula. We have,

[0131]

[0132] The in-plane coordinate p of the beacon is bound to a characteristic peak of the moiré pattern S; h is regarded as the "image point" of p;

[0133] The calibration algorithm of phase unwrapping is as follows: The direct relationship between light and pattern is established through the control point g:

[0134]

[0135] Also because ,have:

[0136]

[0137] (β-λ * )v T c+(ψ-n c )=0,

[0138] where λ * =(1+α)(1-λ) is an internal parameter obtained by calibration; period m c and n c Solve, and you get

[0139] Where Round{..} represents the rounding function; m c and n c are integers, β, λ * ,u,v,c, and ψ are values ​​that do not require high precision;

[0140] The calibration algorithm of the positioning model is as follows: what is read from I is O I Pixel coordinates in the coordinate system and The relationship between them is Where w is the pixel size, (a, b) T O C The pixel coordinates in the coordinate system. Therefore,

[0141]

[0142] This positioning model is equivalent to an optical center at (0, 0, γ * ) T , the pinhole model with a focal length of f is regarded as a virtual camera fixed on the system, and its coordinate system is O G =O C -(0, 0, γ* ) T , projective geometry techniques are used to calibrate the ghost camera and solve P through PnP or binocular triangulation.

[0143] In point positioning systems, such as Figure 1 As shown, a photolithography glass plate (mask) with masks on both sides is attached to the bare camera sensor (sensor in the figure); the side close to the bare camera sensor is marked as T1, and the other side is marked as T2; the beacon P is projected on the sensor to generate Therefore, the Moiré image S is generated by the Moiré effect; the movement of S can be observed from the movement of the beacon P, and the spatial position of the beacon P can be located based on the movement of S, so P is also called the target point;

[0144] In the positioning spectrum method, the hovering theorem accompanying the positioning spectrum is used; specifically: Figure 3 In , a mask-based visual measurement system is used, which regards the mask as N transmission planes. When there are two mask plates, the transmittance function is recorded as , where F is a two-dimensional cosine function;

[0145] Each transmission plane is divided into points (p, p z ) T =(p x , p y , p z ) T The transmittance projection of the projection center on the XY plane is recorded as The comprehensive transmittance T and the moiré pattern S formed thereon are:

[0146]

[0147] As shown in the figure, given a point q on S, the light path from P to q is recorded as Place P along Push to infinity, that is, P ∞ =(p,∞) T , that is, the projective transformation becomes a parallel transformation; the transformed transmittance projection is recorded as Assuming that the optical path from P to q remains unchanged, we have T and There is the following relationship

[0148] Then, assuming that the light path nearby The same is true under projective transformation and parallel transformation.

[0149] in, It is called the positioning spectrum, q is called the hovering point, the transmittance of the hovering point is called the hovering value, and the above relationship is called the hovering theorem.

[0150] In step 2, the Moiré image is used for phase unwrapping calibration to obtain the unwrapped phases m and n, as Figure 4 As shown, the following steps are included:

[0151] Step A1: Low-pass filter the Moiré image to smooth the image and reduce the signal noise, and perform histogram equalization on the image to improve the image contrast and signal-to-noise ratio, and detect the image coordinates c of the control point; Step A2: Use quadratic surface fitting to extract the sub-pixel position of the signal peak in the image, and estimate the Moiré period u, v and the corresponding phase and ψ;

[0152] Step A3: Calibrate λ by least squares method * , in order to calibrate λ * Attach a beacon to the one-dimensional grating ruler and move the beacon to capture a series of have:

[0153]

[0154] (β j -λ * )v T C j +(ψ j -n j )=0

[0155] If m j and n j The absolute value of cannot be determined, so the image is taken continuously to monitor its increase or decrease; therefore, there is m j =m0+Δm j and n j =n0+Δn j , where m0 and n0 are unknown, Δm j and Δn j known;

[0156] In matrix form, we have,

[0157]

[0158] When there are enough samples, the least squares algorithm is used to solve λ * .

[0159] Step A4: Using the formula Perform the least squares method to solve for m and n.

[0160] The pinhole model with periodic expansion range is as follows: Figure 6 As shown, the periodic expansion is Figure 7 as shown; Figure 6 As shown, it is regarded as a virtual camera fixed on the system, that is, it can be expressed as O G =O C -(0, 0, γ * ) T ,

[0161] The virtual imaging plane is on the virtual camera sensor plane O G Compared with the actual system camera sensor plane O C Between the virtual imaging plane and O G The distance is called the focal length f of the pinhole model; then, the projective geometry technique can be used to calibrate the virtual camera and solve P through PnP or binocular triangulation; specifically:

[0162] Given a W×W sensor and a moving range of P (beacon), in order to optimize the positioning resolution, the camera in the pinhole model needs to cover this range through its field of view, such as Figure 7 As shown in part a of FIG. 3-D Moiré model, a large f is concentrated in a small range.

[0163] like Figure 8 As shown in the figure, the peak point of the blue dot Moiré pattern is

[0164] If the obtained Moiré pattern is periodic, its period is extended by solving the phase ambiguity, and then its extended range is used to cover other ranges, see Figure 7 Part b of the resolution limit can improve R = (Wex / W) 2 , where Wex is the side length of the extended range.

[0165] Example:

[0166] Point positioning methods based on 3-D Moiré effect, such as Figures 1 to 8 As shown, the following steps are included:

[0167] Acquire the Moiré image required for calibration through the system;

[0168] The Moiré image is used for phase unwrapping calibration to obtain the unwrapped phases m and n;

[0169] Calculate the image point h in the image;

[0170] Use h to calibrate the system internal parameters to obtain the system internal parameters f, a, b;

[0171] Perform spatial point positioning;

[0172] The system proposed in this example consists of a bare camera sensor and two masks etched on opposite sides of a quartz glass substrate, which is then glued to the sensor cover glass.

[0173] The mask and sensor are parallel to each other, and the thickness of the cover glass and substrate are h and s respectively. The transmittance of the mask is denoted by T i , where i = 1, 2, representing the front and back sides of the mask respectively. i Designed as a checkerboard pattern, it can be simplified to T i (x) = F(Φ i (x)), where F is a two-dimensional cosine function with period (1, 1). Align the mask origin and direction and define the camera origin O at the perpendicular projection of its origin C , thus obtaining the unwrapped phase of the mask:

[0174] Φ1(x)=(u T x,v T x) T

[0175] Φ2(x)=(1+α)Φ1(x)

[0176] Where (1+α)∈[1,2] represents the frequency scaling of the two masks. It is worth noting that u and v are not necessarily aligned with the X and Y axes. Etch a control point at the origin of T1, denoted as g=(0,0,h+s) T , which is used to resolve the periodic blur of the moiré pattern. The image coordinate system O is defined at the upper left of the sensor. I , in O C -(0, 0, t) T A virtual coordinate system O is defined at G This coordinate system is used to describe the system as a pinhole camera coordinate system. C , O I and O G The XY axes are parallel to the edge of the sensor.

[0177] Beacon in O C The coordinates in are expressed as P = (p T , z) T , where p represents the coordinate in the plane and z represents the depth. The distance from P to the front mask is d=zhs. T i The projection pattern on the sensor is projected by the beacon and is denoted as Their composite graph T and Moiré pattern S are

[0178]

[0179] Where LP{..} is a low-pass filter, the image of the Moiré pattern is I=Img(S), where Img{…} involves operations such as sampling, quantization, and blurring.

[0180] Therefore, the model of the present invention can be written as S(x)=F(Φ(x)) and

[0181]

[0182] where β is inversely proportional to the period length of S. φ and ψ are initial phases in the interval [0, 1).

[0183] The present invention studies the optical path connecting P and p Along the positioning spectrum. The projection of is an orthogonal transformation and is not affected by refraction. Therefore, The projection spectrum on is S ∞ (x) = LP{T1(x)T2(x)}. According to the product and identity, we have Φ ∞ (x) = Φ2(x) - Φ1(x). Therefore,

[0184] Φ ∞ (x) = αΦ1(x)

[0185] And because F(Φ(p))=F(Φ ∞ (p)), so there is,

[0186]

[0187] (β-α)v T p+(ψ-n)=0

[0188] Define h as the (m, n) peak of S, that is

[0189]

[0190] βv T h+ψ=n

[0191] And combine it with the above formula. We have,

[0192]

[0193] This example demonstrates an interesting and neat fact: the in-plane coordinate p of the beacon is bound to a characteristic peak of the moiré pattern S. Therefore, h is considered to be the “image point” of p.

[0194] Phase unwrapping calibration algorithm: The direct relationship between light and pattern is established through the control point g:

[0195]

[0196] Also because ,have:

[0197]

[0198]

[0199] where λ * =(1+α)(1-λ) is an internal parameter that can be obtained through calibration. Period m c and n c It can be solved, that is

[0200]

[0201] Where RoundP..} represents the rounding function. c and n c are integers, β, λ * ,u,v,c, and ψ do not need to be very precise.

[0202] Calibration algorithm of positioning model: The present invention can read out from I is O I Pixel coordinates in the coordinate system and , instead of O C The physical coordinates h and c in the coordinate system. The relationship between them is Where w is the pixel size, (a, b) T O C The pixel coordinates in the coordinate system. Therefore,

[0203]

[0204] This means that the proposed positioning model is equivalent to a light center at (0, 0, γ * ) T , a pinhole model with a focal length of f. We can imagine a virtual camera fixed on this system, whose coordinate system is O G =O C -(0, 0, γ * ) T Then, the ghost camera can be calibrated and P can be solved by PnP or binocular triangulation using mature projective geometry techniques.

[0205] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. All technical solutions belonging to the principle of the present invention belong to the protection scope of the present invention. For those skilled in the art, several improvements made without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A point positioning method based on 3-DMoiré effect, characterized in that: The method adopts a point positioning system based on the 3-DMoiré effect, which uses the 3-DMoiré effect caused by the shadow produced by the series mask fixed in front of the naked camera sensor to accurately locate the spatial point, and is illuminated by a beacon marked on the target point; the beacon is the target point to be located; the method adopts a positioning spectrum method to derive the geometric relationship between the beacon and the Moiré pattern; the positioning model of the positioning spectrum method is regarded as a pinhole model with a periodic expansion range, and the frequency of its mask pattern determines the analog focal length; The steps include: Step 1: Obtain the Moiré image S required for calibration through the point positioning system; Step 2: Use Moiré image to perform phase unwrapping calibration to obtain unwrapped phases m and n; Step 3, calculate the image point h in the image; Step 4: Use h to calibrate the system internal parameters to obtain the system internal parameters f, a, b; Step 5: Position the spatial point.

2. A point positioning method based on 3-DMoiré effect according to claim 1, characterized in that: The beacon may be in the form of a point light source, a laser, or a single-mode optical fiber.

3. The point positioning method based on 3-DMoiré effect according to claim 1, characterized in that: The positioning spectrum method includes a phase unwrapping calibration algorithm, which is used to accurately locate the phase ambiguity of the decoded periodic pattern; the positioning model of the positioning spectrum method includes a calibration algorithm, which is used to obtain the internal parameters of the system for accurate positioning, specifically calibrating the phase unwrapping of 3-DMoiré through projective geometry, thereby calibrating the system internal parameters; The calibration algorithm includes three steps: image processing, quadratic surface fitting estimation of periodic phase and least squares method; In the image processing step, the Moiré image is low-pass filtered to smooth the image to reduce signal noise, and the image is histogram equalized to increase the image contrast to improve the image signal-to-noise ratio; In the step of estimating the periodic phase by quadratic surface fitting, a complete cycle of the 3-DMoiré signal is regarded as a quadratic surface. The sub-pixel position of the peak position of the signal in the image is extracted by quadratic surface fitting, and the Moiré period u, v and the corresponding phase are estimated. and ψ; In the least squares step, the basic model for solving periodic ambiguity is derived through projective geometry and positioning spectrum. In this model, the unknowns of the unwrapped phase that need to be solved are fixed and the number does not change, and they are solved by the least squares method.

4. The point positioning method based on 3-DMoiré effect according to claim 3, characterized in that: The pinhole model with a periodic expansion range uses a checkerboard calibration algorithm to calibrate the system. Specifically, based on the camera imaging principle, calibration images are taken at different angles, and the internal and external parameters of the system are calculated using the feature points in the image to improve accuracy and expand the scope of application.

5. The point positioning method based on 3-DMoiré effect according to claim 4, characterized in that: When taking an image, the beacon is moved in space, and the moving trajectory is a perfect checkerboard array, which replaces the checkerboard calibration plate of the traditional checkerboard calibration algorithm.

6. The point positioning method based on 3-DMoiré effect according to claim 1, characterized in that: The point positioning system includes a bare camera sensor and two masks etched on opposite sides of a quartz glass substrate; the quartz glass substrate is pasted on the sensor cover glass, the mask and the bare camera sensor are parallel to each other, the thickness of the cover glass and the substrate are h and s respectively; the transmittance of the mask is recorded as T i , where i = 1, 2, representing the front and back sides of the mask, respectively; T i Designed as a checkerboard pattern, the simplified form is T i (x) = F(Φ i (x)), where F is a two-dimensional cosine function with a period of (1, 1); Align the mask's origin and orientation and define the camera origin O at the perpendicular projection of its origin C , thus obtaining the unwrapped phase of the mask, which can be expressed as: Φ1(x)=(u T x,v T x) T Φ2(x)=(1+α)Φ1(x) Where (1+α)∈[1,2] represents the frequency scaling of the two masks; u and v are not necessarily aligned with the X and Y axes; a control point is etched at the origin of T1, denoted as g=(0,0,h+s) T , used to resolve the periodic blurring of the moiré pattern; The image coordinate system O is defined at the upper left of the sensor I , in O C -(0, 0, t) T A virtual coordinate system O is defined at G ; This coordinate system is used to describe the system as a pinhole camera coordinate system; C , O I and O G The X and Y axes are parallel to the edge of the sensor; Beacon in O C The coordinates in are expressed as P = (p T , z) T , where p represents the coordinate in the plane, z represents the depth; the distance from P to the front mask is recorded as d=zhs; T i The projection pattern on the sensor is projected by the beacon and is denoted as The composite image T and Moiré pattern S are expressed as Where LP{..} is a low-pass filter, and the image of the Moiré pattern is I = Img(S), where Img{..} involves sampling, quantization, and blurring operations; The positioning model is expressed as S(x) = F(Φ(x)) and where β is inversely proportional to the period length of S; φ and ψ are the initial phases in the interval [0, 1).

7. The point positioning method based on 3-D Moiré effect according to claim 6, characterized in that: The optical path connecting P and p In the positioning spectrum on The projection of is an orthogonal transformation and is not affected by refraction; then The projection spectrum on is S ∞ (x) = LP{T1(x)T2(x)}. According to the product and identity, Φ ∞ (x) = Φ2(x)-Φ1(x); therefore, Φ ∞ (x) = αΦ1(x); and because F(Φ(p)) = F(Φ ∞ (p)), so there is, (b-a)v T p+(ψ-n)=0 Define h as the (m,n) peak of S, that is bv T h+ψ=n And combine it with the above formula. We have, The in-plane coordinate p of the beacon is bound to a characteristic peak of the moiré pattern S; h is regarded as the image point of p; The calibration algorithm of phase unwrapping is as follows: The direct relationship between light and pattern is established through the control point g: Also because have: (b-l * )v T c+(ψ-n c )=0, where λ * =(1+α)(1-λ) is an internal parameter obtained by calibration; period m c and n c Solve, and you get Where Round{..} represents the rounding function; m c and n c are integers, β, λ * ,u,v,c, and ψ are values ​​that do not require high precision; The calibration algorithm of the positioning model is as follows: what is read from I is O I Pixel coordinates in the coordinate system and The relationship between them is Where w is the pixel size, (a, b) T O C The pixel coordinates in the coordinate system. Therefore, This positioning model is equivalent to an optical center at (0, 0, γ * ) T , the pinhole model with a focal length of f is regarded as a virtual camera fixed on the system, and its coordinate system is O G =O C -(0, 0, γ * ) T , projective geometry techniques are used to calibrate the ghost camera and solve P through PnP or binocular triangulation.

8. The point positioning method based on 3-D Moiré effect according to claim 7, characterized in that: In the point positioning system, a mask is attached to the naked camera sensor, and the side close to the naked camera sensor is marked as T1, and the other side is marked as T2. Therefore, the Moiré image S is generated by the Moiré effect; the movement of S can be observed from the movement of the beacon P, and the spatial position of the beacon P can be located based on the movement of S, so P is also called the target point; In the positioning spectrum method, the hovering theorem accompanying the positioning spectrum is adopted; specifically: a mask-based visual measurement system is adopted, which regards the mask plate as N transmission planes. When there are two mask plates, the transmittance function is recorded as, where F is a two-dimensional cosine function; Each transmission plane is divided into points (p, p z ) T =(p x , p y , p z ) T The transmittance projection of the projection center on the XY plane is recorded as The comprehensive transmittance T and the moiré pattern S formed thereon are: Given a point q on S, the light path from P to q is written as Place P along Push to infinity, that is, P ∞ =(p,∞) T , that is, the projective transformation becomes a parallel transformation; The transformed transmittance projection is denoted as Assuming that the optical path from P to q remains unchanged, we have T and There is the following relationship Then, assuming that the light path nearby The same is true under projective transformation and parallel transformation. in, It is called the positioning spectrum, q is called the hovering point, the transmittance of the hovering point is called the hovering value, and the above relationship is called the hovering theorem.

9. The point positioning method based on 3-DMoiré effect according to claim 5, characterized in that: In step 2, the Moiré image is used to perform phase unwrapping calibration to obtain unwrapped phases m and n, including the following steps: Step A1: low-pass filter the Moiré image to smooth the image and reduce signal noise, and perform histogram equalization on the image to improve the image contrast and signal-to-noise ratio, and detect the image coordinates c of the control point; Step A2: Use quadratic surface fitting to extract the sub-pixel position of the signal peak in the image and estimate the Moiré period u, v and the corresponding phase and ψ; Step A3: Calibrate λ by least squares method * , in order to calibrate λ * Attach a beacon to the one-dimensional grating ruler and move the beacon to capture a series of have: (b j -l * )v T c j +(ψ j -n j )=0 If m j and n j The absolute value of cannot be determined, so the image is taken continuously to monitor its increase or decrease; therefore, there is m j =m0+Δm j and n j =n0+Δn j , where m0 and n0 are unknown, Δm j and Δn j known; In matrix form, we have, When there are enough samples, the least squares algorithm is used to solve λ * . Step A4: Using the formula Perform the least squares method to solve for m and n.

10. The point positioning method based on 3-DMoiré effect according to claim 5, characterized in that: The pinhole model with a periodic expansion range is regarded as a virtual camera fixed on the system, which can be expressed as G =O C -(0, 0, γ * ) T , The virtual imaging plane is on the virtual camera sensor plane O G Compared with the actual system camera sensor plane O C Between the virtual imaging plane and O G The distance is called the focal length f of the pinhole model; then, the projective geometry technique can be used to calibrate the virtual camera and solve P through PnP or binocular triangulation; specifically: given a W×W sensor and the moving range of P beacon, in order to optimize the positioning resolution, the camera in the pinhole model needs to cover this range through its field of view; 3-D Moiré model sets a large f concentrated in a small range; if the obtained Moiré pattern is periodic, its period is extended by resolving the phase blur, and then its extended range is used to cover other ranges; most of the upper limit of resolution can be improved by R = (Wex / W) 2 , where Wex is the side length of the extended range.

Images