Photogrammetry methods, systems, equipment and storage media via parallel plane

Abstract

This invention discloses a photogrammetry method, system, device, and storage medium through a parallel plate. The method includes: a vision sensor and a parallel plate both mounted and fixed to one side of the object being measured, with the parallel plate positioned between the vision sensor and the object and close to the vision sensor; the vision sensor is used to acquire images of a calibration plate during the calibration process of the photogrammetry system and to acquire images of the object being measured during the measurement process; the parallel plate isolates the measurement system from the object being measured; the calibration plate is mounted and fixed on a translation stage, with the normal vector of the calibration plate target surface parallel to the translation direction of the translation stage; the translation stage is used to translate the calibration plate during the calibration process, providing a one-dimensional scale for establishing a world coordinate system; and a data processing system is used to perform image processing and corresponding mathematical operations. This invention employs an imaging ray tracing calibration method to calibrate the measurement system and corrects the system errors of this method, achieving high-precision visual measurement through a parallel plate.

Description

Technical Field

[0001] This invention relates to the field of photogrammetry, and more particularly to a photogrammetric method, system, device, and storage medium that uses a parallel plate. Background Technology

[0002] With the upgrading of vision sensor and lens manufacturing processes, the accuracy of photogrammetry is constantly improving, and its advantages are continuously expanding. Photogrammetry uses vision sensors to acquire images of the object being measured through area acquisition, and reconstructs the shape and three-dimensional information of the object from the two-dimensional image. It has advantages such as non-contact operation, high precision, and wide measurement range, and plays an irreplaceable role in measurement fields such as map reconstruction and 3D reconstruction.

[0003] In many situations, the object being measured needs to be isolated from the measurement system. For example, if the measurement system operates in harsh environments such as high or low temperatures or corrosive environments, a transparent parallel plate is needed to protect the vision sensor; or if the object being measured is a special substance such as an artifact or a toxic substance, a transparent parallel plate is needed to enclose it. Therefore, in the above scenarios, the measurement system needs to measure the object through a transparent plate, and the light will undergo two refractions before entering the vision sensor.

[0004] For the above measurement scenarios, most photogrammetry solutions directly ignore the influence of the parallel plate on the image during the image acquisition process of the visual sensor, resulting in poor measurement accuracy. Summary of the Invention

[0005] The present invention provides a photogrammetry method, system, device and storage medium through a parallel plate to solve the above-mentioned technical problems.

[0006] The technical solution adopted in this invention is: to provide a photogrammetry method through a parallel plate, comprising the following steps:

[0007] S1. Set up the fixed calibration plate on the translation stage, making the translation vector of the translation stage perpendicular to the surface of the calibration plate. Set the starting calibration scale δ1 and the ending calibration scale δ1 of the translation stage for the imaging ray tracing calibration process. k And the calibration step size Δδ;

[0008] S2. Calibrate the vision sensor without setting up a parallel plate. Use the vision sensor to capture one image of the calibration plate at different angles, and at the beginning and end of the calibration scale δ1 and δ2 on the translation stage. k The image below is used to extract the pixel coordinates of the target points and, using the calibration results from the vision sensor, to calibrate the target points p at the start and end calibration positions on the calibration board. b1 …p bn p a1 …p anThree-dimensional reconstruction was performed using the target point p at two calibration positions on the calibration plate. b1 …p bn p a1 …p an Calculate the translation vector γ of the translation stage and the target surface normal vector η;

[0009] S3. Set up a fixed parallel plate between the object being measured and the vision sensor and close to the vision sensor. Perform imaging ray tracing calibration of the vision sensor through the parallel plate. Use a polynomial mapping model to realize the unidirectional mapping from the image surface pixel (u,v) of the vision sensor to the corresponding point (x,y) on the target surface at k+1 calibration scales. Define the world coordinate system Z-axis as parallel to the translation vector γ of the translation stage. Therefore, the displacement of the corresponding point on the target surface mapped by the image surface pixel of the vision sensor at each calibration scale is equivalent to the change of the calibration scale. At the same time, define the plane perpendicular to the translation vector γ of the translation stage at the initial calibration scale δ1 as the XOY plane of the world coordinate system. Define the plane parallel to the world coordinate system at each calibration scale as the standard calibration plane.

[0010] S4. Set the origin O of the world coordinate system to establish the world coordinate system, calculate the angle θ between the translation vector γ of the translation stage and the normal vector η of the target surface, and use θ to complete the projection transformation and similarity transformation between the target surface and the standard calibration plane to realize the scale confirmation of the standard calibration coordinate system.

[0011] S5. Acquire images of the test point from different viewpoints using a vision sensor, extract the pixel coordinates of the test point, first calculate the target surface mapping point of the calibration plate corresponding to the pixel at each calibration position using step S3, and then calculate the mapping point p of the pixel at each calibration scale using the standard calibration coordinate system scale verification method in step S4. l1 …p ln p r1 …p rn Using mapping point p l1 …p ln p r1 …p rn Fitting spatial light l l l r Calculate spatial rays l l l r The intersection point p, and the coordinates of point p (x, y, z) are the world coordinates of the measured point.

[0012] Furthermore, the measured point is within the space enclosed by the start calibration scale and the end calibration scale of the calibration plate in step S1.

[0013] Furthermore, in step S2, the calibration plate target point p at the two calibration positions... b1 …p bn p a1 …pan The n pairs of translation vectors γ1…γ can be calculated. n Using n pairs of translation vectors γ1…γ n The mean γ is used as the initial value to perform nonlinear optimization on the translation vector γ of the translation stage. The cost function is given by the following equation:

[0014]

[0015] The target surface normal vector η is the target point p on the calibration plate at two calibration positions. b1 …p bn p a1 …p an The mean values ​​of the fitted target surface normal vectors η1 and η2.

[0016] Furthermore, the imaging ray tracing calibration process in step S3 is as follows:

[0017] The translation stage is controlled starting from the initial calibration scale δ1. First, an image of the calibration plate is acquired using a vision sensor. Each time, the translation stage is adjusted to move the calibration plate by a calibration step size Δδ, and an image of the calibration plate is acquired again using the vision sensor. This process continues until the final calibration scale δ1 is reached. k Stop. A total of k+1 images of the calibration board at k+1 calibration scales were acquired. A polynomial mapping model was used to perform a one-way mapping between the visual sensor pixels and the target points on the calibration board at each calibration scale. The reprojection error E was used to measure the fitting effect of the polynomial mapping model. The reprojection error equation is given by the following formula:

[0018]

[0019] Where n is the number of target points p represents the spatial coordinates of the extracted pixels mapped by the calculated polynomial fitting parameters. i This represents the true coordinates of a point in space.

[0020] Furthermore, in step S4, projection transformation is used to convert the coordinates of the target points under the target surface at each calibration scale into coordinates under the standard calibration plane. Similarity transformation is used to add the Z-axis offset of the target points under each calibration scale under the influence of the angle θ between the translation vector γ of the translation stage and the target surface normal vector η to the current calibration scale, thereby realizing the scale confirmation of the calibration coordinate system.

[0021] Furthermore, in step S5, the desired intersection point p satisfies the condition that p lies on the common perpendicular of the two lines and p is rayed to the spatial ray l. l l r The distances are equal.

[0022] The present invention also provides a photogrammetry system through a parallel plate, including a vision sensor, a parallel plate, a translation stage and a calibration plate;

[0023] The calibration plate is mounted and fixed on the translation stage, with the translation vector of the translation stage perpendicular to the surface of the calibration plate. The starting calibration scale δ1 and the ending calibration scale δ1 of the translation stage are set during the imaging ray tracing calibration process. k And the calibration step size Δδ;

[0024] The vision sensor was calibrated without a parallel plate. One image of the calibration plate was captured from different angles using the vision sensor at the beginning and end of the calibration scale δ1 on the translation stage. k The image below is used to extract the pixel coordinates of the target points and, using the calibration results from the vision sensor, to calibrate the target points p at the start and end calibration positions on the calibration board. b1 …p bn p a1 …p an Three-dimensional reconstruction was performed using the target point p at two calibration positions on the calibration plate. b1 …p bn p a1 …p an Calculate the translation vector γ of the translation stage and the target surface normal vector η;

[0025] The parallel plate is fixed between the object being measured and the vision sensor and close to the vision sensor. The vision sensor is calibrated by imaging ray tracing through the parallel plate. A polynomial mapping model is used to realize the unidirectional mapping from the pixels (u,v) of the vision sensor imaging surface to the corresponding points (x,y) of the target surface at k+1 calibration scales. The Z-axis of the world coordinate system is defined to be parallel to the translation vector γ of the translation stage. Therefore, the displacement of the corresponding points of the target surface mapped by the pixels of the vision sensor imaging surface at each calibration scale is equivalent to the change of the calibration scale. At the same time, the plane perpendicular to the translation vector γ of the translation stage at the initial calibration scale δ1 is defined as the XOY plane of the world coordinate system, and the plane parallel to the world coordinate system at each calibration scale is defined as the standard calibration plane.

[0026] Establish the world coordinate system by setting the origin O, calculate the angle θ between the translation vector γ of the translation stage and the normal vector η of the target surface, and use θ to complete the projection transformation and similarity transformation between the target surface and the standard calibration plane, so as to realize the scale confirmation of the standard calibration coordinate system.

[0027] Images of the test point are acquired from different viewpoints using a vision sensor, and the pixel coordinates of the test point are extracted. First, based on the calculated target surface mapping points of the calibration plate corresponding to the pixel points at each calibration position, the mapping points p of the pixel points at each calibration scale are calculated using the scale verification method of the standard calibration coordinate system. l1 …p ln p r1 …p rn Using mapping point p l1 …p ln p r1…p rn Fitting spatial light l l l r Calculate spatial rays l l l r The intersection point p, and the coordinates of point p (x, y, z) are the world coordinates of the measured point.

[0028] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the photogrammetry method through a parallel plate as described above.

[0029] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the photogrammetry method through a parallel plate as described above.

[0030] The beneficial effects of the present invention are: the present invention uses an imaging ray tracing calibration method to calibrate the measurement system and corrects the system error of the method, thereby realizing high-precision visual measurement through a parallel plate. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the structure of the photogrammetry system disclosed in an embodiment of the present invention.

[0032] Figure 2 This is a diagram showing the relationship between the standard calibration plane and the target surface under any calibration scale disclosed in the embodiments of the present invention.

[0033] Figure 3 This is an experimental absolute error diagram disclosed in an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in further detail below with reference to the accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0035] Example 1:

[0036] This invention discloses a photogrammetry method using a parallel plate. The method employs a visual sensor, a parallel plate, a calibration plate, a translation stage, and a data processing system. Both the visual sensor and the parallel plate are mounted and fixed to one side of the object being measured. The parallel plate is positioned between the visual sensor and the object, close to the visual sensor. The visual sensor is used to acquire images of the calibration plate during the photogrammetry system calibration process and to acquire images of the object being measured during the measurement process. The parallel plate isolates the measurement system from the object being measured, meeting the requirements of special measurement environments. The calibration plate is mounted and fixed on the translation stage. During mounting and fixing, the normal vector of the calibration plate's target surface should be kept as parallel as possible to the translation direction of the translation stage. The calibration plate has corresponding feature points to facilitate target point extraction during the calibration process and improve extraction accuracy. The translation stage is used to translate the calibration plate during the calibration process, providing a one-dimensional scale for establishing a world coordinate system. The data processing system is used to perform image processing and corresponding mathematical operations.

[0037] Reference Figure 1 The overall structure of the photogrammetric system shown, and the photogrammetric method through a parallel plate according to the present invention, specifically include the following steps:

[0038] S1. Set up the fixed calibration plate 3 on the translation stage 4, making the translation vector of the translation stage 4 as perpendicular as possible to the surface of the calibration plate 3. Set the starting calibration scale δ1 and the ending calibration scale δ1 of the translation stage for the imaging ray tracing calibration process. k And the calibration step size Δδ; the measured point should be as close as possible to the calibration plate at the beginning calibration scale δ1 and the end calibration scale δ1. k Within the enclosed space, the calibration step size Δδ and calibration sequence k should be selected according to the specific measurement range.

[0039] S2. Calibrate the vision sensor 1 without setting up the parallel plate 2. Use the vision sensor 1 to collect one image of the calibration plate from different angles. Start and end the calibration scale δ1 on the translation stage 4. k The image below; distortion correction is performed on the acquired image based on the distortion parameters obtained from calibration; the pixel coordinates of the calibration board target point in the image are extracted (u l ,v l ), (u r ,v r The target point's world coordinates (x, y, z) are reconstructed based on the calibration parameters obtained from the two angles. The transformation matrix from pixel coordinates to world coordinates is expressed by the following formula:

[0040]

[0041] in, as well as Calibrate the parameters of the vision sensor;

[0042] Based on the calculated world coordinates p of the calibration plate target point at the start and end of the calibration scale. b1 (x b1 ,y b1 ,z b1 ...p bn (x bn ,y bn ,z bn ), p a1 (x a1 ,y a1 ,z a1 ...p an (x an ,y an ,z an It is possible to calculate n pairs of translation vectors γ1…γ n Using n pairs of translation vectors γ1…γ n mean The translation vector γ of the translation stage is used as an initial value for nonlinear optimization, and the cost function is given by the following equation:

[0043]

[0044] The target surface normal vector η is the target point p on the calibration plate at two calibration positions. b1 …p bn p a1 …p an The mean values ​​of the fitted target surface normal vectors η1 and η2.

[0045] S3. Set up a fixed parallel plate 2 between the object being measured and the vision sensor, and close to the vision sensor 1. Calibrate the vision sensor using the imaging ray tracing method through the parallel plate 2. The calibration process is as follows: Control the translation stage to start from the initial calibration scale δ1. First, use the vision sensor to acquire an image of the calibration plate. Each time, adjust the translation stage to move the calibration plate by the calibration step size Δδ, and use the vision sensor to acquire an image of the calibration plate until the final calibration scale δ1 is reached. k Stop, and collect k+1 images of the calibration board at k+1 calibration scales; use a polynomial mapping model to complete the one-way mapping between the visual sensor pixels and the target points on the calibration board at each calibration scale; use the reprojection error E to measure the fitting effect of the polynomial mapping model, and the reprojection error equation is given by the following formula:

[0046]

[0047] Where n is the number of target points p represents the spatial coordinates of the extracted pixels mapped by the calculated polynomial fitting parameters. iThe coordinates of the points in space are the true values. The order of the polynomial should be selected according to the specific fitting accuracy to avoid overfitting and underfitting, so as to minimize the reprojection error. The Z-axis of the world coordinate system is defined to be parallel to the translation vector γ of the translation stage. Therefore, the displacement of the corresponding point on the target surface mapped by the pixel point of the visual sensor imaging surface at each calibration scale is equivalent to the change of the calibration scale. At the same time, the plane perpendicular to the translation vector γ of the translation stage at the initial calibration scale δ1 is defined as the XOY plane of the world coordinate system, and the plane parallel to the world coordinate system at each calibration scale is defined as the standard calibration plane.

[0048] S4. Establish the world coordinate system by setting the origin O. The method for determining the scale of the imaging ray tracing calibration coordinate system is to use a calibration plate as the standard calibration plane, that is, to use the scale of the calibration plate to confirm the scale of the XOY plane of the world coordinate system, and at the same time, to use the translation direction of the translation stage as the Z-axis direction of the world coordinate system, and to use the calibration step size Δδ to confirm the scale of the Z-axis of the world coordinate system, thus finally establishing the overall scale of the coordinate system; refer to Figure 2 As shown, since it is difficult to ensure that the calibration plate target surface is completely parallel to the standard calibration plane in reality, and the scale of the plane parallel to the world coordinate system XOY plane needs to be determined by the calibration plate target surface, a projection transformation is used to convert the coordinates of the target point under the target surface at each calibration scale to the coordinates under the standard calibration plane; a similarity transformation is used to calculate the angle θ between the translation stage translation vector γ and the target surface normal vector η, and the Z-axis offset of the target point under the influence of the angle θ between the translation stage translation vector γ and the target surface normal vector η is added to the current calibration scale, and finally the scale of the calibration coordinate system is confirmed.

[0049] S5. Acquire images of the test point from different viewpoints using a vision sensor, extract the pixel coordinates of the test point, first calculate the target surface mapping point of the calibration plate corresponding to the pixel at each calibration position using step S3, and then calculate the mapping point p of the pixel at each calibration scale using the standard calibration coordinate system scale verification method in step S4. l1 …p ln p r1 …p rn Using mapping point p l1 …p ln p r1 …p rn Fitting spatial light l l l r Calculate spatial rays l l l r The intersection point p, since it is impossible to achieve two-way fitting of spatial rays l in actual measurement. l l r The lines intersect, therefore the required intersection point p satisfies the condition that p lies on the common perpendicular of the two lines and p is rayd to the space ray l. l l rThe distances are equal, and the final calculated coordinates (x, y, z) of point p are the world coordinates of the measured point.

[0050] In summary, the core of this invention lies in selecting the order of the fitting polynomial, calculating the fitting parameters, determining the translation accuracy of the translation stage, and confirming the scale of the world coordinate system in the imaging ray tracing calibration.

[0051] The following example uses an experiment to illustrate the specific experimental steps and data. The terms "first" and "second" used below do not indicate the importance of the steps, but only relate to the order of description:

[0052] First, the visual sensor selected for the experiment has a resolution of 2592×2048 and a pixel size of 4.8μm×4.8μm, and a 75mm lens is used; the calibration board uses a 9×9 circular directional target with a center distance of 12mm.

[0053] Adjust the lens focus to make the working distance of the vision sensor 1000mm. Set the start calibration scale to 0.0mm, the end calibration scale to 5.0mm, and the calibration step size to 1mm. Therefore, the number of calibrations is 6 and the measurement range is the space included by the calibration plate in the calibration scale from 0.0 to 5.0mm.

[0054] Set up a fixed calibration plate on the translation stage, and make the translation vector of the translation stage as perpendicular as possible to the surface of the calibration plate.

[0055] Second, the vision sensor was calibrated using Zhang's calibration method. Two images of the calibration plate were acquired from different viewpoints at 0.0 mm and 5.0 mm scales on the translation stage. Distortion parameters were used to correct the acquired images, and the world coordinates p of the target point on the calibration plate at 0.0 mm and 5.0 mm scales on the translation stage were calculated. b1 (x b1 ,y b1 ,z b1 ...p b81 (x b81 ,y b81 ,z b81 ), p a1 (x a1 ,y a1 ,z a1 ...p an (x an ,y an ,z an ).

[0056] 81 translation vectors γ1…γ were calculated by subtracting the world coordinates of the target point at two different scales. 81 Using the mean of the translation vector As the initial value, the translation vector γ of the translation stage is optimized nonlinearly using the Levenberg-Marquardt iterative method (the most widely used nonlinear least squares algorithm). The cost function is given by the following equation:

[0057]

[0058] The target surface normal vectors η1 and η2 at different scales are calculated using the world coordinates of the target point at two scales. The plane fitting method first uses the RANSAC method to remove outliers, and then uses the least squares method to calculate the fitting normal vectors η1 and η2 of the two planes.

[0059] Third, a fixed parallel plate is set up between the object being measured and the vision sensor, and close to the vision sensor. The vision sensor is calibrated using the imaging ray tracing method through the parallel plate. Based on the experiment, the fitting order is selected as 4, and the fitting polynomial is expressed as follows:

[0060]

[0061] In the formula C ij and D ij The parameters are for the polynomial mapping model, and the reprojection error is 1.8766 μm.

[0062] Fourth, the origin of the calibration plate target surface at the initial calibration scale (0.0 mm) is set as the origin O of the world coordinate system, and the world coordinate system is established. A projection transformation is used to convert the coordinates of the target surface points at each calibration scale to coordinates on the standard calibration plane. In this experiment, the homography matrix is ​​used to complete the transformation from target surface coordinates q1(u1,v1) to standard calibration plane coordinates q2(u2,v2). The transformation equation is as follows:

[0063] q2∝Hq1

[0064] Where H is the homography matrix of the projection transformation, and the homography matrix is ​​represented as follows:

[0065]

[0066] Where H 11 …H 22 These are the parameters of the homography matrix.

[0067] In this experiment, at each calibration scale, eight points were first randomly selected, using the target surface coordinates q1(u1,v1) and the standard calibration plane coordinates q2(u2,v2). The initial value of the homography matrix H was calculated using the least squares method. Then, the Levenberg-Marquardt iterative method was used to perform nonlinear optimization of H. The cost function is expressed as follows:

[0068]

[0069] Calculate the angle θ between the translation stage translation vector γ and the target surface normal vector η. Then, use similarity transformation to calculate the Z-axis offset Δz of the target point on the calibration plate at each calibration scale. Since the origin of the calibration plate target surface is taken as the origin O of the world coordinate system, the offset Δz is also related to the distance χ from the target origin. The relationship between the angle θ between the translation stage translation vector γ and the target surface normal vector η, the offset Δz, and the distance χ from the target origin is shown in the following equation:

[0070] Δz=χsinθ

[0071] The target point offset Δz is compared with the current calibration scale δ. i By adding them together, the scale of the calibrated coordinate system can be confirmed.

[0072] Fifth, using a vision sensor to acquire images of the measured point from different perspectives, extract the pixel coordinates of the measured point. First, calculate the mapping point of the pixel at each calibration scale according to the method described in step three. Then, optimize the world coordinates of the mapping point using the method described in step four, and use the mapping point p... l1 …p ln p r1 …p rn Fitting spatial light l l l r Calculate spatial rays l l l r The intersection point p, since it is impossible to achieve two-way fitting of spatial rays l in actual measurement. l l r The lines intersect, therefore the required intersection point p satisfies the condition that p lies on the common perpendicular of the two lines and p is rayd to the space ray l. l l r The distances are equal, and the final calculated coordinates (x, y, z) of point p are the world coordinates of the measured point.

[0073] Using a translation stage, the calibration plate is translated a distance Δr within the range described in the first step. Specifically, in the experiment, the initial scale is 0.0 mm and the final scale is 1.7 mm. A photogrammetric system is used to acquire images of the calibration plate before and after translation through a parallel plate. (Refer to...) Figure 3 As shown, the absolute values ​​of the errors r1…r1 between the translation distance of all target points and the true value (1.7 mm) are calculated. 81 The mean square error was 9.783 μm.

[0074] In summary, by adopting the above-described photogrammetry method through a parallel plate, non-contact measurement through a parallel plate can be achieved with high measurement accuracy.

[0075] Example 2:

[0076] See Figure 1This embodiment discloses a photogrammetry system through a parallel plate, including a vision sensor 1, a parallel plate 2, a translation stage 4, and a calibration plate 3.

[0077] The calibration plate is mounted and fixed on the translation stage, with the translation vector of the translation stage perpendicular to the surface of the calibration plate. The starting calibration scale δ1 and the ending calibration scale δ1 of the translation stage are set during the imaging ray tracing calibration process. k And the calibration step size Δδ.

[0078] The vision sensor was calibrated without a parallel plate. One image of the calibration plate was captured from different angles using the vision sensor at the beginning and end of the calibration scale δ1 on the translation stage. k The image below is used to extract the pixel coordinates of the target points and, using the calibration results from the vision sensor, to calibrate the target points p at the start and end calibration positions on the calibration board. b1 …p bn p a1 …p an Three-dimensional reconstruction was performed using the target point p at two calibration positions on the calibration plate. b1 …p bn p a1 …p an Calculate the translation vector γ of the translation stage and the target surface normal vector η.

[0079] The parallel plate is fixed between the object under test and the vision sensor and close to the vision sensor. The vision sensor is calibrated by imaging ray tracing through the parallel plate. A polynomial mapping model is used to realize the unidirectional mapping from the pixel point (u,v) of the vision sensor imaging surface to the corresponding point (x,y) of the target surface at k+1 calibration scales. The Z-axis of the world coordinate system is defined to be parallel to the translation vector γ of the translation stage. Therefore, the displacement of the corresponding point of the target surface mapped by the pixel point of the vision sensor imaging surface at each calibration scale is equivalent to the change of the calibration scale. At the same time, the plane perpendicular to the translation vector γ of the translation stage at the initial calibration scale δ1 is defined as the XOY plane of the world coordinate system, and the plane parallel to the world coordinate system at each calibration scale is defined as the standard calibration plane.

[0080] Establish the world coordinate system by setting the origin O, calculate the angle θ between the translation vector γ of the translation stage and the normal vector η of the target surface, and use θ to complete the projection transformation and similarity transformation between the target surface and the standard calibration plane, thereby realizing the scale confirmation of the standard calibration coordinate system.

[0081] Images of the test point are acquired from different viewpoints using a vision sensor, and the pixel coordinates of the test point are extracted. First, based on the calculated target surface mapping points of the calibration plate corresponding to the pixel points at each calibration position, the mapping points p of the pixel points at each calibration scale are calculated using the scale verification method of the standard calibration coordinate system. l1 …p ln p r1…p rn Using mapping point p l1 …p ln p r1 …p rn Fitting spatial light l l l r Calculate spatial rays l l l r The intersection point p, and the coordinates of point p (x, y, z) are the world coordinates of the measured point.

[0082] Example 3:

[0083] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the photogrammetry method through a parallel plate as described in Embodiment 1.

[0084] Example 4:

[0085] This embodiment discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the photogrammetry method through a parallel plate as described in Embodiment 1.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A photogrammetric method through a parallel plate, characterized in that, Includes the following steps: S1. Set up the fixed calibration plate on the translation stage, making the translation vector of the translation stage perpendicular to the surface of the calibration plate. Set the starting calibration scale δ1 and the ending calibration scale δ1 of the translation stage for the imaging ray tracing calibration process. k And the calibration step size Δδ; S2. Calibrate the vision sensor without setting up a parallel plate. Use the vision sensor to capture one image of the calibration plate at different angles, and at the beginning and end of the calibration scale δ1 and δ2 on the translation stage. k The image below is used to extract the pixel coordinates of the target points and, using the calibration results from the vision sensor, to calibrate the target points p at the start and end calibration positions on the calibration board. b1 …p bn p a1 …p an Three-dimensional reconstruction was performed using the target point p at two calibration positions on the calibration plate. b1 …p bn p a1 …p an Calculate the translation vector γ of the translation stage and the target surface normal vector η; S3. Set up a fixed parallel plate between the object being measured and the vision sensor and close to the vision sensor. Perform imaging ray tracing calibration of the vision sensor through the parallel plate. Use a polynomial mapping model to realize the unidirectional mapping from the image surface pixel (u,v) of the vision sensor to the corresponding point (x,y) on the target surface at k+1 calibration scales. Define the world coordinate system Z-axis as parallel to the translation vector γ of the translation stage. Therefore, the displacement of the corresponding point on the target surface mapped by the image surface pixel of the vision sensor at each calibration scale is equivalent to the change of the calibration scale. At the same time, define the plane perpendicular to the translation vector γ of the translation stage at the initial calibration scale δ1 as the XOY plane of the world coordinate system. Define the plane parallel to the world coordinate system at each calibration scale as the standard calibration plane. S4. Set the origin O of the world coordinate system to establish the world coordinate system, calculate the angle θ between the translation vector γ of the translation stage and the normal vector η of the target surface, and use θ to complete the projection transformation and similarity transformation between the target surface and the standard calibration plane to realize the scale confirmation of the standard calibration coordinate system. S5. Acquire images of the test point from different viewpoints using a vision sensor, extract the pixel coordinates of the test point, first calculate the target surface mapping point of the calibration plate corresponding to the pixel at each calibration position using step S3, and then calculate the mapping point p of the pixel at each calibration scale using the standard calibration coordinate system scale verification method in step S4. l1 …p ln p r1 …p rn Using mapping point p l1 …p ln p r1 …p rn Fitting spatial light l l l r Calculate spatial rays l l l r The intersection point p, and the coordinates of point p (x, y, z) are the world coordinates of the measured point.

2. The photogrammetry method through a parallel plate according to claim 1, characterized in that: The measured point is within the space enclosed by the start calibration scale and the end calibration scale of the calibration plate in step S1.

3. The photogrammetry method through a parallel plate according to claim 1, characterized in that: In step S2, the calibration plate target point p at the two calibration positions b1 …p bn p a1 …p an The n pairs of translation vectors γ1…γ can be calculated. n Using n pairs of translation vectors γ1…γ n mean The translation vector γ of the translation stage is used as an initial value for nonlinear optimization, and the cost function is given by the following equation: The target surface normal vector η is the target point p on the calibration plate at two calibration positions. b1 …p bn p a1 …p an The mean values ​​of the fitted target surface normal vectors η1 and η2.

4. The photogrammetric method through a parallel plate according to claim 1, characterized in that: The imaging ray tracing calibration process in step S3 is as follows: The translation stage is controlled starting from the initial calibration scale δ1. First, an image of the calibration plate is acquired using a vision sensor. Each time, the translation stage is adjusted to move the calibration plate by a calibration step size Δδ, and an image of the calibration plate is acquired again using the vision sensor. This process continues until the final calibration scale δ1 is reached. k Stop. A total of k+1 images of the calibration board at k+1 calibration scales were acquired. A polynomial mapping model was used to perform a one-way mapping between the visual sensor pixels and the target points on the calibration board at each calibration scale. The reprojection error E was used to measure the fitting effect of the polynomial mapping model. The reprojection error equation is given by the following formula: Where n is the number of target points p represents the spatial coordinates of the extracted pixels mapped by the calculated polynomial fitting parameters. i This represents the true coordinates of a point in space.

5. The photogrammetry method through a parallel plate according to claim 1, characterized in that: In step S4, projection transformation is used to convert the coordinates of the target points under the target surface at each calibration scale into coordinates under the standard calibration plane. Similarity transformation is used to add the Z-axis offset of the target points under each calibration scale under the influence of the angle θ between the translation vector γ of the translation stage and the target surface normal vector η to the current calibration scale, thereby realizing the scale confirmation of the calibration coordinate system.

6. The photogrammetry method through a parallel plate according to claim 1, characterized in that: In step S5, the intersection point p satisfies the condition that p lies on the common perpendicular of the two lines and p is rayd to the space ray l. l l r The distances are equal.

7. A photogrammetric system through a parallel plane, characterized in that, Includes vision sensors, parallel plates, translation stages, and calibration boards; The calibration plate is mounted and fixed on the translation stage, with the translation vector of the translation stage perpendicular to the surface of the calibration plate. The starting calibration scale δ1 and the ending calibration scale δ1 of the translation stage are set during the imaging ray tracing calibration process. k And the calibration step size Δδ; The vision sensor was calibrated without a parallel plate. One image of the calibration plate was captured from different angles using the vision sensor at the beginning and end of the calibration scale δ1 on the translation stage. k The image below is used to extract the pixel coordinates of the target points and, using the calibration results from the vision sensor, to calibrate the target points p at the start and end calibration positions on the calibration board. b1 …p bn p a1 …p an Three-dimensional reconstruction was performed using the target point p at two calibration positions on the calibration plate. b1 …p bn p a1 …p an Calculate the translation vector γ of the translation stage and the target surface normal vector η; The parallel plate is fixed between the object being measured and the vision sensor and close to the vision sensor. The vision sensor is calibrated by imaging ray tracing through the parallel plate. A polynomial mapping model is used to realize the unidirectional mapping from the pixels (u,v) of the vision sensor imaging surface to the corresponding points (x,y) of the target surface at k+1 calibration scales. The Z-axis of the world coordinate system is defined to be parallel to the translation vector γ of the translation stage. Therefore, the displacement of the corresponding points of the target surface mapped by the pixels of the vision sensor imaging surface at each calibration scale is equivalent to the change of the calibration scale. At the same time, the plane perpendicular to the translation vector γ of the translation stage at the initial calibration scale δ1 is defined as the XOY plane of the world coordinate system, and the plane parallel to the world coordinate system at each calibration scale is defined as the standard calibration plane. Establish the world coordinate system by setting the origin O, calculate the angle θ between the translation vector γ of the translation stage and the normal vector η of the target surface, and use θ to complete the projection transformation and similarity transformation between the target surface and the standard calibration plane, so as to realize the scale confirmation of the standard calibration coordinate system. Images of the test point are acquired from different viewpoints using a vision sensor, and the pixel coordinates of the test point are extracted. First, based on the calculated target surface mapping points of the calibration plate corresponding to the pixel points at each calibration position, the mapping points p of the pixel points at each calibration scale are calculated using the scale verification method of the standard calibration coordinate system. l1 …p ln p r1 …p rn Using mapping point p l1 …p ln p r1 …p rn Fitting spatial light l l l r Calculate spatial rays l l l r The intersection point p, and the coordinates of point p (x, y, z) are the world coordinates of the measured point.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the photogrammetry method through a parallel plate as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the photogrammetry method through a parallel plate as described in any one of claims 1-6.

Images