Panoramic reconstruction method based on virtual multi-objective cone reflector ray tracing

Abstract

The invention discloses a ray tracing panoramic reconstruction method based on a virtual multi-objective cone reflector, an optimization function and a virtual multi-objective method are introduced, when processing is performed through the optimization function, an initial value is endowed as a target parameter, an error is output after calculation by the target function, and then the target parameter is adjusted by an L-M algorithm, so that the ray tracing panoramic reconstruction method based on the virtual multi-objective cone reflector is realized. The target function is returned for calculation, a new error is output, iteration is carried out until the error is zero, the calculated angular point P coincides with the real angular point P, and at the moment, the corresponding target parameter serves as an optimized value to be output outwards, so that the optimized value serves as the target parameter to accurately describe the actual space position of the conical reflector, the reconstructed surface shape is more accurate, and the reconstruction precision is improved. Compared with the defects that in the prior art, a conical reflector needs to be strictly located below a camera, and errors are high, the measuring accuracy is improved, and due to the fact that strict correction is not needed, the measuring efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of surface shape reconstruction, and particularly to a panoramic reconstruction method based on virtual multi-purpose cone mirror ray tracing. Background Art

[0002] With the continuous development of modern manufacturing, the demand for high-precision three-dimensional measurement technology in various fields is increasing. Existing pipeline inner wall measurements mainly rely on contact and non-contact methods. Contact methods such as micrometers and coordinate measuring machines rely on manual operation or mechanical probes, and have problems such as low efficiency and limited measurement depth; non-contact technologies such as ultrasonic detection and laser vision measurement can obtain local features, but are limited by single-point or single-view detection and are difficult to realize the three-dimensional shape reconstruction of the entire inner wall. In addition, existing three-dimensional measurement schemes mostly use multi-device splicing or rotary scanning. The former requires multiple laser / vision sensors to be arranged at intervals, and the latter relies on a high-precision rotating shaft system. Splicing is prone to error accumulation at the seams, and rotary scanning faces problems such as rotating shaft reference drift, cross-section correction difficulties, and no closed-loop calibration;

[0003] In some methods of inner wall reconstruction, a camera is used, and a conical mirror is placed inside the pipeline, and surface shape reconstruction is carried out by adding and shooting. However, this method requires the central axis of the conical mirror to be directly below the optical center of the camera, with strict conditions and low measurement accuracy. Summary of the Invention

[0004] In view of the above defects or deficiencies in the prior art, it is desirable to provide a panoramic reconstruction method based on virtual multi-purpose cone mirror ray tracing.

[0005] A panoramic reconstruction method based on virtual multi-purpose cone mirror ray tracing provided by the present invention specifically includes the following steps:

[0006] S100. Place a calibration board on an optical platform, set an industrial camera above the calibration board, and virtually form a number of virtual cameras by the virtual multi-view method, and mark serial numbers for each of the virtual cameras;

[0007] S200. Calibrate each of the virtual cameras by the Zhang Zhengyou calibration method to obtain the internal parameters of each of the virtual cameras and the external parameters between adjacent serial numbers of the virtual cameras;

[0008] S300. Establish a measurement system, place a pipeline to be measured directly below the industrial camera, attach a checkerboard calibration board to the inner wall of the pipeline, and place a conical mirror at the central position inside the pipeline;

[0009] S400. Use each of the virtual cameras to photograph the conical mirror, which has a spatial coordinate expression that includes target parameters. The target parameters include the coordinates of the vertex of the conical mirror, the direction vector of the central axis of the conical mirror, and the cone angle of the conical mirror;

[0010] S500. Take each corner point P of the checkerboard calibration board as a point to be measured C, and calculate respectively through the ray tracing method to obtain the spatial positions of each corner point P;

[0011] S600. Calculate the error between the iterative spacing and the standard spacing. The standard spacing is the true spacing between adjacent corner points P in the checkerboard calibration board, and the iterative spacing is the spacing between adjacent corner points P in step S500;

[0012] S700. Input the target parameters and the objective function into the optimization function. After being processed by the optimization function, output the optimized value as the target parameters. Among them, the objective function is based on the target parameters, the internal and external parameters of each virtual camera, perform the calculations in step S400, step S500, and step S600, and output the error value;

[0013] S800. Take all points of the checkerboard calibration board as points to be measured C, and calculate respectively through the ray tracing method to obtain the true spatial positions of all points.

[0014] Preferably, in step S100, the virtual multi-camera method specifically includes the following steps:

[0015] S110. Use the industrial camera to photograph the calibration board to obtain a primary imaging image containing the calibration board;

[0016] S120. Divide the primary image into four secondary imaging images of equal pixel size and each containing the calibration board according to the upper left corner, upper right corner, lower left corner, and lower right corner respectively;

[0017] S130. The industrial camera forms four virtual cameras according to the four secondary imaging images, corresponding to the four secondary imaging images respectively.

[0018] Preferably, in step S200, the internal parameters of each virtual camera are respectively the focal length f n and the optical center coordinates (u 0n , v 0n ). The external parameters between each virtual camera are respectively the rotation matrix R n，n+1 and the translation matrix t n，n+1 ; where, n and n + 1 represent the corresponding virtual camera numbers, and n = 1, 2, 3.

[0019] Preferably, in the step S400, the spatial coordinate expression of the conical mirror is as follows:

[0020] ((x - x0)a+(y - y0)b+(z - z0)c) 2 =cos 2 (θ)((x - x0) 2 +(y - y0) 2 +(z - z0) 2 )

[0021] Wherein, (x, y, z) represents the coordinate parameters of any point on the surface of the conical mirror in space, (x0, y0, z0) represents the coordinate parameters of the vertex U of the conical mirror, represents the direction vector parameters of the central axis of the conical mirror, and θ is the cone angle parameter of the conical mirror.

[0022] Preferably, in the step S500, the ray tracing method specifically includes the following steps:

[0023] S510. There are corresponding pixel points A n on the secondary imaging images of the four virtual cameras corresponding to the point C to be measured, where n represents the serial number of the virtual camera, and n = 1, 2, 3, 4;

[0024] S520. Connect the pixel point A n and the corresponding optical center coordinate O n to obtain the incident sub-ray

[0025] S530. Unify the coordinate parameters of each incident sub-ray ;

[0026] S540. After the incident sub-ray extends, there is a reflection point B n between it and the surface of the conical mirror, and the incident ray

[0027] S550. There is a conical generatrix n passing through the reflection point B on the surface of the conical mirror. Connect the vertex U of the conical mirror and the reflection point B n to form the conical generatrix

[0028] S560. Calculate and obtain the unit vector n passing through the reflection point B and perpendicular to the conical generatrix and mark it as the normal of specular reflection;

[0029] S570. Calculate the outgoing ray based on the incident ray and the normal line according to the law of reflection

[0030] S580. Calculate the intersection points of the n outgoing rays , that is, the point C to be measured, and obtain the spatial position of the point C to be measured

[0031] Preferably, in the step S520, the vector parameters of the incident sub-ray are obtained by calculating through the first set of formulas. The first set of formulas is as follows An , Y An , Z An ) are obtained by calculating through the first set of formulas. The first set of formulas is as follows

[0032] X An =(u An -u 0n )×dx n

[0033] Y An =(v An -v 0n )×dy n

[0034] Z An =f n

[0035] where n represents the corresponding virtual camera number; f n represents the focal length of the corresponding virtual camera, (u 0n , v 0n ) represents the origin parameters of the corresponding secondary imaging image; (u An , v An ) represents the pixel coordinate parameters of the pixel point A n in the corresponding secondary imaging image; (dx n , dy n ) are the single pixel lengths in the X direction and Y direction in the corresponding virtual camera

[0036] Preferably, in the step S530, the coordinate parameters of each incident sub-ray are unified through the second formula. Specifically, the camera coordinate system of the virtual camera numbered 1 is selected as the world coordinate system, and the camera coordinate systems of other virtual cameras are converted into the world coordinate system through the second formula. The second formula is as follows

[0037] α n =R n,n+1 α n+1 +t n,n+1

[0038] Among them, n and n + 1 represent the corresponding virtual camera numbers, where n = 1, 2, 3; α n represents the coordinate parameters of the camera coordinate system of the corresponding virtual camera; R n,n+1 represents the rotation matrix between the virtual camera with number n and the virtual camera with number n + 1; t n,n+1 represents the translation matrix between the virtual camera with number n and the virtual camera with number n + 1.

[0039] Preferably, in the step S540, the incident sub-ray extends s unit vectors towards the conical mirror side and intersects with the surface of the conical mirror to form a reflection point B n . Through the third formula group and the conical mirror expression, the incident ray is calculated The third formula group is as follows:

[0040]

[0041] Among them, n represents the number of the corresponding virtual camera, is the incident sub-ray, B n is the intersection point of the outgoing ray and the surface of the conical mirror, s is the solution coefficient, (p xn , p yn , p zn ) is the vector parameter of the unit vector .

[0042] Preferably, in the step S560, according to the fourth formula of the law of specular reflection, the vector parameters (l , l xn , l yn , l zn ) of the normal line are calculated. The fourth formula is as follows:

[0043]

[0044] Among them, is the unit vector perpendicular to the conical generatrix n passing through the reflection point B , is the conical generatrix passing through the reflection point B n ;

[0045] In the step S570, through the sixth formula of the vector reflection law, the outgoing ray is calculated. The sixth formula is as follows:

[0046]

[0047] Among them, is the outgoing light ray from the measurement point C towards the surface of the conical mirror, is the outgoing light ray after passing through the reflection point B n and entering the virtual camera optical center as the incident light ray, is the incident light ray and the outgoing light ray is the axis of symmetry.

[0048] Preferably, in the step S580, the vector intersection point, i.e., the measurement point C, is calculated through the seventh formula group of the vector intersection law. The seventh formula group is as follows:

[0049] x Bn +h n *x rn =x Bn+1 +h n+1 *x rn+1

[0050] y Bn +h n *y rn =y Bn+1 +h n+1 *y rn+1

[0051] z Bn +h n *z rn =z Bn+1 +h n+1 *z rn+1

[0052] Among them, n and n + 1 represent the corresponding virtual camera numbers, n = 1, 2, 3; (x rn , y rn , z rn ) is the vector parameter of the outgoing light ray ; (x Bn , y Bn , z Bn ) is the coordinate parameter of the refraction point B n , and h n is the solution coefficient.

[0053] Compared with the prior art, the beneficial effects of the present invention are:

[0054] The present invention introduces an optimization function and a virtual multi-view method. When processing through the optimization function, an initial value is first assigned as the target parameter. The error is output after being calculated by the objective function, and then the L-M algorithm is used to adjust the target parameter, and the objective function is returned to calculate and output a new error. Such iteration is performed until the error is 0, indicating that the calculated corner point P coincides with the true corner point P. At this time, the corresponding target parameter is output as the optimized value. Therefore, the optimized value as the target parameter can accurately describe the actual spatial position of the conical mirror, making the reconstructed surface shape more accurate. Compared with the prior art, which requires the conical mirror to be strictly below the camera and has disadvantages such as high error, in the present invention, not only the measurement accuracy is improved, but also the measurement efficiency is increased because strict calibration is not required.

[0055] In addition, based on the use of an industrial camera, the present invention virtualizes the industrial camera into multiple virtual cameras through the virtual multi-view method, and photographs the conical mirror from different directions. Through the laws of ray tracing and light reflection, the surface shape of the inner wall of the pipeline is reconstructed pixel by pixel from the captured secondary imaging images, improving the accuracy of the reconstruction of the inner wall of the pipeline. The multiple virtual cameras can perform a full-circle rapid reconstruction at one time, with the advantages of fast speed and high accuracy. Moreover, compared with the high cost of using multiple cameras, the present invention only uses one industrial camera, with low cost and easy to promote and use.

[0056] It should be understood that the content described in the summary of the invention section is not intended to limit the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. Brief Description of the Drawings

[0057] Other features, objects, and advantages of the present invention will become more obvious by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0058] Figure 1 It is a flowchart of the steps of a method for panoramic reconstruction of conical mirror ray tracing based on virtual multi-view provided by an embodiment of the present application;

[0059] Figure 2 It is a schematic diagram of ray tracing in a method for panoramic reconstruction of conical mirror ray tracing based on virtual multi-view provided by an embodiment of the present application;

[0060] Figure 3 It is a schematic diagram of specular reflection in a method for panoramic reconstruction of conical mirror ray tracing based on virtual multi-view provided by an embodiment of the present application;

[0061] Figure 4 It is a schematic diagram of the segmentation of the secondary imaging image in a method for panoramic reconstruction of conical mirror ray tracing based on virtual multi-view provided by an embodiment of the present application. Detailed Description of the Embodiments

[0062] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the relevant invention and do not limit the invention. In addition, it should be noted that for the convenience of description, only the parts related to the invention are shown in the drawings.

[0063] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the drawings and embodiments.

[0064] Please refer to Figures 1 to 4 , an embodiment of the present invention provides a panoramic reconstruction method based on virtual multi-purpose conical mirror ray tracing, which specifically includes the following steps:

[0065] S100. Place the calibration plate on the optical platform, set an industrial camera above the calibration plate, virtually form a number of virtual cameras by the virtual multi-view method, and mark serial numbers for each virtual camera;

[0066] In some embodiments, in step S100, the virtual multi-view method specifically includes the following steps:

[0067] S110. Take a picture of the calibration plate with the industrial camera to obtain a primary imaging image containing the calibration plate;

[0068] S120. Divide the primary image into four secondary imaging images with equal pixel sizes and all containing the calibration plate according to the upper left corner, upper right corner, lower left corner, and lower right corner respectively;

[0069] S130. The industrial camera forms four virtual cameras according to the four secondary imaging images, corresponding to the four secondary imaging images respectively;

[0070] Among them, the industrial camera is a physical camera, which can be virtually formed into multiple virtual cameras for use through the above virtual multi-view method. Further, the virtual multi-view method is based on virtual camera technology, which is a technical method of dividing the field of view of a physical camera into multiple virtual camera views, each view corresponding to a virtual camera, so as to virtually create multiple virtual cameras. Refer to Figure 4 , with the four corners as the corners of the four virtual cameras, four secondary imaging images with equal pixel sizes and all containing the calibration plate are formed. In this embodiment, the industrial camera is virtually formed into four virtual cameras, corresponding to the four secondary imaging images respectively, and the serial numbers are virtual camera 1, virtual camera 2, virtual camera 3, and virtual camera 4.

[0071] S200. Calibrate each virtual camera by the Zhang Zhengyou calibration method to obtain the internal parameters of each virtual camera and the external parameters between adjacent serial-numbered virtual cameras;

[0072] Among them, the Zhang-Zhengyou calibration method is a camera calibration method based on a planar checkerboard proposed by Professor Zhang Zhengyou in 1998. By taking checkerboard images at different angles and combining the homography matrix and non-linear optimization technology, this method solves the internal parameters, external parameters and distortion coefficients of the camera. It combines the high precision of traditional calibration methods and the convenience of self-calibration methods, and has become a classic method in the field of computer vision.

[0073] In some embodiments, in step S200, the internal parameters of each virtual camera are respectively the focal length f n and the optical center coordinates (u 0n , v 0n ). The external parameters between each virtual camera are respectively the rotation matrix R n，n+1 and the translation matrix t n，n+1 ; where, n and n + 1 represent the corresponding virtual camera numbers, and n = 1, 2, 3;

[0074] Among them, for the convenience of understanding, an example is given. The internal parameters of virtual camera 1 are the focal length f1 and the optical center coordinates (u 01 , v 01 ). The pose relationship between virtual camera 1 and virtual camera 2, that is, the external parameters, are represented as the rotation matrix R 1,2 and the translation matrix t 1,2 . These parameters are obtained by calibration based on the secondary imaging images through the Zhang-Zhengyou calibration method. The specific process is prior art and will not be elaborated here.

[0075] S300. Establish a measurement system. Place the pipeline to be measured directly below the industrial camera. Attach a checkerboard calibration board to the inner wall of the pipeline, and place a conical reflector at the center position inside the pipeline;

[0076] Among them, since the pipeline is placed directly below the industrial camera and the conical reflector is placed at the center of the pipeline, the central axis of the pipeline, the central axis of the conical reflector and the optical axis of the industrial camera coincide. However, due to inevitable deviations during actual operation and the inability to ensure the coincidence accuracy, when establishing the measurement system, only try to keep the three coincident as much as possible. In the present invention, the actual position of the conical reflector is obtained through subsequent optimization functions, and then the reconstructed surface shape of the inner wall of the pipeline is obtained.

[0077] S400. Use each of the virtual cameras to photograph the conical reflector. The conical reflector has a spatial coordinate expression, and the spatial coordinate expression includes target parameters. The target parameters include the coordinates of the vertex of the conical reflector, the direction vector of the central axis of the conical reflector, and the cone angle of the conical reflector;

[0078] In some embodiments, in step S400, the spatial coordinate expression of the conical reflector is as follows:

[0079] ((x - x0)a + (y - y0)b + (z - z0)c) 2 = cos 2 (θ)((x - x0) 2 + (y - y0) 2 + (z - z0) 2 )

[0080] Wherein, (x, y, z) represents the coordinate parameters of any point on the surface of the conical mirror in space, (x0, y0, z0) represents the coordinate parameters of the vertex U of the conical mirror, represents the direction vector parameter of the central axis of the conical mirror, θ is the cone angle parameter of the conical mirror, and the initial values include the coordinate parameters of the vertex U, the direction vector parameter of the central axis, and the cone angle parameter of the conical mirror;

[0081] First, it is clear that each virtual camera has its own corresponding camera coordinate system. For example, virtual camera 1 corresponds to camera coordinate system 1, virtual camera 2 corresponds to camera coordinate system 2, and so on. The above spatial coordinate expression of the conical mirror is described based on camera coordinate system 1.

[0082] S500. Take each corner point P of the checkerboard calibration plate as the measurement point C, and calculate the spatial positions of each corner point P respectively through the ray tracing method.

[0083] In some embodiments, in step S500, reference can be made to Figure 2 and Figure 3 , and the ray tracing method specifically includes the following steps:

[0084] S510. There are corresponding pixel points A on the secondary imaging images of the four virtual cameras corresponding to the measurement point C n , n represents the serial number of the virtual camera, n = 1, 2, 3, 4; referring to Figure 2 , the light rays emitted from the measurement point C are used as the outgoing light rays to shoot at the surface of the conical mirror, and after being reflected by the surface of the conical mirror, the incident light rays passing through the corresponding virtual camera optical center are formed and fall on the imaging plane of the virtual camera, forming the corresponding pixel points A on the secondary imaging image n .

[0085] S520. Connect the pixel point A n and the corresponding optical center coordinate O n to obtain the incident sub-ray

[0086] In some embodiments, in step S520, the vector parameter of the incident sub-ray is (X An , Y An , Z An) Obtained by calculating through the first set of formulas, the first set of formulas is as follows:

[0087] X An =(u An -u 0n )×dx n

[0088] Y An =(v An -v 0n )×dy n

[0089] Z An =f n

[0090] Wherein, n represents the corresponding virtual camera serial number; f n represents the focal length of the corresponding virtual camera, (u 0n , v 0n ) represents the origin parameters of the corresponding secondary imaging image; (u An , v An ) represents the pixel coordinate parameters of pixel point A n in the corresponding secondary imaging image; (dx n , dy n ) is the single pixel length in the X direction and Y direction in the corresponding virtual camera; the focal length f n and the optical center coordinates (u 0n , v 0n ) are the internal parameters of the virtual camera, which have been obtained in the above steps, and (u An , v An ) are the pixel coordinate parameters in the secondary imaging image, while (dx n , dy n ) is the actual length of a single pixel in the industrial camera. The industrial camera is produced according to specific specifications, and the corresponding parameters can be directly queried.

[0091] S530. Unify the coordinate parameters of each incident sub-ray .

[0092] In some embodiments, in step S530, the coordinate parameters of each incident sub-ray are unified through the second formula. Specifically, the camera coordinate system of the virtual camera with serial number 1 is selected as the world coordinate system, and the camera coordinate systems of other virtual cameras are converted into the world coordinate system through the second formula. The second formula is as follows:

[0093] α n =R n,n+1 α n+1 +t n,n+1

[0094] Among them, n and n + 1 represent the corresponding virtual camera numbers, where n = 1, 2, 3; α n represents the coordinate parameters of the camera coordinate system of the corresponding virtual camera; R n,n+1 represents the rotation matrix between the virtual camera with number n and the virtual camera with number n + 1; t n,n+1 represents the translation matrix between the virtual camera with number n and the virtual camera with number n + 1;

[0095] Once again, it is clear that each virtual camera has its own corresponding camera coordinate system. For example, virtual camera 1 corresponds to camera coordinate system 1, virtual camera 2 corresponds to camera coordinate system 2, and so on. Therefore, the coordinate parameters provided on virtual camera 2 are all based on camera coordinate system 2. Define the camera coordinate system 1 of camera 1 as the world coordinate system, which serves as the global reference framework to uniformly describe the positions and orientations of all virtual cameras. In order to unify the camera coordinate systems of virtual cameras with numbers other than 1 to the camera coordinate system of virtual camera 1 (i.e., the world coordinate system), coordinate transformation needs to be performed on the camera coordinate systems of different virtual cameras. Specifically, it is to transform the camera coordinate system of each virtual camera, through rotation and translation, into the camera coordinate system 1 of virtual camera 1. To elaborate further, it is to convert the coordinate parameters of virtual camera 2, virtual camera 3, and virtual camera 4 into the coordinate parameters of camera coordinate system 1. For example, when converting the coordinate parameters of virtual camera 2 into the coordinate parameters of camera coordinate system 1, the second formula becomes α1 = R 1,2 α2 + t 1,2 . To simplify the form, homogeneous coordinates are introduced and the formula is transformed as follows:

[0096]

[0097] For simplicity, we abbreviate the homogeneous coordinates of α1 and α2 as α1 and α2, and denote the matrix on the right side of the equation as T 12 , then we obtain α1 = T 12 α2. Similarly, we can get α2 = T 23 α3, α3 = T 34 α4; the coordinate parameters in virtual camera 3 can be transformed into α1 = T 12 T 23 α3, and the coordinate parameters in virtual camera 4 can be transformed into α1 = T 12 T 23 T 34 α4. For example, for a point M in space, the coordinate parameters in camera coordinate system 4 are α m4 , then the coordinate parameters α m1 , α m1 based on camera coordinate system 1 are α 12 T 23 T 34 αm4 , all subsequent points M can be calculated from α m1 , thus completing the unification of the coordinate parameters of all points.

[0098] S540, the incident sub-ray After extension, there is a reflection point B on the surface of the conical mirror n , and the incident ray is calculated to obtain

[0099] In some embodiments, in step S540, the incident sub-ray extends s unit vectors towards the conical mirror side and intersects the surface of the conical mirror to form a reflection point B n , and through the third formula group and the expression of the conical mirror, the incident ray The third formula group is as follows:

[0100]

[0101] where n represents the serial number of the corresponding virtual camera, is the incident sub-ray, B n is the intersection point of the outgoing ray and the surface of the conical mirror, s is the solution coefficient, (p xn , p yn , p zn ) is the vector parameter of the unit vector ;

[0102] Refer to Figure 2 and Figure 3 , according to the third formula group, the vector parametric equation of can be obtained as follows:

[0103]

[0104] Substitute the vector parameter formula into the space expression of the conical mirror, solve to obtain the coefficient s, then the incident ray can be represented by the above vector parameters.

[0105] S550, the conical mirror surface has a conical generatrix passing through the reflection point B n Connect the vertex U of the conical mirror and the reflection point B to form a conical generatrix n

[0106] S560, calculate the unit vector n that passes through the reflection point B and is perpendicular to the conical generatrix and mark it as the normal of the specular reflection;

[0107] ​In some embodiments, in step S560, according to the fourth formula of the law of specular reflection, the normal vector is calculated. The vector parameters (l xn , l yn , l zn ) of the normal vector are obtained. The fourth formula is as follows:

[0108]

[0109] Wherein, is the unit vector perpendicular to the conical generatrix passing through the reflection point B n . The conical generatrix passing through the reflection point B represents the normal vector n and is perpendicular to the conical generatrix .

[0110] S570. Calculate the outgoing ray based on the incident ray and the normal vector

[0111] according to the law of reflection. In step S570, the outgoing ray is calculated through the sixth formula of the vector reflection law. The sixth formula is as follows:

[0112]

[0113] Wherein, is the outgoing ray from the point C to be measured towards the surface of the conical mirror, is the incident ray that the outgoing ray enters the corresponding virtual camera optical center after passing through the reflection point B n , is the symmetry axis of the incident ray and the outgoing ray ;

[0114] For reference, see Figure 3 . The specular reflection law in vector form is a mathematical expression based on the law of specular reflection of light (the angle of incidence is equal to the angle of reflection). In addition, it should be supplemented that the vector calculated by this formula for the outgoing ray is a vector that passes through the reflection point B n , has the same direction as , but different lengths.

[0115] S580. Calculate the intersection point of the n outgoing rays , which is the point C to be measured, to obtain the spatial position of the point C to be measured;

[0116] In some embodiments, in step S580, the vector intersection point, i.e., the point C to be measured, is calculated through the seventh formula group of the vector intersection point rule. The seventh formula group is as follows:

[0117] x Bn +h n *x rn =x Bn+1 +h n+1 *x rn+1

[0118] y Bn +h n *y rn =y Bn+1 +h n+1 *y rn+1

[0119] z Bn +h n *z rn =z Bn+1 +h n+1 *z rn+1

[0120] Wherein, n and n + 1 represent the corresponding virtual camera numbers, n = 1, 2, 3; (x rn , y rn , z rn ) is the unit vector parameter of the outgoing light ray . (x Bn , y Bn , z Bn ) is the coordinate parameter of the refraction point B n , and h n is the solution coefficient;

[0121] The above formula is the coordinate conversion formula of the point C to be measured. That is, the point C to be measured can be represented by the outgoing light ray as the unit vector. Specifically, the coordinate parameter of the point C to be measured is (x Bn +h n *x rn , y Bn +h n *y rn , z Bn +h n *z rn ). By solving the system of binary linear equations composed of three equations, h n and h n+1 are obtained. If n = 2, it can be understood that in the world coordinate system, the point C to be measured is the landing point after extending h2 unit vectors along the direction of the outgoing light ray from the point B2.

[0122] It should be noted that the above-mentioned outgoing light ray Incident light Generatrix of the cone and incident sub-rays etc., only represent the light rays of the corresponding virtual camera, and the parameters of each vector are based on the world coordinate system, that is, based on the camera coordinate system 1. For example, the incident light represents the incident light corresponding to the virtual camera 2, but after coordinate transformation, its corresponding coordinate parameters are marked based on the world coordinate system.

[0123] S600. Calculate the error between the iterative spacing and the standard spacing. The standard spacing is the actual spacing between adjacent corner points P on the checkerboard calibration plate, and the iterative spacing is the spacing between adjacent corner points P in step S500; among them, the standard spacing has been planned during the production of the checkerboard calibration plate and belongs to known parameters.

[0124] S700. Input the target parameters and the objective function into the optimization function. After being processed by the optimization function, output the optimized value as the target parameter. Among them, the objective function is based on the target parameters, the internal and external parameters of each virtual camera, and performs the calculations in steps S400, S500, and S600, and outputs the error value;

[0125] Among them, the optimization function adopts the Levenberg-Marquardt algorithm, and optimizes and adjusts the target parameters through the L-M algorithm. Because the objective function calculates the error of the model based on the target parameters, the L-M algorithm adjusts the target parameters to minimize this error and finally outputs the optimized value;

[0126] In the present invention, the error of the model refers to the error between the iterative spacing and the standard spacing. When the optimization function processes, first assign an initial value as the target parameter. The objective function performs steps S400, S500, and S600, calculates and outputs the error, then the L-M algorithm adjusts the target parameter, returns to the objective function to re-execute these three steps, calculates and outputs a new error, and so on until the error is 0, that is, there is no error between the iterative spacing and the standard spacing, and the calculated corner point P coincides with the actual corner point P. At this time, the corresponding target parameter is output as the optimized value. Thus, the optimized value as the target parameter can accurately describe the actual spatial position of the conical mirror, making the reconstructed surface shape more accurate. Compared with the prior art where the conical mirror needs to be strictly below the camera and has disadvantages such as high error, in the present invention, not only the measurement accuracy is improved, but also the measurement efficiency is improved because strict calibration is not required.

[0127] S800. Take all the points on the checkerboard calibration board as the points to be measured C, and calculate them respectively through the ray tracing method to obtain the true spatial positions of all the points. The ray tracing method here is the same as the one used in step S500. And all the points refer to all the positions on the checkerboard calibration board. Each position has a corresponding pixel point in the four secondary imaging images. Then, according to the ray tracing method, obtain the true point positions corresponding to every four pixel points, and then realize pixel-by-pixel reconstruction to obtain the complete inner wall shape of the pipeline. It should be added that in this embodiment, four virtual cameras are virtually formed. In actual applications, multiple virtual cameras can be virtually formed.

[0128] In the description of this specification, terms such as "connection", "installation", "fixation", etc. should all be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0129] In the description of this specification, the description of terms such as "one embodiment", "some embodiments", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0130] The above is only the preferred embodiment of this application and is not used to limit this application. For those skilled in the art, this application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

1. A method for panoramic reconstruction of light ray tracing based on a virtual multi-purpose conical mirror, characterized in that, Specifically, it includes the following steps: S100: Place the calibration board on the optical platform, set an industrial camera above the calibration board, virtually form several virtual cameras from the industrial camera by the virtual multi-camera method, and label serial numbers for each of the virtual cameras; S200: Calibrate each of the virtual cameras by the Zhang Zhengyou calibration method to obtain the internal parameters of each of the virtual cameras and the external parameters between the virtual cameras with adjacent serial numbers; S300: Establish a measurement system, place the pipeline to be measured directly below the industrial camera, attach a checkerboard calibration board to the inner wall of the pipeline, and place a conical reflector at the central position inside the pipeline; S400: Use each of the virtual cameras to photograph the conical reflector. The conical reflector has a spatial coordinate expression, and the spatial coordinate expression includes target parameters. The target parameters include the coordinates of the vertex of the conical reflector, the direction vector of the central axis of the conical reflector, and the cone angle of the conical reflector; S500: Take each corner point P of the checkerboard calibration board as the point to be measured C, and calculate the spatial positions of each of the corner points P respectively through the ray tracing method; S600: Calculate the error between the iterative distance and the standard distance. The standard distance is the true distance between adjacent corner points P on the checkerboard calibration board, and the iterative distance is the distance between adjacent corner points P in step S500; S700: Input the target parameters and the objective function into the optimization function. After being processed by the optimization function, output the optimized value as the target parameter. Among them, the objective function is to perform the calculations in step S400, step S500, and step S600 based on the target parameters, the internal parameters and external parameters of each of the virtual cameras, and output the error value; S800: Take all points of the checkerboard calibration board as the points to be measured C, and calculate the true spatial positions of all points respectively through the ray tracing method.

2. The panoramic reconstruction method based on virtual multi-purpose conical mirror ray tracing according to claim 1, wherein In the step S100, the virtual multi-camera method specifically includes the following steps: S110: Photograph the calibration board by the industrial camera to obtain a primary imaging image containing the calibration board; S120: Divide the primary image into four secondary imaging images with equal pixel sizes and each containing the calibration board according to the upper left corner, upper right corner, lower left corner, and lower right corner respectively; S130: The industrial camera forms four virtual cameras according to the four secondary imaging images, corresponding to the four secondary imaging images respectively.

3. The panoramic reconstruction method based on virtual multi-purpose conical mirror ray tracing according to claim 2, wherein, In the step S200, the internal parameters of each of the virtual cameras are respectively the focal length f n and the optical center coordinates (u 0n , v 0n ). The external parameters between the virtual cameras are respectively the rotation matrix R n，n+1 and the translation matrix t n，n+1 ; where n and n + 1 represent the corresponding serial numbers of the virtual cameras, and n = 1, 2, 3.

4. The panoramic reconstruction method based on virtual multi-purpose conical mirror ray tracing according to claim 3, characterized in that In the step S400, the spatial coordinate expression of the conical reflector is as follows: ((x - x0)a + (y - y0)b + (z - z0)c) 2 = cos 2 (θ)((x - x0) 2 + (y - y0) 2 + (z - z0) 2 ) Among them, (x, y, z) represents the coordinate parameters of any point on the surface of the conical mirror in space, and (x0, y0, z0) represents the coordinate parameters of the vertex U of the conical mirror. represents the direction vector parameter of the central axis of the conical mirror, and θ is the cone angle parameter of the conical mirror.

5. The panoramic reconstruction method based on virtual multi-purpose cone mirror ray tracing according to claim 4, characterized in that In the step S500, the ray tracing method specifically includes the following steps: On the secondary imaging images of the four virtual cameras corresponding to the measurement point C, there are corresponding pixel points A respectively n , where n represents the serial number of the virtual camera, and n = 1, 2, 3, 4; S520. Connect the pixel point A n and the corresponding optical center coordinate O n to obtain the incident sub-ray S530. Unify the coordinate parameters of each of the incident sub-rays ; S540. The incident sub-ray has a reflection point B with the surface of the conical mirror after extension n , and the incident ray is obtained by calculation S550. The surface of the conical mirror has a conical generatrix passing through the reflection point B n of the conical generatrix Connect the vertex U of the conical mirror and the reflection point B n to form the conical generatrix S560. Calculate and obtain the unit vector that passes through the reflection point B n and is perpendicular to the conical generatrix and mark it as the normal line of specular reflection; ​ S570. The outgoing light is calculated according to the law of reflection from the incident light and the normal ​ S580. Calculate the intersection points of the n outgoing light rays to obtain the spatial position of the point C to be measured, which is the intersection point of the n outgoing light rays.

6. The panoramic reconstruction method based on virtual multi-purpose conical mirror ray tracing according to claim 5, wherein In the step S520, the incident sub-ray has a vector parameter of (X An , Y An , Z An ), which is obtained by calculating through the first set of formulas as follows: X An = (u An - u 0n ) × dx n Y An = (v An - v 0n ) × dy n Z An = f n Among them, n represents the corresponding virtual camera serial number; f n represents the focal length of the corresponding virtual camera, (u 0n , v 0n ) represents the origin parameters of the corresponding secondary imaging image; (u An , v An ) represents the pixel coordinate parameters of the pixel point A n in the corresponding secondary imaging image; (dx n , dy n ) are the single pixel lengths in the X direction and Y direction in the corresponding virtual camera.

7. The panoramic reconstruction method based on virtual multi-purpose conical mirror ray tracing according to claim 6, wherein In the step S530, the coordinate parameters of each of the incident sub-rays are unified by a second formula Specifically, the camera coordinate system of the virtual camera numbered 1 is selected as the world coordinate system, and the camera coordinate systems of other virtual cameras are converted into the world coordinate system by the second formula. The second formula is as follows: α n = R n,n+1 α n+1 + t n,n+1 Among them, n and n + 1 represent the corresponding virtual camera serial numbers, where n = 1, 2, 3; α n represents the coordinate parameters of the camera coordinate system of the corresponding virtual camera; R n,n+1 represents the rotation matrix between the virtual camera with serial number n and the virtual camera with serial number n + 1; t n,n+1 represents the translation matrix between the virtual camera with serial number n and the virtual camera with serial number n + 1.

8. The method for panoramic reconstruction of virtual multi-purpose cone mirror ray tracing according to claim 7, wherein In the step S540, the incident sub-ray extends s unit vectors toward the conical mirror side and intersects with the surface of the conical mirror to form a reflection point B n , and through the third formula group and the expression of the conical mirror, the incident ray is calculated The third formula group is as follows: where n represents the serial number of the corresponding virtual camera, is the incident sub-ray, and B n is the intersection point of the outgoing ray and the surface of the conical mirror, s is the solution coefficient, and (p xn , p yn , p zn ) is the unit vector vector parameter.

9. The panoramic reconstruction method based on virtual multi-purpose conical mirror ray tracing according to claim 8, characterized in that, In the step S560, according to the fourth formula of the law of specular reflection, the normal line is calculated and obtained with vector parameters (l xn , l yn , l zn ), and the fourth formula is as follows: Among them, is the unit vector perpendicular to the conical generatrix n passing through the reflection point B ; is the conical generatrix passing through the reflection point B n . In the step S570, the outgoing light is calculated through the sixth formula of the vector reflection law The sixth formula is as follows: Among them, is the outgoing ray from the measurement point C towards the surface of the conical mirror, is the outgoing ray after passing through the reflection point B n and then entering the incident ray of the corresponding virtual camera optical center, is the incident ray and the outgoing ray is the axis of symmetry.

10. The panoramic reconstruction method based on virtual multi-purpose conical mirror ray tracing according to claim 9, wherein In the step S580, calculate the vector intersection point, that is, the point to be measured C, through the seventh formula group of the vector intersection rule. The seventh formula group is as follows: x Bn +h n *x rn =x Bn+1 +h n+1 *x rn+1 y Bn +h n *y rn =y Bn+1 +h n+1 *y rn+1 z Bn +h n *z rn =z Bn+1 +h n+1 *z rn+1 Among them, n and n + 1 represent the corresponding virtual camera numbers, where n = 1, 2, 3; (x rn , y rn , z rn ) is the vector parameter of the outgoing light ray ; (x Bn , y Bn , z Bn ) is the coordinate parameter of the refraction point B n , and h n is the solution coefficient.

Images