A panoramic reconstruction method based on ray tracing of virtual multi-purpose conic reflectors

Abstract

The present invention discloses a panoramic reconstruction method based on ray tracing of a virtual multi-object conical reflector. The method introduces an optimization function and a virtual multi-objective method. When processing through the optimization function, an initial value is first assigned as a target parameter, and an error is output after calculation by the target function. The target parameter is then adjusted by the L-M algorithm, and the target function is returned to calculate and a new error is output. This iteration is repeated until the error is 0, indicating that the calculated corner point P coincides with the actual corner point P. At this time, the corresponding target parameter is output as an optimized value. Therefore, the optimized value as the target parameter can accurately describe the actual spatial position of the conical reflector, making the reconstructed surface shape more accurate. Compared with the prior art that requires the conical reflector to be strictly located below the camera and has disadvantages such as higher error, the present invention not only improves the measurement accuracy, but also improves the measurement efficiency because it does not require strict calibration.

Description

Technical Field

[0001] The present invention relates to the technical field of surface reconstruction, and in particular to a panoramic reconstruction method based on ray tracing of virtual multi-purpose conic reflectors. Background Art

[0002] With the continuous development of modern manufacturing, the demand for high-precision three-dimensional measurement technology in various fields is increasing. Existing pipeline inner wall measurement mainly relies on contact and non-contact methods. Contact methods such as micrometers and three-dimensional coordinate machines rely on manual operation or mechanical probes, and have problems such as low efficiency and limited measurement depth. Non-contact technologies such as ultrasonic testing and laser vision measurement can obtain local features, but are limited to single-point or single-angle detection, making it difficult to achieve full-area three-dimensional reconstruction of the inner wall. In addition, existing three-dimensional measurement solutions often use multi-device splicing or rotational scanning. The former requires multiple laser / vision sensors to be arranged at intervals, while the latter relies on a high-precision rotating shaft system. Splicing is prone to error accumulation at the seams, and rotational scanning faces problems such as axis reference drift, difficult cross-section correction, and lack of closed-loop calibration.

[0003] One method of partially reconstructing the inner wall is to use a camera, place a conical mirror inside the pipe, and perform surface reconstruction by adding photos. However, this method requires the central axis of the conical mirror to be directly below the optical center of the camera, which is a harsh condition and has low measurement accuracy. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a panoramic reconstruction method based on virtual multi-purpose conic reflectors and ray tracing.

[0005] The present invention provides a panoramic reconstruction method based on virtual multi-purpose conic reflector ray tracing, which specifically includes the following steps:

[0006] S100, placing a calibration plate on an optical platform, setting an industrial camera above the calibration plate, virtualizing the industrial camera into a plurality of virtual cameras using a virtual multi-camera method, and marking a serial number on each of the virtual cameras;

[0007] S200, calibrating each of the virtual cameras using the Zhang Zhengyou calibration method to obtain an intrinsic parameter of each of the virtual cameras and an extrinsic parameter between the virtual cameras with adjacent serial numbers;

[0008] S300, establishing a measurement system, placing a pipe to be measured directly below the industrial camera, attaching a checkerboard calibration plate to the inner wall of the pipe, and placing a conical reflector at the center of the pipe;

[0009] S400, using each of the virtual cameras to photograph the conical reflector, wherein the conical reflector has a spatial coordinate expression, the spatial coordinate expression includes target parameters, and the target parameters include the coordinates of the conical reflector vertex, the direction vector of the conical reflector central axis, and the cone angle of the conical reflector;

[0010] S500, taking each corner point P of the checkerboard calibration plate as a point to be measured C, and calculating the spatial position of each corner point P using a ray tracing method;

[0011] S600, calculating and obtaining an error between an iterative spacing and a standard spacing, wherein the standard spacing is a true spacing between adjacent corner points P in the checkerboard calibration plate, and the iterative spacing is a spacing between adjacent corner points P in step S500;

[0012] S700, inputting target parameters and an objective function into the optimization function, and outputting an optimized value as the target parameter after processing by the optimization function, wherein the objective function is based on the target parameters, the intrinsic parameters and the extrinsic parameters of each virtual camera, performing the calculations of steps S400, S500 and S600, and outputting an error value;

[0013] S800: All points of the checkerboard calibration plate are used as points to be measured C, and are calculated using a ray tracing method to obtain the real spatial positions of all points.

[0014] Preferably, in step S100, the virtual multi-user method specifically includes the following steps:

[0015] S110, photographing the calibration plate with the industrial camera to obtain a primary imaging image including the calibration plate;

[0016] S120, dividing the primary imaging image into four secondary imaging images of equal pixel size and each containing the calibration plate according to the upper left corner, the upper right corner, the lower left corner, and the lower right corner;

[0017] S130: The industrial camera forms four virtual cameras according to the four secondary imaging images, respectively corresponding to the four secondary imaging images.

[0018] Preferably, in step S200, the internal parameters of each virtual camera are focal length f n and the optical center coordinates (u 0n ,v 0n ), the external parameters between the virtual cameras are the rotation matrix R n，n+1 and the translation matrix t n，n+1 ; Among them, n and n+1 represent the corresponding virtual camera numbers.

[0019] Preferably, in step S400, the spatial coordinate expression of the conical reflector is as follows:

[0020] ((x-x0)a+(y-y0)b+(z-z0)c) 2 =cos 2 (θ)((x-x0) 2 +(y-y0) 2 +(z-z0) 2 )

[0021] Wherein, (x, y, z) represents the coordinate parameters of any point on the surface of the conical reflector in space, (x0, y0, z0) represents the coordinate parameters of the vertex U of the conical reflector, represents the direction vector parameter of the central axis of the conical reflector, and θ is the cone angle parameter of the conical reflector.

[0022] Preferably, in step S500, the ray tracing method specifically includes the following steps:

[0023] S510: The point C to be measured corresponds to the four secondary imaging images of the virtual cameras, and there are corresponding pixel points A respectively. n , n represents the serial number of the virtual camera;

[0024] S520, connecting the pixel point A n and the corresponding optical center coordinate O n , get the incident sub-ray

[0025] S530, each incident sub-light The coordinate parameters are unified;

[0026] S540, the incident sub-light After extension, there is a reflection point B between the surface of the conical reflector and the conical reflector. n , calculate the incident light

[0027] S550, the conical reflector surface has a portion passing through the reflection point B n Conical generatrix Connect the vertex U of the conical reflector and the reflection point B n Forming the conical generatrix

[0028] S560, calculate and obtain the value passing through the reflection point B n And with the conical generatrix Perpendicular unit vector Labeled as the normal for specular reflection;

[0029] S570, the incident light and the normal Calculate the outgoing light according to the law of reflection

[0030] S580, calculating n outgoing rays The intersection point of the two points is the point C to be measured, and the spatial position of the point C to be measured is obtained.

[0031] Preferably, in step S520, the incident sub-light The vector parameter is (X An ,Y An ,Z An ) is calculated by the first formula group, which is as follows:

[0032] X An =(u An -u 0n )×dx n

[0033] Y An =(v An -v 0n )×dy n

[0034] Z An =f n

[0035] Among them, n represents the corresponding virtual camera number; f n Represents the focal length of the corresponding virtual camera; (u An ,v An ) represents the pixel A in the corresponding secondary imaging image n Pixel coordinate parameters; (dx n ,dy n ) corresponds to the length of a single pixel in the virtual camera along the X and Y directions.

[0036] Preferably, in step S530, each incident sub-ray is converted into Specifically, the camera coordinate system of the virtual camera with sequence number 1 is selected and marked as the world coordinate system, and the camera coordinate systems of other virtual cameras are converted into the world coordinate system using the second formula. The second formula is as follows:

[0037] α n =R n,n+1 α n+1 +t n,n+1

[0038] Among them, n and n+1 represent the corresponding virtual camera numbers; α nRepresents the coordinate parameters of the camera coordinate system corresponding to the virtual camera; R n,n+1 Represents the rotation matrix between the virtual camera with sequence number n and the virtual camera with sequence number n+1; t n,n+1 Represents the translation matrix between the virtual camera with sequence number n and the virtual camera with sequence number n+1.

[0039] Preferably, in step S540, the incident sub-light Extend s unit vectors toward the conical reflector side Intersecting with the conical reflector surface to form a reflection point B n , through the third formula group and the conical reflector expression, the incident light is calculated The third formula group is as follows:

[0040]

[0041] Wherein, n represents the serial number of the corresponding virtual camera, is the incident sub-ray, B n is the intersection point of the outgoing light and the conical reflector surface, s is the solution coefficient, (p xn ,p yn ,p zn ) is a unit vector The vector parameter of .

[0042] Preferably, in step S560, the normal is calculated according to the fourth formula of the law of mirror reflection. The vector parameter (l xn ,l yn ,l zn ), the fourth formula is as follows:

[0043]

[0044] in, It passes through reflection point B n With cone busbar The perpendicular unit vector, Passing through reflection point B n The cone generatrix of

[0045] In step S570, the outgoing light is calculated by the sixth formula of the vector reflection law. The sixth formula is as follows:

[0046]

[0047] in, is the outgoing light from the measured point C toward the surface of the conical reflector, is the outgoing light Passing through reflection point B n The incident light that enters the optical center of the virtual camera is is the incident light and outgoing light axis of symmetry.

[0048] Preferably, in step S580, the vector intersection point, i.e., the point to be measured C, is obtained by calculation using the seventh formula group of the vector intersection rule. The seventh formula group is as follows:

[0049] x Bn +h n *x rn =x Bn+1 +h n+1 *x rn+1

[0050] yB n +h n *y rn =y Bn+1 +h n+1 *y rn+1

[0051] z Bn +h n *z rn =z Bn+1 +h n+1 *z rn+1

[0052] Among them, n and n+1 represent the corresponding virtual camera numbers; (x rn ,y rn ,z rn ) is the outgoing light Vector parameters; (x Bn ,y Bn ,z Bn ) is the refraction point B n Coordinate parameters, h n To solve the coefficient.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention introduces an optimization function and a virtual multi-objective method. When processing through the optimization function, an initial value is first assigned as a target parameter, the error is output after calculation by the target function, and then the target parameter is adjusted by the LM algorithm. The target function is returned to calculate and a new error is output. This iteration is repeated until the error is 0, indicating that the calculated corner point P coincides with the actual corner point P. At this time, the corresponding target parameter is output as the optimized value, so that the optimized value as the target parameter can accurately describe the actual spatial position of the conical reflector, making the reconstructed surface shape more accurate. Compared with the prior art that requires the conical reflector to be strictly located below the camera, which has disadvantages such as higher errors, the present invention not only improves the measurement accuracy, but also improves the measurement efficiency because it does not require strict proofreading.

[0055] In addition, based on the use of one industrial camera, the present invention uses a virtual multi-camera method to virtually form multiple virtual cameras from the industrial camera, shoots the conical reflector from different directions, and reconstructs the surface shape of the inner wall of the pipeline pixel by pixel from the captured secondary imaging image through ray tracing and light reflection rules, thereby improving the accuracy of the reconstruction of the inner wall of the pipeline. Multiple virtual cameras can quickly reconstruct the entire circumference at one time, which has the advantages of fast speed and high precision. Compared with the high cost of using multiple cameras, the present invention only uses one industrial camera, which is low in cost and easy to promote and use.

[0056] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0058] Figure 1 A flowchart of a method for panoramic reconstruction based on ray tracing of a virtual multi-purpose conic reflector provided in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of ray tracing in a panoramic reconstruction method based on virtual multi-purpose conic reflectors provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of mirror reflection in a panoramic reconstruction method based on ray tracing of a virtual multi-purpose conic reflector provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of the segmentation of a secondary imaging image in a panoramic reconstruction method based on virtual multi-purpose conic reflector ray tracing provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0063] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0064] Please refer to Figures 1 to 4 The embodiment of the present invention provides a panoramic reconstruction method based on virtual multi-purpose conic reflector ray tracing, which specifically includes the following steps:

[0065] S100, placing a calibration plate on an optical platform, setting an industrial camera above the calibration plate, virtualizing the industrial camera into several virtual cameras using a virtual multi-camera method, and marking each virtual camera with a serial number;

[0066] In some embodiments, in step S100, the virtual multi-user method specifically includes the following steps:

[0067] S110, photographing the calibration plate with an industrial camera to obtain a primary imaging image including the calibration plate;

[0068] S120, dividing the primary imaging image into four secondary imaging images of equal pixel size and each containing a calibration plate according to the upper left corner, the upper right corner, the lower left corner, and the lower right corner;

[0069] S130, the industrial camera forms four virtual cameras according to the four secondary imaging images, respectively corresponding to the four secondary imaging images;

[0070] Among them, the industrial camera is a physical camera, which can be used as multiple virtual cameras through the above virtual multi-camera method. Furthermore, the virtual multi-camera method is based on virtual camera technology, which is a technical method that divides the field of view of a physical camera into multiple virtual camera views, each view corresponds to a virtual camera, thereby virtualizing multiple virtual cameras. Figure 4 , with the four corners as the corners of the four virtual cameras, four secondary imaging images with equal pixel size and all containing the calibration plate are formed. In this embodiment, the industrial camera is virtualized into four virtual cameras, corresponding to the four secondary imaging images respectively, and the sequence is virtual camera 1, virtual camera 2, virtual camera 3 and virtual camera 4.

[0071] S200, calibrating each virtual camera using the Zhang Zhengyou calibration method to obtain the intrinsic parameters of each virtual camera and the extrinsic parameters between virtual cameras with adjacent serial numbers;

[0072] Among them, the Zhang Zhengyou calibration method is a camera calibration method based on a planar chessboard proposed by Professor Zhang Zhengyou in 1998. This method takes chessboard images at different angles and combines the homography matrix with nonlinear optimization technology to solve the camera's intrinsic parameters, extrinsic parameters and distortion coefficients. It combines the high precision of traditional calibration methods with the convenience of self-calibration methods, and has become a classic method in the field of computer vision.

[0073] In some embodiments, in step S200, the internal parameters of each virtual camera are focal length f n and the optical center coordinates (u 0n ,v 0n ), the external parameters between each virtual camera are the rotation matrix R n，n+1 and the translation matrix t n，n+1 ; Among them, n and n+1 represent the corresponding virtual camera numbers;

[0074] For ease of understanding, let us take an example to illustrate that the internal parameters of the virtual camera 1 are the focal length f1 and the optical center coordinates (u 01 ,v 01 ); The position relationship between virtual camera 1 and virtual camera 2, that is, the external parameter is expressed as the rotation matrix R 1,2 and the translation matrix t 1,2 These parameters are obtained by calibration based on the secondary imaging image using Zhang Zhengyou calibration method. The specific process is existing technology and will not be described here.

[0075] S300: Establish a measurement system, place the pipe to be measured directly below the industrial camera, attach a checkerboard calibration plate to the inner wall of the pipe, and place a conical reflector at the center of the pipe.

[0076] Among them, because the pipeline is placed directly below the industrial camera and the conical reflector is placed at the center of the pipeline, the central axis of the pipeline, the central axis of the conical reflector and the optical axis of the industrial camera coincide with each other. However, due to inevitable deviations in actual operation, the accuracy of the coincidence cannot be guaranteed. Therefore, when establishing the measurement system, it is only necessary to keep the three coincident as much as possible. The present invention obtains the actual position of the conical reflector through subsequent optimization function optimization, and then obtains the reconstructed surface shape of the inner wall of the pipeline.

[0077] S400, using each of the virtual cameras to photograph the conical reflector, wherein the conical reflector has a spatial coordinate expression, the spatial coordinate expression includes target parameters, and the target parameters include the coordinates of the conical reflector vertex, the direction vector of the conical reflector central axis, and the cone angle of the conical reflector;

[0078] In some embodiments, in step S400, the spatial coordinate expression of the conical reflector is as follows:

[0079] ((x-x0)a+(y-y0)b+(z-z0)c) 2 =cos 2 (θ)((x-x0) 2 +(y-y0) 2 +(z-z0) 2 )

[0080] Among them, (x, y, z) represents the coordinate parameters of any point on the surface of the conical reflector in space, (x0, y0, z0) represents the coordinate parameters of the vertex U of the conical reflector, Represents the direction vector parameter of the central axis of the conical reflector, θ is the cone angle parameter of the conical reflector, and the initial value includes the coordinate parameter of the vertex U, the direction vector parameter of the central axis and the cone angle parameter of the conical reflector;

[0081] First of all, it is clear that each virtual camera has its own corresponding camera coordinate system. For example, virtual camera 1 corresponds to camera coordinate system 1, virtual camera 2 corresponds to camera coordinate system 2, and so on. The above-mentioned spatial coordinate expression of the conical reflector is described based on camera coordinate system 1.

[0082] S500, taking each corner point P of the checkerboard calibration plate as a point to be measured C, and calculating the spatial position of each corner point P using a ray tracing method;

[0083] In some embodiments, in step S500, reference may be made to Figure 2 and Figure 3 , and the ray tracing method specifically includes the following steps:

[0084] S510, the point to be measured C corresponds to the secondary imaging images of the four virtual cameras and has corresponding pixel points A respectively. n , n represents the serial number of the virtual camera; refer to Figure 2 The light emitted from the test point C is emitted as the outgoing light to the conical reflector surface, and is reflected by the conical reflector surface to form the incident light passing through the optical center of the corresponding virtual camera, and falls on the imaging plane of the virtual camera to form the corresponding pixel point A on the secondary imaging image. n .

[0085] S520, connect pixel point A n and the corresponding optical center coordinates O n , get the incident sub-ray

[0086] In some embodiments, in step S520, the incident sub-ray The vector parameter is (X An ,Y An ,Z An ) is calculated by the first formula group, which is as follows:

[0087] X An =(u An -u 0n )×dx n

[0088] Y An =(v An -v 0n )×dy n

[0089] Z An =f n

[0090] Among them, n represents the corresponding virtual camera number; f n Represents the focal length of the corresponding virtual camera; (u An ,v An ) represents the pixel A in the corresponding secondary imaging image n Pixel coordinate parameters; (dx n ,dy n ) is the length of a single pixel in the X and Y directions of the corresponding virtual camera; the focal length f n and the optical center coordinates (u 0n ,v 0n ) is the virtual camera internal parameter, which has been obtained in the above steps, and (u An ,v An ) is the pixel coordinate parameter in the secondary imaging image, and (dx n ,dy n ) is the actual length of a single pixel in an industrial camera. Industrial cameras are manufactured to specific specifications and can be found by directly querying the corresponding parameters.

[0091] S530, each incident sub-ray The coordinate parameters are unified;

[0092] In some embodiments, in step S530, each incident sub-ray is converted into Specifically, the camera coordinate system of the virtual camera with sequence number 1 is marked as the world coordinate system, and the camera coordinate systems of other virtual cameras are converted into the world coordinate system through the second formula. The second formula is as follows:

[0093] α n =R n,n+1 α n+1 +t n,n+1

[0094] Among them, n and n+1 represent the corresponding virtual camera numbers; α n Represents the coordinate parameters of the camera coordinate system corresponding to the virtual camera; R n,n+1Represents the rotation matrix between the virtual camera with sequence number n and the virtual camera with sequence number n+1; t n,n+1 Represents the translation matrix between the virtual camera with sequence number n and the virtual camera with sequence number n+1;

[0095] Once again, it is clear that each virtual camera has its own corresponding camera coordinate system. For example, virtual camera 1 corresponds to camera coordinate system 1, virtual camera 2 corresponds to camera coordinate system 2, and so on. Therefore, the coordinate parameters provided on virtual camera 2 are all based on camera coordinate system 2. The camera coordinate system 1 of camera 1 is defined as the world coordinate system. As a global reference frame, it is used to uniformly describe the position and orientation of all virtual cameras. In order to convert the camera coordinate system of virtual cameras whose serial number is not 1 to the camera coordinate system of virtual camera 1 (that is, the world coordinate system), it is necessary to perform coordinate transformation on the camera coordinate systems of different virtual cameras. Specifically, the camera coordinate system of each virtual camera is converted to the camera coordinate system 1 of virtual camera 1 through rotation and translation. To explain in detail, the coordinate parameters of virtual camera 2, virtual camera 3 and virtual camera 4 are all converted into the coordinate parameters of camera coordinate system 1. For example, the coordinate parameters of virtual camera 2 are converted into the coordinate parameters of camera coordinate system 1, and the second formula becomes α1=R 1,2 α2+t 1,2 , in order to simplify the form, homogeneous coordinates are introduced and the formula is transformed as follows:

[0096]

[0097] For simplicity, we abbreviate the homogeneous coordinates of α1 and α2 as α1 and α2, and denote the matrix on the right side of the equation as T 12 , then we get α1=T 12 α2, similarly, α2=T 23 α3, α3 = T 34 α4; The coordinate parameters in virtual camera 3 can be converted into α1=T 12 T 23 α3, the coordinate parameters of virtual camera 4 can be converted into α1=T 12 T 23 T 34 α4, for example, a point M in space has a coordinate parameter in the camera coordinate system 4 of α m4 , then the coordinate parameter α based on the camera coordinate system 1 m1 , α m1 =T 12 T 23 T 34 α m4 , the subsequent points M can be obtained by α m1 Perform calculations to unify the coordinate parameters of all points.

[0098] S540, incident sub-ray After extension, there is a reflection point B between the conical reflector surface and the conical reflector surface. n , calculate the incident light

[0099] In some embodiments, in step S540, the incident sub-ray Extend s unit vectors toward the conical reflector side Intersection with the conical reflector surface forms reflection point B n , through the third formula group and the conical reflector expression, the incident light is calculated The third formula group is as follows:

[0100]

[0101]

[0102] Among them, n represents the serial number of the corresponding virtual camera, is the incident sub-ray, B n is the intersection point of the outgoing light and the conical reflector surface, s is the solution coefficient, (p xn ,p yn ,p zn ) is a unit vector vector parameters of ;

[0103] refer to Figure 2 and Figure 3 , according to the third formula group, we can get The vector parameter formula is as follows:

[0104]

[0105] Substitute the vector parameter formula into the spatial expression of the conical reflector and solve to obtain the coefficient s, then the incident light It can be represented by the above vector parameters.

[0106] S550, the conical reflector surface has a reflection point B n Conical generatrix Connect the vertex U of the conical reflector and the reflection point B n Forming a conical generatrix

[0107] S560, calculate and obtain the reflection point B n And with the cone busbar Perpendicular unit vector Labeled as the normal for specular reflection;

[0108] In some embodiments, in step S560, the normal is calculated according to the fourth formula of the law of mirror reflection. The vector parameter (l xn ,l yn ,l zn ), the fourth formula is as follows:

[0109]

[0110] in, It passes through reflection point B n With cone busbar The perpendicular unit vector, Passing through reflection point B n The cone generatrix of With cone busbar Perpendicular to each other.

[0111] S570, by the incident light and normals Calculate the outgoing light according to the law of reflection

[0112] In step S570, the outgoing light is calculated by the sixth formula of the vector reflection law. The sixth formula is as follows:

[0113]

[0114] in, is the outgoing light from the measured point C toward the surface of the conical reflector, is the outgoing light Passing through reflection point B n The incident light that enters the optical center of the virtual camera is is the incident light and outgoing light The axis of symmetry;

[0115] For reference Figure 3 The vector form of the mirror reflection law is a mathematical expression based on the mirror reflection law of light (the angle of incidence is equal to the angle of reflection). In addition, it should be added that the vector calculated by the calculation formula of the outgoing light is It passes through reflection point B n , direction and Identical vectors, but of different lengths.

[0116] S580, calculate n outgoing rays The intersection point of is the point to be measured C, and the spatial position of the point to be measured C is obtained;

[0117] In some embodiments, in step S580, the vector intersection point, i.e., the point to be measured C, is calculated using the seventh formula group of the vector intersection rule. The seventh formula group is as follows:

[0118] x Bn +h n *x rn =x Bn+1 +h n+1 *x rn+1

[0119] y Bn +h n *y rn =y Bn+1 +h n+1 *y rn+1

[0120] z Bn +h n *z rn =z Bn+1 +h n+1 *z rn+1

[0121] Among them, n and n+1 represent the corresponding virtual camera numbers; (x rn ,y rn ,z rn ) is the outgoing light The unit vector parameter (x Bn ,y Bn ,z Bn ) is the refraction point B n Coordinate parameters, h n To solve the coefficient;

[0122] The above formula is the coordinate conversion formula of the point C to be measured, that is, the point C to be measured can be converted to the outgoing light. As a unit vector, the coordinate parameters of the point C to be measured are specifically (x Bn +h n *x rn ,y Bn +h n *y rn ,z Bn +h n *z rn ), solve the system of two-variable linear equations consisting of three equations to obtain h n and h n+1 If n = 2, it can be understood that in the world coordinate system, the point C to be measured is from point B2 along the outgoing ray The point where h2 unit vectors are extended in the direction of .

[0123] In particular, the above-mentioned outgoing light Incident light Conical busbar and the incident sub-ray etc., only represents the corresponding virtual camera light, and the parameters of each vector are based on the world coordinate system, that is, based on the camera coordinate system 1, such as the incident light Represents the incident light corresponding to virtual camera 2, but because of the coordinate conversion, its corresponding coordinate parameters are marked based on the world coordinate system.

[0124] S600. Calculate the error between the iterative spacing and the standard spacing, where the standard spacing is the actual spacing between adjacent corner points P in the checkerboard calibration plate, and the iterative spacing is the spacing between adjacent corner points P in step S500. The standard spacing is planned when the checkerboard calibration plate is produced and is a known parameter.

[0125] S700, inputting target parameters and an objective function into the optimization function, and outputting an optimized value as the target parameter after processing by the optimization function, wherein the objective function is based on the target parameters, the intrinsic parameters and the extrinsic parameters of each virtual camera, performing the calculations of steps S400, S500 and S600, and outputting an error value;

[0126] The optimization function uses the Levenberg-Marquardt algorithm and optimizes and adjusts the target parameters through the LM algorithm. Because the objective function calculates the model error based on the target parameters, the LM algorithm minimizes this error by adjusting the target parameters and ultimately outputs the optimized value.

[0127] In the present invention, the error of the model refers to the error between the iteration spacing and the standard spacing. When the optimization function is processed, an initial value is first assigned as the target parameter. The target function executes steps S400, S500 and S600, and the error is output after calculation. The target parameter is then adjusted by the LM algorithm, and the target function is returned to re-execute these three steps to calculate and output a new error. This iteration is repeated until the error is 0, that is, there is no error between the iteration spacing and the standard spacing, and the calculated corner point P coincides with the actual corner point P. At this time, the corresponding target parameter is output as the optimized value, so that the optimized value as the target parameter can accurately describe the actual spatial position of the conical reflector, making the reconstructed surface shape more accurate. Compared with the prior art, which requires the conical reflector to be strictly located below the camera and has disadvantages such as higher errors, the present invention not only improves the measurement accuracy, but also improves the measurement efficiency because it does not require strict calibration.

[0128] S800: All points on the checkerboard calibration plate are used as points C to be measured, and the real spatial positions of all points are calculated using the ray tracing method. The ray tracing method used here is the same as that used in step S500, and all points refer to all points on the checkerboard calibration plate. Each point has a corresponding pixel point in the four secondary imaging images. Then, the real point position corresponding to each four pixels is obtained according to the ray tracing method, and pixel-by-pixel reconstruction is achieved to obtain the complete inner wall shape of the pipeline. It should be noted that in this embodiment, four virtual cameras are virtually formed. In actual application, multiple virtual cameras can be virtually formed.

[0129] In this specification, the terms "connect," "install," and "fix" should be understood broadly. For example, "connect" can refer to a fixed connection, a detachable connection, or an integral connection; it can refer to a direct connection or an indirect connection through an intermediary. Those skilled in the art will understand the specific meanings of these terms in this application based on the specific circumstances.

[0130] Throughout this specification, terms such as "one embodiment" or "some embodiments" mean that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0131] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A panoramic reconstruction method based on virtual multi-purpose conic reflector ray tracing, characterized in that: The specific steps include: S100, placing a calibration plate on an optical platform, setting an industrial camera above the calibration plate, virtualizing the industrial camera into a plurality of virtual cameras using a virtual multi-camera method, and marking a serial number on each of the virtual cameras; S200, calibrating each of the virtual cameras using the Zhang Zhengyou calibration method to obtain an intrinsic parameter of each of the virtual cameras and an extrinsic parameter between the virtual cameras with adjacent serial numbers; S300, establishing a measurement system, placing a pipe to be measured directly below the industrial camera, attaching a checkerboard calibration plate to the inner wall of the pipe, and placing a conical reflector at the center of the pipe; S400, using each of the virtual cameras to photograph the conical reflector, wherein the conical reflector has a spatial coordinate expression, the spatial coordinate expression includes target parameters, and the target parameters include the coordinates of the conical reflector vertex, the direction vector of the conical reflector central axis, and the cone angle of the conical reflector; S500, taking each corner point P of the checkerboard calibration plate as a point to be measured C, and calculating the spatial position of each corner point P using a ray tracing method; S600, calculating and obtaining an error between an iterative spacing and a standard spacing, wherein the standard spacing is a true spacing between adjacent corner points P in the checkerboard calibration plate, and the iterative spacing is a spacing between adjacent corner points P in step S500; S700, inputting target parameters and an objective function into the optimization function, and outputting an optimized value as the target parameter after processing by the optimization function, wherein the objective function is based on the target parameters, the intrinsic parameters and the extrinsic parameters of each virtual camera, performing the calculations of steps S400, S500 and S600, and outputting an error value; S800: All points of the checkerboard calibration plate are used as points to be measured C, and are calculated using a ray tracing method to obtain the real spatial positions of all points.

2. The method for panoramic reconstruction based on virtual multi-purpose conic reflector ray tracing according to claim 1, characterized in that: In step S100, the virtual multi-user method specifically includes the following steps: S110, photographing the calibration plate with the industrial camera to obtain a primary imaging image including the calibration plate; S120, dividing the primary imaging image into four secondary imaging images of equal pixel size and each containing the calibration plate according to the upper left corner, the upper right corner, the lower left corner, and the lower right corner; S130: The industrial camera forms four virtual cameras according to the four secondary imaging images, respectively corresponding to the four secondary imaging images.

3. The method for panoramic reconstruction based on virtual multi-purpose conic reflector ray tracing according to claim 2, characterized in that: In step S200, the internal parameters of each virtual camera are focal length f n and the optical center coordinates (u 0n ,v 0n ), the external parameters between the virtual cameras are the rotation matrix R n，n+1 and the translation matrix t n，n+1 ; Among them, n and n+1 represent the corresponding virtual camera numbers.

4. The method for panoramic reconstruction based on virtual multi-purpose conic reflector ray tracing according to claim 3, characterized in that: In step S400, the spatial coordinate expression of the conical reflector is as follows: ((x-x0)a+(y-y0)b+(z-z0)c) 2 =cos 2 (θ)((x−x0) 2 +(y-y0) 2 +(z-z0) 2 ) Wherein, (x, y, z) represents the coordinate parameters of any point on the surface of the conical reflector in space, (x0, y0, z0) represents the coordinate parameters of the vertex U of the conical reflector, represents the direction vector parameter of the central axis of the conical reflector, and θ is the cone angle parameter of the conical reflector.

5. The method for panoramic reconstruction based on virtual multi-purpose conic reflector ray tracing according to claim 4, characterized in that: In step S500, the ray tracing method specifically includes the following steps: S510: The point C to be measured corresponds to the four secondary imaging images of the virtual cameras, and there are corresponding pixel points A respectively. n , n represents the serial number of the virtual camera; S520, connecting the pixel point A n and the corresponding optical center coordinate O n , get the incident sub-ray S530, each incident sub-light The coordinate parameters are unified; S540, the incident sub-light After extension, there is a reflection point B between the surface of the conical reflector and the conical reflector. n , calculate the incident light S550, the conical reflector surface has a portion passing through the reflection point B n Conical generatrix Connect the vertex U of the conical reflector and the reflection point B n Forming the conical generatrix S560, calculate and obtain the value passing through the reflection point B n And with the conical generatrix Perpendicular unit vector Labeled as the normal for specular reflection; S570, the incident light and the normal Calculate the outgoing light according to the law of reflection S580, calculating n outgoing rays The intersection point of the two points is the point C to be measured, and the spatial position of the point C to be measured is obtained.

6. The method for panoramic reconstruction based on virtual multi-purpose conic reflector ray tracing according to claim 5, characterized in that: In step S520, the incident sub-light The vector parameter is (X An ,Y An ,Z An ) is calculated by the first formula group, which is as follows: X An =(u An -u 0n )×dx n Y An =(v An -v 0n )×dy n Z An =f n Among them, n represents the corresponding virtual camera number; f n Represents the focal length of the corresponding virtual camera; (u An ,v An ) represents the pixel A in the corresponding secondary imaging image n Pixel coordinate parameters; (dx n ,dy n ) corresponds to the length of a single pixel in the virtual camera along the X and Y directions.

7. The method for panoramic reconstruction based on virtual multi-purpose conic reflector ray tracing according to claim 6, characterized in that: In step S530, each incident sub-ray is converted into Specifically, the camera coordinate system of the virtual camera with sequence number 1 is selected and marked as the world coordinate system, and the camera coordinate systems of other virtual cameras are converted into the world coordinate system using the second formula. The second formula is as follows: a n =R n,n+1 a n+1 +t n,n+1 Among them, n and n+1 represent the corresponding virtual camera numbers; α n Represents the coordinate parameters of the camera coordinate system corresponding to the virtual camera; R n,n+1 Represents the rotation matrix between the virtual camera with sequence number n and the virtual camera with sequence number n+1; t n,n+1 Represents the translation matrix between the virtual camera with sequence number n and the virtual camera with sequence number n+1.

8. The method for panoramic reconstruction based on virtual multi-purpose conic reflector ray tracing according to claim 7, characterized in that: In step S540, the incident sub-light Extend s unit vectors toward the conical reflector side Intersecting with the conical reflector surface to form a reflection point B n , through the third formula group and the conical reflector expression, the incident light is calculated The third formula group is as follows: Wherein, n represents the serial number of the corresponding virtual camera, is the incident sub-ray, B n is the intersection point of the outgoing light and the conical reflector surface, s is the solution coefficient, (p xn ,p yn ,p zn ) is a unit vector The vector parameter of .

9. The method for panoramic reconstruction based on virtual multi-purpose conic reflector ray tracing according to claim 8, characterized in that: In step S560, the normal is calculated according to the fourth formula of the law of mirror reflection. The vector parameter (l xn ,l yn ,l zn ), the fourth formula is as follows: in, It passes through reflection point B n With cone busbar The perpendicular unit vector, Passing through reflection point B n The cone generatrix of In step S570, the outgoing light is calculated by the sixth formula of the vector reflection law. The sixth formula is as follows: in, is the outgoing light from the measured point C toward the surface of the conical reflector, is the outgoing light Passing through reflection point B n The incident light that enters the optical center of the virtual camera is is the incident light and outgoing light axis of symmetry.

10. The method for panoramic reconstruction based on virtual multi-purpose conic reflector ray tracing according to claim 9, characterized in that: In step S580, the vector intersection point, i.e., the point to be measured C, is calculated by the seventh formula group of the vector intersection rule. The seventh formula group is as follows: x Bn +h n *x rn =x Bn+1 +h n+1 *x rn+1 y Bn +h n *y rn =y Bn+1 +h n+1 *y rn+1 z Bn +h n *z rn =z Bn+1 +h n+1 *z rn+1 Among them, n and n+1 represent the corresponding virtual camera numbers; (x rn ,y rn ,z rn ) is the outgoing light Vector parameters; (x Bn ,y Bn ,z Bn ) is the refraction point B n Coordinate parameters, h n To solve the coefficient.

Images