A fast registration method based on offset camera extrinsics obtained from point cloud images

Abstract

The present invention belongs to the technical field of intelligent operation and inspection of power transmission lines, and more specifically, it relates to a rapid registration method based on obtaining the external parameters of an offset camera of a point cloud image. The method comprises: obtaining the original image and the offset image taken by the same camera, completing the pole tower detection and marking the main line pole tower respectively; using the feature extraction and matching method to complete the image matching of the two images and calculate the offset; using the registration information between the original image and the point cloud collected at the same time, combined with the feature matching information between the original image and the deflected image, to complete the automatic registration between the deflected image and the point cloud collected at the original time, and then realize the point cloud ranging under the offset image. The present invention can effectively solve the ranging error problem caused by camera offset in power transmission scenarios, and provide solid data support for the safety of power transmission lines.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent operation and inspection of power transmission lines, and more specifically, relates to a fast registration method based on obtaining offset camera extrinsic parameters of a point cloud image. Background Art

[0002] With the rapid development of the power industry, the scale and complexity of power systems are increasing, which puts higher requirements on the safety monitoring and hidden danger detection of transmission lines. The traditional inspection method relies on manual on-site surveys, which is not only inefficient but also has safety hazards. In recent years, with the advancement of three-dimensional scanning technology and image processing technology, the use of tower point cloud data and high-definition images obtained by drones or lidar has provided new possibilities for automated inspections. However, in practical applications, the manually processed point cloud image registration model often fails due to camera offset and requires manual recalibration. How to quickly and accurately register point cloud data with offset image data is of great significance to ensuring the normal operation of the point cloud image ranging model in the power system.

[0003] Traditional point cloud image registration methods mainly rely on manual registration. In actual work, due to environmental changes, mechanical structure errors and other factors, the camera will inevitably have a certain deflection. The use of traditional manual registration methods is time-consuming, labor-intensive and costly. With the development of computer technology, the automatic registration method of point cloud images has been widely studied. For example: Chinese patent document CN117934570A discloses an image point cloud registration method based on a self-calibration attention mechanism, including: constructing an image point cloud network model; constructing a training data set and a test data set to extract deep features of images and point clouds; performing position embedding operations, self-attention and cross-attention processing, cross-modal calibration and multi-scale feature extraction in sequence; achieving precise matching of images and point clouds through dense matching modules; using PNP and RANSAC algorithms to obtain image point cloud registration results, and obtaining a trained image point cloud registration network model; using the test data set as the input of the trained image point cloud registration network model to obtain the final image point cloud registration results.

[0004] This method builds a model, trains data, and uses PNP and RANSAC methods to achieve calibration. It requires sufficiently detailed point cloud data, takes a long time to calculate, and has high equipment costs. It cannot meet the needs of updating the massive image point cloud registration model in the transmission line. Therefore, a method is needed to quickly and efficiently register the offset image with the original point cloud, so as to ensure the long-term effectiveness of the point cloud image ranging model under the transmission line.

[0005] In view of this, the present invention designs a fast registration method based on obtaining offset camera extrinsics of point cloud images. Summary of the invention

[0006] In view of the shortcomings of the prior art, the purpose of the present invention is to disclose a fast registration method based on the external parameters of the offset camera of the point cloud image, which can effectively reduce the problem of increased point cloud image ranging error caused by camera offset, provide solid data support for the safety of transmission lines, and have very important reference value for the rational inspection of transmission channels.

[0007] The detailed technical scheme of the present invention is as follows:

[0008] A fast registration method based on obtaining offset camera extrinsics of a point cloud image, the method comprising:

[0009] S1, obtaining the original image taken by the camera, the original point cloud at the same time and the offset image after the camera is deflected, and performing pole tower detection on the original image and the offset image respectively, marking the position coordinates of the main line pole tower, and performing sparse preprocessing on the main line tower point cloud;

[0010] S2, using an image matching method to complete feature point registration between the original image and the image of the location of the main line tower in the offset image, and calculate the offset of the two images;

[0011] S3. If the offset between the two images is greater than 3 pixels, the registration information between the original image and the original point cloud at the same time is used, combined with the feature matching information between the original image and the offset image, to calculate the registration information between the offset image and the original point cloud at the same time;

[0012] S4. The point cloud image registration model is corrected according to the registration information between the calculated offset image and the original point cloud at the same time, so as to complete the rapid registration between the offset image and the point cloud.

[0013] Preferably, according to the present invention, the original image taken by the camera is obtained by installing a monitoring device in the transmission channel, and the image taken by the device at a certain moment is used as the original image, the original point cloud at the same moment refers to the point cloud of the real transmission scene scanned at the same moment, and the point cloud sparseness preprocessing is performed on the tower point cloud after segmenting it, and the offset image after the camera deflection refers to the image taken after the monitoring device itself moves or is passively offset.

[0014] Preferably, the pole tower detection for the original image and the offset image refers to using yolov5 in combination with the self-training model to detect the pole towers in the original image and the offset image, mark the main line pole towers, and output the upper left corner and lower right corner pixel coordinates of the location of the main line pole tower. , .

[0015] Preferably, according to the present invention, step S2 is specifically as follows:

[0016] S21, intercepting the image of the location of the main line tower in the original image and the offset image according to the position coordinates of the main line tower, and preprocessing the intercepted image using grayscale and Gaussian filtering methods;

[0017] S22, using the SuperPoint deep learning algorithm to complete feature point detection on the preprocessed images;

[0018] S23, use the lightGlue algorithm to complete feature point matching;

[0019] S24, combining the RANSAC algorithm to filter out mismatched points, and outputting valid pixel point pairs of the original image and the image at the location of the main line tower in the offset image one by one ;

[0020] S25, according to the one-to-one corresponding valid pixel point group Calculate the x-coordinate offset and the y-coordinate offset , and calculate the overall offset L, the calculation formula is as follows:

[0021] (1)

[0022] In formula (1), is the offset of the x coordinate, The offset of the y coordinate.

[0023] Preferably, according to the present invention, the registration information between the original image and the original point cloud at the same time includes the camera intrinsic parameter matrix M and the extrinsic parameter matrix , ,in , Respectively represent the rotation matrix and translation matrix between the original image and the original point cloud at the same moment;

[0024] Specifically: manually fine-tune the registration of the original point cloud and the original image until the original point cloud matches the original image to obtain the camera intrinsic parameter matrix M and extrinsic parameter matrix of the captured image , ;

[0025] The camera intrinsic parameter matrix M can also be obtained by the captured image size, camera focal length F and camera sensor size, and the specific form is as follows:

[0026] (2)

[0027] In formula (2), is the camera intrinsic parameter matrix, Indicates the length of the camera's horizontal focal length along the x-axis. Indicates the length of the camera's horizontal focal length on the y-axis. represents the pixel coordinate x of the center point of the image, Indicates the pixel coordinate y of the center point of the image;

[0028] The external parameter matrix , It is the three-dimensional point cloud data in the world coordinate system After translation and rotation, the new 3D point cloud data is taken with the current monitoring device as the origin, the opposite tower as the positive direction of the y-axis, and the x-axis direction is determined by the right-hand coordinate system. obtained;

[0029] The world coordinate system has a world origin and defines the coordinates of any point defined in the space relative to the origin. The axis is horizontal to the right, Axis down, The camera coordinate system is determined by the optical center of the camera when the image is taken as the origin and the horizontal direction is Axis, vertical direction is Axis, pointing to the direction the camera was looking at when taking the first image Axis, which can be used as depth information;

[0030] The above three-dimensional point cloud data based on the world coordinate system Taking the surveillance camera as the origin, it is rotated and translated by a 3×3 rotation matrix and a 3×1 translation matrix, namely:

[0031] (3)

[0032] In formula (3), It is the point cloud after translation and rotation; is the rotation matrix, jointly constitute; Represents the rotation matrix parameters; is the translation matrix, , , jointly constitute; , , Represents the translation matrix parameters, which represent the translation vectors of the x, y, and z axes respectively; Represents the three-dimensional coordinates of the original point cloud; Indicates a point in it; Indicates the total number of points in the point cloud data.

[0033] Preferably, the feature matching information is a group of effective pixel points of the image at the location of the main line tower in the original image. and the effective pixel point group of the image at the location of the main line tower in the offset image , fp0 and fp1 form a valid pixel pair, fp0 and fp1 contain the same number of n pixels that correspond one to one;

[0034] The registration information between the offset image and the original point cloud at the same time includes the camera intrinsic parameter matrix M and the extrinsic parameter matrix and ,in and Respectively represent the rotation matrix and translation matrix between the offset image and the original point cloud at the same moment.

[0035] Preferably, according to the present invention, the feature matching information between the original image and the offset image is combined to calculate the registration information between the offset image and the original point cloud at the same time, specifically as follows:

[0036] First, the original image and the offset image are images taken from different positions by the same camera without lens distortion. The effective pixel points of the image at the location of the main line tower in the original image and the offset image are matched. Find the homography matrix , the conversion relationship is as follows:

[0037] (4)

[0038] In formula (4), Represents the offset image pixel group The coordinates of a pixel point in Represents the original image pixel group The coordinates of a pixel in the is the homography matrix The specific parameters of Divide by Normalize to get the remaining 8 unknowns. Then, as long as there is and The remaining 8 unknown numbers can be solved by using the 4 pairs of pixels in .

[0039] Then the SVD singular value decomposition method is used to transform the homography matrix and the camera intrinsic matrix Convert to rotation matrix and translation matrix , there are usually multiple solutions, and it is necessary to determine the optimal solution;

[0040] The original image and the point cloud image are manually registered. , Matrix, relative offset matrix between fused images , Get the rotation matrix between the offset image and the original point cloud and translation matrix , under the specific formula:

[0041] (5)

[0042] (6)

[0043] Preferably, the specific steps of determining the optimal solution are as follows:

[0044] 1. The original point cloud 3D coordinates Convert to pixel coordinates , the conversion formula is as follows:

[0045] (7)

[0046] In formula (7), is the camera intrinsic parameter matrix, is the rotation matrix between the general original point cloud and any image, is the translation vector between the general original point cloud and any image, is the three-dimensional coordinate of the original point cloud;

[0047] 2. Use the upper left and lower right pixel coordinates of the main line tower in the offset image , , the number of point clouds in this area is counted according to the following judgment formula:

[0048] (8);

[0049] 3. Use the obtained rotation matrix and translation matrix , Project the tower point cloud separately and overlap it with the offset image, and record the rotation matrix and translation matrix used when the number of point clouds is the largest. , , at this time we get , It is the registration information between the offset image and the original point cloud.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] (1) The present invention integrates pole tower detection and deep learning-based image matching technology. This innovation greatly improves the matching accuracy for transmission pole towers with relatively small image offsets, thereby providing a more accurate prerequisite for subsequent point cloud image registration. The present invention can determine whether subsequent registration is required based on this feature, which can greatly improve efficiency in large-volume data such as transmission lines.

[0052] (2) The present invention optimizes the registration of deflected images and original point clouds. In view of the fact that the monitoring equipment has mechanical errors due to external environment (wind, rain, etc.) and the offset of the pan / tilt, there is a large registration error between the captured image and the original point cloud, which affects the ranging accuracy. The present invention uses faster image information to obtain the homography matrix to achieve automatic registration. At the same time, the point cloud sparse method is used to greatly improve its overall registration speed. According to the number of tower point clouds at limited positions, the problem of multiple solutions during the singular value decomposition of the homography matrix is ​​solved, which greatly improves its registration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a fast registration method based on obtaining offset camera extrinsic parameters of a point cloud image described in the present invention.

[0054] Figure 2 Schematic diagram of an original visible light image of a power transmission scene in an embodiment of the present invention.

[0055] Figure 3 It is a schematic diagram of the original point cloud at the same time of the power transmission scene in an embodiment of the present invention.

[0056] Figure 4a It is the original tower point cloud in the power transmission scene in the embodiment of the present invention.

[0057] Figure 4b It is the tower point cloud after sparse processing in the embodiment of the present invention.

[0058] Figure 5 is a schematic diagram of a visible light image after the power transmission scene is offset in an embodiment of the present invention.

[0059] Figure 6a It is a tower detection effect diagram of the original image of the power transmission scene in an embodiment of the present invention.

[0060] Figure 6b It is a tower detection effect diagram of the power transmission scene offset image in an embodiment of the present invention.

[0061] Figure 7 It is a feature matching effect diagram of the original image and the offset image of the power transmission scene in the embodiment of the present invention.

[0062] Figure 8 It is a registration effect diagram of the original image of the power transmission scene and the original point cloud in an embodiment of the present invention.

[0063] Fig. 9 This is a diagram showing the registration effect of the offset image of the power transmission scene and the original point cloud in an embodiment of the present invention.

[0064] Fig.10This is a point cloud image registration effect diagram obtained by using the PNP method in the power transmission scene in an embodiment of the present invention.

[0065] Fig.11 This is a point cloud image registration effect diagram obtained by using the method of the present invention for a power transmission scene in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0067] Embodiment 1,

[0068] Ginseng Figure 1 This embodiment provides a fast registration method based on obtaining offset camera extrinsic parameters of a point cloud image, the method comprising:

[0069] S1, obtaining the original image taken by the camera, the original point cloud at the same time and the offset image after the camera is deflected, and performing pole tower detection on the original image and the offset image respectively, marking the position coordinates of the main line pole tower, and performing sparse preprocessing on the main line tower point cloud;

[0070] S2, using an image matching method to complete feature point registration between the original image and the image of the location of the main line tower in the offset image, and calculate the offset of the two images;

[0071] S3. If the offset between the two images is greater than 3 pixels, the registration information between the original image and the original point cloud at the same time is used, combined with the feature matching information between the original image and the offset image, to calculate the registration information between the offset image and the original point cloud at the same time;

[0072] S4. The point cloud image registration model is corrected according to the registration information between the calculated offset image and the original point cloud at the same time, so as to complete the rapid registration between the offset image and the point cloud.

[0073] The obtaining of the original image taken by the camera refers to installing a monitoring device in the transmission channel, and the image taken by the device at a certain moment is used as the original image. The original point cloud at the same moment refers to the point cloud of the real transmission scene scanned at the same moment, and the point cloud sparseness preprocessing is performed on the tower point cloud after segmenting it. The offset image after camera deflection refers to the image taken after the monitoring device itself moves or is passively offset.

[0074] The pole tower detection for the original image and the offset image refers to using yolov5 in combination with the self-training model to detect the pole towers in the original image and the offset image respectively, marking the main line pole towers, and outputting the upper left corner and lower right corner pixel coordinates of the location of the main line pole tower , .

[0075] The step S2 is specifically as follows:

[0076] S21, intercepting the image of the location of the main line tower in the original image and the offset image according to the position coordinates of the main line tower, and preprocessing the intercepted image using grayscale and Gaussian filtering methods;

[0077] S22, using the SuperPoint deep learning algorithm to complete feature point detection on the preprocessed images;

[0078] S23, use the lightGlue algorithm to complete feature point matching;

[0079] S24, combining the RANSAC algorithm to filter out mismatched points, and outputting valid pixel point pairs of the original image and the image at the location of the main line tower in the offset image one by one ;

[0080] S25, according to the one-to-one corresponding valid pixel point group Calculate the x-coordinate offset and the y-coordinate offset , and calculate the overall offset L, the calculation formula is as follows:

[0081] (1)

[0082] In formula (1), is the offset of the x coordinate, The offset of the y coordinate.

[0083] The registration information between the original image and the original point cloud at the same time includes the camera intrinsic parameter matrix M and the extrinsic parameter matrix , ,in , Respectively represent the rotation matrix and translation matrix between the original image and the original point cloud at the same moment;

[0084] Specifically: manually fine-tune the registration of the original point cloud and the original image until the original point cloud matches the original image to obtain the camera intrinsic parameter matrix M and extrinsic parameter matrix of the captured image , ;

[0085] The camera intrinsic parameter matrix M can also be obtained by the captured image size, camera focal length F and camera sensor size, and the specific form is as follows:

[0086] (2)

[0087] In formula (2), is the camera intrinsic parameter matrix, Indicates the length of the camera's horizontal focal length along the x-axis. Indicates the length of the camera's horizontal focal length on the y-axis. represents the pixel coordinate x of the center point of the image, Indicates the pixel coordinate y of the center point of the image;

[0088] The external parameter matrix , It is obtained by manual registration and fine-tuning, which is the 3D point cloud data in the world coordinate system After translation and rotation, the new 3D point cloud data is taken with the current monitoring device as the origin, the opposite tower as the positive direction of the y-axis, and the x-axis direction is determined by the right-hand coordinate system. , to obtain the external parameter matrix , ;

[0089] The world coordinate system has a world origin and defines the coordinates of any point defined in the space relative to the origin. The axis is horizontal to the right, Axis down, The camera coordinate system is determined by the optical center of the camera when the image is taken as the origin and the horizontal direction is Axis, vertical direction is Axis, pointing to the direction the camera was looking at when taking the first image Axis, which can be used as depth information;

[0090] The above three-dimensional point cloud data based on the world coordinate system Taking the surveillance camera as the origin, it is rotated and translated by a 3×3 rotation matrix and a 3×1 translation matrix, namely:

[0091] (3)

[0092] In formula (3), is the point cloud after translation and rotation; is the rotation matrix, jointly constitute; Represents the rotation matrix parameters; is the translation matrix, , , jointly constitute; , , Represents the translation matrix parameters, which represent the translation vectors of the x, y, and z axes respectively; Represents the three-dimensional coordinates of the original point cloud; Indicates a point in it; Indicates the total number of points in the point cloud data.

[0093] The feature matching information is a group of valid pixel points of the image at the location of the main line tower in the original image. and the effective pixel point group of the image at the location of the main line tower in the offset image , fp0 and fp1 form a valid pixel pair, fp0 and fp1 contain the same number of n pixels that correspond one to one;

[0094] The registration information between the offset image and the original point cloud at the same time includes the camera intrinsic parameter matrix M and the extrinsic parameter matrix and ,in and Respectively represent the rotation matrix and translation matrix between the offset image and the original point cloud at the same moment.

[0095] The feature matching information between the original image and the offset image is combined to calculate the registration information between the offset image and the original point cloud at the same time, as follows:

[0096] First, the original image and the offset image are images taken from different positions by the same camera without lens distortion. The effective pixel points of the image at the location of the main line tower in the original image and the offset image are matched. Find the homography matrix , the conversion relationship is as follows:

[0097] (4)

[0098] In formula (4), Represents the offset image pixel group The coordinates of a pixel point in Represents the original image pixel group The coordinates of a pixel in the is the homography matrix The specific parameters of Divide by Normalize to get the remaining 8 unknowns. Then, as long as there is and The remaining 8 unknown numbers can be solved by using the 4 pairs of pixels in .

[0099] Then the SVD singular value decomposition method is used to transform the homography matrix and the camera intrinsic matrix Convert to rotation matrix and translation matrix , there are usually multiple solutions, and it is necessary to determine the optimal solution;

[0100] The original image and the point cloud image are manually registered. , Matrix, relative offset matrix between fused images , Get the rotation matrix between the offset image and the original point cloud and translation matrix , under the specific formula:

[0101] (5)

[0102] (6)

[0103] The specific steps of determining the optimal solution are as follows:

[0104] 1. The original point cloud 3D coordinates Convert to pixel coordinates , the conversion formula is as follows:

[0105] (7)

[0106] In formula (7), is the camera intrinsic parameter matrix, is the rotation matrix between the general original point cloud and any image, is the translation vector between the general original point cloud and any image, is the three-dimensional coordinate of the original point cloud;

[0107] 2. Use the upper left and lower right pixel coordinates of the main line tower in the offset image , , the number of point clouds in this area is counted according to the following judgment formula:

[0108] (8);

[0109] 3. Use the obtained rotation matrix and translation matrix , Project the tower point cloud separately and overlap it with the offset image, and record the rotation matrix and translation matrix used when the number of point clouds is the largest. , , at this time we get , It is the registration information between the offset image and the original point cloud.

[0110] Embodiment 2,

[0111] Step 1: Install a monitoring device in the transmission channel and obtain the image taken by the device at a certain moment as the original image. Figure 2 , point cloud data is the point cloud of the real power transmission scene scanned at the same time. Figure 3 , after segmenting the tower point cloud, perform point cloud sparse preprocessing on it. Figure 4a is the original tower point cloud, Figure 4b The offset image is the image taken after the monitoring device moves or is passively offset. Figure 5 .

[0112] Step 2: Detect the pole towers: Use yolov5 combined with the self-training model to detect the pole towers in the original image and the offset image, and mark the main line pole towers. Figure 6a This is the tower detection effect diagram of the original image. Figure 6b It is the pole tower detection effect diagram of the offset image, outputting the pixel coordinates of the upper left corner and lower right corner of the pole tower location , .

[0113] Step 3: First, preprocess the image and extract the image of the tower position; then use the SuperPoint deep learning algorithm to complete the feature point detection; finally, use the lightGlue algorithm to complete the feature matching between the images; combine the RANSAC algorithm to filter out the mismatched points, and output the valid pixel point pairs of the two images one by one ; The matching effect of the two images is as follows Figure 7 shown.

[0114] Calculate the x-coordinate offset based on the one-to-one corresponding pixel pairs of the two images and the y-coordinate offset , and calculate the overall offset .

[0115] Step 4: Determine whether the offset between the two images is greater than 3 pixels. If so, extract the registration information generated by manual registration, i.e., the internal reference M and the external reference , ,in , Respectively represent the rotation matrix and translation matrix between the original image and the point cloud. The registration effect between the original image and the original point cloud is as follows: Figure 8 As shown in Figure 2, the registration effect between the offset image and the original point cloud is as follows: Fig. 9 shown.

[0116] Step 5: First, obtain the feature point information based on the original image and the offset image Compute the homography matrix , then the homography matrix and camera intrinsics Combine and use SVD singular value decomposition method to convert it into a rotation matrix and translation matrix , there are usually multiple solutions, so it is necessary to determine the optimal solution.

[0117] The original image and the point cloud image are manually registered , Matrix, relative offset matrix between fused images , Get a new rotation matrix and translation matrix , , under the specific formula:

[0118]

[0119]

[0120] The specific steps to determine the optimal solution are as follows:

[0121] The three-dimensional coordinates of the tower point cloud data Convert to pixel coordinates .

[0122] The tower area information obtained during tower detection in the offset image is used, that is, the pixel coordinates of the upper left corner and the lower right corner , , and count the number of point clouds in this area according to the judgment formula;

[0123]

[0124] Using the obtained multiple rotation matrices and translation matrices , Project the tower point cloud separately and overlap it with the offset image, and record the rotation matrix and translation matrix used when the number of point clouds is the largest. , At this time, the registration effect between the offset image and the registered point cloud is as follows: Fig. 9 As shown, in order to more intuitively express the superiority of this method, the PNP method is used to align it and the alignment effect is as follows Fig.10 As shown, the effect after using this method to align it is as follows Fig.11 As shown, by comparison Fig.10 , Fig.11 It can be seen that after using the method of the present invention, the point cloud mask and the tower can be perfectly integrated, which proves the advantage of the present invention.

[0125] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A fast registration method based on obtaining offset camera extrinsics of point cloud images, characterized in that: The method comprises the following steps: S1, obtaining the original image taken by the camera, the original point cloud at the same time and the offset image after the camera is deflected, and performing pole tower detection on the original image and the offset image respectively, marking the position coordinates of the main line pole tower, and performing sparse preprocessing on the main line tower point cloud; S2, using an image matching method to complete feature point registration between the original image and the image of the location of the main line tower in the offset image, and calculate the offset of the two images; S3. If the offset between the two images is greater than 3 pixels, the registration information between the original image and the original point cloud at the same time is used, combined with the feature matching information between the original image and the offset image, to calculate the registration information between the offset image and the original point cloud at the same time, as follows: First, the effective pixel points of the original image and the image at the location of the main line tower in the offset image are matched. Find the homography matrix , the conversion relationship is as follows: (1) In formula (1), Represents the offset image pixel group The coordinates of a pixel point in Represents the original image pixel group The coordinates of a pixel in the is the homography matrix The specific parameters of Divide by Normalize to get the remaining 8 unknowns. Then, as long as there is and The remaining 8 unknown numbers can be solved by using the 4 pairs of pixels in ; Then the SVD singular value decomposition method is used to transform the homography matrix and the camera intrinsic matrix Convert to rotation matrix and translation matrix , there are usually multiple solutions, and it is necessary to determine the optimal solution; The original image and the point cloud image are manually registered. , Matrix, relative offset matrix between fused images , Get the rotation matrix between the offset image and the original point cloud and translation matrix , under the specific formula: （2） （3）； S4. The point cloud image registration model is corrected according to the registration information between the calculated offset image and the original point cloud at the same time, so as to complete the rapid registration between the offset image and the point cloud.

2. The fast registration method based on obtaining the offset camera extrinsic parameters of the point cloud image according to claim 1, characterized in that: The original image taken by the camera is obtained by installing a monitoring device in the transmission channel, and the image taken by the device at a certain moment is used as the original image. The original point cloud at the same moment refers to the point cloud of the real transmission scene scanned at the same moment. The offset image after camera deflection refers to the image taken after the monitoring device itself moves or is passively offset.

3. The fast registration method based on obtaining the offset camera extrinsic parameters of the point cloud image according to claim 1, characterized in that: The pole tower detection for the original image and the offset image refers to using yolov5 in combination with the self-training model to detect the pole towers in the original image and the offset image respectively, marking the main line pole towers, and outputting the upper left corner and lower right corner pixel coordinates of the location of the main line pole tower , .

4. The fast registration method based on obtaining the offset camera extrinsic parameters of the point cloud image according to claim 1, characterized in that: The step S2 is specifically as follows: S21, intercepting the image of the location of the main line tower in the original image and the offset image according to the position coordinates of the main line tower, and preprocessing the intercepted image using grayscale and Gaussian filtering methods; S22, using the SuperPoint deep learning algorithm to complete feature point detection on the preprocessed images; S23, use the lightGlue algorithm to complete feature point matching; S24, combining the RANSAC algorithm to filter out mismatched points, and outputting valid pixel point pairs of the original image and the image at the location of the main line tower in the offset image one by one ; S25, according to the one-to-one corresponding valid pixel point group Calculate the x-coordinate offset and the y-coordinate offset , and calculate the overall offset L, the calculation formula is as follows: (4) In formula (4), is the offset of the x coordinate, The offset of the y coordinate.

5. The fast registration method based on obtaining the offset camera extrinsic parameters of the point cloud image according to claim 1, characterized in that: The registration information between the original image and the original point cloud at the same time includes the camera intrinsic parameter matrix M and the extrinsic parameter matrix , ,in , Respectively represent the rotation matrix and translation matrix between the original image and the original point cloud at the same moment; Specifically: manually fine-tune the registration of the original point cloud and the original image until the original point cloud matches the original image to obtain the camera intrinsic parameter matrix M and extrinsic parameter matrix of the captured image , ; The specific form of the camera intrinsic parameter matrix M is as follows: (5) In formula (5), is the camera intrinsic parameter matrix, Indicates the length of the camera's horizontal focal length along the x-axis. Indicates the length of the camera's horizontal focal length on the y-axis. represents the pixel coordinate x of the center point of the image, Indicates the pixel coordinate y of the center point of the image; The external parameter matrix , It is the three-dimensional point cloud data in the world coordinate system After translation and rotation, the new 3D point cloud data is taken with the current monitoring device as the origin, the opposite tower as the positive direction of the y-axis, and the x-axis direction is determined by the right-hand coordinate system. obtained; The world coordinate system has a world origin, and the coordinates of any point defined in the world coordinate system are defined relative to the world origin. The axis is horizontal to the right, Axis down, The camera coordinate system is determined by the optical center of the camera when the image is taken as the origin and the horizontal direction is Axis, vertical direction is Axis, pointing to the direction the camera was looking at when taking the first image Axis, which can be used as depth information; The above three-dimensional point cloud data based on the world coordinate system Taking the surveillance camera as the origin, it is rotated and translated by a 3×3 rotation matrix and a 3×1 translation matrix, namely: (6) In formula (6), It is the point cloud after translation and rotation; is the rotation matrix, jointly constitute; Represents the rotation matrix parameters; is the translation matrix, , , jointly constitute; , , Represents the translation matrix parameters, which represent the translation vectors of the x, y, and z axes respectively; Represents the three-dimensional coordinates of the original point cloud; Indicates a point in it; Indicates the total number of points in the point cloud data.

6. The fast registration method based on obtaining the offset camera extrinsic parameters of the point cloud image according to claim 1, characterized in that: The feature matching information is a group of valid pixel points of the image at the location of the main line tower in the original image. and the effective pixel point group of the image at the location of the main line tower in the offset image , fp0 and fp1 form a valid pixel pair, fp0 and fp1 contain the same number of n pixels that correspond one to one; The registration information between the offset image and the original point cloud at the same time includes the camera intrinsic parameter matrix M and the extrinsic parameter matrix and ,in and Respectively represent the rotation matrix and translation matrix between the offset image and the original point cloud at the same moment.

7. The fast registration method based on obtaining the offset camera extrinsic parameters of the point cloud image according to claim 1, characterized in that: The specific steps of determining the optimal solution are as follows: a. The original point cloud 3D coordinates Convert to pixel coordinates , the conversion formula is as follows: （7） In formula (7), is the camera intrinsic parameter matrix, is the rotation matrix between the general original point cloud and any image, is the translation vector between the general original point cloud and any image, is the three-dimensional coordinate of the original point cloud; b. Use the upper left and lower right pixel coordinates of the main line tower in the offset image , , the number of point clouds in this area is counted according to the following judgment formula: （8）； c. Use the obtained rotation matrix and translation matrix , Project the tower point cloud separately and overlap it with the offset image, and record the rotation matrix and translation matrix used when the number of point clouds is the largest. , , at this time we get , It is the registration information between the offset image and the original point cloud.

Images