Image-based spatial reconstruction method of single lead, computer readable storage medium and computer program product

Through image and point cloud data registration, instance segmentation and fitting technology, the ranging error problem caused by wire changes in traditional methods is solved, and high-precision single wire space reconstruction is achieved, suitable for real-time monitoring and patrol of transmission lines Inspection.

CN119006707APending Publication Date: 2024-11-22SHANDONG ZHIYANG ELECTRIC
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411050659.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The traditional single wire space reconstruction method relies on manual measurement, which is time-consuming and labor-intensive and has low accuracy, and cannot effectively deal with the problem of increasing distance measurement error caused by wire changes in the transmission channel.

Method used

Using an image-based single wire space reconstruction method, the changing wire point cloud is calculated by obtaining visible light pictures and point cloud data for registration, instance segmentation and density clustering, combined with camera internal parameters and pixel displacement, and the changing wire point cloud is calculated and fitted Methods generate smooth wire point clouds.

Benefits of technology

Accurate three-dimensional reconstruction of wires is realized, which reduces distance measurement errors, improves reconstruction accuracy and reliability, can adapt to wire shape changes in real time, and provides solid data support and patrol references.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006707A_ABST
    Figure CN119006707A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of intelligent operation inspection of power transmission lines, and particularly relates to an image-based spatial reconstruction method of a single wire, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an original picture and a channel point cloud at the same moment, and performing registration by using the picture and the point cloud to determine a camera internal reference matrix and depth image information data of monitoring and photographing equipment; performing instance segmentation on the original picture and the newly-shot picture, searching a corresponding pixel point in the newly-shot picture closest to the lead pixel of the original picture, and recording the displacement of the pixel point; a traverse point cloud is segmented into single traverse point clouds, a changed rough traverse point cloud is calculated by combining a camera internal reference matrix, depth image information and pixel displacement, and the changed rough traverse point cloud is fitted into a smooth traverse point cloud through a traverse point cloud fitting method. The problem of distance measurement error increase caused by power transmission scene wire change can be effectively reduced, and a solid data support is provided for power transmission line safety.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent operation and inspection of power transmission lines, and more specifically, relates to a method for spatial reconstruction of a single conductor based on an image, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of science and technology, the power system plays an increasingly important role in people's lives. In order to meet the growing demand for electricity, the scale and complexity of the power system are also increasing. In the power system, the transmission line is an important component connecting various power facilities, and a single conductor is the basic unit of the transmission line. Therefore, spatial reconstruction of a single conductor in order to better understand its status and performance is of great significance to ensure the normal operation of the power system.

[0003] Traditional single-wire spatial reconstruction methods mainly rely on manual measurement and calculation, which is time-consuming, labor-intensive and has low accuracy. With the development of computer vision and image processing technology, image-based single-wire spatial reconstruction methods have gradually attracted attention. For example: Chinese patent document CN116958669A discloses a point cloud classification method for transmission line based on point cloud image fusion based on convolutional neural network, including: collecting and preprocessing visible light images and point cloud data of transmission lines; inputting visible light images and point cloud data into the fusion target detection network for feature extraction and feature fusion respectively; inputting the extracted fusion features into the convolutional neural network for wire point cloud data feature learning, and finally realizing the reconstruction calculation of the wire point cloud.

[0004] This method obtains images of the conductors and uses computer vision and image processing technology to identify, locate and measure the conductors, thereby achieving spatial reconstruction of the conductors. However, due to the characteristics of the conductor material and shape, the conductors in the transmission channel will change all the time, while a single image and the point cloud collected at the same time are fixed. When the conductors change, it is necessary to re-acquire the conductor point cloud and re-map the conductors in the newly captured image. Therefore, a method is needed to combine the newly captured image with the original point cloud data to achieve accurate reconstruction of the spatial position of the conductors. Summary of the invention

[0005] In view of the deficiencies of the prior art, an object of the present invention is to disclose a method for spatial reconstruction of a single wire based on an image.

[0006] The invention also discloses a computer-readable storage medium for implementing the above method.

[0007] The present invention also discloses a program product for realizing the above method, which effectively reduces the problem of increased ranging error caused by changes in the conductors in the transmission scene, provides solid data support for the safety of the transmission line, and has very important reference value for the rational inspection of the transmission channel and other work.

[0008] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0009] A method for spatial reconstruction of a single wire based on an image, characterized in that the method comprises the following steps:

[0010] S1. Obtain the visible light image taken by the visual monitoring device based on the transmission line as the original image and the channel point cloud data at the same time, and use the image and point cloud data for registration to determine the camera intrinsic parameter matrix and depth image information data of the monitoring device;

[0011] S2, perform instance segmentation on the original image and the newly captured image respectively, find the label of each wire pixel category in all labels, find the pixel point in the corresponding vertical axis direction in the newly captured image that is closest to the wire pixel in the original image, and record its displacement information;

[0012] S3, extract the wire point cloud in the point cloud set corresponding to the current transmission channel, and segment the wire point cloud into different single wire point clouds. At the same time, since the point cloud data and the image have been registered, the single wire point cloud and the segmented wire pixels have a one-to-one correspondence;

[0013] S4. Combining the camera intrinsic parameter matrix, depth information and pixel displacement, the rough wire point cloud after the change is calculated, and it is fitted into a smooth wire point cloud through the wire point cloud fitting method.

[0014] Preferably, according to the present invention, the acquisition of visible light pictures taken by the transmission line visualization monitoring equipment and the channel point cloud data at the same time refers to installing the monitoring equipment in the transmission channel, the image taken by the equipment at a certain moment is used as the original picture, and the point cloud data is the point cloud of the real transmission scene scanned at the same moment, and the newly taken picture refers to the image taken by the monitoring equipment after a certain interval.

[0015] Preferably, according to the present invention, the registration of the image and the point cloud data refers to the registration of the original image and the wire point cloud, and the registration parameters include an intrinsic parameter M and extrinsic parameters R and T, wherein R represents a rotation matrix and T represents a translation matrix;

[0016] Specifically, first, the image captured during point cloud acquisition is taken as the reference of the channel image, and the camera intrinsic parameter matrix is ​​manually adjusted and the translation and rotation of the point cloud are fine-tuned until the point cloud matches the channel image to obtain the camera intrinsic parameter matrix data of the captured image: it is calibrated by the camera intrinsic parameter matrix data or calculated by the visible light image size, the camera focal length f and the camera CCD or COMS sensor size. The specific form is as follows:

[0017]

[0018] In formula (1), M is the camera internal parameter matrix, f x Indicates the length of the horizontal focal length of the surveillance camera on the x-axis, f y Indicates the length of the vertical focal length of the surveillance camera on the y-axis, c x Indicates the horizontal center position of the image coordinate system, c y Indicates the vertical center position of the image coordinate system;

[0019] The image and wire data are used for registration. The external parameters need to be the 3D point cloud data P in the world coordinate system. w Translate and rotate to convert the point cloud P to the camera coordinate system with the current tower as the origin and the opposite tower as the positive direction of the y-axis c ;

[0020] The world coordinate system has a world origin and defines the coordinates of any point defined in the space relative to the origin. w The axis is horizontal to the right, Y w Axis down, Z w Determined by the right-hand rule, the camera coordinate system is the optical center of the camera when the image is taken as the origin and the horizontal direction is X c Axis, vertical direction is Y c Axis, pointing to the direction the camera was looking at when taking the first image is Z c axis;

[0021] The three-dimensional point cloud data based on the world coordinate system is rotated and translated with the monitoring camera as the origin through a 3*3 rotation matrix and a 3*1 translation matrix, namely:

[0022]

[0023] In formula (2): P rt is the point cloud after translation and rotation; R is the rotation matrix; T is the translation matrix, P w =[x i ,y i ,z i ], represents the original point cloud set of the transmission channel, x iis the horizontal position of the i-th point in the point cloud, y i is the vertical position of the i-th point in the point cloud, z i is the position of the i-th point in the point cloud set perpendicular to the horizontal plane; is the direction vector of the x-axis in the original coordinate system after rotation; [r 12 r 22 r 32 ] T is the direction vector of the y-axis in the original coordinate system after rotation; [r 13 r 23 r 33 ] T is the direction vector of the z-axis in the original coordinate system after rotation; [t x t y t z ] T Respectively represent P W Convert to P rt The translation amount on the x-axis, y-axis and z-axis, and n is the total number of points in the point cloud data.

[0024] Preferably, according to the present invention, performing instance segmentation on the original picture and the newly taken picture respectively is a task in computer vision, and each object instance in the original picture and the newly taken picture is segmented using an instance segmentation algorithm based on deep learning to identify different objects in the picture and depict the outline of each object.

[0025] Preferably, according to the present invention, the segmentation of the wire point cloud into different single wire point clouds refers to using the density clustering method DBSCAN to divide the data points into high-density areas and low-density areas, wherein the high-density areas are regarded as clusters:

[0026] First, the point cloud data of the wire is taken as input, where each point contains three-dimensional space coordinate information;

[0027] Next, select the DBSCAN parameters, neighborhood radius, and minimum number of points, and traverse all points using the DBSCAN algorithm:

[0028] If a point is a core point, try to expand the current cluster and add all density-reachable points, that is, points reachable from the core point through other core points, to the same cluster;

[0029] If a point is a boundary point, it is assigned to the cluster of its core points;

[0030] Noise points are ignored and not assigned to any cluster;

[0031] Finally, for the wire point cloud, each cluster corresponds to a wire.

[0032] According to a preferred embodiment of the present invention, the step of finding the pixel point in the newly captured image corresponding to the longitudinal axis direction that is closest to the wire pixel of the original image and recording its displacement information refers to establishing a one-to-one matching relationship between the wire pixels of the newly captured image that are closest to each wire pixel of the original image in the longitudinal axis u direction by finding the pixel point by point, and recording the displacement Δu between the corresponding pixels in the longitudinal axis u direction of the image. j .

[0033] According to the preferred embodiment of the present invention, the estimated rough wire point cloud after the change refers to converting the depth image information of each pixel into a rough spatial wire point cloud using the camera intrinsic parameter matrix and the pixel displacement. Specifically, the pixel displacement Δu of each wire pixel point in the u direction on the reference image and the current image is converted using the registration parameters obtained in the above steps. j Converted into the spatial variation Δz of the wire point in the three-dimensional space coordinate system j , by adding the spatial variation of each wire point to the original wire point cloud, the reconstructed wire point cloud is obtained;

[0034] The pixel displacement Δu of each wire pixel point in the u direction on the reference image and the current image is calculated. j Converted into the spatial variation Δz of the wire point in the three-dimensional space coordinate system j The process is the inverse process of point cloud to depth map conversion, by comparing the original wire pixel position in the image with the known pixel depth depth j , the pixel displacement Δu of the jth wire pixel in the u direction on the reference image and the current image is calculated under the current depth map depth by the pinhole imaging principle j The real-world distance represented by , accurately maps the wire points in the image to the spatial locations in the real world:

[0035]

[0036] In formula (3), Δz j Indicates the spatial variation corresponding to the j-th pixel displacement, Dis p represents the horizontal distance from the jth 3D wire point to the camera, Δu j represents the pixel displacement of the jth wire pixel in the u direction between the reference image and the current image, and focal is the focal length of the camera;

[0037] Finally, after obtaining the spatial variation of the wire points in the three-dimensional space coordinate system, the reconstructed wire point cloud set P can be obtained by adding the spatial variation to the corresponding original wire point cloud. re , specifically:

[0038] P re=[x j ,y j ,z j +Δz j ],j=1,2,...,m; (4)

[0039] In formula (4), x j Indicates the horizontal position of the space point corresponding to the j-th wire pixel point, y j Indicates the vertical position of the space point corresponding to the j-th wire pixel point, z j Indicates the position of the space point corresponding to the j-th wire pixel in the direction perpendicular to the horizontal plane, Δz j It represents the spatial variation corresponding to the displacement of the jth pixel, and m is the total number of wire pixels.

[0040] Preferably, according to the present invention, the wire point cloud fitting method refers to fitting a rough wire point cloud with noise into a smooth and continuous wire point cloud using the least squares method. Since the wires of the transmission line can be assumed to be ideal flexible curves, various curve models can be used to approximate their spatial paths, such as using polynomial curves in three-dimensional space to fit the wire point cloud, such as a cubic spline curve. At the same time, since the wire information contained in the image is not complete, the wire fitting needs to intercept the part of the point cloud closest to the lens side of the original point cloud set and add it to the reconstructed rough point cloud set, and then fit it into a smooth and complete wire point cloud.

[0041] A computer-readable storage medium stores computer instructions, which can implement the steps of the above method when executed by a processor.

[0042] A computer program product loaded with the above method comprises computer instructions, which can implement the steps of the above method when executed by a processor.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) The present invention integrates the conversion mechanism of two-dimensional pixel displacement and three-dimensional space displacement, and is supplemented by sophisticated image segmentation technology. This innovation enables the accurate capture of subtle changes in the image with only two images and the corresponding point cloud data, and then realizes the three-dimensional reconstruction of a single wire in the real space. More importantly, through the fitting processing of the point cloud data, the present invention can efficiently eliminate the noise generated in the reconstruction process, thereby generating a smooth and fluent wire point cloud, greatly improving the reconstruction accuracy and visual effect, and can more accurately calculate hidden dangers and wire clearance distances.

[0045] (2) The present invention has the ability to reconstruct a single conductor in real time. Since transmission lines often change shape due to external environmental factors (such as wind, temperature, etc.), these changes often cause significant errors in traditional distance measurement methods. However, through the real-time reconstruction technology of the present invention, the obtained conductor point cloud can closely fit the conductor shape in the latest photograph, effectively reducing the accumulation of distance measurement errors caused by changes in conductor shape, and significantly improving the accuracy and reliability of distance measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of a method for spatial reconstruction of a single wire based on an image provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of an original power transmission scenario is provided for an embodiment of the present invention;

[0048] Figure 3 Provide a point cloud visualization diagram of a power transmission scene for an embodiment of the present invention;

[0049] Figure 4 An example segmentation effect diagram of an original picture of a power transmission scene is provided for an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of a new picture taken of a power transmission scene is provided for an embodiment of the present invention;

[0051] Figure 6 Provides a segmentation effect diagram of a newly taken picture of a power transmission scene for an embodiment of the present invention;

[0052] Figure 7 Provide a visualization diagram of the power transmission scene after the conductor point cloud is segmented for the embodiment of the present invention;

[0053] Figure 8 Provides a nearest pixel search principle image for the embodiment of the present invention;

[0054] Fig. 9 A point cloud visualization diagram of a single rough wire provided in an embodiment of the present invention;

[0055] Fig.10 A visualization of a point cloud of a single smooth wire after rough fitting provided by an embodiment of the present invention;

[0056] Fig.11 The embodiment of the present invention provides original wire depth image information (left), depth image information of the original wire in the newly taken image (middle), and depth image information of the reconstructed wire in the newly taken image (right). DETAILED DESCRIPTION

[0057] The present invention will be further described below in conjunction with the embodiments.

[0058] Embodiment 1,

[0059] like Figure 1 As shown, a method for spatial reconstruction of a single wire based on an image comprises the following steps:

[0060] S1. Obtain the visible light image taken by the visual monitoring equipment based on the power transmission line as the original image and the channel point cloud data at the same time, and use the image and point cloud data for registration. The point cloud data is adjusted through registration, and the shape in a fixed posture is basically consistent with the scene in the image, thereby determining the camera intrinsic parameter matrix and depth image information data of the monitoring equipment;

[0061] S2, perform instance segmentation on the original image and the newly captured image respectively, find the label of each wire pixel category in all labels, find the pixel point in the corresponding vertical axis direction in the newly captured image that is closest to the wire pixel in the original image, and record its displacement information;

[0062] S3. Extract the wire point cloud from the point cloud set corresponding to the current transmission channel. Considering the large space between different wires, the density-based clustering method is used to segment the wire point cloud into different single wire point clouds. At the same time, since the point cloud and the image have been registered, the single wire point cloud has a one-to-one correspondence with the segmented wire pixels.

[0063] S4. Combine the camera's intrinsic parameter matrix, depth information and pixel displacement to calculate the rough conductor point cloud after the change, fit it into smooth conductor points through the conductor point cloud fitting method, and combine it with the hidden danger positioning module to realize the acquisition of the spatial position information of the hidden danger of the transmission line.

[0064] The acquisition of visible light images taken by the visual monitoring equipment based on the transmission line and the channel point cloud data at the same time refers to installing a monitoring equipment in the transmission channel, and the image taken by the equipment at a certain moment is used as the original image, and the point cloud data is the point cloud of the real transmission scene scanned at the same moment, and the newly taken picture refers to the image taken by the monitoring equipment after a certain interval.

[0065] The method of obtaining the camera intrinsic parameter matrix data of the monitoring device by registration refers to the registration of the original monitoring image with the wire point cloud, and the registration parameters include the intrinsic parameter M and the extrinsic parameters R and T, where R represents the rotation matrix and T represents the translation matrix.

[0066] Specifically, first, the image captured during point cloud acquisition is used as the reference for the channel image, and the camera intrinsic matrix is ​​manually adjusted and the translation and rotation of the point cloud are fine-tuned until the point cloud matches the channel image. The camera intrinsic matrix data of the captured image is obtained: it is a 3*3 matrix that represents the mapping relationship between the point cloud and the depth map. It is calibrated by the camera intrinsic matrix data or calculated by the visible light image size, the camera focal length f and the camera CCD or COMS sensor size. The specific form is as follows:

[0067]

[0068] Among them, M is the camera internal parameter matrix, f x Indicates the focal length of the surveillance camera in the x-axis (horizontal) direction; f y Indicates the length of the focal length of the surveillance camera in the y-axis (vertical) direction; c x Indicates the horizontal center position of the image coordinate system, c y Indicates the vertical center position of the image coordinate system.

[0069] Using image and point cloud data for registration, the external parameter needs to be the three-dimensional point cloud data P in the world coordinate system w Translate and rotate to convert the point cloud P to the camera coordinate system with the current tower as the origin and the opposite tower as the positive direction of the y-axis c .

[0070] In this embodiment, the world coordinate system has a world origin, and the coordinates of any point defined in the space are defined relative to the origin. w The axis is horizontal to the right, Y w Axis down, Z w Determined by the right-hand rule, the camera coordinate system is the optical center (pinhole) of the camera when the image is taken as the origin and the horizontal direction is X c Axis, vertical direction is Y c Axis, pointing to the direction the camera was looking at when taking the first image is Z c axis.

[0071] The three-dimensional point cloud data based on the world coordinate system is rotated and translated with the monitoring camera as the origin through a 3*3 rotation matrix and a 3*1 translation matrix, namely:

[0072]

[0073] In formula (1): P rt is the point cloud after translation and rotation; R is the rotation matrix; T is the translation matrix, P w =[x i ,y i ,z i], represents the original point cloud set of the transmission channel, x i is the horizontal position of the i-th point in the point cloud, y i is the vertical position of the i-th point in the point cloud, z i is the position of the i-th point in the point cloud set perpendicular to the horizontal plane; is the direction vector of the x-axis in the original coordinate system after rotation; [r 12 r 22 r 32 ] T is the direction vector of the y-axis in the original coordinate system after rotation; [r 13 r 23 r 33 ] T is the direction vector of the z-axis in the original coordinate system after rotation; [t x t y t z ] T Respectively represent P W Convert to P rt The translation amount on the x-axis, y-axis and z-axis, and n is the total number of points in the point cloud data.

[0074] The instance segmentation of the original image and the newly captured image is a task in computer vision. Each object instance in the original image and the newly captured image is segmented using an instance segmentation algorithm based on deep learning to identify different objects in the image and outline each object, even if they belong to the same category.

[0075] The segmentation of the wire point cloud corresponding to the channel into a single wire point cloud refers to segmenting the wires into different categories using a density-based clustering method DBSCAN, wherein the density-based clustering method DBSCAN is used to divide data points into high-density areas and low-density areas, wherein the high-density areas are regarded as clusters:

[0076] First, the point cloud data of the wire is taken as input, where each point contains three-dimensional space coordinate information;

[0077] Next, select the DBSCAN parameters, neighborhood radius, and minimum number of points, and traverse all points using the DBSCAN algorithm:

[0078] If a point is a core point, try to expand the current cluster and add all density-reachable points, that is, points reachable from the core point through other core points, to the same cluster;

[0079] If a point is a boundary point, it is assigned to the cluster of its core points;

[0080] Noise points are ignored and not assigned to any cluster;

[0081] Finally, for the wire point cloud, each cluster corresponds to a wire. In this way, even if the wires are close to each other or cross each other in space, the density-based clustering method DBSCAN can segment them into different categories according to the change of density.

[0082] The method of finding the nearest neighbor pixel points of the wire pixels in the original image and the corresponding wire pixels in the newly captured image and recording their displacement information is specifically to establish a one-to-one matching relationship between the wire pixels of the newly captured image and the nearest neighbor pixels in the vertical axis u direction of each wire pixel in the original image point by point, and to record the displacement Δu between the corresponding pixels in the vertical axis u direction of the image point by point. j .

[0083] The estimated rough wire point cloud after the change refers to converting the depth image information of each pixel into a rough spatial wire point cloud using the camera intrinsic parameter matrix and the pixel displacement. Specifically, the pixel displacement Δu of each wire pixel point in the u direction on the reference image and the current image is calculated using the registration parameters obtained in the above steps. j Converted into the spatial variation Δz of the wire point in the three-dimensional space coordinate system j , by adding the spatial variation of each wire point to the original wire point cloud, the reconstructed wire point cloud is obtained;

[0084] The pixel displacement Δu of each wire pixel point in the u direction on the reference image and the current image is calculated. j Converted into the spatial variation Δz of the wire point in the three-dimensional space coordinate system j The process is the inverse process of point cloud to depth map conversion, by comparing the original wire pixel position in the image with the known pixel depth depth j , the pixel displacement Δu of the jth wire pixel in the u direction on the reference image and the current image is calculated under the current depth map depth by the pinhole imaging principle j The real-world distance represented by , accurately maps the wire points in the image to the spatial locations in the real world:

[0085]

[0086] In formula (3), Δz j Indicates the spatial variation corresponding to the j-th pixel displacement, Dis p represents the horizontal distance from the jth 3D wire point to the camera, Δu j represents the pixel displacement of the jth wire pixel in the u direction between the reference image and the current image, and focal is the focal length of the camera;

[0087] Finally, after obtaining the spatial variation of the wire points in the three-dimensional space coordinate system, the reconstructed wire point cloud set P can be obtained by adding the spatial variation to the corresponding original wire point cloud. re , specifically:

[0088] P re =[x j ,y j ,z j +Δz j ],j=1,2,...,m; (4)

[0089] In formula (4), x j Indicates the horizontal position of the space point corresponding to the j-th wire pixel point, y j Indicates the vertical position of the space point corresponding to the j-th wire pixel point, z j Indicates the position of the space point corresponding to the j-th wire pixel in the direction perpendicular to the horizontal plane, Δz j It represents the spatial variation corresponding to the displacement of the jth pixel, and m is the total number of wire pixels.

[0090] The wire point cloud fitting method refers to fitting a rough wire point cloud with noise into a smooth and continuous wire point cloud using the least square method. Since the wires of the transmission line can be assumed to be ideal flexible curves, various curve models can be used to approximate their spatial paths. For example, a polynomial curve in three-dimensional space (such as a cubic spline curve) is used to fit the wire point cloud. At the same time, since the wire information contained in the image is not complete, the wire fitting needs to intercept the part of the point cloud closest to the lens in the original point cloud set and add it to the reconstructed rough point cloud set, and then fit it into a smooth and complete wire point cloud.

[0091] The acquisition of the spatial position information of the hidden dangers in the transmission line means that after the above steps, the newly generated conductor point cloud is more consistent with the spatial form of the changed conductor in the newly taken image. Therefore, if the spatial position of the hidden danger in the transmission channel is known, the clearance distance between the hidden danger and the conductor can be calculated more accurately.

[0092] Embodiment 2,

[0093] Step 1: Install a monitoring device in the transmission channel, and use the image taken by the device at a certain moment as the original image ( Figure 2 ), while the point cloud data is the point cloud of the real power transmission scene scanned at the same time ( Figure 3 ), due to the different ways of collecting image and point cloud data, the collection time cannot be completely consistent. The same time here means that the collection time is similar. The monitoring device takes a new picture after a certain interval as the new picture ( Figure 5 ).

[0094] Step 2: Use the original surveillance image to align with the wire point cloud to obtain the camera parameters of the captured image, which is a 3*3 matrix. It represents the mapping relationship between the point cloud and the depth map, which can be obtained through camera parameter calibration or through calculation of the visible light image size, camera focal length f and camera CCD or COMS sensor size. The specific form is as follows:

[0095]

[0096] Step 3: The deep learning-based instance segmentation algorithm performs fine segmentation on each object instance in the new and old images. Figure 4 , Figure 6 As shown in Figure 2. It is not only necessary to identify different objects in an image, but also to accurately outline each object, even if they belong to the same category.

[0097] Step 4: Use the density-based clustering method DBSCAN to segment the wires into different categories, such as Figure 7 As shown, the point cloud is classified into 2 categories

[0098] Step 5: Find the nearest neighbor pixel points of the wire pixels in the original image and the corresponding wire pixels in the newly captured image. This is to find the wire pixels of the newly captured image that are the nearest neighbors of each wire pixel in the original image point by point. Figure 8 As shown, a one-to-one matching relationship is established.

[0099] Step 6: Estimating the rough wire point after the change means that after the new image is taken to obtain the depth image information, the depth image information of each pixel can be converted into a rough spatial wire point cloud using the camera parameter information ( Fig. 9 ), it is worth noting that due to the camera perspective, the reconstructed rough wire point cloud cannot be like the point cloud scanned by the drone, so there is a certain blind area above the camera's field of view. Therefore, part of the point cloud near the camera of the original wire point cloud can be intercepted.

[0100] Rough wire point cloud refers to the point cloud that is only the depth image information obtained by the neighbor matching method. After being converted into a spatial point cloud, it is inevitable that certain errors will occur, and even some noisy wire point clouds will appear. These wire point cloud sets are collectively referred to as rough wire point clouds.

[0101] Step 7: The rough wire point cloud with noise and the original point cloud near the camera are fitted into a smooth and continuous wire point cloud using the least squares method ( Fig.10 After the above steps, the newly generated wire point cloud is more consistent with the spatial form of the wire after the change in the new image ( Fig.11 ), so if the spatial location of the hidden danger in the transmission channel is known, the clearance distance between the hidden danger and the conductor can be calculated more accurately.

Claims

1. A method for spatial reconstruction of a single wire based on an image, characterized in that: The method comprises the following steps: S1. Obtain the visible light image taken by the visual monitoring device based on the transmission line as the original image and the channel point cloud data at the same time, and use the image and point cloud data for registration to determine the camera intrinsic parameter matrix and depth image information data of the monitoring device; S2, perform instance segmentation on the original image and the newly captured image respectively, find the label of each wire pixel category in all labels, find the pixel point in the corresponding vertical axis direction in the newly captured image that is closest to the wire pixel in the original image, and record its displacement information; S3, extracting the wire point cloud in the point cloud set corresponding to the current transmission channel, and segmenting the wire point cloud into different single wire point clouds. At the same time, since the point cloud data and the image have been registered, the single wire point cloud and the segmented wire pixels have a one-to-one correspondence; S4. Combining the camera intrinsic parameter matrix, depth image information and pixel displacement, the rough wire point cloud after the change is calculated, and it is fitted into a smooth wire point cloud through the wire point cloud fitting method.

2. The image-based spatial reconstruction method of a single wire according to claim 1, characterized in that: The acquisition of visible light images taken by the visual monitoring equipment based on the transmission line and the channel point cloud data at the same time refers to installing a monitoring equipment in the transmission channel, and the image taken by the equipment at a certain moment is used as the original image, and the point cloud data is the point cloud of the real transmission scene scanned at the same moment, and the newly taken picture refers to the image taken by the monitoring equipment after a certain interval.

3. The image-based spatial reconstruction method of a single wire according to claim 1, characterized in that: The image and point cloud data are registered by using the original image and the wire point cloud for registration, and the registration parameters include an internal parameter M and external parameters R and T, where R represents a rotation matrix and T represents a translation matrix; Specifically, first, the image captured during point cloud acquisition is taken as the reference of the channel image, and the camera intrinsic parameter matrix is ​​manually adjusted and the translation and rotation of the point cloud are fine-tuned until the point cloud matches the channel image to obtain the camera intrinsic parameter matrix data of the captured image: it is calibrated by the camera intrinsic parameter matrix data or calculated by the visible light image size, the camera focal length f and the camera CCD or COMS sensor size. The specific form is as follows: In formula (1), M is the camera intrinsic parameter matrix, f x Indicates the length of the horizontal focal length of the surveillance camera on the x-axis, f y Indicates the length of the vertical focal length of the surveillance camera on the y-axis, c x Indicates the horizontal center position of the image coordinate system, c y Indicates the vertical center position of the image coordinate system; The image and wire data are used for registration. The external parameters need to be the 3D point cloud data P in the world coordinate system. w Translate and rotate to convert the point cloud P to the camera coordinate system with the current tower as the origin and the opposite tower as the positive direction of the y-axis c ; The world coordinate system has a world origin and defines the coordinates of any point defined in the space relative to the origin. w The axis is horizontal to the right, Y w Axis down, Z w Determined by the right-hand rule, the camera coordinate system is the optical center of the camera when the image is taken as the origin and the horizontal direction is X c Axis, vertical direction is Y c Axis, pointing to the direction the camera was looking at when taking the first image is Z c axis; The three-dimensional point cloud data based on the world coordinate system is rotated and translated with the monitoring camera as the origin through a 3*3 rotation matrix and a 3*1 translation matrix, namely: In formula (2): P rt is the point cloud after translation and rotation; R is the rotation matrix; T is the translation matrix, P w =[x i ,y i ,z i ], represents the original point cloud set of the transmission channel, x i is the horizontal position of the i-th point in the point cloud, y i is the vertical position of the i-th point in the point cloud, z i is the position of the ith point in the point cloud in the direction perpendicular to the horizontal plane; [r 11 r 12 r 13 ] T is the direction vector of the x-axis in the original coordinate system after rotation; [r 12 r 22 r 32 ] T is the direction vector of the y-axis in the original coordinate system after rotation; [r 13 r 23 r 33 ] T is the direction vector of the z-axis in the original coordinate system after rotation; [t x t y t z ] T Respectively represent P W Convert to P rt The translation amount on the x-axis, y-axis and z-axis, and n is the total number of points in the point cloud data.

4. The image-based spatial reconstruction method of a single wire according to claim 1, characterized in that: The instance segmentation of the original image and the newly taken image is a task in computer vision. Each object instance in the original image and the newly taken image is segmented using an instance segmentation algorithm based on deep learning to identify different objects in the image and depict the outline of each object.

5. The image-based spatial reconstruction method of a single wire according to claim 1, characterized in that: The segmentation of the wire point cloud into different single wire point clouds refers to using the density clustering method DBSCAN to divide the data points into high-density areas and low-density areas, where the high-density areas are regarded as clusters: First, the point cloud data of the wire is taken as input, where each point contains three-dimensional space coordinate information; Next, select the DBSCAN parameters, neighborhood radius, and minimum number of points, and traverse all points using the DBSCAN algorithm: If a point is a core point, try to expand the current cluster and add all density-reachable points, that is, points reachable from the core point through other core points, to the same cluster; If a point is a boundary point, it is assigned to the cluster of its core points; Noise points are ignored and not assigned to any cluster; Finally, for the wire point cloud, each cluster corresponds to a wire.

6. The image-based spatial reconstruction method of a single wire according to claim 1, characterized in that: The method of finding the pixel point in the newly captured image that is closest to the wire pixel in the original image in the corresponding vertical axis direction and recording its displacement information refers to establishing a one-to-one matching relationship between the wire pixels in the newly captured image that are closest to each wire pixel in the original image in the vertical axis u direction by finding the pixel point by point, and recording the displacement Δu between the corresponding pixels in the vertical axis u direction of the image. j .

7. The image-based spatial reconstruction method of a single wire according to claim 1, characterized in that: The estimated rough wire point cloud after the change refers to converting the depth image information of each pixel into a rough spatial wire point cloud using the camera intrinsic parameter matrix and the pixel displacement. Specifically, the pixel displacement Δu of each wire pixel point in the u direction on the reference image and the current image is calculated using the registration parameters obtained in the above steps. j Converted into the spatial variation Δz of the wire point in the three-dimensional space coordinate system j , by adding the spatial variation of each wire point to the original wire point cloud, the reconstructed wire point cloud is obtained; The pixel displacement Δu of each wire pixel point in the u direction on the reference image and the current image is calculated. j Converted into the spatial variation Δz of the wire point in the three-dimensional space coordinate system j The process is the inverse process of point cloud to depth map conversion, by comparing the original wire pixel position in the image with the known pixel depth depth j , the pixel displacement Δu of the jth wire pixel in the u direction on the reference image and the current image is calculated under the current depth map depth by the pinhole imaging principle j The real-world distance represented by , accurately maps the wire points in the image to the spatial locations in the real world: In formula (3), Δz j Indicates the spatial variation corresponding to the j-th pixel displacement, Dis p represents the horizontal distance from the jth 3D wire point to the camera, Δu j represents the pixel displacement of the jth wire pixel in the u direction between the reference image and the current image, and focal is the focal length of the camera; Finally, after obtaining the spatial variation of the wire points in the three-dimensional space coordinate system, the reconstructed wire point cloud set P can be obtained by adding the spatial variation to the corresponding original wire point cloud. re , specifically: P re =[x j ,y j ,z j +Δz j ],j=1,2,...,m; (4) In formula (4), x j Indicates the horizontal position of the space point corresponding to the j-th wire pixel point, y j Indicates the vertical position of the space point corresponding to the j-th wire pixel point, z j Indicates the position of the space point corresponding to the j-th wire pixel in the direction perpendicular to the horizontal plane, Δz j It represents the spatial variation corresponding to the displacement of the jth pixel, and m is the total number of wire pixels.

8. The image-based spatial reconstruction method of a single wire according to claim 1, characterized in that: The wire point cloud fitting method refers to fitting a rough wire point cloud with noise into a smooth and continuous wire point cloud using the least square method. Since the wires of the transmission line can be assumed to be ideal flexible curves, various curve models can be used to approximate their spatial paths, such as using polynomial curves in three-dimensional space to fit the wire point cloud, such as cubic spline curves. At the same time, since the wire information contained in the image is not complete, the wire fitting needs to intercept the part of the point cloud closest to the lens side in the original point cloud set and add it to the reconstructed rough point cloud set, so as to fit it into a smooth and complete wire point cloud.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the above method can be implemented.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the above method can be implemented.

Citation Information

Cited By

  • Rapid registration method for offset camera extrinsic parameters based on acquired point cloud image

    CN119579665A

  • Monitoring method and monitoring system based on intelligent medical waste collection vehicle

    CN120877233A