Wire change detection and correction method and device of power transmission channel, electronic equipment and storage medium

Abstract

The invention belongs to the field of intelligent operation inspection of power transmission lines, and particularly relates to a wire change detection and correction method and device of a power transmission channel, electronic equipment and a storage medium. The method comprises the steps of converting three-dimensional point cloud data of a power transmission channel under a world coordinate system into point cloud data under a camera coordinate system, and performing registration on reference image data and the point cloud data to obtain camera registration parameters; classifying the point cloud data, extracting a traverse point cloud in the point cloud data, and converting the traverse point cloud into a depth map by using camera registration parameters; capturing wire changes in the reference image data and the current image data by using a dense optical flow estimation algorithm, and generating an optical flow matrix describing wire pixel motion; and converting the optical flow matrix into a spatial variation of the traverse point in a three-dimensional space coordinate system in combination with the depth map, and adding the spatial variation to the corresponding traverse point cloud in the point cloud data so as to correct the traverse point cloud. According to the invention, distance measurement errors caused by wire changes can be effectively suppressed and eliminated.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent operation and maintenance of transmission lines, and more specifically, relates to a method, device, electronic device and storage medium for detecting and correcting wire changes in a transmission corridor. Background Art

[0002] Overhead transmission lines are important power transmission facilities connecting power generation stations and power consumption areas. They often span vast areas and are exposed to the natural environment for a long time. Since most of these lines are erected outdoors, they will inevitably be affected by weather conditions such as wind, rain, snow, and hail. Temperature changes, especially the occurrence of extreme temperatures, will have a significant impact on the physical properties of transmission lines, thereby affecting the stability of their spatial forms.

[0003] In the operation of the power system, the sag of the wire is a key parameter. Sag refers to the vertical offset of the wire under the action of its own weight and external forces, and it is directly related to the safe distance between the wire and the ground or other objects. Temperature fluctuations will affect the length of the wire through the principle of thermal expansion and contraction, thereby changing the size of the sag. In the hot summer, the wire expands due to heat, resulting in an increase in sag; while in the cold winter, the wire contracts, resulting in a decrease in sag. If this change is not considered, it will cause errors in the ranging results.

[0004] In current transmission corridor ranging projects, the impact of wire changes caused by external factors such as temperature difference on ranging errors has not been effectively solved. Although the combination of images and point clouds can map two-dimensional images to three-dimensional space, and then realize three-dimensional ranging of hidden danger targets and protected objects (such as wires) to give users different levels of safety warning levels in real time and intelligently. However, the changing transmission lines obviously pose a severe challenge to the ranging accuracy of this solution. Collecting three-dimensional information at intervals clearly does not meet the requirements of existing unmanned intelligent monitoring, and it is also not conducive to the cost control needs of users and enterprises themselves. Many feedbacks from on-site power users show that the sag change of the wire caused by external climate and other factors is the main reason for the ranging error of hidden dangers in the transmission scenario. Therefore, it is very urgent and necessary to study new technologies for inversing three-dimensional space information from two-dimensional dynamic changes of sag.

[0005] In order to improve the accuracy and reliability of ranging technology, it is necessary to develop new methods to suppress and eliminate ranging errors caused by wire changes, enabling it to consider the thermal expansion and contraction effect of the wire in real time, thereby improving the accuracy of transmission corridor ranging, effectively enhancing the ranging accuracy of overhead transmission lines, and ensuring the safe and stable operation of the power system. Summary of the Invention

[0006] The present invention aims to overcome at least one defect of the above-mentioned prior art, and provides a method for detecting and correcting wire changes in a power transmission channel, so as to suppress and eliminate the ranging error caused by wire changes.

[0007] The present invention also discloses a device loaded with the method for detecting and correcting wire changes in a power transmission channel.

[0008] The detailed technical solution of the present invention is as follows:

[0009] A method for detecting and correcting wire changes in a power transmission channel, the method comprising:

[0010] S1. Obtain the three-dimensional point cloud data P of the power transmission channel in the world coordinate system, and perform translation and rotation on the three-dimensional point cloud data P to convert it into the point cloud data P in the camera coordinate system; W and perform translation and rotation on the three-dimensional point cloud data P W to convert it into the point cloud data P in the camera coordinate system; C ;

[0011] S2. Obtain the reference image data of the power transmission channel, and register the reference image data with the point cloud data P to obtain camera registration parameters, where the camera registration parameters include the internal parameter M and the external parameters R and T, where R represents the rotation matrix and T represents the translation matrix; C ;

[0012] S3. Classify the point cloud data P using the PointNet algorithm, extract the wire point cloud in the point cloud data P, and convert the wire point cloud into a depth map depth using the camera registration parameters; C and extract the wire point cloud in the point cloud data P C ;

[0013] S4. Obtain the current image data of the power transmission channel, and use the dense optical flow estimation algorithm to capture the wire changes in the reference image data and the current image data, and generate an optical flow matrix for describing the wire pixel movement;

[0014] S5. Combine the depth map depth, convert the optical flow matrix into the spatial change amount of the wire points in the three-dimensional space coordinate system, and add the spatial change amount to the corresponding wire point cloud in the point cloud data P to correct the wire point cloud. C ;

[0015] Preferably according to the present invention, in S1, with the camera as the origin, the three-dimensional point cloud data P W is rotated and translated through a 3×3 rotation matrix and a 3×1 translation matrix respectively to be converted into the point cloud data P in the camera coordinate system as: C :

[0016]

[0017] In formula (1): P C represents the point cloud after translation and rotation, that is, the point cloud data in the camera coordinate system; P W = [X i , Y i , Z i represents the point cloud before translation and rotation, that is, the three-dimensional point cloud data in the world coordinate system, and X i represents the position of the i-th point cloud in the three-dimensional point cloud data in the horizontal direction, Y i represents the position of the i-th point cloud in the three-dimensional point cloud data in the vertical direction, Z i represents the position of the i-th point cloud in the three-dimensional point cloud data in the direction perpendicular to the horizontal plane, n represents the number of point clouds; R represents the rotation matrix, and T represents the translation matrix;

[0018] represents the direction vector of the x-axis of the rotated camera coordinate system in the world coordinate system;

[0019] represents the direction vector of the y-axis of the rotated camera coordinate system in the world coordinate system;

[0020] represents the direction vector of the z-axis of the rotated camera coordinate system in the world coordinate system;

[0021] correspondingly represents the translation amounts of the three-dimensional point cloud data P W after translation on the x-axis, y-axis, and z-axis of the camera coordinate system respectively;

[0022] Among them, the camera is installed on the pole tower and is used to obtain the reference image data and the current image data of the transmission line corridor.

[0023] According to the preference of the present invention, in S2, registering the reference image data with the point cloud data P C specifically includes:

[0024] Taking the transmission line corridor image taken at the same time as when collecting the three-dimensional point cloud data P W as the reference for the reference image data, and manually adjusting the internal parameter M of the camera to make the point cloud data P C match the reference image data.

[0025] According to the preference of the present invention, in S2, the matrix of the internal parameter M is:

[0026]

[0027] In formula (2): f x represents the focal length of the camera in the horizontal direction; f yRepresents the length of the focal length of the camera in the vertical direction; c x Represents the position of the center point in the horizontal direction of the image coordinate system; c y Represents the position of the center point in the vertical direction of the image coordinate system.

[0028] According to a preferred embodiment of the present invention, in step S3, the wire point cloud is converted into a depth map depth by using the camera internal parameter M, that is:

[0029]

[0030] In formula (3): x i , y i respectively represent the horizontal and vertical coordinates of the i-th wire point cloud in the point cloud data P C , and n represents the number of point clouds; u i and v i respectively represent the pixel positions of the i-th wire point cloud in the u direction and the v direction.

[0031] According to a preferred embodiment of the present invention, in step S5, the optical flow matrix is converted into the spatial change amount of the wire points in the three-dimensional space coordinate system, specifically:

[0032]

[0033]

[0034] In formulas (4) and (5): Δu i and Δv i represent the pixel displacement amounts of the i-th wire pixel point in the u direction and the v direction in the reference image and the current image respectively, Δuv i represents the comprehensive displacement amount of the i-th wire pixel point corresponding in the reference image and the current image, focal is the focal length of the camera, Dis p represents the horizontal distance from the i-th wire point cloud to the camera in the camera coordinate system, ΔP i represents the spatial displacement amount corresponding to the pixel displacement amount of the i-th wire pixel point, that is, the spatial change amount.

[0035] According to a preferred embodiment of the present invention, in step S5, the spatial change amount is added to the corresponding wire point cloud in the point cloud data P C to obtain the corrected wire point cloud set as:

[0036] P cor =[x i , y i , z i +ΔP i , i = 1, 2,..., n; (6);

[0037] In Equation (6): P cor represents the corrected wire point cloud set, and the point cloud data P C = [x i , y i , z i , where x i represents the position of the i-th wire point cloud in the point cloud data in the horizontal direction, y i represents the position of the i-th wire point cloud in the point cloud data in the vertical direction, z i represents the position of the i-th wire point cloud in the point cloud data in the direction perpendicular to the horizontal plane, and n represents the number of point clouds.

[0038] In another aspect of the present invention, there is provided a device for implementing a method for detecting and correcting wire changes in a transmission channel, the device comprising:

[0039] A first acquisition module, configured to acquire three-dimensional point cloud data P of a transmission channel in a world coordinate system W , and perform translation and rotation on the three-dimensional point cloud data P W to convert it into point cloud data P C in a camera coordinate system;

[0040] A second acquisition module, configured to acquire reference image data of a transmission channel, and register the reference image data with the point cloud data P C to obtain camera registration parameters, where the camera registration parameters include an internal parameter M and external parameters R, T, where R represents a rotation matrix and T represents a translation matrix;

[0041] A classification and extraction module, configured to classify the point cloud data P C using the PointNet algorithm, and extract the wire point cloud in the point cloud data P C , and convert the wire point cloud into a depth map depth using the camera registration parameters;

[0042] A third acquisition module, configured to acquire current image data of a transmission channel, and capture wire changes in the reference image data and the current image data using a dense optical flow estimation algorithm to generate an optical flow matrix for describing the movement of wire pixels;

[0043] A correction module, configured to combine the depth map depth to convert the optical flow matrix into a spatial change amount of wire points in a three-dimensional space coordinate system, and add the spatial change amount to the corresponding wire point cloud in the point cloud data P C to correct the wire point cloud.

[0044] In another aspect of the present invention, there is also provided an electronic device, comprising:

[0045] At least one processor; and

[0046] A memory that stores instructions which, when executed by the at least one processor, cause the at least one processor to perform the method for detecting and correcting wire changes in a power transmission channel as described above.

[0047] In another aspect of the present invention, a machine-readable storage medium is also provided, which stores executable instructions that, when executed, cause the machine to perform the method for detecting and correcting wire changes in a power transmission channel as described above.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] (1) A method for detecting and correcting wire changes in a power transmission channel provided by the present invention captures two-dimensional dynamic changes of wires at different times based on the method of dense optical flow estimation. At the same time, by using the correspondence and conversion relationship between three-dimensional point cloud data and two-dimensional images, real-time update of wire point clouds with images is realized for dynamic correction of wires.

[0050] (2) The method of the present invention can achieve accurate wire change detection and correction effects. Dynamic correction of wire point clouds can be realized only based on wire point cloud data and image information, avoiding the problem of invalid registration information caused by unchanged wire point clouds while image changes occur, providing a strong data basis and technical support for intelligent power transmission channel inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flowchart of the method for detecting and correcting wire changes in a power transmission channel according to the present invention.

[0052] Figure 2 is a schematic diagram of the power transmission channel point cloud collected according to an embodiment of the present invention.

[0053] Figure 3 is a schematic diagram of the power transmission channel image collected according to an embodiment of the present invention.

[0054] Figure 4 (a) is a schematic diagram of the original power transmission channel point cloud according to an embodiment of the present invention.

[0055] Figure 4 (b) is a schematic diagram of the power transmission channel point cloud after rotation and translation operations according to an embodiment of the present invention.

[0056] Figure 5 is a schematic diagram of the power transmission channel point cloud after manual registration operation according to an embodiment of the present invention.

[0057] Figure 6It is a schematic diagram of the wire point cloud obtained after point cloud classification provided by an embodiment of the present invention.

[0058] Figure 7 It is the depth map after the wire point cloud provided by an embodiment of the present invention is converted.

[0059] Figure 8 It is the optical flow map generated by the reference image, the current image and the dense optical flow estimation method provided by an embodiment of the present invention.

[0060] Figure 9 It is the superimposed image of the depth map converted from the wire point cloud before correction and the reference image provided by an embodiment of the present invention.

[0061] Figure 10 It is the superimposed image of the depth map converted from the wire point cloud before correction and the current image provided by an embodiment of the present invention.

[0062] Figure 11 It is the superimposed image of the depth map converted from the wire point cloud after correction and the current image provided by an embodiment of the present invention. Detailed implementation manners

[0063] The following further describes the present disclosure in conjunction with the accompanying drawings and embodiments.

[0064] It should be noted that the following detailed descriptions are all exemplary and are intended to provide further descriptions of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs.

[0065] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary implementation manners according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0066] Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0067] Aiming at the problem of the dynamic change of the wire in space caused by the influence of the external temperature difference on the transmission line wire, the present invention provides a method for detecting and correcting the change of the wire in the transmission channel, which can effectively correct the problem of the invalidation of the registration information caused by the change of the wire in the transmission line monitoring, and can effectively improve the ranging accuracy of the intelligent inspection. The following further describes the method and device for detecting and correcting the change of the wire in the transmission channel of the present invention in conjunction with specific embodiments.

[0068] Embodiment 1

[0069] Reference Figure 1 , this embodiment provides a method for detecting and correcting wire changes in a transmission channel. The method includes: S1. Obtain the three-dimensional point cloud data P of the transmission channel in the world coordinate system W , and perform translation and rotation on the three-dimensional point cloud data P W to convert it into point cloud data P in the camera coordinate system C .

[0070] In this embodiment, a drone equipped with a lidar (LiDAR) system can be used to collect the three-dimensional point cloud data P of the transmission channel W . During the data collection process, the drone scans the transmission channel comprehensively according to the preset flight path and altitude to obtain high-precision and high-density spatial information, ensuring the integrity and coverage of the point cloud data. The collected three-dimensional point cloud results are as Figure 2 shown, which includes wire point cloud, tower point cloud, ground point cloud and other data

[0071] It should be understood that the above-collected three-dimensional point cloud data P W is the point cloud data based on the world coordinate system. The world coordinate system is defined relative to the world Cartesian coordinate system. This coordinate system has a world origin, and the coordinates of any point defined in this space are defined relative to this origin

[0072] The camera coordinate system takes the optical center (pinhole) of the camera when taking images as the origin, the horizontal direction as the X-axis, the vertical direction as the Y-axis, and the direction pointing to the direction observed by the camera when taking the first image as the Z-axis

[0073] The translation and rotation of the three-dimensional point cloud data P W to convert it into point cloud data P in the camera coordinate system C is specifically: taking the camera as the origin, and performing rotation and translation on the three-dimensional point cloud data P W through a 3×3 rotation matrix and a 3×1 translation matrix respectively, that is:

[0074]

[0075] In formula (1): P C represents the point cloud after translation and rotation, that is, the point cloud data in the camera coordinate system; P W = [X i , Y i , Z i represents the point cloud before translation and rotation, that is, the three-dimensional point cloud data in the world coordinate system, and X i represents the position of the i-th point cloud in the three-dimensional point cloud data in the horizontal direction, and Y iRepresents the position of the \(i\)-th point cloud in the vertical direction in the three-dimensional point cloud data, \(Z\) i Represents the position of the \(i\)-th point cloud in the direction perpendicular to the horizontal plane in the three-dimensional point cloud data, \(n\) represents the number of point clouds; \(R\) represents the rotation matrix, and \(T\) represents the translation matrix;

[0076] Represents the direction vector of the \(x\)-axis of the rotated camera coordinate system in the world coordinate system;

[0077] Represents the direction vector of the \(y\)-axis of the rotated camera coordinate system in the world coordinate system;

[0078] Represents the direction vector of the \(z\)-axis of the rotated camera coordinate system in the world coordinate system;

[0079] Correspondingly represents the three-dimensional point cloud data \(P\) W The translation amounts on the \(x\)-axis, \(y\)-axis, and \(z\)-axis of the camera coordinate system after translation respectively.

[0080] The result is as Figure 4 shown, where Figure 4 (a) represents the three-dimensional point cloud data \(P\) based on the world coordinate system before conversion W , Figure 4 (b) represents the point cloud after rotation and translation, that is, the point cloud data \(P\) in the obtained camera coordinate system C . It should be noted here that the camera refers to the monitoring camera installed on the pole carrying the wire, which is used to obtain the reference image data and the current image data of the transmission channel.

[0081] S2. Obtain the reference image data of the transmission channel, and register the reference image data with the point cloud data \(P\) C to obtain the camera registration parameters. The camera registration parameters include the internal parameter \(M\) and the external parameters \(R\), \(T\), where \(R\) represents the rotation matrix and \(T\) represents the translation matrix.

[0082] In this embodiment, the reference image data of the transmission channel (i.e., the original image) can be obtained by using the camera, and the camera is installed on the pole (and the pole on which the camera is installed is used as the current pole). The obtained reference image data includes wires, opposite poles, and the ground, etc., and the result is as Figure 3 shown.

[0083] Then, register the reference image data with the point cloud data \(P\) C to obtain the camera registration parameters. Specifically: using the transmission channel image taken at the same time as when collecting the three-dimensional point cloud data \(P\) W as the reference, and manually adjusting the internal parameter \(M\) of the camera to make the point cloud data \(P\) CMatches the reference image data. The result parameter Figure 5 As shown, the camera registration parameters, i.e., the internal parameter M and the external parameters R and T, can be obtained.

[0084] Among them, the camera internal parameter M is its own property, and the matrix is as follows:

[0085]

[0086] In formula (2): f x represents the length of the focal length of the camera in the horizontal (x-axis) direction; f y represents the length of the focal length of the camera in the vertical (y-axis) direction; c x represents the position of the center point in the horizontal direction of the image coordinate system; c y represents the position of the center point in the vertical direction of the image coordinate system.

[0087] In this embodiment, the specific parameters of the above camera internal parameter M and external parameters R and T are:

[0088]

[0089] T = [2.98e-09 -1.74e+01 4.52e+01].

[0090] S3. Use the PointNet algorithm to classify the point cloud data P C and extract the wire point cloud in the point cloud data P C . Convert the wire point cloud into a depth map depth using the camera registration parameters.

[0091] The classification of the point cloud data mentioned above refers to classifying the point cloud into different point cloud sets according to different scenery categories of the transmission scene.

[0092] In this embodiment, the PointNet deep learning algorithm can be used to perform feature learning and classification on the point cloud data P C . Input the unclassified scattered point cloud and select the point cloud set with the wire category label. As Figure 6 shown, the white dotted line in it is the wire point cloud, and the black points are the tower point cloud and the ground point cloud respectively. In this embodiment, only the wire point cloud needs to be extracted, which is the wire point cloud before correction.

[0093] Then, use the camera registration parameters to convert the extracted wire point cloud in the point cloud data P C into a depth map depth. That is, through the camera imaging principle, use the camera internal parameter M in the registration parameters to convert the point cloud data P CThe wire point cloud in it is converted into the wire depth map in the power transmission channel from the perspective of the monitoring camera, realizing the conversion from the camera coordinate system to the image coordinate system; then, through the coordinates of the projection position of the camera optical axis in the pixel coordinate system, the image coordinate system is converted into the pixel coordinate system, and finally the depth map depth after the conversion of the wire point cloud is obtained as Figure 7 shown. Its calculation method is as follows:

[0094]

[0095] In Equation (3): x i , y i respectively represent the horizontal and vertical coordinates of the i-th wire point cloud in the point cloud data P C , n represents the number of point clouds; u i and v i respectively represent the pixel positions in the u direction and the v direction corresponding to the i-th wire point cloud.

[0096] S4. Obtain the current image data of the power transmission channel, and use the dense optical flow estimation algorithm to capture the wire changes in the reference image data and the current image data, and generate an optical flow matrix for describing the movement of wire pixels. In this embodiment, the current image data of the power transmission channel refers to the current captured image taken by the camera at regular intervals. Input the current image data of the power transmission channel and the initially obtained reference image data into the dense optical flow estimation algorithm, and use this algorithm to capture the wire changes in the reference image data and the current image data to generate an optical flow matrix for describing the movement of wire pixels.

[0097] The dense optical flow estimation algorithm is an image registration method for point-by-point matching of images. It calculates the offset of all points on the image to form a dense optical flow field.

[0098] Dense optical flow describes the optical flow of each pixel in the image moving to the next frame. The movement amount of the pixel points representing the same object (object) in one frame moving to the next frame is represented by a two-dimensional vector. The optical flow matrix can reflect the dynamic changes of the wire in the image sequence and provide an important basis for the calculation of spatial changes.

[0099] In this embodiment, the dense optical flow estimation algorithm is used to capture the wire changes in the reference image and the current captured image, and the finally generated optical flow matrix for describing the movement of wire pixels is as Figure 8 shown.

[0100] S5. Combine the depth map depth, convert the optical flow matrix into the spatial change amount of the wire points in the three-dimensional space coordinate system, and add the spatial change amount to the corresponding wire point clouds in the point cloud data P C to correct the wire point cloud.

[0101] The process of converting the optical flow matrix (two-dimensional displacement) of the wire pixel movement into the spatial change of the wire point in the three-dimensional space coordinate system is the inverse process of the point cloud to depth map conversion. By using the optical flow information [Δu i , Δv i at a certain wire point and the depth map depth i of the i-th wire pixel point known, the real spatial distance represented by the pixel displacement under the current depth map depth can be calculated. Therefore, after calculating the comprehensive displacement Δuv i of the i-th wire pixel point, the real spatial displacement corresponding to the pixel displacement can be obtained through the principle of pinhole imaging, and the wire point in the image can be accurately mapped to the spatial position in the real world. As Figure 8 shown. Specifically:

[0102]

[0103]

[0104] In Equations (4) and (5): Δu i and Δv i represent the pixel displacement amounts of the i-th wire pixel point in the u-direction and v-direction of the reference image and the current image respectively, Δuv i represents the comprehensive displacement amount of the i-th wire pixel point corresponding in the reference image and the current image, focal is the focal length of the camera, Dis p represents the horizontal distance from the i-th wire point cloud to the camera in the camera coordinate system, and ΔP i represents the spatial displacement amount corresponding to the pixel displacement amount of the i-th wire pixel point, that is, the spatial change amount.

[0105] After the wire is corrected to obtain the spatial displacement of the wire point cloud, each point cloud displacement amount is added to the elevation of the corresponding point cloud. Since the change of the wire is often caused by the external temperature difference, this influencing factor makes the sag of the wire prone to vertical changes in elevation. This change is a slow dynamic process, while the left-right spatial dancing of the wire caused by factors such as wind resistance will have small-amplitude high-frequency changes in a very short space. Based on this, in this embodiment, the latter change is ignored, and mainly the sag change of the wire is corrected. Therefore, only the change amount is added to the elevation of the point cloud.

[0106] That is, after obtaining the spatial change amount of the wire point in the three-dimensional space coordinate system, by adding this spatial change amount to the corresponding wire point cloud in the point cloud data P C , the correction of the wire point cloud can be achieved.

[0107] Finally, the corrected wire point cloud set obtained is:

[0108] Pcor = [x i , y i , z i + ΔP i , i = 1, 2,..., n; (6);

[0109] In formula (6): P cor represents the corrected wire point cloud set. The point cloud data P C = [x i , y i , z i , and x i represents the position of the i-th wire point cloud in the point cloud data in the horizontal direction, y i represents the position of the i-th wire point cloud in the point cloud data in the vertical direction, z i represents the position of the i-th wire point cloud in the point cloud data in the direction perpendicular to the horizontal plane, and n represents the number of point clouds.

[0110] As shown in Figure 9 and Figure 10 , they are the superimposed diagrams of the wire before correction in the reference image and the current image respectively. As shown in Figure 11 , it is the superimposed diagram of the wire after correction in the current image. It can be seen that this method has the ability to track the change of the wire and correct the wire point cloud.

[0111] In summary, for the wire change detection and correction method of the transmission channel described in the present invention, it adopts the method of combining three-dimensional scattered point cloud and two-dimensional depth map. By integrating depth image processing, optical flow estimation, image registration, and coordinate transformation technologies, it realizes the precise correction of the wire point cloud data of the transmission line, thereby obtaining the dynamic change information of the wire in the real space and realizing the implementation update of the wire point cloud that changes with time in the transmission channel. The method of the present invention not only improves the accuracy of transmission line monitoring, but also provides technical support for the unmanned and intelligent hidden danger detection of transmission network lines and the maintenance and safe operation of lines.

[0112] Embodiment 2

[0113] This embodiment provides a device for implementing the wire change detection and correction method of the transmission channel. The device includes: a first acquisition module for acquiring the three-dimensional point cloud data P W of the transmission channel in the world coordinate system, and performing translation and rotation on the three-dimensional point cloud data P W to convert it into the point cloud data P C in the camera coordinate system;

[0114] A second acquisition module for acquiring the reference image data of the transmission channel and combining the reference image data with the point cloud data P CPerform registration to obtain camera registration parameters, where the camera registration parameters include an internal parameter M and external parameters R and T. Here, R represents a rotation matrix and T represents a translation matrix;

[0115] A classification and extraction module for classifying the point cloud data P using the PointNet algorithm C and extracting the wire point cloud in the point cloud data P C and converting the wire point cloud into a depth map depth using the camera registration parameters;

[0116] A third acquisition module for acquiring the current image data of the transmission line corridor and capturing the changes in the wires in the reference image data and the current image data using a dense optical flow estimation algorithm to generate an optical flow matrix for describing the movement of wire pixels;

[0117] A correction module for converting the optical flow matrix into a spatial change amount of the wire points in a three-dimensional space coordinate system in combination with the depth map depth, and adding the spatial change amount to the corresponding wire point cloud in the point cloud data P C to correct the wire point cloud.

[0118] Example 3

[0119] This embodiment also provides an electronic device, including:

[0120] At least one processor; and

[0121] A memory storing instructions that, when executed by the at least one processor, cause the at least one processor to execute the method for detecting and correcting wire changes in the transmission line corridor as described above. In this embodiment, the electronic device may include, but is not limited to: a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile computing device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable computing device, a consumer electronic device, etc.

[0122] Example 4

[0123] This embodiment also provides a machine-readable storage medium storing executable instructions that, when executed, cause the machine to execute the method for detecting and correcting wire changes in the transmission line corridor as described above. Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes for implementing the functions of any one of the above embodiments are stored, and the computer or processor of the system or device reads and executes the instructions stored in the readable storage medium.

[0124] In this case, the program code read from the readable medium itself can implement the functions of any one of the above-described embodiments. Therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of this specification.

[0125] Examples of the readable storage medium include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code can be downloaded from a server computer or a cloud via a communication network.

[0126] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0127] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0128] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are performed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide means for implementing the functions specified in Figure 1One process or multiple processes and / or boxes Figure 1 Steps of functions specified in one box or multiple boxes.

[0130] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solutions of the present invention, rather than limitations on the specific implementation manners of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the claims of the present invention shall be included within the protection scope of the claims of the present invention.

Claims

1. A method for detecting and correcting wire changes in a power transmission channel, characterized in that, The method includes: S1. Obtain the three-dimensional point cloud data P of the power transmission channel in the world coordinate system W , and perform translation and rotation on the three-dimensional point cloud data P W to convert it into the point cloud data P in the camera coordinate system C ; S2. Obtain the reference image data of the transmission channel, and register the reference image data with the point cloud data P C to obtain camera registration parameters, where the camera registration parameters include the internal parameter M and the external parameters R and T. Here, R represents the rotation matrix and T represents the translation matrix; S3. Classify the point cloud data P using the PointNet algorithm C and extract the wire point cloud in the point cloud data P C ; convert the wire point cloud into a depth map depth using the camera registration parameters. S4. Obtain the current image data of the transmission line corridor, and use the dense optical flow estimation algorithm to capture the wire changes in the reference image data and the current image data, and generate an optical flow matrix for describing the movement of wire pixels; S5. Combine with the depth map depth, convert the optical flow matrix into the spatial variation of the wire points in the three-dimensional space coordinate system, and add the spatial variation to the corresponding wire point cloud in the point cloud data P C to correct the wire point cloud.

2. The method for detecting and correcting wire changes in a power transmission channel according to claim 1, characterized in that, In the above S1, taking the camera as the origin, the three-dimensional point cloud data P W is rotated and translated through a 3×3 rotation matrix and a 3×1 translation matrix respectively, and is converted into the point cloud data P in the camera coordinate system C as follows: In formula (1): P C represents the translated and rotated point cloud, i.e., the point cloud data in the camera coordinate system; P W =[X i , Y i , Z i represents the point cloud before translation and rotation, i.e., the three-dimensional point cloud data in the world coordinate system, and X i represents the position of the i-th point cloud in the three-dimensional point cloud data in the horizontal direction, Y i represents the position of the i-th point cloud in the three-dimensional point cloud data in the vertical direction, Z i represents the position of the i-th point cloud in the three-dimensional point cloud data in the direction perpendicular to the horizontal plane. n represents the number of point clouds; R represents the rotation matrix, and T represents the translation matrix; represents the direction vector of the x-axis of the rotated camera coordinate system in the world coordinate system; represents the direction vector of the y-axis of the rotated camera coordinate system in the world coordinate system; represents the direction vector of the z-axis of the rotated camera coordinate system in the world coordinate system; correspondingly represents the translation amounts of the three-dimensional point cloud data P W after translation on the x-axis, y-axis, and z-axis of the camera coordinate system respectively; Among them, the camera is installed on the pole tower and is used to obtain the reference image data and the current image data of the transmission line corridor.

3. The method for detecting and correcting wire changes in a power transmission channel according to claim 1, characterized in that, In S2, register the reference image data with the point cloud data P C Specifically, the registration is performed as follows: Taking the transmission line corridor image captured at the same time as the acquisition of the three-dimensional point cloud data P as the reference, the internal parameters M of the camera are manually adjusted to make the point cloud data P W match the reference image data. C ​ 4. The method for detecting and correcting wire changes in a power transmission channel according to claim 1, characterized in that In the S2, the matrix of the internal parameter M is: In Equation (2): f x represents the focal length of the camera in the horizontal direction; f y represents the focal length of the camera in the vertical direction; c x represents the position of the center point in the horizontal direction of the image coordinate system; c y represents the position of the center point in the vertical direction of the image coordinate system.

5. The method for detecting and correcting wire changes in a transmission channel according to claim 4, characterized in that In the S3, use the camera internal parameter M to convert the wire point cloud into a depth map depth, that is: In formula (3): x i , y i respectively represent the horizontal and vertical coordinates of the i-th wire point cloud in the point cloud data P C , n represents the number of point clouds; u i and v i respectively represent the pixel positions in the u direction and the v direction corresponding to the i-th wire point cloud.

6. The method for detecting and correcting wire changes in a transmission channel according to claim 5, characterized in that, In the S5, convert the optical flow matrix into the spatial change amount of the wire points in the three-dimensional space coordinate system, specifically: In formulas (4) and (5): Δu i and Δv i represent the pixel displacement amounts of the i-th wire pixel point in the u-direction and v-direction of the reference image and the current image respectively, and Δuv i represents the comprehensive displacement amount of the corresponding i-th wire pixel point in the reference image and the current image. Focal is the focal length of the camera, and Dis p represents the horizontal distance from the i-th wire point cloud to the camera in the camera coordinate system, and ΔP i represents the spatial displacement amount corresponding to the pixel displacement amount of the i-th wire pixel point, that is, the spatial variation amount.

7. The method for detecting and correcting wire changes in a power transmission channel according to claim 6, characterized in that, In the step S5, add the spatial variation amount to the corresponding wire point cloud in the point cloud data P C to obtain the corrected wire point cloud set as follows: P cor = [x i , y i , z i + ΔP i , i = 1, 2, ..., n; (6); In formula (6): P cor represents the corrected wire point cloud set, and the point cloud data P C = [x i , y i , z i , where x i represents the position of the i-th wire point cloud in the horizontal direction in the point cloud data, y i represents the position of the i-th wire point cloud in the vertical direction in the point cloud data, z i represents the position of the i-th wire point cloud in the direction perpendicular to the horizontal plane, and n represents the number of point clouds.

8. An apparatus for implementing a method for detecting and correcting wire changes in a power transmission channel, characterized in that, The device includes: The first acquisition module is used to acquire the three-dimensional point cloud data P of the power transmission channel in the world coordinate system W , and perform translation and rotation on the three-dimensional point cloud data P W to convert it into the point cloud data P in the camera coordinate system C ; A second acquisition module, configured to acquire reference image data of a power transmission channel, and register the reference image data with point cloud data P C to obtain camera registration parameters, where the camera registration parameters include an internal parameter M and external parameters R and T, where R represents a rotation matrix and T represents a translation matrix; The classification and extraction module is used to classify the point cloud data P by using the PointNet algorithm C and extract the wire point cloud in the point cloud data P C and convert the wire point cloud into a depth map depth by using the camera registration parameters; A third acquisition module, configured to obtain the current image data of the transmission line corridor, and use the dense optical flow estimation algorithm to capture the wire changes in the reference image data and the current image data, and generate an optical flow matrix for describing the movement of wire pixels; A correction module, which is used to combine the depth map depth to convert the optical flow matrix into the spatial variation of the wire points in the three-dimensional space coordinate system, and add the spatial variation to the corresponding wire point cloud in the point cloud data P C to correct the wire point cloud.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory, the memory stores instructions, and when the instructions are executed by the at least one processor, the at least one processor executes the method for detecting and correcting wire changes in the transmission line corridor according to any one of claims 1 to 7.

10. A machine-readable storage medium, characterized in that, An executable instruction is stored on the machine-readable storage medium, and when the instruction is executed, the machine executes the method for detecting and correcting wire changes in the transmission line corridor according to any one of claims 1 to 7.

Images