Power transmission line inspection method based on three-dimensional laser point cloud and image combination

Abstract

The invention relates to a three-dimensional laser point cloud and image combination-based power transmission line inspection method. The method comprises the following steps of collecting three-dimensional point cloud data and image data of a power transmission line; constructing an electric power facility component identification model to identify position data of each electric power facility component in the image data; constructing a point cloud image fusion model to project the three-dimensional point cloud data into the image data, and obtaining the position data of the three-dimensional point cloud of the electric power facility component in a three-dimensional space through the position data of each electric power facility component in the image data; constructing a three-dimensional model of the electric power facility component based on the position data of the three-dimensional point cloud of the electric power facility component in the three-dimensional space, and storing the three-dimensional model in a database; the corresponding three-dimensional model is extracted from the database according to the electric power facility component needing to be inspected, a three-dimensional model aggregate target is constructed, a power transmission line inspection route is planned according to the three-dimensional model aggregate target, and the unmanned aerial vehicle executes a flight task according to the planned power transmission line inspection route.

Description

Technical Field

[0001] The invention relates to a power transmission line inspection method based on the combination of three-dimensional laser point cloud and image, and belongs to the technical field of unmanned aerial vehicle path planning. Background Art

[0002] Most current autonomous inspection solutions based on drones still rely on manual teaching to plan flight paths: first, an experienced pilot controls the drone to various key points and uses the onboard equipment to complete the inspection of specific facilities; then the specific trajectory of the drone and the sequence of actions performed during the entire process are recorded, and a set of fixed inspection routes are generated based on this; then, the established route is regularly repeated with the help of an automated control system to collect data information. Although this method can meet actual needs to a certain extent, its high reliance on manual operation not only limits work efficiency, but also increases potential safety risks.

[0003] The invention patent with publication number CN117589167A discloses a method for planning the route of drone inspection based on a three-dimensional point cloud model. The solution achieves a nearly fully automated process through a series of sophisticated technical means. First, the original point cloud data is filtered and clustered to remove noise and extract key features. Subsequently, the processed point cloud is classified using the random forest algorithm to distinguish different terrains and objects. Finally, the classification results are combined with terrain information, and the A* algorithm is used to automatically generate the optimal flight path. This process significantly reduces the need for human intervention and improves work efficiency.

[0004] However, for some smaller and inconspicuous parts on power facilities, such as insulators or grounding devices, due to their tiny size and usually existing as part of a larger structure, it is quite challenging to extract these detailed features directly from the point cloud through random forests. Therefore, this method is more suitable for identifying large targets such as transmission towers. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention proposes a transmission line inspection method based on the combination of three-dimensional laser point cloud and image.

[0006] The technical solution of the present invention is as follows:

[0007] On the one hand, the present invention provides a transmission line inspection method based on a combination of three-dimensional laser point cloud and image, comprising the following steps:

[0008] Plan the initial route of the transmission line. The drone will perform the flight mission along the initial route and collect 3D point cloud data and image data of the transmission line.

[0009] Constructing a power facility component recognition model, and identifying the position data of each power facility component in the image data through the power facility component recognition model;

[0010] Preprocess the 3D point cloud data, build a point cloud image fusion model to project the 3D point cloud data into the image data, and obtain the position data of the 3D point cloud of the power facility components in the 3D space through the position data of each power facility component in the image data;

[0011] Constructing a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in three-dimensional space and storing it in a database;

[0012] The corresponding three-dimensional models of the power facility components that need to be inspected are extracted from the database and constructed as a three-dimensional model aggregate target. The transmission line inspection route is planned according to the three-dimensional model aggregate target, and the UAV performs the flight mission according to the planned transmission line inspection route.

[0013] As a preferred embodiment of the present invention, the power facility component recognition model is constructed based on a machine learning model, and the location data of each power facility component is obtained by training the image data.

[0014] As a preferred embodiment of the present invention, the point cloud image fusion model is constructed based on the collinear equation model, and the three-dimensional point cloud data is projected onto the image data to obtain the three-dimensional spatial coordinates corresponding to each pixel point in the image data, as shown in the following formula:

[0015]

[0016] Where: (x ′ ,y ′ ) represents the two-dimensional coordinates of the pixel; (X, Y, Z) represents the three-dimensional point cloud coordinates corresponding to the pixel; R -1 represents the inverse rotation matrix of the camera external parameters; D represents the depth value of the point cloud; f x 、f y Respectively represent the pixel values ​​of the camera focal length in the x and y directions; (X0, Y0, Z0) represents the three-dimensional space coordinates of the camera center; (c x ,c y ) represents the coordinates of the camera optical center.

[0017] As a preferred embodiment of the present invention, a three-dimensional model of the power facility component is constructed by multi-view geometric reconstruction technology based on the position data of the three-dimensional point cloud of the power facility component in three-dimensional space.

[0018] As a preferred embodiment of the present invention, a transmission line inspection route is planned by constructing a multi-factor flight path planning objective function, and the multi-factor flight path planning objective function is specifically shown in the following formula:

[0019]

[0020] Where: P represents the drone path, which is represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between node P[i] and node P[j+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery life risk weight; R b Indicates the battery life risk factor of the corresponding node.

[0021] As a preferred embodiment of the present invention, the constraint conditions for constructing the optimized path for the multi-factor flight path planning objective function specifically include:

[0022] The drone path P must pass through all nodes T = {T1, T2, ..., T m}Build constraints:

[0023]

[0024] Where: T m Indicates the node where the mth 3D model aggregate target is located;

[0025] Construct the UAV flight altitude constraint:

[0026]

[0027] Where: h(P[j]) represents the height of node P[j]; h min The minimum permitted flight altitude of a drone; h max Indicates the maximum allowed flight altitude of the drone;

[0028] Constructing battery life constraints:

[0029]

[0030] Where: d max Indicates the maximum range of the drone.

[0031] On the other hand, the present invention also provides a power transmission line inspection system based on the combination of three-dimensional laser point cloud and image, including a data acquisition module, a power facility component marking module, a point cloud image fusion module, a power facility component three-dimensional model building module and a path planning module;

[0032] The data acquisition module is used to plan the initial route of the power transmission line, perform the flight mission through the initial route and collect the three-dimensional point cloud data and image data of the power transmission line;

[0033] The electric facility component marking module is used to construct an electric facility component recognition model, and recognize the position data of each electric facility component in the image data through the electric facility component recognition model;

[0034] The point cloud image fusion module is used to pre-process the three-dimensional point cloud data, build a point cloud image fusion model to project the three-dimensional point cloud data into the image data, and obtain the position data of the three-dimensional point cloud of the power facility components in the three-dimensional space through the position data of each power facility component in the image data;

[0035] The power facility component three-dimensional model building module is used to build a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in three-dimensional space and store it in a database;

[0036] The path planning module is used to extract corresponding three-dimensional models from the database according to the power facility components that need to be inspected and construct them into a three-dimensional model assembly target, plan a transmission line inspection route according to the three-dimensional model assembly target, and execute the flight mission according to the planned transmission line inspection route.

[0037] As a preferred embodiment of the present invention, the power facility component recognition model is constructed based on a machine learning model, and the location data of each power facility component is obtained by training the image data.

[0038] As a preferred embodiment of the present invention, the point cloud image fusion model is constructed based on the collinear equation model, and the three-dimensional point cloud data is projected onto the image data to obtain the three-dimensional spatial coordinates corresponding to each pixel point in the image data, as shown in the following formula:

[0039]

[0040] Where: (x ′ ,y ′ ) represents the two-dimensional coordinates of the pixel; (X, Y, Z) represents the three-dimensional point cloud coordinates corresponding to the pixel; R -1 represents the inverse rotation matrix of the camera external parameters; D represents the depth value of the point cloud; f x 、f y Respectively represent the pixel values ​​of the camera focal length in the x and y directions; (X0, Y0, Z0) represents the three-dimensional space coordinates of the camera center; (c x ,c y ) represents the coordinates of the camera optical center.

[0041] As a preferred embodiment of the present invention, a three-dimensional model of the power facility component is constructed by multi-view geometric reconstruction technology based on the position data of the three-dimensional point cloud of the power facility component in three-dimensional space.

[0042] As a preferred embodiment of the present invention, a transmission line inspection route is planned by constructing a multi-factor flight path planning objective function, and the multi-factor flight path planning objective function is specifically shown in the following formula:

[0043]

[0044] Where: P represents the drone path, which is represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between node P[i] and node P[j+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery life risk weight; R b Indicates the battery life risk factor of the corresponding node.

[0045] As a preferred embodiment of the present invention, the constraint conditions for constructing the optimized path for the multi-factor flight path planning objective function specifically include:

[0046] The drone path P must pass through all nodes T = {T1, T2, ..., T m}Build constraints:

[0047]

[0048] Where: T m Indicates the node where the mth 3D model aggregate target is located;

[0049] Construct the UAV flight altitude constraint:

[0050]

[0051] Where: h(P[j]) represents the height of node P[j]; h min The minimum permitted flight altitude of a drone; h max Indicates the maximum allowed flight altitude of the drone;

[0052] Constructing battery life constraints:

[0053]

[0054] Where: d max Indicates the maximum range of the drone.

[0055] On the other hand, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present invention when executing the program.

[0056] In yet another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present invention.

[0057] The present invention has the following beneficial effects:

[0058] 1. The present invention can automatically identify key elements in the target area, automatically generate the optimal flight path, and flexibly adjust the strategy according to actual conditions, thereby achieving effective monitoring and rapid response to the status of power facilities in complex and changeable environments.

[0059] 2. The present invention is suitable for fully autonomous inspection operations of drones in machine nests on transmission line towers. It can automatically plan the optimal route according to the harsh field operation environment and improve the energy utilization efficiency of the machine nest. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0063] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0064] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0065] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0066] Embodiment 1:

[0067] See also Figure 1 A transmission line inspection method based on three-dimensional laser point cloud and image combination includes the following steps:

[0068] Plan the initial route of the transmission line (pre-design an initial route suitable for drone flight based on the specific direction of the line and the surrounding environmental characteristics (such as mountain distribution, building location, etc.), and the multi-rotor drone equipped with a high-resolution camera and lidar sensor will fly along the initial route to perform flight missions and collect 3D point cloud data and image data of the transmission line;

[0069] Constructing a power facility component recognition model, and identifying the position data of each power facility component in the image data through the power facility component recognition model;

[0070] Preprocess the 3D point cloud data (denoising, filtering and deletion, etc.), build a point cloud image fusion model to project the 3D point cloud data into the image data, and obtain the position data of the 3D point cloud of the power facility components in the 3D space through the position data of each power facility component in the image data;

[0071] Based on the position data of the 3D point cloud of the power facility components in 3D space, a 3D model of the power facility components is constructed and stored in the database (the database not only serves as an important reference for inspection route planning, but also provides valuable historical data support for future maintenance work. In addition, the database design is highly flexible and can expand fields according to actual needs to facilitate the addition of more types of equipment or update of information on existing equipment in the future);

[0072] According to the power facility components that need to be inspected, the corresponding three-dimensional models are extracted from the database and constructed as a three-dimensional model aggregate target. The transmission line inspection route is planned according to the three-dimensional model aggregate target. The UAV performs the flight mission according to the planned transmission line inspection route and pauses for a moment after arriving at each predetermined location to observe the target status in more detail. At the same time, a new round of image data transmitted back in real time will be immediately sent to the background analysis system for processing, and the corresponding entries in the existing database will be updated. Over time, as more and more data are accumulated, the system will gradually learn to identify more types of abnormal phenomena, and make more accurate and effective early warning suggestions based on this.

[0073] As a preferred implementation mode of this embodiment, the power facility component recognition model is constructed based on a machine learning model (for example, traditional model training such as U-Net, or pre-trained large models such as SAM), and the location data of each power facility component is obtained by training the image data.

[0074] As a preferred implementation of this embodiment, a statistical filter is applied to remove noise points in the three-dimensional point cloud data, and methods such as VoxelGrid are used to further simplify the point cloud structure and reduce redundant information. A ground segmentation algorithm is executed to separate non-ground objects from the original point cloud.

[0075] As a preferred implementation of this embodiment, the point cloud image fusion model is constructed based on the collinear equation model, and the three-dimensional point cloud data is projected onto the image data to obtain the three-dimensional spatial coordinates corresponding to each pixel point in the image data, as shown in the following formula:

[0076]

[0077] Where: (x ′ ,y ′ ) represents the two-dimensional coordinates of the pixel; (X, Y, Z) represents the three-dimensional point cloud coordinates corresponding to the pixel; R -1 represents the inverse rotation matrix of the camera external parameters; D represents the depth value of the point cloud; f x 、f y Respectively represent the pixel values ​​of the camera focal length in the x and y directions; (X0, Y0, Z0) represents the three-dimensional space coordinates of the camera center; (c x ,c y ) represents the coordinates of the camera optical center.

[0078] As a preferred implementation of this embodiment, a three-dimensional model of the power facility component is constructed based on the position data of the three-dimensional point cloud of the power facility component in three-dimensional space through multi-view geometric reconstruction technology (such as SfM, MVS, etc.).

[0079] As a preferred implementation of this embodiment, a transmission line inspection route is planned by constructing a multi-factor flight path planning objective function, and the multi-factor flight path planning objective function is specifically shown in the following formula:

[0080]

[0081] Where: P represents the drone path, which is represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between node P[i] and node P[i+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery life risk weight; R b Indicates the battery life risk factor of the corresponding node.

[0082] The calculation formula of the wind speed risk factor is:

[0083]

[0084] Where: v wind (P[i]) represents the real-time wind speed at the node; v safe Indicates the upper limit of the safe wind speed of the drone; v max Indicates the maximum wind speed allowed for the drone;

[0085] The calculation formula of the terrain risk factor is:

[0086]

[0087] Where: g(P[i]) represents the altitude of the node; α represents the terrain sensitivity parameter. In this embodiment, α takes a value of 0.1 to 0.5 for flat terrain, 0.5 to 1.5 for mountainous or complex terrain, and greater than 1.5 for extremely complex terrain; g avg Indicates the average altitude;

[0088] The calculation formula of the battery life risk factor is:

[0089]

[0090] Where: P r Represents the drone's return node; β represents the endurance margin tolerance coefficient, which is set according to the task priority. A smaller margin is tolerated for high-priority tasks, and a higher margin is set for low-priority tasks.

[0091] As a preferred implementation of this embodiment, the constraints for constructing the optimized path for the multi-factor flight path planning objective function specifically include:

[0092] The drone path P must pass through all nodes T = {T1, T2, ..., T m}Build constraints:

[0093]

[0094] Where: T m Indicates the node where the mth 3D model aggregate target is located;

[0095] Construct the UAV flight altitude constraint:

[0096]

[0097] Where: h(P[j]) represents the height of node P[j]; h min The minimum permitted flight altitude of a drone; h max Indicates the maximum allowed flight altitude of the drone;

[0098] Constructing battery life constraints:

[0099]

[0100] Where: d max Indicates the maximum range of the drone.

[0101] As a preferred implementation of this embodiment, a drone nest is set up, and the image and point cloud data collected by the drone are preliminarily processed through the edge computing module embedded in the nest, thereby reducing the burden on the central server;

[0102] Assume that there are m processing devices in the machine nest, and the processing capacity of each device is C k , then the task allocation is optimized as follows:

[0103]

[0104] Where: T k Indicates the total time for the kth device to complete the task; N total Indicates the total data task volume; x k represents the amount of tasks assigned to the kth device;

[0105] After completing the flight mission, each drone uploads the data to the machine nest database, which is synchronized to the central server via the wireless network;

[0106] In order to achieve UAV coverage of the mission area and reduce energy consumption, the location of the drone nest needs to meet the following requirements:

[0107]

[0108] Among them: Cast L represents the cost of building a nest at position L; L represents the coordinates of the nest position; P i represents the set of task points; w setup represents the weight of the machine nest construction cost; c setup (L) represents the cost of deploying the machine nest (taking into account terrain, environment, etc.);

[0109] The location with the lowest cost calculated by the above formula is used as the construction point of the machine nest.

[0110] Embodiment 2:

[0111] A power transmission line inspection system based on the combination of three-dimensional laser point cloud and image, including a data acquisition module, a power facility component marking module, a point cloud image fusion module, a power facility component three-dimensional model building module and a path planning module;

[0112] The data acquisition module is used to plan the initial route of the power transmission line. The UAV performs the flight mission through the initial route and collects the three-dimensional point cloud data and image data of the power transmission line.

[0113] The electric facility component marking module is used to construct an electric facility component recognition model, and recognize the position data of each electric facility component in the image data through the electric facility component recognition model;

[0114] The point cloud image fusion module is used to pre-process the three-dimensional point cloud data, build a point cloud image fusion model to project the three-dimensional point cloud data into the image data, and obtain the position data of the three-dimensional point cloud of the power facility components in the three-dimensional space through the position data of each power facility component in the image data;

[0115] The power facility component three-dimensional model building module is used to build a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in three-dimensional space and store it in a database;

[0116] The path planning module is used to extract the corresponding three-dimensional model from the database according to the power facility components that need to be inspected and construct it into a three-dimensional model assembly target, plan the transmission line inspection route according to the three-dimensional model assembly target, and the UAV performs the flight mission according to the planned transmission line inspection route.

[0117] As a preferred implementation of this embodiment, the power facility component recognition model is constructed based on a machine learning model, and the location data of each power facility component is obtained by training the image data.

[0118] As a preferred implementation of this embodiment, the point cloud image fusion model is constructed based on the collinear equation model, and the three-dimensional point cloud data is projected onto the image data to obtain the three-dimensional spatial coordinates corresponding to each pixel point in the image data, as shown in the following formula:

[0119]

[0120] Where: (x ′ ,y ′ ) represents the two-dimensional coordinates of the pixel; (X, Y, Z) represents the three-dimensional point cloud coordinates corresponding to the pixel; R -1 represents the inverse rotation matrix of the camera external parameters; D represents the depth value of the point cloud; f x 、f y Respectively represent the pixel values ​​of the camera focal length in the x and y directions; (X0, Y0, Z0) represents the three-dimensional space coordinates of the camera center; (c x ,c y ) represents the coordinates of the camera optical center.

[0121] As a preferred implementation of this embodiment, a three-dimensional model of the power facility component is constructed by multi-view geometric reconstruction technology based on the position data of the three-dimensional point cloud of the power facility component in three-dimensional space.

[0122] As a preferred implementation of this embodiment, a transmission line inspection route is planned by constructing a multi-factor flight path planning objective function, and the multi-factor flight path planning objective function is specifically shown in the following formula:

[0123]

[0124] Where: P represents the drone path, which is represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between node P[i] and node P[i+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery life risk weight; R b Indicates the battery life risk factor of the corresponding node.

[0125] As a preferred implementation of this embodiment, the constraints for constructing the optimized path for the multi-factor flight path planning objective function specifically include:

[0126] The drone path P must pass through all nodes T = {T1, T2, ..., T m}Build constraints:

[0127]

[0128] Where: T m Indicates the node where the mth 3D model aggregate target is located;

[0129] Construct the UAV flight altitude constraint:

[0130]

[0131] Where: h(P[j]) represents the height of node P[j]; h min The minimum permitted flight altitude of a drone; h max Indicates the maximum allowed flight altitude of the drone;

[0132] Constructing battery life constraints:

[0133]

[0134] Where: d max Indicates the maximum range of the drone.

[0135] The system is used to implement the method in Example 1, which will not be described in detail here.

[0136] Embodiment three:

[0137] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present invention when executing the program.

[0138] Embodiment 4:

[0139] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any embodiment of the present invention is implemented.

[0140] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0141] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0143] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0144] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A transmission line inspection method based on three-dimensional laser point cloud and image combination, characterized in that: The following steps are involved: Plan the initial route of the transmission line. The drone will perform the flight mission along the initial route and collect 3D point cloud data and image data of the transmission line. Constructing a power facility component recognition model, and identifying the position data of each power facility component in the image data through the power facility component recognition model; Preprocess the 3D point cloud data, build a point cloud image fusion model to project the 3D point cloud data into the image data, and obtain the position data of the 3D point cloud of the power facility components in the 3D space through the position data of each power facility component in the image data; Constructing a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in three-dimensional space and storing it in a database; The corresponding three-dimensional models of the power facility components that need to be inspected are extracted from the database and constructed as a three-dimensional model aggregate target. The transmission line inspection route is planned according to the three-dimensional model aggregate target, and the UAV performs the flight mission according to the planned transmission line inspection route.

2. A power transmission line inspection method based on three-dimensional laser point cloud and image combination according to claim 1, characterized in that: The electric power facility component recognition model is constructed based on a machine learning model, and the location data of each electric power facility component is obtained by training the image data.

3. The power transmission line inspection method based on the combination of three-dimensional laser point cloud and image according to claim 2 is characterized in that: The point cloud image fusion model is constructed based on the collinear equation model, and the three-dimensional point cloud data is projected onto the image data to obtain the three-dimensional spatial coordinates corresponding to each pixel in the image data, as shown in the following formula: Where: (x ′ ,y ′ ) represents the two-dimensional coordinates of the pixel; (X, Y, Z) represents the three-dimensional point cloud coordinates corresponding to the pixel; R -1 represents the inverse rotation matrix of the camera external parameters; D represents the depth value of the point cloud; f x 、f y Respectively represent the pixel values ​​of the camera focal length in the x and y directions; (X0, Y0, Z0) represents the three-dimensional space coordinates of the camera center; (c x ,c y ) represents the coordinates of the camera optical center.

4. The power transmission line inspection method based on the combination of three-dimensional laser point cloud and image according to claim 1 is characterized in that: Based on the position data of the three-dimensional point cloud of the power facility components in three-dimensional space, a three-dimensional model of the power facility components is constructed through multi-view geometric reconstruction technology.

5. The power transmission line inspection method based on the combination of three-dimensional laser point cloud and image according to claim 1 is characterized in that: The transmission line inspection route is planned by constructing a multi-factor flight path planning objective function, and the multi-factor flight path planning objective function is specifically shown in the following formula: Where: P represents the drone path, which is represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between node P[i] and node P[i+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery life risk weight; R b Indicates the battery life risk factor of the corresponding node.

6. The power transmission line inspection method based on the combination of three-dimensional laser point cloud and image according to claim 5 is characterized in that: The constraints for optimizing the path are constructed for the multi-factor flight path planning objective function, including: The drone path P must pass through all nodes T = {T1, T2, ..., T m }Build constraints: Where: T m Indicates the node where the mth 3D model aggregate target is located; Construct the UAV flight altitude constraint: Where: h(P[j]) represents the height of node P[j]; h min The minimum permitted flight altitude of a drone; h max Indicates the maximum allowed flight altitude of the drone; Constructing battery life constraints: Where: d max Indicates the maximum range of the drone.

7. A power transmission line inspection system based on three-dimensional laser point cloud and image combination, characterized in that: It includes a data acquisition module, a power facility component marking module, a point cloud image fusion module, a power facility component 3D model building module, and a path planning module; The data acquisition module is used to plan the initial route of the power transmission line, perform the flight mission through the initial route and collect the three-dimensional point cloud data and image data of the power transmission line; The electric facility component marking module is used to construct an electric facility component recognition model, and recognize the position data of each electric facility component in the image data through the electric facility component recognition model; The point cloud image fusion module is used to pre-process the three-dimensional point cloud data, build a point cloud image fusion model to project the three-dimensional point cloud data into the image data, and obtain the position data of the three-dimensional point cloud of the power facility components in the three-dimensional space through the position data of each power facility component in the image data; The power facility component three-dimensional model building module is used to build a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in three-dimensional space and store it in a database; The path planning module is used to extract corresponding three-dimensional models from the database according to the power facility components that need to be inspected and construct them into a three-dimensional model assembly target, plan a transmission line inspection route according to the three-dimensional model assembly target, and execute the flight mission according to the planned transmission line inspection route.

8. The power transmission line inspection system based on the combination of three-dimensional laser point cloud and image according to claim 7 is characterized in that: The electric power facility component recognition model is constructed based on a machine learning model, and the location data of each electric power facility component is obtained by training the image data.

9. The power transmission line inspection system based on the combination of three-dimensional laser point cloud and image according to claim 8, characterized in that: The point cloud image fusion model is constructed based on the collinear equation model, and the three-dimensional point cloud data is projected onto the image data to obtain the three-dimensional spatial coordinates corresponding to each pixel in the image data, as shown in the following formula: Where: (x ′ ,y ′ ) represents the two-dimensional coordinates of the pixel; (X, Y, Z) represents the three-dimensional point cloud coordinates corresponding to the pixel; R -1 represents the inverse rotation matrix of the camera external parameters; D represents the depth value of the point cloud; f x 、f y Respectively represent the pixel values ​​of the camera focal length in the x and y directions; (X0, Y0, Z0) represents the three-dimensional space coordinates of the camera center; (c x ,c y ) represents the coordinates of the camera optical center.

10. The power transmission line inspection system based on the combination of three-dimensional laser point cloud and image according to claim 7, characterized in that: Based on the position data of the three-dimensional point cloud of the power facility components in three-dimensional space, a three-dimensional model of the power facility components is constructed through multi-view geometric reconstruction technology.

11. The power transmission line inspection system based on the combination of three-dimensional laser point cloud and image according to claim 7, characterized in that: The transmission line inspection route is planned by constructing a multi-factor flight path planning objective function, and the multi-factor flight path planning objective function is specifically shown in the following formula: Where: P represents the drone path, which is represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between node P[i] and node P[i+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery life risk weight; R b Indicates the battery life risk factor of the corresponding node.

12. The power transmission line inspection system based on the combination of three-dimensional laser point cloud and image according to claim 11, characterized in that: The constraints for optimizing the path are constructed for the multi-factor flight path planning objective function, including: The drone path P must pass through all nodes T = {T1, T2, ..., T m }Build constraints: Where: T m Indicates the node where the mth 3D model aggregate target is located; Construct the UAV flight altitude constraint: Where: h(P[j]) represents the height of node P[j]; h min The minimum permitted flight altitude of a drone; h max Indicates the maximum allowed flight altitude of the drone; Constructing battery life constraints: Where: d max Indicates the maximum range of the drone.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images