A power transmission line inspection method based on three-dimensional laser point cloud and image combination

Abstract

The application relates to a power transmission line inspection method based on three-dimensional laser point cloud and image combination, which comprises the following steps: collecting three-dimensional point cloud data and image data of a power transmission line; constructing a power facility component identification model to identify the position data of each power facility component in the image data; constructing a point cloud image fusion model to project the three-dimensional point cloud data into the image data, and obtaining the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space through the position data of each power facility component in the image data; constructing a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space and storing the three-dimensional model in a database; extracting the corresponding three-dimensional model from the database according to the power facility component to be inspected and constructing the three-dimensional model into a three-dimensional model assembly target; planning a power transmission line inspection route according to the three-dimensional model assembly target; and executing a flight task according to the planned power transmission line inspection route.

Description

TECHNICAL FIELD

[0001] The present application relates to a power transmission line inspection method based on three-dimensional laser point cloud and image joint, belonging to the technical field of unmanned aerial vehicle path planning. BACKGROUND

[0002] Most current autonomous inspection solutions based on unmanned aerial vehicles still rely on manual demonstration to plan flight paths: that is, first, an experienced pilot controls the unmanned aerial vehicle to each key point, uses the equipment to complete the inspection of the specific facility; then record the specific trajectory of the unmanned aerial vehicle and the action sequence it performs during the entire process, and generate a fixed inspection route based on this; then use an automated control system to periodically repeat the execution of this established route to collect data information. Although this method can meet the actual needs to some extent, its high dependence on human operation not only limits work efficiency, but also increases potential safety risks.

[0003] The invention patent with publication number CN117589167A discloses a method for unmanned aerial vehicle inspection route planning based on three-dimensional point cloud model. This solution realizes a nearly fully automated process through a series of ingenious technical means. First, the original point cloud data is filtered and clustered to remove noise and extract key features. Then, the processed point cloud is classified using a random forest algorithm to distinguish different terrains and objects. Finally, combining the classification results with terrain information, an optimal flight path is automatically generated using the A* algorithm. This process significantly reduces the need for human intervention and improves work efficiency.

[0004] However, for some smaller and inconspicuous components on power facilities, such as insulators or grounding devices, it is challenging to extract these detailed features directly from point clouds using random forests due to their small size and often being part of larger structures. Therefore, this method is more suitable for identifying large targets like power transmission towers. SUMMARY

[0005] To solve the problems existing in the prior art, the present application provides a power transmission line inspection method based on three-dimensional laser point cloud and image joint.

[0006] The technical solution of the present application is as follows:

[0007] On the one hand, the present application provides a power transmission line inspection method based on three-dimensional laser point cloud and image joint, comprising the following steps:

[0008] Planning an initial route for the power transmission line, and the unmanned aerial vehicle executes the flight task and collects three-dimensional point cloud data and image data of the power transmission line through the initial route;

[0009] A power facility component recognition model is constructed, and position data of each power facility component in the image data is recognized through the power facility component recognition model.

[0010] The three-dimensional point cloud data is preprocessed, a point cloud image fusion model is constructed, the three-dimensional point cloud data is projected into the image data, and position data of the three-dimensional point cloud of the power facility component in the three-dimensional space is obtained through the position data of each power facility component in the image data.

[0011] A three-dimensional model of the power facility component is constructed based on the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space and stored in a database.

[0012] According to the power facility component to be inspected, the corresponding three-dimensional model is extracted from the database and constructed into a three-dimensional model assembly target, a transmission line inspection route is planned according to the three-dimensional model assembly target, and the unmanned aerial vehicle performs a flight task according to the planned transmission line inspection route.

[0013] As a preferred embodiment of the present application, the power facility component recognition model is constructed based on a machine learning model, and the position data of each power facility component is obtained by training the image data.

[0014] As a preferred embodiment of the present application, the point cloud image fusion model is constructed based on a collinear equation model, and the three-dimensional space coordinates corresponding to each pixel point in the image data are obtained by projecting the three-dimensional point cloud data into the image data, and are specifically as follows:

[0015]

[0016] wherein (x ′ ,y ′ ) represents the two-dimensional coordinates of the pixel point; (X,Y,Z) represents the three-dimensional point cloud point coordinates corresponding to the pixel point; R -1 represents the inverse rotation matrix of the camera external parameter; D represents the depth value of the point cloud; f x , f y respectively represent the pixel values of the camera focal length in the x and y directions; (X0,Y0,Z0) represents the three-dimensional space coordinates of the camera center; (c x ,c y ) represents the camera optical center coordinates.

[0017] As a preferred embodiment of the present application, the three-dimensional model of the power facility component is constructed based on the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space through a multi-view geometric reconstruction technology.

[0018] As a preferred embodiment of the present application, the transmission line inspection route is planned by constructing a multi-factor flight path planning objective function, and the multi-factor flight path planning objective function is specifically as follows:

[0019]

[0020] wherein: P represents the UAV path, represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between the node P[i] and the node P[i+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents the terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery endurance risk weight; R b represents the battery endurance risk factor of the corresponding node.

[0021] As a preferred embodiment of the present application, the constraint conditions for constructing the optimized path of the multi-factor flight path planning objective function are specifically as follows:

[0022] The UAV path P must pass through all nodes T = {T1, T2, …, T m} of the three-dimensional model assembly target, and the constraint condition is constructed as follows:

[0023]

[0024] wherein: T m represents the node of the mth three-dimensional model assembly target;

[0025] The UAV flight height constraint condition is constructed as follows:

[0026]

[0027] wherein: h(P[j]) represents the height of the node P[j]; h min represents the minimum allowed flight height of the UAV; h max represents the maximum allowed flight height of the UAV;

[0028] The battery endurance constraint condition is constructed as follows:

[0029]

[0030] wherein: d max represents the maximum range of the UAV.

[0031] In another aspect, the present application also provides a power transmission line inspection system based on three-dimensional laser point cloud and image combination, comprising a data acquisition module, a power facility component marking module, a point cloud image fusion module, a power facility component three-dimensional model construction module, and a path planning module.

[0032] The data acquisition module is used for planning an initial flight route of the power transmission line, performing a flight task through the initial flight route, and collecting three-dimensional point cloud data and image data of the power transmission line;

[0033] The power facility component marking module is used for constructing a power facility component identification model, and identifying position data of each power facility component in the image data through the power facility component identification model;

[0034] The point cloud image fusion module is used for preprocessing the three-dimensional point cloud data, constructing a point cloud image fusion model, projecting the three-dimensional point cloud data into the image data, and obtaining position data of the three-dimensional point cloud of the power facility component in the three-dimensional space through the position data of each power facility component in the image data;

[0035] The power facility component three-dimensional model construction module is used for constructing a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space, and storing the three-dimensional model in a database;

[0036] The path planning module is used for extracting a corresponding three-dimensional model from the database according to a power facility component to be inspected, constructing a three-dimensional model aggregate target, planning a power transmission line inspection flight route according to the three-dimensional model aggregate target, and performing a flight task according to the planned power transmission line inspection flight route.

[0037] As a preferred embodiment of the present application, the power facility component identification model is constructed based on a machine learning model, and the position data of each power facility component is obtained by training the image data.

[0038] As a preferred embodiment of the present application, the point cloud image fusion model is constructed based on a collinear equation model, and three-dimensional space coordinates corresponding to each pixel point in the image data are obtained by projecting the three-dimensional point cloud data into the image data, and are specifically as follows:

[0039]

[0040] wherein (x ′ ,y ′ ) represents two-dimensional coordinates of a pixel point; (X,Y,Z) represents three-dimensional point cloud point coordinates corresponding to the pixel point; R -1 represents an inverse rotation matrix of a camera external parameter; D represents a depth value of the point cloud; f x , f y respectively represent pixel values of a camera focal length in x and y directions; (X0,Y0,Z0) represents three-dimensional space coordinates of a camera center; (c x ,c y ) represents camera optical center coordinates.

[0041] As a preferred embodiment of the present application, the position data of the power facility component three-dimensional point cloud in three-dimensional space is used to construct a three-dimensional model of the power facility component through multi-view geometry reconstruction technology.

[0042] As a preferred embodiment of the present application, the power transmission line patrol route is planned by constructing a multi-factor flight path planning objective function, which is specifically shown in the following formula:

[0043]

[0044] wherein: P represents the UAV path, represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between node P[i] and node P[j+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents the terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery endurance risk weight; R b represents the battery endurance risk factor of the corresponding node.

[0045] As a preferred embodiment of the present application, the multi-factor flight path planning objective function is constructed to have a constraint condition for the optimized path, which specifically includes:

[0046] The UAV path P must pass through all nodes T = {T1, T2, …, T m} of the three-dimensional model collection target to construct the constraint condition:

[0047]

[0048] wherein: T m represents the node of the mth three-dimensional model collection target;

[0049] The UAV flight height constraint condition is constructed:

[0050]

[0051] wherein: h(P[j]) represents the height of node P[j]; h min represents the minimum allowed flight height of the UAV; h max represents the maximum allowed flight height of the UAV;

[0052] The battery endurance constraint condition is constructed:

[0053]

[0054] wherein: d max represents the maximum flight range of the UAV.

[0055] In still another aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of the embodiments of the present application when executing the program.

[0056] In still another aspect, the present application provides a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the method according to any one of the embodiments of the present application.

[0057] The present application has the following beneficial effects:

[0058] 1. The present application can automatically identify key elements in the target area, automatically generate the optimal flight path, and flexibly adjust the strategy according to the actual situation, thereby realizing effective monitoring and rapid response to the status of power facilities in complex and variable environments.

[0059] 2. The present application is suitable for unmanned aerial vehicle autonomous inspection operation in the machine nest on the power transmission line tower, and can automatically plan the optimal route according to the harsh field operation site environment, thereby improving the energy utilization efficiency of the machine nest. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The present application is a method flowchart. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0062] It should be understood that the step numbers used herein are only for the convenience of description, and are not limited to the execution sequence of the steps.

[0063] It should be understood that the terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clearly indicated by the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0064] The terms "comprise" and "comprising" indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0065] The term "and / or" means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0066] Embodiment one:

[0067] Referring to Figure 1 A power transmission line inspection method based on three-dimensional laser point cloud and image combination, comprising the following steps:

[0068] Planning an initial flight route of the power transmission line (designing an initial flight route suitable for the flight of a UAV according to the specific route of the section of the line and the surrounding environmental features (such as the distribution of mountains, the location of buildings, etc.), a multi-rotor UAV equipped with a high-resolution camera and a laser radar sensor flies through the initial flight route to perform a flight task and collects three-dimensional point cloud data and image data of the power transmission line;

[0069] Constructing a power facility component recognition model to recognize the position data of each power facility component in the image data through the power facility component recognition model;

[0070] Pretreating the three-dimensional point cloud data (denoising, filtering, and deleting, etc.), constructing a point cloud image fusion model to project the three-dimensional point cloud data into the image data, and obtaining the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space through the position data of each power facility component in the image data;

[0071] Constructing a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space and storing it in a database (the database not only serves as an important reference for the planning of the inspection flight route, but also provides valuable historical data support for future maintenance work. In addition, the database is designed to be highly flexible, and the fields can be expanded according to actual needs, so as to facilitate the addition of more types of equipment or the update of the information of existing equipment in the future);

[0072] According to the need to inspect the power facility components from the database corresponding to the three-dimensional model and build a three-dimensional model of the target set, according to the three-dimensional model of the target set, the power transmission line inspection route is planned, and the unmanned aerial vehicle executes the flight task according to the planned power transmission line inspection route, and pauses for a while after reaching each predetermined position in order to observe the target state in more detail; At the same time, the new round of image data transmitted back in real time will be immediately sent to the background analysis system for processing, updating the corresponding entries in the existing database; With the passage of time, as more and more data accumulates, the system can gradually learn to identify more types of abnormal phenomena, and accordingly make more accurate and effective early warning recommendations.

[0073] As a preferred embodiment of the present embodiment, the power facility component recognition model is constructed based on a machine learning model (such as a traditional model training such as U-Net, or a pre-trained large model such as SAM), and the position data of each power facility component is obtained by training the image data.

[0074] As a preferred embodiment of the present embodiment, a statistical filter is applied to remove noise points from the three-dimensional point cloud data, and a method such as VoxelGrid is used to further simplify the point cloud structure and reduce redundant information, and a ground segmentation algorithm is executed to separate non-ground objects from the original point cloud.

[0075] As a preferred embodiment of the present embodiment, the point cloud image fusion model is constructed based on a collinear equation model, and the three-dimensional point cloud data is projected into the image data to obtain the three-dimensional space coordinates corresponding to each pixel point in the image data, which is specifically shown in the following formula:

[0076]

[0077] Where: (x ′ ,y ′ ) represents the two-dimensional coordinates of the pixel point; (X,Y,Z) represents the three-dimensional point cloud point coordinates corresponding to the pixel point; R -1 represents the inverse rotation matrix of the camera external parameter; D represents the depth value of the point cloud; f x , f y respectively represent the pixel values of the camera focal length in the x and y directions; (X0,Y0,Z0) represents the three-dimensional space coordinates of the camera center; (c x ,c y ) represents the camera optical center coordinates.

[0078] As a preferred embodiment of the present embodiment, the three-dimensional model of the power facility component is constructed based on the position data of the power facility component three-dimensional point cloud in the three-dimensional space through multi-view geometric reconstruction technology (such as SfM, MVS, etc.).

[0079] As a preferred embodiment of the present embodiment, the power transmission line inspection flight path is planned by constructing a multi-factor flight path planning objective function, which is specifically as shown in the following formula:

[0080]

[0081] wherein P represents the UAV path, represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between the node P[i] and the node P[i+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents the terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery endurance risk weight; R b represents the battery endurance risk factor of the corresponding node.

[0082] The calculation formula of the wind speed risk factor is:

[0083]

[0084] wherein v wind (P[i]) represents the real-time wind speed at the node; v safe represents the upper limit of the safe wind speed of the UAV; v max represents the maximum wind speed allowed by the UAV;

[0085] The calculation formula of the terrain risk factor is:

[0086]

[0087] wherein g(P[i]) represents the altitude at the node; a represents the terrain sensitivity parameter, in the present embodiment, a is valued at 0.1 to 0.5 for flat terrain, a is valued at 0.5 to 1.5 for mountainous or complex terrain, and a is valued at greater than 1.5 for extremely complex terrain; g avg represents the average altitude;

[0088] The calculation formula of the battery endurance risk factor is:

[0089]

[0090] wherein P r represents the UAV return node; b represents the endurance margin tolerance coefficient, which is set according to the task priority, a smaller margin is tolerated for a high-priority task, and a higher margin is set for a low-priority task.

[0091] As a preferred embodiment of the present embodiment, the constraint condition of constructing the optimization path for the multi-factor flight path planning objective function is specifically as follows:

[0092] The path P of the unmanned aerial vehicle must pass through all nodes T = {T1, T2, …, Tn} of the three-dimensional model assembly target; m The constraint condition is constructed as follows:

[0093]

[0094] Wherein: Tm represents the node of the mth three-dimensional model assembly target; m h(P[j]) represents the height of the node P[j];

[0095] The flight height constraint condition of the unmanned aerial vehicle is constructed as follows:

[0096]

[0097] Wherein: h(P[j]) represents the height of the node P[j]; h min represents the minimum allowed flight height of the unmanned aerial vehicle; h max represents the maximum allowed flight height of the unmanned aerial vehicle;

[0098] The battery endurance constraint condition is constructed as follows:

[0099]

[0100] Wherein: d max represents the maximum range of the unmanned aerial vehicle.

[0101] As a preferred embodiment of the present embodiment, the unmanned aerial vehicle nest is set, and the edge computing module embedded in the nest is used to preliminarily process the image and point cloud data collected by the unmanned aerial vehicle, thereby reducing the burden of the center server;

[0102] Suppose that there are m processing devices in the nest, and the processing capacity of each device is C k The task allocation optimization is as follows:

[0103]

[0104] Wherein: T k represents the total time for the kth device to complete the task; N total represents the total data task amount; x k represents the task amount allocated to the kth device;

[0105] After each unmanned aerial vehicle completes the flight task, the data is uploaded to the nest database, and the nest is synchronized to the center server through the wireless network;

[0106] To realize the unmanned aerial vehicle covering the task area and reduce the energy consumption, the position of the nest needs to meet:

[0107]

[0108] Wherein: Cast L represents the cost of building a nest at L position; L represents the nest position coordinates; P i represents a set of task points; w setup represents the nest construction cost weight; c setup (L) represents the nest deployment cost (considering terrain, environment, etc.);

[0109] The site with the minimum cost calculated by the above formula is taken as the nest construction point.

[0110] Embodiment two:

[0111] A power transmission line inspection system based on three-dimensional laser point cloud and image combination includes a data acquisition module, a power facility component marking module, a point cloud image fusion module, a power facility component three-dimensional model construction module, and a path planning module.

[0112] The data acquisition module is used to plan the initial route of the power transmission line. The unmanned aerial vehicle performs the flight task through the initial route and collects the three-dimensional point cloud data and image data of the power transmission line.

[0113] The power facility component marking module is used to construct a power facility component recognition model to identify the position data of each power facility component in the image data.

[0114] The point cloud image fusion module is used to preprocess the three-dimensional point cloud data, construct a point cloud image fusion model, project the three-dimensional point cloud data into the image data, and obtain the position data of the power facility component three-dimensional point cloud in the three-dimensional space through the position data of each power facility component in the image data.

[0115] The power facility component three-dimensional model construction module is used to construct the three-dimensional model of the power facility component based on the position data of the power facility component three-dimensional point cloud in the three-dimensional space and store it in the database.

[0116] The path planning module is used to extract the corresponding three-dimensional model from the database according to the power facility component that needs to be inspected and construct a three-dimensional model set target. The power transmission line inspection route is planned according to the three-dimensional model set target, and the unmanned aerial vehicle performs the flight task according to the planned power transmission line inspection route.

[0117] As a preferred embodiment of the present embodiment, the power facility component recognition model is constructed based on a machine learning model. The position data of each power facility component is obtained by training the image data.

[0118] As a preferred embodiment of the present embodiment, the point cloud image fusion model is constructed based on a collineation equation model, and three-dimensional point cloud data is projected into image data to obtain three-dimensional space coordinates corresponding to each pixel point in the image data, which is specifically shown in the following formula:

[0119]

[0120] wherein (x ′ ,y ′ ) represents the two-dimensional coordinates of the pixel point; (X,Y,Z) represents the three-dimensional point cloud point coordinates corresponding to the pixel point; R -1 represents the inverse rotation matrix of the camera external parameter; D represents the depth value of the point cloud; f x , f y respectively represent the pixel values of the camera focal length in the x and y directions; (X0,Y0,Z0) represents the three-dimensional space coordinates of the camera center; (c x ,c y ) represents the camera optical center coordinates.

[0121] As a preferred embodiment of the present embodiment, the three-dimensional model of the power facility component is constructed based on the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space through multi-view geometric reconstruction technology.

[0122] As a preferred embodiment of the present embodiment, the transmission line patrol route is planned by constructing a multi-factor flight path planning objective function, which is specifically shown in the following formula:

[0123]

[0124] wherein P represents the path of the unmanned aerial vehicle, represented by the node sequence {P[1], P[2], …, P[n]}; n represents the total number of nodes; d(P[i], P[i+1]) represents the distance between the node P[i] and the node P[i+1]; w d represents the distance cost weight; w s represents the wind speed risk weight; R w represents the wind speed risk factor of the corresponding node; w t represents the terrain risk weight; R t represents the terrain risk factor of the corresponding node; w b represents the battery endurance risk weight; R b represents the battery endurance risk factor of the corresponding node.

[0125] As a preferred embodiment of the present embodiment, the multi-factor flight path planning objective function is constructed to have a constraint condition for the optimized path, which specifically includes:

[0126] The UAV path P must pass through all nodes T = {T1, T2, …, Tn} where the three-dimensional model aggregate target is located. m The construction constraint condition is:

[0127]

[0128] Wherein: T m represents the node where the mth three-dimensional model aggregate target is located.

[0129] The construction of the UAV flight height constraint condition is:

[0130]

[0131] Wherein: h(P[j]) represents the height of node P[j]; h min The minimum allowed flight height of the UAV; h max represents the maximum allowed flight height of the UAV.

[0132] The construction of the battery endurance constraint condition is:

[0133]

[0134] Wherein: d max represents the maximum range of the UAV.

[0135] The system is used to implement the method in Embodiment One, and will not be described here.

[0136] Embodiment Three:

[0137] This embodiment proposes an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to realize the method as described in any embodiment of the present application.

[0138] Embodiment Four:

[0139] This embodiment proposes a computer-readable storage medium, which stores a computer program, and the program is executed by a processor to realize the method as described in any embodiment of the present application.

[0140] In the embodiments of the present application, “at least one” means one or more, and “multiple” means two or more. “And / or” describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the cases of A alone, A and B together, and B alone. Wherein A and B can be singular or plural. The character “ / ” generally represents an “or” relationship between the front and rear associated objects. “At least one of the following” and the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0141] Those of ordinary skill in the art can realize that the units and algorithm steps described in the embodiments disclosed herein can be realized in electronic hardware, computer software, and a combination of electronic hardware and computer software. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solutions. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0143] In several embodiments provided in the present application, any function realized in the form of a software function unit and sold or used as an independent product can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory; hereinafter referred to as: ROM), a random access memory (Random Access Memory; hereinafter referred to as: RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0144] The above only describes the embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation based on the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A power transmission line inspection method based on a combination of three-dimensional laser point cloud and image, characterized in that, The method comprises the following steps: planning an initial flight path of the power transmission line, and performing a flight task by the unmanned aerial vehicle through the initial flight path and collecting three-dimensional point cloud data and image data of the power transmission line; constructing a power facility component recognition model, and identifying position data of each power facility component in the image data through the power facility component recognition model; preprocessing the three-dimensional point cloud data, constructing a point cloud image fusion model, projecting the three-dimensional point cloud data into the image data, and obtaining position data of the three-dimensional point cloud of the power facility component in the three-dimensional space through the position data of each power facility component in the image data; the point cloud image fusion model is constructed based on a collinear equation model, and three-dimensional space coordinates corresponding to each pixel point in the image data are obtained by projecting the three-dimensional point cloud data into the image data, and the specific formula is as follows: in: Represents the two-dimensional coordinates of a pixel; Represents the coordinates of a 3D point cloud corresponding to a pixel; The inverse rotation matrix representing the camera's extrinsic parameters; This represents the depth value of the point cloud; , These represent the camera's focal length at... and Pixel value for direction; Represents the three-dimensional spatial coordinates of the camera center; Indicates the coordinates of the camera's optical center; constructing a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space and storing the three-dimensional model in a database; extracting the corresponding three-dimensional model from the database according to the power facility component to be inspected and constructing a three-dimensional model aggregate target, planning a power transmission line inspection flight path according to the three-dimensional model aggregate target, and performing a flight task by the unmanned aerial vehicle according to the planned power transmission line inspection flight path; planning the power transmission line inspection flight path by constructing a multi-factor flight path planning target function, and the specific formula of the multi-factor flight path planning target function is as follows: wherein: represents a drone path, by a sequence of nodes represents; represents the total number of nodes; represents a node distance between nodes ; represents a distance cost weight; represents a wind speed risk weight; represents a wind speed risk factor for a corresponding node; represents a terrain risk weight; represents a terrain risk factor for a corresponding node; represents a battery endurance risk weight; represents a battery endurance risk factor for a corresponding node; constructing constraint conditions for the optimization path of the multi-factor flight path planning target function, specifically including: The UAV path P must pass through all nodes where the three-dimensional model assembly target is located Construction constraints: wherein: represents the node in which the third three-dimensional model collective target is located; constructing an unmanned aerial vehicle flight height constraint condition: wherein: represents the height of the node ; represents the minimum allowable flight height of the drone; represents the maximum allowable flight height of the drone; constructing a battery endurance constraint condition: wherein: represents the maximum range of the drone. 2.The power transmission line inspection method based on three-dimensional laser point cloud and image combination according to claim 1, characterized in that, the power facility component recognition model is constructed based on a machine learning model, and the position data of each power facility component is obtained by training the image data. 3.The power transmission line inspection method based on three-dimensional laser point cloud and image combination of claim 1, wherein, constructing a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space through multi-view geometric reconstruction technology.

4. A power transmission line inspection system based on three-dimensional laser point cloud and image combination, characterized in that, The method for any one of claims 1 to 3 comprises a data collection module, a power facility component marking module, a point cloud image fusion module, a power facility component three-dimensional model construction module, and a path planning module; the data collection module is used for planning an initial flight path of the power transmission line, performing a flight task through the initial flight path, and collecting three-dimensional point cloud data and image data of the power transmission line; the power facility component marking module is used for constructing a power facility component recognition model, and identifying position data of each power facility component in the image data through the power facility component recognition model; the point cloud image fusion module is used for preprocessing the three-dimensional point cloud data, constructing a point cloud image fusion model, projecting the three-dimensional point cloud data into the image data, and obtaining position data of the three-dimensional point cloud of the power facility component in the three-dimensional space through the position data of each power facility component in the image data; the power facility component three-dimensional model construction module is used for constructing a three-dimensional model of the power facility component based on the position data of the three-dimensional point cloud of the power facility component in the three-dimensional space and storing the three-dimensional model in a database; The path planning module is configured to extract corresponding three-dimensional models of power facility components to be inspected from a database and construct a three-dimensional model assembly target, plan a power transmission line inspection route according to the three-dimensional model assembly target, and perform a flight task according to the planned power transmission line inspection route.

5. The power line inspection system based on three-dimensional laser point cloud and image combination of claim 4, wherein, The power facility component recognition model is constructed based on a machine learning model, and position data of each power facility component is obtained by training image data. 6.The power transmission line inspection system based on three-dimensional laser point cloud and image combination of claim 4, wherein, A three-dimensional model of the power facility component is constructed based on position data of the three-dimensional point cloud of the power facility component in a three-dimensional space through multi-view geometric reconstruction technology.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the method of any one of claims 1 to 3 when executing the program.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1 to 3.

Images