Power adaptive inspection method, unmanned aerial vehicle and storage medium

Abstract

The invention provides a power adaptive inspection method, an unmanned aerial vehicle and a storage medium, and belongs to the technical field of unmanned aerial vehicle inspection. The method comprises the steps of controlling an unmanned aerial vehicle to move to a position right above a to-be-detected tower based on an identification result of a real-time environment image; collecting a first tower image of the to-be-detected tower right above the to-be-detected tower; controlling the unmanned aerial vehicle to move to a plurality of target positions, and collecting a second tower image at each target position; and controlling the unmanned aerial vehicle to move towards the next pole tower to be detected based on the direction of the electric wire in the real-time environment image under the condition that the routing inspection ending condition is not met. Thus, through the recognition result of the environment image, the unmanned aerial vehicle is enabled to adaptively move to the position right above and around the tower to collect tower images in different directions, and is moved to the next to-be-detected tower based on the direction of the electric wire in the image, so that adaptive electric power inspection is realized, a route does not need to be manually planned in advance, and the labor cost is greatly saved.

Description

Technical Field

[0001] The present application relates to the field of drone inspection technology, and more specifically, to an adaptive power inspection method, a drone, and a storage medium. Background Art

[0002] In recent years, drone inspection technology has been widely used. With its advantages of high efficiency, flexibility and safety, it has gradually replaced traditional inspection methods and become a new means of efficient inspection in modern society. For example, drone inspections are becoming more and more common in industries such as electricity, oil and gas, photovoltaic power stations, agriculture and urban management.

[0003] The core of drone power inspection is to use a pre-planned and precisely set designated route to systematically and meticulously photograph and inspect power towers in the target area. This method makes full use of the high maneuverability and flexibility of drones, enabling them to accurately and quickly reach each scheduled inspection point according to the established flight path. However, it requires more human participation and requires manual route planning, which makes the inspection method complicated and costly. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide an adaptive power inspection method, a drone and a storage medium, which realize adaptive power inspection without the need for manual route planning in advance, thereby reducing inspection complexity and labor costs.

[0005] In order to achieve the above purpose, the technical solution adopted in this application is as follows:

[0006] In a first aspect, the present application provides a power adaptive inspection method, which is applied to a central control device of a drone, wherein the central control device is communicatively connected to a camera of the drone, and the method comprises:

[0007] Based on the recognition results of the real-time environmental images collected by the camera, the drone is controlled to move directly above the tower to be tested;

[0008] Directly above the pole tower to be measured, controlling a camera to collect a first pole tower image of the pole tower to be measured;

[0009] Control the drone to move to a plurality of target positions, and at each of the target positions, control the camera to collect an image of the second pole tower; wherein the plurality of target positions are located around the pole tower to be measured;

[0010] When the inspection end conditions are not met, the drone is controlled to move to the next tower to be tested based on the direction of the wires in the real-time environmental image, and the step of returning to execute the recognition result based on the real-time environmental image captured by the camera to control the drone to move to the top of the tower to be tested.

[0011] Optionally, the step of controlling the drone to move to directly above the pole tower to be tested based on the recognition result of the real-time environment image collected by the camera includes:

[0012] The real-time environment image collected by the camera is input into the pole tower detection model to obtain the pole tower detection result;

[0013] When a tower head detection frame of the tower to be detected exists in the tower detection result, and no pole head detection frame exists, adjusting the direction of the drone according to the long side direction of the tower head detection frame;

[0014] Obtain the direction of the electric wire from the real-time environment image collected by the camera, control the UAV to move forward along the direction of the electric wire, and return to execute the step of inputting the real-time environment image collected by the camera into the pole tower detection model to obtain the pole tower detection result;

[0015] When a pole head detection frame appears for the first time in the pole tower detection result, the position of the drone is adjusted to be directly above the pole tower of the pole tower to be detected according to the pole head detection frame.

[0016] Optionally, the step of acquiring the direction of the electric wire from the real-time environment image captured by the camera and controlling the drone to move forward along the direction of the electric wire includes:

[0017] Inputting the real-time environment image collected by the camera into the wire detection model to obtain the wire detection result; wherein the wire detection result includes a plurality of first key points and second key points;

[0018] Fitting each of the first key points to obtain a first electric wire, and fitting each of the second key points to obtain a second electric wire;

[0019] Superimposing the direction vectors of the first electric wire and the second electric wire to obtain the direction of the electric wire;

[0020] Control the drone to move according to the direction of the wires.

[0021] Optionally, the step of adjusting the position of the drone to be directly above the pole tower to be tested based on the pole head detection frame includes:

[0022] Detecting whether the center point of the club head detection frame coincides with the camera center point of the camera;

[0023] If not, adjusting the position of the drone based on the center point of the rod head detection frame;

[0024] The center point of the club head detection frame is obtained from the real-time environment image collected by the camera, and the step of detecting whether the center point of the club head detection frame coincides with the camera center point of the camera is returned to be executed.

[0025] Optionally, the step of controlling the drone to move to multiple target locations includes:

[0026] For each preset shooting orientation, the coordinates of a target position are obtained according to the current coordinates of the drone and the shooting orientation;

[0027] According to the coordinates of each target position, the drone is controlled to move to each target position in sequence.

[0028] Optionally, the step of controlling the drone to move to each of the target positions in sequence according to the coordinates of each of the target positions comprises:

[0029] Selecting one of the target positions as a target shooting position;

[0030] Obtaining an attitude adjustment amount according to the coordinates of the target shooting position and the current coordinates of the drone;

[0031] Controlling the motion state of the drone according to the attitude adjustment amount to move the drone to the target shooting position;

[0032] When the image acquisition of the target shooting position is completed, a new target shooting position is selected from the remaining target positions, and the step of obtaining the attitude adjustment amount according to the coordinates of the target shooting position and the current coordinates of the drone is returned to be executed.

[0033] Optionally, the multiple target positions include the right position, the front position, the left position and the rear position of the tower to be measured;

[0034] The step of controlling the drone to move to multiple target locations includes:

[0035] Control the drone to move rightward by a first preset distance and rotate leftward by a first preset angle to reach the right position of the tower to be measured;

[0036] At the right position of the pole tower to be measured, the drone is controlled to rotate rightward by a second preset angle, move forward by a second preset distance, and rotate leftward by a third preset angle to reach the front position of the pole tower to be measured;

[0037] At the front position of the pole tower to be measured, the drone is controlled to rotate rightward by a second preset angle, move forward by a second preset distance, and rotate leftward by a third preset angle to reach the left position of the pole tower to be measured;

[0038] At the left position of the pole tower to be measured, the drone is controlled to rotate rightward by a second preset angle, move forward by a second preset distance, and rotate leftward by a third preset angle to reach the rear position of the pole tower to be measured.

[0039] Optionally, the central control device is communicatively connected with the service device, and the step of the service device obtaining the tower detection model includes:

[0040] Obtain multiple sample images of electric poles; wherein each sample image has different weather conditions and / or viewing angles;

[0041] Label each sample image according to the electric poles in the sample image; wherein the label includes any one of the tower head detection frame, the pole tower detection frame and the pole head detection frame of the electric pole;

[0042] The initial model is trained with the labeled sample images to obtain the tower detection frame.

[0043] In a second aspect, the present application provides an unmanned aerial vehicle, comprising a central control device, the central control device comprising a processor and a memory, the memory storing a computer program that can be executed by the processor, and the processor executing the computer program to implement the power adaptive inspection method as described in the first aspect.

[0044] In a third aspect, the present application provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the power adaptive inspection method as described in the first aspect.

[0045] In a fourth aspect, the present application provides an adaptive power inspection device, which is applied to a central control device of a drone, wherein the central control device is communicatively connected to a camera of the drone, and the adaptive power inspection device includes a recognition movement module and an image acquisition module;

[0046] The identification and movement module is used to control the drone to move to the top of the pole tower to be tested based on the identification result of the real-time environment image collected by the camera;

[0047] The image acquisition module is used to control a camera to acquire a first tower image of the tower to be measured directly above the tower to be measured;

[0048] The image acquisition module is used to control the drone to move to multiple target positions, and at each of the target positions, control the camera to acquire the second tower image; wherein the multiple target positions are located around the tower to be measured;

[0049] The identification and movement module is used to control the drone to move to the next pole tower to be tested based on the direction of the wires in the real-time environmental image when the inspection end condition is not met, and return to execute the identification result based on the real-time environmental image captured by the camera to control the drone to move to the top of the pole tower to be tested.

[0050] The embodiment of the present application provides an adaptive power inspection method, a drone and a storage medium, the method comprising: based on the recognition result of the real-time environmental image collected by the camera, controlling the drone to move to the top of the pole tower to be tested; above the pole tower to be tested, controlling the camera to collect the first pole tower image of the pole tower to be tested; controlling the drone to move to multiple target positions, and at each target position, controlling the camera to collect the second pole tower image, and multiple target positions are located around the pole tower to be tested; if the inspection end condition is not met, based on the direction of the wires in the real-time environmental image, controlling the drone to move to the next pole tower to be tested, and returning to the step of executing the recognition result based on the real-time environmental image collected by the camera, controlling the drone to move to the top of the pole tower to be tested. In this way, through the recognition result of the environmental image, the drone is adaptively moved to the top and the top of the pole tower to collect pole tower images in different directions, and is moved to the next pole tower to be tested based on the direction of the wires in the image, so as to realize adaptive power inspection, without the need for artificial planning of the route in advance, which greatly saves labor costs.

[0051] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A schematic diagram of the system architecture of the power adaptive inspection system provided in an embodiment of the present application is shown.

[0054] Figure 2 A schematic diagram of the module architecture of an electronic device provided in an embodiment of the present application is shown.

[0055] Figure 3 One of the flow charts of the power adaptive inspection method provided in an embodiment of the present application is shown.

[0056] Figure 4 Shows Figure 3 Schematic diagram of the process flow of some sub-steps of step 11.

[0057] Figure 5 Shows Figure 4 A flowchart of some sub-steps of step 115.

[0058] Figure 6 Shows Figure 4A flowchart of some sub-steps of step 117.

[0059] Figure 7 The second flowchart of the power adaptive inspection method provided in the embodiment of the present application.

[0060] Figure 8 Shows Figure 3 One of the flowcharts of some sub-steps of step 15.

[0061] Fig. 9 Shows Figure 3 The second flowchart of some sub-steps of step 15.

[0062] Fig.10 Shows Fig. 9 A flowchart of some sub-steps of step 153B.

[0063] Icons: 10-power adaptive inspection system; 110-server; 120-central control equipment; 130-power module; 140-camera; 20-electronic equipment; 210-memory; 220-processor; 230-communication module. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0065] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0066] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0067] The power adaptive inspection method provided in the embodiment of the present application can be applied to Figure 1 In the power adaptive inspection system 10 shown, the power adaptive inspection system 10 includes a server 110, and a central control device 120, a power module 130 and a camera 140 of the drone. The power module 130 includes a battery, a motor, an electric speed controller and a propeller. The central control device 120 is respectively communicated with the server 110, the camera 140 and the electric speed controller. The battery powers the motor, and the output shaft of the motor is connected to the propeller.

[0068] The server 110 is used to train a pole tower detection model and a wire detection model, and download and deploy the pole tower detection model and the wire detection model to the central control device 120 .

[0069] The camera 140 is used to collect real-time environmental images of the environment in a working state and transmit them to the central control device 120.

[0070] The central control device 120 is used to implement the power adaptive inspection method provided in the embodiment of the present application, including: based on the recognition result of the real-time environmental image captured by the camera 140, controlling the drone to move to the top of the tower to be tested; directly above the tower to be tested, controlling the camera 140 to collect a first tower image of the tower to be tested; controlling the drone to move to multiple target positions, and at each target position, controlling the camera 140 to collect a second tower image, and the multiple target positions are located around the tower to be tested; if the inspection end condition is not met, based on the direction of the wires in the real-time environmental image, controlling the drone to move to the next tower to be tested, and returning to execute the step of controlling the drone to move to the top of the tower to be tested based on the recognition result of the real-time environmental image captured by the camera 140.

[0071] When the drone needs to be controlled to rotate, move forward, move backward, climb, or descend, the central control device 120 sends a flight control signal to the ESC to indicate the desired motor speed. After receiving the flight control signal, the ESC quickly switches the battery of the motor on and off, and adjusts the average voltage supplied to the motor by changing the switching frequency (i.e., duty cycle) of the battery output voltage, thereby controlling the motor speed to drive the motor to connect to the propeller, thereby realizing the rotation, forward, backward, climb, and descent of the drone.

[0072] Please refer to Figure 2 , is a block diagram of an electronic device 20, which may be Figure 1 The central control device 120 in the power adaptive inspection system 10 is shown. The electronic device 20 includes a memory 210, a processor 220 and a communication module 230. The memory 210, the processor 220 and the communication module 230 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0073] The memory 210 is used to store programs or data and can be, but not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable read-only memory, an electrically erasable read-only memory, and the like.

[0074] The processor 220 is used to read / write data or programs stored in the memory 210 and execute corresponding functions. For example, Figure 1 In the power adaptive inspection system 10 shown, the processor 220 of the central control device 120 executes the computer program stored in the memory 210 to implement the power adaptive inspection method provided in the embodiment of the present application.

[0075] The communication module 230 is used to establish a communication connection between the electronic device 20 and other communication terminals, and to send and receive data. Figure 1 In the electric power adaptive inspection system 10 shown, the central control device 120 transmits and receives data with the camera 140 , the electric regulator, etc. through the communication module 230 .

[0076] It should be understood that Figure 2 The structure shown is only a schematic diagram of the structure of the electronic device 20. The electronic device 20 may also include Figure 2 More or fewer components as shown, or with Figure 2 Different configurations are shown. Figure 2 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0077] In order to improve the problem that the current UAV power inspection requires more human participation, requires manual route planning, and cannot realize autonomous inspection operations, the embodiment of the present application provides a power adaptive inspection method, referring to Figure 3 , including steps 11 to 17. And, Figure 1 The central control device 120 in the power adaptive inspection system 10 shown in FIG. Figure 2 The structure shown implements the execution of steps 11 to 17 when the processor 220 reads the computer program stored in the memory 210.

[0078] Step 11: Based on the recognition result of the real-time environmental image collected by the camera, the drone is controlled to move to the top of the tower to be tested.

[0079] Step 13: Control a camera to collect a first tower image of the tower to be measured, just above the tower to be measured.

[0080] Step 15, controlling the UAV to move to multiple target locations, and at each target location, controlling the camera to collect an image of the second tower.

[0081] Among them, multiple target positions are located around the tower to be tested.

[0082] Step 17: If the inspection end condition is not met, the drone is controlled to move to the next pole tower to be inspected based on the direction of the wires in the real-time environment image. After step 17, the process returns to step 11.

[0083] For example, in combination Figure 1 The power adaptive inspection system 10 shown, when it is necessary to inspect the transmission lines in a certain area or a certain transmission line, after the drone autonomously flies to the starting point, the central control device 120 controls the camera 140 to work in mode, and the camera 140 collects real-time environmental images of the environment in which it is located. At the same time, the central control device 120 controls the drone to move to the top of the first tower to be tested based on the recognition result of the real-time environmental image collected by the camera 140.

[0084] Directly above the pole tower to be tested, the central control device 120 controls the camera 140 to collect the first pole tower image of the pole tower to be tested. Then, the central control device 120 controls the drone to move to multiple target positions by sending a flight control signal to the electric controller to control the working state of the propeller of the drone, and controls the camera 140 to collect the second pole tower image at each target position. In this way, the image collection of the pole tower to be tested is completed.

[0085] Next, the central control device 120 detects whether the inspection end condition is met. If it is met, the inspection ends and the drone is controlled to fly back to the gangster's nest or base. If it is not met, the central control device 120 controls the drone to move to the next tower to be tested based on the direction of the wires in the real-time environment image, and continues to repeat the above process until the inspection end condition is met.

[0086] In the adaptive power inspection method provided in the embodiment of the present application, in steps 11 to 17, the drone is adaptively moved to the top and around the tower through the recognition results of the environmental image to collect tower images in different directions, and moves to the next tower to be tested based on the direction of the wires in the image, thereby realizing adaptive power inspection. There is no need for manual planning of the route in advance, which greatly saves labor costs.

[0087] For step 11, there are many ways to control the drone to move to the top of the pole tower to be tested based on the recognition result of the real-time environment image collected by the camera 140. For example, the real-time environment image can be input into the model, and the model can output the key points of the wires and the pole tower to be tested, and then any path planning algorithm can be used to generate a route to the top of the pole tower to be tested based on the key points, and the movement of the drone can be controlled according to the route. It can also be based on the recognition result and the movement of the drone can be controlled according to preset rules. The above methods are all examples, and their implementation methods are not limited.

[0088] In order to make the inspection more accurate and efficient and realize the adaptive movement of the UAV, the moving direction of the UAV is adjusted according to the tower head of the tower to be tested in the real-time environment image in step 11, and the position of the UAV is adjusted to be directly above the tower according to the tower head of the tower to be tested in the image. Figure 4 The process of controlling the drone to move to the top of the pole to be tested in step 11 includes steps 111 to 117.

[0089] Step 111, input the real-time environment image collected by the camera into the pole tower detection model to obtain the pole tower detection result.

[0090] Step 113: if there is a tower head detection frame of the tower to be detected in the tower detection result, and there is no tower head detection frame, adjust the direction of the drone according to the long side direction of the tower head detection frame.

[0091] Step 115, obtaining the direction of the wire from the real-time environment image collected by the camera, and controlling the drone to move forward along the direction of the wire. After step 115, return to step 111.

[0092] Step 117, when the pole head detection frame appears for the first time in the pole tower detection result, the position of the drone is adjusted to be just above the pole tower to be detected according to the pole head detection frame.

[0093] The pole tower detection model is pre-trained, and the tower head detection frame, pole tower detection frame and pole head detection frame of the pole tower to be detected in the real-time environment image are inferred. In the case where there is a pole tower to be detected in the real-time environment image, the pole tower detection result in step 11 includes at least one of the tower head detection frame, the pole tower detection frame and the pole head detection frame.

[0094] Since the tower head detection frame>pole tower detection frame>pole head detection frame, ideally, if the tower head detection frame, pole tower detection frame and pole head detection frame cannot be obtained together, the tower head detection frame appears before the pole tower detection frame and the pole head detection frame.

[0095] In step 113, when the long side direction of the tower head detection frame is parallel to the width direction of the real-time environment image, it means that the positive direction of the drone is in a parallel position with the positive direction of the tower head, and the direction of the drone is adjusted in place, and it can move forward directly. When the long side direction of the tower head detection frame is not parallel to the width direction of the real-time environment image, the angle between the long side direction of the tower head detection frame and the width direction of the real-time environment image can be calculated. Then, the angle is input into the attitude adjustment model of the drone (which can be obtained by using historical data or test data, model fitting or model training), and the rotation angle and moving distance are obtained. According to the rotation angle and moving distance, the posture of the drone is adjusted so that the long side direction of the tower head detection frame is parallel to the width direction of the real-time environment image.

[0096] In this way, the moving direction and shooting angle of the drone's camera are ensured to be consistent with the positive direction of the tower to be tested, realizing the adaptive inspection of the drone. At the same time, it avoids the drone's inspection flight from deviating from the tower to be tested, which helps to improve the detection accuracy and efficiency.

[0097] In order to further improve the detection accuracy, in step 113, the pixel coordinates of the center point of the tower head detection frame can also be converted to the camera coordinate system according to the internal and external parameter matrix of the camera to obtain the camera coordinates of the center point of the tower head detection frame in the camera coordinate system. If the camera coordinates coincide with the coordinates of the center point of the camera, and the long side direction of the tower head detection frame is parallel to the width direction of the real-time environment image, it means that the positive direction of the drone is parallel to the positive direction of the tower head, and the direction of the drone is adjusted in place, and it can move forward directly.

[0098] When the camera coordinates do not coincide with the center point coordinates of the camera, and / or the long side direction of the tower head detection frame is not parallel to the width direction of the real-time environment image, the distance between the camera coordinates and the center point of the camera, and the angle between the long side direction of the tower head detection frame and the width direction of the real-time environment image can be calculated. Then, the distance and angle are input into the attitude adjustment model of the drone to obtain the rotation angle and the moving distance. Adjust the posture of the drone according to the rotation angle and the moving distance. Until the camera coordinates coincide with the center point coordinates of the camera, and the long side direction of the tower head detection frame is parallel to the width direction of the real-time environment image.

[0099] In this way, ensuring that the pole tower to be tested is located in the center of the field of view of the drone's camera can avoid the inspection from deviating from the pole tower to be tested, thereby improving the detection accuracy. At the same time, it can ensure that the reference objects required for the subsequent movement of the drone (i.e. the pole tower to be tested and the wires) are always located in the center of the field of view, thereby ensuring that the drone can perform stable and accurate inspection flights.

[0100] When the direction of the drone is adjusted in place, the drone is controlled to move in the direction of the power line through step 115, so that the drone flight inspection is always carried out along the power line to ensure the inspection efficiency. There are many ways to implement it. For example, the direction of a power line in the power line can be obtained from the real-time environmental image, and the drone is controlled to move based on the direction as the route. It is also possible to extract the direction of the wire according to a preset rule, and control the movement of the drone based on this as the route. The implementation method is not limited.

[0101] In order to make the transmission line located at the center of the camera's viewing angle and improve the clarity of the inspection image, the idea of ​​moving the center line of the two wires of the transmission line is introduced. Figure 5 Step 115 obtains the direction of the wire from the real-time environment image captured by the camera, and controls the drone to move along the direction of the wire, including steps 1151 to 1157.

[0102] Step 1151, input the real-time environment image collected by the camera into the wire detection model to obtain the wire detection result.

[0103] The wire detection result includes a plurality of first key points and second key points.

[0104] Step 1153, fitting each first key point to obtain a first wire, and fitting each second key point to obtain a second wire.

[0105] Step 1155, superimpose the direction vectors of the first electric wire and the second electric wire to obtain the direction of the electric wire.

[0106] Step 1157, control the movement of the drone according to the direction of the wires.

[0107] In the above steps 1151 to 1157, the direction vectors of the first wire and the second wire are parallel to the positive direction of the drone, and the obtained scalar of the wire direction is located between the first wire and the second wire, so that the transmission line is always located at the center of the camera's viewing angle to improve the clarity of the inspection image and ensure the inspection quality. At the same time, this also helps that after the tower to be tested falls into the shooting range of the camera, the tower to be tested is located along the line of the center of the camera's viewing angle.

[0108] In step 117, when the pole head detection frame appears for the first time in the pole tower detection result of the real-time environment image, in order to quickly, efficiently and adaptively adjust the drone to the top of the pole tower, the idea of ​​fine-tuning the position of the drone based on the center point of the pole head detection frame and the camera center point of the camera is introduced. Figure 6 In step 117 , the process of adjusting the position of the drone to be directly above the pole tower to be tested according to the pole head detection frame includes steps 1171 to 1177 .

[0109] Step 1171 , detect whether the center point of the club head detection frame coincides with the camera center point of the camera. If yes, execute step 1173 , if not, execute step 1175 .

[0110] Step 1173, determine that the position of the drone is directly above the tower to be tested.

[0111] Step 1175, adjusting the position of the drone based on the center point of the rod head detection frame.

[0112] Step 1177, obtaining the center point of the club head detection frame from the real-time environment image captured by the camera.

[0113] After step 1177, the process returns to step 1171 to repeatedly adjust the position of the drone until the drone is located directly above the tower to be measured.

[0114] In the above steps 1171 to 1177, in order to ensure the accuracy of position adjustment, the pixel coordinates of the center point of the pole head detection frame are first converted to the camera coordinate system according to the internal and external parameter matrix of the camera, and the coordinates of the center point of the pole head detection frame in the camera coordinate system are obtained. If the coordinates are consistent with the coordinates of the center point of the camera, the drone is located directly above the pole tower to be tested.

[0115] If the coordinates of the center point of the rod head detection frame in the camera coordinate system do not coincide with the camera coordinate system, the distance between the two is calculated, and the model is adjusted according to the distance and the posture of the drone to obtain an adjustment value, and the position of the drone is adjusted according to the adjustment value.

[0116] Through the above method, the drone is located directly above the tower to be tested, so that the first tower image can be captured with the tower to be tested as the visual center, which helps to ensure the clarity of the tower to be tested in the first tower image, thereby improving the inspection quality.

[0117] The tower detection model and key point detection model used in each sub-step of the above step 11 are all pre-trained models, and the training process includes two stages: data collection and model training.

[0118] In the data collection stage, the sample collection route on the scene to be detected can be manually planned, and the drone can be controlled to collect data along the sample collection route under different weather conditions and / or different viewing angles to obtain multiple sample images with different weather conditions and / or viewing angles. Then, the collected sample images are annotated, and the annotation content includes the relevant detection frames of the power lines and poles, the tower head detection frame, the pole tower detection frame, and the pole head detection frame.

[0119] Among them, the tower head detection frame, the pole tower detection frame and the pole head detection frame can all be rectangular frames, and in other examples can also be circular frames, etc.

[0120] In the model training stage, the labeled sample images are preprocessed (such as denoising, cropping, data enhancement, etc.), and then based on the sample images, the initial model (such as the YOLO11 network, etc.) is trained using deep learning methods to obtain the pole tower detection model and the wire detection model.

[0121] In order to ensure the detection accuracy of the tower head detection frame and the pole tower detection frame inferred by the pole tower detection model, the idea of ​​simultaneously marking the tower head detection frame, the pole tower detection frame and the pole head detection frame on the sample image is introduced. Figure 7 , Figure 1 The step of obtaining the tower detection model by the server 110 in the power adaptive inspection system 10 includes steps 21 to 25 .

[0122] Step 21, obtaining multiple sample images of electric poles.

[0123] Step 23, label each sample image according to the electric poles in the sample image.

[0124] Step 25, training the initial model with the labeled sample images to obtain the tower detection frame.

[0125] The labels include any of a tower head detection frame, a pole tower detection frame and a pole head detection frame of a power pole, and each sample image has a different weather and / or viewing angle.

[0126] In the above steps 21 to 25, the pole tower detection frame that is not needed during inspection is introduced as a distinguishing category in the labeling process, so that during the training process, the pole tower detection model can learn the difference between the tower head detection frame, the pole tower detection frame and the pole head detection frame, so as to improve the pole tower detection model's recognition ability of the tower head detection frame and the pole head detection frame.

[0127] After the drone is located directly above the pole tower to be tested, in step 13, the camera is controlled to capture the first image of the first pole tower at the original position, and then the drone is controlled to descend, and after descending to a preset height, the camera is controlled to capture the second image of the first pole tower. One or more images of the first pole tower may also be captured in other ways, which are not limited here.

[0128] Furthermore, in step 15, various methods may be used to control the drone to move to multiple target positions, so as to collect the second tower image (ie, the circumferential image) of the tower to be measured at multiple target positions around the tower to be measured.

[0129] In one example, in order to reduce the computational complexity, multiple target positions are introduced, namely the right position, the front position, the left position and the rear position of the tower to be measured, and the central control device moves to these positions in sequence according to the preset rotation movement rules. Figure 8 At this time, the process of step 15 controlling the drone to move to multiple target locations includes steps 151A to 157A.

[0130] Step 151A, controlling the drone to move rightward by a first preset distance and to rotate leftward by a first preset angle to reach the right position of the tower to be measured.

[0131] Step 153A, at the right position of the tower to be measured, control the drone to rotate rightward by a second preset angle, move forward by a second preset distance, and rotate leftward by a third preset angle to reach the front position of the tower to be measured.

[0132] Step 155A, at the front position of the tower to be measured, control the drone to rotate rightward by a second preset angle, move forward by a second preset distance, and rotate leftward by a third preset angle to reach the left position of the tower to be measured.

[0133] Step 157A, at the left position of the tower to be measured, control the drone to rotate rightward by a second preset angle, move forward by a second preset distance, and rotate leftward by a third preset angle to reach the rear position of the tower to be measured.

[0134] In one example, the first preset distance may be 1 meter, the first preset angle may be 90 degrees, the second preset angle may be 45 degrees, the second preset distance may be 1 meter, and the third preset angle may be 135 degrees. In another example, the first preset distance may be 1.5 meters, the first preset angle may be 90 degrees, the second preset angle may be 45 degrees, the second preset distance may be 1 meter, and the third preset angle may be 135 degrees. The above two examples are for example only, and the specific values ​​of the preset distances and preset angles are not limited.

[0135] Through the above steps 151A to 157A, the drone is controlled to move in the order of right position → front position → left position → rear position with the pole tower to be tested as the center point and any value as the radius, so as to collect images at different positions around the pole tower to be tested, thereby improving the richness and comprehensiveness of the inspection images. In this process, there is no need to introduce complex algorithms, which greatly simplifies the calculation complexity.

[0136] In another example, in order to improve the flexibility of inspection, the idea of ​​moving by coordinates is introduced in step 15 so that the front, back, left, and right do not need to be moved in sequence and can be combined at will. Fig. 9 At this time, the process of step 15 controlling the drone to move to multiple target locations includes steps 151B to 153B.

[0137] Step 151B, for each preset shooting orientation, obtain the coordinates of a target position according to the current coordinates of the drone and the shooting orientation.

[0138] The preset shooting direction includes the distance value and position relationship with the pole tower to be measured. The current coordinates of the drone can be the coordinates of the drone in the world coordinate system. The central control device can obtain the coordinates of the drone in the world coordinate system (i.e. longitude and latitude) through GPS navigation, Beidou navigation, etc. Since the drone is located directly above the pole tower to be measured, the longitude and latitude of the drone at this time are the longitude and latitude of the pole tower to be measured.

[0139] Furthermore, based on the coordinates of the drone (i.e., longitude and latitude) and the shooting direction (i.e., the distance value and position relationship with the tower to be measured), a longitude and latitude coordinate can be calculated, which is the target shooting position.

[0140] Step 153B, according to the coordinates of each target position, control the drone to move to each target position in sequence.

[0141] In one example, refer to Fig.10 , the implementation method of step 153B includes steps 31 to 37.

[0142] Step 31, select a target position as the target shooting position.

[0143] Step 33, obtaining the attitude adjustment amount according to the coordinates of the target shooting position and the current coordinates of the UAV.

[0144] Step 35, controlling the motion state of the drone according to the attitude adjustment amount to move the drone to the target shooting position.

[0145] Step 37, in the case of image acquisition of the target shooting position, a new target shooting position is selected from the remaining target positions. After step 37, the process returns to step 33.

[0146] In step 33, the coordinates of the target shooting position and the current coordinates of the drone are both longitude and latitude. Moreover, the two can be input into a pre-trained computing model, and the computing model can infer the attitude adjustment amount. Alternatively, a pre-built computing formula can be used to calculate the attitude adjustment amounts corresponding to the two. The implementation method is not limited.

[0147] In one example, the attitude adjustment amount may include a horizontal distance and a rotation angle, that is, a horizontal distance and a rotation angle that the drone needs to move from the current position to the target shooting position. On this basis, in step 35, the drone is first controlled to move according to the horizontal distance and then rotate according to the rotation angle, or first rotate according to the rotation angle and then move according to the horizontal distance to move to the target shooting position.

[0148] In another example, in order to increase the speed of the drone moving to the target shooting position, in step 35, the speed values ​​of the drone on the X-axis and Y-axis can be calculated by the rotation angle, horizontal distance and set moving speed, and the aircraft can be controlled according to the speed to move the drone to the target shooting position. In this case, only the speed control of the X-axis and Y-axis is required to move to the target shooting position at one time, and the drone does not need to rotate and move in steps, which greatly improves the speed of position adjustment.

[0149] For step 17, the inspection end condition can be set flexibly. For example, it can be that the total number of towers to be inspected that have completed the inspection reaches a preset number, or that the inspection of all towers to be inspected in a specified area is completed, or that the battery power of the drone only supports returning to the nest. The implementation method is not restricted.

[0150] In addition, in step 17, the method of controlling the drone to move to the next tower to be tested based on the direction of the wires in the real-time environmental image is the same as steps 1151 to 1155 above. Please refer to the detailed content and explanation of steps 1151 to 1155 above, which will not be repeated here.

[0151] Based on the same concept as the above-mentioned power adaptive inspection method, the embodiment of the present application also provides a power adaptive inspection device, including a recognition movement module and an image acquisition module. The power adaptive inspection device can be deployed as a virtual device to Figure 1 In the central control device 120 of the power adaptive inspection system 10 shown.

[0152] The recognition and movement module is used to control the drone to move to the top of the tower to be tested based on the recognition results of the real-time environmental image collected by the camera.

[0153] The image acquisition module is used to control a camera to acquire a first tower image of the tower to be measured just above the tower to be measured.

[0154] The image acquisition module is used to control the drone to move to multiple target positions and control the camera to acquire the second tower image at each target position. The multiple target positions are located around the tower to be measured.

[0155] The identification and movement module is used to control the drone to move to the next pole tower to be tested based on the direction of the wires in the real-time environmental image when the inspection end conditions are not met, and return to execute the recognition result based on the real-time environmental image captured by the camera to control the drone to move to the top of the pole tower to be tested.

[0156] Under the synergistic effect of the recognition and movement module and the image acquisition module, the above-mentioned power adaptive inspection device enables the drone to adaptively move to the top and around the tower through the recognition results of the environmental image, to collect images of the tower in different directions, and to move to the next tower to be tested based on the direction of the wires in the image, thereby realizing adaptive power inspection. There is no need for manual route planning in advance, which greatly saves labor costs.

[0157] For the specific implementation and effect of the power adaptive inspection device, please refer to the description of the implementation of the power adaptive inspection method above. For example, for the specific implementation and effect of the mobile identification module, please refer to the description of the relevant contents of step 11 and step 17 above. For the specific implementation and effect of the image acquisition module, please refer to the description of the relevant contents of step 13 and step 15 above. No further details will be given here.

[0158] In addition, each module of the above-mentioned power adaptive inspection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor 220 in the electronic device 20 in the form of hardware, or can be stored in the memory 210 of the electronic device 20 in the form of software, so that the processor 220 can call and execute the operations corresponding to the above modules to implement the power adaptive inspection method provided above.

[0159] An embodiment of the present application also provides a drone, including a central control device 120, the central control device 120 includes a processor 220 and a memory 210, the memory 210 stores a computer program that can be executed by the processor 220, and the processor 220 can execute the computer program to implement the power adaptive inspection method provided above.

[0160] The embodiment of the present application also provides an electronic device 20, including a processor 220 and a memory 210, wherein the memory 210 stores a computer program that can be executed by the processor 220, and the processor 220 can execute the computer program to implement the power adaptive inspection method provided above.

[0161] The embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by the processor 220, the power adaptive inspection method proposed in the embodiment of the present application is implemented.

[0162] In summary, the power adaptive inspection method, drone, and storage medium provided in the embodiments of the present application achieve at least the following effects:

[0163] (1) Based on the image-based recognition of poles and wires, adaptive movement and image acquisition solve the problem of traditional UAV pole inspections requiring advance route planning, and minimize human involvement in the entire operation process;

[0164] (2) Using AI detection technology, the wires on both sides are identified, and the direction of the wires is obtained based on this, so that the drone can fly along the direction of the wires for inspection;

[0165] (3) Using the identified attribute information of the tower head and pole head of the electric pole, the drone can be used to adjust its posture and take photos, which will be used for the subsequent expansion of AI recognition services;

[0166] (iv) The location information of the next pole tower can be autonomously found according to the direction of the wires until all the pole towers have been detected.

[0167] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0168] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0169] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

[0170] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A power adaptive inspection method, characterized in that: A central control device applied to a drone, wherein the central control device is communicatively connected with a camera of the drone, and the method comprises: Based on the recognition results of the real-time environmental images collected by the camera, the drone is controlled to move directly above the tower to be tested; Directly above the pole tower to be measured, controlling a camera to collect a first pole tower image of the pole tower to be measured; Control the drone to move to a plurality of target positions, and at each of the target positions, control the camera to collect an image of the second pole tower; wherein the plurality of target positions are located around the pole tower to be measured; When the inspection end conditions are not met, the drone is controlled to move to the next tower to be tested based on the direction of the wires in the real-time environmental image, and the step of returning to execute the recognition result based on the real-time environmental image captured by the camera to control the drone to move to the top of the tower to be tested.

2. The power adaptive inspection method according to claim 1, characterized in that: The step of controlling the drone to move to the top of the pole tower to be tested based on the recognition result of the real-time environment image collected by the camera includes: The real-time environment image collected by the camera is input into the pole tower detection model to obtain the pole tower detection result; When a tower head detection frame of the tower to be detected exists in the tower detection result, and no pole head detection frame exists, adjusting the direction of the drone according to the long side direction of the tower head detection frame; Obtain the direction of the electric wire from the real-time environment image collected by the camera, control the UAV to move forward along the direction of the electric wire, and return to execute the step of inputting the real-time environment image collected by the camera into the pole tower detection model to obtain the pole tower detection result; When a pole head detection frame appears for the first time in the pole tower detection result, the position of the drone is adjusted to be directly above the pole tower of the pole tower to be detected according to the pole head detection frame.

3. The power adaptive inspection method according to claim 2, characterized in that: The step of obtaining the direction of the wire from the real-time environment image collected by the camera and controlling the drone to move along the direction of the wire includes: Inputting the real-time environment image collected by the camera into the wire detection model to obtain the wire detection result; wherein the wire detection result includes a plurality of first key points and second key points; Fitting each of the first key points to obtain a first electric wire, and fitting each of the second key points to obtain a second electric wire; Superimposing the direction vectors of the first electric wire and the second electric wire to obtain the direction of the electric wire; Control the drone to move according to the direction of the wires.

4. The power adaptive inspection method according to claim 2, characterized in that: The step of adjusting the position of the drone to be directly above the pole tower to be tested according to the pole head detection frame comprises: Detecting whether the center point of the club head detection frame coincides with the camera center point of the camera; If not, adjusting the position of the drone based on the center point of the rod head detection frame; The center point of the club head detection frame is obtained from the real-time environment image collected by the camera, and the step of detecting whether the center point of the club head detection frame coincides with the camera center point of the camera is returned to be executed.

5. The power adaptive inspection method according to any one of claims 1 to 4, characterized in that: The step of controlling the drone to move to multiple target locations includes: For each preset shooting orientation, the coordinates of a target position are obtained according to the current coordinates of the drone and the shooting orientation; According to the coordinates of each target position, the drone is controlled to move to each target position in sequence.

6. The power adaptive inspection method according to claim 5, characterized in that: The step of controlling the drone to move to each of the target positions in sequence according to the coordinates of each of the target positions comprises: Selecting one of the target positions as a target shooting position; Obtaining an attitude adjustment amount according to the coordinates of the target shooting position and the current coordinates of the drone; Controlling the motion state of the drone according to the attitude adjustment amount to move the drone to the target shooting position; When the image acquisition of the target shooting position is completed, a new target shooting position is selected from the remaining target positions, and the step of obtaining the attitude adjustment amount according to the coordinates of the target shooting position and the current coordinates of the drone is returned to be executed.

7. The power adaptive inspection method according to any one of claims 1 to 4, characterized in that: The multiple target positions include the right position, the front position, the left position and the rear position of the tower to be measured; The step of controlling the drone to move to multiple target locations includes: Control the drone to move rightward by a first preset distance and rotate leftward by a first preset angle to reach the right position of the tower to be measured; At the right position of the pole tower to be measured, the drone is controlled to rotate rightward by a second preset angle, move forward by a second preset distance, and rotate leftward by a third preset angle to reach the front position of the pole tower to be measured; At the front position of the pole tower to be measured, the drone is controlled to rotate rightward by a second preset angle, move forward by a second preset distance, and rotate leftward by a third preset angle to reach the left position of the pole tower to be measured; At the left position of the pole tower to be measured, the drone is controlled to rotate rightward by a second preset angle, move forward by a second preset distance, and rotate leftward by a third preset angle to reach the rear position of the pole tower to be measured.

8. The power adaptive inspection method according to claim 2, characterized in that: The central control device is connected to the service device for communication, and the service device obtains the tower detection model, including: Obtain multiple sample images of electric poles; wherein each sample image has different weather conditions and / or viewing angles; Label each sample image according to the electric poles in the sample image; wherein the label includes any one of the tower head detection frame, the pole tower detection frame and the pole head detection frame of the electric pole; The initial model is trained with the labeled sample images to obtain the tower detection frame.

9. A drone, characterized in that: It comprises a central control device, the central control device comprises a processor and a memory, the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the power adaptive inspection method as described in any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the power adaptive inspection method according to any one of claims 1 to 8 is implemented.

Images