Power adaptive inspection method, unmanned aerial vehicle and storage medium

Abstract

The application provides a power adaptive inspection method, a UAV and a storage medium, and belongs to the technical field of UAV inspection. The method comprises the following steps: based on the identification result of a real-time environment image, the UAV is controlled to move to the top of a to-be-inspected tower; the first tower image of the to-be-inspected tower is collected at the top of the to-be-inspected tower; the UAV is controlled to move to a plurality of target positions, and the second tower image is collected at each target position; and in the case that the inspection end condition is not met, the UAV is controlled to move to the next to-be-inspected tower based on the wire direction in the real-time environment image. In this way, the UAV is adaptively moved to the top and the surroundings of the tower through the identification result of the environment image, different direction tower images are collected, and the next to-be-inspected tower is moved to based on the wire direction in the image, so that adaptive power inspection is realized, the flight route does not need to be planned in advance by humans, and the labor cost is greatly saved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle inspection, in particular to a power adaptive inspection method, an unmanned aerial vehicle and a storage medium. BACKGROUND

[0002] In recent years, unmanned aerial vehicle inspection technology has been widely used. With its advantages of high efficiency, flexibility, safety and the like, it has gradually replaced the traditional inspection method and become a new means of efficient inspection in modern society. For example, in the power, oil and gas, photovoltaic power station, agriculture and city management industries, the application of unmanned aerial vehicle inspection is becoming more and more common.

[0003] The core of unmanned aerial vehicle power inspection is to use a designated flight path planned in advance and accurately set to perform systematic and detailed photographing and inspection operations on the power towers in the target area. This method makes full use of the high mobility and flexibility of the unmanned aerial vehicle, so that it can accurately and quickly reach each predetermined inspection point according to the established flight path. However, it requires more human involvement, such as manually planning the flight path, which leads to a complex inspection method and high cost. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a power adaptive inspection method, an unmanned aerial vehicle and a storage medium, which realize adaptive power inspection without human planning of the flight path in advance, reduce the complexity of inspection and reduce the labor cost.

[0005] In order to achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows:

[0006] In a first aspect, the present application provides a power adaptive inspection method applied to a central control device of an unmanned aerial vehicle, wherein the central control device is in communication connection with a camera of the unmanned aerial vehicle, and the method comprises:

[0007] controlling the unmanned aerial vehicle to move to directly above a to-be-inspected tower based on the recognition result of the real-time environment image collected by the camera;

[0008] controlling the camera to collect a first tower image of the to-be-inspected tower directly above the to-be-inspected tower;

[0009] controlling the unmanned aerial vehicle to move to a plurality of target positions, and controlling the camera to collect a second tower image at each target position; wherein the plurality of target positions are located around the to-be-inspected tower;

[0010] in the case where the inspection end condition is not met, controlling the unmanned aerial vehicle to move to a next to-be-inspected tower based on the direction of the power line in the real-time environment image, and returning to the step of controlling the unmanned aerial vehicle to move to directly above the to-be-inspected tower based on the recognition result of the real-time environment image collected by the camera.

[0011] Optionally, the step of controlling the UAV to move to directly above the tower to be measured based on the recognition result of the real-time environment image collected by the camera comprises:

[0012] inputting the real-time environment image collected by the camera into a tower detection model to obtain a tower detection result;

[0013] in a case where a tower head detection frame of the tower to be measured exists in the tower detection result and a tower head detection frame does not exist, adjusting the direction of the UAV according to the long side direction of the tower head detection frame;

[0014] obtaining a wire direction from the real-time environment image collected by the camera, controlling the UAV to move forward along the wire direction, and returning to execute the step of inputting the real-time environment image collected by the camera into the tower detection model to obtain the tower detection result;

[0015] in a case where a tower head detection frame first appears in the tower detection result, adjusting the position of the UAV to directly above the tower to be measured according to the tower head detection frame.

[0016] Optionally, the step of obtaining a wire direction from the real-time environment image collected by the camera, controlling the UAV to move forward along the wire direction comprises:

[0017] inputting the real-time environment image collected by the camera into a wire detection model to obtain a wire detection result; wherein the wire detection result comprises a plurality of first key points and second key points;

[0018] fitting each of the first key points to obtain a first wire, and fitting each of the second key points to obtain a second wire;

[0019] superimposing the direction vectors of the first wire and the second wire to obtain a wire direction;

[0020] controlling the UAV to move according to the wire direction.

[0021] Optionally, the step of adjusting the position of the UAV to directly above the tower to be measured according to the tower head detection frame comprises:

[0022] detecting whether the center point of the tower head detection frame and the camera center point of the camera coincide;

[0023] if not, adjusting the position of the UAV based on the center point of the tower head detection frame;

[0024] obtaining the center point of the tower head detection frame from the real-time environment image collected by the camera, and returning to execute the step of detecting whether the center point of the tower head detection frame and the camera center point of the camera coincide.

[0025] Optionally, the step of controlling the UAV to move to the plurality of target positions comprises:

[0026] For each preset shooting direction, a target position coordinate is obtained according to the current coordinate of the UAV and the shooting direction;

[0027] According to the target position coordinates, the UAV is controlled to move to each target position in sequence.

[0028] Optionally, the step of controlling the UAV to move to the plurality of target positions in sequence according to the target position coordinates comprises:

[0029] One of the target positions is selected as a target shooting position;

[0030] According to the target shooting position coordinate and the current coordinate of the UAV, a pose adjustment amount is obtained;

[0031] The motion state of the UAV is controlled according to the pose adjustment amount, so as to move the UAV to the target shooting position;

[0032] In the case where the image acquisition of the target shooting position is completed, a new target shooting position is selected from the remaining target positions, and the step of obtaining the pose adjustment amount according to the target shooting position coordinate and the current coordinate of the UAV is executed.

[0033] Optionally, the plurality of target positions comprise a right position, a front position, a left position and a rear position of the tower to be measured.

[0034] The step of controlling the UAV to move to the plurality of target positions comprises:

[0035] The UAV is controlled to move a first preset distance to the right and rotate a first preset angle to the left to reach the right position of the tower to be measured.

[0036] At the right position of the tower to be measured, the UAV is controlled to rotate a second preset angle to the right, move a second preset distance forward, and rotate a third preset angle to the left to reach the front position of the tower to be measured.

[0037] At the front position of the tower to be measured, the UAV is controlled to rotate a second preset angle to the right, move a second preset distance forward, and rotate a third preset angle to the left to reach the left position of the tower to be measured.

[0038] At the left position of the tower to be measured, the UAV is controlled to rotate a second preset angle to the right, move a second preset distance forward, and rotate a third preset angle to the left to reach the rear position of the tower to be measured.

[0039] Optionally, the central control device is in communication connection with a service device, and the service device obtains the tower detection model by:

[0040] a plurality of sample pictures of the power poles are acquired, wherein each sample picture has different weather and / or perspective;

[0041] a label is labeled for each sample picture according to the power pole in the sample picture, wherein the label includes any one of a tower head detection frame, a pole tower detection frame and a pole head detection frame of the power pole;

[0042] the initial model is trained by using the labeled sample pictures to obtain the pole tower detection frame.

[0043] In a second aspect, the present application provides a UAV, comprising a central control device, wherein the central control device comprises a processor and a memory, the memory stores a computer program capable of being executed by the processor, and the processor can execute the computer program to implement the power adaptive inspection method according to the first aspect.

[0044] In a third aspect, the present application provides a storage medium, which stores a computer program, and the computer program is executed by a processor to implement the power adaptive inspection method according to the first aspect.

[0045] In a fourth aspect, the present application provides a power adaptive inspection device applied to a central control device of a UAV, wherein the central control device is in communication connection with a camera of the UAV, and the power adaptive inspection device comprises an identification moving module and an image acquisition module.

[0046] The identification moving module is configured to control the UAV to move to the top of a to-be-inspected tower based on an identification result of a real-time environment image acquired by the camera.

[0047] The image acquisition module is configured to control the camera to acquire a first tower image of the to-be-inspected tower at the top of the to-be-inspected tower.

[0048] The image acquisition module is configured to control the UAV to move to a plurality of target positions, and control the camera to acquire a second tower image at each target position, wherein the plurality of target positions are located around the to-be-inspected tower.

[0049] The identification moving module is configured to, in a case where an inspection end condition is not met, control the UAV to move to a next to-be-inspected tower based on a power line direction in a real-time environment image, and return to the step of controlling the UAV to move to the top of the to-be-inspected tower based on the identification result of the real-time environment image acquired by the camera.

[0050] The power adaptive inspection method, the unmanned aerial vehicle and the storage medium provided by the embodiments of the present application, the method comprises: based on the recognition result of the real-time environment image collected by the camera, controlling the unmanned aerial vehicle to move to the top of the to-be-tested tower; in the top of the to-be-tested tower, controlling the camera to collect the first tower image of the to-be-tested tower; controlling the unmanned aerial vehicle to move to a plurality of target positions, and in each target position, controlling the camera to collect a second tower image, the plurality of target positions are located around the to-be-tested tower; in the case where the inspection end condition is not met, based on the wire direction in the real-time environment image, controlling the unmanned aerial vehicle to move to the next to-be-tested tower, and returning to execute the step of controlling the unmanned aerial vehicle to move to the top of the to-be-tested tower based on the recognition result of the real-time environment image collected by the camera. In this way, through the recognition result of the environment image, the unmanned aerial vehicle is adaptively moved to the top and around of the tower to collect tower images in different directions, and is moved to the next to-be-tested tower based on the wire direction in the image, realizing adaptive power inspection, without manually planning the flight route in advance, greatly saving the labor cost.

[0051] In order to make the above objectives, characteristics and advantages of the present application more apparent, clear and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are referred to, and the detailed description is as follows. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and it should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0053] Figure 1 The system architecture schematic diagram of the power adaptive inspection system provided by the embodiments of the present application is shown.

[0054] Figure 2 The module architecture schematic diagram of the electronic device provided by the embodiments of the present application is shown.

[0055] Figure 3 The flowchart of one of the power adaptive inspection methods provided by the embodiments of the present application is shown.

[0056] Figure 4 The flowchart of one of the power adaptive inspection methods provided by the embodiments of the present application is shown. Figure 3 The flowchart of part of the sub-steps of step 11 in the method is shown.

[0057] Figure 5 The flowchart of part of the sub-steps of step 115 in the method is shown. Figure 4 The flowchart of part of the sub-steps of step 115 in the method is shown.

[0058] Figure 6 The flowchart of part of the sub-steps of step 115 in the method is shown. Figure 4Flowchart of part of the sub-steps of step 117.

[0059] Figure 7 Flowchart of the power adaptive inspection method provided by the embodiment of the present application.

[0060] Figure 8 It is shown that Figure 3 Flowchart of part of the sub-steps of step 15.

[0061] Figure 9 It is shown that Figure 3 Flowchart of part of the sub-steps of step 15.

[0062] Figure 10 It is shown that Figure 9 Flowchart of part of the sub-steps of step 153B.

[0063] Icon: 10-power adaptive inspection system; 110-server; 120-central control device; 130-power module; 140-camera; 20-electronic device; 210-memory; 220-processor; 230-communication module. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0065] Therefore, the detailed description of the embodiments of the present application provided in the accompanying drawings below is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0066] It should be noted that the relational terms herein, such as first and second and the like, are used solely to distinguish one from another entity or action, without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element preceded by "comprises... " does not exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0067] The power adaptive inspection method provided by the embodiments of the present application can be applied to the power adaptive inspection system 10 shown in the figure. Figure 1 The power adaptive inspection system 10 shown in the figure includes a server 110, and a central control device 120, a power module 130 and a camera 140 of a UAV. The power module 130 includes a battery, a motor, an electronic speed controller and a propeller. The central control device 120 is in communication connection with the server 110, the camera 140 and the electronic speed controller respectively. The battery supplies power to the motor. The output shaft of the motor is connected with the propeller.

[0068] The server 110 is configured to train a tower detection model and a wire detection model, and download and deploy the tower detection model and the wire detection model to the central control device 120.

[0069] The camera 140 is configured to, in a working state, collect real-time environment images of an environment where the camera 140 is located, and transmit the real-time environment images to the central control device 120.

[0070] The central control device 120 is configured to implement the power adaptive inspection method provided by the embodiments of the present application, including: based on a recognition result of a real-time environment image collected by the camera 140, controlling the UAV to move to directly above a to-be-detected tower; in the directly above the to-be-detected tower, controlling the camera 140 to collect a first tower image of the to-be-detected tower; controlling the UAV to move to a plurality of target positions, and in each target position, controlling the camera 140 to collect a second tower image, the plurality of target positions being located around the to-be-detected tower; in a case where an inspection end condition is not met, based on a wire direction in the real-time environment image, controlling the UAV to move to a next to-be-detected tower, and returning to perform the step of controlling the UAV to move to directly above the to-be-detected tower based on the recognition result of the real-time environment image collected by the camera 140.

[0071] When it is required to control the unmanned aerial vehicle to rotate, move forward, move backward, climb, descend, etc., the central control device 120 sends a flight control signal to the electronic speed controller to indicate the desired motor speed. After receiving the flight control signal, the electronic speed controller quickly switches the battery of the motor, controls the average voltage supplied to the motor by changing the switching frequency (i.e. duty cycle) of the battery output voltage, and then controls the motor speed to drive the motor connected to the propeller, so as to realize the rotation, forward movement, backward movement, climbing, descending, etc. of the unmanned aerial vehicle.

[0072] Please refer to Figure 2 , which is a block diagram of an electronic device 20. The electronic device 20 can be a central control device 120 in the power adaptive inspection system 10 shown in Figure 1 . The electronic device 20 includes a memory 210, a processor 220, and a communication module 230. The memory 210, the processor 220, and the communication module 230 are directly or indirectly electrically connected to each other to realize the transmission or interaction of data. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines.

[0073] The memory 210 is used to store programs or data. The memory 210 can be, but is not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable read-only memory, an electrically erasable read-only memory, etc.

[0074] The processor 220 is used to read / write the data or program stored in the memory 210 and perform corresponding functions. For example, Figure 1 The processor 220 of the central control device 120 in the power adaptive inspection system 10 shown in

[0075] The communication module 230 is used to establish a communication connection between the electronic device 20 and other communication terminals, and is used to transmit and receive data. For example, Figure 1 In the power adaptive inspection system 10 shown in

[0076] It should be understood that Figure 2 The structure shown in Figure 2 The electronic device 20 can include more or fewer components than those shown in Figure 2 , or have a different configuration from that shown in Figure 2 Each component shown in

[0077] In order to improve the current unmanned aerial vehicle power inspection which needs more human involvement, needs human to formulate the route, and cannot realize autonomous inspection operation, the embodiment of the application provides a power adaptive inspection method, referring to Figure 3 , comprising steps 11 to 17. And, Figure 1 The central control device 120 in the power adaptive inspection system 10 shown in the figure can Figure 2 The structure shown in the figure, when the processor 220 reads the computer program stored in the memory 210, the execution of steps 11 to 17 is realized.

[0078] Step 11, based on the recognition result of the real-time environment image collected by the camera, the unmanned aerial vehicle is controlled to move to the top of the tower to be measured.

[0079] Step 13, in the top of the tower to be measured, the first tower image of the tower to be measured is collected by controlling the camera.

[0080] Step 15, control the unmanned aerial vehicle to move to a plurality of target positions, and in each target position, control the camera to collect a second tower image.

[0081] Among them, a plurality of target positions are located around the tower to be measured.

[0082] Step 17, in the case where the inspection end condition is not met, based on the direction of the power line in the real-time environment image, the unmanned aerial vehicle is controlled to move to the next tower to be measured. After step 17, return to execute step 11.

[0083] For example, in combination with Figure 1 The power adaptive inspection system 10 shown in the figure, in the case where the power line in a certain area or a certain power line needs to be inspected, after the unmanned aerial vehicle autonomously flies to the starting point, the central control device 120 controls the working mode of the camera 140, the camera 140 collects the real-time environment image of the environment, and at the same time, the central control device 120 controls the unmanned aerial vehicle to move to the top of the first tower to be measured based on the recognition result of the real-time environment image collected by the camera 140.

[0084] In the top of the tower to be measured, the central control device 120 controls the camera 140 to collect the first tower image of the tower to be measured. Then, the central control device 120 controls the unmanned aerial vehicle to move to a plurality of target positions by sending a flight control signal to the electronic governor to control the working state of the propeller of the unmanned aerial vehicle, and in each target position, controls the camera 140 to collect a second tower image. In this way, the image collection of the tower to be measured is completed.

[0085] Then, the central control device 120 detects whether the inspection end condition is reached. If yes, the inspection is ended, and the UAV is controlled to fly back to the nest or base. If not, the central control device 120 controls the UAV to move to the next tower to be detected based on the direction of the electric wire in the real-time environment image, and the above process is repeated until the inspection end condition is met.

[0086] The power adaptive inspection method provided by the embodiments of the present application enables the UAV to adaptively move to the top and surroundings of the tower by the recognition result of the environment image in steps 11 to 17, to collect tower images in different directions, and to move to the next tower to be detected based on the direction of the electric wire in the image, so as to realize adaptive power inspection, without planning the flight route in advance, and greatly save the labor cost.

[0087] For step 11, there are various implementation manners for controlling the UAV to move to the top of the tower to be detected based on the recognition result of the real-time environment image collected by the camera 140. For example, the real-time environment image can be input into a model, the model outputs the key points of the electric wire and the tower to be detected, and then any path planning algorithm is used to generate a flight route to the top of the tower to be detected according to the key points, and the movement of the UAV is controlled according to the flight route. Alternatively, the movement of the UAV can be controlled according to the recognition result according to a preset rule. The above manners are all examples, and the implementation manner is not limited.

[0088] In order to make the inspection more accurate and efficient, and to realize the adaptive movement of the UAV, the movement direction of the UAV is adjusted according to the tower head of the tower to be detected in the real-time environment image in step 11, and the position of the UAV is adjusted to the top of the tower according to the tower head of the tower to be detected in the image. Referring to Figure 4 , the process of controlling the UAV to move to the top of the tower to be detected in step 11 includes steps 111 to 117.

[0089] In step 111, the real-time environment image collected by the camera is input into a tower detection model to obtain a tower detection result.

[0090] In step 113, in the case that the tower head detection frame of the tower to be detected exists in the tower detection result, and the tower head detection frame does not exist, the direction of the UAV is adjusted according to the long side direction of the tower head detection frame.

[0091] In step 115, the direction of the electric wire is obtained from the real-time environment image collected by the camera, and the UAV is controlled to move forward along the direction of the electric wire. After step 115, step 111 is executed.

[0092] In step 117, in the case that the tower head detection frame first appears in the tower detection result, the position of the UAV is adjusted to the top of the tower to be detected according to the tower head detection frame.

[0093] The tower detection model is pre-trained, and a model for inferring a tower head detection frame, a tower detection frame and a tower head detection frame of a to-be-detected tower in a real-time environment image. In the case that the to-be-detected tower exists in the real-time environment image, the tower detection result in step 11 at least includes one of the tower head detection frame, the tower detection frame and the tower head detection frame.

[0094] Since the tower head detection frame > the tower detection frame > the tower head detection frame, ideally, if the tower head detection frame, the tower detection frame and the tower head detection frame cannot be obtained together, the tower head detection frame appears earlier than the tower detection frame and the tower head detection frame.

[0095] In step 113, in the case that the long side direction of the tower head detection frame is parallel to the width direction of the real-time environment image, it means that the positive direction of the unmanned aerial vehicle is parallel to the positive direction of the tower head, and the direction of the unmanned aerial vehicle is adjusted to be in place, and the unmanned aerial vehicle can be directly moved forward. In the case that the long side direction of the tower head detection frame is not parallel to the width direction of the real-time environment image, the included angle between the long side direction of the tower head detection frame and the width direction of the real-time environment image can be calculated. Further, the included angle is input into a posture adjustment model of the unmanned aerial vehicle (which can be obtained by model fitting or model training using historical data or experimental data), to obtain a rotation angle and a moving distance. According to the rotation angle and the moving distance, the pose of the unmanned aerial vehicle is adjusted, so that the long side direction of the tower head detection frame is parallel to the width direction of the real-time environment image.

[0096] In this way, the moving direction and the shooting angle of the camera of the unmanned aerial vehicle are ensured to be consistent with the positive direction of the to-be-detected tower, the adaptive patrol inspection of the unmanned aerial vehicle is realized, and at the same time, the patrol flight of the unmanned aerial vehicle deviating from the to-be-detected tower is avoided, which helps to improve the detection accuracy and the detection efficiency.

[0097] In order to further improve the detection accuracy, in step 113, the pixel coordinates of the center point of the tower head detection frame can also be converted to camera coordinates in the camera coordinate system according to the internal and external parameter matrices of the camera, to obtain the camera coordinates of the center point of the tower head detection frame in the camera coordinate system. If the camera coordinates coincide with the center point coordinates of the camera, and the long side direction of the tower head detection frame is parallel to the width direction of the real-time environment image, it means that the positive direction of the unmanned aerial vehicle is parallel to the positive direction of the tower head, and the direction of the unmanned aerial vehicle is adjusted to be in place, and the unmanned aerial vehicle can be directly moved forward.

[0098] In the case that the camera coordinate system and the center point coordinate of the camera do not coincide, and / or the long side direction of the tower head detection frame is not parallel to the width direction of the real-time environment image, the distance between the camera coordinate system and the center point of the camera, and the included angle between the long side direction of the tower head detection frame and the width direction of the real-time environment image can be calculated. Further, the distance and the included angle are input into the attitude adjustment model of the unmanned aerial vehicle to obtain the rotation angle and the movement distance. According to the rotation angle and the movement distance, the pose of the unmanned aerial vehicle is adjusted. Until the camera coordinate system and the center point coordinate of the camera coincide, and the long side direction of the tower head detection frame is parallel to the width direction of the real-time environment image.

[0099] In this way, the to-be-tested tower is ensured to be located at the center of the view angle of the camera of the unmanned aerial vehicle, which can avoid deviation of the inspection from the to-be-tested tower and improve the detection accuracy. At the same time, the reference objects (i.e. the to-be-tested tower and the power line) required for subsequent movement of the unmanned aerial vehicle are ensured to be located at the center of the view angle, which can further ensure stable and accurate inspection flight of the unmanned aerial vehicle.

[0100] In the case that the direction of the unmanned aerial vehicle is adjusted in place, the unmanned aerial vehicle is controlled to move forward along the direction of the power line in step 115, so that the unmanned aerial vehicle flies and inspects along the power transmission line at all times, ensuring the inspection efficiency. The implementation manner is not limited. For example, the direction of a power line in the power transmission line can be obtained from the real-time environment image, and the unmanned aerial vehicle is controlled to move according to the direction as the flight route. The direction of the power line can also be extracted according to a preset rule, and the unmanned aerial vehicle is controlled to move according to the direction as the flight route. The implementation manner is not limited.

[0101] In order to make the power transmission line located at the center of the view angle of the camera to improve the clarity of the inspection image, the concept of moving along the center line of the two power lines of the power transmission line is introduced. Referring to Figure 5 , the process of step 115 of obtaining the direction of the power line from the real-time environment image collected by the camera and controlling the unmanned aerial vehicle to move forward along the direction of the power line includes steps 1151 to 1157.

[0102] In step 1151, the real-time environment image collected by the camera is input into the power line detection model to obtain a power line detection result.

[0103] The power line detection result includes a plurality of first key points and second key points.

[0104] In step 1153, each first key point is fitted to obtain a first power line, and each second key point is fitted to obtain a second power line.

[0105] In step 1155, the direction vectors of the first power line and the second power line are superimposed to obtain a power line direction.

[0106] In step 1157, the unmanned aerial vehicle is controlled to move according to the power line direction.

[0107] In the steps 1151 to 1157, the direction vectors of the first and second electric wires are parallel to the positive direction of the unmanned aerial vehicle, and the scalar of the obtained electric wire direction is located in the middle of the first and second electric wires, so that the power transmission line is always located in the center of the view angle of the camera, thereby improving the clarity of the inspection image and ensuring the inspection quality. Meanwhile, this also helps the to-be-measured tower to be located on the line of the center of the view angle of the camera after the to-be-measured tower falls into the shooting range of the camera.

[0108] In step 117, in the case that the tower head detection frame first appears in the tower detection result of the real-time environment image, in order to quickly, efficiently and adaptively adjust the unmanned aerial vehicle to the top of the tower, the center point of the tower head detection frame and the camera center point of the camera are introduced to finely adjust the position of the unmanned aerial vehicle. Referring to Figure 6 , the process of adjusting the position of the unmanned aerial vehicle to the top of the to-be-measured tower in step 117 according to the tower head detection frame includes steps 1171 to 1177.

[0109] In step 1171, it is detected whether the center point of the tower head detection frame and the camera center point of the camera coincide. If yes, step 1173 is executed, and if no, step 1175 is executed.

[0110] In step 1173, it is determined that the position of the unmanned aerial vehicle has been located at the top of the to-be-measured tower.

[0111] In step 1175, the position of the unmanned aerial vehicle is adjusted based on the center point of the tower head detection frame.

[0112] In step 1177, the center point of the tower head detection frame is obtained from the real-time environment image collected by the camera.

[0113] After step 1177, step 1171 is returned to repeatedly adjust the position of the unmanned aerial vehicle until the position of the unmanned aerial vehicle has been located at the top of the to-be-measured tower.

[0114] In the steps 1171 to 1177, in order to ensure the accuracy of the position adjustment, the pixel coordinates of the center point of the tower head detection frame are first converted to the camera coordinate system according to the internal and external parameter matrices of the camera to obtain the coordinates of the center point of the tower head detection frame in the camera coordinate system. If the coordinates of the center point of the tower head detection frame in the camera coordinate system coincide with the coordinates of the camera center point, the unmanned aerial vehicle has been located at the top of the to-be-measured tower.

[0115] If the coordinates of the center point of the tower head detection frame in the camera coordinate system do not coincide with the camera coordinate system, the distance between the two is calculated, and an adjustment value is obtained according to the distance and the pose adjustment model of the unmanned aerial vehicle. The position of the unmanned aerial vehicle is adjusted according to the adjustment value.

[0116] By the above manner, the unmanned aerial vehicle is located directly above the to-be-detected tower, so that the to-be-detected tower is taken as a visual center to shoot the first tower image, which helps to ensure the clarity of the to-be-detected tower in the first tower image, and thus the inspection quality can be improved.

[0117] The tower detection model and the key point detection model used in each sub-step of step 11 are pre-trained models, and the training process includes two stages of data collection and model training.

[0118] In the data collection stage, a sample collection route on the to-be-detected scene can be planned manually, and the unmanned aerial vehicle is controlled to collect data according to the sample collection route under different weather and / or different angles to obtain multiple sample pictures with different weather and / or angles. Then, the collected sample pictures are labeled, and the label content includes the detection boxes of the tower head, the tower, and the tower head of the electric wire.

[0119] The tower head detection box, the tower detection box, and the tower head detection box can be rectangular boxes, and in other examples, they can also be circular boxes, etc.

[0120] In the model training stage, the pre-processed sample pictures (such as denoising, cropping, data enhancement, etc.) are pre-processed, and then based on the sample pictures, the initial model (such as yolo11 network, etc.) is trained by using the deep learning method to obtain the tower detection model and the electric wire detection model.

[0121] In order to ensure the detection accuracy of the tower head detection box and the tower detection box inferred by the tower detection model, the concept of simultaneously labeling the tower head detection box, the tower detection box, and the tower head detection box of the electric wire pole on the sample picture is introduced. At this time, referring to the electric power adaptive inspection system 10 shown in Figure 7 , Figure 1 The step of obtaining the tower detection model in the server 110 of the electric power adaptive inspection system 10 includes steps 21 to 25.

[0122] Step 21, obtaining multiple sample pictures of electric wire poles.

[0123] Step 23, labeling each sample picture with a label according to the electric wire pole in the sample picture.

[0124] Step 25, training the initial model with the labeled sample pictures to obtain the tower detection box.

[0125] The label includes any one of the tower head detection box, the tower detection box, and the tower head detection box of the electric wire pole, and each sample picture has different weather and / or angles.

[0126] In the above steps 21 to 25, the tower detection frame not needed in the inspection process is introduced as a region class in the labeling process, so that the tower detection model can learn the differences between the tower head detection frame, the tower detection frame and the pole head detection frame in the training process, to improve the recognition ability of the tower detection model for the tower head detection frame and the pole head detection frame.

[0127] After the UAV is located directly above the to-be-tested tower, in step 13, the first tower image is captured by controlling the camera in the original position, and then the UAV is controlled to descend, and after descending by a preset height, the second tower image is captured by controlling the camera. One or more first tower images can also be captured in other ways, which is not limited here.

[0128] Further, in step 15, the UAV can be controlled to move to a plurality of target positions in a plurality of target positions around the to-be-tested tower to collect the second tower image (i.e. the side image) of the to-be-tested tower.

[0129] In an example, in order to reduce the computational complexity, the plurality of target positions are introduced as the right position, the front position, the left position and the rear position of the to-be-tested tower, and the central control device is moved to these positions in turn according to the preset rotation movement rule. Referring to Figure 8 At this time, the process of step 15 of controlling the UAV to move to a plurality of target positions includes steps 151A to 157A.

[0130] Step 151A, control the UAV to move to the right direction by a first preset distance, and rotate to the left by a first preset angle, to reach the right position of the to-be-tested tower.

[0131] Step 153A, in the right position of the to-be-tested tower, control the UAV to rotate to the right by a second preset angle, move forward by a second preset distance, and rotate to the left by a third preset angle, to reach the front position of the to-be-tested tower.

[0132] Step 155A, in the front position of the to-be-tested tower, control the UAV to rotate to the right by a second preset angle, move forward by a second preset distance, and rotate to the left by a third preset angle, to reach the left position of the to-be-tested tower.

[0133] Step 157A, in the left position of the to-be-tested tower, control the UAV to rotate to the right by a second preset angle, move forward by a second preset distance, and rotate to the left by a third preset angle, to reach the rear position of the to-be-tested tower.

[0134] In an example, the first preset distance can be 1 meter, the first preset angle can be 90 degrees, the second preset angle can be 45 degrees, the second preset distance can be 1 meter, and the third preset angle can be 135 degrees. In another example, the first preset distance can be 1.5 meters, the first preset angle can be 90 degrees, the second preset angle can be 45 degrees, the second preset distance can be 1 meter, and the third preset angle can be 135 degrees. The above two examples are examples, and the specific values of the preset distance and the preset angle are not limited.

[0135] Through the above steps 151A to 157A, the unmanned aerial vehicle is controlled to move in the order of right orientation→front orientation→left orientation→rear orientation with the to-be-tested tower as the center point and with an arbitrary value as the radius, so as to collect images in different orientations around the to-be-tested tower, and improve the richness and comprehensiveness of the inspection images. Moreover, in this process, a complex algorithm does not need to be introduced, and the calculation complexity is greatly simplified.

[0136] In another example, in order to move in front, back, left and right without the need for sequential movement, the movement by coordinates can be introduced in step 15 to improve the flexibility of the inspection. Referring to Figure 9 At this time, the process of step 15 for controlling the unmanned aerial vehicle to move to the plurality of target positions includes steps 151B to 153B.

[0137] Step 151B, for each preset shooting orientation, a target position coordinate is obtained according to the current coordinate of the unmanned aerial vehicle and the shooting orientation.

[0138] The preset shooting orientation includes the distance value and the positional relationship with the to-be-tested tower. The current coordinate of the unmanned aerial vehicle can be the coordinate of the unmanned aerial vehicle in the world coordinate system, and the central control device can obtain the coordinate (i.e., the latitude and longitude) of the unmanned aerial vehicle in the world coordinate system (i.e., the latitude and longitude) through GPS navigation, Beidou navigation, etc. Since the unmanned aerial vehicle is located directly above the to-be-tested tower, the latitude and longitude of the unmanned aerial vehicle at this time is the latitude and longitude of the to-be-tested tower.

[0139] Further, according to the coordinate (i.e., the latitude and longitude) of the unmanned aerial vehicle and the shooting orientation (i.e., the distance value and the positional relationship with the to-be-tested tower), a latitude and longitude coordinate can be calculated, which is the target shooting position.

[0140] Step 153B, according to the coordinates of the target positions, the unmanned aerial vehicle is controlled to move to each target position in turn.

[0141] In an example, referring to Figure 10 , the implementation mode of step 153B includes steps 31 to 37.

[0142] Step 31, a target position is selected as the target shooting position.

[0143] Step 33, obtaining the attitude adjustment amount according to the coordinates of the target shooting position and the current coordinates of the UAV.

[0144] Step 35, controlling the movement state of the UAV according to the attitude adjustment amount to move the UAV to the target shooting position.

[0145] Step 37, selecting a new target shooting position from the remaining target positions in the case of image collection at the target shooting position. After step 37, return to step 33.

[0146] In step 33, the coordinates of the target shooting position and the current coordinates of the UAV are both latitude and longitude. Moreover, the two can be input into a pre-trained calculation model to infer the attitude adjustment amount from the calculation model. Alternatively, a pre-constructed calculation formula can be used to calculate the attitude adjustment amount corresponding to the two. The implementation manner is not limited.

[0147] In an example, the attitude adjustment amount can include a horizontal distance and a rotation angle, i.e., the horizontal distance and the rotation angle required for the UAV to move from the current position to the target shooting position. On this basis, in step 35, the UAV is first controlled to move by the horizontal distance, and then rotated by the rotation angle, or first rotated by the rotation angle, and then moved by the horizontal distance, to move to the target shooting position.

[0148] In another example, in order to improve the speed of the UAV moving to the target shooting position, in step 35, the speed values of the UAV on the X-axis and the Y-axis can be calculated through the rotation angle, the horizontal distance, and the set moving speed, and the UAV is controlled at the speed to move to the target shooting position. At this time, the UAV can be moved to the target shooting position at one time only by controlling the speed on the X-axis and the Y-axis, without the need for the UAV to rotate and move in steps, greatly improving the speed of position adjustment.

[0149] For step 17, the inspection end condition can be flexibly set, for example, the total number of the inspected tower to be tested that has completed the inspection can reach a preset number, the inspection of all the tower to be tested in a specified area can be completed, or the battery power of the UAV can only support returning to the nest, and the implementation manner is not limited.

[0150] In addition, in step 17, the implementation manner of controlling the UAV to move to the next tower to be tested based on the direction of the power line in the real-time environment image is the same as steps 1151 to 1155 described above, and the detailed content and elaboration of steps 1151 to 1155 are referred to, which will not be repeated here.

[0151] Based on the same concept as the above power adaptive inspection method, the embodiments of the present application also provide a power adaptive inspection device, which comprises an identification moving module and an image collection module. The power adaptive inspection device can be deployed as a virtual device toFigure 1 The central control device 120 of the power adaptive inspection system 10 shown.

[0152] The identification moving module is configured to control the UAV to move to the top of the to-be-inspected tower based on the identification result of the real-time environment image collected by the camera.

[0153] The image collection module is configured to control the camera to collect a first tower image of the to-be-inspected tower at the top of the to-be-inspected tower.

[0154] The image collection module is configured to control the UAV to move to a plurality of target positions, and control the camera to collect a second tower image at each of the target positions. The plurality of target positions are located around the to-be-inspected tower.

[0155] The identification moving module is configured to, in a case where the inspection end condition is not met, control the UAV to move to a next to-be-inspected tower based on the direction of the power line in the real-time environment image, and return to the step of controlling the UAV to move to the top of the to-be-inspected tower based on the identification result of the real-time environment image collected by the camera.

[0156] The power adaptive inspection device described above, under the synergistic action of the identification moving module and the image collection module, moves the UAV to the top and around of the tower based on the identification result of the environment image, collects tower images in different directions, and moves to the next to-be-inspected tower based on the direction of the power line in the image, thereby realizing adaptive power inspection, without the need for human to plan the flight route in advance, and greatly saving the labor cost.

[0157] The specific implementation and effects of the power adaptive inspection device can be seen from the description of the implementation of the power adaptive inspection method in the foregoing, for example, the specific implementation and effects of the identification moving module can be seen from the description of steps 11 and 17 in the foregoing, and the specific implementation and effects of the image collection module can be seen from the description of steps 13 and 15 in the foregoing, which will not be described herein again.

[0158] In addition, each module of the power adaptive inspection device described above can be realized wholly or partially by software, hardware, and a combination thereof. Each module described above can be embedded in or independent of the processor 220 in the electronic device 20 in hardware form, or can be stored in the memory 210 of the electronic device 20 in software form, so as to be called and executed by the processor 220 to perform the operations corresponding to each module, so as to realize the power adaptive inspection method as provided in the foregoing.

[0159] The embodiment of the present application also provides a UAV, which comprises a central control device 120, the central control device 120 comprises a processor 220 and a memory 210, the memory 210 stores a computer program capable of being executed by the processor 220, and the processor 220 can execute the computer program to realize the power adaptive inspection method as provided above.

[0160] The embodiment of the present application also provides an electronic device 20, which comprises a processor 220 and a memory 210, the memory 210 stores a computer program capable of being executed by the processor 220, and the processor 220 can execute the computer program to realize the power adaptive inspection method as provided above.

[0161] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor 220 to realize the power adaptive inspection method as provided by the embodiment of the present application.

[0162] In summary, the power adaptive inspection method, the UAV and the storage medium provided by the embodiment of the present application at least realize the following effects:

[0163] (1) Based on the tower and wire identification result of the image, the image is adaptively moved and collected, which solves the problem that the traditional UAV tower inspection needs to plan the route in advance, and as much as possible, human intervention is reduced in the whole operation process;

[0164] (2) The AI detection technology is used to identify the two sides of the wire, and the direction of the wire is obtained according to the identification result, so that the UAV flies along the direction of the wire;

[0165] (3) The attribute information of the tower head and the pole head of the identified wire pole is used for the posture adjustment and photographing of the UAV, which is used for the expansion of the subsequent AI identification business;

[0166] (4) According to the direction of the wire, the position information of the next tower can be autonomously found until all the towers are detected.

[0167] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented by other manners. The apparatus embodiments described above are merely illustrative, for example, the flowcharts and block diagrams in the drawings show the possible implementation architecture, function and operation of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementation manners, the functions noted in the blocks can also occur in different order from that noted in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can also be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for executing the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0168] In addition, the function modules in the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0169] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0170] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A power adaptive patrol method, characterized by, The application relates to a central control device applied to a UAV, wherein the central control device is in communication connection with a camera of the UAV, and the method comprises the following steps: controlling the UAV to move to the top of a to-be-inspected tower based on the recognition result of a real-time environment image collected by the camera; controlling the camera to collect a first tower image of the to-be-inspected tower when the UAV is at the top of the to-be-inspected tower; controlling the UAV to move to a plurality of target positions, and controlling the camera to collect a second tower image at each target position; wherein the plurality of target positions are located around the to-be-inspected tower; in the case that the inspection end condition is not met, controlling the UAV to move to the next to-be-inspected tower based on the direction of a power line in the real-time environment image, and returning to the step of controlling the UAV to move to the top of the to-be-inspected tower based on the recognition result of the real-time environment image collected by the camera; the step of controlling the UAV to move to the top of the to-be-inspected tower based on the recognition result of the real-time environment image collected by the camera comprises the following steps: inputting the real-time environment image collected by the camera into a tower detection model to obtain a tower detection result; if the tower head detection frame, the tower detection frame and the tower head detection frame cannot be obtained together, the tower head detection frame appears earlier than the tower detection frame and the tower head detection frame, the tower head detection frame of the to-be-inspected tower exists in the tower detection result, and the tower head detection frame does not exist, if the long side direction of the tower head detection frame is parallel to the width direction of the real-time environment image, the UAV is controlled to move forward, if the long side direction of the tower head detection frame is not parallel to the width direction of the real-time environment image, the direction of the UAV is adjusted according to the included angle between the long side direction of the tower head detection frame and the width direction of the real-time environment image; obtaining the direction of the power line from the real-time environment image collected by the camera, controlling the UAV to move forward along the direction of the power line, and returning to the step of inputting the real-time environment image collected by the camera into the tower detection model to obtain the tower detection result; in the case that the tower head detection frame first appears in the tower detection result, the position of the UAV is adjusted to the top of the to-be-inspected tower according to the tower head detection frame; the step of obtaining the direction of the power line from the real-time environment image collected by the camera, and controlling the UAV to move forward along the direction of the power line comprises the following steps: inputting the real-time environment image collected by the camera into a power line detection model to obtain a power line detection result; wherein the power line detection result comprises a plurality of first key points and second key points; fitting each first key point to obtain a first power line, and fitting each second key point to obtain a second power line; superimposing the direction vectors of the first power line and the second power line to obtain the direction of the power line; controlling the UAV to move according to the direction of the power line.

2. The power adaptive patrol method of claim 1, wherein, the step of adjusting the position of the UAV to the top of the to-be-inspected tower according to the tower head detection frame comprises the following steps: detecting whether the center point of the tower head detection frame and the camera center point of the camera coincide; if not, adjusting the position of the UAV based on the center point of the tower head detection frame. The center point of the rod head detection frame is obtained from a real-time environment image collected by a camera, and whether the center point of the rod head detection frame coincides with a camera center point of the camera is returned.

3. The power adaptive patrol method according to claim 1 or 2, wherein, The step of controlling the unmanned aerial vehicle to move to the plurality of target positions comprises: For each preset shooting direction, a target position coordinate is obtained according to the current coordinate of the unmanned aerial vehicle and the shooting direction; According to the coordinates of the target positions, the unmanned aerial vehicle is controlled to move to each target position in turn.

4. The power adaptive patrol method of claim 3, wherein, The step of controlling the unmanned aerial vehicle to move to the plurality of target positions comprises: A target shooting position is selected from the target positions; An attitude adjustment amount is obtained according to the coordinate of the target shooting position and the current coordinate of the unmanned aerial vehicle; The motion state of the unmanned aerial vehicle is controlled according to the attitude adjustment amount, so that the unmanned aerial vehicle moves to the target shooting position; When the image collection of the target shooting position is completed, a new target shooting position is selected from the remaining target positions, and the step of obtaining the attitude adjustment amount according to the coordinate of the target shooting position and the current coordinate of the unmanned aerial vehicle is executed.

5. The power adaptive patrol method according to claim 1 or 2, wherein, The plurality of target positions include a right position, a front position, a left position and a rear position of the tower to be measured. The step of controlling the unmanned aerial vehicle to move to the plurality of target positions comprises: The unmanned aerial vehicle is controlled to move to the right direction by a first preset distance, and to rotate to the left by a first preset angle, so as to reach the right position of the tower to be measured. At the right position of the tower to be measured, the unmanned aerial vehicle is controlled to rotate to the right by a second preset angle, to move forward by a second preset distance, and to rotate to the left by a third preset angle, so as to reach the front position of the tower to be measured. At the front position of the tower to be measured, the unmanned aerial vehicle is controlled to rotate to the right by a second preset angle, to move forward by a second preset distance, and to rotate to the left by a third preset angle, so as to reach the left position of the tower to be measured. At the left position of the tower to be measured, the unmanned aerial vehicle is controlled to rotate to the right by a second preset angle, to move forward by a second preset distance, and to rotate to the left by a third preset angle, so as to reach the rear position of the tower to be measured.

6. The power adaptive patrol method of claim 1, wherein, The central control device is in communication connection with a service device, and the service device obtains a tower detection model, comprising: A plurality of sample pictures of power poles are obtained; each sample picture has different weather and / or viewing angles; Each sample picture is labeled with a label according to the power pole in the sample picture; the label includes any one of a tower head detection frame, a tower detection frame and a rod head detection frame of the power pole; The initial model is trained with the labeled sample pictures to obtain a tower detection frame.

7. A drone, characterized in that, The central control device comprises a processor and a memory, the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the power adaptive inspection method according to any one of claims 1 to 6.

8. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the power adaptive inspection method according to any one of claims 1 to 6.

Images