Unmanned aerial vehicle control method and device, computer equipment, storage medium and program product

Abstract

The invention relates to an unmanned aerial vehicle control method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring a target image including power equipment acquired by an unmanned aerial vehicle, and height information of the power equipment actually measured by the unmanned aerial vehicle; generating a height control instruction according to the preset height and the height information; determining a course control instruction based on the position information of the power equipment in the target image; and sending the height control instruction and the course control instruction to the unmanned aerial vehicle. By adopting the method, the control flexibility and the control accuracy of the unmanned aerial vehicle can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of UAV power inspection, and in particular to a control method, device, computer equipment, storage medium and program product for a UAV. Background Art

[0002] Drone inspection is a modern technology that uses drones equipped with advanced sensors and equipment to conduct regular or irregular inspections of specific areas or facilities. Due to its advantages of efficient and accurate aerial inspections, it is widely used in power system inspections.

[0003] In the traditional drone power inspection process, the drone's inspection path is determined based on static geometric data and a preset path, and path control instructions are sent to the drone based on the drone's inspection path to instruct the drone to perform inspections along the preset path.

[0004] However, there is currently a problem of poor flexibility in the path control of drones. Summary of the Invention

[0005] Based on this, it is necessary to provide a drone control method, device, computer equipment, storage medium and program product that can improve the path control flexibility of the drone to address the above technical problems.

[0006] In a first aspect, the present application provides a method for controlling a drone, comprising:

[0007] Obtain target images including power equipment collected by the drone, as well as height information of the power equipment measured by the drone;

[0008] Generate altitude control instructions based on preset altitude and altitude information;

[0009] Determine heading control instructions based on the position information of the power equipment in the target image;

[0010] Send altitude control commands and heading control commands to the drone.

[0011] In one embodiment, the determining of the heading control instruction based on the position information of the power equipment in the target image includes:

[0012] Analyze the power equipment in the target image to obtain the location information of the power equipment;

[0013] Obtaining heading control parameters according to the position information of the power equipment and the center position information of the target image;

[0014] Generate heading control instructions based on heading control parameters.

[0015] In one embodiment, the above-mentioned analysis of the electric power equipment in the target image to obtain the location information of the electric power equipment includes:

[0016] Input the target image into the preset segmentation network for segmentation processing to obtain the segmented image corresponding to the target image;

[0017] The position of the electric power equipment in the segmented image is analyzed to obtain the position information of the electric power equipment.

[0018] In one embodiment, the above-mentioned obtaining of the heading control parameter based on the position information of the power equipment and the center position information of the target image includes:

[0019] determining a yaw angle error according to a difference between the position information of the power equipment and the center position information of the target image;

[0020] The yaw angle error is processed by the first proportional integral differential PID control algorithm to obtain the heading adjustment parameter;

[0021] The heading control parameters are determined according to the current heading parameters and heading adjustment parameters of the UAV.

[0022] In one embodiment, the generating of the altitude control instruction according to the preset altitude and altitude information includes:

[0023] The difference between the preset altitude and the altitude information is determined as the altitude error;

[0024] The height error is processed by the second proportional integral differential PID control algorithm to obtain the height control parameter;

[0025] Generate altitude control instructions based on altitude control parameters.

[0026] In one embodiment, after sending the altitude control instruction and the heading control instruction to the drone, the method further includes:

[0027] Receive inspection images after drone inspection;

[0028] The inspection image is recognized based on the preset recognition network to obtain the recognition result of the inspection image.

[0029] In a second aspect, the present application further provides a control device for a drone, comprising:

[0030] An acquisition module, configured to acquire target images including power equipment captured by a drone, and height information of the power equipment measured by the drone;

[0031] A generation module, used to generate a height control instruction according to a preset height and height information;

[0032] a determination module, configured to determine a heading control instruction based on position information of the power equipment in the target image;

[0033] The sending module is used to send altitude control instructions and heading control instructions to the drone.

[0034] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Obtain target images including power equipment collected by the drone, as well as height information of the power equipment measured by the drone;

[0036] Generate altitude control instructions based on preset altitude and altitude information;

[0037] Determine heading control instructions based on the position information of the power equipment in the target image;

[0038] Send altitude control commands and heading control commands to the drone.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0040] Obtain target images including power equipment collected by the drone, as well as height information of the power equipment measured by the drone;

[0041] Generate altitude control instructions based on preset altitude and altitude information;

[0042] Determine heading control instructions based on the position information of the power equipment in the target image;

[0043] Send altitude control commands and heading control commands to the drone.

[0044] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0045] Obtain target images including power equipment collected by the drone, as well as height information of the power equipment measured by the drone;

[0046] Generate altitude control instructions based on preset altitude and altitude information;

[0047] Determine heading control instructions based on the position information of the power equipment in the target image;

[0048] Send altitude control commands and heading control commands to the drone.

[0049] The above-mentioned drone control method, device, computer equipment, storage medium and program product obtain the drone's altitude control instructions and the drone's heading control instructions by respectively analyzing the images of the power equipment and the height information of the power equipment collected by the drone, so as to control the drone according to the altitude control instructions and the heading control instructions. Compared with the existing method of controlling the drone to fly according to a preset path, this solution generates the drone's control instructions in real time through the real-time collected images and altitude, greatly improving the control flexibility of the drone; in addition, the altitude control and heading control are determined separately, further improving the control accuracy of the drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A diagram showing an application environment of a method for controlling a drone in one embodiment;

[0052] Figure 2 1 is a flow chart of a method for controlling a drone in one embodiment;

[0053] Figure 3 is a flow chart of a method for controlling a drone according to another embodiment;

[0054] Figure 4 is a flow chart of a method for controlling a drone according to another embodiment;

[0055] Figure 5 A schematic diagram of an internal processing flow of a preset segmentation network in one embodiment;

[0056] Figure 6 A schematic diagram of a training process for a preset segmentation network in one embodiment;

[0057] Figure 7 is a flow chart of a method for controlling a drone according to another embodiment;

[0058] Figure 8 is a flow chart of a method for controlling a drone according to another embodiment;

[0059] Figure 9 A schematic diagram of an internal processing flow for identifying a network in one embodiment;

[0060] Figure 10is a flow chart of a method for controlling a drone according to another embodiment;

[0061] Figure 11 is a flow chart of a method for controlling a drone according to another embodiment;

[0062] Figure 12 FIG. 4 is a structural block diagram of a control device for a drone in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] Drone inspection is a modern technology that uses drones equipped with advanced sensors and equipment to conduct regular or irregular inspections of specific areas or facilities. Due to its advantages of efficient and accurate aerial inspections, it is widely used in power system inspections.

[0065] Traditional drone power inspections determine the inspection path based on static geometric data and a preset path. Path control instructions are then sent to the drone based on the inspection path, instructing it to perform inspections along the preset path. However, current drone path control suffers from limited flexibility. This application aims to address this issue.

[0066] After introducing the background technology of the drone control method provided by the embodiment of the present application, the following briefly describes the implementation environment involved in the drone control method provided by the embodiment of the present application. The drone control method provided by the embodiment of the present application can be applied to Figure 1The computer device shown in the figure includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be achieved via Wi-Fi, mobile cellular networks, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a method for controlling a drone. The display unit of the computer device is used to produce a visual image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0067] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0068] After introducing the application scenarios of the drone control method provided by the embodiments of the present application, the following focuses on the drone control method described in the present application.

[0069] In one embodiment, Figure 2 As shown, a control method for a drone is provided, which is applied to Figure 1 The computer device in the example is used to illustrate the process, including the following steps:

[0070] S201. Obtain target images including power equipment collected by a drone, and height information of the power equipment measured by the drone.

[0071] The target image refers to an image captured by a drone and includes images of power equipment, which may be equipment such as transmission lines and power towers.

[0072] In this embodiment, an image acquisition device (e.g., a camera) installed on the drone can be used to capture target images including power equipment around the flight path of the drone during flight, as well as height information of the power equipment from the ground measured by a laser radar carried on the drone.

[0073] Optionally, the UAV is equipped with a laser radar to measure the distance of the target transmission line at a high frequency (e.g., 10 Hz or higher) to obtain the current height of the power equipment measured by the laser radar in real time. .

[0074] S202: Generate an altitude control instruction according to the preset altitude and altitude information.

[0075] The "preset altitude" refers to the preset altitude of the drone in mission planning. It should be noted that mission planning involves presetting the drone's path information before takeoff. Mission planning includes both altitude planning and position planning. For example, the altitude and position planning corresponding to the drone at the first moment may differ from those corresponding to the second moment. This means that the altitude and position planning for the drone at different moments are different.

[0076] In this embodiment, after obtaining the height information of the power equipment, the difference between the height information and the preset height can be processed to obtain a calculation result, and a height control instruction can be generated based on the calculation result. Optionally, the height information and the preset height can be input into a control algorithm for processing to obtain a height control instruction; optionally, the height information and the preset height can be input into a preset neural network model for prediction to obtain a height control instruction.

[0077] S203: Determine a heading control instruction based on the position information of the power equipment in the target image.

[0078] In this embodiment, after the target image of the power equipment is obtained as described above, image processing can be performed on the target image of the power equipment to obtain the position information of the power equipment in the target image, and operation processing can be performed on the position information of the power equipment in the target image to obtain the operation processing result, and the heading control instruction can be determined based on the operation processing result.

[0079] Optionally, the position information of the power equipment in the target image can be input into the control algorithm for processing to obtain a heading control instruction; optionally, the position information of the power equipment in the target image can also be input into a preset neural network model for prediction to obtain a heading control instruction.

[0080] S204: Send the altitude control command and the heading control command to the UAV.

[0081] In this embodiment, after the altitude control command and the heading control command are determined, the altitude control command and the heading control command may be sent to the UAV, so that the UAV flies according to the received altitude control command and the heading control command.

[0082] The control method for the drone provided in this embodiment obtains the altitude control instruction and the heading control instruction of the drone by respectively analyzing the image of the power equipment and the height information of the power equipment collected by the drone, so as to control the drone according to the altitude control instruction and the heading control instruction. Compared with the existing method of controlling the drone to fly according to a preset path, this solution generates the control instructions of the drone in real time through the real-time collected images and altitude, which greatly improves the control flexibility of the drone; in addition, the altitude control and heading control are determined separately, which further improves the control accuracy of the drone.

[0083] In one embodiment, Figure 2 Based on the embodiment shown, the process of determining the heading control instruction can be described as follows: Figure 3 As shown, the above S203 "determining the heading control instruction based on the position information of the power equipment in the target image" includes:

[0084] S301: Analyze the power equipment in the target image to obtain location information of the power equipment.

[0085] In this embodiment, after the target image including the power equipment collected by the drone is obtained, the power equipment in the target image can be analyzed to obtain the location information of the power equipment.

[0086] Optionally, the following provides a specific implementation method for obtaining the location information of the power equipment, see Figure 4 , that is, the above-mentioned S301 "analyzing the power equipment in the target image to obtain the location information of the power equipment" includes:

[0087] S3011. Input the target image into a preset segmentation network for segmentation processing to obtain a segmented image corresponding to the target image.

[0088] In this embodiment, after obtaining the target image including the power equipment captured by the drone, the target image can be input into a preset segmentation network for segmentation processing to obtain multiple segmented images corresponding to the target image.

[0089] It should be noted that the preset segmentation network can use the UNet architecture to extract the features of the target image, see Figure 5This approach primarily addresses the issue of excessive model parameters by splitting the original FC layer into horizontal and vertical feature extraction. This approach also improves the ability of spatial information to model long-range semantic features. Secondly, features are reused at the high-level semantic feature layer, and a hybrid attention structure constructed by a convolutional module with enhanced local perception and a Transformer module with enhanced global perception is used for feature refinement, enhancing model recognition in complex environments. The decoder then constructs a feature upsampling structure based on kernel reorganization to effectively capture neighborhood pixel information for feature reduction. Finally, a 1×1 convolution is performed on the output to obtain multi-category features. A convolutional kernel module with enhanced global perception is used for feature refinement, enhancing model recognition in complex environments. Finally, the decoder uses convolutional blocks to extract features, ensuring model lightweightness. A feature upsampling structure based on kernel reorganization is constructed to effectively capture neighborhood pixel information for feature reduction. Finally, a 1×1 convolution is performed on the output to obtain multi-category features, and our improved UNet model is used for transmission line segmentation.

[0090] It should be noted that before using the preset segmentation network to segment the target image, the preset segmentation network needs to be trained. The image processing process before training includes:

[0091] Step 1: Data collection equipment and environment setup. Data will be collected using an M30T drone, equipped with a 4K camera and supporting up to 2.8x lossless zoom, providing high-resolution aerial video and imagery. Data collection will be conducted across multiple locations to ensure scene diversity and avoid the impact of background noise on model training. The selection of collection locations will be based on randomness, encompassing a variety of typical power inspection environments, such as urban, rural, and mountainous areas.

[0092] During step 2, the data collection process, the drone will capture footage from various perspectives, including frontal, top, and side views. This multi-perspective capture method is designed to enhance the model's object detection capabilities from various angles. All aerial footage will be recorded at a resolution of 3840×2160 and a frame rate of 30 fps. A zoom function will be used to ensure clear details of transmission lines and power towers, eliminating the need for manual cropping.

[0093] The next step involves video processing and image extraction. The collected raw data will be organized into a dataset of at least 100 videos. Key frames will be extracted from these videos to generate high-quality static image samples. The estimated number of sample images extracted is 5,000, each of which will contain two main types of objects: power transmission lines and power towers.

[0094] In step 4, object labeling and data formatting, Labelme software is used to label all collected sample images frame by frame, ensuring pixel-level accurate labeling of target objects. The labeling results are saved as a COCO-formatted dataset to facilitate the training and evaluation of deep learning models.

[0095] Step 5: Target pixel distribution statistics. Due to the complex distribution of targets in images, it is not possible to simply calculate the number of targets based on the number of images. Therefore, we plan to perform a statistical analysis of the pixel distribution of each target type to provide more balanced sample training data for the model.

[0096] Step 6, data quality verification, to ensure the quality and diversity of the dataset, the collected data is further screened and verified, low-quality samples are eliminated, and the final dataset is ensured to cover a wide range of target appearance changes and scene diversity.

[0097] Further, the training process of the preset segmentation network can be found in Figure 6 As shown in the figure, the collected data is used as model input for training. During the training process, the model is optimized, weight parameters are updated, and model performance is tested. Finally, the training is completed and the optimal result is saved. To verify the model's generalization ability, the test set data is selected for verification. On the one hand, the performance indicators of the model under the test set are obtained, and on the other hand, the image detection speed is analyzed and the detection results are visualized.

[0098] S3012: Analyze the position of the power equipment in the segmented image to obtain the position information of the power equipment.

[0099] In this embodiment, after the segmented image corresponding to the target image is obtained, the position of the power equipment in the segmented image may be analyzed to obtain the position information of the power equipment.

[0100] S302: Obtain heading control parameters according to the position information of the power equipment and the center position information of the target image.

[0101] The center position information of the target image refers to a position that is equidistant from each vertex of the target image.

[0102] In this embodiment, after obtaining the position information of the power device, the center position information of the target image can be obtained, and the difference between the position information of the power device and the center position information of the target image can be processed to obtain a processing result, and a heading control instruction can be generated based on the processing result. Optionally, the position information of the power device and the center position information of the target image can be input into a control algorithm for processing to obtain a heading control instruction; optionally, the position information of the power device and the center position information of the target image can also be input into a preset neural network model for prediction to obtain a heading control instruction.

[0103] Optionally, a specific implementation method for obtaining the heading control parameters is provided below. Figure 7 The above-mentioned S302 “obtaining a heading control parameter according to the position information of the power equipment and the center position information of the target image” includes:

[0104] S3021. Determine a yaw angle error based on a difference between the position information of the power equipment and the center position information of the target image.

[0105] Among them, the yaw angle error is used to quantify the degree of deviation between the current heading and the target heading.

[0106] In this embodiment, after obtaining the position information of the power equipment and the center position information of the target image as mentioned above, the difference between the position information of the power equipment and the center position information of the target image can be further obtained, and the difference between the position information of the power equipment and the center position information of the target image can be determined as the yaw angle error.

[0107] For example, referring to the following formula (1), a method for obtaining the yaw angle error is disclosed:

[0108]

[0109] in, The horizontal coordinate representing the center of gravity of the power equipment, The horizontal coordinate representing the center position of the target image serves as the reference point for the ideal line position. Represents the camera focal length, used to map image coordinate offsets to angular deviations. It is the core variable for adjusting the heading and reflects the offset of the line in the image.

[0110] in, It can be expressed by the following formula (2):

[0111]

[0112] in, Refers to the horizontal coordinate of the pixel point in the segmented image corresponding to the target image, n refers to the number of segmented images, The vertical coordinate indicating the center of gravity of the power equipment, Refers to the vertical coordinate of the pixel point in the segmented image corresponding to the target image.

[0113] It should be noted that the center of gravity coordinates of the power equipment It represents the average position of the power lines on the segmented image and is the core indicator for measuring line offset.

[0114] S3022: Perform a first proportional-integral-differential (PID) control algorithm on the yaw angle error to obtain a heading adjustment parameter.

[0115] In this embodiment, after the yaw angle error is obtained, the yaw angle error can be processed by a first proportional integral differential (PID) control algorithm to obtain a heading adjustment parameter. For example, the process of obtaining the heading adjustment parameter can be referred to the following formula (3):

[0116]

[0117] in, , , They represent proportional gain, integral gain and differential gain respectively, and are used to adjust the system response speed, error accumulation and dynamic changes. Represents the integral of the yaw angle error, which is used to eliminate steady-state errors. Indicates the rate of change of yaw angle error, used to suppress heading oscillation caused by error change. PID control algorithm can calculate heading adjustment parameters in real time and use it for heading correction.

[0118] S3023. Determine the heading control parameters according to the current heading parameters and heading adjustment parameters of the UAV.

[0119] In this embodiment, after the heading adjustment parameter is obtained as described above, the current heading parameter of the UAV may be obtained, and the heading control parameter may be determined based on the difference between the current heading parameter of the UAV and the heading adjustment parameter.

[0120] Optionally, a process for determining the heading control parameters is provided below, see formula (4):

[0121]

[0122] in, Indicates the heading control parameters, Refers to the current heading parameters of the drone, Refers to the heading adjustment parameter.

[0123] S303: Generate a heading control instruction according to the heading control parameters.

[0124] In this embodiment, after the heading control parameters are determined as described above, a heading control instruction can be generated based on the heading control parameters, and the heading control instruction can be sent to the motor control module of the drone control system so that the motor control module adjusts the rotor speed to achieve accurate correction of the drone heading.

[0125] The method for determining the heading control instruction provided in this embodiment obtains the location information of the power equipment by analyzing the target image including the power equipment collected by the UAV, and obtains the heading control parameters by analyzing the location information of the power equipment and the center position information of the target equipment, thereby providing a data basis for subsequently controlling the heading of the UAV based on the heading control parameters.

[0126] In one embodiment, Figure 2-7 Based on any of the embodiments shown, the process of generating the height control instruction can be described as follows: Figure 8 As shown, the above S202 "generating an altitude control instruction according to the preset altitude and altitude information" includes:

[0127] S401: Determine the difference between the preset altitude and the altitude information as an altitude error.

[0128] In this embodiment, after obtaining the height information of the power equipment obtained by the drone through the laser radar, the difference between the height information and the preset height can be further obtained, and the difference between the height information and the preset height can be determined as the height error.

[0129] For example, referring to the following formula (5), a method for determining the height error is provided:

[0130]

[0131] in, Indicates the preset altitude, i.e., the preset safe flight altitude for the mission, which is usually set dynamically based on the specific installation conditions of the transmission line (such as line type, environmental requirements, etc.). Indicates that the current height of the power equipment is measured by the lidar in real time. Indicates the height error, which is used in subsequent thrust calculations.

[0132] S402: Perform a second proportional-integral-differential (PID) control algorithm on the altitude error to obtain an altitude control parameter.

[0133] In this embodiment, after the height error is obtained, the height error can be processed by a PID control algorithm to obtain a height control parameter. For example, the process of obtaining the height control parameter can be referred to the following formula (6):

[0134]

[0135] in, Represents the proportional gain, which is used to quickly respond to the current error. Represents the integral gain, which is used to accumulate historical errors and eliminate the steady-state error of altitude deviation. It is the differential gain, which is used to suppress the high oscillation caused by the error change rate. It represents the accumulated error value (integral term) and plays an important role in eliminating system deviation. Indicates the instantaneous rate of change of error (differential term), which can improve the dynamic response performance of the system. 、 、 By adjusting the parameters, the drone can achieve fast and precise altitude control while avoiding overshoot and oscillation during altitude adjustment.

[0136] S403: Generate an altitude control instruction according to the altitude control parameters.

[0137] In this embodiment, after the altitude control parameters are determined as described above, an altitude control instruction can be generated based on the altitude control parameters, and the altitude control instruction can be sent to the motor control module of the drone control system so that the motor control module adjusts the rotor speed to achieve accurate correction of the drone heading.

[0138] Optionally, after the altitude control parameter is determined as above, the altitude control parameter T will directly affect the motor speed of the drone, ultimately adjusting the flight altitude of the drone. The specific execution steps are: according to the size of the altitude control parameter T, the speed of each rotor is changed, thereby increasing or decreasing the lift of the drone. When T>0, the drone rises; when T<0, the drone lowers. Furthermore, the system monitors the adjusted altitude in real time. , forming a closed-loop control to ensure that the drone's altitude keeps approaching .

[0139] The method for determining the height control instruction provided in this embodiment obtains the height control parameter by analyzing the difference between the preset height and the height information of the power equipment actually measured by the UAV, thereby providing a data basis for subsequently controlling the height of the UAV based on the height control parameter.

[0140] In one embodiment, Figure 2-8 Based on any of the embodiments shown, see Figure 9After sending the altitude control command and the heading control command to the UAV, the method further includes:

[0141] S205: Receive inspection images after the drone inspection.

[0142] In this embodiment, after the altitude control instructions and heading control instructions are sent to the drone, the drone will fly according to the altitude control instructions and heading control instructions, and complete the inspection of the power equipment during the flight, obtain the inspection image after the inspection, and send the inspection image after the drone inspection to the computer device.

[0143] S206 : Recognize the inspection image based on a preset recognition network to obtain a recognition result of the inspection image.

[0144] The inspection image recognition results include the defect category, location, size, and confidence level. The recognition results are visualized as heat maps showing the defect distribution, helping personnel quickly identify and locate defects.

[0145] In this embodiment, after the inspection image is received, the inspection image may be recognized based on a preset recognition network to obtain a recognition result of the inspection image.

[0146] It should be noted that the preset recognition network generally consists of two parts: Figure 10, feature extraction network, and detection head. The feature extraction network encodes and extracts key feature information from inspection images, such as the structural features of power lines, insulators, fasteners, and pylons. The detection head predicts the location and category of target defects based on the extracted feature maps. To meet the diverse target detection requirements in power inspection scenarios, this solution improves upon the YOLOv4 model, focusing on improving the detection accuracy of small and blurred targets (such as power lines photographed from a distance, tiny cracks on insulators, or rust). Specific improvements include enhanced feature extraction capabilities and optimized multi-scale feature fusion strategies. To enhance the detection of small and blurred targets, this patent adds a 152×152 feature scale to the original YOLOv4 model's detection head design for fine-grained target prediction. This implementation method includes: 1) Multi-scale feature extraction: The original 76×76 feature map is convolutionally and up-sampled, then fused with the shallow feature map to generate a higher-resolution feature map (152×152). This feature map captures detailed information about small targets while preserving deep semantic features. 2) Routing Layer Feature Fusion: The routing layer fuses the model's multi-layer features, combining shallow texture features with deep semantic information to effectively capture both global and local features of power line component defects. This improved architecture utilizes feature maps at four scales (e.g., 19×19, 38×38, 76×76, and 152×152) for target detection, ensuring the ability to extract both large-scale global features and small-scale local features of power lines and components.

[0147] Optionally, after receiving the inspection image as mentioned above, the inspection image can be preprocessed to obtain a preprocessed inspection image, and then feature extraction can be performed on the preprocessed inspection image to obtain feature parameters of the inspection image. The feature parameters of the inspection image can then be input into a preset recognition network for recognition to obtain a recognition result of the inspection image.

[0148] Exemplarily, the process of preprocessing the inspection images includes: standardizing the inspection images obtained by drone inspection, including resolution scaling, contrast enhancement, etc., to obtain standardized inspection images, thereby ensuring that the image quality meets the model requirements.

[0149] For example, the feature extraction process for inspection images involves using a deep learning model to extract features of power lines and their components using a convolutional neural network (CNN). This includes features such as the geometry of power lines and insulators, as well as areas of rust on the tower surface. The feature map then uses multiple layers of convolution to extract both shallow texture information (such as surface cracks) and deep semantic information (such as the overall component morphology).

[0150] For example, the image recognition process for inspection images includes: processing the feature map by the detection head, predicting the defect category (such as broken wires, corrosion, or damaged insulators) through the classification branch, and predicting the defect location and bounding box through the regression branch. The improved detection head can accurately locate smaller objects and adapt to the recognition of ambiguous objects in complex environments. Furthermore, the detection results output from multiple feature scales are combined, and redundant bounding boxes are removed using the non-maximum suppression (NMS) algorithm to generate the final recognition result.

[0151] In an exemplary embodiment, the present application also provides a real-time online defect recognition and report generation system for generating corresponding defect recognition reports based on the recognition results of the inspection images. The real-time online defect recognition and report generation system includes: a data acquisition module, a defect recognition module, a data processing module, and a report generation module. The data acquisition module completes multi-dimensional data collection of the transmission line through the laser radar and high-resolution camera on the drone. The laser radar provides accurate three-dimensional spatial data and can record the structural information of the transmission line in detail, while the camera collects high-resolution image data for further visual analysis. In order to ensure the synchronization and accuracy of the data, the collected images and point cloud data will be paired through timestamps to ensure the temporal consistency of the two, providing a reliable data basis for subsequent processing.

[0152] The Defect Recognition Module processes image and point cloud data in real time based on deep learning algorithms. Utilizing a trained convolutional neural network (CNN) model, this module efficiently identifies and classifies common defects on transmission lines, such as broken lines, contaminated insulators, and corroded towers. Through in-depth analysis of image and point cloud data, the system can promptly identify potential problems and generate alerts, providing precise support for maintenance work.

[0153] The data processing module performs necessary preprocessing on the collected raw data to improve the accuracy and efficiency of subsequent analysis. Image data undergoes denoising and normalization, while point cloud data undergoes filtering and voxelization to reduce noise interference and ensure data clarity and accuracy. The processed data is used for feature extraction and defect identification, providing refined input for further analysis.

[0154] The report generation module automatically generates detailed inspection reports based on defect identification results. These reports include defect classification, location, severity, and recommended repair times, helping personnel quickly understand the current status and priority of issues. Reports can be exported to PDF format for archiving and further analysis, and can also be used by maintenance personnel and decision-makers as a reference to optimize maintenance and management decisions.

[0155] Furthermore, the process of generating an inspection report by the report generation module includes:

[0156] The real-time defect recognition module is responsible for instantly analyzing collected images and point cloud data, using deep learning models to identify and classify defects. By processing image and point cloud data in real time, the system can quickly identify various common defect types and accurately mark their locations within power equipment. Through the efficient analysis of deep learning models, the system provides timely feedback on defect type and location, ensuring that potential issues are promptly identified and addressed.

[0157] Once a defect is identified, the Defect Location and Description module compares it with the drone's flight path map, accurately pinpointing the defect's location and generating a detailed description. This description includes key data such as defect type, location, and severity level, allowing personnel to quickly assess the defect's impact. The system also displays the spatial distribution of defects using a heat map, enabling personnel to fully understand the distribution of various defects within the inspection area and optimize subsequent treatment plans.

[0158] After defect identification and location are complete, the automatic report generation module automatically generates a detailed inspection report based on the system's processing results. This report includes key inspection task information, such as inspection time, location, and equipment number, to ensure data traceability. Furthermore, the report lists the number and distribution of each defect type and provides a defect location map, accurately displaying the severity and location of the defect. Based on the specific defect, the system also generates repair recommendations, assessing repair priority based on defect severity and recommending appropriate treatment measures, providing a scientific basis for subsequent repairs and decision-making.

[0159] In this embodiment, the inspection image is recognized to obtain the recognition result, thereby achieving the effect of drone inspection, thereby ensuring that when there is a fault in the power equipment during the inspection process, appropriate treatment measures can be recommended in a timely manner, providing a scientific basis for subsequent maintenance and decision-making.

[0160] In one embodiment, see Figure 11 , also provides a drone control method, including:

[0161] S10, obtaining a target image including the power equipment captured by the drone, and height information of the power equipment measured by the drone;

[0162] S11, determining the difference between the preset height and the height information as the height error;

[0163] S12, performing a second proportional-integral-differential (PID) control algorithm on the altitude error to obtain an altitude control parameter;

[0164] S13, generating an altitude control instruction according to the altitude control parameters;

[0165] S14, inputting the target image into a preset segmentation network for segmentation processing to obtain a segmented image corresponding to the target image;

[0166] S15, analyzing the position of the power equipment in the segmented image to obtain position information of the power equipment;

[0167] S16, determining a yaw angle error based on a difference between the position information of the power equipment and the center position information of the target image;

[0168] S17, performing a first proportional integral differential (PID) control algorithm on the yaw angle error to obtain a heading adjustment parameter;

[0169] S18. Determine a heading control parameter based on the current heading parameter and the heading adjustment parameter of the UAV;

[0170] S19. Generate a heading control instruction according to the heading control parameters;

[0171] S20, sending the altitude control command and the heading control command to the UAV;

[0172] S21, receiving inspection images after the drone inspection;

[0173] S22. Recognize the inspection image based on a preset recognition network to obtain a recognition result of the inspection image.

[0174] The control method for the drone provided in this embodiment obtains the altitude control instruction and the heading control instruction of the drone by respectively analyzing the image of the power equipment and the height information of the power equipment collected by the drone, so as to control the drone according to the altitude control instruction and the heading control instruction. Compared with the existing method of controlling the drone to fly according to a preset path, this solution generates the control instructions of the drone in real time through the real-time collected images and altitude, which greatly improves the control flexibility of the drone; in addition, the altitude control and heading control are determined separately, which further improves the control accuracy of the drone.

[0175] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0176] Based on the same inventive concept, the present application also provides a drone control device for implementing the aforementioned drone control method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of the one or more drone control device embodiments provided below can be found in the above-mentioned limitations of the drone control method and will not be further elaborated here.

[0177] In an exemplary embodiment, Figure 12 As shown, a control device for a drone is provided, comprising: an acquisition module 10, a generation module 11, a determination module 12, and a sending module 13, wherein:

[0178] The acquisition module 10 is used to acquire target images including power equipment collected by the drone, and height information of the power equipment measured by the drone.

[0179] The generating module 11 is used to generate a height control instruction according to a preset height and height information.

[0180] The determination module 12 is configured to determine a heading control instruction based on the position information of the power equipment in the target image.

[0181] The sending module 13 is used to send the altitude control instruction and the heading control instruction to the UAV.

[0182] In an exemplary embodiment, the determination module 12 includes: an analysis unit, an acquisition unit, and a generation unit, wherein:

[0183] An analysis unit, specifically configured to analyze the power equipment in the target image to obtain location information of the power equipment;

[0184] an acquisition unit, specifically configured to obtain a heading control parameter based on the position information of the power equipment and the center position information of the target image;

[0185] The generating unit is specifically used to generate a heading control instruction according to the heading control parameters.

[0186] In an exemplary embodiment, the above-mentioned analysis unit is further used to input the target image into a preset segmentation network for segmentation processing to obtain a segmented image corresponding to the target image; and analyze the position of the power equipment in the segmented image to obtain the position information of the power equipment.

[0187] In an exemplary embodiment, the acquisition unit is further used to determine the yaw angle error based on the difference between the position information of the power equipment and the center position information of the target image; perform a first proportional integral differential (PID) control algorithm on the yaw angle error to obtain a heading adjustment parameter; and determine the heading control parameter based on the current heading parameter and the heading adjustment parameter of the UAV.

[0188] In an exemplary embodiment, the apparatus further includes: a determination module, a processing module, and a generation module, wherein:

[0189] A determination module, configured to determine a difference between a preset height and the height information as a height error;

[0190] a processing module, configured to process the altitude error using a second proportional-integral-differential (PID) control algorithm to obtain an altitude control parameter;

[0191] The generation module is used to generate altitude control instructions according to altitude control parameters.

[0192] In an exemplary embodiment, the apparatus further includes: a receiving module and an identification module, wherein:

[0193] A receiving module is used to receive inspection images after the drone inspection;

[0194] The recognition module is used to recognize the inspection image based on a preset recognition network to obtain a recognition result of the inspection image.

[0195] Each module in the aforementioned drone control device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in hardware form, or may be stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0196] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0197] Obtain target images including power equipment collected by the drone, as well as height information of the power equipment measured by the drone;

[0198] Generate altitude control instructions based on preset altitude and altitude information;

[0199] Determine heading control instructions based on the position information of the power equipment in the target image;

[0200] Send altitude control commands and heading control commands to the drone.

[0201] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0202] Analyze the power equipment in the target image to obtain the location information of the power equipment;

[0203] Obtaining heading control parameters according to the position information of the power equipment and the center position information of the target image;

[0204] Generate heading control instructions based on heading control parameters.

[0205] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0206] Input the target image into the preset segmentation network for segmentation processing to obtain the segmented image corresponding to the target image;

[0207] The position of the electric power equipment in the segmented image is analyzed to obtain the position information of the electric power equipment.

[0208] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0209] determining a yaw angle error according to a difference between the position information of the power equipment and the center position information of the target image;

[0210] The yaw angle error is processed by the first proportional integral differential PID control algorithm to obtain the heading adjustment parameter;

[0211] The heading control parameters are determined according to the current heading parameters and heading adjustment parameters of the UAV.

[0212] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0213] The difference between the preset altitude and the altitude information is determined as the altitude error;

[0214] The height error is processed by the second proportional integral differential PID control algorithm to obtain the height control parameter;

[0215] Generate altitude control instructions based on altitude control parameters.

[0216] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0217] Receive inspection images after drone inspection;

[0218] The inspection image is recognized based on the preset recognition network to obtain the recognition result of the inspection image.

[0219] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0220] Obtain target images including power equipment collected by the drone, as well as height information of the power equipment measured by the drone;

[0221] Generate altitude control instructions based on preset altitude and altitude information;

[0222] Determine heading control instructions based on the position information of the power equipment in the target image;

[0223] Send altitude control commands and heading control commands to the drone.

[0224] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0225] Analyze the power equipment in the target image to obtain the location information of the power equipment;

[0226] Obtaining heading control parameters according to the position information of the power equipment and the center position information of the target image;

[0227] Generate heading control instructions based on heading control parameters.

[0228] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0229] Input the target image into the preset segmentation network for segmentation processing to obtain the segmented image corresponding to the target image;

[0230] The position of the electric power equipment in the segmented image is analyzed to obtain the position information of the electric power equipment.

[0231] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0232] determining a yaw angle error according to a difference between the position information of the power equipment and the center position information of the target image;

[0233] The yaw angle error is processed by the first proportional integral differential PID control algorithm to obtain the heading adjustment parameter;

[0234] The heading control parameters are determined according to the current heading parameters and heading adjustment parameters of the UAV.

[0235] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0236] The difference between the preset altitude and the altitude information is determined as the altitude error;

[0237] The height error is processed by the second proportional integral differential PID control algorithm to obtain the height control parameter;

[0238] Generate altitude control instructions based on altitude control parameters.

[0239] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0240] Receive inspection images after drone inspection;

[0241] The inspection image is recognized based on the preset recognition network to obtain the recognition result of the inspection image.

[0242] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0243] Obtain target images including power equipment collected by the drone, as well as height information of the power equipment measured by the drone;

[0244] Generate altitude control instructions based on preset altitude and altitude information;

[0245] Determine heading control instructions based on the position information of the power equipment in the target image;

[0246] Send altitude control commands and heading control commands to the drone.

[0247] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0248] Analyze the power equipment in the target image to obtain the location information of the power equipment;

[0249] Obtaining heading control parameters according to the position information of the power equipment and the center position information of the target image;

[0250] Generate heading control instructions based on heading control parameters.

[0251] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0252] Input the target image into the preset segmentation network for segmentation processing to obtain the segmented image corresponding to the target image;

[0253] The position of the electric power equipment in the segmented image is analyzed to obtain the position information of the electric power equipment.

[0254] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0255] determining a yaw angle error according to a difference between the position information of the power equipment and the center position information of the target image;

[0256] The yaw angle error is processed by the first proportional integral differential PID control algorithm to obtain the heading adjustment parameter;

[0257] The heading control parameters are determined according to the current heading parameters and heading adjustment parameters of the UAV.

[0258] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0259] The difference between the preset altitude and the altitude information is determined as the altitude error;

[0260] The height error is processed by the second proportional integral differential PID control algorithm to obtain the height control parameter;

[0261] Generate altitude control instructions based on altitude control parameters.

[0262] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0263] Receive inspection images after drone inspection;

[0264] The inspection image is recognized based on the preset recognition network to obtain the recognition result of the inspection image.

[0265] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0266] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0267] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for controlling a drone, characterized in that: The method comprises: Acquire a target image including the power equipment captured by a drone, and height information of the power equipment measured by the drone; generating an altitude control instruction according to the preset altitude and the altitude information; determining a heading control instruction based on the position information of the electric power equipment in the target image; The altitude control instruction and the heading control instruction are sent to the UAV.

2. The method according to claim 1, characterized in that The determining of the heading control instruction based on the position information of the electric power equipment in the target image includes: Analyzing the electric power equipment in the target image to obtain location information of the electric power equipment; Obtaining a heading control parameter according to the position information of the power equipment and the center position information of the target image; The heading control instruction is generated according to the heading control parameter.

3. The method according to claim 2, characterized in that The analyzing the electric power equipment in the target image to obtain the location information of the electric power equipment includes: Inputting the target image into a preset segmentation network for segmentation processing to obtain a segmented image corresponding to the target image; The position of the electric power equipment in the segmented image is analyzed to obtain the position information of the electric power equipment.

4. The method according to claim 2, characterized in that The obtaining of the heading control parameter according to the position information of the power equipment and the center position information of the target image includes: determining a yaw angle error according to a difference between the position information of the electric power equipment and the center position information of the target image; Processing the yaw angle error using a first proportional integral differential (PID) control algorithm to obtain a heading adjustment parameter; The heading control parameter is determined according to the current heading parameter of the UAV and the heading adjustment parameter.

5. The method according to any one of claims 1 to 4, characterized in that The generating of the altitude control instruction according to the preset altitude and the altitude information includes: Determine the difference between the preset height and the height information as a height error; Processing the altitude error using a second proportional-integral-differential (PID) control algorithm to obtain an altitude control parameter; The altitude control instruction is generated according to the altitude control parameter.

6. The method according to any one of claims 1 to 4, characterized in that After sending the altitude control instruction and the heading control instruction to the UAV, the method further includes: Receiving inspection images after the drone inspection; The inspection image is recognized based on a preset recognition network to obtain a recognition result of the inspection image.

7. A control device for a drone, characterized in that: The device comprises: an acquisition module, configured to acquire a target image including power equipment captured by a drone, and height information of the power equipment measured by the drone; A generating module, configured to generate a height control instruction according to a preset height and the height information; a determination module, configured to determine a heading control instruction based on the position information of the electric power equipment in the target image; A sending module is used to send the altitude control instruction and the heading control instruction to the UAV.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images