Control method and system for airborne camera equipment of unmanned aerial vehicle

Abstract

The invention discloses a control method and system for an airborne camera device of an unmanned aerial vehicle, and the method comprises the steps: obtaining image data of an airborne camera of the unmanned aerial vehicle, and carrying out the preprocessing of the image data, and obtaining first image data; a multi-task learning strategy is introduced, and loss functions of different tasks are weighed to construct a multi-task joint model; training a multi-task joint model by using the first image data to obtain a trained multi-task joint model; introducing a back propagation algorithm to update the trained multi-task joint model to obtain a first multi-task joint model, and deploying the first multi-task joint model on an edge computing platform; and acquiring real-time image data, and controlling the airborne camera of the unmanned aerial vehicle in real time by the edge computing platform according to the output result of the first multi-task joint model, thereby improving the task execution efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) control, and particularly to a method and system for controlling an on-board camera device of a UAV. Background Art

[0002] With the development of UAV technology, its application scope has been continuously expanding, and higher requirements have been put forward for the performance and intelligence level of on-board cameras. Modern on-board cameras of UAVs generally support high-resolution imaging and integrate a variety of sensors, such as infrared sensors, thermal imaging sensors, etc., which can capture light information in different bands and provide rich image data. In addition, the on-board camera of the UAV based on artificial intelligence can automatically identify and track targets, providing support for the autonomous flight and precise strike of the UAV;

[0003] However, at present, the application of UAVs in transmission lines is relatively limited. Due to the weak computing power of UAVs, processing a large amount of image data will result in too high a time delay, increase energy consumption, and reduce the endurance time. In practical applications, UAVs need to perform multiple tasks simultaneously, such as fault detection, fault analysis, maintenance personnel detection, etc. The traditional single-task model is difficult to efficiently process these tasks, resulting in low task execution efficiency. In some key tasks, such as transmission line fault detection and maintenance, it is necessary to obtain and process image data in real time so as to take timely measures. The existing processing methods are difficult to meet the real-time requirements. Therefore, a method for controlling an on-board camera device of a UAV is proposed to solve the above problems. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method and system for controlling an on-board camera device of a UAV to solve the problems that it is currently difficult to meet the real-time requirements, the traditional single-task model is difficult to efficiently process these tasks, resulting in low task execution efficiency.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for controlling an on-board camera device of a UAV, including:

[0008] Obtaining image data of the on-board camera of the UAV, and performing preprocessing to obtain first image data;

[0009] Introducing a multi-task learning strategy, weighing the loss functions of different tasks to construct a multi-task joint model;

[0010] Training the multi-task joint model with the first image data to obtain a trained multi-task joint model;

[0011] The backpropagation algorithm is introduced to update the trained multi-task joint model, obtaining the first multi-task joint model, and deploying the first multi-task joint model on the edge computing platform;

[0012] Real-time image data is obtained, and the edge computing platform controls the on-board camera of the drone in real time according to the output result of the first multi-task joint model.

[0013] As a preferred solution of the control method for the on-board camera device of the drone in the present invention, wherein: constructing the multi-task joint model by weighing the loss functions of different tasks includes:

[0014] The different tasks at least include the fault detection task, fault analysis task, and maintenance personnel detection task of the transmission line.

[0015] As a preferred solution of the control method for the on-board camera device of the drone in the present invention, wherein: the fault detection task includes:

[0016] If the fault detection result of the first time series output of the first multi-task joint model is a fault, the edge computing platform sends a first operation instruction to the drone to control the drone to perform a first operation;

[0017] After the first operation is completed, the drone sends a completion instruction of the first operation to the edge computing platform, and at this time, the output of the multi-task joint model in the edge computing platform is obtained again.

[0018] As a preferred solution of the control method for the on-board camera device of the drone in the present invention, wherein: the fault analysis task includes:

[0019] If the cause of the fault is identified in the second time series output of the first multi-task joint model;

[0020] When the cause of the fault is a human factor, the edge computing platform sends a second operation instruction to the drone to control the drone to perform a second operation;

[0021] When the cause of the fault is a non-human factor, the edge computing platform sends a third operation instruction to the drone to control the drone to perform a third operation.

[0022] As a preferred solution of the control method for the on-board camera device of the drone in the present invention, wherein: it further includes:

[0023] If the cause of the fault is not identified in the second time series output of the first multi-task joint model, the edge computing platform sends a fourth operation instruction to the drone to control the drone to perform a fourth operation.

[0024] As a preferred solution of the control method for the on-board camera device of the drone in the present invention, wherein: it further includes:

[0025] If the second temporal output of the first multi-task joint model identifies the cause of the fault, the UAV sends a first personnel arrangement instruction to the edge computing platform to arrange corresponding technicians for maintenance;

[0026] If the second temporal output of the first multi-task joint model does not identify the cause of the fault, the UAV sends a second personnel arrangement instruction to the edge computing platform to arrange technicians of at least two types for maintenance.

[0027] As a preferred solution of the control method of the UAV airborne camera device of the present invention, wherein: the maintenance personnel detection task includes:

[0028] The third temporal output of the first multi-task joint model obtains whether a technician arrives on site.

[0029] In a second aspect, the present invention provides a control system for a UAV airborne camera device, including:

[0030] A preprocessing module, configured to obtain image data of the UAV airborne camera and perform preprocessing to obtain first image data;

[0031] A model construction module, configured to introduce a multi-task learning strategy and balance loss functions of different tasks to construct a multi-task joint model;

[0032] A training module, configured to use the first image data to train the multi-task joint model to obtain a trained multi-task joint model;

[0033] A deployment module, configured to introduce a backpropagation algorithm to update the trained multi-task joint model to obtain a first multi-task joint model, and deploy the first multi-task joint model on the edge computing platform;

[0034] A control module, configured to obtain real-time image data, and the edge computing platform controls the airborne camera of the UAV in real time according to the output result of the first multi-task joint model.

[0035] In a third aspect, the present invention provides a computing device, including:

[0036] A memory and a processor;

[0037] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the control method of the UAV airborne camera device are implemented.

[0038] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for controlling an airborne camera device of a drone.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows: By introducing a multi-task joint model and an edge computing platform, the drone can automatically execute multiple tasks, such as fault detection, fault analysis, and maintenance personnel detection. The edge computing platform controls the airborne camera of the drone in real time according to the output results of the model, greatly improving the inspection efficiency and reducing manual intervention. The multi-task joint model can process multiple tasks simultaneously, avoiding the time delay and resource waste during task switching of the traditional single-task model and improving the efficiency of task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0041] Figure 1 It is a schematic diagram of the overall process logic of the method for controlling an airborne camera device of a drone according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the specific embodiments of the present invention in detail with reference to the accompanying drawings of the specification. Obviously, the described embodiments are some but not all embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0043] Embodiment 1

[0044] Referring to Figure 1 , an embodiment of the present invention provides a method for controlling an airborne camera device of a drone, including:

[0045] S100: Obtain the image data of the airborne camera of the drone, and perform preprocessing to obtain first image data;

[0046] In the embodiment of the present application, the preprocessing includes:

[0047] After decoding the image data of the UAV airborne camera, it is converted into a grayscale image. Gaussian filtering is used to remove the noise in the image data. The size of the convolution kernel of Gaussian filtering can be 3x3 or 5x5, and the standard deviation can be 1.0 or 1.5. By histogram equalization or linear transformation, the brightness and contrast of the image data are adjusted to optimize the readability of the image data and the visibility of features. The edge information in the image is enhanced through an edge detection algorithm, and the preprocessed image data is the first image data.

[0048] The edge detection algorithm can include the Sobel operator, which detects edges by calculating the gradient of the image. The Sobel operator performs convolution on the horizontal and vertical directions of the image respectively to obtain the gradient images in the horizontal and vertical directions.

[0049] It should be noted that Gaussian filtering can effectively remove the random noise in the image while retaining the edge information of the image. By selecting an appropriate convolution kernel size and standard deviation, the denoising intensity can be flexibly adjusted to ensure that the image remains clear while removing noise. Histogram equalization and linear transformation can significantly adjust the brightness and contrast of the image, making the pixel value distribution of the image more uniform and optimizing the readability of the image and the visibility of features. This helps with subsequent feature extraction and target recognition, improving the processing effect. Edge detection helps with subsequent fault detection and target recognition tasks, improving the accuracy and reliability of detection.

[0050] S200: Introduce a multi-task learning strategy, weigh the loss functions of different tasks to construct a multi-task joint model;

[0051] S300: Use the first image data to train the multi-task joint model to obtain the trained multi-task joint model;

[0052] In the embodiments of the present application, the first image data is divided into a training set and a validation set. The training set is used for model training, and the validation set is used to evaluate the model performance. Select appropriate loss functions for each task, and according to the importance or data volume of the task, weight and combine the loss functions of different tasks to obtain a joint loss function. For example, set the loss weight of the fault detection task to 0.5, the loss weight of the fault analysis task to 0.3, and the loss weight of the maintenance personnel detection task to 0.2;

[0053] Select an appropriate optimizer, set hyperparameters such as the learning rate and batch size, introduce regularization techniques to avoid overfitting, perform forward propagation on the training set, calculate the output and loss of each task, calculate the gradient of the joint loss function, and update the model parameters through the backpropagation algorithm. Periodically evaluate the model performance on the validation set, and repeat the training steps until the preset number of iterations is met.

[0054] S400: Introduce the backpropagation algorithm to update the trained multi-task joint model, obtain the first multi-task joint model, and deploy the first multi-task joint model on the edge computing platform;

[0055] Specifically, pass the input data through the forward propagation of the network, calculate the output value of each node, calculate the error between the network output and the actual label, backpropagate the error from the output layer to the input layer, calculate the gradient of each parameter using the chain rule, and update the network parameters according to the gradient and learning rate;

[0056] Save the trained multi-task joint model in a format suitable for deployment, select the edge computing platform, configure the running environment on the edge computing platform, load the saved model on the edge computing platform, input the real-time acquired image data, perform real-time inference, control the on-board camera of the drone in real time according to the model output result, quantize the model to reduce the model size and calculation amount, improve the inference speed, and adopt an asynchronous processing mechanism to improve the system's response speed and processing efficiency;

[0057] It should be noted that the preprocessed image data is used to train the multi-task joint model, and the model parameters are updated through the backpropagation algorithm. Finally, the trained model is deployed on the edge computing platform to realize the real-time control of the on-board camera of the drone, improve the training effect and inference speed of the model, and ensure the efficient operation and high-precision detection of the system, significantly improving the application effect of the drone in the power grid.

[0058] S500: Obtain real-time image data, and the edge computing platform controls the on-board camera of the drone in real time according to the output result of the first multi-task joint model.

[0059] It should be noted that offloading the computing task from the drone to the edge server reduces the computing burden of the drone, reduces latency and energy consumption, increases the battery life, the edge computing platform can quickly process a large amount of data, provide real-time analysis results, support efficient inspection tasks, update the trained multi-task joint model through the backpropagation algorithm, ensure the accuracy and reliability of the model, and improve the efficiency and quality of data processing.

[0060] In the embodiment of the present application, the above step S200 includes the following sub-step A1;

[0061] In A1: Different tasks at least include the fault detection task, fault analysis task, and maintenance personnel detection task of the transmission line.

[0062] Specifically, different tasks can also include environmental monitoring tasks, supervisor tasks, emergency rescue tasks, etc.;

[0063] It should be noted that the combination of the multi-task joint model and the edge computing platform enables the UAV to automatically execute multiple tasks, reducing manual intervention and improving the inspection efficiency.

[0064] In the embodiment of the present application, the above step S500 includes the following sub-steps B1 - B2;

[0065] In B1: If the fault detection result of the first temporal output of the first multi-task joint model indicates a fault, the edge computing platform sends a first operation instruction to the UAV to control the UAV to perform a first operation;

[0066] In B2: After the first operation is completed, the UAV sends a completion instruction of the first operation to the edge computing platform. At this time, the output of the multi-task joint model in the edge computing platform is retrieved again.

[0067] Specifically, the first operation is any type of operation that makes the UAV approach the fault point or the fault target;

[0068] In an optional embodiment, the first operation may include, when the UAV approaches the fault point, hovering near the fault point. In the hovering state, the UAV observes the position and proportion of the fault point in the image data in real time through the on-board camera, and makes fine adjustments to the UAV, such as moving forward and backward, left and right, up and down, and adjusting the pitch angle and yaw angle of the camera, so that the fault point is within the set area of the image data;

[0069] The first operation may also include that the UAV obtains the accurate position information of itself and the fault point through the global positioning system, calculates the distance and azimuth angle between the two, and the UAV autonomously plans a flight path and flies along the route to find the fault point;

[0070] In the embodiment of the present application, the first operation includes sending a trajectory instruction to the UAV to adjust the UAV directly above the fault, so that the fault point or the fault target is located at the center of the image data of the UAV's on-board camera, and the fault point or the fault target occupies 70% - 80% of the image data area of the UAV's on-board camera;

[0071] The first timing is the representation type indicating whether there is a fault when the first multi-task joint model performs the fault detection task;

[0072] In an optional embodiment, the first timing may include that if the fault detection result of the 1st order output of the first multi-task joint model is "1", there is a fault, and if the output fault detection result is "0", there is no fault;

[0073] In an optional embodiment, the first timing may also include that if the fault detection result of the A-order output of the first multi-task joint model is "A", there is a fault, and if the output fault detection result is "a", there is no fault;

[0074] In the embodiment of the present application, the first timing sequence is "fault_detect_i", where i is any upper or lower case letter. If the fault detection result output by the fault_detect timing sequence of the first multi-task joint model is "fault_detect_A", then there is a fault; if the output fault detection result is "fault_detect_a", then there is no fault.

[0075] It should be noted that the precise positioning method greatly reduces the time for the drone to hover over the target area to find the fault point, improves the efficiency of fault detection, helps to improve the accuracy and reliability of the fault detection algorithm, reduces the situations of misjudgment and missed judgment, and can more accurately identify the type and degree of the fault.

[0076] In the embodiment of the present application, after completing steps B1 - B2 in the above step S500, the following steps B3 - B8 are further included;

[0077] In B3: If the second timing sequence output of the first multi-task joint model identifies the cause of the fault;

[0078] In B4: When the cause of the fault is a human factor, the edge computing platform sends a second operation instruction to the drone to control the drone to perform a second operation;

[0079] In B5: When the cause of the fault is a non-human factor, the edge computing platform sends a third operation instruction to the drone to control the drone to perform a third operation;

[0080] In B6: If the second timing sequence output of the first multi-task joint model does not identify the cause of the fault, the edge computing platform sends a fourth operation instruction to the drone to control the drone to perform a fourth operation;

[0081] In B7: If the second timing sequence output of the first multi-task joint model identifies the cause of the fault, the drone sends a first personnel arrangement instruction to the edge computing platform to arrange the corresponding technical personnel for maintenance;

[0082] In B8: If the second timing sequence output of the first multi-task joint model does not identify the cause of the fault, the drone sends a second personnel arrangement instruction to the edge computing platform to arrange at least two types of technical personnel for maintenance.

[0083] Specifically, the second timing sequence is the representation type for outputting the cause of the fault when the first multi-task joint model performs the fault analysis task;

[0084] In an alternative embodiment, the second timing sequence may include that if the 2 timing sequence output of the first multi-task joint model identifies the cause of the fault as "2", then the cause of the fault is a human factor; if the output identifies the cause of the fault as "0", then it is a non-human cause;

[0085] In an alternative embodiment, the second time sequence may further include that if the fault cause identified by the B time sequence output of the first multitask joint model is "B", the fault cause is a human factor; if the identified fault cause is "b", it is a non-human cause.

[0086] In the embodiment of the present application, the second time sequence is "FR_i", where i is any upper or lower case letter; if the fault cause output by the FR_i time sequence is "FR_B", the fault cause is a human factor; if the identified fault cause is "FR_b", it is a non-human cause.

[0087] Specifically, the second operation is any type of operation that enables the drone to perform personnel tracking or personnel detection.

[0088] In an alternative embodiment, the drone takes off to an initially set height, sets the rotational angular velocity to search for a target, the binocular camera acquires a video stream, the on-board processor detects the target object through the input frame, and when the target is located and locked, the on-board computer calculates the relative pose between the two. The pose information of the current frame is input to obtain a drone control instruction, and it is determined whether the drone enters the "tracking" mode. In the tracking mode, the drone determines the next maneuvering behavior based on the relative distance between the drone and the target object obtained from the depth data.

[0089] In the embodiment of the present application, the second operation includes controlling the drone to take off, ensuring it hovers stably at a preset height, flying the drone near the target area, performing a preliminary scan to find the target person. Once the target person is found, controlling the drone to stay within the visual range of the target person, adjusting the flight direction and speed so that the target person is always at the center of the camera. According to the moving direction and speed of the target person, the flight attitude of the drone is adjusted in real time to ensure that the target person does not leave the visual field of the camera, maintaining a distance of 10 - 20 meters from the target person, and ensuring that the image of the target person in the camera is clearly visible by adjusting the height and horizontal position of the drone.

[0090] Specifically, the third operation is any type of operation that enables the drone to perform detailed shooting of the fault point.

[0091] In an alternative embodiment, the third operation may include sending a shooting instruction to the drone, causing the drone to hover above the fault point, and adjusting the shooting parameters of the on-board camera, such as focal length, exposure, etc., to obtain clearer and more detailed image data of the fault point.

[0092] In the embodiment of the present application, the third operation includes flying the drone above the fault area so that the drone hovers above the fault point, adjusting the pitch angle and yaw angle of the drone to align the camera with the fault point, ensuring that the field of view of the camera can cover the fault area, capturing details of the fault point from different angles and distances, and slowly moving the drone to take pictures of the fault point from different positions;

[0093] Specifically, the fourth operation is any type of operation in which the drone takes off again when no fault cause is found and continues to search for the fault cause;

[0094] In an alternative embodiment, the fourth operation may include the drone returning to a height of 10 - 20 meters, flying the drone to the periphery of the fault area to expand the search range, adopting a spiral flight path to gradually expand the search area, and adjusting the pitch angle and yaw angle of the drone to take pictures of the area from different angles to cover all possible fault points. The fourth operation may also include re-checking key equipment to eliminate the fault cause;

[0095] In an alternative embodiment, the fourth operation may also include lowering the flight height of the drone to 5 - 10 meters, flying at a slower speed to ensure that the camera can clearly capture details on the ground, and focusing on checking possible fault areas such as around equipment and connection points according to previous flight experience and observed situations.

[0096] In the embodiment of the present application, the fourth operation includes the drone returning to the initial take-off position, dividing the fault area into several small areas, flying to the center point of each small area one by one according to a preset order for fixed-point hovering and shooting. At each point, adjust the angle of the drone to be able to comprehensively cover the ground conditions of the area. At the ground control station, record the images and videos of each inspection point, mark any suspicious or abnormal areas, and there should be a certain overlapping area between adjacent inspection points to avoid incomplete coverage caused by shooting angle or drone position deviation;

[0097] It should be noted that through the second operation, the third operation, and the fourth operation, not only the accuracy and efficiency of task execution are improved, but also the flexibility and adaptability of the operation are enhanced, the data quality and analysis accuracy are improved, the safety and compliance of the operation are guaranteed, and at the same time, the operation difficulty and training cost are reduced, which has significant practical value and application prospects.

[0098] In an alternative embodiment, the personnel arrangement instruction can directly display the required type and quantity of technicians. For example, if the second timing output of the first multi-task joint model identifies that 1 technician is required to find the fault cause, it will display the required technician type: 1;

[0099] In an alternative embodiment, the first personnel arrangement instruction and the second personnel arrangement instruction are described numerically. The first personnel arrangement instruction is expressed as: 1(F, T, N), where 1 represents the first personnel arrangement instruction, F represents the type of fault, T represents the type of technical personnel required, and N represents the number of technical personnel required; the second personnel arrangement instruction is expressed as: 2(T 1 , N 1 , T 2 , N 2 … T k , N k );

[0100] In the embodiment of the present application, the first personnel arrangement instruction is P 1 (T, N), and the second personnel arrangement instruction is P 2 (T 1 , N 1 , T 2 , N 2 … T k , N k ), where T represents the type of technical personnel required and N represents the number of technical personnel required;

[0101] Exemplarily, if the cause of the fault is an electrical short circuit and 1 electrical engineer is required for repair, it can be expressed as: P 1 (electrical short circuit, electrical engineer, 1);

[0102] If the cause of the fault is not identified and at least 1 electrical engineer and 1 mechanical engineer are required for repair, it is expressed as: P 2 (electrical engineer, 1, mechanical engineer, 1)

[0103] It should be noted that the clear instruction format enables the edge computing platform to quickly understand and execute the personnel arrangement task, reduces the delay of task allocation, and through precise task allocation, ensures that each type of technical personnel can be reasonably utilized, avoiding resource waste. When the cause of the fault is not identified, the collaborative work of multiple types of technical personnel can improve the reliability of fault diagnosis and handling. This operation not only improves the accuracy and efficiency of task execution, but also enhances the systematicness and reliability of task management, simplifies the operation process, and improves the resource utilization efficiency.

[0104] In the embodiment of the present application, after completing steps B3 - B8 in the above step S500, the following step B9 is further included;

[0105] In B9: The third timing output of the first multi - task joint model is used to obtain whether a technical personnel arrives on the scene.

[0106] Specifically, if it is detected that a technical personnel comes over, the drone receives the next instruction and returns to the origin to resume the inspection;

[0107] If no technician is detected within the set time, it is determined whether the scope of the fault has expanded. If it has expanded, the alarm level is adjusted, and the buzzer of the drone is sounded to notify the nearby inspection personnel to conduct an inspection.

[0108] It should be noted that the drone can not only improve the efficiency and reliability of task execution, but also optimize resource allocation, enhance the flexibility and adaptability of the system, and improve the automation and intelligence level of the system.

[0109] The above is a schematic solution of a control method for a drone-borne camera device in this embodiment. It should be noted that the technical solution of the control system of the drone-borne camera device belongs to the same concept as the technical solution of the control method of the drone-borne camera device described above. For the details not described in the technical solution of the control system of the drone-borne camera device in this embodiment, reference can be made to the description of the technical solution of the control method of the drone-borne camera device described above.

[0110] The control system of the drone-borne camera device in this embodiment includes:

[0111] A preprocessing module for acquiring the image data of the drone-borne camera and performing preprocessing to obtain first image data;

[0112] A model construction module for introducing a multi-task learning strategy and constructing a multi-task joint model by weighing the loss functions of different tasks;

[0113] A training module for training the multi-task joint model with the first image data to obtain a trained multi-task joint model;

[0114] A deployment module for introducing a backpropagation algorithm to update the trained multi-task joint model to obtain a first multi-task joint model and deploying the first multi-task joint model on an edge computing platform;

[0115] A control module for acquiring real-time image data, and the edge computing platform controls the drone-borne camera in real time according to the output result of the first multi-task joint model.

[0116] This embodiment also provides a computing device applicable to the control of a drone-borne camera device, including:

[0117] A memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the control method for the drone-borne camera device as proposed in the above embodiment.

[0118] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for controlling an airborne camera device of a drone as proposed in the above embodiment.

[0119] The storage medium proposed in this embodiment and the method for controlling an airborne camera device of a drone proposed in the above embodiment belong to the same inventive concept. For technical details not described in detail in this embodiment, reference can be made to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software and necessary general-purpose hardware, and of course, it can also be implemented by hardware. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk, or optical disc of a computer, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

Claims

1. A method for controlling a camera device mounted on a drone, characterized in that: include: Acquire image data from a camera mounted on the drone and perform preprocessing to obtain first image data; Introducing a multi-task learning strategy, weighing the loss functions of different tasks to build a multi-task joint model; Using the first image data to train the multi-task joint model to obtain a trained multi-task joint model; Introducing a back-propagation algorithm to update the trained multi-task joint model to obtain a first multi-task joint model, and deploying the first multi-task joint model on an edge computing platform; Real-time image data is acquired, and the edge computing platform controls the onboard camera of the drone in real time according to the output result of the first multi-task joint model.

2. The method for controlling the drone-mounted camera device according to claim 1, characterized in that: The loss functions of weighing different tasks to construct a multi-task joint model include: The different tasks include at least a transmission line fault detection task, a fault analysis task, and a maintenance personnel detection task.

3. The method for controlling the drone-mounted camera device according to claim 2, characterized in that: Fault detection tasks include: If the fault detection result of the first time sequence output of the first multi-task joint model is faulty, the edge computing platform sends a first operation instruction to the drone to control the drone to perform the first operation; After completing the first operation, the drone sends a completion instruction of the first operation to the edge computing platform, and then reacquires the output of the multi-task joint model in the edge computing platform.

4. The method for controlling the drone-mounted camera device according to claim 3, characterized in that: Fault analysis tasks include: If the second timing output of the first multi-task joint model identifies the cause of the fault; When the cause of the failure is human factors, the edge computing platform sends a second operation instruction to the drone to control the drone to perform the second operation; When the cause of the fault is non-human factors, the edge computing platform sends a third operation instruction to the drone to control the drone to perform the third operation.

5. The method for controlling the camera device onboard a drone as claimed in claim 4, characterized in that: Also includes: If the second timing output of the first multi-task joint model fails to identify the cause of the fault, the edge computing platform sends a fourth operation instruction to the drone to control the drone to perform a fourth operation.

6. The method for controlling the drone-mounted camera device according to claim 4 or 5, characterized in that: Also includes: If the second time series output of the first multi-task joint model identifies the cause of the fault, the UAV sends a first personnel arrangement instruction to the edge computing platform to arrange corresponding technicians to perform maintenance; If the second time sequence output of the first multi-task joint model does not identify the cause of the fault, the drone sends a second personnel arrangement instruction to the edge computing platform to arrange at least two types of technicians to perform maintenance.

7. The method for controlling the camera device onboard a drone as claimed in claim 6, characterized in that: Maintenance personnel inspection tasks include: The third time series output of the first multi-task joint model is whether the presence of a technician is detected.

8. A control system for a drone airborne camera device according to any one of claims 1 to 7, characterized in that: A preprocessing module is used to obtain image data from the camera on the drone and perform preprocessing to obtain first image data; Model building module, which is used to introduce multi-task learning strategies and weigh the loss functions of different tasks to build a multi-task joint model; A training module, used to train the multi-task joint model using the first image data to obtain a trained multi-task joint model; A deployment module, used to introduce a back-propagation algorithm to update the trained multi-task joint model, obtain a first multi-task joint model, and deploy the first multi-task joint model on an edge computing platform; The control module is used to obtain real-time image data, and the edge computing platform controls the drone's airborne camera in real time according to the output result of the first multi-task joint model.

9. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for controlling the drone-mounted camera device according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for controlling the drone-mounted camera device as described in any one of claims 1 to 7.

Images