Unmanned aerial vehicle low-altitude data acquisition method and device, computer equipment, readable storage medium and program product
By equiping image acquisition equipment and recognition models on the drone, identifying and following the power grid equipment for image acquisition, the problem of low data acquisition efficiency of drone is solved, and efficient data acquisition of power grid equipment is achieved.
Patent Information
- Application Number
- CN202411323605.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-12-03
AI Technical Summary
The existing drones have low data acquisition efficiency in power grid systems, and each device needs to be docked at a low altitude and photographed, resulting in a decrease in acquisition efficiency.
The drone is equipped with a first image acquisition device and a second image acquisition device. By identifying the model, it recognizes whether there is a preset target in the area to be collected, controls the viewing angle of the second image acquisition device to follow the target object, and acquires the target image during flight. , improve data collection efficiency.
Through the cooperation of image acquisition and recognition models during drone flight, efficient data acquisition of power grid equipment is achieved, and the acquisition efficiency is improved, and the inefficiency problem of each device having to dock and shoot in traditional methods is avoided.
Smart Images

Figure CN119071445A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for collecting low-altitude data of unmanned aerial vehicles. Background Art
[0002] With the development of computer technology, drone technology has emerged. Users can remotely control the flight of drones. Drones are usually equipped with cameras, and users can control drones to collect image data using the cameras. In the power grid system, there are many types of power equipment. When using drones to inspect the power grid system, it is necessary to use drones to collect data from various power grid equipment. At present, the way drones collect data from power grid systems is usually to dock at low altitudes at the location of each device and take pictures to achieve data collection.
[0003] However, there are many devices in the power grid, and the collection process of low-altitude parking and taking pictures of each device will lead to a decrease in data collection efficiency.
[0004] Therefore, the current method of using drones to collect data has the defect of low collection efficiency. Summary of the invention
[0005] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for collecting low-altitude data of unmanned aerial vehicles that can improve the collection efficiency in response to the above-mentioned technical problems.
[0006] In a first aspect, the present application provides a method for collecting low-altitude data of a drone, which is applied to a drone, wherein a first image acquisition device and a second image acquisition device are provided in the drone, and the method comprises:
[0007] Acquire the image to be identified captured by the first image acquisition device for the area to be acquired during flight;
[0008] Input the image to be identified into a trained recognition model, and let the recognition model identify whether there is an image corresponding to a preset target object in the image to be identified, and output a corresponding recognition result;
[0009] If the recognition result is that there is an image corresponding to the preset target object, the viewing angle of the second image acquisition device is controlled to follow the preset target object in the area to be acquired, and the target image corresponding to the preset target object is acquired by the second image acquisition device.
[0010] In one embodiment, the step of acquiring the image to be identified captured by the first image acquisition device for the area to be acquired during flight includes:
[0011] When a flight start instruction is detected, adjusting the acquisition direction of the first image acquisition device to match the flight direction;
[0012] The image to be identified acquired by the adjusted first image acquisition device for the area to be acquired is acquired.
[0013] In one embodiment, the step of acquiring the image to be identified captured by the first image acquisition device for the area to be acquired during flight includes:
[0014] Adjusting the field of view of the first image acquisition device to a first field of view, and acquiring the image to be recognized acquired by the first image acquisition device based on the first field of view of the area to be acquired during flight;
[0015] The step of acquiring the target image corresponding to the preset target object by the second image acquisition device includes:
[0016] Adjusting the field of view of the second image acquisition device to a second field of view, and acquiring a target image corresponding to the preset target object based on the second field of view by the second image acquisition device;
[0017] Wherein, the first field of view is larger than the second field of view.
[0018] In one embodiment, a rotation drive mechanism is provided in the drone, and the second image acquisition device is connected to the rotation drive mechanism;
[0019] If the recognition result is that there is an image corresponding to the preset target object, controlling the viewing angle of the second image acquisition device to follow the preset target object in the area to be acquired during the flight includes:
[0020] If the recognition result is that there is an image corresponding to the preset target object, acquiring acceleration information of the drone;
[0021] Determining the rotational power corresponding to the rotational drive mechanism according to the acceleration information;
[0022] The viewing angle of the second image acquisition device is controlled according to the rotational power to follow the preset target object in the area to be acquired.
[0023] In one embodiment, determining the rotational power corresponding to the rotational drive mechanism according to the acceleration information includes:
[0024] If the acceleration information is greater than a preset threshold, the rotational power corresponding to the rotational drive mechanism is increased according to the acceleration information to obtain the increased rotational power;
[0025] or,
[0026] If the acceleration information is less than or equal to the preset threshold, the rotational power corresponding to the rotational drive mechanism is reduced according to the acceleration information to obtain the reduced rotational power.
[0027] In one embodiment, the method further comprises:
[0028] Obtaining a recognition model to be trained, image samples to be recognized, and preset target object samples;
[0029] Inputting the image sample to be identified into the recognition model to be trained, the recognition model identifies whether the image sample to be identified has an image corresponding to the preset target object sample, and outputs corresponding recognition result prediction information;
[0030] If the recognition result prediction information indicates that there is an image corresponding to the preset target object sample, obtaining a comparison result between the image corresponding to the preset target object sample recognized by the recognition model and the preset target object sample;
[0031] The model parameters of the recognition model are adjusted according to the comparison result until a preset training end condition is met, thereby obtaining the trained recognition model.
[0032] In a second aspect, the present application provides a low-altitude data acquisition device for a drone, which is applied to a drone, wherein a first image acquisition device and a second image acquisition device are provided in the drone, and the device comprises:
[0033] An acquisition module, used for acquiring the image to be identified acquired by the first image acquisition device for the area to be acquired during flight;
[0034] A recognition module, used to input the image to be recognized into a trained recognition model, and the recognition model recognizes whether there is an image corresponding to a preset target object in the image to be recognized, and outputs a corresponding recognition result;
[0035] The acquisition module is used to control the viewing angle of the second image acquisition device to follow the preset target object in the area to be acquired if the recognition result is that there is an image corresponding to the preset target object, and to acquire the target image corresponding to the preset target object through the second image acquisition device.
[0036] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.
[0037] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.
[0038] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.
[0039] The above-mentioned method, device, computer equipment, computer-readable storage medium and computer program product for collecting low-altitude data of unmanned aerial vehicles use the first image collection device to collect the image to be identified when the unmanned aerial vehicle is flying, identify whether there is a preset target object in the image to be identified, and if so, control the perspective of the second image collection device to follow the preset target object in the area to be collected during flight, and collect the target image corresponding to the preset target object through the second image collection device. Compared with the traditional method of docking and shooting at each device, this solution uses two image collection devices in the unmanned aerial vehicle to respectively realize the identification of the preset target object and the perspective tracking and image collection of the preset target object during the flight of the unmanned aerial vehicle, thereby improving the efficiency of data collection for power grid equipment. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 A schematic diagram of a process for collecting low-altitude data of a drone in one embodiment;
[0042] Figure 2 A schematic diagram of a flow chart of a method for collecting low-altitude data of a drone in another embodiment;
[0043] Figure 3 It is a structural block diagram of a low-altitude data acquisition device for a UAV in one embodiment;
[0044] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0046] In one embodiment, Figure 1 As shown, a method for collecting low-altitude data of a drone is provided. This embodiment uses the method applied to a drone as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a drone and a server, and is implemented through the interaction between the drone and the server, including the following steps S202 to S206. Among them:
[0047] Step S202: acquiring the image to be identified captured by the first image acquisition device for the area to be captured during flight.
[0048] Among them, the drone can also be called an unmanned aircraft, which is an unmanned aircraft controlled by a radio remote control device and a self-contained program control device. From a technical point of view, it can be divided into: unmanned helicopters, unmanned fixed-wing aircraft, unmanned multi-rotor aircraft, unmanned airships, unmanned paragliders, etc.; with the development of science and technology, drones have gradually been widely used in the civilian field. In this embodiment, a first image acquisition device and a second image acquisition device are provided in the drone. Specifically, two rotating drive mechanisms are also provided in the drone, and the rotating drive mechanisms are installed on the lower surface of the drone or on one side of the forward direction of the drone through a mounting seat. The first image acquisition device and the second image acquisition device are respectively connected to each rotating drive mechanism to achieve installation on the drone and connection with the drone.
[0049] The first image acquisition device may be a camera with an image recognition function, and the second image acquisition device may be a conventional camera. When the drone inspects the power grid area, it is necessary to acquire images of the equipment in the power grid. To collect data on each power device in the power grid, the drone may fly within a preset height threshold, i.e., fly at low altitude, and use the first image acquisition device and the second image acquisition device to acquire images of each power device in the power grid during the flight.
[0050] When the drone is flying, it can obtain the image to be identified that is captured by the first image acquisition device for the area to be collected during flight. The area to be collected can be the area of the field of view covered by the first image acquisition device in the area where the power grid is located, and the image to be identified can be an image that needs to be identified. The drone can identify whether the image to be identified contains a preset target object. The preset target object can be various power equipment in the pre-set power grid. The above-mentioned first image acquisition device can be in a moving state, so that the drone can adjust the posture of the first image acquisition device during flight to better collect and identify images of the power grid area.
[0051] Step S204, inputting the above-mentioned image to be recognized into a trained recognition model, allowing the above-mentioned recognition model to recognize whether there is an image corresponding to a preset target object in the above-mentioned image to be recognized, and outputting a corresponding recognition result.
[0052] The drone may have a built-in trained recognition model. For example, the drone may pre-train the recognition model to be recognized. After each iteration, the model parameters of the recognition model are adjusted based on the output result. When the preset training end condition is met, the trained recognition model is obtained. The above recognition model may be a deep learning neural network model.
[0053] Among them, the drone can use a variety of preset target object samples to identify the recognition model in advance, for example, obtain images of various power equipment in the power grid input by relevant managers as preset target object samples to train the recognition model, or obtain personnel who need to be searched and rescued at the power grid site input by relevant managers as preset target object samples to train the recognition model, and then the above-mentioned recognition model can realize the timely recognition of images of various power equipment or personnel in the power grid.
[0054] The above-mentioned trained recognition model can realize the recognition of multiple preset targets. Then, the drone can input the image to be recognized acquired by the first image acquisition device into the above-mentioned trained recognition model. The recognition model can recognize the preset target based on the image to be recognized, that is, the recognition model can recognize whether there is an image corresponding to the preset target in the image to be recognized. For example, for each preset target, the recognition model recognizes whether there is an image corresponding to the preset target in the image to be recognized, and then outputs the corresponding recognition result. Among them, the recognition result includes images corresponding to the preset target and images corresponding to the preset target.
[0055] In addition, in some embodiments, the above-mentioned preset targets may also be pre-specified by relevant staff, for example, a preset number of preset targets may be determined from multiple preset target samples that have participated in training as preset targets that the recognition model needs to identify from the image to be identified.
[0056] Step S206, if the recognition result is that there is an image corresponding to the preset target object, control the viewing angle of the second image acquisition device to follow the preset target object in the area to be acquired, and acquire the target image corresponding to the preset target object through the second image acquisition device.
[0057] The above-mentioned recognition model can output a corresponding recognition result based on the image to be recognized collected by the first image acquisition device. The drone can perform different processing for different output results. For example, if the recognition result is that there is no image corresponding to the preset target object, the drone can continue to control the first image acquisition device to collect the image to be recognized, and recognize the preset target object for the new image to be recognized.
[0058] If the drone detects that the recognition result is an image corresponding to a preset target object, the drone can control the viewing angle of the second image acquisition device to follow the preset target object in the area to be collected. The second image acquisition device can be in motion, so that the drone can control the viewing angle of the second image acquisition device to aim at the preset target object. For example, the drone can achieve the following and locking of the viewing angle of the second image acquisition device by rotating, so that the drone can collect the target image corresponding to the preset target object through the second image acquisition device, and realize the image collection of the preset target object in the area to be collected. The drone can also send the collected target image to the management end of the relevant management personnel, so that the management end can analyze the preset target object based on the target image, such as detecting whether there is an abnormality in the preset target object, or determining the geographical location of the preset target object.
[0059] In the above-mentioned drone low-altitude data collection method, the first image acquisition device is used to collect the image to be identified when the drone is flying, and it is identified whether there is a preset target object in the image to be identified. If there is, the second image acquisition device is controlled to follow the preset target object in the area to be collected during flight, and the target image corresponding to the preset target object is collected by the second image acquisition device. Compared with the traditional method of docking and shooting at each device, this solution uses two image acquisition devices in the drone to respectively realize the identification of the preset target object and the perspective tracking and image acquisition of the preset target object during the flight of the drone, thereby improving the efficiency of data collection for power grid equipment.
[0060] In one embodiment, obtaining the image to be identified captured by the first image acquisition device for the area to be acquired during flight includes: when a flight start instruction is detected, adjusting the acquisition direction of the first image acquisition device to match the flight direction; and obtaining the image to be identified captured by the adjusted first image acquisition device for the area to be acquired.
[0061] In this embodiment, the drone can fly in response to the flight start instruction of the control device. The flight start instruction can be sent to the drone by the control device through wireless communication. The drone can be provided with a communication module. After the drone receives the flight start instruction through the communication module, it can start the flight based on the flight start instruction. At this time, the drone can adjust the direction of the first image acquisition device and the second image acquisition device. Among them, the first image acquisition device and the second image acquisition device are both devices that can rotate. As the drone flies, the first image acquisition device and the second image acquisition device can move with the flight action of the drone. During the flight of the drone, the first image acquisition device and the second image acquisition device can both be facing the forward direction of the drone in the initial state. Then, the drone can adjust the acquisition direction of the first image acquisition device to match the flight direction of the drone. Among them, the acquisition direction of the first image acquisition device can represent the direction in which the perspective of the first image acquisition device points. In addition, in some embodiments, when the drone is flying, the acquisition direction of the second image acquisition device in the initial state can also be pointing to the direction of the drone's flight.
[0062] After the drone adjusts the acquisition direction of the first image acquisition device, the adjusted first image acquisition device can be obtained, so that the drone can obtain the image to be identified acquired by the adjusted first image acquisition device for the area to be acquired. Specifically, the first image acquisition device can be a camera with an image recognition function, and the second image acquisition device can be a conventional camera. As the drone flies, the first image acquisition device and the second image acquisition device move accordingly. In the initial state, the first image acquisition device and the second image acquisition device are both facing the forward direction of the drone, and the first image acquisition device can be in operation, and acquire the image to be identified for the area to be acquired at a low altitude below the drone.
[0063] Through this embodiment, the drone can adjust the initial state of the first image acquisition device when taking off to match the flight direction of the drone, so that it can more quickly start the image acquisition process of the area to be acquired, thereby improving the efficiency of the drone in acquiring image data of the power grid area.
[0064] In one embodiment, obtaining the image to be identified that is captured by the first image acquisition device during flight for the area to be acquired includes: adjusting the field of view of the first image acquisition device to a first field of view, and obtaining the image to be identified that is captured by the first image acquisition device during flight for the area to be acquired based on the first field of view.
[0065] In this embodiment, the field of view of the first image acquisition device and the field of view of the second image acquisition device may be different. For example, when the drone is flying, the field of view of the first image acquisition device may be adjusted to the first field of view, and when the drone is flying, the image to be identified acquired by the first image acquisition device based on the first field of view of the area to be acquired. For the second image acquisition device, the drone may use a different field of view to further acquire images of the preset target object.
[0066] In one embodiment, acquiring the target image corresponding to the preset target object by the second image acquisition device includes: adjusting the field of view of the second image acquisition device to a second field of view, and acquiring the target image corresponding to the preset target object by the second image acquisition device based on the second field of view; wherein the first field of view is larger than the second field of view.
[0067] In this embodiment, the field of view of the second image acquisition device may be different from the field of view of the first image acquisition device. After the drone recognizes the image of the preset target object using the image to be recognized acquired by the first image acquisition device, the second image acquisition device may be used to further acquire the image of the preset target object. For example, the drone may adjust the field of view of the second image acquisition device to the second field of view, and after determining that there is an image corresponding to the preset target object in the image to be recognized, the second image acquisition device may acquire the target image corresponding to the preset target object based on the second field of view. The second image acquisition device may acquire the target image of the preset target object during the flight of the drone.
[0068] Among them, the first field of view can be larger than the second field of view, so that the drone can collect the image to be identified with a wider viewing angle through the first image acquisition device, and then quickly detect whether the image of the preset target object is contained in the image to be identified with a wider viewing angle. After detecting the image of the preset target object, the drone can collect a clearer target image by aiming at the preset target object through the second image acquisition device with a narrower field of view during the flight of the drone. Specifically, taking the first image acquisition device as the first camera and the second image acquisition device as the second camera as an example, the first field of view corresponding to the first image acquisition device can be the first focal length, that is, the drone can adjust the focal length of the first image acquisition device to the first focal length; the second field of view corresponding to the second image acquisition device can be the second focal length, and the drone can adjust the focal length of the second image acquisition device to the second focal length. Among them, the first focal length can be a wide-angle focal length, and the second focal length can be a telephoto focal length, so that the first field of view is wider than the second field of view, so that the image to be identified collected by the first image acquisition device can contain more information, and the target image collected by the second image acquisition device can more clearly display the target preset object.
[0069] Through this embodiment, the drone can set different focal lengths for the first image acquisition device and the second image acquisition device respectively, so that the image to be identified acquired by the first image acquisition device contains more information, and can quickly identify whether it contains an image corresponding to a preset target object. The target image acquired by the second image acquisition device more clearly displays the preset target object, thereby improving the efficiency of the drone in collecting image data of the power grid area.
[0070] In one embodiment, if the above-mentioned recognition result is that there is an image corresponding to the above-mentioned preset target object, then during the above-mentioned flight, the viewing angle of the above-mentioned second image acquisition device is controlled to follow the above-mentioned preset target object in the above-mentioned area to be acquired, including: if the above-mentioned recognition result is that there is an image corresponding to the above-mentioned preset target object, then the acceleration information of the above-mentioned UAV is obtained; the rotational power corresponding to the above-mentioned rotational drive mechanism is determined according to the above-mentioned acceleration information; and the viewing angle of the above-mentioned second image acquisition device is controlled to follow the above-mentioned preset target object in the above-mentioned area to be acquired according to the above-mentioned rotational power.
[0071] In this embodiment, a rotary drive mechanism is provided in the drone, the second image acquisition device is connected to the rotary drive mechanism, and the first image acquisition device can also be connected to the rotary drive mechanism. The drone can identify the preset target object for the image to be identified acquired by the first image acquisition device through the recognition model. If the drone determines through the recognition result that there is an image corresponding to the preset target object in the image acquired by the first image acquisition device, the drone can control the second image acquisition device through the rotary drive mechanism, and aim at the preset target object in the area to be collected during the flight of the drone to collect images. For example, the drone can obtain the acceleration information of the drone through the acceleration sensor, and determine the rotational power corresponding to the rotary drive mechanism according to the acceleration information, that is, the drone needs to determine the rotational power required to control the second image acquisition device to aim at the preset target object based on the flight acceleration of the drone.
[0072] In one embodiment, the rotational power corresponding to the rotational drive mechanism is determined based on the acceleration information, including: if the acceleration information is greater than a preset threshold, the rotational power corresponding to the rotational drive mechanism is increased according to the acceleration information to obtain the increased rotational power; or, if the acceleration information is less than or equal to the preset threshold, the rotational power corresponding to the rotational drive mechanism is reduced according to the acceleration information to obtain the reduced rotational power.
[0073] In this embodiment, for acceleration information of different magnitudes, the drone can provide different rotational powers to the rotation drive mechanism. For example, the drone can pre-set a preset threshold corresponding to the acceleration information, which can be zero. When the drone detects that the acceleration is greater than the preset threshold, it means that the drone is accelerating. The drone can increase the rotational power corresponding to the rotation drive mechanism according to the acceleration information to obtain the increased rotational power. When the drone detects that the acceleration is less than the preset threshold, it means that the drone is decelerating. The drone can reduce the rotational power corresponding to the rotation drive mechanism according to the acceleration information to obtain the reduced rotational power.
[0074] That is, the drone can control the rotation drive mechanism to aim at the preset target in the area to be captured according to the position of the preset target, and output the corresponding rotational power based on the flight acceleration of the drone, increase the rotational power when the drone accelerates, and reduce the rotational power when the drone decelerates, so that the second image capture device maintains the same viewing angle aimed at the preset target. Therefore, the drone can control the viewing angle of the second image capture device to follow the preset target in the area to be captured according to the above-mentioned output rotational power, and capture the target image.
[0075] Specifically, the drone performs the above steps through a processor. The signal output ends of the first image acquisition device and the second image acquisition device are connected to the processor, and the output end of the processor is connected to the communication module. The drone receives the image data transmitted by the first image acquisition device and the second image acquisition device using the communication module through the processor. The output end of the processor in the drone is connected to the rotating drive mechanism. Among them, the processor can be processed by PLC (Programmable Logic Controller), for example, it can be a small programmable controller with an integrated 24V load power supply. The processor can be directly connected to sensors and transmitters (actuators), etc., including CPU (Central Processing Unit) 221, CPU 222, CPU 224, CPU 224XP and CPU 226. Among them, CPU221 and 222 have 180mA output, CPU 224, CPU 224XP, CPU 226 output 280 and 400mA respectively, which can be used as load power for different types of equipment. CPU 221~226 each has two types of CPUs with different power supply voltages and control voltages.
[0076] Among them, each of the above-mentioned CPUs corresponds to digital input and output points and analog input and output points respectively. For example, for digital input and output points, CPU 221 has 6 input points and 4 output points, CPU 222 has 8 input points and 6 output points, CPU 224 has 14 input points and 10 output points, CPU 224XP has 14 input points and 10 output points, and CPU 226 has 24 input points and 16 output points. For analog input and output points, CPU 224XP has 2 input points and 1 output point. The above-mentioned processor can also perform interrupt input, that is, it allows to respond to the rising edge of the process signal at an extremely fast speed, so that it can respond quickly based on the change in the speed of the drone and adjust the rotational power output by the second image acquisition device in time.
[0077] The above CPU 221 / 222 can be a high-speed counter, specifically four high-speed counters (30KHz), which are programmable and have reset inputs. Two independent input terminals can simultaneously count up and down, and can be connected to two A / B phase incremental encoders with a phase difference of 90°. For CPU224 / 224XP / 226, it can be 6 high-speed counters (30KHz) with the same functions as CPU221 / 222. For CPU 222 / 224 / 224XP / 226, the drone can be expanded with digital and analog expansion modules, and the simulator can be used to simulate the local input signal for debugging user programs.
[0078] Among them, the above-mentioned rotation drive mechanism can be composed of a drive motor, a reducer and a rotating shaft. The rotating shaft of the drive motor is connected to the input shaft of the reducer, and the output shaft of the reducer is connected to the rotating shaft. The first image acquisition device and the second image acquisition device are respectively installed on the rotating shafts of the two rotation drive mechanisms. The two drive motors can respectively drive the two rotating shafts to rotate, and the first image acquisition device and the second image acquisition device are driven to rotate based on the rotation of the two rotating shafts, so that the first image acquisition device and the second image acquisition device can acquire image data at different positions.
[0079] When the drone identifies that there is an image corresponding to a preset target object based on the image to be identified, for example, the drone pre-acquires the preset target object input by the relevant staff, and identifies the preset target object on the image to be identified acquired by the first image acquisition device, when the drone identifies that there is an image corresponding to the preset target object, the drone can determine the position of the preset target object in the area to be acquired based on the image of the preset target object, and then control the rotation drive mechanism corresponding to the second image acquisition device to output the corresponding rotational power, drive the second image acquisition device to rotate, so that the second image acquisition device can aim at the preset target object to acquire the target image, so as to perform further data processing on the target image later.
[0080] Furthermore, the drone can also control the rotation angle of the second image acquisition device based on the flight speed of the drone. For example, if the altitude of the drone remains unchanged, the faster the flight speed, the faster the second image acquisition device moves away from the preset target object. At this time, the drone can control the rotation drive mechanism to increase the output rotational power, so that the second image acquisition device rotates relatively faster; the slower the drone speed, the slower the second image acquisition device moves away from the preset target object. At this time, the drone can control the rotation drive mechanism to reduce the output rotational power, so that the second image acquisition device rotates relatively slower. In this way, the second image acquisition device can remain aligned with the preset target object to capture the target image.
[0081] Through the above-mentioned embodiments, the UAV can control the rotational power output by the rotational drive mechanism based on the rotational drive mechanism in combination with the acceleration of the UAV, so that the second image acquisition device can maintain alignment with the preset target object for target image acquisition during the flight of the UAV, thereby improving the efficiency of the UAV in collecting image data of the power grid area.
[0082] In one embodiment, it also includes: obtaining a recognition model to be trained, an image sample to be recognized, and a preset target object sample; inputting the above-mentioned image sample to be recognized into the above-mentioned recognition model to be trained, and having the above-mentioned recognition model recognize whether the above-mentioned image sample to be recognized has an image corresponding to the above-mentioned preset target object sample, and outputting corresponding recognition result prediction information; if the above-mentioned recognition result prediction information indicates that there is an image corresponding to the above-mentioned preset target object sample, obtaining a comparison result between the image corresponding to the above-mentioned preset target object sample recognized by the above-mentioned recognition model and the above-mentioned preset target object sample; adjusting the model parameters of the above-mentioned recognition model according to the above-mentioned comparison result until the preset training end condition is met, thereby obtaining the trained above-mentioned recognition model.
[0083] In this embodiment, the drone may have a built-in recognition model, and the drone may pre-iterate the recognition model to be trained to obtain a trained recognition model. The drone may obtain the recognition model to be trained, the image sample to be recognized, and the preset target object sample. The image sample to be recognized may be a sample of an image corresponding to the preset target object sample, and the preset target object sample may be selected according to actual conditions.
[0084] The drone can input the above-mentioned image samples to be identified into the above-mentioned recognition model to be trained. The recognition model can identify whether the above-mentioned image samples to be identified have images corresponding to the above-mentioned preset target samples, and after generating corresponding recognition result prediction information, output the corresponding recognition result prediction information. The drone can detect whether the above-mentioned output recognition result prediction information indicates that there is an image corresponding to the preset target sample. If the above-mentioned recognition result prediction information indicates that there is an image corresponding to the above-mentioned preset target sample, the drone can compare the image corresponding to the above-mentioned preset target sample identified by the above-mentioned recognition model with the above-mentioned preset target sample to obtain a corresponding comparison result. Therefore, the drone can adjust the model parameters of the above-mentioned recognition model according to the above-mentioned comparison result until the preset training end condition is met, and obtain the above-mentioned trained recognition model. Among them, the preset training end condition can be included in the preset training times, the similarity of the above-mentioned comparison result is greater than the preset similarity threshold, or the above-mentioned training times reach the above-mentioned preset training times.
[0085] Through this embodiment, the drone can use the image samples to be identified and the preset target object samples to identify the recognition model, and then can collect image data for each preset target object in the power grid area based on the trained recognition model, thereby improving the efficiency of the drone in collecting image data in the power grid area.
[0086] In an exemplary embodiment, Figure 2 As shown, Figure 2 It is a flow chart of a method for collecting low-altitude data of a drone in another embodiment. In this embodiment, a first image acquisition device and a second image acquisition device can be set on the drone. The drone also includes a processor and a communication module. Among them, the first image acquisition device can use a camera with an image recognition function, and the second image acquisition device can use a conventional camera. As the drone flies, the first image acquisition device and the second image acquisition device move accordingly. In the initial state, the first image acquisition device and the second image acquisition device are both facing the direction of the drone's forward movement. The first image acquisition device is in operation and collects images of the low-altitude situation on the lower side.
[0087] The drone can identify the image to be identified collected by the first image acquisition device, and identify whether there is an image corresponding to a preset target object in the image. When the drone finds that there is a preset target object in the collected image, the drone controls the rotating drive mechanism corresponding to the second image acquisition device through the processor output, and drives the second image acquisition device to rotate, so that the second image acquisition device keeps shooting the preset target object, thereby being able to obtain a complete and accurate target image of the preset target object, so as to further analyze and process the target image later. In the process of the drone flying, when the second image acquisition device continuously acquires the target image for the preset target object, the first image acquisition device can search and identify the next preset target object. After the recognition is completed, the drone controls the second image acquisition device to keep aiming at the new preset target object to acquire a new target image, so that more targets can be quickly discovered and the acquisition efficiency is improved. That is, the first image acquisition device and the second image acquisition device can work asynchronously, the first image acquisition device is used to identify each preset target object, and the second image acquisition device is used to acquire each target image of each preset target object.
[0088] Through the above-mentioned embodiment, the drone realizes the recognition of the preset target object and the perspective tracking and image acquisition of the preset target object respectively during the flight of the drone through the two image acquisition devices in the drone, thereby improving the efficiency of data acquisition of the power grid equipment. In addition, by setting the first image acquisition device and the second image acquisition device, wherein the first image acquisition device is used for rapid capture of the target object and the second image acquisition device is used for continuous acquisition of the target object, the first image acquisition device and the second image acquisition device form a coordinated effect, and it is not easy to have incomplete data. When the second image acquisition device continuously acquires the picture, the first image acquisition device can search for the next target object, and can quickly discover more targets, thereby improving the overall efficiency and not easily missing the target object.
[0089] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0090] Based on the same inventive concept, the embodiment of the present application also provides a drone low-altitude data acquisition device for implementing the drone low-altitude data acquisition method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more drone low-altitude data acquisition device embodiments provided below can refer to the limitations of the drone low-altitude data acquisition method above, and will not be repeated here.
[0091] In an exemplary embodiment, Figure 3 As shown, a UAV low-altitude data acquisition device is provided, including: an acquisition module 500, an identification module 502 and a collection module 504, wherein:
[0092] The acquisition module 500 is used to acquire the image to be identified acquired by the first image acquisition device for the area to be acquired during flight.
[0093] The recognition module 502 is used to input the above-mentioned image to be recognized into a trained recognition model, and the above-mentioned recognition model recognizes whether there is an image corresponding to a preset target object in the above-mentioned image to be recognized, and outputs a corresponding recognition result.
[0094] The acquisition module 504 is used to control the viewing angle of the second image acquisition device to follow the preset target object in the area to be acquired if the recognition result shows that there is an image corresponding to the preset target object, and acquire the target image corresponding to the preset target object through the second image acquisition device.
[0095] In one embodiment, the acquisition module 500 is used to adjust the acquisition direction of the first image acquisition device to match the flight direction when a flight start instruction is detected; and acquire the image to be identified acquired by the adjusted first image acquisition device for the area to be acquired.
[0096] In one embodiment, the acquisition module 500 is used to adjust the field of view of the first image acquisition device to the first field of view, and acquire the image to be identified acquired by the first image acquisition device based on the first field of view of the area to be acquired during flight.
[0097] In one embodiment, the acquisition module 504 is used to adjust the field of view of the second image acquisition device to a second field of view, and acquire the target image corresponding to the preset target object based on the second field of view through the second image acquisition device; wherein the first field of view is larger than the second field of view.
[0098] In one embodiment, the acquisition module 504 is used to obtain the acceleration information of the UAV if the recognition result is that there is an image corresponding to the preset target object; determine the rotational power corresponding to the rotational drive mechanism according to the acceleration information; and control the viewing angle of the second image acquisition device to follow the preset target object in the area to be acquired according to the rotational power.
[0099] In one embodiment, the acquisition module 504 is used to increase the rotational power corresponding to the rotational drive mechanism according to the acceleration information to obtain the increased rotational power if the acceleration information is greater than a preset threshold.
[0100] In one embodiment, the acquisition module 504 is used to reduce the rotational power corresponding to the rotational drive mechanism according to the acceleration information if the acceleration information is less than or equal to the preset threshold, so as to obtain the reduced rotational power.
[0101] In one embodiment, the above-mentioned device also includes: a training module, which is used to obtain a recognition model to be trained, an image sample to be recognized, and a preset target object sample; inputting the above-mentioned image sample to be recognized into the above-mentioned recognition model to be trained, and the above-mentioned recognition model recognizes whether the above-mentioned image sample to be recognized has an image corresponding to the above-mentioned preset target object sample, and outputs corresponding recognition result prediction information; if the above-mentioned recognition result prediction information is that there is an image corresponding to the above-mentioned preset target object sample, obtain a comparison result between the image corresponding to the above-mentioned preset target object sample recognized by the above-mentioned recognition model and the above-mentioned preset target object sample; adjust the model parameters of the above-mentioned recognition model according to the above-mentioned comparison result until the preset training end condition is met, and obtain the trained above-mentioned recognition model.
[0102] Each module in the above-mentioned UAV low-altitude data acquisition device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0103] In an exemplary embodiment, a computer device is provided. The computer device can be applied to a drone, and its internal structure diagram can be as follows: Figure 4As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a method for collecting low-altitude data of a drone is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or may be a key, trackball or touchpad provided on the computer device housing.
[0104] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0105] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned UAV low-altitude data collection method when executing the computer program.
[0106] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned UAV low-altitude data collection method is implemented.
[0107] In one embodiment, a computer program product is provided, including a computer program, which implements the above-mentioned UAV low-altitude data collection method when executed by a processor.
[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0109] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0111] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A method for collecting low-altitude data from an unmanned aerial vehicle, characterized in that: Applied to a drone, the drone is provided with a first image acquisition device and a second image acquisition device, and the method comprises: Acquire the image to be identified captured by the first image acquisition device for the area to be acquired during flight; Input the image to be identified into a trained recognition model, and let the recognition model identify whether there is an image corresponding to a preset target object in the image to be identified, and output a corresponding recognition result; If the recognition result is that there is an image corresponding to the preset target object, the viewing angle of the second image acquisition device is controlled to follow the preset target object in the area to be acquired, and the target image corresponding to the preset target object is acquired by the second image acquisition device.
2. The method according to claim 1, characterized in that The acquiring of the image to be identified acquired by the first image acquisition device for the area to be acquired during flight includes: When a flight start instruction is detected, adjusting the acquisition direction of the first image acquisition device to match the flight direction; The image to be identified acquired by the adjusted first image acquisition device for the area to be acquired is acquired.
3. The method according to claim 1, characterized in that The acquiring of the image to be identified acquired by the first image acquisition device for the area to be acquired during flight includes: Adjusting the field of view of the first image acquisition device to a first field of view, and acquiring the image to be recognized acquired by the first image acquisition device based on the first field of view of the area to be acquired during flight; The step of acquiring the target image corresponding to the preset target object by the second image acquisition device includes: Adjusting the field of view of the second image acquisition device to a second field of view, and acquiring a target image corresponding to the preset target object based on the second field of view by the second image acquisition device; Wherein, the first field of view is larger than the second field of view.
4. The method according to claim 1, characterized in that The drone is provided with a rotation drive mechanism, and the second image acquisition device is connected to the rotation drive mechanism; If the recognition result is that there is an image corresponding to the preset target object, controlling the viewing angle of the second image acquisition device to follow the preset target object in the area to be acquired during the flight includes: If the recognition result is that there is an image corresponding to the preset target object, acquiring acceleration information of the drone; Determining the rotational power corresponding to the rotational drive mechanism according to the acceleration information; The viewing angle of the second image acquisition device is controlled according to the rotational power to follow the preset target object in the area to be acquired.
5. The method according to claim 4, characterized in that The determining the rotational power corresponding to the rotational drive mechanism according to the acceleration information comprises: If the acceleration information is greater than a preset threshold, the rotational power corresponding to the rotational drive mechanism is increased according to the acceleration information to obtain the increased rotational power; or, If the acceleration information is less than or equal to the preset threshold, the rotational power corresponding to the rotational drive mechanism is reduced according to the acceleration information to obtain the reduced rotational power.
6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining a recognition model to be trained, image samples to be recognized, and preset target object samples; Inputting the image sample to be identified into the recognition model to be trained, allowing the recognition model to identify whether the image sample to be identified contains an image corresponding to the preset target object sample, and outputting corresponding recognition result prediction information; If the recognition result prediction information indicates that there is an image corresponding to the preset target object sample, obtaining a comparison result between the image corresponding to the preset target object sample recognized by the recognition model and the preset target object sample; The model parameters of the recognition model are adjusted according to the comparison result until a preset training end condition is met, thereby obtaining the trained recognition model.
7. A low-altitude data collection device for unmanned aerial vehicles, characterized in that: Applied to a drone, the drone is provided with a first image acquisition device and a second image acquisition device, and the device comprises: An acquisition module, used for acquiring the image to be identified acquired by the first image acquisition device for the area to be acquired during flight; A recognition module, used to input the image to be recognized into a trained recognition model, and the recognition model recognizes whether there is an image corresponding to a preset target object in the image to be recognized, and outputs a corresponding recognition result; The acquisition module is used to control the viewing angle of the second image acquisition device to follow the preset target object in the area to be acquired if the recognition result is that there is an image corresponding to the preset target object, and to acquire the target image corresponding to the preset target object through the second image acquisition device.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
