Unmanned aerial vehicle control method, storage medium, processor and electronic device

Abstract

The invention discloses an unmanned aerial vehicle control method, a storage medium, a processor and an electronic device. The method comprises the following steps: acquiring image information, and determining a first relative position relationship between a mounting point and an unmanned aerial vehicle based on the image information; obtaining point cloud information, wherein the point cloud information is obtained by performing radar scanning on the target area by the unmanned aerial vehicle; determining a second relative position relationship between the mounting point and the unmanned aerial vehicle based on the point cloud information; determining positioning information based on the first relative position relation and the second relative position relation; and generating a control strategy based on the positioning information, wherein the control strategy is used for controlling the unmanned aerial vehicle to move into the installation area of the installation point. The technical problem that an existing unmanned aerial vehicle is inaccurate in positioning when positioning a power transmission tower installation point is solved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control technology, and in particular to a control method, storage medium, processor and electronic device for an unmanned aerial vehicle. Background Art

[0002] With the continuous development of social economy, the demand for electricity continues to rise, and the scale of the power network continues to expand and upgrade. This development trend has directly led to a sharp increase in the number of transmission towers, and their heights are also increasing. In the maintenance of transmission towers, the installation of anti-fall devices is a crucial task, which is directly related to the life safety of operators. In order to achieve the accurate installation of anti-fall devices, high-precision positioning technology is indispensable.

[0003] At present, in the field of transmission tower installation point positioning, the existing positioning technology is mainly based on GPS modules, which can provide a certain degree of positioning services in an open, unobstructed environment. However, in the surrounding environment of the transmission tower, there are many factors that affect the accuracy of GPS positioning. On the one hand, the transmission tower itself has a complex structure, and its metal components and auxiliary equipment may block satellite signals, resulting in obstructed signal reception and inaccurate positioning information. On the other hand, the surrounding terrain, such as mountains, woods, etc., as well as possible buildings, electromagnetic field interference, etc., will cause multipath effects, causing satellite signals to reach the GPS module after multiple reflections, thereby generating signal deviations, and ultimately resulting in large positioning errors, which cannot meet high-precision positioning requirements.

[0004] In addition, the existing visual assistance technology only uses unilateral visual features to identify targets in the process of identifying installation points, which cannot adapt to the identification of targets in complex scenes and lacks comprehensive analysis and effective fusion of multiple visual features. The response time cannot be guaranteed.

[0005] In summary, although the existing technology has made some progress in the positioning of transmission towers, there are still problems such as insufficient positioning accuracy, poor environmental adaptability, and unstable response time. Therefore, a new technical solution is needed to solve these problems in order to improve the high-precision positioning capability of UAVs for transmission tower installation points.

[0006] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0007] The embodiments of the present invention provide a control method, storage medium, processor and electronic device for an unmanned aerial vehicle, so as to at least solve the technical problem that the existing unmanned aerial vehicles are inaccurate in locating the installation point of the transmission tower.

[0008] According to one aspect of an embodiment of the present invention, a method for controlling a drone is provided, comprising: acquiring image information, the image information being obtained by the drone performing image acquisition on a target area; determining a first relative position relationship between an installation point and the drone based on the image information; acquiring point cloud information, the point cloud information being obtained by the drone performing a radar scan on the target area; determining a second relative position relationship between the installation point and the drone based on the point cloud information; determining positioning information based on the first relative position relationship and the second relative position relationship; and generating a control strategy based on the positioning information, the control strategy being used to control the drone to move to within the installation area of ​​the installation point.

[0009] Optionally, determining the positioning information based on the first relative position relationship and the second relative position relationship includes: fusing data of the first relative position relationship and the second relative position relationship to determine the positioning information.

[0010] Optionally, determining a first relative position relationship between the installation point and the drone based on the image information includes: extracting features from the image information to obtain visual features of at least one target point; comparing the visual features with standard features stored in a feature library to obtain a similarity result; determining whether the target point is an installation point based on the similarity result; in response to the target point being an installation point, determining a first relative position relationship based on a current position of the target point and a current position of the drone.

[0011] Optionally, generating a control strategy based on the positioning information includes: obtaining the position information of the UAV at a first moment; generating path planning information based on the position information and the positioning information; and generating a control strategy based on the path planning information.

[0012] Optionally, obtaining the position information of the UAV at the first moment includes: obtaining first light information, the first light information is obtained by the UAV detecting the light emitted by a preset infrared light device; determining auxiliary positioning information based on the first light information, the auxiliary positioning information is used to characterize the relative position relationship between the preset infrared light device and the UAV; obtaining GPS positioning information, the GPS positioning information is obtained by positioning the UAV with a GPS module; and determining the position information based on the auxiliary positioning information and the GPS positioning information.

[0013] Optionally, path planning information is generated based on the position information and the positioning information, including: obtaining a path planning model, the path planning model is used to generate initial planning path information according to the position information and the positioning information; determining an estimated position of the UAV at a second moment based on the initial planning path information, wherein the second moment is located after the first moment; obtaining the actual position of the UAV at the second moment; adjusting parameters of the path planning model based on the actual position and the estimated position to obtain an updated path planning model; and generating path planning information using the updated path planning model based on the position information and the positioning information.

[0014] Optionally, the control method further includes: when it is detected that the drone moves to an adjacent area of ​​the installation point, using a differential positioning module of the drone to obtain the position information of the drone at the first moment, wherein the installation area is located in the adjacent area.

[0015] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above method when running.

[0016] According to yet another aspect of an embodiment of the present invention, a processor is provided, and the processor is used to run a program, wherein the program is configured to execute the above method when running.

[0017] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method through the computer program.

[0018] In an embodiment of the present invention, a method of determining a first relative position relationship between an installation point and a UAV based on image information and a second relative position relationship between the installation point and the UAV based on point cloud information is adopted. Positioning information is determined by the first relative position relationship and the second relative position relationship, thereby achieving the purpose of improving the positioning accuracy of the UAV for the installation point, thereby realizing the technical effect of precise positioning and navigation, and further solving the technical problem of inaccurate positioning of existing UAVs when locating the installation point of a transmission tower. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 is a hardware structure block diagram of a computer terminal of a method for controlling a drone according to one optional embodiment of the present invention;

[0021] Figure 2 is a flow chart of a method for controlling a drone according to one optional embodiment of the present invention;

[0022] Figure 3 is a flow chart of a method for controlling a drone according to one optional embodiment of the present invention;

[0023] Figure 4 is a positioning schematic diagram of a control method for a drone according to one of the optional embodiments of the present invention;

[0024] Figure 5is a schematic diagram of a control device for a drone according to one optional embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] According to one embodiment of the present invention, an embodiment of a method for controlling an unmanned aerial vehicle is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0028] The method embodiment can be executed in an electronic device or similar computing device including a memory and a processor in a drone. For example, the method can be executed in an electronic device of a drone. Figure 1 As shown, the electronic device of the drone may include one or more processors 102 (the processor may include but is not limited to a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP) chip, a microprocessor (MCU), a field programmable logic device (FPGA), a neural network processor (NPU), a tensor processing unit (TPU), an artificial intelligence (AI) type processor, etc.) and a memory 104 for storing data. Optionally, the electronic device of the above-mentioned car may also include a transmission device 106 for communication functions, an input and output device 108, and a display device 110. It can be understood by those skilled in the art that Figure 1The structure shown is for illustration only and does not limit the structure of the electronic device of the drone. For example, the electronic device of the drone may include more or fewer components than those described in the above structure, or may have a different configuration than those described in the above structure.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the control method of the drone in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the control method of the drone described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0031] The display device 110 may be, for example, a touch screen type liquid crystal display (LCD) and a touch display (also referred to as a "touch screen" or a "touch display screen"). The liquid crystal display may enable a user to interact with a user interface of the mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), and a user may interact with the GUI by finger contact and / or gestures on a touch-sensitive surface, wherein the human-computer interaction functions here may optionally include the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital videos, playing digital music and / or web browsing, etc. The executable instructions for executing the above human-computer interaction functions are configured / stored in a computer program product or a readable storage medium executable by one or more processors.

[0032] In the prior art, drones have the following disadvantages when locating installation points:

[0033] 1. Insufficient positioning accuracy: Existing technologies mainly rely on GPS modules to locate drones. However, since satellite signals may be affected by atmospheric interference, multipath effects, and urban canyon effects, the positioning accuracy is insufficient and cannot meet the centimeter-level accuracy requirements of transmission tower installation points.

[0034] 2. Lack of multimodal data fusion: Existing technologies fail to fully utilize multimodal sensor data such as visual sensors and Lidar, resulting in the inability to accurately identify and locate transmission tower installation points in complex scenarios, limiting the application of drones in power inspection and maintenance.

[0035] 3. Lack of real-time dynamic calibration: The existing GPS positioning technology lacks a real-time dynamic calibration mechanism and cannot correct positioning deviations caused by satellite signal errors, multipath effects, etc. in real time, resulting in instability in positioning results.

[0036] This embodiment provides a method for controlling a drone running on the electronic device of the drone. Figure 2 FIG. 1 is a flow chart of a method for controlling a drone according to one embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0037] Step S10, obtaining image information, where the image information is obtained by the drone collecting images of the target area;

[0038] Step S20, determining a first relative position relationship between the installation point and the drone based on the image information;

[0039] Step S30, obtaining point cloud information, where the point cloud information is obtained by the drone performing radar scanning on the target area;

[0040] Step S40, determining a second relative position relationship between the installation point and the drone based on the point cloud information;

[0041] The mounting points can be used to install fall protection devices, etc., and other devices can also be installed.

[0042] Step S50, determining positioning information based on the first relative position relationship and the second relative position relationship;

[0043] Step S60: generating a control strategy based on the positioning information, where the control strategy is used to control the drone to move to an installation area of ​​the installation point.

[0044] Through the above steps, a method of determining the first relative position relationship between the installation point and the drone based on image information and a method of determining the second relative position relationship between the installation point and the drone based on point cloud information is adopted, and positioning information is determined through the first relative position relationship and the second relative position relationship, thereby achieving the purpose of improving the positioning accuracy of the drone for the installation point, thereby achieving the technical effect of precise positioning and navigation, and further solving the technical problem of inaccurate positioning of existing drones when locating the installation point of the transmission tower.

[0045] Optionally, determining the positioning information based on the first relative position relationship and the second relative position relationship includes: fusing data of the first relative position relationship and the second relative position relationship to determine the positioning information.

[0046] Optionally, determining a first relative position relationship between the installation point and the drone based on the image information includes: extracting features from the image information to obtain visual features of at least one target point; comparing the visual features with standard features stored in a feature library to obtain a similarity result; determining whether the target point is an installation point based on the similarity result; in response to the target point being an installation point, determining a first relative position relationship based on a current position of the target point and a current position of the drone.

[0047] An optical camera is used to collect images, and the images are preprocessed in the order of grayscale conversion, threshold processing, and morphological operations. Then, the features of the installation points such as white circular landmarks are extracted, and the relevant parameters are calculated to determine the position of the installation points in the image. The specific process includes:

[0048] (1) Start the camera to collect images according to the preset acquisition strategy, and perform preprocessing such as grayscale conversion, threshold processing, and corrosion operations on the image to simplify the image, highlight the target object, and remove noise interference, which is conducive to subsequent more accurate image analysis and target feature extraction;

[0049] (2) Extract white circular landmarks and calculate features. In the binary image after morphological processing, further screen out possible white circular landmarks. For the extracted white circular landmarks, calculate the length-width ratio of its circumscribed rectangle and other installation point features;

[0050] (3) Match the calculated white circular landmark and its related features with the pre-established transmission tower installation point feature model, calculate the similarity or difference between the features, and complete the preliminary location confirmation of the installation point;

[0051] (4) During the installation process, the Lidar continuously measures the height between the UAV and the installation point. The visual features of the installation point area detected by the visual sensor are combined with the height information to fully confirm the relative position of the UAV and the installation point.

[0052] Lidar measures the height between the drone and the mounting point, combined with visual features of the mounting point area detected by the visual sensor, including:

[0053] Use Lidar to perform horizontal and vertical scans to cover the installation point area below and obtain a series of data points. These data points contain distance information and corresponding angle information.

[0054] By integrating and processing these data points, a three-dimensional point cloud of the drone’s surroundings can be constructed;

[0055] The visual sensor detects the visual features of the installation point area, extracts color features by performing color space conversion on the collected images, analyzes the grayscale co-occurrence matrix of the image, and other methods to extract texture features. The image processing algorithm is used to detect the shape features of the installation point area, and the multimodal features are matched with the established transmission tower installation point feature model to optimize the model structure and complete the high-precision positioning of the installation point.

[0056] Performing data fusion on the first relative position relationship and the second relative position relationship to determine positioning information includes:

[0057] Optionally, generating a control strategy based on the positioning information includes: obtaining the position information of the UAV at a first moment; generating path planning information based on the position information and the positioning information; and generating a control strategy based on the path planning information.

[0058] Optionally, obtaining the position information of the UAV at the first moment includes: obtaining first light information, the first light information is obtained by the UAV detecting the light emitted by a preset infrared light device; determining auxiliary positioning information based on the first light information, the auxiliary positioning information is used to characterize the relative position relationship between the preset infrared light device and the UAV; obtaining GPS positioning information, the GPS positioning information is obtained by positioning the UAV with a GPS module; and determining the position information based on the auxiliary positioning information and the GPS positioning information.

[0059] Visual sensors are used to detect infrared beams of specific wavelength and intensity, and are integrated with the position information obtained by the GPS module to improve the accuracy of drone positioning.

[0060] The visual sensor works with the GPS module. The visual sensor can accurately detect infrared beams of specific wavelength and intensity. By capturing infrared beams from transmission towers and surrounding areas, the positioning information obtained by the GPS module is fused and corrected to improve positioning accuracy.

[0061] Specifically, determining the location information based on the auxiliary positioning information and the GPS positioning information includes:

[0062] (1) After using the GPS module on the drone to obtain the drone's own location information, use the infrared sensitive element contained in the visual sensor to detect infrared light in a specific wavelength range, and convert the infrared light signal into an electrical signal through the photoelectric conversion element in the sensor;

[0063] (2) Use the signal conditioning circuit inside the sensor to amplify, filter, and perform analog-to-digital conversion on the electrical signal, so as to provide the digital signal for subsequent processing;

[0064] (3) Use built-in algorithms and pattern recognition technology to perform feature analysis on the detected infrared signal to determine the direction, intensity and other parameters of the infrared beam;

[0065] (4) Call the data fusion algorithm module to process the infrared beam information detected by the visual sensor and the position information obtained by the GPS module, and use data fusion algorithms such as Kalman filtering and extended Kalman filtering to fuse the GPS position information and the position information calculated by the infrared beam.

[0066] Optionally, path planning information is generated based on the position information and the positioning information, including: obtaining a path planning model, the path planning model is used to generate initial planning path information according to the position information and the positioning information; determining an estimated position of the UAV at a second moment based on the initial planning path information, wherein the second moment is located after the first moment; obtaining the actual position of the UAV at the second moment; adjusting parameters of the path planning model based on the actual position and the estimated position to obtain an updated path planning model; and generating path planning information using the updated path planning model based on the position information and the positioning information.

[0067] That is, a data fusion algorithm framework is established to fuse these measurements with the predicted state when the Lidar measures new height information and the visual sensor detects new visual features.

[0068] The fusion results are combined with the expected visual features based on the predicted position, and the state prediction and measurement update steps are continuously repeated. After a certain period of training, the system can accurately estimate the relative position of the drone and the installation point.

[0069] Optionally, the control method further includes: when it is detected that the drone moves to an adjacent area of ​​the installation point, using a differential positioning module of the drone to obtain the position information of the drone at the first moment, wherein the installation area is located in the adjacent area.

[0070] RTK GPS technology is used to calibrate the position of the drone. Through the collaboration between the base station and the GPS module on the drone, the differential distance correction information is calculated to achieve high-precision positioning of the drone.

[0071] When the UAV is about to approach the installation point, RTK GPS technology is used for position calibration. The base station compares the calculated precise position with the mobile station position to calculate the differential distance correction information, and uses this information to correct its own measured position to ensure that the UAV is accurately positioned at the installation point.

[0072] For example, the adjacent area is a first spherical space with a radius of 10 cm from the installation center, and the installation area is a second spherical space with a radius of 5 cm from the installation center. When the drone moves into the first spherical space, it is necessary to switch the differential positioning module for positioning to correct the positioning deviation.

[0073] Real-time kinematic differential GPS (RTK GPS) technology is used to calibrate the position of the UAV. The technology consists of two parts: a base station and a mobile station (GPS module on the UAV). The base station is set at a known precise location point and receives satellite signals for differential calculation and position calibration, thereby achieving high-precision positioning of the UAV.

[0074] The specific RTK positioning process includes the following steps:

[0075] (1) Determine the precise location of the base station through total station measurement, and install and fix the GPS receiver, antenna and related communication equipment of the base station;

[0076] (2) The GPS receiver of the base station receives radio frequency signals from multiple satellites, including the satellite's location information, time information, and the state of the satellite when sending the signal. The receiver amplifies and pre-processes the received radio frequency signal and converts it into an intermediate frequency signal suitable for subsequent processing;

[0077] (3) Sample and digitize the intermediate frequency signal to measure the time delay (i.e., pseudorange) of the satellite signal reaching the base station, and use the carrier phase observation value to obtain more accurate measurement information. Then, decode the navigation message in the satellite signal to obtain the satellite's ephemeris data, clock correction parameters and other information.

[0078] The drone GPS module receives satellite signals for data collection and preliminary processing, performs differential calculation and position calibration on the data received by the base station, and also includes:

[0079] (1) The UAV GPS (mobile station) module establishes a connection with the base station through a wireless communication link. Before the communication is established, relevant communication parameter configurations are performed, such as setting the communication frequency, link detection, and handshake.

[0080] (2) The observation data is first packaged and formatted, and organized into data frames according to a predefined format. The UAV sends the satellite observation data (including pseudorange, carrier phase, etc.) it has collected to the base station.

[0081] (3) After receiving the satellite observation data sent by the drone, the base station compares it with its own local observation data and calculates the corresponding error correction number. After receiving the differential correction number sent by the base station, the drone applies these correction numbers to its own observation data. Using the corrected observation data, the drone calculates its position again.

[0082] In this application, the GPS module is used to obtain its own position, and the visual sensor is used to detect the infrared beam to obtain the accurate position of the drone itself. The image captured by the optical camera of the drone is subjected to grayscale conversion, threshold processing and morphological operations, and white circular landmarks are extracted. The installation point features such as aspect ratio expansion operations are calculated, and the installation point is preliminarily confirmed. Lidar is used to measure spatial features such as height, and the relevant visual features of the installation point area are detected by the visual sensor. After feature fusion, the feature is input into the installation point recognition model to accurately locate the location of the installation point. The technical solution of this application is adopted to achieve the following technical effects:

[0083] 1. By combining the infrared beam detected by the visual sensor and the location information obtained by the GPS module, the data fusion algorithm is used to improve the accuracy of drone positioning to meet the needs of high-precision positioning of the transmission tower installation point.

[0084] 2. By integrating data from multiple sensors such as visual sensors, Lidar and GPS, the feature information of the transmission tower installation points can be fully extracted to improve the accuracy and reliability of positioning.

[0085] 3. Using RTK GPS technology, through the collaboration between the base station and the GPS module on the drone, the real-time dynamic calibration of the drone's position is realized, the positioning deviation is corrected, and the stability and accuracy of the positioning results are improved.

[0086] In this application, the system structure and modular design: a modular system including a vision-GPS fusion positioning unit, an image acquisition and processing unit, a visual feature detection unit, a data fusion and position determination unit, and an RTK GPS calibration unit is designed, and each unit works together to improve the overall performance and reliability of the system.

[0087] Figure 3 : is a flow chart of a method for accurately locating the installation point of a UAV transmission tower based on visual assistance and differential GPS technology according to an embodiment of the present invention. Figure 3 As shown, the method comprises the following steps:

[0088] Step S101: The visual sensor and the GPS module work together to capture infrared beams of the transmission tower and the surrounding area through the visual sensor, and fuse and correct the positioning information obtained by the GPS module.

[0089] The accuracy of positioning information is crucial in the positioning of the UAV transmission tower installation point. The GPS module is a commonly used positioning device that determines its own location information by receiving satellite signals. However, in actual applications, the positioning accuracy of the GPS module may be affected by many factors, such as satellite signal obstruction, multipath effect, etc., resulting in certain errors in the positioning information.

[0090] When the drone approaches the transmission tower, the visual sensor starts working. It first uses the infrared sensitive element contained in it to detect infrared light in a specific wavelength range. These infrared lights are converted into electrical signals under the action of the photoelectric conversion element inside the sensor. Then, the signal conditioning circuit inside the sensor is used to amplify, filter, and perform analog-to-digital conversion on the electrical signals, converting them into digital signals suitable for subsequent processing.

[0091] The amplifier in the signal conditioning circuit amplifies the weak electrical signal. During the amplification process, some noise and interference signals may be introduced. In order to remove these unwanted signal components, the signal conditioning circuit uses filtering technology. The analog electrical signal after amplification and filtering needs to be converted into a digital signal so that it can be processed by a computer or other digital processing device. The analog-to-digital conversion process is to sample the continuous analog signal at a certain sampling frequency and convert the analog value of each sampling point into a corresponding digital value. This process involves the application of the sampling theorem to ensure that the sampling frequency is high enough to accurately represent the original analog signal.

[0092] Then, using built-in algorithms and pattern recognition technology, the detected infrared signal is analyzed to determine the direction, intensity and other parameters of the infrared beam. At the same time, the GPS module obtains the drone's own location information.

[0093] Finally, the data fusion algorithm module is called to process the infrared beam information detected by the visual sensor and the position information obtained by the GPS module. In this process, the Kalman filter and extended Kalman filter data fusion algorithm can be used to fuse the GPS position information and the position information calculated by the infrared beam, thereby realizing the fusion correction of the positioning information obtained by the GPS module and improving the accuracy of positioning.

[0094] Kalman filtering is a recursive linear minimum variance estimation method. It makes an optimal estimate of the state of a system based on the state equation and measurement equation of the system. In this application, the GPS position information is considered as one measurement of the system, and the position information calculated by the infrared beam is considered as another measurement. The Kalman filter algorithm first predicts the next state of the system based on the prior knowledge and initial state of the system. Then, the predicted state is corrected based on the actual measurement values ​​(GPS position information and position information calculated by the infrared beam) to obtain a more accurate state estimate. This process will be repeated continuously, and as new measurement values ​​are continuously input, the state estimate of the system will become more and more accurate.

[0095] System state equation:

[0096] x k =F k-1 x k-1 +w k-1

[0097] Among them, F k-1 is the state transfer matrix, which describes the transition law of the system state from time k-1 to time k, x k-1 is the state vector at the previous moment, w k-1 is the process noise vector, usually assumed to have a mean of zero and a cosquare matrix of Q k-1 Gaussian white noise.

[0098] Measurement equation:

[0099] z k =H k x k +v k

[0100] Among them, H k is the measurement matrix, which maps the system state to the measurement space, v k is the measurement noise vector, also assumed to have a mean of zero and a covariance matrix of R k Gaussian white noise.

[0101] State update equation:

[0102]

[0103] Correct the predicted state according to the actual measured value. To obtain the updated optimal state estimate, K k is the Kalman gain matrix.

[0104] Step S102, using an optical camera to capture an image, pre-processing the image, and then extracting installation point features such as white circular landmarks to accurately determine the position of the installation point in the image.

[0105] The image acquisition process starts with the camera according to a carefully designed acquisition strategy, which fully considers multiple factors such as the UAV's flight attitude, distance from the transmission tower, and lighting conditions. For example, the camera shooting angle and focal length will be adjusted at different flight stages. When approaching the transmission tower, a smaller focal length is used to obtain detailed local images, and a larger focal length is used to cover the entire area when the distance is far. After the image is acquired, grayscale conversion is first performed. Because the color information of the color image may increase the complexity when extracting target features and is not necessary, it is converted into a grayscale image through a specific algorithm (weighted average method, according to the human eye's sensitivity to color, the red, green, and blue channels are weighted to calculate the pixel grayscale value) to simplify the data and retain the basic structure and contrast, providing a suitable format for subsequent processing. Then threshold processing is performed, that is, the image pixels are divided into two categories according to the grayscale value and the set threshold. If the target object (such as the installation point) has a high grayscale value and a low background, selecting a suitable threshold can retain the pixels above the threshold and set the pixels below the threshold to a fixed value (such as black), thereby highlighting the difference between the target and the background and reducing background interference. Finally, the erosion operation is performed, which is a morphological operation that slides the structural element across the image. If the pixels covered by the structural element meet the conditions (such as being greater than or less than a certain value), the pixels are retained, otherwise they are set to a fixed value. It can remove small noise points and isolated pixels, making the shape of the target object more regular and compact, and further simplifying the image to improve the accuracy of subsequent feature extraction.

[0106] Grayscale conversion formula (weighted average method):

[0107] Y=0.299R+0.587G+0.114B

[0108] In the binary image after morphological processing, possible white circular landmarks are further screened. First, preliminary screening is performed based on the color and shape characteristics of the white circular landmarks. In the binary image, the white area represents the possible target object. By analyzing the size, shape, connectivity and other properties of the white area, some areas that obviously do not meet the characteristics of the white circular landmark can be excluded. For example, white areas that are too small or too large, irregular in shape, and disconnected may be excluded.

[0109] For the extracted white circular landmark, the mounting point features such as the aspect ratio of its circumscribed rectangle are calculated. Calculating the aspect ratio of the circumscribed rectangle can provide shape information about the circular landmark. The center position of the circle can be determined by a moment-based method or a Hough transform. The radius can be calculated by measuring the distance from the center of the circle to the pixel on the boundary of the circular landmark. The circularity can be measured by calculating the relationship between the circumference and area of ​​the circular landmark. A perfect circle has a circularity of 1, while the actual circular landmark may deviate from 1 due to factors such as noise or image resolution.

[0110] The calculated white circular landmark and its related features are matched with the pre-established transmission tower installation point feature model. The pre-established feature model contains the ideal feature information of the transmission tower installation point, which is obtained by analyzing and counting a large number of transmission tower installation point images. During the matching process, the similarity or difference between the features is calculated. The similarity can be measured by calculating the distance between feature vectors, such as Euclidean distance, cosine similarity, etc. The difference can be measured by calculating the difference or relative difference between the features.

[0111]

[0112] The value of cosine similarity is between -1 and 1. The closer the value is to 1, the more similar the two feature vectors are. In feature matching, it can be used to measure the similarity between the features related to the white circular landmark and the features of the transmission tower installation point feature model.

[0113] The precise position confirmation of the installation point is completed based on the result of feature matching. If the similarity between the features is high and meets certain threshold conditions, it can be considered that the correct installation point has been found. At this time, the precise coordinates of the installation point in the image can be determined based on the position of the white circular landmark in the image and other related features. For example, if the white circular landmark is the center mark of the installation point, then its center position can be used as a reference for the horizontal and vertical coordinates of the installation point in the image. Combined with other features such as the radius size and the relative position relationship with other reference points, the accuracy of position confirmation can be further improved.

[0114] Step 103, using Lidar to measure the precise height, combining with the visual sensor to detect the relevant visual features of the installation point area, adjusting the UAV flight attitude in real time, and accurately navigating to the transmission tower installation point.

[0115] Lidar obtains information about the surrounding environment by emitting laser beams and receiving reflected light. When scanning the area where the transmission tower is installed, Lidar is used for horizontal and vertical scanning. The laser beam is emitted at a certain frequency. After being reflected by the surface of an object, the reflected light is received and the time from the emission to the reception of the laser is measured. The distance between the surface point of the object and the Lidar device can be calculated based on the speed of light and the time difference. At the same time, its angle measurement system records the angle information when the laser is emitted and received. Each laser pulse can obtain a data point containing distance and corresponding angle information. Continuously emitting laser pulses for measurement can obtain a series of data points covering the installation point area below. Integrating and processing these data points is the key to constructing three-dimensional point cloud data. First, filtering is required to remove abnormal data points. For example, statistical filtering methods can be used to eliminate data points that are obviously deviated from the normal distribution according to the distribution law of data points.

[0116] By processing a large number of data points in this way, a 3D point cloud data of the drone's surroundings can be constructed. This 3D point cloud data can intuitively reflect the geometric shape and structural characteristics of the surrounding environment of the installation point area, providing a basis for subsequent analysis and processing.

[0117] When the visual sensor detects the visual features of the installation point area, it must first convert the collected image into a color space. Commonly used color spaces include RGB (red, green, blue) and HSV (hue, saturation, brightness), etc. By converting the image from the RGB color space to the HSV color space, it is easier to extract color features. For the transmission tower installation point area, there are some specific color logos or marks, which can be extracted as color features in this way.

[0118] In addition to color features, texture features are also an important aspect of describing images. In order to extract texture features, analyzing the grayscale co-occurrence matrix of an image is a common method. The grayscale co-occurrence matrix describes the frequency relationship of pixels with different grayscale values ​​in an image in a certain direction and distance. By calculating the grayscale co-occurrence matrix, some texture feature parameters such as contrast, correlation, energy, and entropy can be obtained. These parameters can reflect the image's texture fineness, directionality, and randomness. By analyzing these features, the installation point area can be better identified.

[0119] Image processing algorithms are used to detect the shape features of the installation point area. Edge detection algorithms are used to extract the edge contours of objects in the image. Common edge detection algorithms include Sobel operator and Canny operator. These algorithms determine the position of the edge by calculating the gradient change of pixels in the image. For the installation point area, its shape has a certain regularity, such as a circle, square or other specific geometric shapes. By detecting the shape features, the location and range of the installation point can be further determined.

[0120] The extracted color features, texture features and shape features are associated and identified with the installation points and integrated into the established feature model. This feature model continuously optimizes its structure and strengthens the recognition of transmission tower installation points by analyzing and learning a large number of images of known installation points.

[0121] Step 104, using RTK GPS technology to perform position calibration, the base station compares the calculated precise position with the mobile station position to calculate differential distance correction information, and uses this information to correct the position measured by itself.

[0122] First, it is necessary to set it in the precise position by measuring with a total station. The total station can determine the position of a spatial point by measuring angles and distances. When determining the location of the base station, a series of measurement operations are required to ensure accuracy.

[0123] After determining the location, pay special attention when installing the base station's GPS receiver, antenna, and communication equipment. The performance and installation location of the GPS receiver will affect the quality of signal reception. The antenna installation direction and height must be reasonably adjusted to receive the strongest signal, and the communication equipment must ensure stable and reliable communication. The base station's GPS receiver begins to receive radio frequency signals from multiple satellites. These radio frequency signals contain a wealth of information, such as the satellite's location information, time information, and the state of the satellite when sending the signal.

[0124] After receiving, the RF signal needs to be amplified and filtered for preprocessing. After amplification and filtering, the RF signal will be converted into an intermediate frequency signal, which will be sampled and digitized. Sampling is to convert the continuous analog signal into a discrete digital signal, and digitization is to convert the sampled value into a digital value. In this process, the pseudo-range of the satellite signal to the base station is measured (calculated based on the signal propagation time and the speed of light), and the carrier phase observation value is used to improve the positioning accuracy.

[0125] Finally, the navigation message in the satellite signal must be decoded to obtain key information such as the satellite's ephemeris data and clock correction parameters. This information is very important for subsequent differential calculations and position calibration.

[0126] Figure 4 The schematic diagram of a method for positioning the installation point of a transmission tower of a UAV based on visual assistance and differential GPS according to an embodiment of the present invention is shown, presenting an efficient positioning system, and the positioning method is divided into four key processes. In the positioning information acquisition stage, the GPS module receives satellite signals to obtain the position of the UAV itself, and at the same time introduces a visual sensor to detect the infrared beam of the transmission tower and the surrounding area to obtain additional position information, and uses data fusion technology to fuse the information of the two for correction to improve the positioning accuracy. In the image acquisition and processing stage, starting from the optical camera to collect the image, followed by grayscale conversion, threshold processing, morphological operations and other preprocessing, then extracting the white circular landmark and calculating the installation point features, and finally determining the position of the installation point in the image through feature matching. In the visual feature detection stage, the Lidar sensor measures the height information between the UAV and the installation point, and the visual sensor detects the color, texture and shape of the installation point area again. In the position calibration stage, RTK GPS technology is used to calculate the differential distance correction information, and the position is calibrated through the cooperation between the base station and the GPS module on the UAV, so that the UAV can accurately navigate to the installation point of the transmission tower.

[0127] The present embodiment provides a UAV transmission tower installation point precise positioning device based on visual assistance and differential GPS technology, which includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment.

[0128] Figure 5FIG. 1 is a schematic diagram of an optional UAV transmission tower installation point positioning device based on visual assistance and differential GPS according to an embodiment of the present invention. Figure 5 As shown, the device may include: a positioning information acquisition unit 30, an image acquisition and processing unit 31, a visual feature detection unit 32, and a position calibration unit 33.

[0129] Among them, the positioning information acquisition unit 30 is used to obtain the drone's own position information through the GPS module, and at the same time use the visual sensor to detect the infrared light beams of the transmission tower and the surrounding area, and fuse the information of the two to correct the positioning information.

[0130] The image acquisition and processing unit 31 is used to acquire images of the transmission tower and its surrounding environment through an optical camera, perform preprocessing such as grayscale conversion, threshold processing and morphological operations, extract white circular landmarks and calculate their installation point features, and then match these features with a pre-established feature model to determine the position of the installation point in the image.

[0131] The height and visual feature detection unit 32 is used to measure the height information between the drone and the installation point using the Lidar sensor, and to detect visual features such as color, texture and shape of the installation point area using the visual sensor.

[0132] The position calibration unit 33 is used to perform position calibration using RTK GPS technology, calculate differential distance correction information, and achieve accurate positioning of the transmission tower installation point by the UAV when the position calibration meets high-precision requirements.

[0133] The above-mentioned UAV transmission tower installation point positioning device based on visual assistance and differential GPS can obtain the UAV's own position information by using the GPS module through the positioning information acquisition unit 30, and detect the infrared beam of the transmission tower and the surrounding area with the help of the visual sensor, and then fuse the information of the two to correct the positioning. The image acquisition and processing unit 31 uses an optical camera to collect the transmission tower and the surrounding environment image, and then performs pre-processing such as grayscale conversion, threshold processing and morphological operation, extracts the white circular landmark and calculates its installation point features, and then matches it with the pre-established feature model to determine the installation point position. The height information between the UAV and the installation point is measured by the Lidar sensor through the height and visual feature detection unit 32, and the visual features such as color, texture and shape of the installation point area are detected by the visual sensor. The position calibration unit 33 uses RTK GPS technology to perform position calibration, calculate the differential distance correction information, and realize the precise positioning of the transmission tower installation point when the position calibration meets the high-precision requirements. This embodiment significantly improves the accuracy and stability of positioning by using visual assistance and differential GPS technology, and solves the technical problems of insufficient accuracy and poor reliability when the UAV locates the installation point of the transmission tower.

[0134] Optionally, the positioning information acquisition unit 30 includes: a GPS positioning module, which is used to receive satellite signals through a GPS module carried on the drone to obtain the initial position information of the drone itself. In this process, GPS is used to obtain the position information of the drone, but since the satellite signal may be affected by factors such as atmospheric interference and multipath effects, there is a certain error in GPS positioning. Therefore, a visual sensor is introduced, which can obtain additional information related to the relative position of the transmission tower, such as distance, angle, etc., by analyzing these infrared beams. An information fusion module is used to fuse and correct the position information obtained by the GPS positioning module with the infrared beam information detected by the infrared detection module. First, the data format of the two types of information is unified and preprocessed to remove abnormal values ​​and noise. Then, the Kalman filter algorithm is used to comprehensively consider the accuracy and stability of GPS positioning and the real-time and precision of infrared detection, and the two types of information are weighted and fused. By continuously adjusting the weight coefficient, the fused position information is made more accurate and reliable.

[0135] Optionally, the image acquisition and processing unit 31 includes: an image acquisition and preprocessing module, the image acquisition module controls an optical camera with high resolution and good color reproduction ability to acquire images of the transmission tower and the surrounding environment, and can be adjusted according to preset parameters such as focal length, exposure time, etc. during the acquisition process to adapt to different lighting conditions and shooting distances. The grayscale conversion module performs grayscale conversion on the acquired color image, first reads the values ​​of the red, green, and blue channels of each pixel of the color image, and then assigns different weights to the three channels according to a specific weighted average method to calculate the grayscale value of each pixel, thereby converting the color image into a grayscale image. The threshold processing module sets a suitable threshold for the grayscale image, compares the pixel with the threshold according to the grayscale value, marks the pixel greater than or equal to the threshold as the target pixel, and marks the pixel less than the threshold as the background pixel, highlighting the difference between the target object and the background to separate the target object. The feature extraction module is used to extract the installation point features in the image. First, the edge contour of the object in the image is detected by an edge detection algorithm (Canny operator). For areas that may be installation points, their edges usually have certain features, such as circular edges. Then, the white circular landmark is extracted, and its circumscribed rectangle and other features such as the aspect ratio are calculated. In addition, the features of the installation point can be further determined by analyzing the local features of the image, such as texture features, color features, etc. The feature matching module is used to match the extracted installation point features with the pre-established transmission tower installation point feature model. First, the similarity or difference between the extracted features and each feature in the feature model is calculated. A variety of methods are used to calculate the similarity, and the features are sorted according to the size of the similarity, and the features with the highest similarity are selected as the matching results.

[0136] Optionally, the visual feature detection unit 32 includes: the height measurement module uses the Lidar sensor to perform multiple measurements from different angles and positions to accurately determine the height information between the drone and the installation point. The visual feature detection module makes full use of the visual sensor to comprehensively detect the color, texture and shape features of the installation point area to better identify the installation point area. First, the height information is screened to remove outliers and noise, and then the most representative and distinguishing features are selected from the rich visual features. The feature matching module matches the key features selected by the feature selection module with the pre-established installation point feature model to determine the position of the installation point in the image and the relative position relationship between the drone and the installation point. The weight allocation module assigns weights to each matching pair based on the similarity between the key features. The higher the similarity of the matching pair, the greater the weight assigned, and the matching pairs with high similarity can be given more attention in subsequent processing, thereby significantly improving the accuracy and reliability of positioning.

[0137] Optionally, the position calibration unit 33 includes: a base station selection and reception module, which determines the precise position point of the base station through total station measurement, and the receiver amplifies and filters the received radio frequency signal to convert it into an intermediate frequency signal suitable for subsequent processing. The intermediate frequency signal is sampled and digitized, and the time delay (i.e., pseudorange) of the satellite signal reaching the base station is measured. At the same time, the carrier phase observation value is used to obtain more accurate measurement information, and the navigation message in the satellite signal is decoded to obtain the satellite's ephemeris data, clock correction parameters and other information. The information transmission module transmits the differential distance correction information of the base station to the drone, and the position calibration module enables the drone to use the correction information to correct the position to achieve high-precision positioning. The error evaluation module defines the error function to evaluate the quality of the position calibration, and the parameter update module adjusts the parameters according to the error evaluation results to ensure positioning accuracy and stability.

[0138] The above-mentioned UAV transmission tower installation point positioning device based on visual assistance and differential GPS can also include a processor and a memory. The above-mentioned positioning information acquisition unit 30, image acquisition and processing unit 31, height and visual feature detection unit 32, position calibration unit 33, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0139] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the installation point of the UAV transmission tower can be accurately located by adjusting the kernel parameters.

[0140] Compared with the prior art, this application has the following advantages:

[0141] 1. High-precision positioning capability: The existing technology mainly relies on GPS modules for positioning, but in complex environments, such as around transmission towers, it will be affected by signal shielding and multipath effects, resulting in insufficient positioning accuracy. The present invention uses visual-GPS fusion positioning technology, combines the visual sensor to detect infrared beams and the location information obtained by the GPS module, and uses data fusion algorithms to improve positioning accuracy, meeting the centimeter-level accuracy requirements of the transmission tower installation point.

[0142] 2. Strong environmental adaptability: In the existing technology, GPS signals are easily blocked in complex environments, such as forest areas, resulting in inaccurate positioning information. The present invention uses visual sensors and Lidar sensors to enable drones to accurately locate even in environments where GPS signals are limited, thereby enhancing the adaptability and robustness of the system.

[0143] 3. Multimodal data fusion: In terms of visual assistance, the existing technology only uses unilateral visual features for target recognition, lacking comprehensive analysis and effective fusion of multiple visual features. The present invention uses a feature matching and data fusion algorithm framework to compare and match the visual features detected in real time with the information in the feature library, realize multimodal data fusion, and improve the accuracy and reliability of positioning.

[0144] 4. Real-time dynamic calibration: The existing technology lacks a real-time dynamic calibration mechanism and cannot correct the positioning deviation caused by satellite signal errors, multipath effects, etc. in real time. The present invention adopts RTK GPS technology, and through the cooperation of the base station and the GPS module on the drone, calculates the differential distance correction information, realizes the real-time dynamic calibration of the drone position, and improves the stability and accuracy of the positioning results.

[0145] Traditional positioning technology has many shortcomings in the positioning of transmission tower installation points. When relying solely on GPS modules for positioning, it is often unable to meet the requirements of high-precision positioning due to interference from various factors, such as satellite signal obstruction and multipath effects. In recent years, with the continuous development of computer vision technology, sensor technology and satellite positioning technology, new ideas and methods have been provided for the application of high-precision positioning technology in transmission tower installation points. Computer vision technology has made significant progress in target recognition and positioning. In the positioning of transmission tower installation points, optical cameras are used to collect images of poles and surrounding environments. After preprocessing steps such as grayscale conversion, threshold processing, and morphological operations, installation point features such as white circular landmarks can be extracted, providing an important basis for accurately determining the location of installation points. The fusion application of multiple sensors has become an effective means to improve positioning accuracy. Lidar sensors can measure distances by emitting and receiving laser beams, obtain height information between drones and installation points, and obtain three-dimensional point cloud data of the surrounding environment. Visual sensors can detect infrared beams of specific wavelengths and intensities to provide additional auxiliary information for positioning. The emergence of real-time dynamic differential GPS (RTK GPS) technology provides strong support for high-precision positioning. This technology sets up a base station at a known precise location to receive satellite signals for differential calculation and position calibration. The GPS module (rover) on the drone communicates with the base station, receives the differential correction number and applies it to its own position calculation, thereby greatly improving the accuracy of positioning.

[0146] In this application, the GPS module is used to obtain its own position, and the infrared beam is detected by the visual sensor to obtain the accurate position of the drone itself. Then, the drone optical camera is used to collect images, and the collected images are sequentially subjected to grayscale conversion, threshold processing and morphological operations, and then the white circular landmark is extracted, and the installation point features such as aspect ratio expansion operation are calculated, so as to accurately confirm the position of the installation point. On this basis, Lidar is used to measure the precise height, and the relevant visual features of the installation point area are detected by the visual sensor, so as to adjust the flight attitude of the drone in real time and realize accurate navigation to the installation point of the transmission tower. In addition, RTK GPS technology is used for position calibration, that is, the calculated precise position is compared with the mobile station position through the base station, so as to calculate the differential distance correction information, and use this information to correct the position measured by itself. The present invention effectively solves the problem of insufficient positioning accuracy when the drone relies solely on the GPS module for positioning, improves the position confirmation accuracy of the visual system for the target object in the drone image, and uses the technology of multi-module fusion to successfully realize the high-precision positioning of the installation point on the transmission tower by the drone.

[0147] The present application also provides a device for accurately locating the installation point of a UAV transmission tower based on visual assistance and differential GPS technology, comprising:

[0148] Vision-GPS fusion positioning unit: This unit is mainly responsible for combining GPS positioning technology with visual sensor technology to improve the accuracy and reliability of drone positioning. The initial location information of the drone is obtained through the GPS module to provide basic data for positioning; the visual sensor detects specific infrared beams and uses the information it detects to correct and supplement GPS positioning.

[0149] Image acquisition and processing unit: The function of this unit is to use an optical camera to collect images of the transmission tower and its surrounding environment, and perform a series of processing operations on the images to extract the feature information of the installation point and ultimately determine the precise location of the installation point in the image.

[0150] Height and visual feature detection unit: Its function is to comprehensively detect the relative position relationship between the drone and the installation point from the two aspects of height and visual features by using Lidar and visual sensors. Lidar measures the height information between the drone and the installation point, providing data support for the height control and precise landing of the drone; the visual sensor detects various visual features of the installation point area, such as color, texture and shape.

[0151] Data fusion and position determination unit: This unit is responsible for fusing data from different sensors (such as visual sensors, Lidar) and different data sources (such as visual features, height information, GPS information, etc.). By establishing a feature library and data fusion algorithm framework, various measurements and feature information are integrated and analyzed to accurately estimate the relative position of the drone and the installation point, and enable the drone to accurately navigate to the installation point.

[0152] RTK GPS calibration unit: This unit uses real-time kinematic differential GPS (RTK GPS) technology to calibrate the position of the drone through collaboration between a base station and the GPS (rover) module on the drone. The base station is set at a known precise location point, receives and processes satellite signals, and calculates differential corrections; the drone (rover) module communicates with the base station, receives corrections and applies them to its own position calculation, thereby achieving high-precision positioning of the drone. An embodiment of the present invention also provides a storage medium, which stores a computer program, wherein the computer program is configured to execute the steps of any of the above method embodiments when it is run.

[0153] An embodiment of the present invention further provides a processor, which is configured to run a computer program to execute the steps in any of the above method embodiments.

[0154] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps in any one of the above method embodiments through the computer program.

[0155] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0156] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0157] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0159] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0160] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0162] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for controlling an unmanned aerial vehicle, characterized in that: include: Acquiring image information, wherein the image information is obtained by collecting images of a target area by a drone; Determine a first relative position relationship between the installation point and the drone based on the image information; Acquiring point cloud information, where the point cloud information is obtained by performing radar scanning of a target area by a drone; Determine a second relative position relationship between the installation point and the drone based on the point cloud information; Determine positioning information based on the first relative position relationship and the second relative position relationship; A control strategy is generated based on the positioning information, and the control strategy is used to control the drone to move to the installation area of ​​the installation point.

2. The method for controlling a drone according to claim 1, characterized in that: Determining positioning information based on the first relative position relationship and the second relative position relationship includes: Data fusion is performed on the first relative position relationship and the second relative position relationship to determine the positioning information.

3. The method for controlling a drone according to claim 1, characterized in that: Determining a first relative position relationship between the installation point and the drone based on the image information includes: Extracting features from the image information to obtain visual features of at least one target point; Comparing the visual features with standard features stored in a feature library to obtain a similarity result; Determine whether the target point is an installation point based on the similarity result; In response to the target point being the installation point, the first relative position relationship is determined based on a current position of the target point and a current position of the drone.

4. The method for controlling a drone according to claim 1, characterized in that: Generating a control strategy based on the positioning information includes: Get the location information of the drone at the first moment; generating path planning information based on the position information and the positioning information; The control strategy is generated based on the path planning information.

5. The method for controlling a drone according to claim 4, characterized in that: Get the drone's location information at the first moment, including: Acquire first light information, where the first light information is obtained by the drone detecting light emitted by a preset infrared light device; Determine auxiliary positioning information based on the first light information, where the auxiliary positioning information is used to characterize the relative position relationship between the preset infrared light device and the drone; Obtaining GPS positioning information, where the GPS positioning information is obtained by positioning the GPS module of the drone; The location information is determined based on the auxiliary positioning information and the GPS positioning information.

6. The method for controlling a drone according to claim 4, characterized in that: Generating path planning information based on the position information and the positioning information includes: Acquire a path planning model, wherein the path planning model is used to generate initial planned path information according to the position information and the positioning information; Determine an estimated position of the UAV at a second moment based on the initial planned path information, wherein the second moment is after the first moment; Get the actual position of the drone at the second moment; Adjusting the path planning model based on the actual position and the estimated position to obtain an updated path planning model; Based on the position information and the positioning information, path planning information is generated using an updated path planning model.

7. The method for controlling a drone according to claim 4, characterized in that: The control method further comprises: When it is detected that the drone moves to a vicinity of the installation point, the position information of the drone at the first moment is acquired using the differential positioning module of the drone, wherein the installation area is located in the vicinity.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

9. A processor, characterized in that: The processor is used to run a program, wherein the program is configured to execute the method described in any one of claims 1 to 7 when run.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 7.

Images