Unmanned aerial vehicle composite navigation positioning method and device for complex environment

Abstract

The application discloses a complex environment-oriented unmanned aerial vehicle composite navigation positioning method and device, and belongs to the technical field of unmanned aerial vehicle navigation positioning. The method comprises the following steps: based on real-time image data and a preloaded reference map, surface texture features are extracted through SLAM technology and matched with the reference map to generate visual positioning data; dynamic error compensation is performed on the visual positioning data based on real-time motion estimation data to obtain compensated visual positioning data; inertial motion data and the compensated visual positioning data are input into a multi-source data fusion model, error correction is performed on the inertial motion data through a filtering unit to obtain corrected inertial motion data, feature matching optimization is performed on the compensated visual positioning data through a neural network unit to obtain optimized visual positioning data, and the corrected inertial motion data and the optimized visual positioning data are jointly solved to generate fusion positioning results, so that the precision of unmanned aerial vehicle navigation positioning in a complex environment is improved.

Description

Technical Field

[0001] This application belongs to the field of unmanned aerial vehicle (UAV) navigation and positioning technology, and particularly relates to a composite navigation and positioning method and device for UAVs in complex environments. Background Technology

[0002] Unmanned aerial vehicles (UAVs) are increasingly used in surveying and exploration, emergency rescue, inspection and security, and military reconnaissance. In actual operations, they often face complex scenarios such as densely built-up urban environments, lush outdoor environments, and severely obstructed indoor environments. These scenarios present problems such as GPS signal loss / weak signals, blurred visual features, and dynamic environmental changes, which place extremely high demands on the accuracy and robustness of navigation and positioning.

[0003] Existing navigation and positioning methods for unmanned aerial vehicles (UAVs) mostly employ single inertial or visual navigation, or simple multi-source data stitching and fusion. Pure inertial navigation suffers from cumulative errors, and its positioning accuracy drops sharply after prolonged operation. Pure visual navigation is susceptible to ambient lighting, feature occlusion, and motion blur, resulting in poor positioning stability. Traditional multi-source fusion methods often use fixed-weight fusion strategies, failing to consider the reliability differences of sensor data under different environments, and lacking targeted compensation for dynamic errors in visual positioning. In complex environments, they struggle to achieve high-precision and highly stable continuous positioning, exhibiting problems such as positioning drift, outlier interference, and insufficient robustness.

[0004] Furthermore, existing fusion algorithms only optimize visual positioning data at a shallow feature matching level, failing to fully explore the high-dimensional semantic features of images and thus unable to effectively distinguish similar features in complex environments, resulting in a high feature matching error rate. At the same time, the error correction of inertial navigation data is not linked to the confidence level of visual observations. When there are errors in the visual data, it will still generate incorrect corrections for the inertial data, further reducing the overall positioning accuracy. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a composite navigation and positioning method and device for unmanned aerial vehicles (UAVs) in complex environments. This method improves the accuracy, robustness, and dynamic adaptability of UAV navigation and positioning in complex environments through multi-sensor data acquisition, dynamic compensation for visual positioning, and intelligent fusion of multi-source data.

[0006] To address the aforementioned problems, according to a first aspect of the present invention, a composite navigation and positioning method for unmanned aerial vehicles (UAVs) in complex environments is provided, the method comprising: Collect inertial motion data, real-time image data, and real-time motion estimation data of the unmanned aerial vehicle; Based on the real-time image data and the preloaded reference map, surface texture features are extracted using SLAM technology and matched with the reference map to generate visual positioning data. Based on the real-time motion estimation data, dynamic error compensation is performed on the visual positioning data to obtain compensated visual positioning data. The inertial motion data and the compensated visual positioning data are input into the multi-source data fusion model. The inertial motion data is corrected for errors by the filtering unit to obtain the corrected inertial motion data. The compensated visual positioning data is optimized by feature matching by the neural network unit to obtain the optimized visual positioning data. The corrected inertial motion data and the optimized visual positioning data are jointly solved to generate the fused positioning result.

[0007] The multi-source data fusion model includes a filtering unit and a neural network unit.

[0008] According to one embodiment of this application, the step of extracting surface texture features using SLAM technology and matching them with the reference map based on the real-time image data and a pre-loaded reference map to generate visual positioning data includes: The real-time image data is input into the SLAM front end to extract feature points from the real-time image data and construct a local map. The local map is registered with the preloaded reference map by feature points, and the pose of the local map in the reference map coordinate system is calculated. The pose is converted into position and attitude information in a geographic coordinate system to generate visual positioning data.

[0009] According to one embodiment of this application, the step of performing dynamic error compensation on the visual positioning data based on the real-time motion estimation data to obtain compensated visual positioning data includes: The real-time motion estimation data is input into the motion compensation model to obtain the visual feature pixel offset caused by the posture change; The position estimate in the visual positioning data is corrected based on the pixel offset to eliminate positioning drift caused by motion blur, thus obtaining compensated visual positioning data.

[0010] According to one embodiment of this application, the step of correcting the inertial motion data through a filtering unit to obtain corrected inertial motion data includes: The compensated visual positioning data is used as the observation value by the filtering unit, and the state of the inertial motion data is updated by the extended Kalman filter algorithm to obtain the corrected inertial motion data.

[0011] According to one embodiment of this application, the step of performing feature matching optimization on the compensated visual positioning data through a neural network unit to obtain optimized visual positioning data includes: The real-time image data is input into a pre-trained neural network to extract high-dimensional semantic features; The high-dimensional semantic features are matched with features in the baseline map, and the confidence level of the compensated visual positioning data is evaluated and outliers are removed to obtain optimized visual positioning data.

[0012] According to one embodiment of this application, the step of jointly solving the corrected inertial motion data and the optimized visual positioning data to generate a fused positioning result includes: The corrected inertial motion data is input into the inertial navigation mechanics equations to construct a system state vector, which includes the three-dimensional position, three-dimensional velocity, quaternion attitude of the unmanned aerial vehicle, and the zero bias error of the inertial measurement unit. The system state vector is updated over time to obtain the predicted state values ​​and their covariance matrix; The optimized visual positioning data is input into the visual observation model, and the confidence factor of the current visual observation is output based on the pre-trained neural network. The noise covariance matrix of the visual observation model is adaptively adjusted based on the confidence factor to obtain the adjusted noise covariance matrix. The predicted state value, covariance matrix, and adjusted noise covariance matrix are input into the Kalman filter update equation to perform measurement update and obtain the posterior estimate of the system state vector. Position and attitude information are extracted from the posterior estimate to generate a fused localization result.

[0013] According to one embodiment of this application, the step of adaptively adjusting the noise covariance matrix of the visual observation model based on the confidence factor to obtain the adjusted noise covariance matrix includes: The confidence factor is input into a pre-trained confidence estimation network, and the confidence score is output. The confidence score is compared with a preset threshold, and the noise covariance matrix of the visual observation model is adjusted according to the comparison result to obtain the adjusted noise covariance matrix.

[0014] According to a second aspect of the present invention, a composite navigation and positioning device for unmanned aerial vehicles (UAVs) oriented towards complex environments is provided, the device comprising: The data acquisition module is used to collect inertial motion data, real-time image data, and real-time motion estimation data of the unmanned aerial vehicle. The visual positioning module is used to extract surface texture features based on the real-time image data and the preloaded reference map, and match them with the reference map to generate visual positioning data. The optical flow compensation module is used to perform dynamic error compensation on the visual positioning data based on the real-time motion estimation data to obtain compensated visual positioning data. The fusion positioning module is used to input the inertial motion data and the compensated visual positioning data into the multi-source data fusion model, correct the error of the inertial motion data through the filtering unit to obtain the corrected inertial motion data, optimize the feature matching of the compensated visual positioning data through the neural network unit to obtain the optimized visual positioning data, and jointly solve the corrected inertial motion data and the optimized visual positioning data to generate the fusion positioning result.

[0015] According to a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the composite navigation and positioning method for unmanned aerial vehicles in complex environments as described in the first aspect above.

[0016] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the composite navigation and positioning method for unmanned aerial vehicles in complex environments as described in the first aspect above.

[0017] According to a fifth aspect of the present invention, a chip is provided, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the composite navigation and positioning method for unmanned aerial vehicles in complex environments as described in the first aspect.

[0018] According to a sixth aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the composite navigation and positioning method for unmanned aerial vehicles in complex environments as described in the first aspect above.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application.

[0020] The present invention provides a composite navigation and positioning method for unmanned aerial vehicles in complex environments, which has the following advantages over the prior art: (1) By integrating the advantages of inertial navigation and visual navigation, and combining SLAM technology, dynamic error compensation and multi-source data intelligent fusion model, this invention solves the positioning defects of a single navigation method in complex environments, effectively suppresses the cumulative error of inertial navigation and the environmental sensitivity of visual navigation, and improves the positioning accuracy and continuous working capability of unmanned aerial vehicles in complex scenarios such as GPS loss of lock, feature occlusion and motion ambiguity. It does not require hardware upgrades to a single sensor and has high practicality and cost-effectiveness.

[0021] (2) This invention performs dynamic error compensation on visual positioning data based on real-time motion estimation data, accurately calculates the offset of visual feature pixels caused by changes in aircraft attitude, and specifically corrects the positioning drift caused by motion blur, effectively improving the original accuracy of visual positioning data, laying a high-quality data foundation for subsequent multi-source fusion, and enhancing the stability of visual positioning during dynamic flight.

[0022] (3) This invention integrates the filter unit and the neural network unit through the designed multi-source data fusion model, performs accurate error correction on the inertial motion data through the extended Kalman filter algorithm, and extracts high-dimensional semantic features of the image using the pre-trained neural network to realize the confidence assessment and outlier removal of the visual positioning data. It fully explores the effective information of the sensor data, avoids the errors of shallow feature matching and the interference of outliers on the fusion results, and improves the reliability of the positioning data.

[0023] (4) By introducing a confidence factor in the joint solution process, the present invention enables adaptive adjustment of the noise covariance matrix of the visual observation model, so that the fusion algorithm can dynamically adjust the fusion weight according to the reliability of the visual data in the current environment. When the confidence of the visual data is low, its influence on the fusion result is reduced, and when the confidence of the visual data is high, its correction effect is enhanced, which greatly enhances the robustness and environmental adaptability of the fusion positioning. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating the composite navigation and positioning method for unmanned aerial vehicles in complex environments provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of the composite navigation and positioning device for unmanned aerial vehicles in complex environments provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0026] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate to allow embodiments of this application to be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, the use of "and" and / or to indicate at least one of the connected objects in the specification and claims, and the character " / ", generally indicates an "or" relationship between the preceding and following objects.

[0027] The following description, in conjunction with the accompanying drawings, details the composite navigation and positioning method, composite navigation and positioning device, electronic device, and readable storage medium for unmanned aerial vehicles (UAVs) in complex environments provided in this application, through specific embodiments and application scenarios.

[0028] Among them, the composite navigation and positioning method for unmanned aerial vehicles in complex environments can be applied to the terminal, specifically executed by the hardware or software in the terminal.

[0029] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0030] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0031] The composite navigation and positioning method for unmanned aerial vehicles (UAVs) in complex environments provided in this application embodiment can be executed by an electronic device or a functional module or entity within an electronic device that can implement the composite navigation and positioning method for UAVs in complex environments. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The following description uses an electronic device as the execution subject to illustrate the composite navigation and positioning method for UAVs in complex environments provided in this application embodiment.

[0032] Figure 1 This is a flowchart illustrating the composite navigation and positioning method for unmanned aerial vehicles in complex environments provided in this application embodiment. Figure 1 As shown, the composite navigation and positioning method for unmanned aerial vehicles in complex environments includes steps 110, 120, 130 and 140.

[0033] Step 110: Collect inertial motion data, real-time image data, and real-time motion estimation data of the unmanned aerial vehicle; In this embodiment, inertial motion data, including angular velocity, acceleration, and attitude angle, is collected by the inertial measurement unit onboard the unmanned aerial vehicle (UAV). Real-time image data is collected by the onboard visual sensors (such as high-definition cameras or fisheye cameras), with the frame rate set to 30-60fps according to the flight speed to ensure the continuity of image features. Real-time motion estimation data, including the instantaneous velocity, attitude change rate, and displacement increment of the UAV, is collected by the optical flow sensor or the motion calculation module of the flight control system. All sensors are time-synchronized to ensure the consistency of data acquisition timing, and the sampling period is uniformly 10ms.

[0034] Step 120: Based on the real-time image data and the preloaded reference map, extract the surface texture features using SLAM technology and match them with the reference map to generate visual positioning data; In some embodiments, the step of extracting surface texture features using SLAM technology and matching them with the reference map based on the real-time image data and a preloaded reference map to generate visual positioning data includes: The real-time image data is input into the SLAM front end to extract feature points from the real-time image data and construct a local map. The local map is registered with the preloaded reference map by feature points, and the pose of the local map in the reference map coordinate system is calculated. The pose is converted into position and attitude information in a geographic coordinate system to generate visual positioning data.

[0035] The real-time image data is input to the SLAM front end, and feature points in the real-time image data are extracted using ORB, SIFT, or SURF feature extraction algorithms. A local map is constructed through feature point tracking and triangulation measurement. The local map contains the three-dimensional spatial coordinates of the feature points and the image pixel coordinates. The local map is then registered with a pre-loaded reference map for feature point registration. The iterative nearest point algorithm or random sampling consensus algorithm is used to remove mismatched feature points. The pose of the local map in the reference map coordinate system is calculated, including position coordinates and attitude angles. The pose is converted from the reference map coordinate system to the position and attitude information in the WGS84 geographic coordinate system through a coordinate transformation matrix to generate visual positioning data. The visual positioning data includes the three-dimensional latitude and longitude, altitude, roll angle, pitch angle, and yaw angle of the UAV.

[0036] Step 130: Based on the real-time motion estimation data, perform dynamic error compensation on the visual positioning data to obtain compensated visual positioning data; In some embodiments, the step of performing dynamic error compensation on the visual positioning data based on the real-time motion estimation data to obtain compensated visual positioning data includes: The real-time motion estimation data is input into the motion compensation model to obtain the visual feature pixel offset caused by the posture change; The position estimate in the visual positioning data is corrected based on the pixel offset to eliminate positioning drift caused by motion blur, thus obtaining compensated visual positioning data.

[0037] In this embodiment, the real-time motion estimation data is input into a pre-trained motion compensation model. This model is trained based on a convolutional neural network. Taking the motion estimation data as input, it outputs the visual feature pixel offset caused by the aircraft's high-speed flight and sudden attitude changes. This offset includes pixel offset values ​​in both the horizontal and vertical directions. Based on the pixel offset, the position estimate in the visual positioning data is linearly corrected using the following formula: in, The compensated position coordinates, Here are the original visual location coordinates, and k is a scaling factor. denoted as pixel offset and f as focal length of the visual sensor, this correction eliminates the positioning drift caused by motion blur, resulting in compensated visual positioning data.

[0038] Step 140: Input the inertial motion data and the compensated visual positioning data into the multi-source data fusion model. The inertial motion data is corrected by the filtering unit to obtain the corrected inertial motion data. The compensated visual positioning data is optimized by the neural network unit to obtain the optimized visual positioning data. The corrected inertial motion data and the optimized visual positioning data are jointly solved to generate the fused positioning result. In some embodiments, the step of correcting the inertial motion data through a filtering unit to obtain corrected inertial motion data includes: The compensated visual positioning data is used as the observation value by the filtering unit, and the state of the inertial motion data is updated by the extended Kalman filter algorithm to obtain the corrected inertial motion data.

[0039] The compensated visual positioning data is used as observations by the filtering unit to establish the state equation and observation equation of inertial navigation. The extended Kalman filter algorithm is used to predict the state and update the measurement of the inertial motion data. The zero bias error and scale factor error of the inertial measurement unit are corrected in real time to obtain the corrected inertial motion data, which effectively suppresses the cumulative error of inertial navigation.

[0040] In some embodiments, the step of performing feature matching optimization on the compensated visual positioning data through a neural network unit to obtain optimized visual positioning data includes: The real-time image data is input into a pre-trained neural network to extract high-dimensional semantic features; The high-dimensional semantic features are matched with features in the baseline map, and the confidence level of the compensated visual positioning data is evaluated and outliers are removed to obtain optimized visual positioning data.

[0041] In some embodiments, the step of jointly solving the corrected inertial motion data and the optimized visual positioning data to generate a fused positioning result includes: The corrected inertial motion data is input into the inertial navigation mechanics equations to construct a system state vector, which includes the three-dimensional position, three-dimensional velocity, quaternion attitude of the unmanned aerial vehicle, and the zero bias error of the inertial measurement unit. The system state vector is updated over time to obtain the predicted state values ​​and their covariance matrix; The optimized visual positioning data is input into the visual observation model, and the confidence factor of the current visual observation is output based on the pre-trained neural network. The noise covariance matrix of the visual observation model is adaptively adjusted based on the confidence factor to obtain the adjusted noise covariance matrix. The predicted state value, covariance matrix, and adjusted noise covariance matrix are input into the Kalman filter update equation to perform measurement update and obtain the posterior estimate of the system state vector. Position and attitude information are extracted from the posterior estimate to generate a fused localization result.

[0042] The corrected inertial motion data is input into the inertial navigation mechanics equations to construct the system state vector. Where is the three-dimensional position, is the three-dimensional velocity, and is the quaternion attitude. For gyroscope zero bias error, This refers to the zero bias error of the accelerometer. The system state vector is updated over time based on the inertial navigation mechanics equations and the state transition matrix to obtain the state prediction value and its covariance matrix; the optimized visual positioning data is input into the visual observation model, and the confidence factor of the current visual observation is output based on the pre-trained confidence evaluation neural network. The confidence factor ranges from 0 to 1, and the larger the value, the higher the reliability of the visual observation data. In some embodiments, the step of adaptively adjusting the noise covariance matrix of the visual observation model according to the confidence factor to obtain the adjusted noise covariance matrix includes: The confidence factor is input into a pre-trained confidence estimation network, and the confidence score is output. The confidence score is compared with a preset threshold, and the noise covariance matrix of the visual observation model is adjusted according to the comparison result to obtain the adjusted noise covariance matrix.

[0043] The confidence factor is input into a pre-trained confidence estimation network to output a confidence score. The confidence score is compared with a preset threshold (usually set to 0.5). If the confidence score is greater than or equal to the preset threshold, the noise covariance matrix is ​​kept unchanged or appropriately reduced to increase the weight of visual observation. If the confidence score is less than the preset threshold, the noise covariance matrix is ​​increased to decrease the weight of visual observation, resulting in an adjusted noise covariance matrix. The state prediction value, covariance matrix, and adjusted noise covariance matrix are input into the Kalman filter update equation to calculate the Kalman gain and perform measurement updates to obtain the posterior estimate and covariance matrix of the system state vector. Three-dimensional position and attitude information are extracted from the posterior estimate to generate a fusion positioning result. The fusion positioning result is output in the form of latitude, longitude, altitude, and attitude angle in a geographic coordinate system for flight control by the flight control system.

[0044] The multi-source data fusion model includes a filtering unit and a neural network unit.

[0045] In this embodiment, by integrating the advantages of inertial navigation and visual navigation, and combining SLAM technology, dynamic error compensation, and a multi-source data intelligent fusion model, the positioning defects of a single navigation method in complex environments are solved. It effectively suppresses the cumulative error of inertial navigation and the environmental sensitivity of visual navigation, and improves the positioning accuracy and continuous working capability of the UAV in complex scenarios such as GPS loss of lock, feature occlusion, and motion ambiguity. It does not require hardware upgrades to a single sensor and has high practicality and cost-effectiveness.

[0046] The composite navigation and positioning method for unmanned aerial vehicles (UAVs) in complex environments provided in this application can be executed by a composite navigation and positioning device for UAVs in complex environments. This application uses an example of a composite navigation and positioning device for UAVs in complex environments executing the composite navigation and positioning method for UAVs in complex environments to illustrate the composite navigation and positioning device for UAVs in complex environments provided in this application.

[0047] This application also provides a composite navigation and positioning device for unmanned aerial vehicles in complex environments, such as... Figure 2 As shown, the composite navigation and positioning device for unmanned aerial vehicles in complex environments includes: a data acquisition module 210, a visual positioning module 220, an optical flow compensation module 230, and a fusion positioning module 240.

[0048] The data acquisition module 210 is used to acquire inertial motion data, real-time image data and real-time motion estimation data of the unmanned aerial vehicle; The visual positioning module 220 is used to extract surface texture features based on the real-time image data and the preloaded reference map, and match them with the reference map to generate visual positioning data. The optical flow compensation module 230 is used to perform dynamic error compensation on the visual positioning data based on the real-time motion estimation data to obtain compensated visual positioning data. The fusion positioning module 240 is used to input the inertial motion data and the compensated visual positioning data into the multi-source data fusion model, correct the error of the inertial motion data through the filtering unit to obtain the corrected inertial motion data, optimize the feature matching of the compensated visual positioning data through the neural network unit to obtain the optimized visual positioning data, and jointly solve the corrected inertial motion data and the optimized visual positioning data to generate the fusion positioning result.

[0049] According to the embodiments of this application, the composite navigation and positioning method for unmanned aerial vehicles in complex environments integrates a multi-source data fusion model that fuses filtering units and neural network units. An extended Kalman filter algorithm is used to accurately correct errors in inertial motion data. Simultaneously, a pre-trained neural network is used to extract high-dimensional semantic features from images, enabling confidence assessment and outlier removal of visual positioning data. This fully leverages the effective information from sensor data, avoids errors in shallow feature matching and interference from outliers on the fusion results, and improves the reliability of positioning data.

[0050] The composite navigation and positioning device for unmanned aerial vehicles in complex environments provided in this application embodiment can achieve... Figure 1 The various processes implemented in the embodiment of the composite navigation and positioning method for unmanned aerial vehicles in complex environments will not be described in detail here to avoid repetition.

[0051] In some embodiments, such as Figure 3 As shown, this application embodiment also provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, it implements the various processes of the above-described embodiment of the composite navigation and positioning method for unmanned aerial vehicles in complex environments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0052] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0053] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described embodiments of the composite navigation and positioning method for unmanned aerial vehicles in complex environments, and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0054] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0055] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described composite navigation and positioning method for unmanned aerial vehicles in complex environments.

[0056] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0057] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described embodiments of the composite navigation and positioning method for unmanned aerial vehicles in complex environments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0058] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a device-level chip, device chip, chip device, or on-chip device chip, etc.

[0059] It should be noted that, in this document, the terms include, encompass, or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, including an element by a statement does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0060] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the complex environment-oriented unmanned aerial vehicle composite navigation and positioning method of the various embodiments of this application.

[0061] In the description of this application, the first feature and the second feature may include one or more of the features.

[0062] In the description of this application, "multiple" means two or more.

[0063] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0064] In the description of this specification, the references to the terms "an embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0065] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A composite navigation and positioning method for unmanned aerial vehicles (UAVs) in complex environments, characterized in that, include: Collect inertial motion data, real-time image data, and real-time motion estimation data of the unmanned aerial vehicle; Based on the real-time image data and the preloaded reference map, surface texture features are extracted using SLAM technology and matched with the reference map to generate visual positioning data. Based on the real-time motion estimation data, dynamic error compensation is performed on the visual positioning data to obtain compensated visual positioning data. The inertial motion data and the compensated visual positioning data are input into the multi-source data fusion model. The inertial motion data is corrected for errors by the filtering unit to obtain the corrected inertial motion data. The compensated visual positioning data is optimized by feature matching by the neural network unit to obtain the optimized visual positioning data. The corrected inertial motion data and the optimized visual positioning data are jointly solved to generate the fused positioning result. The multi-source data fusion model includes a filtering unit and a neural network unit.

2. The composite navigation and positioning method for unmanned aerial vehicles in complex environments according to claim 1, characterized in that, Based on the real-time image data and the pre-loaded reference map, surface texture features are extracted using SLAM technology and matched with the reference map to generate visual positioning data, including: The real-time image data is input into the SLAM front end to extract feature points from the real-time image data and construct a local map. The local map is registered with the preloaded reference map by feature points, and the pose of the local map in the reference map coordinate system is calculated. The pose is converted into position and attitude information in a geographic coordinate system to generate visual positioning data.

3. The composite navigation and positioning method for unmanned aerial vehicles in complex environments according to claim 2, characterized in that, The step of performing dynamic error compensation on the visual positioning data based on the real-time motion estimation data to obtain compensated visual positioning data includes: The real-time motion estimation data is input into the motion compensation model to obtain the visual feature pixel offset caused by the posture change; The position estimate in the visual positioning data is corrected based on the pixel offset to eliminate positioning drift caused by motion blur, thus obtaining compensated visual positioning data.

4. The composite navigation and positioning method for unmanned aerial vehicles in complex environments according to claim 3, characterized in that, The step of correcting the inertial motion data through a filtering unit to obtain corrected inertial motion data includes: The compensated visual positioning data is used as the observation value by the filtering unit, and the state of the inertial motion data is updated by the extended Kalman filter algorithm to obtain the corrected inertial motion data.

5. The composite navigation and positioning method for unmanned aerial vehicles in complex environments according to claim 4, characterized in that, The step of performing feature matching optimization on the compensated visual positioning data through a neural network unit to obtain optimized visual positioning data includes: The real-time image data is input into a pre-trained neural network to extract high-dimensional semantic features; The high-dimensional semantic features are matched with features in the baseline map, and the confidence level of the compensated visual positioning data is evaluated and outliers are removed to obtain optimized visual positioning data.

6. The composite navigation and positioning method for unmanned aerial vehicles in complex environments according to claim 5, characterized in that, The step of jointly solving the corrected inertial motion data and the optimized visual positioning data to generate a fused positioning result includes: The corrected inertial motion data is input into the inertial navigation mechanics equations to construct a system state vector, which includes the three-dimensional position, three-dimensional velocity, quaternion attitude of the unmanned aerial vehicle, and the zero bias error of the inertial measurement unit. The system state vector is updated over time to obtain the predicted state values ​​and their covariance matrix; The optimized visual positioning data is input into the visual observation model, and the confidence factor of the current visual observation is output based on the pre-trained neural network. The noise covariance matrix of the visual observation model is adaptively adjusted based on the confidence factor to obtain the adjusted noise covariance matrix. The predicted state value, covariance matrix, and adjusted noise covariance matrix are input into the Kalman filter update equation to perform measurement update and obtain the posterior estimate of the system state vector. Position and attitude information are extracted from the posterior estimate to generate a fused localization result.

7. The composite navigation and positioning method for unmanned aerial vehicles in complex environments according to claim 6, characterized in that, The step of adaptively adjusting the noise covariance matrix of the visual observation model according to the confidence factor to obtain the adjusted noise covariance matrix includes: The confidence factor is input into a pre-trained confidence estimation network, and the confidence score is output. The confidence score is compared with a preset threshold, and the noise covariance matrix of the visual observation model is adjusted according to the comparison result to obtain the adjusted noise covariance matrix.

8. A composite navigation and positioning device for unmanned aerial vehicles (UAVs) in complex environments, implemented using the composite navigation and positioning method for UAVs in complex environments as described in any one of claims 1 to 7, characterized in that... The device includes: The data acquisition module is used to collect inertial motion data, real-time image data, and real-time motion estimation data of the unmanned aerial vehicle. The visual positioning module is used to extract surface texture features based on the real-time image data and the preloaded reference map, and match them with the reference map to generate visual positioning data. The optical flow compensation module is used to perform dynamic error compensation on the visual positioning data based on the real-time motion estimation data to obtain compensated visual positioning data. The fusion positioning module is used to input the inertial motion data and the compensated visual positioning data into the multi-source data fusion model, correct the inertial motion data through the filtering unit to obtain corrected inertial motion data, optimize the compensated visual positioning data through the neural network unit to obtain optimized visual positioning data, and jointly solve the corrected inertial motion data and the optimized visual positioning data to generate the fusion positioning result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the composite navigation and positioning method for unmanned aerial vehicles in complex environments as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the composite navigation and positioning method for unmanned aerial vehicles in complex environments as described in any one of claims 1 to 7.

Images