Unmanned aerial vehicle synchronous positioning and mapping method, device and equipment and storage medium

Abstract

The invention relates to the technical field of unmanned aerial vehicle synchronous positioning and mapping, and discloses an unmanned aerial vehicle synchronous positioning and mapping method and device, equipment and a storage medium, and the method comprises the steps: obtaining image data collected in a low-altitude environment without an unmanned aerial vehicle; performing four-dimensional Gaussian initialization based on the image data; rendering four-dimensional Gaussian on the basis of a four-dimensional Gaussian rendering technology to obtain a map represented by four-dimensional Gaussian; if new image data is obtained; obtaining camera pose estimation based on a speedometer and the new image data, and updating the optimal parameters of the camera pose based on the re-projection error; densifying and filtering the four-dimensional Gaussian; and updating a map represented by the four-dimensional Gaussian based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimation. According to the method, the unmanned aerial vehicle can be ensured to perform high-precision and high-robustness autonomous pose estimation in an outdoor dynamic large-scale environment, and the dynamic spatial-temporal characteristics of the environment can be captured for mapping, so that the real-time environment detection and analysis requirements of a field emergency scene can be met.

Description

Technical Field

[0001] The present invention relates to the technical field of drone synchronous positioning and mapping, and in particular to a drone synchronous positioning and mapping method, device, equipment and storage medium. Background Art

[0002] Global extreme climate change has brought great challenges to forest fire prevention and control. In particular, with the acceleration of urbanization, the interaction between human activities and forests has become more frequent, and the frequency and intensity of forest fires have gradually increased, posing a huge threat to natural ecology, social economy, and residents' health. Firefighters are the main force in forest fire rescue. However, the complex geographical environment and changeable meteorological conditions make the fire scene highly uncertain, making it difficult for firefighters to carry out safe and efficient firefighting operations, resulting in the evolution of the fire into a larger scale and even causing casualties. Therefore, it is necessary to detect forest wildfires through perception technology to provide comprehensive information for forest fire air defense. Traditional forest scene perception technology is mainly based on remote sensing satellites and observation tower platforms. These technologies have limitations in field fire detection performance. For example, the field of view of the observation tower is limited and the construction cost is high. At the same time, when a forest fire occurs, it is also vulnerable to damage by the fire, resulting in additional maintenance costs. Remote sensing satellites have the advantage of a wide monitoring field of view. However, the satellites have the disadvantages of high cost, long replay cycle, and limited image resolution. This makes it very difficult to monitor forest fires in specific high-risk areas in real time.

[0003] Mapping technology based on real-time drone images is expected to overcome the above limitations and is considered to be the most promising method for future field monitoring. This technology uses low-altitude drones to obtain color and depth images of forest scenes, and then completes the three-dimensional reconstruction of the target scene in an offline environment through point cloud extraction and surface reconstruction. These solutions can provide emergency commanders with high-precision and high-fidelity forest scene perception, and support core services such as forest fire spread prediction and risk warning, thereby helping forest fires to be "fighted early, small, and fought", thereby achieving the goal of improving firefighting efficiency and reducing disaster losses. In addition, since it does not use expensive lidar to provide geometric information, but uses more economical and portable depth cameras, this technology is also more likely to be quickly implemented in the field of emergency rescue.

[0004] However, existing simultaneous positioning and mapping solutions based on depth cameras still have technical shortcomings in the field of UAV low-altitude remote sensing. First, these solutions are mainly designed for indoor scenes, and it is difficult to accurately track and build high-precision maps in large-scale outdoor low-altitude environments; second, most of the existing vision-based tracking solutions only use matched pairs of images for camera pose estimation, which is difficult to provide long-term point tracking to cope with the dynamic characteristics of outdoor environments; third, the existing simultaneous mapping solutions mainly use methods such as artifact filtering to perform three-dimensional modeling for static scenes, which is difficult to fully capture the dynamic features in the scene, which are crucial for the analysis and deduction of emergency response. Summary of the invention

[0005] Based on this, it is necessary to address the technical problem that the existing technology has poor synchronous positioning and mapping effects in the field of low-altitude remote sensing of unmanned aerial vehicles, and propose a method, device, equipment and storage medium for synchronous positioning and mapping of unmanned aerial vehicles. In the first aspect, a method for synchronous positioning and mapping of unmanned aerial vehicles is provided, and the method includes:

[0006] Acquire image data collected by drones in low-altitude environments;

[0007] Performing four-dimensional Gaussian initialization based on the image data;

[0008] Rendering the four-dimensional Gaussian based on the four-dimensional Gaussian rendering technology to obtain a map represented by the four-dimensional Gaussian;

[0009] If new image data is obtained, the camera pose estimation is obtained based on the odometer, the image gradient sampling method and the new image data, and the optimal parameters of the camera pose are updated based on the reprojection error;

[0010] The four-dimensional Gaussian is densified and filtered, and the map represented by the four-dimensional Gaussian is updated based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimate.

[0011] In a second aspect, a UAV synchronous positioning and mapping device is provided, the device comprising:

[0012] An acquisition module is used to acquire image data collected by drones in low-altitude environments;

[0013] An initialization module, used for performing four-dimensional Gaussian initialization based on the image data;

[0014] A rendering module, used for rendering the four-dimensional Gaussian based on the four-dimensional Gaussian rendering technology to obtain a map represented by the four-dimensional Gaussian;

[0015] A first updating module is used for obtaining a camera pose estimate based on the odometer, the image gradient sampling method and the new image data if new image data is obtained, and updating the optimal parameters of the camera pose based on the reprojection error;

[0016] The second updating module is used to densify and filter the four-dimensional Gaussian, and update the map represented by the four-dimensional Gaussian based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimation.

[0017] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for synchronous positioning and mapping of a UAV when executing the computer program.

[0018] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned drone synchronous positioning and mapping method are implemented.

[0019] The method for synchronous positioning and mapping of unmanned aerial vehicles proposed in the present invention obtains image data collected by unmanned aerial vehicles in a low-altitude environment, and then performs four-dimensional Gaussian initialization based on the image data, and then renders the four-dimensional Gaussian based on the four-dimensional Gaussian rendering technology to obtain a map represented by the four-dimensional Gaussian. If new image data is obtained, the camera pose estimation is obtained based on the odometer, the image gradient sampling method and the new image data, and the optimal parameters of the camera pose are updated based on the reprojection error. Finally, the four-dimensional Gaussian is densified and filtered, and the map represented by the four-dimensional Gaussian is updated based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimation. The present invention can ensure that the unmanned aerial vehicle performs high-precision and high-robustness autonomous pose estimation in an outdoor dynamic large-scale environment, and can also capture the dynamic spatiotemporal characteristics of the environment for mapping, so as to meet the real-time environmental detection and analysis needs of outdoor emergency scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] in:

[0022] Figure 1 A diagram of an application environment of a method for synchronous positioning and mapping of a drone in an embodiment;

[0023] Figure 2 Flow chart of a method for synchronous positioning and mapping of a drone in one embodiment;

[0024] Figure 3 It is a structural block diagram of a UAV synchronous positioning and mapping device in one embodiment;

[0025] Figure 4 is a structural block diagram of a computer device in one embodiment;

[0026] Figure 5 It is a structural block diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0028] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] The method for synchronous positioning and mapping of unmanned aerial vehicles provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, the client 110 communicates with the server 120 through a network. The server 120 can obtain image data collected by a drone in a low-altitude environment through the client 110, and then initialize the four-dimensional Gaussian based on the image data, and then render the four-dimensional Gaussian based on the four-dimensional Gaussian rendering technology to obtain a map represented by the four-dimensional Gaussian. Then, if new image data is obtained, the camera pose estimation is obtained based on the odometer, the image gradient sampling method and the new image data, and the optimal parameters of the camera pose are updated based on the reprojection error. Finally, the four-dimensional Gaussian is densified and filtered, and the map represented by the four-dimensional Gaussian is updated based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimation. The present invention can ensure that the drone performs high-precision and high-robustness autonomous pose estimation in an outdoor dynamic large-scale environment, and can capture the dynamic spatiotemporal characteristics of the environment for mapping, so as to meet the real-time environment detection and analysis requirements of outdoor emergency scenes. . Among them, the client 110 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server 120 can be implemented by an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0031] See also Figure 2 As shown, Figure 2 A flowchart of a method for synchronous positioning and mapping of a drone provided by an embodiment of the present invention includes the following steps:

[0032] Step S101: Acquire image data collected by a drone in a low-altitude environment;

[0033] In this embodiment, the image data can be collected in a low-altitude forest environment, the simulation environment data is used to simulate various properties that may appear in real scenes, and the performance of the algorithm is efficiently tested. The real-time data is used to verify the performance of the mapping algorithm and the simulation data set. The main collection objects include the color image and depth image of the drone depth camera. The simulation data set is collected by the drone simulation software Airsim for the forest environment (Electric Dreams Environment) modeled by the virtual engine (Unreal Engine5), which is divided into different heights such as 5 meters to 50 meters, straight lines, surrounds, random and other different collection routes. The real-world collection equipment consists of the Xinjiang M350RTK drone equipped with visible light cameras and depth cameras and other collection equipment. The DJI M350RTK drone is a professional-grade multi-rotor aircraft with high stability and accurate positioning capabilities. The RTK technology (real-time motion positioning) of the DJI M350RTK enables it to provide high-precision location information in low-altitude environments, which is essential for data collection and map making. The visible light camera and the depth camera provide RGB-D images to generate accurate three-dimensional point cloud data. These data contain three-dimensional coordinate information, which is suitable for accurate mapping of urban features such as buildings, roads, and vegetation. The system is capable of operating as soon as it is turned on, making it suitable for a variety of applications. Data collection process: The drone takes off and performs a carpet-scanning flight in a low-altitude forest environment, while collecting posture information and RGB-D images, and transmitting the image data in a timestamp sequence to the synchronous positioning and mapping system.

[0034] Step S102: performing four-dimensional Gaussian initialization based on the image data;

[0035] In this embodiment, for the first frame, the tracking step is skipped and the camera pose is set to the unit matrix, that is, the current position is regarded as the origin of the world coordinate system. In the mapping part, all pixels in the first frame are used to initialize a new four-dimensional Gaussian. The mean and covariance matrix ∑ of each Gaussian are parameterized as four scalars μ = (μ x , μ y , μ z , μ t ) and a four-dimensional covariance matrix ∑ = RSS T R T Specifically, the four-dimensional Gaussian is marginalized along XYZT to obtain one-dimensional Gaussian distributions in X, Y, Z, and T dimensions. The mean of these one-dimensional Gaussian distributions is μ x , μ y , μ z , μ t, represents the mean of the numerical distribution of the four-dimensional Gaussian on the X-axis, Y-axis, Z-axis and T-axis, that is, the position mean of the Gaussian in three-dimensional space and time, while R in the covariance matrix is ​​the four-dimensional rotation matrix, and S is the scale factor. Through this parameterization, the four-dimensional Gaussian can be described by the shape and position of the ellipsoid, and can be rotated arbitrarily in space and time. This representation method can flexibly capture complex motion changes in dynamic scenes.

[0036] Step S103: Rendering the four-dimensional Gaussian based on the four-dimensional Gaussian rendering technology to obtain a map represented by the four-dimensional Gaussian;

[0037] In this embodiment, based on the four-dimensional Gaussian representation, a high-fidelity reconstructed RGB-D image is obtained by a four-dimensional Gaussian rendering technique as a map of the four-dimensional Gaussian representation. Specifically, the four-dimensional Gaussian rendering technique is used to obtain a high-fidelity reconstructed RGB-D image by giving a time t (0 time when initialized) and a viewing angle Assume that there are N Gaussians in the scene, for the i-th four-dimensional Gaussian p i (x, y, z, t), which is first decomposed into a conditional three-dimensional Gaussian p i (x, y, z|t) (characterizing the probability distribution of the Gaussian in three-dimensional space at time t) and an edge one-dimensional Gaussian p i (t) (characterizing the probability distribution of the Gaussian in time). Then, the conditional three-dimensional Gaussian is projected onto a two-dimensional pixel plane to obtain the two-dimensional Gaussian distribution p of the Gaussian at the (u, v) pixel at time t i (u, v|t). Finally, render these N Gaussians to obtain the visual perspective The RGB-D image under needs to be fused with the color c of the Gaussian that changes with time t and viewing angle d (three-dimensional vector). i (d, t), two-dimensional Gaussian distribution p i (u, v|t), one-dimensional time Gaussian distribution p i (t) and the opacity α of the Gaussian point i .

[0038] Step S104: if new image data is obtained, obtaining a camera pose estimate based on the odometer, the image gradient sampling method and the new image data, and updating the optimal parameters of the camera pose based on the reprojection error;

[0039] In this embodiment, the camera pose tracking aims to estimate the camera pose of the currently incoming new image data, so as to minimize the image and depth reconstruction errors of the new image data relative to the camera pose parameters within time t+1.

[0040] In one embodiment, step S104 includes: the odometer selects key points of new image data using an image gradient sampling method, performs bidirectional key point tracking in the most recent multiple frames to obtain the camera pose, and minimizes the reprojection error through gradient optimization to update the optimal parameters of the camera pose.

[0041] In this embodiment, first, the odometer uses the image gradient sampling method to select key points to ensure the uniqueness and good distribution of features. Subsequently, the tracking model performs bidirectional key point tracking in the most recent multiple frames, and uses trajectory quality evaluation for efficient filtering, retaining only high-quality points for optimization. The basis for filtering is based on the uncertainty, visibility, and dynamics of the point trajectory. Afterwards, the reprojection error is minimized by gradient optimization to update the optimal parameters for the camera pose. Sliding window bundling adjustment can be used to optimize the camera pose and three-dimensional point position to ensure the reliability of the trajectory.

[0042] Specifically, assuming that the camera intrinsic parameter is K, the current image sequence is represented by S BA , where the external parameter of the i-th image is T i , the external parameter of the jth image is T j , the position of the nth feature point in the ith image is x i，n , corresponding to the depth d i，n First, the two-dimensional pixel position model of the nth feature point projected to other images (i.e., the jth image) is obtained through the long-term arbitrary point tracking visual odometer i→j (x i，n ), and calculate the corresponding two-dimensional pixel position by reprojection And calculate the distance error between the two positions. For each feature point in i, the distance error is calculated and weighted summed to obtain the reprojection error from the i-th image to the j-th image. Further, calculate S BA The reprojection error between all image pairs in the image sequence is summed up and the formula for the reprojection error is summarized as follows:

[0043]

[0044] Among them, ||·|| ρ is the distance metric, using Manhattan distance, with weight w i→j，n It is derived from the calculation result of the long-term arbitrary point tracking visual odometry, which indicates the importance of the reprojection error from the ith image to the jth image to the optimization of the pose and 3D points.

[0045] Step S105: densify and filter the four-dimensional Gaussian, and update the map represented by the four-dimensional Gaussian based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimation.

[0046] In this embodiment, after the camera pose of the newly incoming image frame is determined, four-dimensional Gaussian densification and filtering are performed. The purpose of densification is to add new Gaussian bodies to fit previously unseen scene elements or to enhance the fitting effect. The purpose of filtering is to remove abnormal Gaussians and Gaussians with poor fitting effects. In terms of Gaussian filtering, due to the use of inverse depth, the calculated points will be unevenly distributed and contain some outliers, which will cause errors in the optimization. To address this problem, a selection method is proposed to sample points and remove outliers, as follows: 1) Divide the image into g×g blocks. 2) Obtain the depth average d of each block mean and standard deviation d std . 3) Only keep one Gaussian with the maximum weight and whose depth mean is d, where d satisfies |dd min |<3d std , and delete other Gaussians in this block. For Gaussian densification, the mask and surface information are used to guide which pixels should be densified. Masks are added to locations where the map is not dense enough, or where new geometry should be in front of the current estimated geometry (i.e., the ground truth depth is in front of the predicted depth, and the depth error is greater than 50 times the median depth error (MDE)). Next, the rendering gradient value and the number of Gaussians at the current pixel are calculated. If the current pixel satisfies the requirement to have a mask added, and the rendering gradient value is higher than the median gradient value of the current keyframe and the Gaussian is dense, a new Gaussian is added following the same process as the first frame initialization.

[0047] After determining the camera pose and the number and position of Gaussians, real-time mapping is performed. This step aims to update the parameters of the four-dimensional Gaussian map based on the currently estimated online camera pose set so that the reprojection error is minimized and the model fits the real scene best. This is done through differentiable rendering and gradient-based optimization. In this step, the camera pose is fixed and the parameters of the Gaussians are updated. Hot-start optimization is started only from the latest constructed map. Instead of optimizing all previous keyframes, frames that may affect the newly added Gaussians are selected. Each latest frame is selected as a keyframe, and k frames are selected for optimization, including the current frame, the most recent keyframe, and the k-2 previous keyframes with the highest overlap with the current frame. The degree of overlap is determined by obtaining the point cloud of the depth map of the current frame and the number of points in the frustum of each keyframe. In this stage, the target loss function is optimized using gradient descent. The loss function is mainly used to optimize the reconstruction and rendering quality of dynamic scenes. It combines multiple loss terms, including L1 loss and structural similarity loss SSIM, to ensure that the difference between the generated image and the real image is minimized while maintaining the smoothness and consistency of the image.

[0048] The present invention has the following advantages: 1. Low cost: It can realize real-time detection of low-altitude environments within 50 meters without using radar, and has the characteristic of low cost. 2. Robustness: The proposed camera pose estimation algorithm has good robustness in the face of outdoor dynamic scenes, and can accurately distinguish between static elements and dynamic elements in the scene, and realize high-precision and high-robustness tracking and camera pose estimation, thereby providing accurate geographic information reference for the three-dimensional model. 3. High efficiency: The proposed outdoor dynamic scene mapping algorithm can perform real-time three-dimensional reconstruction of outdoor dynamic scenes such as forests and fields at a theoretical rendering speed of up to 400 frames per second, greatly improving the reconstruction speed, thereby meeting the needs of rapid modeling of emergency scenes. 4. It can provide more accurate geographic environment information for forest fire prevention and control and decision-making, so as to enhance the intelligence and accuracy of forest fire prevention and control. In addition, the solution can also be used in multiple application fields such as urban planning, environmental monitoring, cultural heritage protection, and emergency response.

[0049] See also Figure 3 As shown, in one embodiment, a UAV synchronous positioning and mapping device is provided, the device comprising:

[0050] An acquisition module 10 is used to acquire image data collected by a drone in a low-altitude environment;

[0051] An initialization module 20, configured to perform four-dimensional Gaussian initialization based on the image data;

[0052] A rendering module 30, used for rendering a four-dimensional Gaussian based on a four-dimensional Gaussian rendering technique to obtain a map represented by a four-dimensional Gaussian;

[0053] A first updating module 40 is used to obtain a camera pose estimate based on the odometer, the image gradient sampling method and the new image data if new image data is obtained, and to update the optimal parameters of the camera pose based on the reprojection error;

[0054] The second updating module 50 is used to densify and filter the four-dimensional Gaussian, and update the map represented by the four-dimensional Gaussian based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimation.

[0055] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the service side of a method for synchronous positioning and mapping of a drone.

[0056] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the client side of a method for synchronous positioning and mapping of a drone.

[0057] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0058] Acquire image data collected by drones in low-altitude environments;

[0059] Performing four-dimensional Gaussian initialization based on the image data;

[0060] Rendering the four-dimensional Gaussian based on the four-dimensional Gaussian rendering technology to obtain a map represented by the four-dimensional Gaussian;

[0061] If new image data is obtained, the camera pose estimation is obtained based on the odometer, the image gradient sampling method and the new image data, and the optimal parameters of the camera pose are updated based on the reprojection error;

[0062] The four-dimensional Gaussian is densified and filtered, and the map represented by the four-dimensional Gaussian is updated based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimate.

[0063] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0064] Acquire image data collected by drones in low-altitude environments;

[0065] Performing four-dimensional Gaussian initialization based on the image data;

[0066] Rendering the four-dimensional Gaussian based on the four-dimensional Gaussian rendering technology to obtain a map represented by the four-dimensional Gaussian;

[0067] If new image data is obtained, the camera pose estimation is obtained based on the odometer, the image gradient sampling method and the new image data, and the optimal parameters of the camera pose are updated based on the reprojection error;

[0068] The four-dimensional Gaussian is densified and filtered, and the map represented by the four-dimensional Gaussian is updated based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimate.

[0069] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0070] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0071] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0072] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for simultaneous positioning and mapping of unmanned aerial vehicles, characterized in that: The method for synchronous positioning and mapping of unmanned aerial vehicles comprises: Acquire image data collected by drones in low-altitude environments; Performing four-dimensional Gaussian initialization based on the image data; Rendering the four-dimensional Gaussian based on the four-dimensional Gaussian rendering technology to obtain a map represented by the four-dimensional Gaussian; If new image data is obtained, the camera pose estimation is obtained based on the odometer, the image gradient sampling method and the new image data, and the optimal parameters of the camera pose are updated based on the reprojection error; The four-dimensional Gaussian is densified and filtered, and the map represented by the four-dimensional Gaussian is updated based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimate.

2. The method for simultaneous positioning and mapping of unmanned aerial vehicles according to claim 1, characterized in that: The step of performing four-dimensional Gaussian initialization based on the image data comprises: All pixels in the first frame of the image data are used to initialize a new four-dimensional Gaussian, where the mean and covariance matrix ∑ of each four-dimensional Gaussian is parameterized as four scalars μ = (μ x , μ y , μ z , μ t ) and a four-dimensional covariance matrix ∑ = RSS T R T Specifically, μ x , μ y , μ z , μ t They refer to the means of the one-dimensional Gaussian distributions in X, Y, Z and T dimensions obtained by marginalizing the four-dimensional Gaussian along the X, Y, Z and T axes respectively. The R in the covariance matrix is ​​the four-dimensional rotation matrix, and S is the scale scaling factor.

3. The method for simultaneous positioning and mapping of unmanned aerial vehicles according to claim 1, characterized in that: If new image data is obtained, the steps of obtaining a camera pose estimate based on the odometer, the image gradient sampling method and the new image data, and updating the optimal parameters of the camera pose based on the reprojection error include: The odometer uses an image gradient sampling method to select key points of new image data, performs bidirectional key point tracking in the most recent multiple frames to obtain the camera pose, and minimizes the reprojection error through gradient optimization to update the optimal parameters of the camera pose.

4. A UAV synchronous positioning and mapping device, characterized in that: The UAV synchronous positioning and mapping device comprises: An acquisition module is used to acquire image data collected by drones in low-altitude environments; An initialization module, used for performing four-dimensional Gaussian initialization based on the image data; A rendering module, used for rendering the four-dimensional Gaussian based on the four-dimensional Gaussian rendering technology to obtain a map represented by the four-dimensional Gaussian; A first updating module is used for obtaining a camera pose estimate based on the odometer, the image gradient sampling method and the new image data if new image data is obtained, and updating the optimal parameters of the camera pose based on the reprojection error; The second updating module is used to densify and filter the four-dimensional Gaussian, and update the map represented by the four-dimensional Gaussian based on the densified and filtered four-dimensional Gaussian and the updated camera pose estimation.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for simultaneous positioning and mapping of a drone as described in any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for simultaneous positioning and mapping of a drone as claimed in any one of claims 1 to 3 are implemented.

Images