Monocular SLAM static initialization method, device, drone and storage medium

Abstract

The present invention discloses a monocular SLAM static initialization method, device, drone and storage medium, the method comprising: obtaining a monocular image of a two-dimensional code target; based on the feature points of the two-dimensional code target in the monocular image, using the PnP algorithm to obtain the pose of the camera coordinate system in the two-dimensional code world coordinate system and the scale information of the feature points; using the PlaneRecNet model to obtain a depth map and multiple planes, using the RANSAC algorithm to calculate the equation and normal vector of each plane; screening multiple normal vectors that meet the orthogonality of each other from the normal vectors, thereby determining the normal vector corresponding to the horizontal plane, and obtaining the gravity vector; rotating the camera coordinate system so that its Z-axis direction is parallel to the gravity vector direction, and obtaining the world coordinate system; completing the initialization based on the world coordinate system, the depth map and the scale information. The above scheme realizes the static initialization of monocular vision SLAM, while restoring the scale information and realizing gravity alignment.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle positioning and navigation, and in particular to a monocular SLAM static initialization method, a device, an unmanned aerial vehicle and a storage medium. Background Art

[0002] When visual SLAM performs positioning and navigation tasks, it often needs to make initial estimates of some specific parameters at the beginning of startup. This parameter estimation process is called initialization. Accurate initialization results can improve the accuracy and solution speed of SLAM pose estimation and map construction; conversely, wrong or inaccurate initialization results will affect the performance of visual SLAM.

[0003] For monocular vision SLAM, monocular initialization is required. Since the traditional monocular method cannot estimate the depth of map points, the traditional monocular initialization requires mobile initialization, obtaining multiple image frame sequences, and restoring the depth of map points by triangulation. This will make SLAM initialization inconvenient. At the same time, the monocular lacks scale information, and without IMU, it is impossible to estimate the gravity vector for gravity alignment. In order to restore the scale information, it is generally necessary to combine the trajectory obtained by IMU integration to restore the scale of the monocular trajectory, and it is also necessary to rely on IMU to obtain the gravity vector for gravity alignment of the posture.

[0004] Therefore, a new initialization scheme needs to be proposed to overcome the above problems. Summary of the invention

[0005] In view of the above problems, the present invention is proposed to provide a monocular SLAM static initialization method, device, drone and storage medium that overcome the above problems or at least partially solve the above problems.

[0006] According to one aspect of the present invention, a monocular SLAM static initialization method is provided, the method comprising:

[0007] Obtain a monocular image of the QR code target;

[0008] Based on multiple feature points of the two-dimensional code target in the monocular image, a PnP algorithm is used to obtain the position and scale information of the feature points of the camera coordinate system in the two-dimensional code world coordinate system;

[0009] The PlaneRecNet model is used to estimate the depth of the monocular image, output a depth map and extract multiple planes, and the equation and normal vector of each plane are calculated using the RANSAC algorithm;

[0010] Filtering out a plurality of normal vectors that are orthogonal to each other from the normal vectors, thereby determining a normal vector corresponding to a horizontal plane, and determining the normal vector of the horizontal plane as a gravity vector;

[0011] Rotate the camera coordinate system so that its Z-axis direction is parallel to or coincides with the gravity vector direction, and determine the rotated camera coordinate system as the world coordinate system;

[0012] Initialization is completed based on the world coordinate system, the depth map and the scale information.

[0013] In some embodiments, the two-dimensional target includes five different two-dimensional codes, and the five two-dimensional codes are designed as follows: one two-dimensional code is placed in the center of the two-dimensional code target plate, and the other four are placed at the four corners of the central two-dimensional code.

[0014] In some embodiments, based on multiple feature points of the two-dimensional code target in the monocular image, obtaining the pose of the camera coordinate system in the two-dimensional code world coordinate system and the scale information of the feature points using the PnP algorithm includes:

[0015] Taking the vertices on the two-dimensional code as feature points, and obtaining the projection coordinates of each feature point in a normalized camera coordinate system through two-dimensional code detection;

[0016] The position and posture of the camera coordinate system in the two-dimensional code world coordinate system are determined by using the feature points, projection coordinates and the minimization of the reprojection error calculation in the PnP algorithm, so as to restore the scale information of each feature point according to the two-dimensional code.

[0017] In some embodiments, acquiring a monocular image of a two-dimensional code target includes:

[0018] The two-dimensional code target is photographed in an indoor environment to obtain the monocular image.

[0019] In some embodiments, rotating the camera coordinate system so that its Z-axis direction is parallel to or coincides with the gravity vector direction, and determining the rotated camera coordinate system as the world coordinate system further includes:

[0020] The world coordinate system is determined with the optical center of the camera as the origin, and the subsequent camera frame poses and map point positions are unified under the world coordinate system.

[0021] In some implementations, completing initialization based on the world coordinate system, the depth map, and the scale information includes:

[0022] Based on the depth map output by the PlaneRecNet model and the scale information obtained by two-dimensional code detection, the scale information of the depth map is restored.

[0023] In some implementations, completing the initialization based on the world coordinate system, the depth map, and the scale information further includes:

[0024] Matching the depth map from which scale information has been restored and the monocular image from which feature points have been extracted to obtain positions of corresponding feature points in the world coordinate system;

[0025] Based on the positional relationship between other points in the monocular image and the feature point, the map point of the other point in the world coordinate system is determined, thereby realizing the static initialization of the monocular SLAM.

[0026] According to another aspect of the present invention, a monocular SLAM static initialization device is provided, the device comprising:

[0027] An image acquisition module, adapted to acquire a monocular image of a two-dimensional code target;

[0028] A posture determination module, adapted to obtain the posture of the camera coordinate system in the two-dimensional code world coordinate system and scale information of the feature points using a PnP algorithm based on multiple feature points of the two-dimensional code target in the monocular image;

[0029] A plane extraction module, adapted to estimate the depth of the monocular image using a PlaneRecNet model, output a depth map and extract multiple planes, and calculate the equation and normal vector of each plane using a RANSAC algorithm;

[0030] a vector determination module, adapted to select a plurality of normal vectors that are orthogonal to each other from the normal vectors, thereby determining a normal vector corresponding to a horizontal plane, and determining the normal vector of the horizontal plane as a gravity vector;

[0031] A coordinate determination module, adapted to rotate the camera coordinate system so that its Z-axis direction is parallel to or coincides with the gravity vector direction, and determine the rotated camera coordinate system as a world coordinate system;

[0032] An initialization module is adapted to complete initialization based on the world coordinate system, the depth map and the scale information.

[0033] According to another aspect of the present invention, there is provided an unmanned aerial vehicle, comprising: a processor and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the monocular SLAM static initialization method according to any one of the above-mentioned embodiments.

[0034] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the monocular SLAM static initialization method according to any one of the above items is implemented.

[0035] From the above, it can be seen that according to the monocular SLAM static initialization method disclosed in the present invention, the pose and scale information of the camera coordinate system are first determined based on the feature points in the monocular image and the PnP algorithm, and then the depth map and gravity vector are determined based on the PlaneRecNet model and the RANSAC algorithm, and then the world coordinate system is determined based on the gravity vector. Finally, the static initialization of SLAM is realized based on the monocular camera and the monocular image, and the scale information and gravity alignment are restored without the aid of IMU.

[0036] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0038] Figure 1 A schematic diagram of a process flow of a monocular SLAM static initialization method according to some embodiments of the present invention is shown;

[0039] Figure 2 A schematic structural diagram of a two-dimensional code target plate according to some embodiments of the present invention is shown;

[0040] Figure 3 shows a depth map obtained by monocular depth estimation using a PlaneRecNet model according to some embodiments of the present invention;

[0041] Figure 4 shows an example diagram of a horizontal plane detected according to some embodiments of the present invention;

[0042] Figure 5 The overall process diagram of the monocular SLAM static initialization method according to some embodiments of the present invention is shown;

[0043] Figure 6 A schematic diagram of the structure of a monocular SLAM static initialization device according to some embodiments of the present invention is shown;

[0044] Figure 7 A schematic structural diagram of a drone according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0045] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0046] The main idea of ​​the embodiments of the present invention is: in view of the fact that mobile initialization is not suitable for some commercial products, and user operation is inconvenient, and mobile initialization may lead to initialization failure if the mobile operation is not standard, a monocular static initialization method is proposed, so that monocular vision SLAM can be statically initialized, and can restore scale information without relying on IMU, and gravity alignment is performed at the same time, so that the SLAM posture can be directly used by intelligent entities such as drones.

[0047] Glossary:

[0048] World Coordinates Definition: The world coordinate system is a global three-dimensional coordinate system used to describe the position of objects in the real world. It is usually represented by (X_w, Y_w, Z_w).

[0049] Camera Coordinates Definition: The camera coordinate system is a three-dimensional coordinate system centered on the camera, which describes the position of the object relative to the camera. It is usually represented by (X_c, Y_c, Z_c).

[0050] Transformation: The transformation from the world coordinate system to the camera coordinate system is usually completed through the camera's external parameter matrix (rotation matrix R and translation vector t).

[0051] Normalized Image Coordinates definition: The normalized image coordinate system is a special form of the camera coordinate system that removes depth information but retains direction information.

[0052] Aruco QR code detection is a computer vision technology used to detect and identify Aruco QR codes, which consist of black and white squares with unique identifiers. Aruco QR code detection can locate and identify these markers in images or videos, thereby achieving tracking and positioning of objects or scenes.

[0053] The PlaneRecNet model is a multi-task convolutional neural network that integrates a single-level instance segmentation network for piecewise plane segmentation and a deep decoder to reconstruct scenes from a single RGB image. It is capable of cross-task consistent multi-task learning for piecewise plane detection and reconstruction from a single RGB image, achieving depth estimation and plane detection for a single image.

[0054] RANSAC algorithm is the abbreviation of Random Sample Consensus algorithm. It is an algorithm that calculates the mathematical model parameters of data based on a set of sample data sets containing abnormal data to obtain valid sample data. It was first proposed by Fischler and Bolles in 1981; RANSAC algorithm is often used in computer vision. For example, in the field of stereo vision, it can simultaneously solve the matching point problem of a pair of cameras and the calculation of the basic matrix.

[0055] The core of the PnP algorithm is to estimate the pose of the camera in the world coordinate system, and optimize the pose of the camera and the spatial coordinates of the 3D points by minimizing the reprojection error. The PnP problem is described as follows: Given the intrinsic parameters K of the camera, the world coordinates Pw of n 3D spatial points and their projected pixel coordinates Puv in the camera, it is necessary to estimate the pose T of the camera in the world coordinate system. By minimizing the reprojection error, the camera pose T and the spatial coordinates Pw of the 3D points are optimized together. Specifically, the difference between the observed position and the projected position of the feature point needs to be minimized, which can be achieved by changing T.

[0056] The optical center of the camera is the point in the center of the convex lens, that is, the focus of the convex lens. In the paraxial light of the convex lens, the intersection of the line connecting the incident point and the exit point and the principal optical axis is the optical center. In the camera coordinate system, the optical center of the camera is located on the optical axis of the camera, parallel to the X and Y axes of the image, and the intersection of the optical axis and the image plane is the origin of the image coordinate system.

[0057] The intrinsic parameters of the camera are parameters related to the camera's own characteristics, such as the camera's focal length, pixel size, etc.

[0058] Camera extrinsic parameters are parameters in the world coordinate system, such as the camera's position, rotation direction, etc.

[0059] Camera calibration (or video camera calibration): is the mapping from world coordinates to pixel coordinates. World coordinates are generally defined artificially. Calibration is to solve the mapping relationship between the world coordinates and pixel coordinates of the known calibration control points. Once this relationship is solved, the world coordinates of the point can be inferred from its pixel coordinates.

[0060] Figure 1The following is a flow chart of a monocular SLAM static initialization method according to an embodiment of the present invention. The method can be implemented by a program in a flight control system of a drone. The method comprises the following steps:

[0061] Step S110, obtaining a monocular image of the two-dimensional code target;

[0062] Step S120, based on the multiple feature points of the two-dimensional code target in the monocular image, using the PnP algorithm to obtain the pose of the camera coordinate system in the two-dimensional code world coordinate system and the scale information of the feature points; wherein the two-dimensional code can be an aruco two-dimensional code;

[0063] Step S130, using the PlaneRecNet model to estimate the depth of the monocular image, output a depth map and extract multiple planes, and use the RANSAC algorithm to calculate the equation and normal vector of each plane; wherein the depth map obtained is as follows Figure 3 As shown;

[0064] Step S140, a plurality of normal vectors that are orthogonal to each other are selected from the normal vectors, so as to determine the normal vector corresponding to the horizontal plane, and determine the normal vector of the horizontal plane as the gravity vector; wherein the horizontal plane obtained is referred to Figure 4 Plane shown: 0.39;

[0065] Step S150, rotating the camera coordinate system so that its Z-axis direction is parallel to or coincides with the gravity vector direction, and determining the rotated camera coordinate system as the world coordinate system;

[0066] Step S160 , completing initialization through the above-mentioned partial matching and fusion based on the world coordinate system, the depth map and the scale information.

[0067] According to an embodiment of the present invention, the pose and scale information of the camera coordinate system are first determined based on the feature points in the monocular image and the PnP algorithm, and then the depth map and gravity vector are determined based on the PlaneRecNet model and the RANSAC algorithm. Then, the world coordinate system is determined based on the gravity vector, and finally, the static initialization of SLAM is realized based on the monocular camera and the monocular image, and the scale information and gravity alignment are restored without the aid of an IMU.

[0068] In some embodiments, the two-dimensional target includes five different two-dimensional codes, and the five two-dimensional codes are designed as follows: one two-dimensional code is placed in the center of the two-dimensional code target plate, and the other four are placed at the four corners of the central two-dimensional code.

[0069] Specific, combined Figure 2As shown, the designed QR code target board is shown below. The four Aruco QR code IDs around it are 0, 1, 2, and 3 clockwise starting from the upper left corner. The center QR code ID is 4, and the corresponding dictionary is DICT_4X4_50. In the calculation, the vertices of the rectangular QR code can be used as feature points to facilitate recognition and calculation.

[0070] In some embodiments, in step S120, based on multiple feature points of the two-dimensional code target in the monocular image, using the PnP algorithm to obtain the pose of the camera coordinate system in the two-dimensional code world coordinate system and the scale information of the feature points includes:

[0071] Taking the vertices on the two-dimensional code as feature points, and obtaining the projection coordinates of each feature point in a normalized camera coordinate system through two-dimensional code detection;

[0072] The position and posture of the camera coordinate system in the two-dimensional code world coordinate system are determined by using the feature points, projection coordinates and the minimization of the reprojection error calculation in the PnP algorithm, so as to restore the scale information of each feature point according to the two-dimensional code.

[0073] In some embodiments, obtaining a monocular image of a two-dimensional code target in step S110 includes:

[0074] The two-dimensional code target is photographed in an indoor environment to obtain the monocular image.

[0075] Because in indoor environments, there are usually obvious planes that are orthogonal to each other in the XYZ directions, the PlaneRecNet model can be used to easily perform monocular depth estimation and extract planes, and then the equations of these planes can be calculated based on the RANSAC algorithm.

[0076] In some embodiments, rotating the camera coordinate system in step S150 so that its Z-axis direction is parallel to or coincides with the gravity vector direction, and determining the rotated camera coordinate system as the world coordinate system further includes:

[0077] The world coordinate system is determined with the optical center of the camera as the origin, and the subsequent camera frame poses and map point positions are unified under the world coordinate system.

[0078] In some embodiments, in step S160, completing the initialization based on the world coordinate system, the depth map and the scale information includes:

[0079] Based on the depth map output by the PlaneRecNet model and the scale information obtained by two-dimensional code detection, the scale information of the depth map is restored.

[0080] In some implementations, completing the initialization based on the world coordinate system, the depth map, and the scale information in step S160 further includes:

[0081] Matching the depth map from which scale information has been restored and the monocular image from which feature points have been extracted to obtain positions of corresponding feature points in the world coordinate system;

[0082] Based on the positional relationship between other points in the monocular image and the feature point, the map point of the other point in the world coordinate system is determined, thereby realizing the static initialization of the monocular SLAM.

[0083] Specifically, according to the above embodiment, the camera system is rotated so that the Z-axis direction of the camera system is parallel to the direction of the gravity vector, and the origin is still the optical center of the camera. The initial frame camera system after rotation, that is, aligned with gravity, is used as the world system, thereby realizing the gravity alignment of the world system that does not rely on the IMU. The subsequent camera frame poses and map point positions are unified to this gravity-aligned world system.

[0084] At the same time, based on the depth map output by the PlaneRecNet model and the scale information obtained by QR code detection, the scale information of the depth map of the PlaneRecNet model can be restored. After matching the restored scaled depth map with the monocular image with feature points extracted, the inverse depth of the corresponding feature point can be directly obtained, and the position of the corresponding feature point in the world system can be obtained, and then more map points can be added. At this time, the static initialization of the monocular SLAM is completed, and the initial camera frame pose and a large number of map points are obtained. The scale is restored and gravity alignment is performed.

[0085] Combination Figure 6 As shown, the present invention also protects a monocular SLAM static initialization device 600, and the monocular SLAM static initialization device 600 includes:

[0086] An image acquisition module 610 is adapted to acquire a monocular image of a two-dimensional code target;

[0087] A posture determination module 620, adapted to obtain the posture of the camera coordinate system in the two-dimensional code world coordinate system and scale information of the feature points by using a PnP algorithm based on multiple feature points of the two-dimensional code target in the monocular image;

[0088] A plane extraction module 630, adapted to estimate the depth of the monocular image using a PlaneRecNet model, output a depth map and extract multiple planes, and calculate the equation and normal vector of each plane using a RANSAC algorithm;

[0089] A vector determination module 640, adapted to select a plurality of normal vectors that are orthogonal to each other from the normal vectors, thereby determining a normal vector corresponding to a horizontal plane, and determining the normal vector of the horizontal plane as a gravity vector;

[0090] A coordinate determination module 650, adapted to rotate the camera coordinate system so that its Z-axis direction is parallel to or coincides with the gravity vector direction, and determine the rotated camera coordinate system as a world coordinate system;

[0091] The initialization module 660 is adapted to complete initialization based on the world coordinate system, the depth map and the scale information.

[0092] According to the monocular SLAM static initialization device disclosed in this embodiment, the static initialization of monocular vision SLAM is realized, and scale information can be restored and gravity alignment can be performed without the help of an IMU module.

[0093] In some embodiments, in combination Figure 2 As shown, the two-dimensional target includes five different Aruco two-dimensional codes, and the five two-dimensional codes are designed as follows: one two-dimensional code is placed in the center of the two-dimensional code target plate, and the other four are placed at the four corners of the central two-dimensional code.

[0094] In some embodiments, the pose determination module 620 is further adapted to:

[0095] Taking the vertices on the two-dimensional code as feature points, and obtaining the projection coordinates of each feature point in a normalized camera coordinate system through two-dimensional code detection;

[0096] The position and posture of the camera coordinate system in the two-dimensional code world coordinate system are determined by using the feature points, projection coordinates and the minimization of the reprojection error calculation in the PnP algorithm, so as to restore the scale information of each feature point according to the two-dimensional code.

[0097] In some embodiments, the image acquisition module 610 is further adapted to:

[0098] The two-dimensional code target is photographed in an indoor environment to obtain the monocular image.

[0099] In some embodiments, the coordinate determination module 650 is further adapted to:

[0100] The world coordinate system is determined with the optical center of the camera as the origin, and the subsequent camera frame poses and map point positions are unified under the world coordinate system.

[0101] In some embodiments, the initialization module 660 is further adapted to:

[0102] Based on the depth map output by the PlaneRecNet model and the scale information obtained by two-dimensional code detection, the scale information of the depth map is restored.

[0103] In some embodiments, the initialization module 660 is further adapted to:

[0104] Matching the depth map from which scale information has been restored and the monocular image from which feature points have been extracted to obtain positions of corresponding feature points in the world coordinate system;

[0105] Based on the positional relationship between other points in the monocular image and the feature point, the map point of the other point in the world coordinate system is determined, thereby realizing the static initialization of the monocular SLAM.

[0106] It should be noted that the specific implementation of the above-mentioned device embodiments can be carried out with reference to the specific implementation of the above-mentioned corresponding method embodiments, which will not be repeated here. Specific embodiments

[0107] Combination Figure 5 The overall flow chart shown in the figure first uses a designed QR code target of known size, combined with Figure 2 As shown in the figure, the center point of the central QR code is used as the origin of the QR code world system, and the four fixed points of the five QR codes are used as map points with known positions. After the camera intrinsic parameters have been calibrated, the projection coordinates of the 16 vertices in the normalized camera system can be obtained through QR code detection. Then, PnP is solved based on minimizing the reprojection error, and the position and posture of the camera system in the QR code world system can be obtained. In this way, the scale information of the map point can be restored by relying on the QR code.

[0108] The designed QR code target board is shown below. There are four Aruco QR code IDs around it, starting from the upper left corner, clockwise, they are 0, 1, 2, 3. The center QR code ID is 4, and the corresponding dictionary is DICT_4X4_50.

[0109] Because in indoor environments, there are usually relatively obvious planes that are orthogonal to each other in the XYZ directions. This specific embodiment can perform monocular depth estimation and extract planes through the PlaneRecNet model. Based on the RANSAC algorithm, the equations of these planes can be calculated, and their corresponding normal vectors can be obtained. Among these normal vectors, three normal vectors that are orthogonal to each other are selected, and then the normal vector corresponding to the horizontal plane is obtained as the gravity direction, thereby estimating a gravity vector based on vision.

[0110] At this time, the camera system is rotated so that the Z-axis direction of the camera system is parallel to the direction of the gravity vector. The origin is still the optical center of the camera. The initial frame camera system after rotation, that is, aligned with gravity, is used as the world system. The gravity alignment of the world system that does not rely on the IMU is achieved. The subsequent camera frame poses and map point positions are unified to this gravity-aligned world system.

[0111] At the same time, based on the depth map output by the PlaneRecNet model and the scale information obtained by QR code detection, the scale information of the PlaneRecNet depth map can be restored. After matching the restored scaled depth map with the monocular image with feature points extracted, the inverse depth of the corresponding feature point can be directly obtained, and the position of the corresponding feature point in the world system can be obtained, and then more map points can be added. At this time, the static initialization of the monocular SLAM is completed, and the initial camera frame pose and a large number of map points are obtained. The scale is restored and gravity alignment is performed.

[0112] It should be noted that:

[0113] The algorithm and display provided herein are not inherently related to any particular computer, virtual device or other equipment. Various general-purpose devices can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of device. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific language is for the purpose of disclosing the best mode of the present invention.

[0114] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0115] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0116] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device so disclosed may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0117] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present invention and to form different embodiments.

[0118] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components in the monocular SLAM static initialization device according to an embodiment of the present invention. The present invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0119] An embodiment of the present invention provides a non-volatile computer storage medium, wherein the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the above-mentioned monocular SLAM static initialization method in any of the above-mentioned method embodiments.

[0120] Figure 7 A schematic diagram of the structure of an embodiment of a drone of the present invention is shown, and the specific embodiment of the present invention does not limit the specific structure of the drone.

[0121] like Figure 7As shown, the UAV (including the flight control system) may include: a processor (processor) 702 , a communications interface (Communications Interface) 704 , a memory (memory) 706 , and a communication bus 708 .

[0122] Wherein: the processor 702, the communication interface 704, and the memory 706 communicate with each other through the communication bus 708. The communication interface 704 is used to communicate with other devices such as a client or other server network elements. The processor 702 is used to execute the program 710, which can specifically execute the relevant steps in the above-mentioned monocular SLAM static initialization method embodiment for the drone.

[0123] Specifically, the program 710 may include program codes, which include computer operation instructions.

[0124] The processor 702 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the drone may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0125] The memory 706 is used to store the program 710. The memory 706 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0126] The program 710 can be specifically used to enable the processor 702 to perform operations corresponding to the above-mentioned monocular SLAM static initialization method embodiment.

[0127] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A monocular SLAM static initialization method, characterized in that: The method comprises: Obtain a monocular image of the QR code target; Based on multiple feature points of the two-dimensional code target in the monocular image, a PnP algorithm is used to obtain the position and scale information of the feature points of the camera coordinate system in the two-dimensional code world coordinate system; The PlaneRecNet model is used to estimate the depth of the monocular image, output a depth map and extract multiple planes, and the equation and normal vector of each plane are calculated using the RANSAC algorithm; Filtering out a plurality of normal vectors that are orthogonal to each other from the normal vectors, thereby determining a normal vector corresponding to a horizontal plane, and determining the normal vector of the horizontal plane as a gravity vector; Rotate the camera coordinate system so that its Z-axis direction is parallel to or coincides with the gravity vector direction, and determine the rotated camera coordinate system as the world coordinate system; Initialization is completed based on the world coordinate system, the depth map and the scale information.

2. The method according to claim 1, characterized in that The two-dimensional code target includes five different two-dimensional codes, and the five two-dimensional codes are designed as follows: one two-dimensional code is placed at the center of the two-dimensional code target plate, and the other four are placed at the four corners of the central two-dimensional code.

3. The method according to claim 2, characterized in that Based on multiple feature points of the two-dimensional code target in the monocular image, the pose of the camera coordinate system in the two-dimensional code world coordinate system and the scale information of the feature points are obtained by using the PnP algorithm, including: Taking the vertices on the two-dimensional code as feature points, and obtaining the projection coordinates of each feature point in a normalized camera coordinate system through two-dimensional code detection; The position and posture of the camera coordinate system in the two-dimensional code world coordinate system are determined by using the feature points, projection coordinates and the minimization of the reprojection error calculation in the PnP algorithm, so as to restore the scale information of each feature point according to the two-dimensional code.

4. The method according to claim 1, characterized in that: Obtaining a monocular image of a QR code target includes: The two-dimensional code target is photographed in an indoor environment to obtain the monocular image.

5. The method according to any one of claims 1 to 4, characterized in that Rotating the camera coordinate system so that its Z-axis direction is parallel to or coincides with the gravity vector direction, and determining the rotated camera coordinate system as the world coordinate system further includes: The world coordinate system is determined with the optical center of the camera as the origin, and the subsequent camera frame poses and map point positions are unified under the world coordinate system.

6. The method according to claim 1, characterized in that Based on the world coordinate system, the depth map and the scale information, completing the initialization includes: Based on the depth map output by the PlaneRecNet model and the scale information obtained by two-dimensional code detection, the scale information of the depth map is restored.

7. The method according to claim 1, characterized in that Based on the world coordinate system, the depth map and the scale information, completing the initialization further includes: Matching the depth map from which scale information has been restored and the monocular image from which feature points have been extracted to obtain positions of corresponding feature points in the world coordinate system; Based on the positional relationship between other points in the monocular image and the feature point, the map point of the other point in the world coordinate system is determined, thereby realizing the static initialization of the monocular SLAM.

8. A monocular SLAM static initialization device, characterized in that: The device comprises: An image acquisition module, adapted to acquire a monocular image of a two-dimensional code target; A posture determination module, adapted to obtain the posture of the camera coordinate system in the two-dimensional code world coordinate system and scale information of the feature points using a PnP algorithm based on multiple feature points of the two-dimensional code target in the monocular image; A plane extraction module, adapted to estimate the depth of the monocular image using a PlaneRecNet model, output a depth map and extract multiple planes, and calculate the equation and normal vector of each plane using a RANSAC algorithm; a vector determination module, adapted to select a plurality of normal vectors that are orthogonal to each other from the normal vectors, thereby determining a normal vector corresponding to a horizontal plane, and determining the normal vector of the horizontal plane as a gravity vector; A coordinate determination module, adapted to rotate the camera coordinate system so that its Z-axis direction is parallel to or coincides with the gravity vector direction, and determine the rotated camera coordinate system as a world coordinate system; An initialization module is adapted to complete initialization based on the world coordinate system, the depth map and the scale information.

9. A drone, characterized in that: The invention comprises a processor and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the monocular SLAM static initialization method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by the processor, the monocular SLAM static initialization method according to any one of claims 1-7 is implemented.

Images