Map construction method, device and equipment

Abstract

The present application provides a map construction method, device and equipment, which relates to the field of robotics, wherein the method comprises: first obtaining the initial posture and visual image of the self-mobile device during movement, obtaining several target key frames from the visual image, determining the local error equation according to the initial posture and visual image, using the local error equation to perform local nonlinear optimization on the target key frames, further adding the optimized posture of each target key frame to the feature map layer to obtain the target feature map layer, and finally determining the global error equation according to the optimized posture, performing global nonlinear optimization on the posture in the target feature map layer according to the global error equation to obtain the target global map. The technical solution provided by the present application enables the self-mobile device to construct a map with high accuracy in scenarios such as when feature points are not abundant, scenes are similar, the signal of the real-time dynamic measuring instrument is poor, and wheels slip.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a map construction method, device and equipment. Background Art

[0002] With the development of society and the advancement of science and technology, the functions of robots have become diversified. These various robots can perform their tasks well in specific environments, and therefore are becoming more and more popular among people.

[0003] In most cases, the robot's working environment is unknown or uncertain, and the robot's autonomous movement and positioning need to rely on environmental maps. The current way robots build maps uses lasers or vision for simultaneous localization and mapping (SLAM). This can easily lead to SLAM matching failures in venues with few feature points or similar scenes, resulting in low accuracy of the constructed map. Summary of the invention

[0004] In view of this, the present application provides a map construction method, device and equipment to improve the accuracy of the map constructed by the robot in a venue with few feature points or similar scenes.

[0005] In order to achieve the above objectives, in a first aspect, an embodiment of the present application provides a map construction method, comprising:

[0006] Obtaining the initial position and visual image of the mobile device during movement;

[0007] extracting a target keyframe from the visual image;

[0008] Determining a local error equation based on the initial pose and the visual image;

[0009] Using the local error equation to perform local nonlinear optimization on the target key frame to obtain the posture of the mobile device after the optimization of the target key frame;

[0010] Adding the optimized pose of each target key frame to the feature map layer to obtain a target feature map layer;

[0011] Determine a global error equation according to the optimized posture;

[0012] The pose in the target feature map layer is globally nonlinearly optimized according to the global error equation to obtain a global map.

[0013] As an optional implementation of the embodiment of the present application, determining the local error equation according to the initial posture and the visual image includes:

[0014] Determine a predicted pose of the mobile device at the target key frame according to the initial pose, and determine a pose measurement error factor according to the predicted pose;

[0015] Extracting visual image feature points of the target key frame;

[0016] Determining a reprojection error factor based on the predicted pose and the visual image feature points;

[0017] Determine a real-time dynamic measurement error factor according to the real-time position information of the self-mobile device;

[0018] Determining a marginalization factor according to the visual image feature points and the real-time position information;

[0019] The local error equation is determined according to the posture measurement error factor, the reprojection error factor, the real-time dynamic measurement error factor and the marginalization factor.

[0020] As an optional implementation of the embodiment of the present application, determining the reprojection error factor according to the predicted pose and the visual image feature points includes:

[0021] Determine the coordinates of the visual image feature points of the target key frame in the feature map layer according to the predicted pose;

[0022] Determining a target feature point matched by the visual image feature point in the feature map layer according to the coordinates of the visual image feature point in the feature map layer;

[0023] According to the predicted position and posture, the target feature points are converted into the visual image to obtain matching feature points;

[0024] A reprojection error factor is determined based on the matching feature points and the visual image feature points.

[0025] As an optional implementation of the embodiment of the present application, determining the global error equation according to the optimized posture includes:

[0026] Determining a relative posture constraint factor of the optimized posture of each target key frame according to the posture corresponding to each target key frame and the inverse of the posture corresponding to each target key frame;

[0027] Obtaining a closed-loop detection pose constraint factor according to the inverse of the pose corresponding to each of the target key frames and the closed-loop detection pose corresponding to each of the target key frames;

[0028] Determine a real-time dynamic measurement position constraint factor according to the inverse of the position and posture corresponding to each of the target key frames and the real-time dynamic measurement position corresponding to each of the target key frames;

[0029] A global error equation is determined according to the relative posture constraint factor, the closed-loop detection posture constraint factor and the real-time dynamic measurement position constraint factor.

[0030] As an optional implementation of an embodiment of the present application, the global map also includes at least one of a target trajectory layer and a boundary layer, the target trajectory layer is used to determine the operating trajectory range of the self-moving device during movement, and the boundary layer is used to distinguish between the operating range and the non-operating range and to plan the operating trajectory range for the self-moving device within the operating range.

[0031] As an optional implementation of the embodiment of the present application, when the global map includes the target trajectory layer, the method further includes:

[0032] Get the initial trajectory layer;

[0033] The optimized pose of each target key frame is added to the initial trajectory layer to obtain the target trajectory layer.

[0034] As an optional implementation of the embodiment of the present application, the visual image includes a color image and a depth image corresponding to the color image, and when the global map includes a boundary layer, the method further includes: acquiring boundary information in the color image, and determining a depth value corresponding to the boundary information from the depth image corresponding to the color image; wherein the boundary information is used to determine the operating range and the non-operating range;

[0035] According to the boundary information and the depth value corresponding to the boundary information, obtaining the coordinates of the boundary information in the first coordinate system of the mobile device;

[0036] Based on the acquired posture of the mobile device during movement and the coordinates of the boundary information in the first coordinate system, the coordinates of the boundary information in the second coordinate system are determined, so that the coordinates of the boundary information in the second coordinate system are used as the coordinate information of the boundary layer.

[0037] As an optional implementation of the embodiment of the present application, before obtaining the initial position and visual image during the movement of the mobile device, the method further includes:

[0038] Control the mobile device to move from an initial position along a preset direction and a preset distance before stopping;

[0039] The roll angle, pitch angle and position information of the self-moving device when it stops are used as the initial position and posture of the self-moving device;

[0040] Transforming the initial posture to a second coordinate system where the feature points in the initial visual image collected after the mobile device stops are located, so that the initial posture corresponds to the coordinates in the second coordinate system one by one;

[0041] An initial feature map layer is obtained according to the coordinates of the feature points in the initial visual image in the second coordinate system.

[0042] In a second aspect, an embodiment of the present application provides a map construction device, including:

[0043] Acquisition module: used to obtain the initial posture and visual image of the mobile device during its movement;

[0044] Extraction module: used for acquiring a number of target key frames from the visual image;

[0045] A local error determination module: used to determine a local error equation according to the initial posture and the visual image;

[0046] Local optimization module: used for performing local nonlinear optimization on the target key frame by using the local error equation to obtain the posture of the mobile device after the optimization of the target key frame;

[0047] Adding module: used for adding the optimized pose of each target key frame to the feature map layer to obtain the target feature map layer;

[0048] A global error determination module: used for determining a global error equation according to the optimized posture;

[0049] Global optimization module: used for performing global nonlinear optimization on the pose in the target feature map layer according to the global error equation to obtain a target global map.

[0050] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method described in the first aspect or any implementation method of the first aspect when calling the computer program.

[0051] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in the first aspect or any one of the embodiments of the first aspect is implemented.

[0052] The map construction scheme provided in the embodiment of the present application first obtains the initial posture and visual image of the self-mobile device during movement, and obtains several target key frames from the visual image, determines the local error equation according to the initial posture and visual image, and uses the local error equation to perform local nonlinear optimization on the target key frame to obtain the posture of the self-mobile device after the optimization of the target key frame, further adds the optimized posture of each target key frame to the feature map layer to obtain the target feature map layer, and finally determines the global error equation according to the optimized posture, and performs global nonlinear optimization on the posture in the target feature map layer according to the global error equation to obtain the target global map. The map construction solution provided in the embodiment of the present application combines the local error equations and global error equations generated by the visual image and posture data to achieve dual optimization of the posture of the self-moving device. It not only reduces the local error of the key frame posture of the visual image during the movement of the self-moving device, but also performs global error reduction on the posture in the entire target feature map layer, thereby improving the accuracy of map construction. The self-moving device can still construct a high-precision map in scenarios where the data obtained by some sensors is poor due to environments such as a lack of feature points, similar scenes, poor signals from real-time dynamic measurement instruments, and wheel slippage. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of a map construction method provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of a process for determining a local error equation provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of a process for determining a reprojection error factor according to an embodiment of the present application;

[0056] Figure 4 A schematic diagram of a process for determining a global error equation provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of a global map constructed by the map construction method provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of a process for generating a target trajectory layer provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of a process for generating a boundary layer according to an embodiment of the present application;

[0060] Figure 8 A flowchart of a positioning initialization process provided in an embodiment of the present application;

[0061] Fig. 9A map construction device provided in an embodiment of the present application;

[0062] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0064] The map construction method provided in the embodiment of the present application can be implemented by a map construction device, which can be a self-moving device, such as a robot, or a chip or circuit used in a robot, or the map construction device can also be an electronic device or a chip or circuit used in an electronic device. For example, the map construction method can be used to build a map on a computer. This embodiment will be described later using the application of the map construction method to a robot as an example. When the map construction device is an electronic device, the map construction device can interact with the robot. For example, the robot can report various sensor data of the robot to the electronic device, etc. This embodiment of the present application does not limit this.

[0065] The robot may be a lawn mowing robot, a sweeping robot, a mine-clearing robot, a cruise robot, etc., and this embodiment does not specifically limit this.

[0066] Figure 1 A flowchart of a map construction method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method may include the following steps:

[0067] S110, obtaining an initial position and visual image from the mobile device during movement.

[0068] The self-mobile device is equipped with multiple different types of sensors and cameras, including but not limited to wheel speed meters, inertial sensors, and real-time kinematic (RTK) meters.

[0069] The wheel speed meter may be a ring-shaped wheel speed sensor, a linear wheel speed sensor, a Hall-type wheel speed sensor, or the like.

[0070] The inertial sensor can be a micro electro mechanical system (MEMS) sensor or an inertial measurement unit (IMU) sensor.

[0071] The camera can be a monocular camera, a binocular camera, an RGB-D camera, an event camera, etc.

[0072] The initial posture may include data collected by the wheel speed meter of the self-moving device during the movement, data collected by the inertial sensor and position information of the self-moving device collected by the real-time dynamic measuring instrument. The self-moving device can collect visual images within the visual range through the camera, and then obtain feature points in the visual image.

[0073] In the following, this embodiment is described by taking the case where the wheel speed meter is a ring wheel speed sensor, the inertial sensor is an IMU, and the camera is an RGB-D camera as an example.

[0074] It is understandable that the initial posture may also include data from other sensors, which is not limited in the embodiments of the present application.

[0075] S120, obtaining a number of target key frames from the visual image.

[0076] Specifically, the mobile device can determine one or more target visual images from the collected multiple frames of visual images except the first frame of visual image, wherein the parallax between the target visual image and the previous frame of visual image of the target visual image is greater than a preset angle.

[0077] The self-mobile device can use the first visual image frame and one or more target visual images as target key frames. Specifically, the number of visual image frames can be set as target key frames according to actual conditions, which is not limited here.

[0078] The mobile device can add the extracted target key frame into the sliding window to facilitate the subsequent local nonlinear optimization of the target key frame.

[0079] The mobile device can also store the extracted target key frames in the visual dictionary database to facilitate subsequent closed-loop detection.

[0080] S130, determining a local error equation according to the initial posture and the visual image.

[0081] Figure 2 A schematic diagram of a process for determining a local error equation provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the process may include the following steps:

[0082] S131. Obtain a predicted posture of the mobile device in a target key frame according to the initial posture, and determine a posture measurement error factor according to the predicted posture.

[0083] The predicted posture may include the prediction result of the wheel speed meter and the prediction result of the IMU. Correspondingly, the posture measurement error factor may include the wheel speed meter error factor and the IMU error factor.

[0084] The target key frame may be the first visual image frame among the visual images captured by the camera and a visual image whose disparity with the previous visual image frame is greater than a preset threshold.

[0085] Specifically, the self-mobile device can pre-integrate the wheel speed meter data and IMU data corresponding to each frame of the collected visual image to obtain the prediction result of the wheel speed meter and the IMU of the self-mobile device in the target key frame.

[0086] The mobile device can pre-integrate the data collected by the wheel speed meter according to the following formula (1):

[0087]

[0088] in, represents the coordinates of the mobile device in the world coordinate system in the jth target key frame, represents the coordinates of the mobile device in the world coordinate system in the i-th target key frame, represents the i-th target key frame, the rotation posture parameter of the mobile device in the world coordinate system, q bo Represents the rotation extrinsic parameter from the IMU coordinate system to the wheel speed meter coordinate system, v oi represents the speed of the wheel speed meter of the i-th target key frame, and Δt represents the time interval between the i-th target key frame and the j-th target key frame, where i≥1, j>i.

[0089] The mobile device can pre-integrate the data collected by the IMU according to the following formula (2):

[0090]

[0091] in, represents the coordinates of the mobile device in the world coordinate system in the jth target key frame, represents the speed of the jth target keyframe from the mobile device, represents the rotational posture parameters of the mobile device in the world coordinate system in the jth target key frame, represents the bias of the IMU gyroscope of the i-th target key frame, represents the bias of the IMU accelerometer of the i-th target keyframe, represents the coordinates of the mobile device in the world coordinate system in the i-th target key frame, represents the speed of the i-th target keyframe from the mobile device, Δt represents the time interval between the i-th target keyframe and the j-th target keyframe, and g w represents gravity, represents the rotational posture parameters of the mobile device in the world coordinate system for the i-th target keyframe. represents the pre-integration result of the IMU translation from the i-th target keyframe to the j-th target keyframe, Represents the pre-integration result of the IMU velocity from the i-th target keyframe to the j-th target keyframe, represents the pre-integration result of the IMU rotation from the i-th target keyframe to the j-th target keyframe, represents the offset of the IMU gyroscope of the jth target keyframe, Represents the bias of the IMU accelerometer of the jth target keyframe.

[0092] After the mobile device pre-integrates the data collected by the wheel speed meter and the data collected by the IMU, the error factors of the wheel speed meter and the IMU can be determined according to the pre-integration results of the wheel speed meter (i.e., the prediction results of the wheel speed meter) and the pre-integration results of the IMU (i.e., the prediction results of the IMU), respectively.

[0093] Specifically, the self-moving device can determine the wheel speed meter error factor according to the following formula (3):

[0094]

[0095] Among them, r o (v oi ,s) represents the wheel speed meter error factor of the i-th target key frame, r o represents the wheel speed meter error, v oi represents the speed of the wheel speed meter of the i-th target key frame, s represents the state quantity of a visual image feature point corresponding to the marginalization, that is, the optimized change in the sliding window, Represents the rotational posture parameters of the mobile device in the world coordinate system at the i-th target key frame. represents the coordinates of the mobile device in the world coordinate system in the jth target key frame, represents the coordinates of the mobile device in the world coordinate system in the i-th target key frame, p bo Represents the translation of the external parameters from the mobile device in the IMU coordinate system to the wheel speed meter coordinate system, represents the rotational posture parameters of the mobile device in the world coordinate system in the jth target key frame, represents the pre-integration result of the wheel speed meter between the i-th target key frame and the j-th target key frame, and ∑o represents the information matrix of the wheel speed meter.

[0096] The mobile device can determine the IMU error factor according to the following formula (4):

[0097]

[0098] Among them, b j represents the jth target key frame, b i represents the i-th target key frame, s represents the state quantity of a visual image feature point corresponding to the marginalization, that is, the optimized change in the sliding window, r p 、r q 、r v 、r ba 、r bg They represent the translation error, rotation error, velocity error, gyroscope error and accelerometer error of IMU respectively. represents the inverse of the pose of the self-mobile device in the i-th target keyframe, and (x, y, z) is the camera coordinate corresponding to the self-mobile device in the target keyframe.

[0099] S132, extracting visual image feature points of the target key frame, and determining a reprojection error factor according to the predicted posture and the visual image feature points.

[0100] Figure 3 A schematic diagram of a process for determining a reprojection error factor provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the process may include the following steps:

[0101] S1321. Determine the coordinates of the visual image feature points of the target key frame in the feature map layer according to the predicted posture.

[0102] The visual image feature point is a representative point in the visual image, for example, it can be a point on the outline of a tree, a person or an object in a frame of an image. The mobile device can obtain the pixel values ​​corresponding to these points in the visual image.

[0103] The number of visual image feature points may be one or more. For example, there are visual image feature points A and B in a frame of visual image. Taking visual image feature point A in the visual image as an example, the self-mobile device may determine the predicted pose of the visual image based on the prediction results of the wheel speed meter and the inertial sensor corresponding to the visual image, and determine the coordinates of the visual image feature point A in the feature map layer based on the predicted pose.

[0104] S1322. Determine the target feature point that matches the visual image feature point in the feature map layer according to the coordinates of the visual image feature point in the feature map layer.

[0105] Specifically, the self-mobile device can find a target feature point A1 in the feature map layer that represents the same actual position and has the same corresponding pixel value as the visual image feature point A in the visual image according to the coordinates of the visual image feature point A in the feature map layer.

[0106] S1323. According to the predicted posture, the target feature points are converted into the visual image to obtain matching feature points.

[0107] Specifically, the self-mobile device can determine the coordinates of the target feature point A1 in the visual image according to the predicted position and posture, and project A1 into the visual image to obtain the matching feature point A' of the feature point A in the visual image.

[0108] S1324. Determine a reprojection error factor based on the matching feature points and the visual image feature points.

[0109] Specifically, the self-mobile device may determine the reprojection error factor according to the visual image feature point A and the matching feature point A' of A in the visual image. The self-mobile device may determine the reprojection error factor according to the following formula (5) and formula (6):

[0110]

[0111]

[0112] in, is the coordinate of a feature point in the jth target key frame in the camera coordinate system, T bc Represents the external parameters of the camera to IMU coordinate system, represents the pose of the i-th target key frame, represents the pose of the jth target key frame, λ represents the inverse depth of the feature point, Represents the coordinates of the feature point in the pixel coordinate system, r c Represents the reprojection error factor.

[0113] S133. Determine a real-time dynamic measurement error factor according to the real-time position information of the self-mobile device.

[0114] The self-moving device can determine the RTK error factor (that is, the real-time dynamic measurement error factor) based on its own current position information collected by RTK and the position information of the self-moving device in the current visual image, wherein the real-time position information includes the coordinates of the self-moving device in the RTK base station coordinate system and the yaw angle of the self-moving device.

[0115] Specifically, the mobile device can determine the RTK error factor according to the following formula (7):

[0116] error=(x1 -x 2 )*(x 1 -x 2 )+(y 1 -y 2 )*(y 1 -y 2 )+|(θ 1 -θ 2 )|.....Formula (7)

[0117] Where error represents the RTK error factor, (x 1 ,y 1 ) represents the coordinates of the mobile device collected by RTK in the RTK base station coordinate system, (x 2 ,y 2 ) represents the coordinates of the mobile device in the current visual image, θ 1 Indicates the current heading angle of the RTK mobile station (that is, the self-moving device), θ 2 Indicates the heading angle of the current self-mobile device in the visual image.

[0118] In order to create a map without directional angle drift, a single-antenna RTK can be used to collect the position information of the mobile device. When RTK is a single antenna, since RTK cannot feedback the heading angle of the RTK mobile station, the mobile device can calculate the current heading angle of the RTK mobile station through inter-frame difference.

[0119] Specifically, when the mobile robot detects that it is moving in a straight line, it can calculate the position coordinates of the mobile device (rtk_x 1 ,rtk_y 1 )(rtk_x 2 ,rtk_y 2 ), determine the current heading angle of the RTK mobile station, wherein the current heading angle of the RTK mobile station can be determined according to the following formula (8):

[0120] θ 1 =atan2(rtk_y 2 -rtk_y 1 ,rtk_x 2 -rtk_x 1 )............Formula (8)

[0121] When the current angular velocity of the self-moving device is less than the set angular velocity, the self-moving device moves in a straight line; otherwise, the self-moving device moves in a non-straight line.

[0122] S134. Determine a marginalization factor according to the visual image feature points and the real-time position information.

[0123] The self-mobile device can determine the marginalization factor according to the visual image feature points and the real-time position information of the self-mobile device itself. Specifically, the self-mobile device can determine the marginalization factor according to the following formula (9):

[0124] er=||r p -J p s|| 2 ....................................Formula (9)

[0125] Among them, er represents the marginalization factor, J p represents the information matrix corresponding to the marginalization, which includes the real-time location information of the mobile device, and s represents the state quantity of a visual image feature point corresponding to the marginalization, that is, the optimized change in the sliding window.

[0126] S135. Determine a local error equation according to the posture measurement error factor, the reprojection error factor, the real-time dynamic measurement error factor and the marginalization factor.

[0127] Specifically, the mobile device can construct a local error equation according to the following formula (10):

[0128]

[0129] Among them, ||r p -J p s|| 2 represents the marginalization factor, represents the error factor of IMU (i.e., the error factor of posture measurement), represents the reprojection error factor, ρ represents the kernel function, ε c represents the visual image pixel projection function, Represents the pixels of a visual image, represents the RTK error factor (i.e., real-time dynamic measurement error factor), (x curi ,y curi ,θ curi ) represents the coordinates of the mobile device in the i-th target key frame, (x RTKi ,y RTKi ,θ RTKi ) represents the coordinates of the mobile device collected by RTK when the camera collects the i-th target key frame.

[0130] It can be understood that when there is a closed-loop detection posture constraint factor, the self-moving device can also construct a local error equation according to the following formula (11):

[0131]

[0132] Among them, ||T cur -1 ·T close || 2 represents the closed-loop detection pose constraint factor, T cur Indicates the current location of the mobile device, T close Indicates the position detected by the closed loop.

[0133] Furthermore, when the data obtained by some sensors of the self-mobile device is poor, such as a lack of feature points, similar scenes, poor RTK signals, and wheel slippage, the local error equation can still be constructed by determining the error factor through normal data obtained by other sensors.

[0134] S140, using a local error equation to perform local nonlinear optimization on the target key frame to obtain an optimized position and posture of the target key frame.

[0135] Specifically, n target key frames can be maintained in the sliding window, and the mobile device can perform nonlinear optimization on the pose of each target key frame maintained in the sliding window according to the local error equation to obtain the optimized pose of the target key frame, where n is an integer greater than 1.

[0136] The pose in the target keyframe can include the following variables:

[0137] s=[x 1 ,…,x n ,x bc ,x bo ,λ 1 ,…,λ m ],m∈[0,z]

[0138]

[0139] x bc =[p bc ,q bc ]

[0140] x bo =[p bo ,q bo ]

[0141] Among them, s represents the optimized change in the sliding window, which can also be expressed as the optimized position of the target key frame in the sliding window, (x 1 ,…,x n ) and (λ 1 ,…,λ m ) represents the state variable that can be optimized in the pose of the target key frame of the sliding window, x bc Represents the external parameter from the IMU coordinate system to the camera coordinate system, xbo represents the external parameter from the IMU coordinate system to the wheel speed meter coordinate system, p bc Represents the translation extrinsic parameter from the IMU coordinate system to the camera coordinate system, q bc Represents the rotation extrinsic parameter from the IMU coordinate system to the camera coordinate system, p bo Represents the translation extrinsic parameter from the IMU coordinate system to the wheel speed meter coordinate system, q bo Represents the rotation extrinsic parameter from the IMU coordinate system to the wheel speed meter coordinate system, x k represents the state variables that can be optimized in the kth target keyframe, represents the position of the kth target keyframe from the mobile device, represents the speed of the kth target keyframe from the mobile device, represents the pose of the kth target keyframe from the mobile device, represents the bias of the IMU accelerometer of the kth target keyframe, Represents the bias of the IMU gyroscope of the kth target keyframe.

[0142] S150, adding the optimized pose of each target key frame to the feature map layer to obtain a target feature map layer.

[0143] The created feature map layer is initialized, that is, the initial pose is added to the feature map layer. After initialization, the mobile device can add the pose of the optimized target key frame in the sliding window to the position corresponding to the target key frame in the feature map layer to obtain a target feature map layer, which includes the optimized pose of each target key frame and the optimized feature points of each target key frame.

[0144] S160. Determine a global error equation according to the optimized posture.

[0145] Figure 4 A schematic diagram of a process for determining a global error equation provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the process may include the following steps:

[0146] S161. Determine a relative posture constraint factor of the optimized posture of each target key frame according to the posture corresponding to each target key frame and the inverse of the posture corresponding to each target key frame.

[0147] The self-mobile device may add n target key frames among the target key frames to the pose graph, wherein, among the n target key frames, the translation difference between each two adjacent target key frames is greater than a set distance or the angle difference is greater than a set angle.

[0148] The mobile device can determine the relative pose constraint factor of the optimized pose of the target key frame in the pose graph based on the optimized pose corresponding to each target key frame in the pose graph and the inverse of the optimized pose corresponding to each target key frame in the pose graph.

[0149] Specifically, the self-moving device can determine the relative posture constraint factor according to the following formula (12):

[0150] error 1 =T 1 .inverse()*T 2 +T 2 .inverse()*T 3 +.....+T i .inverse()*T i+1

[0151] +T n-1 .inverse()*T n .......Formula (12)

[0152] Among them, error 1 Represents the relative posture constraint factor, T i Indicates the optimized pose corresponding to the i-th target keyframe among the n target keyframes, T i .inverse() represents the inverse of the optimized pose corresponding to the i-th target keyframe.

[0153] S162. Obtain a closed-loop detection posture constraint factor according to the inverse of the posture corresponding to each target key frame and the closed-loop detection posture corresponding to each target key frame.

[0154] The self-mobile device can detect whether each target key frame in the pose graph needs to be closed-loop detected. For example, the target key frame in the pose graph can be closed-loop detected at set intervals. That is, when the self-mobile device detects that the interval number between the current target key frame and the previous target key frame that underwent closed-loop detection is the set value, the current target key frame is closed-loop detected. Otherwise, the current target key frame is not closed-loop detected.

[0155] When the mobile device determines that the current target key frame in the pose graph needs to be closed-loop detected, a similar target key frame can be searched in the visual dictionary database based on the feature points extracted from the current target key frame. If not found, the next target key frame is detected to see if closed-loop detection is required. If found, these similar target key frames are combined into a local feature map.

[0156] The mobile device can match the feature points in the current target key frame with the feature points in the local map using the pnp (perspective-n-point) algorithm to obtain a closed-loop detection posture.

[0157] The mobile device can obtain the closed-loop detection pose constraint factor according to the inverse of the optimized pose corresponding to each target key frame in the pose graph and the closed-loop detection pose corresponding to each target key frame.

[0158] Specifically, the self-mobile device can determine the closed-loop detection posture constraint factor according to the following formula (13):

[0159] error 2 =T i .inverse()*T i _close........................Formula (13)

[0160] Among them, error 2 represents the closed-loop detection pose constraint factor, T i _close represents the closed-loop detection pose corresponding to the i-th target keyframe among n target keyframes.

[0161] S163: Determine a real-time dynamic measurement position constraint factor according to the inverse of the position and posture corresponding to each target key frame and the real-time dynamic measurement position corresponding to each target key frame.

[0162] The mobile device can determine the real-time dynamic measurement position constraint factor according to the inverse of the optimized pose corresponding to each target key frame in the pose graph and the RTK collected position information corresponding to each target key frame in the pose graph.

[0163] Specifically, the self-mobile device can determine the real-time dynamic measurement position constraint factor according to the following formula (14):

[0164] error 3 =T i .inverse()*T i _rtk............Formula (14)

[0165] Among them, error 3 represents the real-time dynamic measurement position constraint factor, T i _rtk represents the position information collected by RTK corresponding to the i-th target key frame among N target key frames.

[0166] S164, determining a global error equation according to the relative posture constraint factor, the closed-loop detection posture constraint factor, and the real-time dynamic measurement position constraint factor.

[0167] Specifically, the mobile device can determine the global error equation according to the following formula (15):

[0168] error=error 1 +error 2 +error 3 ............Formula (15).

[0169] The self-mobile device can also add the closed-loop detection posture constraint factor to the next sliding window, so that the closed-loop detection posture constraint factor is added when the local error equation is constructed in the sliding window next time.

[0170] S170. Perform global nonlinear optimization on the pose in the target feature map layer according to the global error equation to obtain a target global map.

[0171] Specifically, the self-moving device can perform global nonlinear optimization on the pose in the target feature map layer according to the global error equation to obtain a global map.

[0172] The global nonlinear optimization may be performed by the mobile device according to the global error equation to perform nonlinear optimization on the positions of all target key frames except the first key frame in the target feature map layer.

[0173] The mobile device can also perform global nonlinear optimization on the map feature points in the target feature map layer, making the land leveling Figure 1 The consistency is more reasonable and closer to the real scene.

[0174] Those skilled in the art will appreciate that the above embodiments are exemplary and are not intended to limit the present application. Where possible, the execution order of one or more of the above steps may be adjusted, or may be selectively combined to obtain one or more other embodiments. Those skilled in the art may select and combine any of the above steps as needed, and all those that do not depart from the essence of the present application fall within the scope of protection of the present application.

[0175] Figure 5 A schematic diagram of a global map constructed by the map construction method provided in an embodiment of the present application, such as Figure 5 As shown, the global map may include a target trajectory layer and a boundary layer in addition to the target feature map layer.

[0176] The target trajectory layer is used to determine the operating trajectory range of the self-moving device during its movement, and the boundary layer is used to distinguish between the operating range and the non-operating range and to plan the operating trajectory range for the self-moving device within the operating range.

[0177] Figure 6 A schematic diagram of a process for generating a target trajectory layer provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, when the global map also includes a target trajectory layer, the process may include the following steps:

[0178] S210: Obtain an initial trajectory layer.

[0179] Specifically, a new blank layer may be created, and the initial posture of the mobile device before moving is projected onto the blank layer to obtain an initial trajectory layer.

[0180] S220: adding the optimized pose of each target key frame to the initial trajectory layer to obtain a target trajectory layer.

[0181] Specifically, after the self-mobile device starts to move, the optimized posture of each target key frame can be added to the initial trajectory layer to obtain the target trajectory layer. The target trajectory layer includes the trajectory formed by the coordinates corresponding to the optimized posture of each target key frame, so that the self-mobile device can only operate within the trajectory range when working.

[0182] Figure 7 A flow chart of the boundary layer generation process provided in the embodiment of the present application is as follows: Figure 7 As shown, when the global map also includes a boundary layer, the process may include the following steps:

[0183] S310: Acquire boundary information in a color image, and determine a depth value corresponding to the boundary information from a depth image corresponding to the color image.

[0184] Specifically, an RGB image (ie, a color image) and a depth image collected by an RGB-D camera of a mobile device can be obtained while fusion positioning is being performed.

[0185] Specifically, boundary information can be extracted from the RGB image, and the boundary information is used to determine the operating range and the non-operating range. The boundary information may include a set of visible points representing the outermost layer and shadow boundary of the self-mobile device and a set of points at the junction of the occlusion and the background. The self-mobile device can determine the depth values ​​corresponding to the above-mentioned points from the depth image corresponding to the RGB image.

[0186] S320: Obtain the coordinates of the boundary information in the first coordinate system of the mobile device according to the boundary information and the depth value corresponding to the boundary information.

[0187] The self-moving device can obtain the coordinates of each point in the boundary information in the camera coordinate system of the self-moving device (ie, the first coordinate system) according to the depth value corresponding to each point in the boundary information.

[0188] S330. Determine the coordinates of the boundary information in the second coordinate system based on the acquired posture of the self-mobile device during movement and the coordinates of the boundary information in the first coordinate system, so that the coordinates of the boundary information in the second coordinate system serve as the coordinate information of the boundary layer.

[0189] The self-moving device can determine the coordinates of each point in the boundary information in the RTK base station coordinate system (i.e., the second coordinate system) based on its own acquired posture and the coordinates of each point in the boundary information in the camera coordinate system, add the coordinates of these points to the boundary layer, and then connect the points added to the boundary layer to form a working boundary. When the self-moving device is working, it can plan the route only within the range contained in the working boundary to ensure that the self-moving device does not exceed the working boundary.

[0190] In another embodiment of the present application, before acquiring the first data during the movement of the mobile device, the mobile device may first perform positioning initialization.

[0191] Figure 8 A flowchart of the positioning initialization process provided in the embodiment of the present application is shown in FIG. Figure 8 As shown, the process may include the following steps:

[0192] S410, controlling the mobile device to move from an initial position along a preset direction and a preset distance and then stop.

[0193] Specifically, the self-moving device can be controlled to move a set distance, such as 0.5 meters, along a straight line. During the movement of the self-moving device, RTK position coordinates can be collected at fixed intervals, and a linear fit can be performed on multiple groups of RTK position coordinates using the least squares method to obtain a straight line, and the azimuth of the straight line is used as the initial yaw angle of the self-moving device.

[0194] S420: Taking the roll angle, pitch angle and position information of the self-moving device when it stops as the initial position and posture of the self-moving device.

[0195] Specifically, the roll angle and pitch angle of the posture of the self-moving device collected by the IMU when the self-moving device stops moving can be used as the initial roll angle and pitch angle of the self-moving device.

[0196] The coordinates collected by RTK when the self-moving device stops moving are used as the initial coordinates of the self-moving device, that is, the first position information of the self-moving device.

[0197] The initial position and orientation of the self-moving device is determined according to the initial roll angle, pitch angle and initial coordinates of the self-moving device.

[0198] S430, transforming the initial posture to a second coordinate system where feature points in an initial visual image collected after the mobile device stops are located, so that the initial posture corresponds to coordinates in the second coordinate system one by one.

[0199] Specifically, the feature points in the visual image collected after the mobile device stops can be transformed into coordinates in the RTK base station coordinate system through the transformation of the initial posture.

[0200] S440: Obtain a feature map layer according to the coordinates of the feature points in the initial visual image in the second coordinate system.

[0201] Specifically, the corresponding feature points can be projected into the feature map layer according to the transformed coordinates in the RTK base station coordinate system to obtain the feature map layer, and the subsequent optimization of the pose in the target key frame is based on the initial pose of the self-moving device.

[0202] The map construction scheme provided in the embodiment of the present application first obtains the initial posture and visual image of the self-mobile device during movement, determines the local error equation based on the initial posture and visual image, then extracts the target key frame from the visual image, performs local nonlinear optimization on the target key frame using the local error equation to obtain the optimized posture of the target key frame, further adds the optimized posture of each target key frame to the target feature map layer, and finally determines the global error equation based on the optimized posture, and performs global nonlinear optimization on the posture in the target feature map layer based on the global error equation to obtain a global map. The map construction scheme provided in the embodiment of the present application combines the local error equation and the global error equation generated by the visual image and posture data to achieve dual optimization of the posture of the self-mobile device, not only reduces the local error of the posture of the key frame of the visual image during the movement of the self-mobile device, but also performs global error reduction on the posture in the entire target feature map layer, thereby improving the accuracy of map construction, so that the self-mobile device can still construct a map with high accuracy in scenes where the data obtained by some sensors are poor due to environments such as poor feature points, similar scenes, poor signals of real-time dynamic measurement instruments, and wheel slippage.

[0203] Based on the same inventive concept, an embodiment of the present application also provides a map construction device. Fig. 9 A schematic diagram of the structure of a map construction device provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the map construction device provided in the embodiment of the present application may include:

[0204] Acquisition module 11: used to acquire the initial position and visual image of the mobile device during its movement;

[0205] Extraction module 12: used for acquiring a plurality of target key frames from the visual image;

[0206] A local error determination module 13: used to determine a local error equation according to the initial posture and the visual image;

[0207] Local optimization module 14: used for performing local nonlinear optimization on the target key frame by using the local error equation to obtain the posture of the mobile device after the optimization of the target key frame;

[0208] Adding module 15: used for adding the optimized pose of each target key frame to the feature map layer to obtain a target feature map layer;

[0209] A global error determination module 16 is used to determine a global error equation according to the optimized posture;

[0210] The global optimization module 17 is used to perform global nonlinear optimization on the position and posture in the target feature map layer according to the global error equation to obtain a global map.

[0211] As an optional implementation manner, the local error determination module 13 is specifically used for:

[0212] Determine a predicted pose of the mobile device at the target key frame according to the initial pose, and determine a pose measurement error factor according to the predicted pose;

[0213] Extracting visual image feature points of the target key frame;

[0214] Determining a reprojection error factor based on the predicted pose and the visual image feature points;

[0215] Determine a real-time dynamic measurement error factor according to the real-time position information of the self-mobile device;

[0216] Determine a marginalization factor according to the feature point and the real-time position information;

[0217] The local error equation is determined according to the posture measurement error factor, the reprojection error factor, the real-time dynamic measurement error factor and the marginalization factor.

[0218] As an optional implementation manner, the local error determination module 13 is further used for:

[0219] Determine the coordinates of the visual image feature points of the target key frame in the feature map layer according to the predicted pose;

[0220] Determining a target feature point matched by the visual image feature point in the feature map layer according to the coordinates of the visual image feature point in the feature map layer;

[0221] According to the predicted position and posture, the target feature points are converted into the visual image to obtain matching feature points;

[0222] A reprojection error factor is determined based on the matching feature points and the visual image feature points.

[0223] As an optional implementation, the global error determination module 16 is used to:

[0224] Determining a relative posture constraint factor of the optimized posture of each target key frame according to the posture corresponding to each target key frame and the inverse of the posture corresponding to each target key frame;

[0225] Obtaining a closed-loop detection pose constraint factor according to the inverse of the pose corresponding to each of the target key frames and the closed-loop detection pose corresponding to each of the target key frames;

[0226] Determine a real-time dynamic measurement position constraint factor according to the inverse of the position and posture corresponding to each of the target key frames and the real-time dynamic measurement position corresponding to each of the target key frames;

[0227] A global error equation is determined according to the relative posture constraint factor, the closed-loop detection posture constraint factor and the real-time dynamic measurement position constraint factor.

[0228] As an optional embodiment, the global map also includes at least one of a target trajectory layer and a boundary layer, the target trajectory layer is used to determine the operating trajectory range of the self-moving device during movement, and the boundary layer is used to distinguish between the operating range and the non-operating range and to plan the operating trajectory range for the self-moving device within the operating range.

[0229] As an optional implementation manner, when the global map includes the target trajectory layer, the adding module 15 is further used for:

[0230] Get the initial trajectory layer;

[0231] The optimized pose of each target key frame is added to the initial trajectory layer to obtain the target trajectory layer.

[0232] As an optional implementation manner, when the global map includes a boundary layer, the visual image includes a color image and a depth image corresponding to the color image, and the map construction device is further used to:

[0233] Acquire boundary information in the color image, and determine a depth value corresponding to the boundary information from a depth image corresponding to the color image, wherein the boundary information is used to determine the operating range and the non-operating range;

[0234] According to the boundary information and the depth value corresponding to the boundary information, obtaining the coordinates of the boundary information in the first coordinate system of the mobile device;

[0235] Based on the acquired posture of the mobile device during movement and the coordinates of the boundary information in the first coordinate system, the coordinates of the boundary information in the second coordinate system are determined, so that the coordinates of the boundary information in the second coordinate system are used as the coordinate information of the boundary layer.

[0236] As an optional implementation, before obtaining the initial position and visual image from the mobile device during movement, the map building device is further used to:

[0237] Control the mobile device to move from an initial position along a preset direction and a preset distance before stopping;

[0238] The roll angle, pitch angle and position information of the self-moving device when it stops are used as the initial position and posture of the self-moving device;

[0239] Transforming the initial posture to a second coordinate system where the feature points in the initial visual image collected after the mobile device stops are located, so that the initial posture corresponds to the coordinates in the second coordinate system one by one;

[0240] A feature map layer is obtained according to the coordinates of the feature points in the initial visual image in the second coordinate system.

[0241] The map construction device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0242] Based on the same inventive concept, an embodiment of the present application also provides an electronic device. Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Fig.10 As shown, the electronic device provided in this embodiment includes: a memory 210 and a processor 220, the memory 210 is used to store computer programs; the processor 220 is used to execute the method described in the above method embodiment when calling the computer program.

[0243] The electronic device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0244] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in the above method embodiment is implemented.

[0245] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the electronic device implements the method described in the above method embodiment when executing the computer program product.

[0246] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0247] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk or a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0248] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media can include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0249] The naming or numbering of the steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0250] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0251] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some feature points can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0252] It should be understood that when used in the present application specification and the appended claims, the term "comprising" indicates the presence of described characteristic points, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other characteristic points, wholes, steps, operations, elements, components and / or collections thereof.

[0253] In the description of this application, unless otherwise specified, " / " indicates that the objects associated with each other are in an "or" relationship, for example, A / B can represent A or B; "and / or" in this application is merely a description of the association relationship between associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.

[0254] Furthermore, in the description of this application, unless otherwise specified, "plurality" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0255] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0256] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0257] References to "one embodiment" or "some embodiments" described in the specification of the present application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in the specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0258] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A map construction method, characterized in that: include: Obtaining the initial position and visual image of the mobile device during movement; Acquire a plurality of target key frames from the visual image; Determining a local error equation based on the initial pose and the visual image; Using the local error equation to perform local nonlinear optimization on the target key frame to obtain the posture of the mobile device after the optimization of the target key frame; Adding the optimized pose of each target key frame to the feature map layer corresponding to the initial visual image to obtain the target feature map layer; Determine a global error equation according to the optimized posture; Performing global nonlinear optimization on the pose in the target feature map layer according to the global error equation to obtain a target global map; Determining a local error equation according to the initial pose and the visual image includes: Acquire a predicted pose of the mobile device in the target key frame according to the initial pose, and determine a pose measurement error factor according to the predicted pose; Extracting visual image feature points of the target key frame; Determining a reprojection error factor based on the predicted pose and the visual image feature points; Determine a real-time dynamic measurement error factor according to the real-time position information of the self-mobile device; Determining a marginalization factor according to the visual image feature points and the real-time position information; Determine a local error equation according to the posture measurement error factor, the reprojection error factor, the real-time dynamic measurement error factor and the marginalization factor; Determining the global error equation according to the optimized posture comprises: Determining a relative posture constraint factor of the optimized posture of each target key frame according to the posture corresponding to each target key frame and the inverse of the posture corresponding to each target key frame; Obtaining a closed-loop detection pose constraint factor according to the inverse of the pose corresponding to each of the target key frames and the closed-loop detection pose corresponding to each of the target key frames; Determine a real-time dynamic measurement position constraint factor according to the inverse of the position and posture corresponding to each of the target key frames and the real-time dynamic measurement position corresponding to each of the target key frames; A global error equation is determined according to the relative posture constraint factor, the closed-loop detection posture constraint factor and the real-time dynamic measurement position constraint factor.

2. The method according to claim 1, characterized in that The determining of the reprojection error factor according to the predicted pose and the visual image feature points comprises: Determine the coordinates of the visual image feature points of the target key frame in the feature map layer according to the predicted pose; Determining a target feature point matched by the visual image feature point in the feature map layer according to the coordinates of the visual image feature point in the feature map layer; According to the predicted position and posture, the target feature points are converted into the visual image to obtain matching feature points; A reprojection error factor is determined based on the matching feature points and the visual image feature points.

3. The method according to claim 1 or 2, characterized in that: The global map also includes at least one of a target trajectory layer and a boundary layer, the target trajectory layer is used to determine the operating trajectory range of the self-moving device during movement, and the boundary layer is used to distinguish between an operating range and a non-operating range and to plan the operating trajectory range for the self-moving device within the operating range.

4. The method according to claim 3, characterized in that When the global map includes the target trajectory layer, the method further includes: Get the initial trajectory layer; The optimized pose of each target key frame is added to the initial trajectory layer to obtain the target trajectory layer.

5. The method according to claim 4, characterized in that When the visual image includes a color image and a depth image corresponding to the color image, and the global map includes a boundary layer, the method further includes: Acquire boundary information in the color image, and determine a depth value corresponding to the boundary information from a depth image corresponding to the color image; wherein the boundary information is used to determine the operating range and the non-operating range; According to the boundary information and the depth value corresponding to the boundary information, obtaining the coordinates of the boundary information in the first coordinate system of the mobile device; Based on the acquired posture of the mobile device during movement and the coordinates of the boundary information in the first coordinate system, the coordinates of the boundary information in the second coordinate system are determined, so that the coordinates of the boundary information in the second coordinate system are used as the coordinate information of the boundary layer.

6. The method according to claim 1 or 2, characterized in that: Before obtaining the initial position and visual image from the mobile device during movement, the method further includes: Control the mobile device to move from an initial position along a preset direction and a preset distance before stopping; The roll angle, pitch angle and position information of the self-moving device when it stops are used as the initial position and posture of the self-moving device; Transforming the initial posture to a second coordinate system where the feature points in the initial visual image collected after the mobile device stops are located, so that the initial posture corresponds to the coordinates in the second coordinate system one by one; A feature map layer is obtained according to the coordinates of the feature points in the initial visual image in the second coordinate system.

7. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method according to any one of claims 1 to 6 when calling the computer program.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images