A method, apparatus and device for processing an environmental map

Abstract

The application discloses a kind of processing method, device and equipment of environmental map, the method includes: the current stereo image that camera equipment of mapping robot is acquired;The current stereo image is processed, and the current key frame of the current stereo image is obtained;When determining that the current key frame is valid frame, the current key frame is inserted into the local environmental map that has been constructed, and new local environmental map is obtained.By the above-mentioned mode, the mapping efficiency of positioning and map creation SLAM is improved.

Description

Technical Field

[0001] This invention relates to the field of map processing technology, and more specifically to a method, apparatus, and device for processing environmental maps. Background Technology

[0002] Simultaneous localization and mapping (SLAM) is fundamental and crucial for mobile robots to solve various problems such as exploration, reconnaissance, and navigation in unknown environments. By placing the robot in an unknown environment and starting from an unknown location, it incrementally creates a map of the environment, and simultaneously uses this map for autonomous localization and navigation. In SLAM, robot localization depends on the environmental map, while the map creation process also depends on the robot's accurate localization; this is a highly correlated process that must be solved simultaneously.

[0003] The implementation of SLAM methods mainly considers four aspects: map representation, information perception, data association, and localization and mapping. Currently, in SLAM localization and mapping methods, map optimization calculations are relatively complex during localization and mapping. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a method, apparatus and device for processing environmental maps that overcomes or at least partially solves the above problems.

[0005] According to one aspect of the present invention, a method for processing an environmental map is provided, the method comprising:

[0006] Acquire the current stereoscopic image captured by the camera equipment of the composition robot;

[0007] The current stereoscopic image is processed to obtain the current keyframe of the current stereoscopic image;

[0008] When the current keyframe is determined to be a valid frame, it is inserted into the constructed local environment map to obtain a new local environment map.

[0009] According to another aspect of the present invention, an environmental map processing apparatus is provided, the apparatus comprising:

[0010] The acquisition module is used to acquire the current stereoscopic image obtained by the camera device of the modeling robot;

[0011] The processing module is used to process the current stereoscopic image to obtain the current keyframe of the current stereoscopic image; when the current keyframe is determined to be a valid frame, the current keyframe is inserted into the constructed local environment map to obtain a new local environment map.

[0012] According to another aspect of the present invention, a computing device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0013] The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described environment map processing method.

[0014] According to another aspect of the present invention, a computer storage medium is provided, the storage medium storing at least one executable instruction, the executable instruction causing a processor to perform an operation corresponding to the above-described environment map processing method.

[0015] According to the solution provided in the above embodiments of the present invention, the environmental map processing method can save a lot of computational costs and improve the efficiency of SLAM mapping, thereby solving the problem that the current SLAM localization and mapping methods have relatively complex calculations for map and attitude optimization; it has achieved the beneficial effects of simplifying keyframe insertion and discrimination time, saving computational costs, and improving the efficiency of SLAM mapping.

[0016] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more obvious and understandable, specific implementation methods of the embodiments of the present invention are described below. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0018] Figure 1 A flowchart of an environmental map processing method provided in an embodiment of the present invention is shown;

[0019] Figure 2 A schematic diagram of a node for calculating motion estimation entropy is shown in another embodiment of the present invention;

[0020] Figure 3 A schematic diagram of the structure of the environmental map processing device provided in an embodiment of the present invention is shown;

[0021] Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of the present invention is shown. Detailed Implementation

[0022] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0023] Figure 1 A flowchart illustrating an environmental map processing method provided in an embodiment of the present invention is shown. Figure 1 As shown, the method includes the following steps:

[0024] Step 11: Obtain the current stereo image captured by the camera device of the composition robot;

[0025] Step 12: Process the current stereoscopic image to obtain the current keyframe of the current stereoscopic image;

[0026] Step 13: When the current keyframe is determined to be a valid frame, the current keyframe is inserted into the constructed local environment map to obtain a new local environment map.

[0027] This embodiment of the invention acquires a current stereo image captured by the camera device of a mapping robot; processes the current stereo image to obtain a current keyframe; and when the current keyframe is determined to be a valid frame, inserts it into an existing local environment map to obtain a new local environment map. This simplifies the keyframe insertion into the environment map determination time, saves significant computational costs, and improves the mapping efficiency of SLAM.

[0028] In an optional embodiment of the present invention, the constructed local environment map is constructed through the following process:

[0029] Step 111: Obtain the existing keyframe set of the acquired stereoscopic images;

[0030] Step 112: Obtain the stereo feature information of the existing keyframes in the existing keyframe set;

[0031] Step 113: Determine the pose information of the camera device based on the stereo feature information of the existing keyframes;

[0032] Step 114: Generate a local environment map based on the stereo feature information of the existing keyframes and the pose information of the camera device.

[0033] In an optional embodiment of the present invention, step 114 may specifically include:

[0034] Step 1141: Obtain standard keyframes for the local environment map based on the stereo feature information of the existing keyframes.

[0035] Step 1142: Obtain the boundary of the local environment map based on the standard keyframes;

[0036] Step 1143: Based on the attitude information of the camera device, obtain the orientation information of the position of the local environment map in the entire map;

[0037] Step 1144: Generate the local environment map based on the boundary and direction information.

[0038] In this embodiment, a set of known keyframes of the stereo image is first processed, and 3D (stereo) point maps and 3D line segment landmarks are detected in the frames in the keyframe set. The generation tree of the graph structure (i.e., 3D point map) and the visibility map (3D line segment landmark) are output based on the detection results. Each node in the 3D point map and the visibility map represents a keyframe. The 3D position of the image in the set is calculated, and stereo feature information is output, including stereo feature values ​​and stereo feature descriptors. Here, the stereo feature values ​​can be stereo coordinate values.

[0039] Point features can be obtained from keyframe images using the ORB feature extraction method. Based on binary feature descriptors, fast and efficient keypoint matching can be performed. ORB is short for Oriented Fast and RotatedBrief, which can be used to quickly create feature vectors for keypoints in an image. These feature vectors can be used to identify objects in the image.

[0040] Line segment features are extracted using the linesegment detection (LSD) algorithm, which has high accuracy and repeatability. It uses the geometric information of the line segments themselves to filter out line segments with mismatched direction and length or high degree of difference, retaining more linear structure and making the map content richer.

[0041] The above steps yield the stereo features and stereo feature descriptors of the keyframes. The camera pose information is calculated using the stereo feature descriptors. Feature matching is performed using the descriptors of feature points and lines extracted from the image, thus obtaining the matching relationship between 3D and 2D features.

[0042] The 3D feature points and lines observed in the current frame are projected onto the 2D image plane of the next frame. Then, based on the error relationship between the projected points and lines and the matched points and lines between adjacent frames, a preliminary estimate of the camera pose is made to obtain the camera pose information.

[0043] The camera's pose information is used to determine the orientation information of the environment map; the standard frame is not fixed, and when the next frame arrives, the current frame can be used as the standard frame for the next frame to construct the environment map.

[0044] Further, obtain the stereo features and descriptors of all keyframes in the set to obtain a list of statistical values. Then, iterate through the keyframe features in the list to build and store the keyframe edges to obtain standard keyframes. Finally, determine the boundaries of the local environment map based on the boundaries of the standard keyframes.

[0045] In an optional embodiment of the present invention, step 12 may include:

[0046] Step 121: Extract point features from the current stereo image to obtain a stereo point map of the current stereo image;

[0047] Step 122: Extract line segment features from the current stereo image to obtain the visibility map of the current stereo image;

[0048] Step 123: Obtain the current keyframe of the current stereo image based on the stereo point map and the visibility map, wherein each node in the stereo point map and the visibility map represents a keyframe.

[0049] In this embodiment, for the current stereo image, 3D point map and 3D line segment landmark detection are also performed; the detection results output a generation tree of the graph structure (3D point map) and a visibility map (3D line segment landmark), where each node of the visibility map represents a keyframe, the 3D position of the image in the set is calculated, and stereo feature information is output, including stereo feature values ​​and stereo feature descriptors, where the stereo feature values ​​can be stereo coordinates.

[0050] In an optional embodiment of the present invention, step 13 above, determining that the current keyframe is a valid frame, includes:

[0051] Step 131: Obtain the covariance matrix of motion estimation between the current keyframe and the standard keyframe; further, this step may be followed by:

[0052] The current keyframe is used as the standard keyframe for the next keyframe, and the covariance matrix of motion estimation between the next keyframe and the current keyframe is obtained.

[0053] Step 132: Based on the covariance matrix, obtain the ratio of the motion estimation entropy of the current keyframe to the motion estimation entropy of the standard frame;

[0054] Step 133: When the ratio is less than a threshold, the current keyframe is determined to be a valid keyframe.

[0055] Here, the threshold is a preset value, or the threshold is the sine value of a line segment. The line segment is constructed by using the sequence of keyframes as the horizontal coordinate points and the sequence of the number of stereo features overlapping between each keyframe and the previous frame as the vertical coordinate points, forming a line segment composed of two coordinate points (SF_Ni, SF_Ci) and (SF_Ni+1, SF_Ci+1). Wherein, SF_Ni is the keyframe, SF_Ci is the number of stereo features overlapping between the keyframe and the previous frame, SF_Ni+1 is the next keyframe, and SF_Ci+1 is the number of stereo features overlapping between the keyframe and the next frame.

[0056] In this embodiment, a correspondence is established between two stereo frames (the new keyframe and the previous standard frame), using... Normal distribution modeling, where As a 6-dimensional vector representing the camera motion between frame t (the previous frame) and t+1 (the new keyframe) (two frames, each in three dimensions), For the covariance of motion estimation (the covariance matrix between the new keyframe and the previous frame), the keypoints and line segments are back-projected from the i-th frame to the i+1-th frame, that is, the current frame is used as the new standard frame.

[0057] To determine when to insert a new keyframe into the map, the uncertainty of relative motion estimation is used through the above formula.

[0058] We use h(ξ)=3(1+log2π)+0.5log(|∑ξ|) as the definition of motion estimation entropy;

[0059] For a given keyframe, calculate the motion estimation entropy h(ξ) between the previous keyframe i and the current keyframe i+u. i,i+u The ratio of the two entropy values ​​is calculated as follows:

[0060]

[0061] Considering the overlapping stereo features among multiple consecutive frames, the motion estimation entropy of a given keyframe may be correlated with multiple frames. Therefore, multiple motion estimation entropies (α) are calculated. i-n ,α i-n+1 ,α i-n+2 ,....α i ,...α i+2n-2 ,α i+2n-1 ,α i+2n The input value is used as the input value of the recurrent neural network, and α is obtained through the recurrent neural network. i Predictive techniques.

[0062] by Figure 2 For example, when n is 4, ci Represents α i Previous motion estimation entropy, p i Represents α i The subsequent motion estimation entropy, that is, using (α) i-4 ,α i-3 ,α i-2 ,α i-1 ,α i ,α i+1 ,α i+2 ,α i+3 ) as input value for α i The numerical prediction yields a value of α. i The motion estimation entropy.

[0063] Based on the calculated α as a threshold, when frame i+u is inserted into the process system as a new key frame, if its calculated value is less than the α value, it is determined to be a valid key frame and is regarded as a key frame that meets the system requirements.

[0064] In the above embodiments of the present invention, the number of overlapping stereo features is used as the threshold for motion estimation entropy. This threshold is represented by α, which is a threshold set according to different environmental scenarios before map construction. When frame i+u is inserted into the process system as a new keyframe, the comparison value α is calculated to determine whether the current frame is a valid keyframe.

[0065] First, the number of stereo features in each keyframe is counted. Each keyframe is represented by SF_Ni, and the number of stereo features that overlap between the keyframe SF_Ni and the previous frame SF_Ni-1 is denoted as SF_Ci. The sequence of SF_Ni and SF_Ci are used as the x and y coordinates, respectively, to construct the image. The sin value of the line segment formed by the two coordinate points (SF_Ni, SF_Ci) and (SF_Ni+1, SF_Ci+1) is used as the threshold α of the SF_Ni frame. That is, if the actual calculated value is less than α, it is determined to be a valid keyframe and is considered to be a keyframe that meets the system requirements.

[0066] In an optional embodiment of the present invention, step 13, inserting the current keyframe into the constructed local environment map to obtain a new local environment map, includes:

[0067] Step 131: Determine the attitude information of the current keyframe based on the current attitude information of the camera device;

[0068] Step 132: Convert the stereo feature information of the current keyframe into planar feature information;

[0069] Step 133: Add the observation values ​​of the landmarks corresponding to the planar feature information to the constructed local environment map to obtain a new local environment map.

[0070] In this embodiment, an index information table for keyframes is established; the 3D pose information of the keyframes is determined based on the current pose information of the camera device, including the absolute pose and the relative pose of the previous keyframe and its related uncertainties. New 3D landmarks are established, the stereo feature information of the current keyframe is converted into planar feature information, and the observation values ​​of existing landmarks are added to the map by storing the 2D image coordinates and descriptors of the frame, resulting in a new local environment map.

[0071] Furthermore, after obtaining the new local environment map, it may also include:

[0072] Step 134: Determine the actual location coordinates of the current keyframe based on the correspondence between the observed values ​​of the landmarks in the current keyframe and the actual landmarks in the local environment in the new local environment map;

[0073] Step 135: Obtain the position information of the camera device in the new local environment map based on the actual position coordinates of the current keyframe.

[0074] In this embodiment, a correspondence is established between the observation value of the current frame and the landmarks in the local environment. The observation feature value extracted from the current frame corresponds to the actual location coordinates in the real scene, thereby realizing the representation of the camera's position in the actual map.

[0075] In an optional embodiment of the present invention, after obtaining the new local environment map, the process may further include:

[0076] Step 136: Obtain the projection error between the observation values ​​of the i-th keyframe and the landmarks of the j-th map point in the new local environment map;

[0077] Step 137: Based on the projection error, update the actual position coordinates of the keyframes in the new local environment map to obtain the updated local environment map.

[0078] In this embodiment, the overall projection error ψ is first optimized.

[0079] use

[0080] Minimize the projection error between observations and the projection error between the landmarks projected onto the observation frame, where κ l , ρ l and L l This refers to the number of local keyframes, the number of point features, and the number of line segment features.

[0081] Define projection error e ij Let e ​​be the two-dimensional distance from the i-th keyframe to the j-th map point observation, mathematically expressed as e.ij =x ij -π(ξ i,w ,X wj Using the function π: Let the 3D point X in the j-th real coordinate system be... wj Transform to the local reference frame of the i-th keyframe, and then project this point onto the image.

[0082] use This represents the error between viewing the k-th line in the i-th frame of the image.

[0083] Where P w,k and Q w,k For the 3D endpoints of a line segment in the world coordinate system, l ik The linear equation for the i-th keyframe corresponding to the K-th line segment is expressed as l using the homogeneous coordinates of the 2D endpoints of the line segment. ik =p ik ×q ik The Levenberg-Marquardt method is used to optimize the projection error ψ, where the error increment is defined as Δψ = [H + λdiag(H)]. -1 J T We, H, and J represent the Hessian and Jacobian matrices, respectively, and the error vector e contains the projection errors of all points and lines. Using The above error is recursively processed until convergence, thus obtaining the optimal ψ. Then, the local keyframes and landmark positions are updated again.

[0084] The above embodiments of the present invention determine whether a new frame is a key frame by calculating the ratio of the motion estimation entropy of the newly inserted frame to that of the original video frame. This simplifies the key frame insertion and discrimination time, saves a lot of computational costs, and improves the efficiency of SLAM mapping.

[0085] Figure 3 A schematic diagram of the structure of the environmental map processing apparatus 30 provided in an embodiment of the present invention is shown. Figure 3 As shown, the device 30 includes:

[0086] The acquisition module 31 is used to acquire the current stereoscopic image obtained by the camera device of the composition robot;

[0087] The processing module 32 is used to process the current stereoscopic image to obtain the current keyframe of the current stereoscopic image; when the current keyframe is determined to be a valid frame, the current keyframe is inserted into the constructed local environment map to obtain a new local environment map.

[0088] Optionally, the constructed local environment map is built through the following process:

[0089] Obtain the existing keyframe set of the acquired stereo images;

[0090] Obtain the stereo feature information of the existing keyframes in the existing keyframe set;

[0091] The pose information of the camera device is determined based on the stereo feature information of the existing keyframes;

[0092] A local environment map is generated based on the stereo feature information of the existing keyframes and the pose information of the camera device.

[0093] Optionally, based on the stereo feature information of the existing keyframes and the pose information of the camera device, a local environment map is generated, including:

[0094] Based on the stereo feature information of the existing keyframes, standard keyframes for the local environment map are obtained.

[0095] Based on the standard keyframes, the boundaries of the local environment map are obtained;

[0096] Based on the attitude information of the camera device, the orientation information of the position of the local environment map in the entire map is obtained;

[0097] The local environment map is generated based on the boundary and direction information.

[0098] Optionally, the current stereoscopic image is processed to obtain the current keyframe of the current stereoscopic image, including:

[0099] Point features are extracted from the current stereo image to obtain a stereo point map of the current stereo image;

[0100] Line segment features are extracted from the current stereo image to obtain the visibility map of the current stereo image;

[0101] Based on the stereo point map and the visibility map, the current keyframe of the current stereo image is obtained, wherein each node in the stereo point map and the visibility map represents a keyframe.

[0102] Optionally, determining that the current keyframe is a valid frame includes:

[0103] Obtain the covariance matrix of motion estimation between the current keyframe and the standard keyframe;

[0104] Based on the covariance matrix, the ratio of the motion estimation entropy of the current keyframe to the motion estimation entropy of the standard frame is obtained;

[0105] When the ratio is less than a threshold, the current keyframe is determined to be a valid keyframe.

[0106] Optionally, the threshold is a preset value; or, the threshold is the sine value of a line segment, wherein the line segment is constructed by using the sequence of keyframes as the horizontal coordinate points and the sequence of the number of stereo features overlapping between each keyframe and the previous frame as the vertical coordinate points, forming a line segment composed of two coordinate points (SF_Ni, SF_Ci) and (SF_Ni+1, SF_Ci+1); where SF_Ni is a keyframe, SF_Ci is the number of stereo features overlapping between the keyframe and the previous frame, SF_Ni+1 is the next keyframe, and SF_Ci+1 is the number of stereo features overlapping between the keyframe and the next frame.

[0107] Optionally, the current keyframe is inserted into the constructed local environment map to obtain a new local environment map, including:

[0108] The attitude information of the current keyframe is determined based on the current attitude information of the camera device;

[0109] Convert the stereo feature information of the current keyframe into planar feature information;

[0110] The observation values ​​of the landmarks corresponding to the planar feature information are added to the constructed local environment map to obtain a new local environment map.

[0111] Optionally, the processing module 32 is further configured to determine the actual location coordinates of the current keyframe based on the correspondence between the observed values ​​of the landmarks in the current keyframe and the actual landmarks in the local environment in the new local environment map;

[0112] Based on the actual position coordinates of the current keyframe, the position information of the camera device in the new local environment map is obtained.

[0113] Optionally, the processing module 32 is further configured to obtain the projection error between the observation values ​​of the i-th keyframe and the landmarks of the j-th map point in the new local environment map;

[0114] Based on the projection error, the actual position coordinates of the keyframes in the new local environment map are updated to obtain the updated local environment map.

[0115] It should be noted that this device is the same as the method described above. All implementations in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0116] This invention provides a non-volatile computer storage medium storing at least one executable instruction that can execute the environment map processing method in any of the above method embodiments.

[0117] Figure 4 The diagram shows a structural schematic of a computing device provided in an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computing device.

[0118] like Figure 4 As shown, the computing device may include a processor, a communications interface, memory, and a communications bus.

[0119] The processor, communication interface, and memory communicate with each other via a communication bus. The communication interface is used to communicate with other network elements, such as clients or other servers. The processor executes programs, specifically the steps described in the embodiment of the method for processing the environment map of the computing device.

[0120] Specifically, the program may include program code, which includes computer operation instructions.

[0121] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0122] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive.

[0123] Specifically, the program can be used to cause the processor to execute the environment map processing method in any of the above method embodiments. The specific implementation of each step in the program can be found in the corresponding descriptions of the steps and units in the above environment map processing embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0124] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the embodiments of the present invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the embodiments of the present invention.

[0125] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0126] Similarly, it should be understood that, in order to streamline the embodiments of the invention and aid in understanding one or more of the various inventive aspects, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0127] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0128] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0129] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The embodiments of the present invention can also be implemented as device or apparatus programs (e.g., computer programs and computer program products) for performing part or all of the methods described herein. Such programs implementing the embodiments of the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0130] It should be noted that the above embodiments are illustrative of the present invention and not restrictive of the invention, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of the present invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for processing environmental maps, characterized in that, The method includes: Acquire the current stereoscopic image captured by the camera equipment of the composition robot; The current stereoscopic image is processed to obtain the current keyframe of the current stereoscopic image; When the current keyframe is determined to be a valid frame, it is inserted into the constructed local environment map to obtain a new local environment map. Determining the current keyframe as a valid frame includes: obtaining the motion estimation covariance matrix between the current keyframe and a standard keyframe; obtaining the standard keyframe of the constructed local environment map based on the stereo feature information of the existing keyframes; obtaining the ratio of the motion estimation entropy of the current keyframe to the motion estimation entropy of the standard frame based on the covariance matrix; and determining the current keyframe when the ratio is less than a threshold. A valid keyframe is defined as follows: the threshold is the sine value of a line segment, which is constructed by using the sequence of keyframes as the horizontal coordinate points and the sequence of the number of stereo features overlapping between each keyframe and the previous frame as the vertical coordinate points, forming a line segment composed of two coordinate points (SF_Ni, SF_Ci) and (SF_Ni+1, SF_Ci+1); where SF_Ni is the keyframe, SF_Ci is the number of stereo features overlapping between the keyframe and the previous frame, SF_Ni+1 is the next keyframe, and SF_Ci+1 is the number of stereo features overlapping between the keyframe and the next frame.

2. The environmental map processing method according to claim 1, characterized in that, The existing local environment map is constructed through the following process: Obtain the existing keyframe set of the acquired stereo images; Obtain the stereo feature information of the existing keyframes in the existing keyframe set; The pose information of the camera device is determined based on the stereo feature information of the existing keyframes; A local environment map is generated based on the stereo feature information of the existing keyframes and the pose information of the camera device.

3. The environmental map processing method according to claim 2, characterized in that, Based on the stereo feature information of the existing keyframes and the pose information of the camera device, a local environment map is generated, including: Based on the stereo feature information of the existing keyframes, standard keyframes for the local environment map are obtained. Based on the standard keyframes, the boundaries of the local environment map are obtained; Based on the attitude information of the camera device, the orientation information of the position of the local environment map in the entire map is obtained; The local environment map is generated based on the boundary and direction information.

4. The environmental map processing method according to claim 3, characterized in that, Processing the current stereo image to obtain the current keyframe of the current stereo image includes: Point features are extracted from the current stereo image to obtain a stereo point map of the current stereo image; Line segment features are extracted from the current stereo image to obtain the visibility map of the current stereo image; Based on the stereo point map and the visibility map, the current keyframe of the current stereo image is obtained, wherein each node in the stereo point map and the visibility map represents a keyframe.

5. The environmental map processing method according to claim 4, characterized in that, Inserting the current keyframe into the constructed local environment map yields a new local environment map, including: The attitude information of the current keyframe is determined based on the current attitude information of the camera device; Convert the stereo feature information of the current keyframe into planar feature information; The observation values ​​of the landmarks corresponding to the planar feature information are added to the constructed local environment map to obtain a new local environment map.

6. The environmental map processing method according to claim 5, characterized in that, After obtaining the new local environment map, it also includes: The actual location coordinates of the current keyframe are determined based on the correspondence between the observed values ​​of the landmarks in the current keyframe and the actual landmarks in the local environment in the new local environment map. Based on the actual position coordinates of the current keyframe, the position information of the camera device in the new local environment map is obtained.

7. The environmental map processing method according to claim 5, characterized in that, After obtaining the new local environment map, it also includes: Obtain the The first keyframe is added to the new local environment map. Projection error between the observations of landmarks at map points; Based on the projection error, the actual position coordinates of the keyframes in the new local environment map are updated to obtain the updated local environment map.

8. An environmental map processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the current stereoscopic image obtained by the camera device of the modeling robot; A processing module is used to process the current stereo image to obtain the current keyframe of the current stereo image; when the current keyframe is determined to be a valid frame, the current keyframe is inserted into the constructed local environment map to obtain a new local environment map; wherein, determining the current keyframe as a valid frame includes: obtaining the motion estimation covariance matrix between the current keyframe and the standard keyframe; the standard keyframe obtains the standard keyframe of the constructed local environment map based on the stereo feature information of the existing keyframes; based on the covariance matrix, the ratio of the motion estimation entropy of the current keyframe to the motion estimation entropy of the standard frame is obtained; in the... When the ratio is less than a threshold, the current keyframe is determined to be a valid keyframe. The threshold is the sine value of a line segment. The line segment is constructed by using the sequence of keyframes as the horizontal coordinate points and the sequence of the number of stereo features overlapping between each keyframe and the previous frame as the vertical coordinate points. The line segment is formed by two coordinate points (SF_Ni, SF_Ci) and (SF_Ni+1, SF_Ci+1). Wherein, SF_Ni is the keyframe, SF_Ci is the number of stereo features overlapping between the keyframe and the previous frame, SF_Ni+1 is the next keyframe, and SF_Ci+1 is the number of stereo features overlapping between the keyframe and the next frame.

9. A computing device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform an operation corresponding to the environmental map processing method as described in any one of claims 1-7.

10. A computer storage medium storing at least one executable instruction that causes a processor to perform an operation corresponding to the environmental map processing method as described in any one of claims 1-7.

Images