A space line position calculation method, a movable robot and an electronic device

Abstract

The embodiment of the application provides a space line position calculation method, a movable robot and electronic equipment, which are applied to the field of information technology, can obtain at least two image frames; a first line segment corresponding to a target space line in a first frame of two target image frames is subjected to coordinate feature extraction; a first grid region matched with the first line segment is identified according to the coordinate feature of the first line segment; a second grid region in a second frame of the two target image frames corresponding to the position of the first grid region in the target image frame is identified; the coordinate features of each line segment in the second grid region are identified, and the coordinate feature of the first line segment is matched with the coordinate features of each line segment in the second grid region to obtain the coordinate feature of a second line segment matched; and the space position of the target space line is calculated according to the coordinate feature of the first line segment and the coordinate feature of the second line segment. Thus, the calculation amount of the matching process is reduced, and the matching efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a method for calculating the position of a spatial line, a mobile robot, and an electronic device. Background Technology

[0002] Currently, it is becoming increasingly common to acquire images of spatial lines using image acquisition devices and then calculate their spatial coordinates based on the positions of the lines in the acquired images. For example, in SLAM (Simultaneous Localization and Mapping), calculating the spatial coordinates of spatial lines can assist in map construction, thereby enhancing map accuracy and positioning robustness.

[0003] Currently, calculating the spatial coordinates of a line often requires matching a line segment in one image frame with line segments in another image frame to determine the coordinates of the corresponding line segment in the other image frame, and then determining the position of the line based on these coordinates. However, existing techniques often use brute-force matching to match a line segment with all line segments in another image frame, which is not only computationally intensive but also inefficient. Summary of the Invention

[0004] The purpose of this application is to provide a method for calculating the position of a spatial line, a mobile robot, and an electronic device, to solve the problem of low line segment matching efficiency when determining the position of a spatial line in the prior art. The specific technical solution is as follows:

[0005] A first aspect of this application provides a method for calculating the position of a spatial line, including:

[0006] Acquire at least two image frames, wherein the at least two image frames include two target image frames acquired by the image acquisition device during movement of the device from the same target spatial line;

[0007] Coordinate features are extracted from the first line segment corresponding to the target space line in the first frame of the two target image frames.

[0008] Based on the coordinate characteristics of the first line segment, a first grid region matching the first line segment is identified, wherein each target image frame includes multiple grid regions, and the grid regions of each target image frame correspond one-to-one;

[0009] Based on the position of the first grid region in the target image frame, identify the second grid region in the second frame of the two target image frames corresponding to that position;

[0010] Identify the coordinate features of each line segment within the second grid area, and match the coordinate features of the first line segment with the coordinate features of each line segment within the second grid area to obtain the coordinate features of the matching second line segment;

[0011] The spatial position of the target spatial line is calculated based on the coordinate characteristics of the first line segment and the second line segment.

[0012] Optionally, after calculating the spatial position of the target spatial line based on the coordinate characteristics of the first line segment and the second line segment, the method further includes:

[0013] Obtain the coordinate positions of multiple spatial lines calculated from multiple image frames, wherein the multiple spatial lines include the target spatial line;

[0014] Determine at least one of the selected spatial lines from the plurality of spatial lines that meets the following conditions:

[0015] For any given spatial line, the ratio of the depth calculated based on the coordinate position of that spatial line to the average depth of all spatial lines is within a preset range;

[0016] For any spatial line, the ratio of the number of image frames containing the line segment corresponding to that spatial line to the total number of image frames in the multi-frame image frame is greater than a preset value;

[0017] A target map is constructed based on at least one filtered spatial line, wherein the target map is a map with spatial straight line features.

[0018] Optionally, constructing the target map based on the at least one filtered spatial line includes:

[0019] Determine the specified image frames among the multiple image frames;

[0020] For each filtered spatial line, determine the number of consecutive frames in the specified multi-frame image containing the line segments corresponding to each spatial line, and set the weight of each spatial line according to the determined number of consecutive frames.

[0021] The target map is constructed based on the weights of each of the aforementioned spatial lines.

[0022] Optionally, the step of identifying a second grid region in the second frame of two target image frames corresponding to the first grid region at the target image frame position includes:

[0023] Create an index relationship between the first grid region and the first line segment;

[0024] Based on the index relationship, the second grid region in the second frame is identified.

[0025] Optionally, the step of identifying the coordinate features of each line segment within the second grid area and matching the coordinate features of the first line segment with the coordinate features of each line segment within the second grid area to obtain the coordinate features of the matching second line segment includes:

[0026] The coordinate features of each line segment within the second grid area are identified and vectorized to obtain the feature vectors of each line segment within the second grid area.

[0027] Calculate the feature vector corresponding to the first line segment based on the coordinate characteristics of the first line segment, and calculate the straight line feature descriptor distance between the feature vector corresponding to the first line segment and the feature vector of each line segment in the second grid area.

[0028] The line segment with the smallest corresponding descriptor distance among all line segments within the second grid area is determined as the second line segment.

[0029] Optionally, calculating the spatial position of the target spatial line based on the coordinate characteristics of the first line segment and the second line segment includes:

[0030] Based on the coordinate characteristics of the first line segment and the second line segment, a reprojection error function for the target spatial line is created using a preset function.

[0031] The spatial position of the target spatial line is obtained by calculating the spatial position when the error corresponding to the reprojection error function is minimized.

[0032] Optionally, the step of creating the reprojection error function of the target spatial line based on the coordinate characteristics of the first line segment and the second line segment using a preset function includes:

[0033] Based on the coordinate characteristics of the first line segment and the second line segment, a reprojection error function for the target spatial line is created using a preset robust kernel function:

[0034]

[0035] Where, ρ p ρ l For Huber robust kernel function, Represents the information matrix, e l e represents the projection error of the line feature. p Represents the projection error of a point.

[0036] A second aspect of this application provides a mobile robot, including an image acquisition module and a processor:

[0037] The image acquisition module is used to acquire at least two image frames, wherein the at least two image frames include two target image frames acquired by the image acquisition device during the movement of the device from the same target spatial line.

[0038] The processor is configured to extract coordinate features from a first line segment corresponding to a target spatial line in the first frame of the two target image frames; identify a first grid region matching the first line segment based on its coordinate features, wherein each target image frame includes multiple grid regions, and the grid regions of each target image frame correspond one-to-one; identify a second grid region in the second frame of the two target image frames corresponding to the position of the first grid region in the target image frame; identify the coordinate features of each line segment in the second grid region, and match the coordinate features of the first line segment with the coordinate features of each line segment in the second grid region to obtain the coordinate features of the matching second line segment; and calculate the spatial position of the target spatial line based on the coordinate features of the first line segment and the coordinate features of the second line segment.

[0039] Optionally, the processor is further configured to acquire the coordinate positions of multiple spatial lines calculated from multiple image frames, wherein the multiple spatial lines include the target spatial line; determine at least one filtered spatial line among the multiple spatial lines that satisfies the following conditions: for any spatial line, the ratio of the depth calculated based on the coordinate position of the spatial line to the average depth of all spatial lines is within a preset range; for any spatial line, the ratio of the number of image frames containing the line segment corresponding to the spatial line to the total number of image frames is greater than a preset value; and construct a target map based on the at least one filtered spatial line, wherein the target map is a map with spatial straight line features.

[0040] Optionally, the processor is specifically configured to: determine multiple specified image frames in the multiple image frames; for each filtered spatial line, determine the number of consecutive frames in the multiple specified image frames containing the line segments corresponding to each spatial line, and set the weight of each spatial line according to the determined number of consecutive frames; and construct a target map according to the weight of each spatial line.

[0041] Optionally, the processor is specifically configured to create an index relationship between the first grid region and the first line segment; and to identify a second grid region in the second frame based on the index relationship.

[0042] Optionally, the processor is specifically configured to identify the coordinate features of each line segment in the second grid area and vectorize them to obtain the feature vector of each line segment in the second grid area; calculate the feature vector corresponding to the first line segment based on the coordinate features of the first line segment, and calculate the straight line feature descriptor distance between the feature vector corresponding to the first line segment and the feature vector of each line segment in the second grid area; and determine the line segment with the smallest corresponding descriptor distance among the line segments in the second grid area as the second line segment.

[0043] Optionally, the processor is specifically configured to create a reprojection error function of the target spatial line based on the coordinate characteristics of the first line segment and the coordinate characteristics of the second line segment using a preset function; calculate the spatial position where the error corresponding to the reprojection error function is minimized, and obtain the spatial position of the target spatial line.

[0044] Optionally, the processor is specifically configured to create a reprojection error function for the target spatial line based on the coordinate characteristics of the first line segment and the coordinate characteristics of the second line segment using a preset robust kernel function.

[0045]

[0046] Where, ρ p ρ l For Huber robust kernel function, Represents the information matrix, e l e represents the projection error of the line feature. p Represents the projection error of a point.

[0047] A third aspect of this application provides a spatial line position calculation device, comprising:

[0048] The image frame acquisition module is used to acquire at least two image frames, wherein the at least two image frames include two target image frames acquired by the image acquisition device during the movement of the device from the same target spatial line.

[0049] The feature extraction module is used to extract coordinate features from the first line segment corresponding to the target space line in the first frame of the two target image frames.

[0050] The range calculation module is used to identify a first grid region that matches the first line segment based on the coordinate characteristics of the first line segment, wherein each target image frame includes multiple grid regions, and the grid regions of each target image frame correspond one-to-one.

[0051] The line segment matching module is used to identify the second grid region in the second frame of two target image frames corresponding to the first grid region at the position of the target image frame;

[0052] The coordinate acquisition module is used to identify the coordinate features of each line segment in the second grid area, and match the coordinate features of the first line segment with the coordinate features of each line segment in the second grid area to obtain the coordinate features of the matching second line segment.

[0053] The position calculation module is used to calculate the spatial position of the target spatial line based on the coordinate characteristics of the first line segment and the coordinate characteristics of the second line segment.

[0054] Optionally, the device further includes:

[0055] The coordinate position acquisition submodule is used to acquire the coordinate positions of multiple spatial lines calculated from multiple image frames, wherein the multiple spatial lines include the target spatial line;

[0056] The spatial line filtering submodule is used to determine at least one filtered spatial line from the plurality of spatial lines that meets the following conditions:

[0057] For any given spatial line, the ratio of the depth calculated based on the coordinate position of that spatial line to the average depth of all spatial lines is within a preset range;

[0058] For any spatial line, the ratio of the number of image frames containing the line segment corresponding to that spatial line to the total number of image frames in the multi-frame image frame is greater than a preset value;

[0059] The map construction submodule is used to construct a target map based on the at least one filtered spatial line, wherein the target map is a map with spatial straight line features.

[0060] Optionally, the map building submodule includes:

[0061] A designated frame determination unit is used to determine multiple designated image frames among the multiple image frames;

[0062] A continuous frame determination unit is used to determine, for each filtered spatial line, the number of continuous frames containing the line segments corresponding to each spatial line in the multi-frame specified image frame, and to set the weight of each spatial line according to the determined number of continuous frames.

[0063] A map building unit is used to construct a target map based on the weights of the aforementioned spatial lines.

[0064] Optionally, the line segment matching module includes:

[0065] The index creation submodule is used to create the index relationship between the first grid region and the first line segment;

[0066] The grid recognition submodule is used to identify the second grid region in the second frame based on the index relationship.

[0067] Optionally, the coordinate acquisition module includes:

[0068] The coordinate quantization submodule is used to identify the coordinate features of each line segment in the second grid area and vectorize them to obtain the feature vector of each line segment in the second grid area.

[0069] The distance calculation submodule is used to calculate the feature vector corresponding to the first line segment based on the coordinate characteristics of the first line segment, and to calculate the straight line feature descriptor distance between the feature vector corresponding to the first line segment and the feature vector of each line segment in the second grid area.

[0070] The coordinate feature acquisition submodule is used to determine the line segment with the smallest corresponding descriptor distance among all line segments within the second grid area as the second line segment.

[0071] Optionally, the location calculation module includes:

[0072] The function creation submodule is used to create the reprojection error function of the target space line based on the coordinate characteristics of the first line segment and the coordinate characteristics of the second line segment using a preset function.

[0073] The function solver submodule is used to calculate the spatial position where the error corresponding to the reprojection error function is minimized, thereby obtaining the spatial position of the target spatial line.

[0074] Optionally, the function creation submodule is specifically used to create a reprojection error function for the target spatial line based on the coordinate characteristics of the first line segment and the coordinate characteristics of the second line segment using a preset robust kernel function.

[0075]

[0076] Where, ρ p ρ l For Huber robust kernel function, Represents the information matrix, e l e represents the projection error of the line feature. p Represents the projection error of a point.

[0077] In another aspect of this application, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0078] Memory, used to store computer programs;

[0079] The processor, when executing a program stored in memory, implements any of the above-mentioned spatial line position calculation methods.

[0080] In another aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program, when executed by a processor, implements any of the above-described spatial line position calculation methods.

[0081] In another aspect of the embodiments of this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the above-described spatial line position calculation methods.

[0082] Beneficial effects of the embodiments in this application:

[0083] This application provides a method for calculating the position of a spatial line, a mobile robot, and an electronic device. The method can acquire at least two image frames, including two target image frames acquired by an image acquisition device during movement of the same target spatial line. The method involves: extracting coordinate features of a first line segment corresponding to the target spatial line in the first frame of the two target image frames; identifying a first grid region matching the first line segment based on its coordinate features, wherein each target image frame includes multiple grid regions, and the grid regions of each target image frame correspond one-to-one; identifying a second grid region in the second frame of the two target image frames corresponding to the first grid region at its position in the target image frame; identifying the coordinate features of each line segment within the second grid region and matching the coordinate features of the first line segment with those of the line segments within the second grid region to obtain the coordinate features of the matching second line segment; and calculating the spatial position of the target spatial line based on the coordinate features of the first and second line segments. By identifying the second grid region in the second frame of the two target image frames corresponding to the first grid region at the target image frame position, and then matching the coordinate features of the first line segment with the coordinate features of each line segment in the second grid region to obtain the coordinate features of the matching second line segment, it is not necessary to match the first line segment with all line segments in the second frame, thereby reducing the computational load of the matching process, improving the matching efficiency, and solving the problem of low line segment matching efficiency when determining the position of spatial lines in the prior art.

[0084] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0085] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0086] Figure 1 A schematic flowchart of a spatial line position calculation method provided in an embodiment of this application;

[0087] Figure 2 A schematic diagram illustrating the process of constructing a target map provided in an embodiment of this application;

[0088] Figure 3 This is a schematic diagram of the process for identifying the second grid region provided in an embodiment of this application;

[0089] Figure 4 A schematic diagram illustrating an example of the spatial line position calculation method provided in this application.

[0090] Figure 5 A schematic diagram of the process for obtaining the coordinate features of the second line segment provided in an embodiment of this application;

[0091] Figure 6 A schematic diagram of the first line segment provided for an embodiment of this application;

[0092] Figure 7 A flowchart illustrating the calculation of the spatial position of a target spatial line provided in an embodiment of this application;

[0093] Figure 8 A schematic diagram illustrating the feature error provided in the embodiments of this application;

[0094] Figure 9 A schematic diagram illustrating another example of the spatial line position calculation method provided in this application embodiment;

[0095] Figure 10 This is a schematic diagram of the structure of a mobile robot provided in an embodiment of this application;

[0096] Figure 11 A schematic diagram of a spatial line position calculation device provided in an embodiment of this application;

[0097] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0098] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0099] First, the technical terms that may be used in the embodiments of this application will be explained:

[0100] SLAM: Simultaneous Localization and Mapping (SLAM) is the problem of localization and mapping of a robot as it moves in its environment.

[0101] LSD (Line Segment Detector) is an algorithm that can obtain sub-pixel-level line detection results in linear time. It is applicable to any digital image without parameter tuning and allows control over the number of false detections.

[0102] LBD: Line Binary Descriptor (LBD) has advantages such as rotation and illumination invariance, and is independent of the length of the line segment. Hamming distance can be used as the distance metric between the feature vectors of two line segments.

[0103] BF Matching: Brute-Force Matcher (BF) selects a feature descriptor from set A, calculates similarity with all other features in set B, and returns the closest match.

[0104] FLANN matching: Fast Library for Approximate Nearest Neighbors (FLANN) is an optimized algorithm library for fast nearest neighbor search on large-scale high-dimensional datasets.

[0105] A first aspect of this application provides a method for calculating the position of a spatial line, see [link to previous document]. Figure 1 ,include:

[0106] Step S11: Acquire at least two image frames.

[0107] The at least two image frames include two target image frames acquired by the image acquisition device during its movement, focusing on the same target spatial line. In this embodiment, the image acquisition device is a camera, specifically a global exposure camera with a fixed focal length, which can be mounted on a movable device. For example, when the method of this embodiment is applied to SLAM, the camera can be mounted on a robot, allowing the robot to acquire images of the surrounding environment and obtain at least two image frames.

[0108] The method of this application embodiment is applied to a smart terminal and can be implemented through a smart terminal. Specifically, when the method of this application embodiment is applied to SLAM, the smart terminal can be a robot that collects environmental images and performs image acquisition and spatial line position calculation through the robot, or it can be a back-end server. For example, when applied to a server, the robot can collect environmental images to obtain multiple image frames, and then send the image frames to the back-end server to perform spatial line position calculation through the back-end server.

[0109] Step S12: Extract coordinate features from the first line segment corresponding to the target space line in the first frame of the two target image frames.

[0110] The first line segment is the line segment in the first frame corresponding to the target spatial line among at least one spatial line. This spatial line can be the edge line of an object or pattern when capturing images of the surrounding environment, such as the lines corresponding to the four sides of a rectangular table, or the lines corresponding to the four edges of a rectangular pattern on the ground. The two target image frames are two image frames obtained by capturing images of the same target spatial line. When extracting coordinate features from the first line segment, the coordinates of its start and end points can be extracted.

[0111] In SLAM systems, line features offer more constraints than point features and are more robust to changes in lighting. Furthermore, line features represent a higher-order representation of the environment than point features, enabling the construction of environmental maps with intuitive geometric information. They are more evenly distributed across the image, less prone to clustering, and more stable at wide viewing angles, exhibiting greater robustness to motion blur. For example, in monocular SLAM systems, if the image acquisition device moves at a constant speed, under this assumption, image grid constraints and spatial constraints can improve the accuracy of monocular spatial line generation, as well as the overall accuracy and robustness of the monocular SLAM system.

[0112] Step S13: Identify the first grid region that matches the first line segment based on the coordinate characteristics of the first line segment.

[0113] Each target image frame includes multiple grid regions, and the grid regions of each target image frame correspond one-to-one. The range of the grid region is smaller than the range of the image frame.

[0114] In this embodiment, the size and specifications of each grid region can be predetermined, and the grid is the same for each image frame. Specifically, the grid region contains the predicted coordinates of the second line segment and is smaller than the range of the second frame image. For example, one or more grids containing the first line segment can be identified based on the start and end points of the first line segment. For example, the grids containing the start and end points of the first line segment can be identified and then marked as the first grid region matching the first line segment.

[0115] Step S14: Based on the position of the first grid region in the target image frame, identify the second grid region in the second frame of the two target image frames corresponding to that position.

[0116] In this embodiment, since the interval between the two image frames is short, it can be assumed that the position of the target spatial line in the first frame and the second frame is similar. Therefore, based on the position of the first grid region in the target image frame, the second grid region in the second frame of the two target image frames corresponding to that position can be identified. It can be assumed that the position of the target spatial line in the two frames is in the second grid region or in the grid adjacent to that grid.

[0117] Specifically, based on the coordinate characteristics of the first line segment, after identifying the first grid region that matches the first line segment, an index can be built based on the identified first grid region, and then the second grid region can be obtained by matching through the index.

[0118] Step S15: Identify the coordinate features of each line segment in the second grid area, and match the coordinate features of the first line segment with the coordinate features of each line segment in the second grid area to obtain the coordinate features of the matching second line segment.

[0119] After establishing an index by grid and roughly matching the index values ​​to identify the second grid region in the second frame, the coordinate features of each line segment within the second grid region can be identified, and the coordinate features of the first line segment can be brute-force matched with the coordinate features of each line segment within the second grid region.

[0120] Optionally, the coordinate features of each line segment within the second grid area can be identified based on the movement characteristics of the image acquisition device. Specifically, the movement characteristics of the image acquisition device can include its speed. Based on the speed of the image acquisition device and the time interval between two image frames, the displacement of the image acquisition device can be calculated. Then, based on this displacement, the corresponding line segment can be identified and matched. For example, the predicted position can be obtained through a kinematic model, and LBD vector matching can be performed based on the position index of the straight line contained in the previous frame to obtain the coordinate features of the matching second line segment.

[0121] Step S16: Calculate the spatial position of the target spatial line based on the coordinate characteristics of the first line segment and the second line segment.

[0122] Specifically, based on the coordinate characteristics of the first and second line segments, the spatial position of the target spatial line is calculated, allowing the creation of an expression for the target spatial line. This expression is then solved using the coordinate characteristics of the first and second line segments. More specifically, a spatial straight line can be established through triangulation based on the coordinate characteristics of the first and second line segments. For example, the pixel positions X1(u1,v1,1) and X2(u2,v1,1) of the two endpoints of the straight line in different image frames... n ,1), Calculate the depths s1 and s2 of the known pose recovery line. 2n The position of the line in space can then be obtained:

[0123] s2X 2n =s1RX1+t

[0124] As can be seen, the method of this application embodiment can identify the second grid region in the second frame of the two target image frames corresponding to the first grid region at the target image frame position, and then match the coordinate features of the first line segment with the coordinate features of each line segment in the second grid region to obtain the coordinate features of the matching second line segment. It is not necessary to match the first line segment with all line segments in the second frame, thereby reducing the amount of computation in the matching process, improving the matching efficiency, and solving the problem of low line segment matching efficiency when determining the position of spatial lines in the prior art.

[0125] Optional, see Figure 2 After calculating the spatial position of the target spatial line based on the coordinate characteristics of the first line segment and the second line segment, the above method further includes:

[0126] Step S21: Obtain the coordinate positions of multiple spatial lines calculated from multiple image frames, wherein the multiple spatial lines include the target spatial line;

[0127] Step S22: Determine at least one of the selected spatial lines that meets the following conditions: for any spatial line, the ratio of the depth calculated based on the coordinate position of the spatial line to the average depth of all spatial lines is within a preset range; for any spatial line, the ratio of the number of image frames containing the line segment corresponding to the spatial line to the total number of image frames is greater than a preset value.

[0128] Step S23: Construct a target map based on at least one filtered spatial line, wherein the target map is a map with spatial straight line features.

[0129] In practical applications, when the method of this embodiment is applied to a SLAM system, to avoid the excessive size of the back-end optimization model in the SLAM system, it is necessary to filter the straight lines in the SLAM map. During the line filtering process, straight lines that can become landmarks need to meet multiple conditions such as parallax, reprojection error, and epipolar constraints.

[0130] In this embodiment, for any spatial line, the ratio of the depth calculated based on the coordinate position of that spatial line to the average depth of all spatial lines falls within a preset range. The average depth of all spatial lines can be pre-calculated, and then for any spatial line, the ratio of the depth calculated based on the coordinate position of that spatial line to the average depth of all spatial lines can be determined. Specifically, the preset range can be set according to the calculation task. The applicant's research has found that a preset range of 0.8 to 1.2 is optimal. For example, for a spatial straight line, determining the depth Z′ of the current spatial straight line... line Is the point's depth between 0.8 and 1.2 times the average depth of the line in the current scene?

[0131]

[0132] Z line Z point n represents the depth of a point or line in space. l n p This represents the number of point and line features in space. Depth determination can remove spatial lines whose depth significantly deviates from the scene average, improving map generation efficiency.

[0133] In this embodiment, the subscripts line or l represent parameters corresponding to a line, and the subscripts point or p represent parameters corresponding to a point. For example, Z line Represents the depth of the centerline in space, Z point n represents the depth of a point in space. l n represents the number of features along the centerline in space. pThis represents the number of point features in space. For any spatial line, if the ratio of image frames containing the line segment corresponding to that spatial line to the total number of image frames is greater than a preset value, then observation of consecutive image frames is possible. It is necessary to determine whether the spatial line is within the field of view of the current frame, i.e., whether the spatial line can be continuously observed. If the line is within the current field of view, the number of observations, n, is counted. visible Simultaneously, count the number of times n that the line can be matched. match Finally, the ratio of observations to matches is calculated. The applicant's research found that a preset value of around 0.25 is optimal, meaning the ratio of image frames containing the line segment corresponding to the spatial line to the total number of image frames should be greater than 0.25. If this ratio is too small, it indicates that the spatial line quality is not high and it needs to be removed during subsequent optimization.

[0134] Optionally, based on at least one filtered spatial line, a target map is constructed, including: determining multiple specified image frames within a multi-frame image set; for each filtered spatial line, determining the number of consecutive frames containing the line segment corresponding to each spatial line within the multiple specified image frames, and assigning a weight to each spatial line based on the determined number of consecutive frames; and constructing the target map based on the weights of each spatial line. Specifically, the weights of each spatial line can be assigned based on the determined number of consecutive frames, with a higher weight assigned to the spatial line for a larger number of consecutive frames. For example, if a spatial line can be observed in three consecutive keyframes, it is considered a high-quality spatial line and can be assigned a higher weight for backend optimization.

[0135] As can be seen, the method of this application embodiment can filter spatial lines by preset conditions, filter out spatial lines whose depth deviates significantly from the scene average and spatial lines of low quality, select higher quality landmarks, and construct a target map based on the selected landmarks. This can not only reduce the impact of poor quality spatial lines, but also reduce the amount of computation in the map construction process and improve computational efficiency.

[0136] Optional, see Figure 3 Step S14, based on the position of the first grid region in the target image frame, identifies the second grid region in the second frame of the two target image frames corresponding to that position, including:

[0137] Step S141: Create the index relationship between the first grid region and the first line segment;

[0138] Step S142: Identify the second grid region in the second frame based on the index relationship.

[0139] In this embodiment, the LSD line detection algorithm can be used to extract line features from each frame of the image, calculate the coordinates of the starting and ending points of the line features, and the LBD descriptor of the line features. By dividing the image into grid regions, lines are assigned to different grids according to the position of the line segments, establishing an index relationship between lines and image grids.

[0140] Since image acquisition devices typically capture images at a high frame rate in practical applications, the motion between two adjacent frames can be assumed to be uniform. Therefore, the pose of the current frame can be estimated using the pose of the previous frame.

[0141]

[0142] In the formula The pose T of the current frame is obtained based on the assumption of uniform motion. k+1,w Calculate the position of the line in the previous frame in the current frame. For example... Figure 4 As shown, the projection of line AB in the previous frame is A1B1. According to the assumption of uniform motion, the projection of the line in the next frame is A2B2.

[0143] In practical applications, the LSD line detection algorithm can be used to divide the line segments in the first and second frames into grids, with the grids in adjacent frames being consistent. The corresponding grids are determined by the coordinates of the start and end points in the first frame, establishing the coordinates of the start and end points of the first line segment in the first frame and the corresponding grid indices. Then, the projections of the start and end points of the first frame into the second frame are calculated, obtaining the indices of each grid and its projection point in the second frame. Finally, based on the coordinate relationship between each grid and its projection point in the second frame, the grids corresponding to the projection points of the start and end points in the first frame are selected, yielding the predicted coordinate range of the second line segment in the second frame. Specifically, the matching relationship between the start and end points of the first frame can be found in the image grid of the second frame, determining the line segment endpoint indices stored in the previous frame's grid, thus limiting the coarse search range in the current frame. Using the distance between LBD descriptors as the matching metric greatly improves matching efficiency.

[0144] Optional, see Figure 5 Step S15 identifies the coordinate features of each line segment within the second grid area, and matches the coordinate features of the first line segment with the coordinate features of each line segment within the second grid area to obtain the coordinate features of the matching second line segment, including:

[0145] Step S151: Identify the coordinate features of each line segment in the second grid area and vectorize them to obtain the feature vector of each line segment in the second grid area.

[0146] Step S152: Calculate the feature vector corresponding to the first line segment based on the coordinate features of the first line segment, and calculate the linear feature descriptor distance between the feature vector corresponding to the first line segment and the feature vectors of each line segment in the second grid area.

[0147] Step S153: Determine the line segment with the smallest corresponding descriptor distance among all line segments in the second grid area as the second line segment.

[0148] In practical use, the LBD descriptor can be calculated by calling the API. Based on the coordinate characteristics of the first line segment, the feature vector corresponding to the first line segment is calculated. Then, the distance between the feature vector corresponding to the first line segment and the linear feature descriptor distance of the feature vectors of each line segment within the second grid area is calculated. Finally, the line segment with the smallest corresponding distance among all line segments within the predicted coordinate range in the second frame is identified as the second line segment, and its coordinate characteristics are obtained. For example... Figure 6 As shown, the endpoint indices of line A1B1 in the previous frame are (4, 2) and (2, 5). Based on these indices, the index values ​​of the corresponding lines are searched in the grid in the next frame. The final matched line is obtained based on the distance of the LBD line segment descriptor. Compared with brute-force matching and FLANN matching of the whole image, under the assumption of uniform motion, the amount of computation is reduced and the matching accuracy is improved.

[0149] Optional, see Figure 7 Step S16 calculates the spatial position of the target spatial line based on the coordinate characteristics of the first line segment and the second line segment, including:

[0150] Step S161: Based on the coordinate characteristics of the first line segment and the second line segment, create a reprojection error function for the target space line using a preset function.

[0151] Step S162: Calculate the spatial position where the error corresponding to the reprojection error function is minimized, and obtain the spatial position of the target spatial line.

[0152] Optionally, based on the coordinate characteristics of the first line segment and the second line segment, a reprojection error function for the target space line is created using a preset function, including:

[0153] Based on the coordinate characteristics of the first and second line segments, the reprojection error function of the target space line is created using a preset robust kernel function as follows:

[0154]

[0155] Where, ρ p ρ l For Huber robust kernel function, Represents the information matrix, e l e represents the projection error of the line feature. pRepresents the projection error of a point.

[0156] During the operation of the SLAM system, point and line features can be observed in the field of view of the image acquisition device at different poses. By constructing constraints through the error functions of points and lines, a precise pose estimate of the current image frame can be obtained, such as... Figure 8 As shown;

[0157] in,

[0158] Among them, e p T represents the projection error of a point, where π is the projection function. cw Let be the pose transformation matrix, and define the projection error of a point as the Euclidean distance between pixels.

[0159]

[0160] Among them, e p T represents the projection error of a point, where π is the projection function. cw Let be the pose transformation matrix, and define the projection error of a point as the Euclidean distance between pixels.

[0161] The method described in this application can be used to optimize the graph through a reprojection function, construct spatial constraints for straight line features, connect the point-line relationships in the SLAM system, continuously and dynamically adjust the spatial position of the straight line, and finally generate a robust and accurate spatial straight line.

[0162] To illustrate the method of the embodiments of this application, see [link to relevant documentation]. Figure 9 The following description is based on specific embodiments:

[0163] 1. LSD line extraction: When a visual frame is input, the LSD line detection algorithm is used to extract the line features in each frame of the image, calculate the coordinates of the starting and ending points of the line features, and the LBD descriptor of the line features.

[0164] 2. Establish grid parameters: Divide the image into grid regions, assign lines to different grids based on the position of line segments, and establish the index relationship between lines and image grids.

[0165] 3. Mesh matching: Due to the high frame rate of camera image acquisition, the motion between two adjacent frames can be assumed to be uniform. Therefore, the pose of the current frame can be estimated using the pose of the previous frame.

[0166]

[0167] In the formula, The pose T of the current frame is obtained based on the assumption of uniform motion. k+1,w Calculate the position of the line in the previous frame in the current frame. For example... Figure 4As shown, the projection of line AB in the previous frame is A1B1. According to the assumption of uniform motion, the projection of the line in the next frame is A2B2.

[0168] The system searches for corresponding matches in the image grid of the current frame based on the index values ​​of the feature endpoints in the previous frame. It also uses the line segment endpoint indices stored in the previous frame's grid to limit the coarse search range in the current frame, and then uses the distance between LBD descriptors as the matching metric, greatly improving matching efficiency. For example... Figure 6 As shown, the endpoint indices of line A1B1 in the previous frame are (4, 2) and (2, 5). Based on these indices, the index values ​​of the corresponding lines are searched in the grid in the next frame. The final matched line is obtained based on the distance of the LBD line segment descriptor. Compared with brute-force matching and FLANN matching of the whole image, under the assumption of uniform motion, the amount of computation is reduced and the matching accuracy is improved.

[0169] 4. Triangulation establishes spatial lines. From multiple pairs of matching lines and their corresponding map points, the optimal pose for the current frame can be calculated. Based on the matching lines in two adjacent frames, the spatial position of the line can be calculated. This is achieved by using the pixel positions X1(u1,v1,1) and X2(u2,v2,1) of the line's two endpoints in different image frames. Based on the known pose, the depths s1 and s2 of the line can be recovered, thus determining the spatial position of the line.

[0170] s2X2=s1RX1+t

[0171] By combining the coordinates of feature points, line endpoints, and initial pose values, the RANSAC algorithm is used to solve for the optimal pose estimate and line depth value. Thus, the line feature is transformed from a two-dimensional plane into a spatial line with scale, which can serve as a landmark in a map, thereby constraining the construction of the SLAM system.

[0172] During the operation of a SLAM system, point and line features can be observed in the camera's field of view at different poses. By constructing constraints using the error functions of points and lines, a precise pose estimate of the current image frame can be obtained, such as... Figure 8 As shown.

[0173] The error of point features is defined as the Euclidean distance between the pixels projected from the map point to the current frame, which is used as the error term.

[0174]

[0175] Where e p T represents the projection error of a point, where π is the projection function. cw Let be the pose transformation matrix, and define the projection error of a point as the Euclidean distance between pixels.

[0176] The error for line features is defined as the distance from the endpoint of the line to the matching line in the current image frame, which is a non-linear error representation using geometric distance definition:

[0177]

[0178] Where e l X represents the projection error of the line feature. s and X e L′(l1,l2,l3) represents the endpoint of the line feature projected onto the current frame, and L′(l1,l2,l3) represents the line equation, while l1,l2,l3 represent the line equation parameters.

[0179] Based on the above definition of error for point and line features, and assuming that the observation error follows a Gaussian distribution, the reprojection error function for all point and line features is constructed as follows:

[0180]

[0181] Where ρ p ρ l The Huber robust kernel function reduces the interference of the error term in the error function on the optimization direction; The information matrix is ​​represented by the observation covariance. A graph optimization problem is constructed using a reprojection function, spatial constraints on the straight-line features are established, the point-line relationships in the SLAM system are connected, and the spatial positions of the straight lines are continuously and dynamically adjusted to ultimately generate robust and accurate spatial straight lines.

[0182] 5. Spatial Constraint Optimization: Adjusting the Position of Straight Lines. Real-world environments are rich in geometric information such as points and lines. To avoid an excessively large back-end optimization model in the SLAM system, straight lines in the SLAM map need to be filtered. During this filtering process, straight lines that can become landmarks must meet various conditions, including parallax, reprojection error, and epipolar constraints.

[0183] For a straight line in space, the first step is to determine the average depth in the scene, i.e., the depth Z′ of the current straight line in space. line The depth of the point must be between 0.8 and 1.2 times the average depth of the line in the current scene, that is:

[0184]

[0185] Z line Z point n represents the depth of a point-line in space. l n p This represents the number of point and line features in space. Based on the above formula, straight lines in space are filtered to remove those whose depth deviates significantly from the scene average.

[0186] For observations within consecutive frames, it is necessary to determine whether the line is within the field of view of the current frame, i.e., whether the line can be observed continuously. If the line is within the current field of view, the number of observations, n, is counted. visible Simultaneously, count the number of times n that the line can be matched. match Finally, the ratio of observations to matches is calculated. This value should be greater than 0.25. If the ratio is too small, it indicates that the quality of the linear features is not high and they need to be removed during subsequent optimization.

[0187] If a spatial line can be observed in three consecutive keyframes, it is considered to be a high-quality spatial line and can be given higher weight for backend optimization.

[0188] A second aspect of this application provides a mobile robot, see [link to relevant documentation]. Figure 10 It includes an image acquisition module 1001 and a processor 1002:

[0189] The image acquisition module 1001 is used to acquire at least two image frames, wherein the at least two image frames include two target image frames acquired by the image acquisition device during the movement of the device from the same target spatial line.

[0190] The processor 1002 is configured to extract coordinate features from a first line segment corresponding to a target spatial line in the first frame of two target image frames; identify a first grid region matching the first line segment based on its coordinate features, wherein each target image frame includes multiple grid regions, and the grid regions of each target image frame correspond one-to-one; identify a second grid region in the second frame of the two target image frames corresponding to the position of the first grid region in the target image frame; identify the coordinate features of each line segment in the second grid region, and match the coordinate features of the first line segment with the coordinate features of each line segment in the second grid region to obtain the coordinate features of the matching second line segment; and calculate the spatial position of the target spatial line based on the coordinate features of the first and second line segments.

[0191] Optionally, the processor 1002 is further configured to acquire the coordinate positions of multiple spatial lines calculated from multiple image frames, wherein the multiple spatial lines include a target spatial line; determine at least one filtered spatial line among the multiple spatial lines that satisfies the following conditions: for any spatial line, the ratio of the depth calculated based on the coordinate position of the spatial line to the average depth of all spatial lines is within a preset range; for any spatial line, the ratio of the number of image frames containing the line segment corresponding to the spatial line to the total number of image frames is greater than a preset value; and construct a target map based on at least one filtered spatial line, wherein the target map is a map with spatial straight line features.

[0192] Optionally, the processor 1002 is specifically used to determine multiple specified image frames in multiple image frames; for each filtered spatial line, determine the number of consecutive frames containing the line segments corresponding to each spatial line in the multiple specified image frames, and set the weight of each spatial line according to the determined number of consecutive frames; and construct a target map according to the weight of each spatial line.

[0193] Optionally, the processor 1002 is specifically used to create an index relationship between the first grid region and the first line segment; and to identify the second grid region in the second frame based on the index relationship.

[0194] Optionally, the processor 1002 is specifically used to identify the coordinate features of each line segment in the second grid area and vectorize them to obtain the feature vector of each line segment in the second grid area; calculate the feature vector corresponding to the first line segment based on the coordinate features of the first line segment, and calculate the straight line feature descriptor distance between the feature vector corresponding to the first line segment and the feature vector of each line segment in the second grid area; and determine the line segment with the smallest corresponding descriptor distance among the line segments in the second grid area as the second line segment.

[0195] Optionally, the processor 1002 is specifically used to create a reprojection error function of the target spatial line based on the coordinate characteristics of the first line segment and the coordinate characteristics of the second line segment through a preset function; calculate the spatial position when the error corresponding to the reprojection error function is minimized, and obtain the spatial position of the target spatial line.

[0196] Optionally, processor 1002 is specifically used to create a reprojection error function for the target space line based on the coordinate characteristics of the first line segment and the coordinate characteristics of the second line segment using a preset robust kernel function.

[0197]

[0198] Where, ρ p ρ l For Huber robust kernel function, Represents the information matrix, e l e represents the projection error of the line feature. p Represents the projection error of a point.

[0199] As can be seen, the mobile robot of this application embodiment can identify the second grid region in the second frame of the two target image frames corresponding to the first grid region at the target image frame position, and then match the coordinate features of the first line segment with the coordinate features of each line segment in the second grid region to obtain the coordinate features of the matching second line segment. It is not necessary to match the first line segment with all line segments in the second frame, thereby reducing the amount of computation in the matching process, improving the matching efficiency, and solving the problem of low line segment matching efficiency when determining the position of spatial lines in the prior art.

[0200] A second aspect of the embodiments of this application, see [link to previous document]. Figure 11 A spatial line position calculation device is provided, comprising:

[0201] The image frame acquisition module 1101 is used to acquire at least two image frames, wherein the at least two image frames include two target image frames acquired by the image acquisition device during the movement of the image acquisition device on the same target spatial line.

[0202] The feature extraction module 1102 is used to extract coordinate features from the first line segment corresponding to the target space line in the first frame of the two target image frames.

[0203] The range calculation module 1103 is used to identify the first grid region that matches the first line segment based on the coordinate characteristics of the first line segment, wherein each target image frame includes multiple grid regions, and the grid regions of each target image frame correspond one-to-one.

[0204] The line segment matching module 1104 is used to identify the second grid region in the second frame of the two target image frames corresponding to the position of the first grid region in the target image frame;

[0205] The coordinate acquisition module 1105 is used to identify the coordinate features of each line segment in the second grid area, and match the coordinate features of the first line segment with the coordinate features of each line segment in the second grid area to obtain the coordinate features of the matching second line segment.

[0206] The position calculation module 1106 is used to calculate the spatial position of the target spatial line based on the coordinate characteristics of the first line segment and the coordinate characteristics of the second line segment.

[0207] Optionally, the above-mentioned device further includes:

[0208] The coordinate position acquisition submodule is used to acquire the coordinate positions of multiple spatial lines calculated from multiple image frames, including the target spatial line.

[0209] The spatial line filtering submodule is used to determine at least one filtered spatial line from multiple spatial lines that meets the following conditions:

[0210] For any given spatial line, the ratio of the depth calculated based on the coordinate position of that spatial line to the average depth of all spatial lines is within a preset range;

[0211] For any given spatial line, the ratio of the number of image frames containing the line segment corresponding to that spatial line to the total number of image frames is greater than a preset value;

[0212] The map building submodule is used to build a target map based on at least one filtered spatial line, wherein the target map is a map with spatial straight line features.

[0213] Optional, the map building submodule includes:

[0214] The specified frame determination unit is used to determine multiple specified image frames in a multi-frame image frame set.

[0215] The continuous frame determination unit is used to determine the number of continuous frames containing the line segments corresponding to each spatial line in multiple specified image frames for each filtered spatial line, and to set the weight of each spatial line according to the determined number of continuous frames.

[0216] Map building unit, used to construct a target map based on the weights of each spatial line.

[0217] Optionally, the line segment matching module 1104 includes:

[0218] The index creation submodule is used to create the index relationship between the first grid region and the first line segment;

[0219] The grid recognition submodule is used to identify the second grid region in the second frame based on the index relationship.

[0220] Optionally, the coordinate acquisition module 1105 includes:

[0221] The coordinate quantization submodule is used to identify the coordinate features of each line segment in the second grid area and vectorize them to obtain the feature vector of each line segment in the second grid area.

[0222] The distance calculation submodule is used to calculate the feature vector corresponding to the first line segment based on the coordinate characteristics of the first line segment, and to calculate the feature vector corresponding to the first line segment and the feature vectors of each line segment in the second grid area, as well as the straight line feature descriptor distance.

[0223] The coordinate feature acquisition submodule is used to identify the line segment with the smallest distance to the corresponding descriptor among all line segments within the second grid area as the second line segment.

[0224] Optionally, the location calculation module 1106 includes:

[0225] The function creation submodule is used to create a reprojection error function for the target space line based on the coordinate characteristics of the first line segment and the second line segment using a preset function.

[0226] The function solver submodule is used to calculate the spatial position where the error corresponding to the reprojection error function is minimized, thereby obtaining the spatial position of the target spatial line.

[0227] Optionally, a function creation submodule is used to create a reprojection error function for the target space line based on the coordinate characteristics of the first and second line segments using a preset robust kernel function.

[0228]

[0229] Where, ρ p ρ l For Huber robust kernel function, Represents the information matrix, e l e represents the projection error of the line feature. p Represents the projection error of a point.

[0230] As can be seen, the apparatus of this application embodiment can identify the second grid region in the second frame of the two target image frames corresponding to the first grid region at the target image frame position, and then match the coordinate features of the first line segment with the coordinate features of each line segment in the second grid region to obtain the coordinate features of the matching second line segment. It is not necessary to match the first line segment with all line segments in the second frame, thereby reducing the computational load of the matching process, improving the matching efficiency, and solving the problem of low line segment matching efficiency when determining the position of spatial lines in the prior art.

[0231] This application also provides an electronic device, such as... Figure 12 As shown, it includes a processor 1201, a communication interface 1202, a memory 1203, and a communication bus 1204. The processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through the communication bus 1204.

[0232] Memory 1203 is used to store computer programs;

[0233] When processor 1201 executes the program stored in memory 1203, it performs the following steps:

[0234] Acquire at least two image frames, wherein the at least two image frames include two target image frames acquired by the image acquisition device during the movement of the device from the same target spatial line;

[0235] Extract coordinate features from the first line segment corresponding to the target space line in the first frame of the two target image frames;

[0236] Based on the coordinate characteristics of the first line segment, the first grid region that matches the first line segment is identified. Each target image frame includes multiple grid regions, and the grid regions of each target image frame correspond one-to-one.

[0237] Based on the position of the first grid region in the target image frame, identify the second grid region in the second frame of the two target image frames corresponding to that position;

[0238] Identify the coordinate features of each line segment within the second grid area, and match the coordinate features of the first line segment with the coordinate features of each line segment within the second grid area to obtain the coordinate features of the matching second line segment;

[0239] Calculate the spatial position of the target spatial line based on the coordinate characteristics of the first line segment and the second line segment.

[0240] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0241] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0242] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0243] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0244] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described spatial line position calculation methods.

[0245] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the spatial line position calculation methods described above.

[0246] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0247] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0248] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for mobile robots, devices, electronic devices, storage media, and computer program products are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0249] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method of spatial line position calculation, characterized by, The method comprises: acquiring at least two image frames, wherein the at least two image frames comprise two target image frames obtained by an image acquisition device during movement of a same target space line; performing coordinate feature extraction on a first line segment corresponding to a target space line in a first target image frame of the two target image frames; identifying a first grid region matching the first line segment according to the coordinate feature of the first line segment, wherein each target image frame comprises a plurality of grid regions, and the grid regions of each target image frame correspond one by one, and the range of the grid region is smaller than the range of the target image frame; creating an index relationship between the first grid region and the first line segment, wherein the index relationship represents the coordinates of the starting point and the ending point of the first line segment, and the index relationship between the grid regions corresponding to the starting point and the ending point of the first line segment; identifying a second grid region in a second target image frame according to the index relationship; identifying the coordinate features of each line segment in the second grid region, and matching the coordinate features of the first line segment with the coordinate features of each line segment in the second grid region to obtain the coordinate features of a second line segment that matches; calculating the spatial position of the target space line according to the coordinate features of the first line segment and the coordinate features of the second line segment; acquiring the coordinate positions of a plurality of space lines calculated through a plurality of image frames, wherein the plurality of space lines comprise the target space line; determining at least one filtered space line from the plurality of space lines that satisfies the following conditions: for any space line, the ratio of the depth of the space line to the average depth of all space lines calculated according to the coordinate position of the space line is within a preset range; for any space line, the ratio of the image frames containing the line segment corresponding to the space line to the total image frames in the plurality of image frames is greater than a preset value; constructing a target map according to the at least one filtered space line, wherein the target map is a map with spatial straight line features.

2. The method of claim 1, wherein, The constructing a target map according to the at least one filtered space line comprises: determining a plurality of specified image frames from the plurality of image frames; for each filtered space line, determining the number of continuous frames containing the line segment corresponding to each space line from the plurality of specified image frames, and setting the weight of each space line according to the determined number of continuous frames; constructing a target map according to the weights of each space line.

3. The method of claim 1, wherein, The identifying the coordinate features of each line segment in the second grid region, and matching the coordinate features of the first line segment with the coordinate features of each line segment in the second grid region to obtain the coordinate features of a second line segment that matches comprises: identifying the coordinate features of each line segment in the second grid region and vectorizing to obtain the feature vectors of each line segment in the second grid region; calculating the feature vector corresponding to the first line segment according to the coordinate features of the first line segment, and calculating the straight line feature descriptor distance between the feature vector corresponding to the first line segment and the feature vectors of each line segment in the second grid region; determining the second line segment as the line segment in the second grid region corresponding to the smallest descriptor distance.

4. The method of claim 1, wherein, The space position of the target space line is calculated according to the coordinate features of the first line segment and the coordinate features of the second line segment, and the method comprises the following steps of: According to the coordinate features of the first line segment and the coordinate features of the second line segment, a re-projection error function of the target space line is created by a preset function; The space position of the target space line is calculated according to the coordinate features of the first line segment and the coordinate features of the second line segment, and the method comprises the following steps of:

5. The method of claim 4, wherein, According to the coordinate features of the first line segment and the coordinate features of the second line segment, a re-projection error function of the target space line is created by a preset function; According to the coordinate features of the first line segment and the coordinate features of the second line segment, a re-projection error function of the target space line is created by a preset function; where ρ p , ρ l are Huber robust kernel functions, is the information matrix, e l is the projection error of the line feature, e p is the projection error of the point.

6. A mobile robot, characterized in that The image acquisition module and the processor are comprised: The image acquisition module is used for acquiring at least two image frames, wherein the at least two image frames comprise two target image frames acquired by the image acquisition device during movement of the image acquisition device; The processor is used for extracting the coordinate features of a first line segment corresponding to a target space line in a first frame of the two target image frames; identifying a first grid region matched with the first line segment according to the coordinate features of the first line segment, wherein each target image frame comprises a plurality of grid regions, and the grid regions of each target image frame are in one-to-one correspondence, and the range of the grid region is smaller than the range of the target image frame; identifying a second grid region in a second frame of the two target image frames according to the position of the first grid region in the target image frame; identifying the coordinate features of each line segment in the second grid region, and matching the coordinate features of the first line segment with the coordinate features of each line segment in the second grid region to obtain the coordinate features of a second line segment matched with the first line segment; and calculating the space position of the target space line according to the coordinate features of the first line segment and the coordinate features of the second line segment. The processor is further used for acquiring the coordinate positions of a plurality of space lines calculated by a plurality of image frames, wherein the plurality of space lines comprise the target space line; determining at least one filtered space line in the plurality of space lines which satisfies the following conditions: for any space line, the ratio of the depth calculated according to the coordinate position of the space line to the average value of the depths of all space lines is located in a preset range; for any space line, the ratio of the image frames containing the line segment corresponding to the space line to the total image frames in the plurality of image frames is greater than a preset value; and constructing a target map according to the at least one filtered space line, wherein the target map is a map with space straight line features. The processor is specifically used for creating an index relationship between the first grid region and the first line segment; identifying the second grid region in the second frame according to the index relationship; and the index relationship represents the coordinates of the starting point and the ending point of the first line segment and the index relationship between the grid regions corresponding to the starting point and the ending point of the first line segment; and the index relationship.

7. The mobile robot of claim 6, wherein, The processor is specifically configured to determine a plurality of designated image frames in the plurality of image frames; determine, for each filtered spatial line, a number of continuous frames in which the plurality of designated image frames contain a line segment corresponding to each spatial line, and set a weight of each spatial line according to the determined number of continuous frames; and construct a target map according to the weight of each spatial line.

8. The mobile robot of claim 6, wherein, The processor is specifically configured to identify coordinate features of each line segment in the second grid region and vectorize to obtain feature vectors of each line segment in the second grid region; calculate a feature vector corresponding to the first line segment according to the coordinate features of the first line segment, and calculate a straight line feature descriptor distance between the feature vector corresponding to the first line segment and the feature vectors of each line segment in the second grid region; and determine the second line segment as the line segment in the second grid region corresponding to the smallest descriptor distance.

9. The mobile robot of claim 6, wherein, The processor is specifically configured to create a re-projection error function of the target spatial line according to the coordinate features of the first line segment and the coordinate features of the second line segment through a preset function; and calculate a spatial position corresponding to a minimum error of the re-projection error function to obtain the spatial position of the target spatial line.

10. The mobile robot of claim 9, wherein, The processor is specifically configured to create a re-projection error function of the target spatial line according to the coordinate features of the first line segment and the coordinate features of the second line segment through a preset robust kernel function: where ρ p , ρ l is the Huber robust kernel function, is the information matrix, e l is the projection error of the line feature, e p is the projection error of the point.

11. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps of any one of claims 1-5. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps of any one of claims 1-5. ​ 12. A computer-readable storage medium, characterized in that, ​

Images