A visual SLAM method, device, robot and storage medium
By acquiring and matching feature lines in the current image and adjacent images in the visual SLAM method, determining the positioning posture and building a map, the problem of inaccurate positioning of traditional visual SLAM in artificial building environments is solved, and the accuracy of visual SLAM is improved and robustness.
Patent Information
- Application Number
- CN202011352975.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-26
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-11-26
AI Technical Summary
Traditional visual SLAM methods are inaccurate in detection and tracking of feature points in artificial built environments, resulting in low positioning accuracy.
By obtaining the feature lines of preset type objects in the current image and adjacent images, matching processing is performed to determine the current position, and a map is constructed based on the position to improve positioning accuracy.
It improves the accuracy and robustness of visual SLAM map building positioning, and solves the problems of low matching success or poor accuracy caused by weak texture and insufficient number of feature points.
Smart Images

Figure CN112381939B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and in particular relates to a visual SLAM method, device, robot and storage medium. Background Art
[0002] At present, Simultaneous Localization And Mapping (SLAM) is a hot topic in the research of autonomous navigation of robots. It is mainly used to perceive the surrounding environment after the robot enters an unknown environment, and then build an incremental map and locate itself at the same time. Generally, traditional visual SLAM methods mainly use feature points in the natural environment for mapping and positioning because feature points in the natural environment are easy to detect and track. However, because it only considers the feature point information in the natural environment, the visual SLAM method will make inaccurate judgments on some man-made building environments such as walls, which will seriously affect the accuracy of visual SLAM and cause low accuracy of visual SLAM mapping and positioning. Summary of the invention
[0003] The embodiments of the present application provide a visual SLAM method, device, robot and storage medium, which can solve the problem of low accuracy of visual SLAM mapping and positioning due to weak texture of visual SLAM, such as low matching success or poor accuracy caused by insufficient number of feature points in the surrounding environment.
[0004] In a first aspect, an embodiment of the present application provides a visual SLAM method, comprising:
[0005] Acquire a current image, and determine characteristic lines of a preset type of target from the current image;
[0006] Acquire adjacent images, determine the historical characteristic lines of the preset type of target from the adjacent images; match the characteristic lines of the preset type of target with the historical characteristic lines;
[0007] When the matching is successful, the current posture is determined according to the characteristic lines of the preset type of target and the historical characteristic lines, and a map is constructed according to the current posture.
[0008] Optionally, determining a characteristic line of a preset type of target from the current image includes:
[0009] Extracting each line segment in the image from the current image;
[0010] Determine a target object where at least two line segments intersect at a point according to the connection status of each line segment;
[0011] When the target object meets the preset type, the line segment corresponding to the preset type target object is used as the characteristic line of the preset type target object.
[0012] Optionally, matching the characteristic line of the preset type of target with the historical characteristic line includes:
[0013] Determine the characteristic value of the historical characteristic line and the characteristic value of the characteristic line of the preset type target;
[0014] Calculating a first error value between a characteristic value of the historical characteristic line and a characteristic value of a characteristic line of the preset type target;
[0015] When the first error value is smaller than a preset first threshold, the matching is successful.
[0016] Optionally, the matching process of the characteristic line of the preset type of target with the historical characteristic line further includes:
[0017] Determining characteristic points of the preset type of target according to the characteristic lines of the preset type of target;
[0018] Determining historical feature points of the preset type of target according to the historical feature line;
[0019] Calculating a second error value between the position of the historical feature point and the position of the feature point of the preset type target;
[0020] When the second error value is smaller than a preset second threshold, the matching is successful.
[0021] Optionally, determining the characteristic points of the preset type of object according to the characteristic lines of the preset type of object includes:
[0022] Determining the intersection points where the characteristic lines of the preset type of target intersect each other;
[0023] The intersection point is used as a feature point of the preset type of target.
[0024] Optionally, determining the current posture according to the characteristic line of the preset type of target and the historical characteristic line includes:
[0025] Acquire a set of sampling points at the same position of the characteristic line of the preset type of target and the historical characteristic line;
[0026] Calculate the sampling point set, the historical feature points and the feature points of the preset type of target according to a preset algorithm, and determine a rotation matrix and a translation matrix between the adjacent image and the current image;
[0027] The current position and posture is determined according to the rotation matrix and the translation matrix.
[0028] Optionally, the determining the current posture according to the characteristic line of the preset type of target and the historical characteristic line further includes:
[0029] Obtain at least three pairs of sampling points at the same position between the characteristic line of the preset type of target and the historical characteristic line;
[0030] Calculate the at least three pairs of sampling points according to a preset algorithm to determine a rotation matrix and a translation matrix between the adjacent image and the current image;
[0031] The current position and posture is determined according to the rotation matrix and the translation matrix.
[0032] In a second aspect, an embodiment of the present application provides a visual SLAM device, comprising:
[0033] An acquisition module, used to acquire a current image and determine a characteristic line of a preset type of target from the current image;
[0034] A matching module, used to obtain adjacent images, determine the historical characteristic line of the preset type of target from the adjacent images; and match the characteristic line of the preset type of target with the historical characteristic line;
[0035] The construction module is used to determine the current posture according to the feature line when the matching is successful, and to construct a map according to the current posture.
[0036] In a third aspect, an embodiment of the present application provides a robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned visual SLAM methods when executing the computer program.
[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned visual SLAM methods are implemented.
[0038] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a robot, the robot executes any one of the visual SLAM methods in the first aspect.
[0039] In the embodiment of the present application, a current image is obtained, and a characteristic line of a preset type of target is determined from the current image, and a characteristic line for comparison in the current environment of the robot is determined; an adjacent image is obtained, and a historical characteristic line of the preset type of target is determined from the adjacent image; the characteristic line of the preset type of target is matched with the historical characteristic line, so as to determine whether the characteristic line currently obtained by the robot is of the same preset type of target as the historical characteristic line. When the match is successful, the current posture is determined according to the characteristic line of the preset type of target and the historical characteristic line, so as to further more accurately derive the current posture of the robot according to the current environment, and construct a map according to the current posture, thereby improving the accuracy and robustness of visual SLAM mapping and positioning. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0041] Figure 1 This is a first flow chart of the visual SLAM method provided in the embodiment of the present application;
[0042] Figure 2 is a schematic diagram of a target object of the visual SLAM method provided in an embodiment of the present application;
[0043] Figure 3 This is a second flow chart of the visual SLAM method provided in the embodiment of the present application;
[0044] Figure 4 Schematic diagram of the third process of the visual SLAM method provided in the embodiment of the present application;
[0045] Figure 5 Schematic diagram of the structure of the visual SLAM device provided in the embodiment of the present application;
[0046] Figure 6 It is a schematic diagram of the structure of the robot provided in the embodiment of the present application. DETAILED DESCRIPTION
[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0048] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0049] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0050] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0051] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0052] Figure 1 FIG. 1 is a flow chart of a visual SLAM method in an embodiment of the present application. The execution subject of the method may be a robot, such as Figure 1 As shown, the visual SLAM method may include the following steps:
[0053] Step S101: Acquire a current image, and determine characteristic lines of a preset type of target from the current image.
[0054] In this embodiment, if visual SLAM only considers the feature point information in the natural environment, the accuracy of visual SLAM mapping and positioning will be low. Therefore, the current image can be collected by using the camera installed on the robot. The robot determines the feature lines of the preset type of target from the current image, so that the robot can compare the features of the special type of target in the current environment, thereby accurately determining its current posture. Figure 2 As shown, if the current target object is a corner, the three line segments forming the corner are the characteristic lines of the corner, and the intersection between the characteristic lines can be the characteristic point of the corner. Among them, the above-mentioned preset type of target object can be a right-angle type target object, that is, the target object has a right-angle feature.
[0055] Optionally, the camera can be installed at the front of the robot at a certain inclination angle. At least one camera can be installed on the robot, and the camera includes at least one of a monocular camera and a binocular camera for sensing the surrounding environment. Although there are many types of sensors for sensing the surrounding environment, the camera equipment has the advantages of being cheap, small in size, and easy to install, which can make the visual SLAM method an important research topic in the field.
[0056] Optionally, step S101 includes:
[0057] Each line segment in the image is extracted from the current image.
[0058] A target object where at least two line segments intersect at a point is determined according to the connection status of each line segment.
[0059] When the target object meets the preset type, the line segment corresponding to the preset type target object is used as the characteristic line of the preset type target object.
[0060] In this embodiment, the robot extracts each line segment in the image from the acquired image of the current environment, and can determine a target object formed by at least two line segments intersecting at a point based on the connection status of each line segment, and judge the type of the target object. When the type of the target object meets the preset type, the at least two line segments of the target object obtained by the above method are used as the characteristic lines in this embodiment.
[0061] Step S102: Acquire adjacent images, determine the historical characteristic lines of the preset type of target from the adjacent images; and match the characteristic lines of the preset type of target with the historical characteristic lines.
[0062] In this embodiment, after the robot obtains the current image, it further obtains the adjacent image at the adjacent moment, that is, the previous frame image. To facilitate the comparison of the two frames of images, the historical feature lines of the preset type of target object can be determined from the adjacent images, so as to facilitate the matching of the historical feature lines with the feature lines of the preset type of target object obtained in the current image, so as to determine whether the current image obtained by the current robot and the target object in the adjacent image can match, and facilitate the robot to further accurately determine its current posture according to the target objects in the two frames of images.
[0063] Optionally, after acquiring the adjacent images, tracking processing may be performed based on the feature values of the feature lines of the preset type of target determined from the current image, so as to determine the historical feature lines in the adjacent images.
[0064] Optional, such as Figure 3 As shown, step S102 includes:
[0065] Step S301: Determine the characteristic value of the historical characteristic line and the characteristic value of the characteristic line of the preset type of target.
[0066] Step S302: Calculate a first error value between a characteristic value of the historical characteristic line and a characteristic value of a characteristic line of the preset type of target.
[0067] Step S303: When the first error value is less than a preset first threshold, the matching is successful.
[0068] In this embodiment, the robot can select a preset number of sampling points from the historical feature line of the preset type of target in the adjacent image, determine their position coordinates, and calculate their mean value according to the position coordinates to obtain the feature value of the historical feature line; and select a preset number of sampling points from the same position of the feature line of the preset type of target in the current image as the sampling points in the selected historical feature line, and determine their position coordinates, and calculate their mean value according to the position coordinates to obtain the feature value of the feature line of the preset type of target, and calculate the difference between the two to obtain a first error value, and compare them. When the first error value is less than the first threshold preset inside the robot, it means that the two are matched successfully. Among them, the above-mentioned first threshold can be set according to the angular resolution, frame rate and movement speed of the camera, which is not limited here. For example, if the current environment is 30 frames per second, 1080p, 120 degrees field of view and the robot moves at a speed of 30 cm / s, the first threshold can be set to 3.6 pixels.
[0069] Optional, such as Figure 4 As shown, step S102 also includes:
[0070] Step S401: determining feature points of the preset type of target according to the feature lines of the preset type of target.
[0071] Optionally, step S401 includes:
[0072] Determine the intersection point between the characteristic lines of the preset type of target; and use the intersection point as the characteristic point of the preset type of target.
[0073] Optionally, points at preset positions around the feature line may be used as feature points of a preset type of target.
[0074] Step S402: determining historical feature points of the preset type of target according to the historical feature line.
[0075] Step S403: Calculate a second error value between the position of the historical feature point and the position of the feature point of the preset type of target.
[0076] Step S404: When the second error value is less than a preset second threshold, the matching is successful.
[0077] In this embodiment, the robot can determine the historical feature points of the preset type of target from the historical feature lines through the above-mentioned feature point selection means, and determine the feature points of the preset type of target from the feature lines of the preset type of target in the current image, and obtain the second error value by calculating the difference between the positions of the two feature points, and compare them. When the second error value is less than the second threshold preset inside the robot, it indicates that the two are matched successfully. Among them, the above-mentioned second threshold can be set according to the angular resolution, frame rate and movement speed of the camera, which is not limited here. For example, if the current environment is 30 frames per second, 1080p, 120 degrees field of view and the robot moves at a speed of 30cm / s, the second threshold can be set to 9 pixels; the above-mentioned feature points are as follows: Figure 2 As shown, it can be the intersection point of the characteristic lines.
[0078] Optionally, when there are at least two preset types of targets, the relative position relationship of the at least two preset types of targets can be determined, and auxiliary registration can be performed during the matching process according to the relative position relationship of the at least two preset types of targets, so as to improve the matching speed. The relative position relationship of the at least two preset types of targets can be determined by the relative position of the intersection of the characteristic lines of the at least two preset types of targets.
[0079] Optionally, the feature points may be selected by randomly setting at least two points on a fixed object in the robot operation scene as feature points, so that the robot can perform matching processing according to the feature points appearing in the image.
[0080] Step S103: When the matching is successful, the current posture is determined according to the characteristic lines of the preset type of target and the historical characteristic lines, and a map is constructed according to the current posture.
[0081] In this embodiment, when the match is successful, it means that the preset type of target object in the current image of the robot is the same as the preset type of target object in the adjacent image, and because it meets the above-mentioned threshold conditions, the current posture of the robot can be calculated in combination. Therefore, the robot can calculate based on the characteristic lines of the preset type of target object in the current image and the historical characteristic lines in the adjacent image to obtain the movement mode of the robot from the moment when the adjacent image is obtained to the moment when the current image is obtained, and thus the posture of the robot at the current moment can be obtained, and then a map can be constructed based on the current posture.
[0082] Optionally, step S103 includes:
[0083] A set of sampling points at the same position between the characteristic line of the preset type of target and the historical characteristic line is obtained.
[0084] The sampling point set, the historical feature points and the feature points of the preset type of target are calculated according to a preset algorithm to determine a rotation matrix and a translation matrix between the adjacent image and the current image.
[0085] The current position and posture is determined according to the rotation matrix and the translation matrix.
[0086] In this embodiment, the robot can determine its current posture according to the combination of feature lines and feature points in a preset type of target. The robot can obtain sampling points at the same position of the feature lines of the preset type of target and the historical feature lines, as well as the historical feature points obtained above and the feature points of the preset type of target, and calculate the above-obtained data points using a preset algorithm to determine the rotation matrix and translation matrix between the adjacent image and the current image. The movement mode of the robot from the moment the adjacent image is obtained to the moment the current image is obtained can be obtained based on the rotation matrix and the translation matrix, and the posture of the previous adjacent image can be used to obtain the posture at the current moment.
[0087] Optionally, a set of sampling points at positions close to the characteristic lines of the preset type of target and the historical characteristic lines may be selected for calculation, wherein the sampling points at similar positions may be sampling points whose position difference between the characteristic lines of the preset type of target and the historical characteristic lines is less than or equal to a preset position threshold.
[0088] It can be understood that the sampling point set obtained by the feature line can be obtained by calculating the mean of the points in the point set to obtain the points representing the feature line of the preset type of target and the points representing the historical feature line. For example, point A, point B, and point C are selected from the feature line of the preset type of target. After calculation, the point representing the feature line is (A+B+C) / 2; similarly, three points at corresponding positions are selected from the historical feature line, and the above calculation is also performed to obtain the points representing the historical feature line.
[0089] As a specific example but not limitation, if the current image contains feature points P1, P2, P3 and feature line L preset in the real scene; then the robot will obtain P1', P2', P3' and point L' representing the feature line under the current image A by acquiring the current image A and the adjacent image B; and P1", P2", P3" and point L" representing the feature line under the adjacent image B; wherein the above-mentioned points representing the feature line L can be obtained by selecting a preset number of sampling points on the feature line and averaging to obtain the points representing the feature line. The above-mentioned data are calculated according to the preset algorithm, and the formula can be obtained:
[0090] P′1=K*P1; P″1=K*(R*P1+T);
[0091] P′2=K*P2; P″2=K*(R*P2+T);
[0092] P′3=K*P3; P″3=K*(R*P3+T);
[0093] L′=K*L; L″=K*(R*L+T);
[0094] Among them, the above K is the camera internal parameter matrix, which is preset in advance inside the robot; according to the above formula, the robot's motion mode (R, T) can be derived, the above R is the rotation matrix, and the above T is the translation matrix. According to the rotation matrix and the translation matrix, the robot's motion mode in the process of moving from the moment when the adjacent image is acquired to the moment when the current image is acquired can be obtained, and the current moment's posture can be obtained according to the posture of the previous adjacent image through the rotation matrix and the translation matrix.
[0095] Optionally, step S103 further includes:
[0096] At least three pairs of sampling points at the same position between the characteristic line of the preset type of target and the historical characteristic line are obtained.
[0097] The at least three pairs of sampling points are calculated according to a preset algorithm to determine a rotation matrix and a translation matrix between the adjacent image and the current image.
[0098] The current position and posture are determined according to the rotation matrix and the translation matrix.
[0099] In this embodiment, the robot can also determine its current posture based on the feature line. The robot can obtain at least three pairs of sampling points at the same position as the feature line of the preset type of target and the historical feature line, and calculate the at least three pairs of sampling points obtained by using an algorithm consistent with the above, thereby determining the rotation matrix and translation matrix between the adjacent image and the current image. The movement mode of the robot from the moment when the adjacent image is obtained to the moment when the current image is obtained can be obtained based on the rotation matrix and the translation matrix, and the posture at the current moment can be obtained based on the posture of the previous adjacent image through the rotation matrix and the translation matrix.
[0100] In the embodiment of the present application, a current image is obtained, and a characteristic line of a preset type of target is determined from the current image, and a characteristic line for comparison in the current environment of the robot is determined; an adjacent image is obtained, and a historical characteristic line of the preset type of target is determined from the adjacent image; the characteristic line of the preset type of target is matched with the historical characteristic line, so as to determine whether the characteristic line currently obtained by the robot is of the same preset type of target as the historical characteristic line. When the match is successful, the current posture is determined according to the characteristic line of the preset type of target and the historical characteristic line, so as to further more accurately derive the current posture of the robot according to the current environment, and construct a map according to the current posture, thereby improving the accuracy and robustness of visual SLAM mapping and positioning.
[0101] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0102] Corresponding to the visual SLAM method described above, Figure 5 FIG. 1 is a schematic diagram of the structure of a visual SLAM device in an embodiment of the present application. Figure 5 As shown, the visual SLAM device may include:
[0103] The acquisition module 501 is used to acquire a current image and determine a characteristic line of a preset type of target from the current image.
[0104] The matching module 502 is used to obtain adjacent images, determine the historical characteristic lines of the preset type of target from the adjacent images, and match the characteristic lines of the preset type of target with the historical characteristic lines.
[0105] The construction module 503 is used to determine the current posture according to the feature line when the matching is successful, and to construct a map according to the current posture.
[0106] Optionally, the acquisition module 501 may include:
[0107] The extraction unit is used to extract each line segment in the image from the current image.
[0108] The first determining unit is used to determine a target object where at least two line segments intersect at a point according to the connection status of each line segment.
[0109] As a characteristic line unit, when the target object meets the preset type, the line segment corresponding to the preset type target object is used as the characteristic line of the preset type target object.
[0110] Optionally, the matching module 502 may include:
[0111] The second determining unit is used to determine the characteristic value of the historical characteristic line and the characteristic value of the characteristic line of the preset type target.
[0112] The first calculation unit is used to calculate a first error value between a characteristic value of the historical characteristic line and a characteristic value of a characteristic line of the preset type target.
[0113] The first matching unit is used to match successfully when the first error value is smaller than a preset first threshold.
[0114] Optionally, the matching module 502 may further include:
[0115] The third determining unit is used to determine the characteristic points of the preset type of target according to the characteristic lines of the preset type of target.
[0116] The fourth determining unit is used to determine the historical feature points of the preset type of target according to the historical feature line.
[0117] The second calculation unit is used to calculate a second error value between the position of the historical feature point and the position of the feature point of the preset type target.
[0118] The second matching unit is used to match successfully when the second error value is smaller than a preset second threshold.
[0119] Optionally, the second determining unit may include:
[0120] The intersection determination subunit is used to determine the intersection points where the characteristic lines of the preset type of target intersect.
[0121] As a feature point subunit, it is used to use the intersection point as a feature point of the preset type of target.
[0122] Optionally, the construction module 503 may include:
[0123] The first acquisition unit is used to acquire a set of sampling points at the same position between the characteristic line of the preset type of target and the historical characteristic line.
[0124] The third calculation unit is used to calculate the sampling point set, the historical feature points and the feature points of the preset type of target according to a preset algorithm, and determine the rotation matrix and the translation matrix between the adjacent image and the current image.
[0125] A fifth determination unit is used to determine a current posture according to the rotation matrix and the translation matrix.
[0126] Optionally, the construction module 503 may further include:
[0127] The second acquisition unit is used to acquire at least three pairs of sampling points at the same position between the characteristic line of the preset type of target and the historical characteristic line.
[0128] The fourth calculation unit is used to calculate the at least three pairs of sampling points according to a preset algorithm to determine the rotation matrix and the translation matrix between the adjacent image and the current image.
[0129] A sixth determination unit is used to determine a current posture according to the rotation matrix and the translation matrix.
[0130] In the embodiment of the present application, a current image is obtained, and a characteristic line of a preset type of target is determined from the current image, and a characteristic line for comparison in the current environment of the robot is determined; an adjacent image is obtained, and a historical characteristic line of the preset type of target is determined from the adjacent image; the characteristic line of the preset type of target is matched with the historical characteristic line, so as to determine whether the characteristic line currently obtained by the robot is of the same preset type of target as the historical characteristic line. When the match is successful, the current posture is determined according to the characteristic line of the preset type of target and the historical characteristic line, so as to further more accurately derive the current posture of the robot according to the current environment, and construct a map according to the current posture, thereby improving the accuracy and robustness of visual SLAM mapping and positioning.
[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned system embodiments and method embodiments, and will not be repeated here.
[0132] Figure 6 This is a schematic diagram of the structure of the robot provided in the embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.
[0133] like Figure 6 As shown, the robot 6 of this embodiment includes: at least one processor 600 ( Figure 6 Only one is shown in the figure), a memory 601 connected to the processor 600, and a computer program 602 stored in the memory 601 and executable on the at least one processor 600, such as a visual SLAM program. When the processor 600 executes the computer program 602, the steps in the above-mentioned various visual SLAM method embodiments are implemented, such as Figure 1 Alternatively, when the processor 600 executes the computer program 602, the functions of the modules in the above-mentioned device embodiments are realized, for example, Figure 5 Functions of modules 501 to 503 are shown.
[0134] Exemplarily, the computer program 602 may be divided into one or more modules, which are stored in the memory 601 and executed by the processor 600 to complete the present application. The one or more modules may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 602 in the robot 6. For example, the computer program 602 may be divided into an acquisition module 501, a matching module 502, and a construction module 503, and the specific functions of each module are as follows:
[0135] An acquisition module 501 is used to acquire a current image and determine a characteristic line of a preset type of target from the current image;
[0136] A matching module 502 is used to obtain adjacent images, determine the historical characteristic line of the preset type of target from the adjacent images, and match the characteristic line of the preset type of target with the historical characteristic line;
[0137] The construction module 503 is used to determine the current posture according to the feature line when the matching is successful, and to construct a map according to the current posture.
[0138] The robot 6 may include, but is not limited to, a processor 600 and a memory 601. Those skilled in the art will appreciate that Figure 6 This is only an example of the robot 6 and does not constitute a limitation on the robot 6. The robot 6 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, buses, etc.
[0139] The processor 600 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0140] In some embodiments, the memory 601 may be an internal storage unit of the robot 6, such as a hard disk or memory of the robot 6. In other embodiments, the memory 601 may also be an external storage device of the robot 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the robot 6. Further, the memory 601 may also include both an internal storage unit and an external storage device of the robot 6. The memory 601 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 601 may also be used to temporarily store data that has been output or is to be output.
[0141] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0142] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0143] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0144] In the embodiments provided in the present application, it should be understood that the disclosed devices / robots and methods can be implemented in other ways. For example, the device / robot embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0146] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0147] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / robot, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.
[0148] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A visual SLAM method, characterized in that: include: Acquire a current image, and determine characteristic lines of a preset type of target from the current image, including: extracting each line segment in the image from the current image; determining a target in which at least two line segments intersect at a point according to the connection status of each line segment; when the target meets the preset type, use the line segment corresponding to the preset type of target as the characteristic line of the preset type of target; Acquire adjacent images, determine the historical characteristic lines of the preset type of target from the adjacent images; match the characteristic lines of the preset type of target with the historical characteristic lines; When the matching is successful, the current posture is determined according to the characteristic lines of the preset type of target and the historical characteristic lines, and a map is constructed according to the current posture.
2. The visual SLAM method according to claim 1, wherein The matching process of the characteristic line of the preset type of target with the historical characteristic line includes: Determine the characteristic value of the historical characteristic line and the characteristic value of the characteristic line of the preset type target; Calculating a first error value between a characteristic value of the historical characteristic line and a characteristic value of a characteristic line of the preset type target; When the first error value is smaller than a preset first threshold, the matching is successful.
3. The visual SLAM method as claimed in claim 1, wherein The matching process of the characteristic line of the preset type of target with the historical characteristic line also includes: Determining characteristic points of the preset type of target according to the characteristic lines of the preset type of target; Determining historical feature points of the preset type of target according to the historical feature line; Calculating a second error value between the position of the historical feature point and the position of the feature point of the preset type target; When the second error value is smaller than a preset second threshold, the matching is successful.
4. The visual SLAM method as claimed in claim 3, wherein: The determining the characteristic points of the preset type of target according to the characteristic lines of the preset type of target includes: Determining the intersection points where the characteristic lines of the preset type of target intersect each other; The intersection point is used as a feature point of the preset type of target.
5. The visual SLAM method as claimed in claim 3, wherein: The determining the current posture according to the characteristic line of the preset type target and the historical characteristic line includes: Acquire a set of sampling points at the same position of the characteristic line of the preset type of target and the historical characteristic line; Calculate the sampling point set, the historical feature points and the feature points of the preset type of target according to a preset algorithm, and determine a rotation matrix and a translation matrix between the adjacent image and the current image; The current position and posture are determined according to the rotation matrix and the translation matrix.
6. The visual SLAM method as claimed in claim 1, wherein: The determining of the current posture according to the characteristic line of the preset type of target and the historical characteristic line also includes: Obtain at least three pairs of sampling points at the same position between the characteristic line of the preset type of target and the historical characteristic line; Calculate the at least three pairs of sampling points according to a preset algorithm to determine a rotation matrix and a translation matrix between the adjacent image and the current image; The current position and posture are determined according to the rotation matrix and the translation matrix.
7. A visual SLAM device, characterized in that: include: An acquisition module, used to acquire a current image and determine a characteristic line of a preset type of target from the current image; The acquisition module includes: an extraction unit, used to extract each line segment in the image from the current image; a first determination unit, used to determine a target object where at least two line segments intersect at a point according to the connection status of each line segment; a characteristic line unit, used to use the line segment corresponding to the preset type of target object as the characteristic line of the preset type of target object when the target object meets the preset type; A matching module, used to obtain adjacent images, determine the historical characteristic line of the preset type of target from the adjacent images; and match the characteristic line of the preset type of target with the historical characteristic line; The construction module is used to determine the current posture according to the feature line when the matching is successful, and to construct a map according to the current posture.
8. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of a visual SLAM method as described in any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a visual SLAM method as claimed in any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Visual SLAM method based on structural lines
CN104077809A
A visual SLAM method and a visual SLAM device based on point-line characteristics
CN109558879A
Visual inertia speedometer method, visual inertia speedometer device and mobile equipment
CN110246147A
Image processing apparatus, image processing method, image processing system and program
JP2017208038A
