Visual processing method and mobile robot
By acquiring environmental images and extracting feature points in a mobile robot, and adding feature labels, the problem of visual localization and map building in unknown environments is solved, achieving efficient visual localization and environmental map building.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ECOVACS ROBOTICS CO LTD
- Filing Date
- 2022-04-25
- Publication Date
- 2026-06-05
AI Technical Summary
Existing mobile robots lack feature identifiers in unknown environments, making it impossible to achieve visual localization and environmental mapping.
By acquiring an environmental image of the initial area, initial feature points are extracted and visually located. When the feature points cannot meet the requirements, location information is provided to the feature label setting subject, and feature labels are added in specific areas to meet the visual positioning requirements.
When feature points cannot meet the requirements for visual positioning, targeted addition of feature markers improves the efficiency and accuracy of visual positioning, ensuring the efficient construction of environmental maps.
Smart Images
Figure CN116989784B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of mobile robots, specifically to a vision processing method and a mobile robot. Background Technology
[0002] Visual navigation technology for mobile robots is a rapidly developing and maturing technology, and a manifestation of the intelligence of mobile robots. It acquires environmental image information through cameras, processes the acquired information, transforms it into map data usable by computers, and then integrates and analyzes the data to achieve visual localization of the mobile robot. Visual localization refers to perceiving the surrounding environment through fixed visual sensors on the mobile robot, extracting environmental feature information through a series of filtering and noise reduction processes, and finally calculating the mobile robot's pose using image processing and machine vision technologies. Simultaneous Localization and Map Building (SLAM) technology is mainly used to solve the localization, navigation, and map building problems of mobile robots operating in unknown environments. It refers to the mobile robot using sensors to perform self-localization and incrementally build a complete environmental map without knowing its own position in the environment.
[0003] Existing mobile robots face challenges in map building due to a lack of feature markers in unknown environments, preventing them from achieving visual localization and constructing environmental maps. Summary of the Invention
[0004] This application provides a visual processing method, a mobile robot, and a method for a mobile robot to construct an environmental map in an initial area, in order to solve the technical problems in the prior art that visual positioning and environmental map construction are impossible due to the lack of feature markers in unknown environments.
[0005] This application provides a vision processing method applied to a mobile robot, comprising:
[0006] The system acquires an environmental image of the initial region and performs feature extraction on the environmental image to obtain initial feature points. The initial region refers to the area where the mobile robot has not acquired an environmental map.
[0007] Visual localization is performed based on the initial feature points;
[0008] In response to the fact that the first initial feature point in the first sub-region of the initial region cannot meet the visual positioning requirements, the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region are provided to the feature identification setting body, so that the feature identification setting body can add feature identifications at the boundary location between the first sub-region and / or the first sub-region and the second sub-region, wherein the second sub-region is a sub-region in the initial region that is adjacent to the first sub-region, and the second initial feature point in the second sub-region meets the visual positioning requirements.
[0009] Optionally, the first initial feature point in the first sub-region of the initial region cannot meet the visual positioning requirements, including: feature point matching cannot be performed based on the first initial feature point; or, the camera motion cannot be calculated based on the matched feature points in the first initial feature point; or, the incremental environmental sub-map corresponding to the first sub-region cannot be constructed based on the matched feature points in the first initial feature point; or, the first initial feature point cannot be extracted from the environmental image.
[0010] Optionally, the inability to perform feature point matching based on the first initial feature point includes at least one of the following:
[0011] The number of the first initial feature points extracted from the environmental image is less than a predetermined threshold.
[0012] The distribution density of the first initial feature points extracted from the environmental image is lower than a predetermined density threshold.
[0013] Optionally, the visual localization based on the initial feature points includes:
[0014] Feature point matching is performed on the initial feature points extracted from adjacent environmental images to obtain the matched feature points;
[0015] Based on the matched feature points, the motion of the mobile robot's camera is estimated to obtain the camera's motion information;
[0016] The spatial location of the matched feature points is calculated based on the motion information, and an incremental environment map is constructed based on the spatial location.
[0017] Optionally, the method further includes:
[0018] Obtain the difference information between the second initial feature point and the first initial feature point; provide the difference information to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the difference information; or,
[0019] Obtain the demand level information for feature points in the first sub-region; provide the demand level information to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the demand level information.
[0020] Optionally, the method further includes:
[0021] Obtain the distribution data of the first initial feature points in the first sub-region;
[0022] The distribution data is provided to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the distribution data.
[0023] Optionally, the distribution data of the first initial feature points in the first sub-region includes one of the following:
[0024] The number of the first initial feature points distributed in different blocks of the first sub-region;
[0025] The distribution pattern of the first initial feature point in different blocks of the first sub-region.
[0026] Optionally, providing the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region to the feature identifier setting entity includes at least one of the following:
[0027] Send the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region to the terminal of the feature identification setting entity;
[0028] The location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region are output to the feature identification setting entity by voice broadcast;
[0029] Marking information is set at the corresponding positions of the first sub-region and / or the boundary between the first sub-region and the second sub-region.
[0030] Optionally, it also includes: providing the location information of the feature identifier to be set in the first sub-region to the feature identifier setting subject.
[0031] Optionally, the feature identifier setting entity adds a feature identifier to the first sub-region, including: the feature identifier setting entity uniformly sets the feature identifier in the first sub-region according to a preset area;
[0032] The feature identification setting entity adds a feature identifier at the boundary position between the first sub-region and the second sub-region, including: the feature identification setting entity uniformly sets the feature identifier at a preset distance along the boundary position between the first sub-region and the second sub-region.
[0033] Optionally, the size of the feature identifier corresponds to the size of the first sub-region.
[0034] Optionally, it also includes: detecting that a feature identifier has been added in the first sub-region, determining that the first sub-region meets the visual positioning requirements, and providing the confirmation information that the first sub-region meets the visual positioning requirements to the feature identifier setting entity.
[0035] This application embodiment also provides a mobile robot, which has a vision processing module;
[0036] The visual processing module is used to: acquire an environmental image captured for an initial region, and extract features from the environmental image to obtain initial feature points, wherein the initial region refers to the region where the mobile robot has not acquired its environmental map; perform visual localization based on the initial feature points; and, in response to the first initial feature point in the first sub-region of the initial region failing to meet the visual localization requirements, provide the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region to the feature identification setting entity, so that the feature identification setting entity can add feature identifiers at the boundary location between the first sub-region and / or the first sub-region and the second sub-region, wherein the second sub-region is a sub-region in the initial region adjacent to the first sub-region, and the second initial feature point in the second sub-region meets the visual localization requirements.
[0037] This application also provides a method for a mobile robot to construct an environmental map in an initial area, including:
[0038] The mobile robot travels to an initial area where no environmental map has been acquired and initiates the environmental map building task;
[0039] The mobile robot captures images of the initial area to obtain a first environmental image, and extracts features from the first environmental image to obtain initial feature points;
[0040] Visual positioning is performed based on the initial feature points; in response to the fact that the first initial feature points in the first sub-region of the initial region cannot meet the visual positioning requirements, the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region are provided to the feature identification setting body, so that the feature identification setting body can add feature identifications at the boundary location between the first sub-region and / or the first sub-region and the second sub-region, wherein the second sub-region is a sub-region in the initial region that is adjacent to the first sub-region, and the second initial feature points in the second sub-region meet the visual positioning requirements;
[0041] After the feature identification setting entity adds feature identification at the boundary position between the first sub-region and / or the second sub-region, the mobile robot takes an image of the initial region again to obtain a second environmental image, and performs feature extraction on the second environmental image to obtain target feature points.
[0042] Visual localization is performed based on the target feature points, and an incremental environment map corresponding to the initial region is constructed.
[0043] The visual processing method provided in this application is applied to a mobile robot, including: acquiring an environmental image captured for an initial region, and extracting features from the environmental image to obtain initial feature points, wherein the initial region refers to the region where the mobile robot has not acquired an environmental map; performing visual localization based on the initial feature points; and, in response to the first initial feature point in a first sub-region of the initial region failing to meet the visual localization requirements, providing the position information of the first sub-region and / or the boundary position information between the first sub-region and the second sub-region to a feature label setting entity, so that the feature label setting entity can add feature labels at the boundary positions of the first sub-region and / or the boundary positions between the first sub-region and the second sub-region, wherein the second sub-region is a sub-region adjacent to the first sub-region in the initial region, and the second initial feature point in the second sub-region meets the visual localization requirements. By using this method, when the first initial feature point in the first sub-region of the initial region fails to meet the visual localization requirements, feature labels can be added in a targeted manner based on the actual visual localization requirements of the mobile robot for the first sub-region. This method can efficiently and accurately determine the location for adding feature labels, and also makes the added feature labels more consistent with the visual localization requirements of the mobile robot. Attached Figure Description
[0044] Figure 1 A flowchart of the visual processing method provided in the first embodiment of this application;
[0045] Figure 2 A schematic diagram illustrating an implementation scenario of the visual processing method provided in the first embodiment of this application;
[0046] Figure 3 A flowchart illustrating a method for a mobile robot to construct an environmental map in an initial region, as provided in the third embodiment of this application. Detailed Implementation
[0047] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.
[0048] When mobile robots build maps in unknown environments, they primarily use SLAM (Simultaneous Localization and Mapping) technology. This involves the mobile robot starting from an unknown location in the unknown environment, performing localization based on position estimation and the map during movement, and simultaneously building an incremental map based on its localization to achieve autonomous localization and navigation. Localization can essentially be defined as an estimation problem (i.e., estimating the camera position using camera observation data) or an optimization problem (i.e., optimizing the estimated camera position using constraints between various observation data). Map building is the process of fusing observation data based on localization. SLAM technology mainly includes the following:
[0049] Read the environmental images captured by the mobile robot's camera and preprocess them;
[0050] The Visual Odometry (VO) task estimates camera motion between adjacent environmental images and estimates the appearance of a local map. This process requires calculating the rotation matrix R and translation vector t of the camera's inter-frame motion (rotation matrix, translation vector, quaternion, Euler angles) based on the environmental images returned by the camera (i.e., calculating how the camera moves), and estimating the approximate spatial coordinates of feature points to estimate the local environmental map (point cloud).
[0051] Backend optimization involves receiving camera pose measurements from visual odometry at different times and loop closure detection information, optimizing them to obtain a globally consistent trajectory and map.
[0052] Loop closure detection determines whether a mobile robot has reached a previous position. If a loop is detected, the information is provided to the backend for processing. When a camera reaches a position it has previously reached in the environment, due to errors in the sensor and calculation process, the point cloud map usually cannot achieve a closed curve. The function of loop closure detection is to determine whether the camera has returned to a previous position, thereby correcting the error and ensuring that the point cloud map matches the actual space.
[0053] Mapping involves creating a map that corresponds to the task requirements based on the estimated trajectory.
[0054] To address the problem that existing mobile robots cannot perform visual processing when building maps of unknown environments due to the lack of feature markers in the unknown environment, the first embodiment of this application provides a visual processing method. The execution subject of this method is a mobile robot, or more precisely, the visual processing module of the mobile robot. This embodiment mainly targets the aforementioned visual odometry (VO) task. That is, when visual odometry estimates camera motion between adjacent environmental images and estimates the appearance of local environmental maps, if it is found that the feature points of a certain area cannot meet the visual positioning requirements of the mobile robot, feature markers are added to that area. The mobile robot can be a smart lawnmower or a cleaning robot, etc.
[0055] Please refer to Figure 1 Understanding this embodiment, Figure 1 This is a flowchart of the visual processing method provided in the first embodiment of this application; the following is in conjunction with... Figure 1 The method will be explained.
[0056] like Figure 1 As shown, the visual processing method provided in this embodiment includes the following steps:
[0057] S101, acquire the environmental image captured for the initial region, and extract features from the environmental image to obtain initial feature points.
[0058] This step involves acquiring an environmental image of the initial region and extracting features from the image to obtain initial feature points. The initial region refers to the area where the mobile robot has not acquired an environmental map. The initial region is essentially the unknown region for the mobile robot. For example, after the mobile robot's camera transmits the environmental image (a series of image frames) of the unknown region to its vision processing module, the module uses an image recognition algorithm to extract features from the image and obtain the extracted initial feature points.
[0059] S102, visual localization based on initial feature points.
[0060] After extracting features from the captured environmental image in the above steps to obtain initial feature points, this step is used to perform visual localization based on these initial feature points and construct an incremental environmental map corresponding to the initial region. This process includes the following:
[0061] First, feature point matching is performed on the initial feature points extracted from adjacent environmental images to obtain matched feature points. For each feature point, a descriptor is used to describe it in order to distinguish it from other feature points. A descriptor is usually a vector containing information about the feature point and its surrounding area. If the descriptors of two feature points are similar, they can be considered to be the same point. Based on the information of the feature points and descriptors, the matched feature points in the two environmental images can be calculated.
[0062] Secondly, based on the matched feature points, the motion of the mobile robot's camera is estimated to obtain the camera's motion information. For example, based on the pixel coordinates of the matched feature points and the distance between the feature points and the camera, the rotation matrix R and translation vector t of the camera's inter-frame motion (rotation matrix, translation vector, quaternion, Euler angles) are calculated.
[0063] Finally, the spatial coordinates of the matched feature points are calculated based on the motion information, and an incremental environment map is constructed based on these spatial coordinates. This process essentially involves establishing a point cloud description of the map; that is, estimating the appearance of the incremental environment map using the spatial positions of a large number of feature points.
[0064] Since the above-described visual localization process based on feature points is existing technology, the specific details will not be elaborated here.
[0065] S103, in response to the fact that the first initial feature point in the first sub-region of the initial region cannot meet the visual positioning requirements, the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region are provided to the feature label setting entity, so that the feature label setting entity can add feature labels at the boundary locations of the first sub-region and / or the first sub-region and the second sub-region. A feature label refers to an object in the real environment that carries feature points; that is, the initial feature point is a point on the feature label. For example, the feature label corresponding to the first initial feature point is a tree, and the feature label corresponding to the second initial feature point is a black and white checkered pattern.
[0066] This step is used when, during visual localization based on the aforementioned initial feature points and the construction of an incremental environment map corresponding to the initial region, the first initial feature point in the first sub-region of the initial region cannot meet the visual localization requirements. In this case, the location information of the first sub-region, or the boundary location information between the first and second sub-regions, or both the location information of the first sub-region and the boundary location information, are provided to the feature identification setting entity. This allows the feature identification setting entity to add feature markers at the location of the first sub-region, or at the boundary location between the first and second sub-regions, or at both the location of the first sub-region and the boundary location. The purpose is to ensure that the added feature points meet the needs of the mobile robot for subsequent visual localization and synchronous construction of the environment map. The aforementioned second sub-region is a sub-region in the initial region adjacent to the first sub-region, and the second initial feature point in the second sub-region meets the visual localization requirements. For example, in different sub-regions of the same unknown region, some sub-regions can be located, while others cannot. Please refer to [reference needed]. Figure 2 This is a schematic diagram of an implementation scenario of the visual processing method provided in this application.
[0067] The inability of the first initial feature points in the aforementioned first sub-region to meet the visual positioning requirements refers to the failure of visual positioning in the first sub-region. This process is essentially a traversal of the initial region based on visual positioning to determine the first sub-region where visual positioning has failed. The inability to meet the positioning requirements can be due to the inability to perform feature point matching, the inability to calculate camera motion based on the matched feature points, the inability to extract feature points or the extraction of too few feature points, or the inability to calculate an incremental environmental sub-map based on the matched feature points. For example, the neatly arranged trees in a forest have high similarity in their feature identifiers due to the high similarity of their environment, making feature point matching impossible. Similarly, narrow passages or open areas lack identifiable feature identifiers, making it impossible to extract feature points from such areas, or the extracted feature points may not meet the visual positioning requirements of the mobile robot.
[0068] In this embodiment, the first initial feature point in the first sub-region of the initial region cannot meet the visual positioning requirements, which can cover the following situations:
[0069] A: Feature point matching cannot be performed based on the first initial feature point. That is, in the feature point matching stage, feature point matching cannot be performed based on feature points extracted from adjacent environmental images. In this embodiment, it can be one or more of the following situations:
[0070] Scenario 1: The number of initial feature points extracted from the environmental image is lower than a predetermined threshold, that is, the number of initial feature points extracted from the environmental image cannot meet the number required for feature point matching;
[0071] Scenario 2: The distribution density of the first initial feature points extracted from the environmental image is lower than the predetermined density threshold. That is, the density of the first initial feature points extracted from the environmental image is too low to meet the density required for feature point matching.
[0072] B: Camera motion cannot be calculated based on the matched feature points in the first initial feature points. Specifically, after matching feature points in the first initial feature points in adjacent environmental images, a set of matched points can be obtained. Camera motion can be calculated based on this set of matched points. For example, when calculating the camera's pose matrix, four sets of matched points are required. However, if the number of matched feature points in the set of matched points is too small, camera motion cannot be calculated based on these matched feature points.
[0073] C: Based on the matched feature points from the first initial feature points, it is impossible to construct an incremental environmental sub-map corresponding to the first sub-region. Specifically, after calculating the camera motion based on the matched feature points, the camera motion result is obtained. Subsequently, it is necessary to reconstruct the spatial location of the special station based on the camera motion result. Through a large number of these spatial locations, the appearance of the environmental map can be estimated. In this process, if the number of reconstructed feature point spatial locations is too small and the environmental map is too sparse when constructing it, it will result in the inability to construct the environmental map.
[0074] D: The first initial feature point cannot be extracted from the adjacent environmental image. That is, the first initial feature point cannot be extracted from the environmental image during the feature point extraction stage. In this embodiment, it can be one or more of the following situations:
[0075] The number of feature points in the environmental image is less than a predetermined threshold.
[0076] The distribution density of feature points in the environmental image is less than a predetermined density threshold.
[0077] It should be noted that, in another embodiment of this application, difference information between the second initial feature point and the first initial feature point can also be obtained; this difference information is provided to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the difference information. This difference information can serve as reference information when adding feature identifiers, and it can be any information characterizing the difference between the second initial feature point and the first initial feature point. For example, the difference information can be the difference in quantity between the second initial feature point and the first initial feature point, or the difference between the category of the feature identifier corresponding to the second initial feature point and the category of the feature identifier corresponding to the first initial feature point. For example, the feature identifier corresponding to the first initial feature point is a tree, and the feature identifier corresponding to the second initial feature point is a black and white checkered pattern. After providing the difference information to the feature identifier setting entity, the feature identifier setting entity can add a corresponding number of feature identifiers in the first sub-region according to the above-mentioned quantity difference, so that the feature points added in the first sub-region can (such as the second initial feature points in the second sub-region) meet the visual positioning requirements, or feature identifiers of the same category as the feature identifiers corresponding to the second initial feature points can be added in the first sub-region, so that after recognizing the environmental image containing the feature identifiers added in the first sub-region, feature points that meet the visual positioning requirements can be extracted.
[0078] It should be noted that, in another embodiment of this application, the demand level information for feature points in the first sub-region can also be obtained. That is, different sub-regions may have different demand levels for feature points, and this demand level can be divided based on the importance of the region. The demand level information is provided to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the demand level information. For example, if the first sub-region in the initial region has a higher importance, then its demand level for feature points is higher; if the third sub-region has a lower importance, then its demand level for feature points is lower. Therefore, more feature points can be added in the first sub-region, while fewer feature points can be added in the third sub-region.
[0079] It should be noted that, in another embodiment of this application, the distribution data of the first initial feature points in the first sub-region can also be obtained. This distribution data can be the number of the first initial feature points distributed in different blocks of the first sub-region, or it can be the distribution pattern of the first initial feature points in different blocks of the first sub-region (for example, the first initial feature points are distributed in different blocks of the first sub-region in decreasing order from near to far). The distribution data is then provided to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the distribution data. In this way, the addition of feature identifiers can be more accurate and targeted. For example, a corresponding number of feature identifiers can be added to different blocks of the first sub-region in a targeted manner, or feature identifiers that conform to the distribution pattern of the first initial feature points in different blocks of the first sub-region can be added.
[0080] In this embodiment, the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region can be provided to the feature identification setting entity through one or more of the following methods:
[0081] Send the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region to the terminal of the feature identification setting entity, for example, send the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region to the mobile APP of the feature identification setting personnel;
[0082] The location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region are output to the feature identification setting entity by voice broadcast;
[0083] Marking information is set at the corresponding position of the boundary between the first sub-region and / or the second sub-region. For example, a mobile robot can set pattern marking information at the corresponding position of the boundary between the first sub-region and / or the second sub-region, and the feature identifier setter can set feature identifiers accordingly based on the pattern marking information.
[0084] In this embodiment, in addition to providing the location information of the first sub-region and / or the boundary location information between the first and second sub-regions to the feature identifier setting body, the specific location for adding the feature identifier can also be provided to the feature identifier setting body. The feature identifier setting body adding the feature identifier in the first sub-region can mean that the feature identifier setting body evenly distributes the feature identifier in the first sub-region according to a preset area, or sets the feature identifier according to the specific location provided by the mobile robot. The feature identifier setting body adding the feature identifier at the boundary location between the first and second sub-regions can mean that the feature identifier setting body evenly distributes the feature identifier at a preset distance along the boundary location between the first and second sub-regions, or sets the feature identifier according to the specific location provided by the mobile robot. In this embodiment, the size of the feature identifier can correspond to the size of the first sub-region.
[0085] It should be noted that when the mobile robot detects that a feature marker has been added to the first sub-region, it can determine that the first sub-region meets the visual positioning requirements. In this case, the confirmation information that the first sub-region meets the visual positioning requirements can be provided to the feature marker setting entity.
[0086] In summary, by using the visual processing method provided in this embodiment, when the first initial feature points in the first sub-region of the initial region cannot meet the visual positioning requirements, feature markers can be added in a targeted manner based on the actual visual positioning requirements of the mobile robot for the first sub-region. This can efficiently and accurately determine the location of the added feature markers, and also make the added feature markers more in line with the visual positioning requirements of the mobile robot.
[0087] Corresponding to the first embodiment, the second embodiment of this application also provides a mobile robot. Since this mobile robot performs visual processing based on the method provided in the first embodiment, the mobile robot provided in this application is basically similar to the first embodiment. The following is only a brief description of this system embodiment; other relevant details can be found in the description of the first embodiment above.
[0088] The mobile robot provided in this embodiment has a vision processing module; the vision processing module is used to: acquire an environmental image captured for an initial region, and extract features from the environmental image to obtain initial feature points, wherein the initial region refers to the region where the mobile robot has not acquired its environmental map; perform visual localization based on the initial feature points, and construct an incremental environmental map corresponding to the initial region; in response to the first initial feature point in the first sub-region of the initial region failing to meet the visual localization requirements, provide the position information of the first sub-region and / or the boundary position information between the first sub-region and the second sub-region to a feature identification setting entity, so that the feature identification setting entity can add feature identifications at the boundary position between the first sub-region and / or the first sub-region and the second sub-region, wherein the second sub-region is a sub-region in the initial region adjacent to the first sub-region, and the second initial feature point in the second sub-region meets the visual localization requirements.
[0089] By using this mobile robot, when the first initial feature point in the first sub-region of the initial area cannot meet the visual positioning requirements, feature markers can be added in a targeted manner based on the actual visual positioning requirements of the mobile robot in the first sub-region. The location of the added feature markers can be obtained efficiently and accurately, and the added feature markers can better meet the visual positioning requirements of the mobile robot.
[0090] Corresponding to the first embodiment described above, the third embodiment of this application combines a specific vision processing scenario for a mobile robot and describes the scenario embodiment using at least one optional implementation method from the first embodiment.
[0091] This method is applied to construct an environmental map of unknown areas after a mobile robot has started. Please refer to [reference needed]. Figure 3 This is a flowchart of a method for a mobile robot to construct an environmental map in an initial area, as provided in the third embodiment of this application.
[0092] like Figure 3 As shown, the method includes the following steps:
[0093] Step 301: The mobile robot travels to the initial area where no environmental map has been acquired and starts the environmental map building task;
[0094] Step 302: The mobile robot captures images of the initial area to obtain a first environmental image, and extracts features from the first environmental image to obtain initial feature points.
[0095] Step 303: When performing visual positioning based on the initial feature points and constructing an incremental environment map corresponding to the initial region, in response to the fact that the first initial feature points in the first sub-region of the initial region cannot meet the visual positioning requirements, the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region are provided to the feature identification setting entity, so that the feature identification setting entity can add feature identifications at the boundary location between the first sub-region and / or the first sub-region and the second sub-region, wherein the second sub-region is a sub-region in the initial region that is adjacent to the first sub-region, and the second initial feature points in the second sub-region meet the visual positioning requirements;
[0096] Step 304: After the feature identification setting entity adds feature identification at the boundary position between the first sub-region and / or the first sub-region and the second sub-region, the mobile robot takes an image of the initial region again to obtain a second environmental image, and performs feature extraction on the second environmental image to obtain target feature points.
[0097] Step 305: Perform visual localization based on target feature points and construct an incremental environment map corresponding to the initial region.
[0098] The third embodiment described above is a specific implementation method provided in conjunction with a specific application scenario. This implementation method is based on the first embodiment described above and is more closely integrated with the specific scenario. Its specific technical features can also be further combined with the first embodiment to form implementation methods for different situations.
[0099] This method addresses situations where the initial feature points in the first sub-region of the initial region cannot meet the visual positioning requirements. Based on the mobile robot's actual visual positioning needs for the first sub-region, it strategically adds feature markers. After adding the feature markers, the mobile robot re-captures images of the initial region to obtain a second environmental image. Feature extraction is then performed on this second environmental image to obtain target feature points. Visual positioning is then performed based on these target feature points, and an incremental environmental map corresponding to the initial region is constructed. By using this method, the location of added feature markers can be efficiently and accurately determined, ensuring that the added feature markers better meet the aforementioned visual positioning requirements of the mobile robot, and enabling the construction of an incremental environmental map corresponding to the initial region.
[0100] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.
Claims
1. A visual processing method, characterized in that, The method is applied to mobile robots and includes: The system acquires an environmental image of the initial region and performs feature extraction on the environmental image to obtain initial feature points. The initial region refers to the area where the mobile robot has not acquired an environmental map. Visual localization is performed based on the initial feature points; In response to the fact that the first initial feature point in the first sub-region of the initial region cannot meet the visual positioning requirements, the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region are provided to the feature identification setting body, so that the feature identification setting body can add feature identifications at the boundary location between the first sub-region and / or the first sub-region and the second sub-region, wherein the second sub-region is a sub-region in the initial region that is adjacent to the first sub-region, and the second initial feature point in the second sub-region meets the visual positioning requirements; The method further includes: Obtain the difference information between the second initial feature point and the first initial feature point; provide the difference information to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the difference information; or, Obtain the demand level information for feature points in the first sub-region; provide the demand level information to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the demand level information.
2. The visual processing method according to claim 1, characterized in that, The first initial feature point in the first sub-region of the initial region cannot meet the visual positioning requirements, including: Feature point matching cannot be performed based on the first initial feature point; or... Based on the matched feature points from the first initial feature points, it is impossible to calculate the camera's motion; or, Based on the matched feature points from the first initial feature points, it is impossible to construct an incremental environmental sub-map corresponding to the first sub-region; or... The first initial feature point could not be extracted from the environmental image.
3. The visual processing method according to claim 2, characterized in that, The inability to perform feature point matching based on the first initial feature point includes at least one of the following: The number of the first initial feature points extracted from the environmental image is less than a predetermined threshold. The distribution density of the first initial feature points extracted from the environmental image is lower than a predetermined density threshold.
4. The visual processing method according to claim 1, characterized in that, The visual localization based on the initial feature points includes: Feature point matching is performed on the initial feature points extracted from adjacent environmental images to obtain the matched feature points; Based on the matched feature points, the motion of the mobile robot's camera is estimated to obtain the camera's motion information; The spatial location of the matched feature points is calculated based on the motion information, and an incremental environment map is constructed based on the spatial location.
5. The visual processing method according to claim 1, characterized in that, The method further includes: Obtain the distribution data of the first initial feature points in the first sub-region; The distribution data is provided to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the distribution data.
6. The visual processing method according to claim 5, characterized in that, The distribution data of the first initial feature point in the first sub-region includes one of the following: The number of the first initial feature points distributed in different blocks of the first sub-region; The distribution pattern of the first initial feature point in different blocks of the first sub-region.
7. The visual processing method according to claim 1, characterized in that, Providing the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region to the feature identification setting entity includes at least one of the following: Send the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region to the terminal of the feature identification setting entity; The location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region are output to the feature identification setting entity by voice broadcast; Marking information is set at the corresponding positions of the first sub-region and / or the boundary between the first sub-region and the second sub-region.
8. The visual processing method according to claim 1, characterized in that, Also includes: The location information of the feature identifier to be set in the first sub-region is provided to the feature identifier setting entity.
9. The visual processing method according to claim 1, characterized in that, The feature identification setting entity adds feature identification to the first sub-region, including: the feature identification setting entity uniformly sets feature identification in the first sub-region according to a preset area, or sets feature identification according to the specific location provided by the mobile robot; The feature identification setting entity adds a feature identifier at the boundary position between the first sub-region and the second sub-region, including: the feature identification setting entity uniformly sets the feature identifier at a preset distance along the boundary position between the first sub-region and the second sub-region, or sets the feature identifier according to the specific position provided by the mobile robot.
10. The visual processing method according to claim 1, characterized in that, The size of the feature identifier corresponds to the size of the first sub-region.
11. The visual processing method according to claim 1, characterized in that, Also includes: If a feature identifier has been added to the first sub-region, it is determined that the first sub-region meets the visual positioning requirements, and the confirmation information that the first sub-region meets the visual positioning requirements is provided to the feature identifier setting entity.
12. A mobile robot, characterized in that, The mobile robot has a vision processing module; The visual processing module is used to: acquire an environmental image captured for an initial region, and extract features from the environmental image to obtain initial feature points, wherein the initial region refers to the region where the mobile robot has not acquired its environmental map; perform visual localization based on the initial feature points; and, in response to the first initial feature point in the first sub-region of the initial region failing to meet the visual localization requirements, provide the position information of the first sub-region and / or the boundary position information between the first sub-region and the second sub-region to the feature identification setting entity, so that the feature identification setting entity can add feature identifiers at the boundary position between the first sub-region and / or the first sub-region and the second sub-region, wherein the second sub-region is a sub-region in the initial region adjacent to the first sub-region, and the second initial feature point in the second sub-region meets the visual localization requirements. Specifically, the difference information between the second initial feature point and the first initial feature point is obtained; the difference information is provided to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the difference information; or... Obtain the demand level information for feature points in the first sub-region; provide the demand level information to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the demand level information.
13. A method for a mobile robot to construct an environmental map in an initial area, characterized in that, include: The mobile robot travels to an initial area where no environmental map has been acquired and initiates the environmental map building task; The mobile robot captures images of the initial area to obtain a first environmental image, and extracts features from the first environmental image to obtain initial feature points. Visual positioning is performed based on the initial feature points; in response to the fact that the first initial feature points in the first sub-region of the initial region cannot meet the visual positioning requirements, the location information of the first sub-region and / or the boundary location information between the first sub-region and the second sub-region are provided to the feature identification setting body, so that the feature identification setting body can add feature identifications at the boundary location between the first sub-region and / or the first sub-region and the second sub-region, wherein the second sub-region is a sub-region in the initial region that is adjacent to the first sub-region, and the second initial feature points in the second sub-region meet the visual positioning requirements; After the feature identification setting entity adds feature identification at the boundary position between the first sub-region and / or the second sub-region, the mobile robot takes an image of the initial region again to obtain a second environmental image, and performs feature extraction on the second environmental image to obtain target feature points. Visual localization is performed based on the target feature points, and an incremental environment map corresponding to the initial region is constructed. Specifically, the difference information between the second initial feature point and the first initial feature point is obtained; the difference information is provided to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the difference information; or... Obtain the demand level information for feature points in the first sub-region; provide the demand level information to the feature identifier setting entity so that the feature identifier setting entity can add feature identifiers based on the demand level information.