Robot mapping method based on gap recognition and robot

Abstract

The invention discloses a robot mapping method based on gap recognition and a robot, and the method comprises the steps: A, enabling the robot to walk along a wall and complete at least one closed-loop path in a manner of target gap recognition, and enabling at least one circle of searched wall chain reserved with at least one target gap to form a first partition; entering a non-traversal area through the identified target gap, walking according to a preset walking mode, and keeping target gap identification; b, when the robot recognizes a new target gap, the robot does not pass through the new target gap, continuously walks according to the preset walking mode and recognizes the target gap until the robot walks a closed-loop area according to the preset walking mode, the closed-loop area is marked as a target subarea, and the target subarea is marked as a target subarea; and then entering a non-traversal area through the identified new target gap, walking according to a preset walking mode, and keeping target gap identification until the robot cannot identify the new target gap.

Description

Technical Field

[0001] The present application relates to the technical field of regional planning, and in particular to a robot mapping method and a robot based on gap recognition. Background Art

[0002] For sweeping robots, zoned cleaning is a very important function. Although the home environment is a highly structured environment, currently sweeping robots basically use lidar or depth camera detection technology to obtain the entire environment contour data before dividing the area. Before the area is divided, the sweeping robot will not recognize the room door area in the map building navigation. Such a solution is not suitable for the sweeping robot to identify the house structure without a two-dimensional map.

[0003] In the robot mapping method disclosed in the Chinese invention patent with application number 202010764284.7, after the robot finishes walking along the preset edge direction, the door at the position of the reference segmentation boundary line is identified based on the image feature information of the position of the reference segmentation boundary line collected by the robot's camera, and the reference segmentation boundary line is marked in the laser map, so that the door divides the indoor working area into different room sub-areas; however, during the robot's walking along the preset edge direction, only the reference segmentation boundary line is set around the same room area in the pre-constructed global laser map, and the door frame is not directly identified; even after the door at the position of the reference segmentation boundary line is identified, the robot is not specifically controlled to navigate to multiple different room areas through the door at the position of the reference segmentation boundary line; therefore, it is not possible to continuously divide new room areas for the entire house environment and identify the door positions of different room areas in real time without constructing a global map. Summary of the invention

[0004] This application discloses a robot mapping method and a robot based on gap recognition, and the specific technical solutions are as follows: The robot mapping method based on gap recognition includes: step A, the robot walks at least one closed-loop path by walking along the wall and identifying the target gap, and encloses the first partition with at least one circle of wall chain with at least one target gap reserved in the search; then the robot enters the untraversed area from the first partition through the identified target gap, and then walks according to a preset walking mode, and keeps identifying the target gap; then executes step B; step B, whenever the robot identifies a new target gap, the robot does not pass through the new target gap, and continues to walk according to the preset walking mode and identify the target gap until the robot walks a closed-loop area according to the preset walking mode, marks the closed-loop area as the target partition, and then the robot enters the untraversed area from the target partition through the identified new target gap, and then walks according to the preset walking mode, and keeps identifying the target gap until the robot cannot identify the new target gap, and the robot completes the area division.

[0005] The technical effect of the present application is that the robot performs step A and step B, identifies the target gap while walking along the wall, so as to enclose the first partition and establish the corresponding map after completing the first circle, and then enter the untraversed area through the identified target gap on the basis of the first partition, and then continue to identify the target gap and walk a closed loop area according to the preset walking mode to divide a target partition, so as to perform incremental mapping based on a single target partition, and walk through each room area recursively until no new target gap is identified, and then complete the global mapping and obtain a complete map, without having to walk the entire working area in advance before starting the partitioning operation or mapping, and improve the operation efficiency and mapping efficiency by identifying the target gap one by one. Compared with the prior art, the present application does not need to add extra markers to the door, and only needs the gap feature formed by the door frame on the wall to appear in the perception field of the robot of the present application to realize the recognition of the target gap, and then locate the door frame and partition it according to the door frame, so as to realize the positioning of the door frame and the area division marking in indoor and outdoor environments through a relatively low computing power algorithm.

[0006] Furthermore, the specific method of step A includes: step A1, the robot searches for the wall, then walks along the wall and identifies the target gap, while building a map, and walks along the target gap when identifying the target gap, until the robot has walked at least one closed-loop path, obtains at least one circle of wall chain with at least one target gap reserved, and encloses the at least one circle of wall chain with at least one target gap reserved into the first partition, and sets the partition map according to the first partition; then executes step A2; step A2, the robot enters the untraversed area from the first partition through the target gap identified in step A1; then the robot walks according to the preset walking mode and keeps identifying the target gap; then executes step B. Step A1 determines multiple vertical projection lines of the wall located in the same plane or the same curved surface, and by identifying the target gap, the relative position relationship between different batches of vertical projection lines of the wall arranged continuously can be obtained. Then step A2 uses the target gap as the boundary to effectively identify each independent space in the environment. It can be seen from the technical solution composed of the above steps A1 and A2 that the application obtains the closed-loop path and the target gap, which helps to divide the first partition in the environment, that is, the first independent space; and determines the passable position information from the first independent space in the environment to the untraversed area based on the target gap. And it has good applicability for rooms with complex layouts.

[0007] Furthermore, the specific method of step B includes: step B1, judging whether the robot recognizes a new target gap, if so, executing step B2, otherwise executing step B4; step B2, the robot does not pass through the new target gap, continues to walk according to the preset walking mode, and keeps identifying the target gap until the robot walks through a closed-loop area, and then marks the closed-loop area as a target partition, and adds the map information corresponding to the target partition to the partition map to form an incremental map; then execute step B3; step B3, the robot enters the untraversed area from the target partition described in step B2 through the new target gap identified by step B1; then the robot walks according to the preset walking mode, and keeps identifying the target gap; then execute step B1; step B4, the robot completes the area division, and sets the latest formed incremental map as the global map. In the process of recursively executing steps B1 to B3, the coverage of the incremental map is continuously expanded, and the number of marked target partitions and target gaps increases. When step B4 is executed, the latest incremental map has been updated to the global map, and an overall map of each target partition and the first partition that the robot has walked in a target area is established. Compared with the prior art that requires the robot to walk along the boundaries of each partition in advance and then build a map for each partition, steps A1 to A2 and steps B1 to B4 are for partition mapping and planning. Mapping is done while walking, and when a target gap is identified, the location leading to the untraversed area and the boundary position of different partitions are recorded in the map, thereby speeding up the efficiency of robot mapping and area division.

[0008] Furthermore, in step A1, the method for searching a wall by the robot includes: the robot collects an environmental image, extracts vertical projection lines of the wall in the environmental image, and then identifies each combination of multiple vertical projection lines of the wall located in the same plane or the same curved surface as a wall to be searched. When the robot walks along the wall, each time it walks in the same direction, the plane or curved surface where the combination of multiple vertical projection lines of the wall located in the same plane or the same curved surface extracted in the same direction is located is regarded as a wall surface extending in one direction, that is, a wall.

[0009] Further, in the step A1, the method for the robot to complete a closed-loop path includes: step A101, when the robot recognizes the current wall, the robot walks along the current wall in a preset clockwise direction, and at the same time determines whether the robot recognizes the target gap, if so, executes step A102, otherwise executes step A103; step A102, determines whether the robot walks back to the preset edge starting point, if so, determines that the robot has completed a closed-loop path, otherwise executes step A103; step A103, the robot walks along the boundary formed by the location of the target gap described in step A101, and keeps searching for the wall; then executes step A104; step A104, when the robot recognizes the next wall, the robot updates the next wall to the current wall, and then executes step A101; wherein, when the robot completes a closed-loop path, the robot forms a circle of wall chains with at least one target gap reserved in accordance with the preset clockwise direction with each wall and each target gap identified in sequence in steps A101 to A104. When the robot completes a closed-loop path, the robot forms a wall chain with at least one target gap reserved in the preset clockwise direction with each wall and each target gap identified in sequence in steps A101 to A104; the projection lines of each wall and each target gap identified in sequence in steps A101 to A104 on the ground are enclosed into continuous line segments to form a complete closed loop.

[0010] Furthermore, in a frame of environmental image collected by the robot in real time, the target gap is between two adjacent walls; the target gap is surrounded by the vertical projection lines of the wall on the opposite side of the two adjacent walls, forming two vertical projection lines of the wall corresponding to the target gap; wherein the pixel distance between the two end points of the target gap is within the preset width threshold range, and the pixel distances between the intersection of the extension lines of the two vertical projection lines of the wall corresponding to the target gap and the nearest end points of the two vertical projection lines of the wall corresponding to the target gap are both within the preset projection distance threshold range; wherein the target gap is used to connect the target partitions marked by each of the two adjacent executions of step B2, or to connect the first partition and the target partition marked by the first execution of step B2. In summary, the positional relationship between the two vertical projection lines of the wall corresponding to the target gap is determined.

[0011] Furthermore, after completing the area division in step B4, the robot obtains the position information of each wall of different target partitions and the position information of the target gap, and then the robot walks along each wall in turn and passes through each target gap in turn to complete the vertical projection lines of each wall or the wall that constitutes the wall. When completing the vertical projection lines of the wall, the robot can quickly walk along the wall of the whole house, which plays a role in checking for gaps and filling gaps and perfecting the boundaries of the target partitions.

[0012] Furthermore, the method for the robot to identify the target gap includes: the robot collects an environmental image and extracts a plurality of straight line segments from the environmental image; based on the length characteristics, angle characteristics and quantity characteristics of the line segments, two reference vertical projection lines and one reference horizontal projection line forming a cross relationship are screened out from the plurality of straight line segments; based on the preset door frame size conditions, two wall vertical projection lines and one door beam line used to form a door frame are screened out from the two reference vertical projection lines and one reference horizontal projection line formed in the screened out cross relationship, and it is determined that the two wall vertical projection lines and one door beam line used to form the door frame enclose the target gap. By screening the two reference vertical projection lines and one reference horizontal projection line forming a cross relationship, the accuracy of the robot in identifying the door frame is improved.

[0013] Furthermore, the method for selecting two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship from the plurality of straight line segments according to the line segment length characteristics, angle characteristics and quantity characteristics comprises: step A111, selecting straight line segments perpendicular to the ground from the plurality of straight line segments; then, among the selected straight line segments perpendicular to the ground, selecting straight line segments whose lengths are greater than a preset wall height threshold and marking them as reference vertical projection lines; then executing step A112; step A112, determining whether the number of reference vertical projection lines marked in step A111 is less than a value of two, If yes, it is determined that the robot cannot filter out two vertical projection lines of the wall from the environment image, otherwise, execute step A113; step A113, within the preset projection distance threshold range corresponding to the center of the environment image, filter out straight line segments whose lengths are within the preset width threshold range, and then mark the filtered straight line segments whose lengths are within the preset width threshold range as reference horizontal projection lines; then execute step A114; step A114, determine whether the number of reference horizontal projection lines marked in step A113 is less than the value one, if yes, it is determined that the robot cannot filter out the door beam line from the environment image, otherwise, execute step A115; step A115, filter out reference horizontal projection lines that meet the door structure by enumerating reference vertical projection lines and reference horizontal projection lines, and then execute step A116 ; Wherein, the reference horizontal projection line that meets the door structure is a reference horizontal projection line that forms an intersection relationship with the two reference vertical projection lines; Step A116, determine whether the number of reference horizontal projection lines that meet the door structure screened out in Step A115 is greater than a value of one, if so, execute Step A117, otherwise determine that the robot cannot screen out the door beam line from the environment image; Step A117, obtain multiple groups of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship; then in each group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship, mark the two reference vertical projection lines as two wall vertical projection lines, and mark the one reference horizontal projection line as a door beam line; wherein, the extension lines of the two reference vertical projection lines intersect. In summary, by executing Steps A111 to A117, two wall vertical projection lines and one door beam line that are candidate door frame constituent line segments are determined. Then, the robot determines the direction information of the door frame where the door beam line is located relative to the current position of the robot based on the positional relationship between the center point of each door beam line and the intersection of the extension lines of the two vertical projection lines of the wall. This is also equivalent to the direction information of the door beam line relative to the coordinates of the current position of the robot in the environment image.

[0014] Furthermore, among the two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship, there is: among the endpoints of each reference vertical projection line, the endpoint closest to the intersection of the extension lines of the two reference vertical projection lines is set as a corner point, and then the corner point of each reference vertical projection line is connected to form the one reference horizontal projection line, so as to form a reference horizontal projection line that conforms to the door structure.

[0015] Furthermore, the method for selecting two vertical wall projection lines and one door beam line for forming a door frame from two reference vertical projection lines and one reference horizontal projection line that have been selected to form an intersection relationship according to a preset door frame size condition includes: step A118, marking a group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship determined in step A117 as a group of candidate door frame lines; then executing step A119; step A119, determining whether the physical distance between the vertical wall corner lines corresponding to two reference vertical projection lines in a group of candidate door frame lines is within a preset door width distance range, if so, executing step A120, otherwise, the two vertical wall projection lines for forming a door frame cannot be selected; wherein the robot uses a depth sensor to detect the physical distance between the vertical wall corner lines corresponding to the two reference vertical projection lines; step A120, calculating a distance between one vertical wall corner line in the group of candidate door frame lines. The ratio between the length of the reference horizontal projection line and the physical distance is then used to convert the length of the vertical corner line corresponding to each reference vertical projection line based on the ratio and the length of each reference vertical projection line, and then execute step A121; step A121, determine whether the length of the vertical corner line corresponding to each reference vertical projection line in a group of candidate door frame lines is within the preset door frame height range, if so, mark two reference vertical projection lines in a group of candidate door frame lines as two wall vertical projection lines for forming a door frame, and mark a reference horizontal projection line in a group of candidate door frame lines as a door beam line for forming a door frame; otherwise, two wall vertical projection lines for forming a door frame cannot be screened out; wherein, the target gap includes the door frame; wherein, the reference vertical projection line is a projection line formed by the robot collecting the vertical corner line into the coordinate system where the environmental image is located, and the vertical corner line is perpendicular to the ground where the robot walks. In summary, the present application converts the vertical projection lines from pixel size to physical size by executing steps A118 to A121, and obtains two vertical wall projection lines and one door beam line used to constitute the door frame based on the physical size, eliminates the misjudgment interference of door leaves and side wall lines perpendicular to the ground, constructs a simplified model of the door frame, obtains the vertical wall projection lines and door beam lines that conform to the simplified model of the aforementioned door frame, and correspondingly identifies a complete and accurate door frame in the environmental image.

[0016] Furthermore, in step A121, the intersections formed by two vertical projection lines of the wall marked in a group of candidate door frame lines and one door beam line marked in step A121 in the same group of candidate door frame lines are marked as corner points, and two corner points are obtained; the angle formed by the two corner points and the center of the environment image is greater than the preset angle threshold; wherein the length of each door beam line marked in step A121 in the same group of candidate door frame lines is greater than the preset door beam projection length; wherein the extension lines of the two reference vertical projection lines in the same group of candidate door frame lines intersect. Thus, the positional relationship between the two vertical projection lines of the wall in the environment image is reflected, and the door beam model is simplified in terms of angle features and door beam line length features.

[0017] A robot is provided with a visual sensor, and the robot is used to execute the robot mapping method. The method includes executing steps A and B to realize both the identification of target gaps and the division of target partitions; for step A, the robot specifically executes steps A1 and A2 to divide the first partition in the environment; for step A1, the robot specifically executes steps A101 to A104, so that the robot walks along the wall to complete a closed-loop path and forms a circle of wall chains with at least one target gap reserved, and the at least one circle of wall chains with at least one target gap reserved can be used to form the first partition; for step A1, the robot also determines the candidate partitions by executing steps A111 to A117. Select two vertical wall projection lines and one door beam line that constitute the door frame segment; for step A1, the robot also converts the vertical projection lines from pixel size to physical size by executing steps A118 to A121, and judges and obtains two vertical wall projection lines and one door beam line that constitute the door frame based on the physical size, eliminates the misjudgment interference of the door leaf and side wall lines perpendicular to the ground, and constructs a simplified model of the door frame. Based on the simplified model of the door frame, room areas close to the actual environment are divided in the indoor working area, thereby improving the rationality of the robot's partitioning and achieving faster room cleaning.

[0018] Furthermore, the visual sensor adopts a camera; the lens of the camera of the robot is installed toward the ceiling of the indoor room area where the robot is located, so that the viewing angle of the camera covers the surface of the ceiling and the space below it. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a robot mapping method based on gap recognition disclosed in an embodiment of the present application.

[0020] Figure 2 It is a schematic diagram of a door frame to be identified in an environmental image disclosed in an embodiment of the present application, wherein the robot captures a schematic diagram marked with a vertical projection line A1B1 of the wall and a door beam line B1C1 in front of or directly below the door frame, and the robot's camera is set to face the ceiling.

[0021] Figure 3 It is a flowchart of the specific steps included in the sequential execution of step A and step B disclosed in an embodiment of the present application.

[0022] Figure 4 This is a flow chart of a method in which the robot walks a closed-loop path in step A1 disclosed in an embodiment of the present application.

[0023] Figure 5 It is a flowchart of a method for selecting a reference horizontal projection line and two reference vertical projection lines that form a cross relationship before a robot identifies a target gap, disclosed in an embodiment of the present application.

[0024] Figure 6 A flowchart of a method for a robot to screen out two vertical wall projection lines and one door beam line for forming a door frame is disclosed in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will describe the technical solutions in the embodiments of the present application in detail in conjunction with the drawings in the embodiments of the present application. To further illustrate each embodiment, the present application provides drawings. These drawings are part of the disclosure of the present application, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments.

[0026] The position of the door frame in the home environment is relatively unchanged. The door can be rotatably installed in the door frame. The door is a passage connecting two rooms. The width and shape of the door frame have a unified standard. At present, the robot determines the key boundary connecting the two room areas in the home environment by identifying the position of the door frame, but it will collect and combine door line features of multiple dimensions while the robot remains in a static condition (if the sample selection is not rich enough, it will not be able to adapt to various door recognitions). It also collects door line features through detection methods such as binocular vision or laser scanning and establishes multiple pairs of template quantities for matching and recognition. All of them require building a global map in advance and marking the recognized quantities in the global map for easy regional division. The design cost and computational complexity of the sensor are high, which affects the efficiency of the robot's cross-regional navigation and regional mapping in the home environment.

[0027] In order to solve the above technical problems, the present application discloses a robot mapping method based on gap recognition. The executor of the robot mapping method is a robot. In order to respond quickly and improve the mapping efficiency, the robot can use a monocular camera to acquire images, identify walls and gaps in the acquired wall images, and build a map while acquiring and identifying walls and gaps. Of course, choosing a binocular camera or a depth sensor to acquire and identify images can meet the corresponding recognition efficiency and partition mapping effect.

[0028] See also Figure 1 It can be seen that the robot mapping method includes: Step A, the robot walks at least one closed-loop path by walking along the wall and identifying the target gap, and encloses at least one circle of the wall chain with at least one target gap reserved in the search into the first partition; wherein, the robot walks along the wall and identifies the target gap in the way of walking on the premise that the robot first searches for the physical elements that constitute the wall, the physical elements can be selected from the image of the vertical wall corner line, and the vertical wall corner lines arranged and extended in a certain direction form a wall, so that the robot establishes the basis for walking along the wall; the robot's action of identifying the target gap in step A and the robot's walking along the wall in step A are performed simultaneously, if the robot When a person recognizes a target gap in the wall, a door frame is marked in the wall, and then walking along the marked door frame can be understood as the robot continuing to walk along the wall. In this way, walking along the wall and identifying target gaps can be completed continuously in the same room area without entering a new area through the target gap. Each closed-loop path corresponds to the collection and combination of a circle of wall chain (not necessarily a closed wall chain, but it will be composed of multiple links in series, each link can be regarded as a path parallel to the robot's walking along the wall, and also as the horizontal outline of the wall). At least one target gap will be identified in a combined wall chain.

[0029] In order to improve the efficiency of mapping and area division, when the robot walks a closed-loop path for the first time, it will enclose at least one circle of wall chains with at least one target gap in the search to form the first partition. In the scene of the actual planned room area, the robot will enclose the collected vertical wall corners and target gaps to form the first room area by following a circle. As the first partition divided by the robot in a target area, the map of the first partition is correspondingly constructed. At least one door frame is set on the boundary of the first partition, and there is only a relatively complete room map for the time being. In addition, the robot uses a monocular camera for visual recognition to quickly complete the first mapping.

[0030] Then, the robot enters the untraversed area from the first partition through the identified target gap, that is, based on the passability of the target gap (the target gap can at least accommodate the robot), after the robot walks along the wall contour for a circle of the first partition, it preferentially walks to an identified target gap, and then passes through the target gap to enter the untraversed area (unmapped area) from the first partition. It can be understood as entering another partition, and then walking in another partition according to a preset walking mode, and keeping identifying the target gap. The preset walking mode may not be a walking mode along the wall. A bow-shaped walking mode may be adopted in another partition to start a walking mode with a higher area coverage rate, and the target gap is still identified in the untraversed area. It is also allowed to superimpose the constructed map in this walking mode, especially to superimpose on the basis of the map of the first partition to implement incremental mapping; then execute step B.

[0031] Step B: Whenever the robot recognizes a new target gap, the robot does not pass through the new target gap, wherein the new target gap is set to be different from the target gap that has been recognized in the first partition or the untraversed area disclosed in step A; the robot continues to walk and identify target gaps according to the preset walking mode, keeps building and updating the map, for example, the robot regenerates the map every 5 minutes, continuously superimposes the real-time walking trajectory on the local map, and then updates it to the global map. Until the robot walks through a closed-loop area according to the preset walking mode, the closed-loop area is marked as a target partition, which is also regarded as a traversed area. It can be understood that the robot uses multiple target gaps to divide a target partition, and realizes dividing a target partition in the untraversed area disclosed in step A; at this time, the identified target gaps are respectively in various directions of the traversed area, and there is no need to repeatedly identify the target gaps, and the repeatedly walked areas and the walking trajectories in the areas can be superimposed and displayed on the global map; then the robot enters the untraversed area from the target partition through the identified new target gaps, and continues to plan new target partitions in the untraversed area. At this time, the untraversed area is in the untraversed area described in step A, and its coverage range is smaller than the untraversed area described in step A; then the robot continues to walk according to the preset walking mode, and keeps identifying the target gaps, and repeatedly walks through different untraversed areas through each target gap until the robot cannot identify the new target gap. At this time, the robot has identified all the target gaps, and correspondingly, the robot completes the area division, and completes the division of all target partitions in the target area.

[0032] The technical effect of the present application is that the robot executes step A and step B, and identifies the target gap while walking along the wall, so as to form the first partition and establish the corresponding map after completing the first circle, and then enter the untraversed area through the identified target gap on the basis of the first partition, and then continue to identify the target gap and walk a closed-loop area according to the preset walking mode to divide a target partition, thereby performing incremental mapping based on a single target partition, and recursively walking through each room area until no new target gap is identified, and then completing the global mapping and obtaining a complete map. There is no need to walk the entire working area in advance before starting the partitioning operation or mapping, and the operation efficiency and mapping efficiency are improved by identifying the target gaps partition by partition.

[0033] Compared with the prior art, the present application does not need to add any extra markers on the door. It only needs the gap features formed by the door frame on the wall to appear within the perception field of view of the robot of the present application to identify the target gap, and then locate the door frame and divide the area according to the door frame, thereby realizing door frame positioning and area division marking in indoor and outdoor environments through a relatively low computing power algorithm.

[0034] As an example, see Figure 3 As shown, it is a flow chart of the robot mapping method provided by the preferred embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0035] like Figure 3 As shown, the specific method of step A includes: Step A1, the robot searches for the wall, then walks along the wall and identifies the target gap, and constructs a map at the same time, specifically the collected wall image and target gap image, and extracts the position information of image features such as points and lines from them, marks them in the map, and realizes real-time mapping; when the robot identifies the target gap, it walks along the target gap. If it has walked past the target gap, it continues to walk along the wall to the next target gap, and walks along the wall in this way until the robot completes a closed-loop path, obtains at least one circle of wall chain with at least one target gap reserved, and surrounds the at least one circle of wall chain with at least one target gap reserved into the first partition, and sets the partition map according to the first partition, and then executes step A2; in step A1, each time the robot completes a closed-loop path, it obtains a circle of wall chain with at least one target gap reserved, and the robot sets the circle of wall chain with at least one target gap reserved as the outline of the first partition and reflects it in the map in real time, and updates it to the partition map.

[0036] In the step A1, the method for searching the wall by the robot includes: the robot collects the environment image, the tool for collecting the image by the robot is a monocular camera, and its viewing angle can at least cover the side walls of the wall, multiple vertical line segments similar to the shape of the door frame, and the door beam line; then the robot extracts the vertical projection lines of the wall in the environment image, and then identifies each batch of the combination of multiple vertical projection lines of the wall located in the same plane or the same curved surface as a wall to be searched. The combination of multiple vertical projection lines of the wall in the same plane or the same curved surface is that the multiple vertical projection lines of the wall correspond to mutually parallel vertical corner lines in the actual walking environment of the robot, so as to form a wall side by side. When the robot walks along the wall, each time it walks in the same direction, the plane or curved surface where the combination of multiple vertical projection lines of the wall located in the same plane or the same curved surface extracted in the same direction is located is regarded as a wall surface extending in one direction, that is, a wall.

[0037] In step A1, the robot can determine the boundary of the corresponding independent space area according to the combination of vertical projection lines of the wall, and the target gap is set on the boundary of the independent space area; the robot passing through the target gap can be understood as the robot crossing the boundary of the independent space area; thus, the boundaries of all independent space areas in the environment and the number of independent space areas can be obtained. The independent space area can refer to an independent room or an independent open space.

[0038] The robot builds a map while walking along the wall, which will reduce walking along the edge of isolated obstacles, mainly to increase speed. The robot is allowed to collide with the wall, and there is no specific limit on the distance between the robot's side and the wall.

[0039] Step A2, the target gap identified by the robot through step A1, specifically, after the robot identifies all the target gaps in the first partition and sets the partition map, it preferentially walks to a target gap identified by step A1, then walks out of the first partition, and enters the untraversed area through the target gap; then the robot walks according to the preset walking mode and keeps identifying the target gap, then the robot will walk according to the preset walking mode in the latest untraversed area, keep identifying the target gap, so as to identify the new target gap, and then determine the new partition connected to the new target gap; then execute step B.

[0040] Step A1 determines multiple vertical projection lines of the wall located in the same plane or the same curved surface, and by identifying the target gap, the relative position relationship between different batches of vertical projection lines of the wall arranged continuously can be obtained. Then step A2 uses the target gap as the boundary to effectively identify each independent space in the environment. It can be seen from the implementation method composed of the above steps A1 and A2 that this embodiment obtains the closed-loop path and the target gap, which helps to divide the first partition in the environment, that is, the first independent space; and determines the passable position information from the first independent space in the environment to the untraversed area based on the target gap. And it has good applicability for rooms with complex layouts.

[0041] On the basis of the above embodiment, after executing step A2, step B is executed; the specific method of step B includes: Step B1, determine whether the robot recognizes a new target gap, if so, execute step B2, otherwise execute step B4; wherein the new target gap is a gap recognized by the robot in an area outside the first partition, and the new target gap does not include the target gap recognized in step A1.

[0042] Step B2, the robot does not pass through the new target gap, but only marks the position of the new target gap in the map. The robot continues to walk according to the preset walking mode and keeps identifying the target gap until the robot completes a closed-loop area. The closed-loop area here is the total area that the robot walks according to the preset walking mode without passing through any target gap, and is surrounded by the wall boundary and the boundary of the target gap. The closed-loop area is then marked as a target partition, and the map information corresponding to the target partition is added to the partition map to form an incremental map; then step B3 is executed; in step B2, the robot walks in the target partition according to the preset walking mode, does not enter the new partition through the new target gap, and continues to walk and build a map in the original target partition. For example, when the robot walks on a bow-shaped path, it does not cross the target gap, and continues to walk the bow-shaped path in the original partition, so that the robot stays in a closed-loop area to complete the bow-shaped path and continuously identifies the updated target gap.

[0043] Step B3, the robot enters the untraversed area from the target partition described in step B2 through the new target gap identified by step B1; then the robot walks according to the preset walking mode and keeps identifying the target gap, and then executes step B1, so as to recursively identify the new target gap in the new partition, continuously add new partition maps and target gaps to the map, and allow the path positions that the robot has repeatedly walked to be superimposed and displayed.

[0044] Step B4: The robot completes the area division and sets the latest incremental map as the global map. At this time, the robot has not identified any new target gaps and has used all the identified target gaps to complete the division of all target partitions in the same map. Based on the fact that there are multiple room areas in the robot's indoor and outdoor working areas, multiple target gaps that can represent door frames are identified, and multiple target gaps are used as entrances and exits between the corresponding two target partitions (room areas). Then, by executing steps A1 and A2 and steps B1 to B4, multiple target gaps can be cumulatively determined and multiple target partitions can be divided.

[0045] In the process of recursively executing steps B1 to B3, the coverage of the incremental map is continuously expanded, and the number of marked target partitions and target gaps increases. When step B4 is executed, the latest incremental map has been updated to the global map, and an overall map of each target partition and the first partition that the robot has walked in a target area is established. Compared with the prior art that requires the robot to walk along the boundaries of each partition in advance and then build a map for each partition, steps A1 to A2 and steps B1 to B4 are for partition mapping and planning. Mapping is done while walking, and when a target gap is identified, the location leading to the untraversed area and the boundary position of different partitions are recorded in the map, thereby speeding up the efficiency of robot mapping and area division.

[0046] As an embodiment, in step A1, a flow chart of a method in which the robot walks a closed-loop path is as follows: Figure 4 As shown, specifically including: Step A101, when the robot recognizes the current wall, the robot walks along the current wall in a preset clockwise direction, specifically, walks along the current wall from the preset edge starting point in a preset clockwise direction, and at the same time determines whether the robot recognizes the target gap, if yes, executes step A102, otherwise executes step A103; the preset clockwise direction is clockwise or counterclockwise; for a wall, it can be divided into the current wall and the next wall according to the search time and search position. As for the method of defining a wall using the vertical projection line of the wall, please refer to the corresponding embodiment of the aforementioned step A1, which will not be repeated here. When executing step A101, after the robot recognizes the current wall, it first walks to the front of the current wall and maintains a certain edge distance with the current wall, thereby determining the preset edge starting point; the preset edge starting point can be located in the vertical direction of the current wall, and the distance from the preset edge starting point to the current wall is between 0.5cm and 10cm, which can represent the distance between the edge of one side of the robot close to the wall and the current wall.

[0047] Step A102, determine whether the robot walks back to the preset edge starting point used when executing step A101 for the first time. If so, determine that the robot has completed a closed-loop path. If the preset edge starting point is located at the end point of an identified target gap, the robot may repeatedly identify the same target gap at the preset edge starting point and be directly below one side of the target gap; otherwise, execute step A103.

[0048] Step A103, the robot walks along the boundary formed by the position of the target gap described in step A101, and keeps searching for the wall; then executes step A104; it can be understood that the two sides of the target gap are different walls (composed of different batches of vertical projection lines of the wall arranged continuously). The boundary formed by the position of the target gap is understood to be directly below the target gap, and the robot walks along the corresponding boundary but does not cross the target gap, while searching for the next wall.

[0049] Step A104, when the robot recognizes the next wall, the robot updates the next wall to the current wall, and when the robot walks the boundary formed by the position of the target gap described in step A101, the position of the next wall recognized by the robot is updated to the preset edge starting point, and then step A101 is executed to realize the recognition of the target gap between two adjacent walls, or search for the next wall each time the target gap is recognized and walked along, until the robot walks back to the preset edge starting point used when step A101 was executed for the first time, and the robot has completed a closed-loop path.

[0050] In this embodiment, when the robot completes a closed-loop path, the robot forms a wall chain with at least one target gap reserved in the preset clockwise direction with each wall and each target gap identified in sequence in steps A101 to A104; the projection lines of each wall and each target gap identified in sequence in steps A101 to A104 on the ground are enclosed into continuous line segments to form a complete closed loop.

[0051] In step A1, in order to search for walls and identify target gaps, the robot will collect images. In a frame of environmental images collected by the robot in real time, the target gap is between two adjacent walls; the target gap is surrounded by vertical projection lines of the walls on the opposite sides of the two adjacent walls, forming two vertical projection lines of the walls corresponding to the target gap; schematically, as shown in FIG. Figure 2 As shown, Figure 2 is a schematic diagram of the environment image collected by the robot in front of or directly below the door frame. The center point of the environment image currently collected by the robot's camera is point O, which can represent the current position of the robot. The straight line segment A1B1 is the vertical projection line of the wall on the left side of the robot extracted by the robot, and the straight line segment D1C1 is the vertical projection line of the wall on the right side of the robot extracted by the robot. Figure 2 The wall vertical projection line A1B1, wall vertical projection line C1D1 and door beam line B1C1 extracted from the image are connected to form a target gap, which may be a door frame existing in the actual environment.

[0052] The pixel distance between the two endpoints of the target gap is within the preset width threshold range, the endpoint close to the center of the environmental image in the vertical projection line A1B1 of the wall is endpoint B1, and the endpoint close to the center of the environmental image in the vertical projection line C1D1 of the wall is endpoint C1. The pixel distance between the two endpoints of the target gap can be the pixel distance between endpoint B1 and endpoint C1, the pixel distance between the two endpoints of the target gap can also be the pixel distance between endpoint B1 and endpoint C1, and the pixel distance between the two endpoints of the target gap can also be the pixel distance between endpoint A1 and endpoint D1; when the pixel distance between endpoint B1 and endpoint C1 is the lower limit value of the preset width threshold range, the pixel distance between endpoint A1 and endpoint D1 is the upper limit value of the preset width threshold range.

[0053] Preferably, the conventional minimum room width ranges from 60 cm to 100 cm, the physical distance range converted from the preset width threshold range is 60 cm to 100 cm, the physical distance between endpoint A1 and endpoint D1 is 100 cm (the value will become smaller when converted to the pixel length occupied in the image), and the physical distance between endpoint B1 and endpoint C1 is 60 cm (the value will become smaller when converted to the pixel length occupied in the image).

[0054] The pixel distances between the intersection of the extension lines of the two vertical projection lines of the wall corresponding to the target gap and the nearest endpoints of the two vertical projection lines of the wall corresponding to the target gap are both within the preset projection distance threshold range; Figure 2 In the figure, the extension line of the vertical projection line A1B1 of the wall intersects with the extension line of the vertical projection line C1D1 of the wall. Preferably, the intersection point of the extension line of the vertical projection line A1B1 of the wall and the extension line of the vertical projection line C1D1 is point O. The pixel distance between point O and the nearest endpoint B1 of the vertical projection line A1B1 of the wall is equal to the length of the line segment OB1. The pixel distance between point O and the nearest endpoint C1 of the vertical projection line C1D1 of the wall is equal to the length of the line segment OC1. The physical distance range converted from the preset projection distance threshold range is preferably 0 to 0.2 m.

[0055] It is known to technicians in the field of machine vision that the pixel distance (the coordinate distance between two pixel points) in an image is converted into the physical distance in the actual environment, and the conversion coefficient involved is determined by the camera parameters, including the focal length, field of view angle, sensor size and other parameters of the camera. Based on this, converting the pixel distance into the physical distance according to the focal length and pixel size of the camera is a conversion method that technicians in the field of machine vision can master.

[0056] In the present application, the target gap is used to connect the target partitions marked by each of the two adjacent executions of step B2, or to connect the first partition and the target partition marked by the first execution of step B2. If the first partition and the target partition are both room areas, the target gap is represented as a door frame. Therefore, the preset width threshold range can allow the robot body to pass through, and correspondingly exerts a constraint on the preset projection distance threshold range. In summary, the positional relationship between the two vertical projection lines of the wall corresponding to the target gap is determined.

[0057] Based on the aforementioned embodiment, after the robot completes the area division in step B4, it obtains the position information of each wall of different target partitions and the position information of the target gap, which are all recorded in the global map. The robot can then use the position information of each wall and the position information of the target gap marked in the global map to perform path planning and locate the target gap. Then the robot walks along each wall in turn and passes through each target gap in turn to complete the vertical projection lines of each wall or the wall that makes up the wall. When completing the vertical projection lines of the wall, the robot can quickly move along the wall throughout the house, which plays a role in checking for gaps and filling gaps and perfecting the boundaries of the target partitions.

[0058] It can be understood that the robot, under user control or in automatic mode, identifies the environmental space in different target partitions, and the target partition may refer to an indoor room, an outdoor room, or a combination of an indoor room and an outdoor room.

[0059] It should be noted that when the robot detects that it has walked to the location of the target gap marked in the map, and detects that the distance between the location of the target gap and the target gap is less than or equal to a preset value, the robot is controlled to walk forward in the direction of the target gap; when the robot detects that the distance between it and the target gap is equal to zero, the robot is controlled to continue walking forward and enter other rooms.

[0060] As an embodiment, the method for the robot to identify a target gap includes: The robot collects an environmental image and extracts multiple straight line segments from the environmental image; preferably, the robot can identify the straight line segments in the image through various edge detection algorithms, such as vertical corner lines and horizontal corner lines, and respectively extract reference vertical projection lines and reference horizontal projection lines that form a cross relationship in the environmental image to represent them, wherein the extension lines of the two reference vertical projection lines intersect, so that the two reference vertical projection lines form a cross relationship; in the environment in which the robot actually walks, the vertical corner line is used to represent the line segment perpendicular to the ground of the indoor and outdoor room area, the vertical corner line and the horizontal corner line are perpendicular to each other, but the reference vertical projection line is not necessarily perpendicular to the horizontal corner line.

[0061] Generally, after the robot can identify the straight line segments in the image through various edge detection algorithms, the robot will calculate the angle of each straight line segment from the environmental image (the environmental image can be binarized in advance) through Hough transform (which can be obtained based on the slope of the equation of the extracted straight line segment). The robot then selects straight line segments that can represent the doorpost line, door beam line, side wall line, etc. based on the length and angle of the straight line segment.

[0062] In this embodiment, in order to allow the camera's field of view to fully cover the door frame, the robot's camera lens is installed facing the ceiling, so the camera's field of view can cover the ceiling above the robot and the area below the ceiling. If the doorpost vertical to the ground and the camera are oriented in the same direction, then based on the perspective principle of the lens, the extension lines of the two reference vertical projection lines in the environmental image will eventually intersect.

[0063] According to the length characteristics, angle characteristics and quantity characteristics of the line segments, two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship are screened out from the multiple straight line segments; thereby, the robot can exclude the reference vertical projection lines and reference horizontal projection lines that cannot form an intersection relationship based on the three judgment quantities of the length characteristics of a single line segment, the angle characteristics between two line segments and the quantity characteristics of the cumulatively extracted straight line segments, thereby reducing the number of straight line segment samples required for screening the wall vertical projection lines and door beam lines.

[0064] According to the preset door frame size condition, two wall vertical projection lines and one door beam line for forming the door frame are selected from the two reference vertical projection lines and one reference horizontal projection line that have been selected to form a cross relationship, and it is determined that the two wall vertical projection lines and one door beam line for forming the door frame surround the target gap, that is, the two currently selected wall vertical projection lines and one door beam line intersect and form a gap in the direction toward the ground. The preset door frame size condition is a further refinement of the aforementioned line segment length feature, which improves the accuracy of the robot in identifying the door frame.

[0065] Since the two vertical wall projection lines and one door beam line used to form the door frame are a combination of the projection line segments of the actual door frame in the environment image, the two vertical wall projection lines used to form the door frame have an intersection relationship, and the position relationship between the intersection of the extension lines of the two vertical wall projection lines used to form the door frame and the center of the door beam line can represent the distribution direction, azimuth angle and other information formed by the door frame (or the target gap) relative to the current position of the robot. When the robot determines the center of the door beam line, the door frame is positioned.

[0066] In some embodiments, the endpoints of the two vertical wall projection lines used to form the door frame that are close to the center of the environmental image in which they are located are set as corner points, and then the corner points are connected to form a reference door beam line. The reference door beam line can be parallel to the reference horizontal projection line (i.e., the door beam line) directly extracted from the environmental image.

[0067] As an embodiment, the method of selecting two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship from the plurality of straight line segments according to the line segment length feature, angle feature and quantity feature is as follows: Figure 5 As shown, specifically including: Step A111, filter out the straight line segments perpendicular to the ground from the multiple straight line segments, where the definition of vertical is the straight line segments perpendicular to the ground of the robot's actual walking environment (including vertical wall corner lines) displayed / projected in the environment image; then mark the filtered straight line segments perpendicular to the ground as reference vertical projection lines; and then execute step A112.

[0068] Step A112, judging whether the number of reference vertical projection lines marked in step A111 is less than the value 2, if yes, determining that the robot cannot select two wall vertical projection lines from the environment image, causing the robot to suspend the execution of the robot mapping method; otherwise, executing step A113, thereby eliminating invalid reference vertical projection lines.

[0069] Step A113, within the preset projection distance threshold range corresponding to the center of the environmental image, filter out the straight line segments whose lengths are within the preset width threshold range, and then mark the filtered straight line segments whose lengths are within the preset width threshold range as reference horizontal projection lines; then execute step A114; since the door beam line is generally set above the door frame and sandwiched between two reference vertical projection lines, the door beam line is projected into the environmental image close to the center of the environmental image. Therefore, a preset projection distance threshold range is set in the center of the environmental image to search for a reference horizontal projection line that can represent the door beam line at a close distance.

[0070] Step A114, determine whether the number of reference horizontal projection lines marked in step A113 is less than the value one. If so, determine that the robot cannot filter out the door beam line from the environment image, and suspend the execution of the robot mapping method; otherwise, execute step A115; thereby excluding invalid reference horizontal projection lines.

[0071] Step A115, filter out reference horizontal projection lines that meet the door structure by enumerating reference vertical projection lines and reference horizontal projection lines, and then execute step A116; wherein, the reference horizontal projection lines that meet the door structure are reference horizontal projection lines that form an intersection relationship with two reference vertical projection lines; therefore, it can be understood that there are at least one reference horizontal projection line and at least two reference vertical projection lines among the reference horizontal projection lines and reference vertical projection lines that meet the door structure, and there are two reference vertical projection lines that intersect with both ends of a reference horizontal projection line respectively, and in addition, the extension lines of the two reference vertical projection lines intersect.

[0072] Step A116, determine whether the number of reference horizontal projection lines that meet the door structure screened out in step A115 is greater than the value one, if so, execute step A117, otherwise determine that the robot cannot screen out the door beam line from the environment image, so that the robot suspends the execution of the robot mapping method, thereby excluding invalid reference horizontal projection lines that meet the door structure.

[0073] Therefore, when executing step A117, there may be multiple door beam lines extending in the same direction, and the position occupied by the door beam line is subsequently determined by the connection node of two reference vertical projection lines connected to the reference horizontal projection line.

[0074] Step A117, obtain multiple groups of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship; then in each group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship, mark the two reference vertical projection lines as two wall vertical projection lines, and mark the one reference horizontal projection line as a door beam line; wherein the extension lines of the two reference vertical projection lines intersect, and the two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship are connected to each other as a gap.

[0075] Step A117 is equivalent to determining that there are two wall vertical projection lines and a door beam line that form an intersection relationship among the reference horizontal projection line and the reference vertical projection line that conform to the door structure. Based on step A113, within the same frame of the environment image, the pixel distance between the intersection point of the extended lines of the two wall vertical projection lines determined in step A117 and the center of the environment image is within the preset projection distance threshold range.

[0076] In summary, two vertical wall projection lines and one door beam line are determined as candidate door frame line segments by executing steps A111 to A117. Then, the robot determines the direction information of the door frame where the door beam line is located relative to the current position of the robot based on the positional relationship between the center point of each door beam line and the intersection of the extension lines of the two vertical wall projection lines, which is also equivalent to the direction information of the door beam line relative to the coordinates of the current position of the robot in the environment image.

[0077] In some embodiments, among the two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship, there is: among the endpoints of each reference vertical projection line, the endpoint closest to the intersection of the extension lines of the two reference vertical projection lines is set as a corner point, and then the corner points of each reference vertical projection line are connected to form the one reference horizontal projection line, so as to form a reference horizontal projection line that conforms to the door structure.

[0078] It should be noted that there is usually a part of the wall above the door. To support this part of the wall, a prefabricated wooden or reinforced concrete beam is needed like a bridge. This beam is called a door beam. It can reduce the impact of the wall sinking on the door frame. The door beam can be processed by the image recognition algorithm in the environmental image of this embodiment to be represented by the reference horizontal projection line; the two sides below the door beam are two door posts perpendicular to the ground, which are also processed by the image recognition algorithm in the environmental image of this embodiment to be represented by the reference vertical projection line. Then, two adjacent reference vertical projection lines and a door beam line can form a door frame, forming a simplified model of a door.

[0079] As an embodiment, the method of selecting two wall vertical projection lines and one door beam line for forming a door frame from two reference vertical projection lines and one reference horizontal projection line that have been selected to form a cross relationship according to a preset door frame size condition is as follows: Figure 6 As shown, specifically including: Step A118, mark a group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship determined in step A117 as a group of candidate door frame lines, so that there are two wall vertical projection lines and one door beam line that form an intersection relationship in a group of candidate door frame lines; then execute step A119.

[0080] Step A119, determine whether the physical distance between the vertical corner lines corresponding to two reference vertical projection lines in a group of candidate door frame lines is within the preset door width distance range. If so, execute step A120, otherwise, the two vertical wall projection lines used to form the door frame cannot be screened out to exclude the two reference vertical projection lines with inappropriate spacing; wherein, the robot uses a depth sensor to detect the physical distance between the vertical corner lines corresponding to the two reference vertical projection lines; the reference vertical projection line is a projection line formed by the robot collecting the vertical corner lines into the coordinate system of the environment image, and the vertical corner line is perpendicular to the ground where the robot walks.

[0081] In the walking environment of the robot, the minimum room width ranges from 60 cm to 100 cm, and correspondingly, the preset door width distance range is preferably 60 cm to 100 cm. The detection angle of the robot using the depth sensor also needs to cover the area between the doorpost lines on both sides of the door frame.

[0082] Step A120, calculate the ratio between the length of a reference horizontal projection line in the set of candidate door frame lines and the physical distance, and then convert the length of the vertical corner line corresponding to each reference vertical projection line based on the ratio and the length of each reference vertical projection line, so that the reference vertical projection line is converted from pixel size to physical size and represented by the length of the vertical corner line. This ratio is also applicable to the physical size conversion operation of the reference horizontal projection line in the same frame of environmental image. Then execute step A121. The length of the reference vertical projection line belongs to the pixel length occupied by the reference vertical projection line in the environmental image. After being converted into the length of the vertical corner line, the actual height of the door frame is obtained so as to judge the actual door frame in step A1.

[0083] Step A121, determine whether the length of the vertical wall corner line corresponding to each reference vertical projection line in a group of candidate door frame lines is within the preset door frame height range, if so, mark two reference vertical projection lines in a group of candidate door frame lines as two wall vertical projection lines used to form the door frame, and mark a reference horizontal projection line in a group of candidate door frame lines as a door beam line used to form the door frame, and then the door frame can be determined, wherein the target gap includes the door frame, that is, the door frame and the straight line segments composed of it (which can correspond to the projection line segments in the environment image) are identified in the wall by executing step A121; otherwise, the two wall vertical projection lines used to form the door frame cannot be screened out to exclude the wall vertical projection lines with unreasonable heights; wherein, the preset door frame height range is set to 1.6m to 2.5m, which meets the door frame height characteristics in the room.

[0084] In summary, this embodiment converts the vertical projection lines from pixel size to physical size by executing steps A118 to A121, and obtains two vertical wall projection lines and one door beam line used to constitute the door frame based on the physical size, eliminates the misjudgment interference of the door leaves and side wall lines perpendicular to the ground, constructs a simplified model of the door frame, obtains the vertical wall projection lines and door beam lines that conform to the simplified model of the aforementioned door frame, and correspondingly identifies a complete and accurate door frame in the environmental image.

[0085] It should be noted that the vertical projection line of the wall in this embodiment is a simplified projection line, and the thickness of the wall is not taken into account.

[0086] Of course, objects that conform to the simplified model of the aforementioned door frame, including the vertical projection lines of the wall and the door beam lines, also include square columns, side door leaves, door panels, walls between the ceiling and the door frame, door sills, side walls and front walls in a home environment.

[0087] In an embodiment of two vertical wall projection lines and one door beam line that can form a door frame, when two reference vertical projection lines in a group of candidate door frame lines are marked by step A121 as two vertical wall projection lines for forming a door frame, and one reference horizontal projection line in a group of candidate door frame lines is marked by step A121 as one door beam line for forming a door frame, there are: intersection marks formed by the two vertical wall projection lines marked by the aforementioned step A121 in a group of candidate door frame lines and the one door beam line marked by step A121 in the same group of candidate door frame lines, respectively. are corner points, and two corner points are obtained; the angle formed by the two corner points and the center of the environmental image is greater than a preset angle threshold, and the preset angle threshold is preferably 20 degrees; in step A121, the length of a door beam line marked in the same group of candidate door frame lines is greater than a preset door beam projection length, and the preset door beam projection length is preferably 35 pixels in length, and the extension lines of two reference vertical projection lines in the same group of candidate door frame lines intersect, thereby reflecting the positional relationship between the two wall vertical projection lines in the environmental image, and completing the simplified door beam model on the angle feature and the door beam line length feature respectively.

[0088] Indicatively, see Figure 2 It can be seen that Figure 2 It is a schematic diagram of the environmental image collected by the robot in front of the door frame. The center of the environmental image currently collected by the robot's camera is located near point O, and point O is the intersection of the extension lines of the two vertical projection lines of the wall. The pixel distance between point O and the center of the environmental image is within the preset projection distance threshold range. The preset projection distance threshold range is used to represent the error range of the vertical wall corner line converted to the pixel coordinate system. The error source includes the deformation and distortion of the image caused by the lens distortion of the camera.

[0089] In some ideal cases, point O is the center of the environment image, and the angle formed by the illustrated corner point B1 and corner point C1 and the center O of the environment image is angle B1OC1, and angle B1OC1 is greater than the preset angle threshold.

[0090] The straight line segment A1B1 is the vertical projection line of the wall on one side of the robot extracted by the robot, and the straight line segment D1C1 is the vertical projection line of the wall on the other side of the robot extracted by the robot. Then, the endpoint B1 close to point O is selected from the two endpoints of the vertical projection line A1B1 of the wall as a corner point, and the endpoint C1 close to point O is selected from the two endpoints of the vertical projection line D1C1 of the wall as another corner point. Then, the corner points B1 and C1 are connected to form a line segment B1C1. Then, the line segment B1C1 forms the door beam line, and then Figure 2 The wall vertical projection line A1B1, the wall vertical projection line C1D1 and the door beam line B1C1 extracted from the image processing apparatus are connected to form the target gap to represent the door frame existing in the actual environment.

[0091] In some embodiments, the current position of the robot is represented by point O, which is the center of the environment image; then the robot sets the direction from the center of the currently collected environment image to the center of the door beam line as the distribution direction of the door frame relative to the current position of the robot, corresponding to Figure 2 In the figure, the center of the door beam line is the midpoint H1 of the straight line segment B1C1, and the direction from point O to point H1 is the distribution direction of the door frame where the door beam line B1C1 is located relative to the current position of the robot (specifically, its pixel coordinate position in the image). It can also be understood that the robot is located in the direction of H1O of the door frame where the door beam line B1C1 is located, and the robot is regarded as walking along the direction OH1 to the front of the door frame where the door beam line B1C1 is located.

[0092] In some embodiments, the robot sets the angle between the direction from one corner point to another corner point and the coordinate axis of the image coordinate system, or the angle converted from the angle into the world coordinate system, as the angle information of the door beam line, so as to convert it into the orientation information of the door beam line in the outdoor room area; the direction from one corner point to another corner point is an extension direction of the door beam line in the image coordinate system, which is schematically represented as Figure 2 Corner point B1 points to the direction of corner point C1. Specifically, the center of the door beam line can be converted into the world coordinate system to realize the conversion of the angle information of the door beam line into the world coordinate system, that is, it falls into the grid map and is positioned with the help of the world coordinate system, which is suitable for the robot to use in the navigation movement process.

[0093] Based on the aforementioned embodiments, the present application discloses a robot, the robot is equipped with a visual sensor, and the visual sensor is generally installed on the top of the robot body to cover a larger viewing space. The robot is used to execute the robot mapping method mentioned in the aforementioned embodiments, including executing steps A and B to realize both the identification of target gaps and the division of target partitions; for step A, the robot specifically executes steps A1 and A2 to divide the first partition in the environment; for step A1, the robot specifically executes steps A101 to A104, so that the robot walks along the wall to complete a closed-loop path and forms a circle of wall chain with at least one target gap reserved, then the at least one circle of wall chain with at least one target gap reserved can be surrounded into the first partition; for step A1, the robot also determines two vertical wall projection lines and a door beam line as candidate door frame constituent line segments by executing steps A111 to A117; for step A1, the robot also converts the vertical projection lines from pixel size to physical size by executing steps A118 to A121, and obtains two vertical wall projection lines and a door beam line for constituting the door frame based on the physical size, eliminates the misjudgment interference of the door leaf and side wall lines perpendicular to the ground, and constructs a simplified model of the door frame. Based on the simplified model of the door frame, room areas close to the actual environment are divided in the indoor working area, thereby improving the rationality of the robot's partitioning and achieving faster room cleaning.

[0094] The program code corresponding to the robot mapping method disclosed in the above embodiment is stored in the robot's built-in controller or memory. When the robot calls the program code, the robot is configured to execute the robot mapping method disclosed in the above embodiment. Compared with the prior art, no laser sensor is installed, which saves sensor costs and ensures the mapping efficiency of the door frames of different room areas when the robot passes through.

[0095] As an embodiment, the visual sensor uses a camera. In order to respond quickly and improve the efficiency of mapping, the robot can use a monocular camera for image acquisition and identify walls and gaps in the acquired wall image; the lens of the robot's camera is installed toward the ceiling of the indoor room where the robot is located, so that the camera's field of view covers the surface of the ceiling and the space below it, wherein a door frame is set in the space below the ceiling as the entrance and exit of different room areas. In this embodiment, the camera is set at the front end of the robot body, and the direction of the robot's camera is set to be horizontal and obliquely upward. The optical axis of the camera can be close to 90 degrees with the horizontal plane, and the environmental image acquired by the camera can at least cover the area between the wall where the door frame or the door beam of the door frame is located and the ceiling. In this embodiment, a single camera can be set, and multi-angle camera can be maintained during the execution of a single room area task.

[0096] Due to the passability of the door, the same door frame can be detected in front, behind, and below the door frame, but the square pillars and door leaves in the home environment can only be detected by the robot in a single direction, and the passability of the door allows the robot's upward camera to detect the same door frame in three viewing angles: in front, behind, and directly below. When the above embodiment uses the target gap to represent the door frame, the robot selects two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship from multiple straight line segments extracted from the environment image according to the line segment length feature, angle feature, and quantity feature. Shadow line; then according to the preset door frame size conditions, two wall vertical projection lines and one door beam line for forming the door frame are screened out from the two reference vertical projection lines and one reference horizontal projection line that have been screened out to form a cross relationship, and it is determined that the two wall vertical projection lines and one door beam line for forming the door frame enclose the target gap, that is, they are connected to form the door frame, so as to further eliminate the influence of multiple vertical line segments similar to the shape of the door frame formed on the boundary between the front wall and the side wall of the wall of the indoor working area, such as the misjudgment caused by the characteristic lines of the square columns and the characteristic lines of the door leaves, thereby improving the accuracy of door frame recognition.

[0097] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A robot mapping method based on gap recognition, characterized in that: include: Step A, the robot walks at least one closed-loop path by walking along the wall and identifying the target gap, and encloses the searched at least one circle of wall chain with at least one target gap reserved into the first partition; then the robot enters the untraversed area from the first partition through the identified target gap, and then walks according to the preset walking mode, and keeps identifying the target gap; then executes step B; Step B: Whenever the robot recognizes a new target gap, the robot does not pass through the new target gap, but continues to walk according to the preset walking mode and identify the target gap until the robot walks through a closed-loop area according to the preset walking mode, marks the closed-loop area as a target partition, and then the robot passes through the identified new target gap to enter the untraversed area from the target partition, and then walks according to the preset walking mode and continues to identify the target gap until the robot cannot recognize the new target gap and completes the area division.

2. The robot mapping method according to claim 1, characterized in that: The specific method of step A includes: Step A1, the robot searches for a wall, then walks along the wall and identifies a target gap, while building a map, and walks along the target gap when identifying the target gap, until the robot walks through at least one closed-loop path, obtains at least one circle of wall chain with at least one target gap reserved, and encloses the at least one circle of wall chain with at least one target gap reserved into a first partition, and sets a partition map according to the first partition; then executes step A2; Step A2: The robot enters the untraversed area from the first partition through the target gap identified in step A1; then the robot walks according to the preset walking mode and keeps identifying the target gap; then executes step B.

3. The robot mapping method according to claim 2, characterized in that: The specific method of step B includes: Step B1, determine whether the robot recognizes a new target gap, if yes, execute step B2, otherwise execute step B4; Step B2: The robot does not pass through the new target gap, but continues to walk according to the preset walking mode, while maintaining the target gap recognition, until the robot walks through a closed-loop area, then marks the closed-loop area as a target partition, and adds the map information corresponding to the target partition to the partition map to form an incremental map; then execute step B3; Step B3, the robot enters the untraversed area from the target partition described in step B2 through the new target gap identified in step B1; then the robot walks according to the preset walking mode and keeps identifying the target gap; then executes step B1; Step B4: The robot completes the area division and sets the latest incremental map as the global map.

4. The robot mapping method according to claim 2, characterized in that: In step A1, the method for the robot to search for a wall includes: The robot collects environmental images, extracts the vertical projection lines of the walls from the environmental images, and then identifies each combination of multiple vertical projection lines of the walls located in the same plane or the same curved surface as a wall to be searched.

5. The robot mapping method according to claim 4, characterized in that: In step A1, the method for the robot to walk a closed-loop path includes: Step A101, when the robot recognizes the current wall, the robot walks along the current wall in a preset clockwise direction, and at the same time determines whether the robot recognizes the target gap, if yes, executes step A102, otherwise executes step A103; Step A102, determining whether the robot has walked back to the preset edge starting point, if yes, determining that the robot has walked a closed-loop path, otherwise, executing step A103; Step A103, the robot walks along the boundary formed by the location of the target gap described in step A101 and keeps searching for the wall; then executing step A104; Step A104: When the robot recognizes the next wall, the robot updates the next wall to the current wall, and then executes step A101; When the robot completes a closed-loop path, the robot forms a wall chain with at least one target gap reserved in the preset clockwise direction according to the walls and target gaps identified in sequence in steps A101 to A104.

6. The robot mapping method according to claim 5, characterized in that: In a frame of environment image collected by the robot in real time, the target gap is between two adjacent walls; the target gap is surrounded by vertical projection lines of the wall on the opposite side of the two adjacent walls, forming two vertical projection lines of the wall corresponding to the target gap; The pixel distance between the two endpoints of the target gap is within a preset width threshold range, and the pixel distances between the intersection of the extended lines of the two vertical projection lines of the wall corresponding to the target gap and the nearest endpoints of the two vertical projection lines of the wall corresponding to the target gap are both within a preset projection distance threshold range; The target gap is used to connect the target partitions marked by the two adjacent executions of step B2, or to connect the first partition and the target partition marked by the first execution of step B2.

7. The robot mapping method according to claim 4, characterized in that: After the robot completes the area division in step B4, it obtains the position information of each wall of different target partitions and the position information of the target gap, and then the robot walks along each wall in turn and passes through each target gap in turn.

8. The robot mapping method according to claim 1, characterized in that: The robot's methods for target gap identification include: The robot collects an environment image and extracts a plurality of straight line segments from the environment image; According to the line segment length characteristics, angle characteristics and quantity characteristics, two reference vertical projection lines and one reference horizontal projection line forming an intersection relationship are selected from the plurality of straight line segments; According to the preset door frame size conditions, two wall vertical projection lines and one door beam line for forming a door frame are screened out from the two reference vertical projection lines and one reference horizontal projection line that have been screened out to form a cross relationship, and it is determined that the two wall vertical projection lines and one door beam line for forming the door frame enclose the target gap.

9. The robot mapping method according to claim 8, characterized in that: The method of selecting two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship from the plurality of straight line segments according to the line segment length feature, angle feature and quantity feature comprises: Step A111, selecting straight line segments perpendicular to the ground from the plurality of straight line segments; then selecting straight line segments whose length is greater than a preset wall height threshold from the selected straight line segments perpendicular to the ground and marking them as reference vertical projection lines; then executing step A112; Step A112, determining whether the number of reference vertical projection lines marked in step A111 is less than a value of two, if yes, determining that the robot cannot filter out two wall vertical projection lines from the environment image, otherwise executing step A113; Step A113, within the preset projection distance threshold range corresponding to the center of the environment image, filter out the straight line segments whose lengths are within the preset width threshold range, and then mark the filtered straight line segments whose lengths are within the preset width threshold range as reference horizontal projection lines; then execute step A114; Step A114, determining whether the number of reference horizontal projection lines marked in step A113 is less than a value of one, if so, determining that the robot cannot filter out the door beam line from the environment image, otherwise executing step A115; Step A115, screening out a reference horizontal projection line that meets the door structure by enumerating the reference vertical projection lines and the reference horizontal projection lines, and then executing step A116; wherein the reference horizontal projection line that meets the door structure is a reference horizontal projection line that forms an intersection relationship with the two reference vertical projection lines; Step A116, determining whether the number of reference horizontal projection lines that meet the door structure selected in step A115 is greater than a value of one, if yes, executing step A117, otherwise determining that the robot cannot select the door beam line from the environment image; Step A117, obtain multiple groups of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship; then, in each group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship, mark the two reference vertical projection lines as two wall vertical projection lines, and mark the one reference horizontal projection line as a door beam line; wherein the extension lines of the two reference vertical projection lines intersect.

10. The robot mapping method according to claim 9, characterized in that: Among the two reference vertical projection lines and one reference horizontal projection line forming the intersection relationship, there are: Among the endpoints of each reference vertical projection line, the endpoint closest to the intersection of the extension lines of the two reference vertical projection lines is set as a corner point, and then the corner points of each reference vertical projection line are connected to form the reference horizontal projection line.

11. The robot mapping method according to claim 9, characterized in that: The method of selecting two wall vertical projection lines and one door beam line for forming a door frame from two reference vertical projection lines and one reference horizontal projection line that have been selected to form a cross relationship according to a preset door frame size condition comprises: Step A118, marking a group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship determined in step A117 as a group of candidate door frame lines; then executing step A119; Step A119, determining whether the physical distance between the vertical corner lines corresponding to the two reference vertical projection lines in a set of candidate door frame lines is within the preset door width distance range, if yes, executing step A120, otherwise, the two vertical wall projection lines for forming the door frame cannot be screened out; wherein the robot uses a depth sensor to detect the physical distance between the vertical corner lines corresponding to the two reference vertical projection lines; Step A120, calculating the ratio between the length of a reference horizontal projection line in the set of candidate door frame lines and the physical distance, and then converting the length of the vertical wall corner line corresponding to each reference vertical projection line based on the ratio and the length of each reference vertical projection line, and then executing step A121; Step A121, determining whether the lengths of the vertical wall corner lines corresponding to each reference vertical projection line in a set of candidate door frame lines are all within a preset door frame height range, if so, marking two reference vertical projection lines in a set of candidate door frame lines as two wall vertical projection lines for forming a door frame, and marking a reference horizontal projection line in a set of candidate door frame lines as a door beam line for forming a door frame; otherwise, no two wall vertical projection lines for forming a door frame can be screened out; wherein, the target gap includes the door frame; The reference vertical projection line is a projection line formed when the robot collects the vertical wall corner line into the coordinate system where the environment image is located, and the vertical wall corner line is perpendicular to the ground where the robot walks.

12. The robot mapping method according to claim 11, characterized in that: Step A121 marks the intersections formed by the two vertical projection lines of the wall marked in a group of candidate door frame lines and the one door beam line marked in the same group of candidate door frame lines in step A121 as corner points, and obtains two corner points; the angle formed by the two corner points and the center of the environment image is greater than a preset angle threshold; Among them, the length of each door beam line marked in the same set of candidate door frame lines in step A121 is greater than the preset door beam projection length.

13. A robot equipped with a visual sensor, characterized in that: The robot is used to execute the robot mapping method described in any one of claims 1 to 12.

14. The robot according to claim 13, characterized in that: The visual sensor uses a camera; The camera lens of the robot is installed toward the ceiling of the indoor room area where the robot is located, so that the viewing angle of the camera covers the surface of the ceiling and the space below it.

Images