Method for recognizing door frame by robot and robot

Abstract

The invention discloses a method for recognizing a door frame through a robot and the robot. The method for recognizing the door frame through the robot comprises the steps that the robot collects an environment image and extracts multiple straight line segments from the environment image; screening out two reference vertical projection lines and one reference horizontal projection line which form a cross relation from the plurality of straight line segments according to length features, angle features and quantity features of the line segments; and according to a preset door frame size condition, screening out two wall surface vertical projection lines and a door beam line used for forming the door frame from the screened two reference vertical projection lines and the screened one reference horizontal projection line which form the cross relationship, and determining that the robot identifies the door frame.

Description

Technical Field

[0001] The present application relates to the technical field of visual recognition, and in particular to a method for a robot to recognize a door frame and a robot. Background Art

[0002] In a door frame positioning method based on image recognition disclosed in the Chinese invention patent with application number 202211023208.6, in order to divide the indoor working area into multiple room areas, it is necessary to first identify the location of the door frame to form a boundary line to divide the area. Identifying the location of the door frame requires the robot to walk through the indoor working area according to the preset planned path, and select a specific location as the matching center for template matching, and then calculate the positioning evaluation score for the matching result, and then locate the center point of the door beam based on the matching conditions satisfied by the positioning evaluation score. Since it involves matching calculations between multiple points and lines, the robot identifies the door at the location of the reference segmentation boundary line through a pre-constructed map, and still consumes relatively more computing resources to calculate the door frame posture. Summary of the invention

[0003] The present application discloses a method and a robot for identifying door frames, and the specific technical solution is as follows: A method for robot recognition of door frames, comprising: the robot collects an environmental image and extracts a plurality of straight line segments from the environmental image; based on line segment length features, angle features and quantity features, two reference vertical projection lines and one reference horizontal projection line forming a cross relationship are screened out from the plurality of straight line segments; based on a preset door frame size condition, two wall vertical projection lines and one door beam line for forming a door frame are screened out from the screened two reference vertical projection lines and one reference horizontal projection line forming a cross relationship, and it is determined that the robot recognizes the door frame. The present application excludes the reference vertical projection lines and reference horizontal projection lines that cannot form an intersection relationship from three judgment quantities, namely the length feature of a single line segment, the angle feature between two line segments, and the number feature of the cumulatively extracted straight line segments, and continues to use the preset door frame size conditions for screening to reduce the number of straight line segments required for processing the wall vertical projection lines and door beam lines, thereby improving the efficiency of the robot in identifying door frames; and by screening the two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship, the influence of multiple vertical line segments similar to the shape of a door frame formed on the boundary between the front wall and the side walls of the indoor working area wall, such as the characteristic lines of square columns and the characteristic lines of door leaves, is eliminated, thereby improving the accuracy of the robot in identifying door frames.

[0004] Furthermore, the method for selecting two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship from the plurality of straight line segments according to the line segment length characteristics, angle characteristics and quantity characteristics comprises: step A111, selecting straight line segments perpendicular to the ground from the plurality of straight line segments; then, among the selected straight line segments perpendicular to the ground, selecting straight line segments whose lengths are greater than a preset wall height threshold and marking them as reference vertical projection lines; then executing step A112; step A112, determining whether the number of reference vertical projection lines marked in step A111 is less than a value of two, If yes, it is determined that the robot cannot filter out two vertical projection lines of the wall from the environment image, otherwise, execute step A113; step A113, within the preset projection distance threshold range corresponding to the center of the environment image, filter out straight line segments whose lengths are within the preset width threshold range, and then mark the filtered straight line segments whose lengths are within the preset width threshold range as reference horizontal projection lines; then execute step A114; step A114, determine whether the number of reference horizontal projection lines marked in step A113 is less than the value one, if yes, it is determined that the robot cannot filter out the door beam line from the environment image, otherwise, execute step A115; step A115, filter out reference horizontal projection lines that meet the door structure by enumerating reference vertical projection lines and reference horizontal projection lines, and then execute step A116; wherein, the reference horizontal projection line that meets the door structure is formed with the two reference vertical projection lines. Reference horizontal projection lines of a crossing relationship; step A116, judging whether the number of reference horizontal projection lines that meet the door structure screened out in step A115 is greater than a value of one, if so, executing step A117, otherwise determining that the robot cannot screen out the door beam line from the environmental image; step A117, obtaining multiple groups of two reference vertical projection lines and one reference horizontal projection line that form a crossing relationship; then in each group of two reference vertical projection lines and one reference horizontal projection line that form a crossing relationship, marking the two reference vertical projection lines as two wall vertical projection lines, and marking the one reference horizontal projection line as a door beam line; wherein the extension lines of the two reference vertical projection lines intersect; in the same frame of the environmental image, the distance between the intersection point of the two wall vertical projection lines determined in step A117 and the center of the environmental image is within the preset projection distance threshold range.

[0005] In summary, the robot determines two vertical wall projection lines and one door beam line as candidate door frame line segments by executing steps A111 to A117. Then, based on the positional relationship between the center point of each door beam line and the intersection of the extension lines of the two vertical wall projection lines, the robot determines the direction information of the door frame where the door beam line is located relative to the current position of the robot, which is also equivalent to the direction information of the door beam line relative to the coordinates of the current position of the robot in the environment image.

[0006] Furthermore, among the two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship, there is: among the endpoints of each reference vertical projection line, the endpoint closest to the intersection of the extension lines of the two reference vertical projection lines is set as a corner point, and then the corner points of each reference vertical projection line are connected to form the one reference horizontal projection line as a reference horizontal projection line that conforms to the door structure.

[0007] Further, the method for selecting two vertical wall projection lines and one door beam line for forming a door frame from two reference vertical projection lines and one reference horizontal projection line that have been selected to form an intersection relationship according to a preset door frame size condition comprises: step A118, marking a group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship determined in step A117 as a group of candidate door frame lines; then executing step A119; step A119, judging whether the physical distance between the vertical wall corner lines corresponding to the two reference vertical projection lines in a group of candidate door frame lines is within the preset door width distance range, if so, executing step A120, otherwise, the two vertical wall projection lines for forming the door frame cannot be selected; wherein the robot uses a depth sensor to detect the physical distance between the vertical wall corner lines corresponding to the two reference vertical projection lines; step A120, calculating the physical distance between the group of candidate door frame lines; Select the ratio between the length of a reference horizontal projection line in the door frame line and the physical distance, and then convert the length of the vertical corner line corresponding to each reference vertical projection line based on the ratio and the length of each reference vertical projection line, and then execute step A121; step A121, determine whether the length of the vertical corner line corresponding to each reference vertical projection line in a group of candidate door frame lines is within the preset door frame height range, if so, mark two reference vertical projection lines in a group of candidate door frame lines as two wall vertical projection lines for forming a door frame, and mark a reference horizontal projection line in a group of candidate door frame lines as a door beam line for forming a door frame; otherwise, two wall vertical projection lines for forming a door frame cannot be screened out; wherein, the reference vertical projection line is a projection line formed by the robot collecting the vertical corner line into the coordinate system where the environmental image is located, and the vertical corner line is perpendicular to the ground where the robot walks. In summary, the present application converts the vertical projection lines from pixel size to physical size by executing steps A118 to A121, and obtains two vertical wall projection lines and one door beam line used to constitute the door frame based on the physical size, eliminates the misjudgment interference of door leaves and side wall lines perpendicular to the ground, constructs a simplified model of the door frame, obtains the vertical wall projection lines and door beam lines that conform to the simplified model of the aforementioned door frame, and correspondingly identifies a complete and accurate door frame in the environmental image.

[0008] Furthermore, the preset door width distance range is set to 60 cm to 100 cm, and the preset door frame height range is set to 1.6 m to 2.5 m, which conforms to the door frame size characteristics set in the actual room area.

[0009] Furthermore, in step A121, the intersections formed by the two vertical projection lines of the wall marked in a group of candidate door frame lines and the one door beam line marked in step A121 in the same group of candidate door frame lines are marked as corner points, and two corner points are obtained; the angle formed by the two corner points and the center of the environment image is greater than the preset angle threshold; wherein the length of each door beam line marked in step A121 in the same group of candidate door frame lines is greater than the preset door beam projection length. Thus, the positional relationship between the two vertical projection lines of the wall in the environment image is reflected, and the door beam model is simplified in terms of angle features and door beam line length features.

[0010] Furthermore, the preset angle threshold is set to 20 degrees, and the preset door beam projection length is equal to 35 pixels in the environmental image.

[0011] Further, after executing step A121, the robot uses the center point of the environment image collected at the current position to represent the current position of the robot; then the robot sets the direction from the center of the environment image collected at the current position to the center of the door beam line as the distribution direction of the door frame relative to the current position of the robot; then the angle formed by the distribution direction relative to a coordinate axis of the image coordinate system is set as the azimuth angle formed by the door frame relative to the current position of the robot, or the angle formed by the distribution direction relative to the door beam line is set as the azimuth angle formed by the door frame relative to the current position of the robot. The azimuth information set in this application is suitable for the robot to locate the entrance and exit of a room during navigation movement.

[0012] A robot is provided with a visual sensor, and the robot is used to perform the method for identifying a door frame by the robot. The robot determines two vertical wall projection lines and a door beam line as candidate door frame constituent line segments by executing steps A111 to A117, and further converts the vertical projection lines from pixel sizes to physical sizes by executing steps A118 to A121, and obtains two vertical wall projection lines and a door beam line for constituting the door frame based on the physical sizes, eliminates the misjudgment interference of door leaves and side wall lines perpendicular to the ground, and constructs a simplified model of the door frame.

[0013] Furthermore, the visual sensor adopts a camera; the lens of the camera of the robot is installed toward the ceiling of the indoor room where the robot is located, so that the viewing angle of the camera covers the surface of the ceiling and the space below it. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of a door frame to be identified in an environmental image disclosed in an embodiment of the present application, wherein the robot captures a schematic diagram marked with a vertical projection line A1B1 of the wall and a door beam line B1C1 in front of or directly below the door frame, and the robot's camera is set to face the ceiling.

[0015] Figure 2 A flowchart of a method for screening out two reference vertical projection lines and one reference horizontal projection line that form a cross relationship before a robot recognizes a door frame is disclosed in an embodiment of the present application.

[0016] Figure 3 A flowchart of a method for a robot to screen out two vertical wall projection lines and one door beam line for forming a door frame is disclosed in one embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the drawings in the embodiments of the present invention. To further illustrate each embodiment, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments.

[0018] The position of the door frame in the home environment is relatively unchanged. The door can be rotatably installed in the door frame. The door is a passage connecting two rooms. The width and shape of the door frame have a unified standard. At present, the robot determines the key boundary connecting the two room areas in the home environment by identifying the position of the door frame, but it will collect and combine door line features of multiple dimensions while the robot remains in a static condition (if the sample selection is not rich enough, it will not be able to adapt to various door recognitions). It also collects door line features through detection methods such as binocular vision or laser scanning and establishes multiple pairs of template quantities for matching and recognition. All of them require building a global map in advance and marking the recognized quantities one by one in the global map. After the robot obtains all map areas, it uses template matching to identify the door frames at the entrances and exits of the room area, which consumes relatively more computing resources to identify the door frames.

[0019] It can be understood by those skilled in the art that identifying a door frame in a wall is equivalent to extracting a gap of a specific size in the wall; and the two sides of the gap are vertical wall corner lines. In addition, the wall can be regarded as a combination of multiple vertical wall corner lines, which surround an area with a gap along the horizontal direction. In order to identify the door frame, it can be understood that the robot recognizes the environmental space in a target area within a certain range under the control of the user or in automatic mode. The target area can refer to an indoor room, an outdoor room, or a combination of an indoor room and an outdoor room. The robot is equipped with a camera with a lens facing upward, and the lens of the camera can be tilted upward on the top surface of the robot, at least covering the area between the wall where the door frame or the door beam of the door frame is located and the ceiling. The center point of the environmental image captured by the camera represents the position above the robot, and the projection position of the center point of the environmental image captured by the camera in the robot walking plane is used to represent the body center of the robot.

[0020] In order to solve the above technical problems, the present application discloses a method for a robot to identify door frames. The executor of the method is a robot. In order to respond quickly and improve the efficiency of door frame recognition, the robot can use a monocular camera to acquire images, identify walls and door frames in the acquired wall images, and build a map while acquiring and identifying walls and door frames. Of course, choosing a binocular camera or a depth sensor to acquire images and identify door frames can meet the corresponding recognition efficiency and accuracy.

[0021] As an embodiment, the method for a robot to identify a door frame includes: The robot collects an environmental image and extracts multiple straight line segments from the environmental image; preferably, the robot can identify the straight line segments in the image through various edge detection algorithms, such as vertical corner lines and horizontal corner lines, and respectively extract reference vertical projection lines and reference horizontal projection lines that form a cross relationship in the environmental image to represent them, wherein the extension lines of the two reference vertical projection lines intersect, so that the two reference vertical projection lines form a cross relationship; in the environment in which the robot actually walks, the vertical corner line is used to represent the line segment perpendicular to the ground of the indoor and outdoor room area, the vertical corner line and the horizontal corner line are perpendicular to each other, but the reference vertical projection line is not necessarily perpendicular to the horizontal corner line.

[0022] Generally, after the robot can identify the straight line segments in the image through various edge detection algorithms, the robot will calculate the angle of each straight line segment from the environmental image (the environmental image can be binarized in advance) through Hough transform (which can be obtained based on the slope of the equation of the extracted straight line segment). The robot then selects straight line segments that can represent the doorpost line, door beam line, side wall line, etc. based on the length and angle of the straight line segment.

[0023] In this embodiment, in order to allow the camera's field of view to fully cover the door frame, the robot's camera lens is installed facing the ceiling, so the camera's field of view can cover the ceiling above the robot and the area below the ceiling. If the doorpost vertical to the ground and the camera are oriented in the same direction, then based on the perspective principle of the lens, the extension lines of the two reference vertical projection lines in the environmental image will eventually intersect.

[0024] The robot selects two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship from the multiple straight line segments according to the length characteristics, angle characteristics and quantity characteristics of the line segments; thereby, the robot can exclude the reference vertical projection lines and reference horizontal projection lines that cannot form an intersection relationship from three judgment quantities, namely, the length characteristics of a single line segment, the angle characteristics between two line segments and the quantity characteristics of the cumulatively extracted straight line segments, thereby reducing the number of straight line segment samples required for selecting the wall vertical projection lines and door beam lines.

[0025] According to the preset door frame size condition, the robot selects two wall vertical projection lines and one door beam line for forming the door frame from the two reference vertical projection lines and one reference horizontal projection line that have been selected to form a cross relationship, and determines that the robot recognizes the door frame. The recognition method here refers to the current two wall vertical projection lines and one door beam line being connected to form a door frame. At this time, the current two wall vertical projection lines and one door beam line intersect and form a gap in the direction toward the ground. The preset door frame size condition is a further refinement of the aforementioned line segment length feature, which improves the accuracy of the robot's recognition of the door frame.

[0026] Since the two vertical wall projection lines and one door beam line used to form the door frame are a combination of the projection line segments of the actual door frame in the environment image, the two vertical wall projection lines used to form the door frame have an intersection relationship, and the position relationship between the intersection of the extension lines of the two vertical wall projection lines used to form the door frame and the center of the door beam line can represent the distribution direction, azimuth angle and other information formed by the door frame (or the target gap) relative to the current position of the robot. When the robot determines the center of the door beam line, the door frame is positioned.

[0027] In some embodiments, the endpoints of the two vertical wall projection lines used to form the door frame that are close to the center of the environmental image in which they are located are set as corner points, and then the corner points are connected to form a reference door beam line. The reference door beam line can be parallel to the reference horizontal projection line (i.e., the door beam line) directly extracted from the environmental image.

[0028] As an embodiment, the method of selecting two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship from the plurality of straight line segments according to the line segment length feature, angle feature and quantity feature is as follows: Figure 2 As shown, specifically including: Step A111, filter out the straight line segments perpendicular to the ground from the multiple straight line segments, where the definition of vertical is the straight line segments perpendicular to the ground of the robot's actual walking environment (including vertical wall corner lines) displayed / projected in the environment image; then mark the filtered straight line segments perpendicular to the ground as reference vertical projection lines; and then execute step A112.

[0029] Step A112, determining whether the number of reference vertical projection lines marked in step A111 is less than a value of two, if yes, determining that the robot cannot screen out two wall vertical projection lines from the environment image, causing the robot to suspend the execution of the area division method; otherwise, executing step A113, thereby eliminating invalid reference vertical projection lines.

[0030] Step A113, within the preset projection distance threshold range corresponding to the center of the environmental image, filter out the straight line segments whose lengths are within the preset width threshold range, and then mark the filtered straight line segments whose lengths are within the preset width threshold range as reference horizontal projection lines; then execute step A114; since the door beam line is generally set above the door frame and sandwiched between two reference vertical projection lines, the door beam line is projected into the environmental image close to the center of the environmental image. Therefore, a preset projection distance threshold range is set in the center of the environmental image to search for a reference horizontal projection line that can represent the door beam line at a close distance.

[0031] Step A114, determine whether the number of reference horizontal projection lines marked in step A113 is less than the value one. If so, determine that the robot cannot filter out the door beam line from the environmental image, and suspend the execution of the area division method; otherwise, execute step A115; thereby excluding invalid reference horizontal projection lines.

[0032] Step A115, filter out reference horizontal projection lines that meet the door structure by enumerating reference vertical projection lines and reference horizontal projection lines, and then execute step A116; wherein, the reference horizontal projection lines that meet the door structure are reference horizontal projection lines that form an intersection relationship with two reference vertical projection lines; therefore, it can be understood that there are at least one reference horizontal projection line and at least two reference vertical projection lines among the reference horizontal projection lines and reference vertical projection lines that meet the door structure, and there are two reference vertical projection lines that intersect with both ends of a reference horizontal projection line respectively, and in addition, the extension lines of the two reference vertical projection lines intersect.

[0033] Step A116, determine whether the number of reference horizontal projection lines that meet the door structure screened out in step A115 is greater than the value one. If so, execute step A117; otherwise, determine that the robot cannot screen out the door beam line from the environmental image, and suspend the execution of the area division method to eliminate invalid reference horizontal projection lines that meet the door structure.

[0034] Therefore, when executing step A117, there may be multiple door beam lines extending in the same direction, and the position occupied by the door beam line is subsequently determined by the connection node of two reference vertical projection lines connected to the reference horizontal projection line.

[0035] Step A117, obtain multiple groups of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship; then, in each group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship, mark the two reference vertical projection lines as two wall vertical projection lines, and mark the one reference horizontal projection line as a door beam line; wherein, the extension lines of the two reference vertical projection lines intersect, and the two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship are connected to each other as a gap, which cannot yet be defined as the door frame.

[0036] Step A117 is equivalent to determining that there are two wall vertical projection lines and a door beam line that form an intersection relationship among the reference horizontal projection line and the reference vertical projection line that conform to the door structure. Based on step A113, within the same frame of the environment image, the pixel distance between the intersection point of the extended lines of the two wall vertical projection lines determined in step A117 and the center of the environment image is within the preset projection distance threshold range.

[0037] In summary, two vertical wall projection lines and one door beam line are determined as candidate door frame line segments by executing steps A111 to A117. Then, the robot determines the direction information of the door frame where the door beam line is located relative to the current position of the robot based on the positional relationship between the center point of each door beam line and the intersection of the extension lines of the two vertical wall projection lines, which is also equivalent to the direction information of the door beam line relative to the coordinates of the current position of the robot in the environment image.

[0038] In some embodiments, among the two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship, there is: among the endpoints of each reference vertical projection line, the endpoint closest to the intersection of the extension lines of the two reference vertical projection lines is set as a corner point, and then the corner points of each reference vertical projection line are connected to form the one reference horizontal projection line as a reference horizontal projection line that conforms to the door structure.

[0039] It should be noted that there is usually a part of the wall above the door. To support this part of the wall, a prefabricated wooden or reinforced concrete beam is needed like a bridge. This beam is called a door beam. It can reduce the impact of the wall sinking on the door frame. The door beam can be processed by the image recognition algorithm in the environmental image of this embodiment to be represented by the reference horizontal projection line; the two sides below the door beam are two door posts perpendicular to the ground, which are also processed by the image recognition algorithm in the environmental image of this embodiment to be represented by the reference vertical projection line. Then, two adjacent reference vertical projection lines and a door beam line can form a door frame, forming a simplified model of a door.

[0040] On the basis of the above embodiment, the method of selecting two wall vertical projection lines and one door beam line for forming a door frame from two reference vertical projection lines and one reference horizontal projection line that have been selected to form a cross relationship according to the preset door frame size condition is as follows: Figure 3 As shown, specifically including: Step A118, mark a group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship determined in step A117 as a group of candidate door frame lines, so that there are two wall vertical projection lines and one door beam line that form an intersection relationship in a group of candidate door frame lines; then execute step A119.

[0041] Step A119, determine whether the physical distance between the vertical corner lines corresponding to two reference vertical projection lines in a group of candidate door frame lines is within the preset door width distance range. If so, execute step A120, otherwise, the two vertical wall projection lines used to form the door frame cannot be screened out to exclude the two reference vertical projection lines with inappropriate spacing; wherein, the robot uses a depth sensor to detect the physical distance between the vertical corner lines corresponding to the two reference vertical projection lines; the reference vertical projection line is a projection line formed by the robot collecting the vertical corner lines into the coordinate system of the environment image, and the vertical corner line is perpendicular to the ground where the robot walks.

[0042] In the walking environment of the robot, the minimum room width ranges from 60 cm to 100 cm, and correspondingly, the preset door width distance range is preferably 60 cm to 100 cm. The detection angle of the robot using the depth sensor also needs to cover the area between the doorpost lines on both sides of the door frame.

[0043] Step A120, calculate the ratio between the length of a reference horizontal projection line in the set of candidate door frame lines and the physical distance, and then convert the length of the vertical corner line corresponding to each reference vertical projection line based on the ratio and the length of each reference vertical projection line, so that the reference vertical projection line is converted from pixel size to physical size and represented by the length of the vertical corner line. This ratio is also applicable to the physical size conversion operation of the reference horizontal projection line in the same frame of environmental image. Then execute step A121. The length of the reference vertical projection line belongs to the pixel length occupied by the reference vertical projection line in the environmental image. After being converted into the length of the vertical corner line, the actual height of the door frame is obtained so as to judge the actual door frame in step A1.

[0044] Step A121, determine whether the length of the vertical wall corner line corresponding to each reference vertical projection line in a group of candidate door frame lines is within the preset door frame height range, if so, mark two reference vertical projection lines in a group of candidate door frame lines as two wall vertical projection lines used to form the door frame, and mark a reference horizontal projection line in a group of candidate door frame lines as a door beam line used to form the door frame, and then the door frame can be determined, wherein the target gap includes the door frame, that is, the door frame and the straight line segments composed of it (which can correspond to the projection line segments in the environment image) are identified in the wall by executing step A121; otherwise, the two wall vertical projection lines used to form the door frame cannot be screened out to exclude the wall vertical projection lines with unreasonable heights; wherein, the preset door frame height range is set to 1.6m to 2.5m, which meets the door frame height characteristics in the room.

[0045] In summary, this embodiment converts the vertical projection lines from pixel size to physical size by executing steps A118 to A121, and obtains two vertical wall projection lines and one door beam line used to constitute the door frame based on the physical size, eliminates the misjudgment interference of the door leaves and side wall lines perpendicular to the ground, constructs a simplified model of the door frame, obtains the vertical wall projection lines and door beam lines that conform to the simplified model of the aforementioned door frame, and correspondingly identifies a complete and accurate door frame in the environmental image.

[0046] It should be noted that the vertical projection line of the wall in this embodiment is a simplified projection line, and the thickness of the wall is not taken into account.

[0047] Of course, objects that conform to the simplified model of the aforementioned door frame, including the vertical projection lines of the wall and the door beam lines, also include square columns, side door leaves, door panels, walls between the ceiling and the door frame, door sills, side walls and front walls in a home environment.

[0048] In an embodiment of two vertical wall projection lines and one door beam line that can form a door frame, when two reference vertical projection lines in a group of candidate door frame lines are marked by step A121 as two vertical wall projection lines for forming a door frame, and one reference horizontal projection line in a group of candidate door frame lines is marked by step A121 as one door beam line for forming a door frame, there exists: intersection marks formed by the two vertical wall projection lines marked by the aforementioned step A121 in a group of candidate door frame lines and one door beam line marked by step A121 in the same group of candidate door frame lines, respectively. Recorded as corner points, two corner points are obtained; the angle formed by the two corner points and the center of the environmental image is greater than a preset angle threshold, and the preset angle threshold is preferably 20 degrees; in step A121, the length of a door beam line marked in the same group of candidate door frame lines is greater than the preset door beam projection length, and the preset door beam projection length is preferably 35 pixels in length, and the extension lines of the two reference vertical projection lines in the same group of candidate door frame lines intersect, thereby reflecting the positional relationship between the two wall vertical projection lines in the environmental image, and simplifying the door beam model in terms of angle features and door beam line length features respectively.

[0049] Indicatively, see Figure 1 It can be seen that Figure 1 It is a schematic diagram of the environment image collected by the robot in front of the door frame. The center of the environment image currently collected by the robot's camera is located near point O. Point O is the intersection of the extension lines of the two vertical projection lines of the wall. The pixel distance between point O and the center of the environment image is within the preset projection distance threshold range. The preset projection distance threshold range is used to represent the error range of the vertical wall corner line converted to the pixel coordinate system. The error source includes the deformation and distortion of the image caused by the lens distortion of the camera. In some ideal cases, point O is the center of the environment image, and the angle formed by the corner point B1 and the corner point C1 shown in the figure and the center O of the environment image is angle B1OC1, and angle B1OC1 is greater than the preset angle threshold.

[0050] The straight line segment A1B1 is the vertical projection line of the wall on one side of the robot extracted by the robot, and the straight line segment D1C1 is the vertical projection line of the wall on the other side of the robot extracted by the robot. Then, the endpoint B1 close to point O is selected from the two endpoints of the vertical projection line A1B1 of the wall as a corner point, and the endpoint C1 close to point O is selected from the two endpoints of the vertical projection line D1C1 of the wall as another corner point. Then, the corner points B1 and C1 are connected to form a line segment B1C1. Then, the line segment B1C1 forms the door beam line, and then Figure 2 The wall vertical projection line A1B1, the wall vertical projection line C1D1 and the door beam line B1C1 extracted from the image processing apparatus are connected to form the target gap to represent the door frame existing in the actual environment.

[0051] As an embodiment, after the robot executes the step A121, the robot uses the center point of the environment image collected at the current position to represent the current position of the robot; then the robot sets the direction from the center of the environment image collected at the current position to the center of the door beam line as the distribution direction of the door frame relative to the current position of the robot; in some embodiments, the current position of the robot is represented by point O, point O represents the center of the environment image, then the robot sets the direction from the center of the currently collected environment image to the center of the door beam line as the distribution direction of the door frame relative to the current position of the robot, corresponding to Figure 1 In the figure, the center of the door beam line is the midpoint H1 of the straight line segment B1C1, and the direction from point O to point H1 is the distribution direction of the door frame where the door beam line B1C1 is located relative to the current position of the robot (specifically, its pixel coordinate position in the image). It can also be understood that the robot is located in the direction of H1O of the door frame where the door beam line B1C1 is located, and the robot is regarded as walking along the direction OH1 to the front of the door frame where the door beam line B1C1 is located.

[0052] Then the robot sets the angle formed by the distribution direction relative to a coordinate axis of the image coordinate system as the azimuth angle formed by the door frame relative to the current position of the robot, or sets the angle formed by the distribution direction relative to the door beam line as the azimuth angle formed by the door frame relative to the current position of the robot. In some embodiments, the robot sets the angle formed by the direction from one corner point to another corner point and the coordinate axis of the image coordinate system, or the angle converted from the angle to the world coordinate system as the angle information of the door beam line, so as to convert it into the azimuth angle of the door beam line in the outdoor room area; the direction from one corner point to another corner point is an extension direction of the door beam line in the image coordinate system, schematically represented as Figure 2 The angle formed by the direction from point O to point H1 and the direction from point B1 to point C1 is used to represent the angle formed by the distribution direction relative to the door beam line, which is correspondingly set to the azimuth angle formed by the door frame relative to the current position of the robot. The center of the door beam line is further converted into the world coordinate system to realize the conversion of the angle information of the door beam line into the world coordinate system, that is, it falls into the grid map and is positioned with the help of the world coordinate system, which is suitable for the robot to locate the entrance and exit of the room during the navigation movement.

[0053] Based on the aforementioned embodiments, the present application discloses a robot, which is equipped with a visual sensor. The visual sensor is generally installed on the top of the robot's body to cover a larger viewing space. The robot is used to execute the method for robot identifying door frames mentioned in the aforementioned embodiments. The robot determines two vertical wall projection lines and one door beam line as candidate door frame constituent line segments by executing steps A111 to A117, and also converts the vertical projection lines from pixel size to physical size by executing steps A118 to A121, and obtains two vertical wall projection lines and one door beam line for constituting the door frame based on the physical size, eliminates the misjudgment interference of door leaves and side wall lines perpendicular to the ground, and constructs a simplified model of the door frame.

[0054] The program code corresponding to the method for robot recognition of door frames disclosed in the above embodiment is stored in the controller or memory built into the robot. When the robot calls the program code, the robot is configured to execute the method for robot recognition of door frames disclosed in the above embodiment. Compared with the prior art, no laser sensor is installed, saving sensor costs. When the robot passes through different room areas, it can quickly recognize the door frame and mark it in the map.

[0055] As an embodiment, the visual sensor uses a camera. In order to respond quickly and improve mapping efficiency, the robot can use a monocular camera to collect images and identify walls and door frames in the collected wall images; the lens of the robot's camera is installed toward the ceiling of the indoor room where the robot is located, so that the camera's field of view covers the surface of the ceiling and the space below it, wherein a door frame is set in the space below the ceiling as an entrance and exit to different room areas.

[0056] The camera is arranged at the front end of the robot body, and the camera of the robot is arranged to face horizontally and obliquely upward, and the optical axis of the camera can be close to 90 degrees with the horizontal plane, and the environmental image collected by the camera can at least cover the area between the wall surface where the door frame or the door beam of the door frame is located and the ceiling. In this embodiment, a single camera can be arranged, and multi-angle camera can be maintained during the process of performing the door frame recognition task.

[0057] Due to the passability of the door, the same door frame can be detected in front, behind, and below the door frame, but the square pillars and door leaves in the home environment can only be detected by the robot in a single direction, and the passability of the door allows the robot's upward camera to detect the same door frame in three viewing angles: in front, behind, and directly below. When the target gap is used to represent the door frame in the aforementioned embodiment, the robot selects two reference lines that form an intersection relationship from multiple straight line segments extracted from the environment image according to the line segment length feature, angle feature, and quantity feature. A reference vertical projection line and a reference horizontal projection line are selected; then according to the preset door frame size conditions, two wall vertical projection lines and a door beam line for forming a door frame are selected from the two reference vertical projection lines and one reference horizontal projection line that have been selected to form a cross relationship, and the door frame is identified and identified, that is, the door frame is connected to form the door frame, so as to further eliminate the influence of multiple vertical line segments similar to the shape of the door frame formed on the boundary between the front wall and the side wall of the indoor working area, such as the misjudgment caused by the characteristic lines of the square columns and the characteristic lines of the door leaves, thereby improving the recognition accuracy of the door frame.

[0058] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0059] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for robot to identify door frames, characterized in that: include: The robot collects an environment image and extracts a plurality of straight line segments from the environment image; According to the line segment length characteristics, angle characteristics and quantity characteristics, two reference vertical projection lines and one reference horizontal projection line forming an intersection relationship are selected from the plurality of straight line segments; According to the preset door frame size conditions, two wall vertical projection lines and one door beam line for forming the door frame are selected from the two reference vertical projection lines and one reference horizontal projection line that have been selected to form a cross relationship, and it is determined that the robot recognizes the door frame.

2. The method according to claim 1, characterized in that: The method of selecting two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship from the plurality of straight line segments according to the line segment length feature, angle feature and quantity feature comprises: Step A111, selecting straight line segments perpendicular to the ground from the plurality of straight line segments; then selecting straight line segments whose length is greater than a preset wall height threshold from the selected straight line segments perpendicular to the ground and marking them as reference vertical projection lines; then executing step A112; Step A112, determining whether the number of reference vertical projection lines marked in step A111 is less than a value of two, if yes, determining that the robot cannot filter out two wall vertical projection lines from the environment image, otherwise executing step A113; Step A113, within the preset projection distance threshold range corresponding to the center of the environment image, filter out the straight line segments whose lengths are within the preset width threshold range, and then mark the filtered straight line segments whose lengths are within the preset width threshold range as reference horizontal projection lines; then execute step A114; Step A114, determining whether the number of reference horizontal projection lines marked in step A113 is less than a value of one, if so, determining that the robot cannot filter out the door beam line from the environment image, otherwise executing step A115; Step A115, screening out a reference horizontal projection line that meets the door structure by enumerating the reference vertical projection lines and the reference horizontal projection lines, and then executing step A116; wherein the reference horizontal projection line that meets the door structure is a reference horizontal projection line that forms an intersection relationship with the two reference vertical projection lines; Step A116, determining whether the number of reference horizontal projection lines that meet the door structure selected in step A115 is greater than a value of one, if yes, executing step A117, otherwise determining that the robot cannot select the door beam line from the environment image; Step A117, obtaining multiple groups of two reference vertical projection lines and one reference horizontal projection line forming a cross relationship; then in each group of two reference vertical projection lines and one reference horizontal projection line forming a cross relationship, marking the two reference vertical projection lines as two wall vertical projection lines, and marking the one reference horizontal projection line as a door beam line; wherein the extension lines of the two reference vertical projection lines intersect; In the same frame of the environment image, the distance between the intersection of the two wall vertical projection lines determined in step A117 and the center of the environment image is within the preset projection distance threshold range.

3. The method according to claim 2, characterized in that: Among the two reference vertical projection lines and one reference horizontal projection line forming the intersection relationship, there are: Among the endpoints of each reference vertical projection line, the endpoint closest to the intersection of the extension lines of the two reference vertical projection lines is set as a corner point, and then the corner points of each reference vertical projection line are connected to form the reference horizontal projection line.

4. The method according to claim 2, characterized in that: The method of selecting two wall vertical projection lines and one door beam line for forming a door frame from two reference vertical projection lines and one reference horizontal projection line that have been selected to form a cross relationship according to a preset door frame size condition comprises: Step A118, marking a group of two reference vertical projection lines and one reference horizontal projection line that form an intersection relationship determined in step A117 as a group of candidate door frame lines; then executing step A119; Step A119, determining whether the physical distance between the vertical corner lines corresponding to the two reference vertical projection lines in a set of candidate door frame lines is within the preset door width distance range, if yes, executing step A120, otherwise, the two vertical wall projection lines for forming the door frame cannot be screened out; wherein the robot uses a depth sensor to detect the physical distance between the vertical corner lines corresponding to the two reference vertical projection lines; Step A120, calculating the ratio between the length of a reference horizontal projection line in the set of candidate door frame lines and the physical distance, and then converting the length of the vertical wall corner line corresponding to each reference vertical projection line based on the ratio and the length of each reference vertical projection line, and then executing step A121; Step A121, determining whether the lengths of the vertical wall corner lines corresponding to each reference vertical projection line in a set of candidate door frame lines are all within a preset door frame height range, if so, marking two reference vertical projection lines in a set of candidate door frame lines as two wall vertical projection lines for forming a door frame, and marking a reference horizontal projection line in a set of candidate door frame lines as a door beam line for forming a door frame; otherwise, no two wall vertical projection lines for forming a door frame can be screened out; The reference vertical projection line is a projection line formed when the robot collects the vertical wall corner line into the coordinate system where the environment image is located, and the vertical wall corner line is perpendicular to the ground where the robot walks.

5. The method according to claim 4, characterized in that: The preset door width distance range is set to 60cm to 100cm; the preset door frame height range is set to 1.6m to 2.5m.

6. The method according to claim 4, characterized in that: Step A121 marks the intersections formed by the two vertical projection lines of the wall marked in a group of candidate door frame lines and the one door beam line marked in the same group of candidate door frame lines in step A121 as corner points, and obtains two corner points; the angle formed by the two corner points and the center of the environment image is greater than a preset angle threshold; Among them, the length of each door beam line marked in the same set of candidate door frame lines in step A121 is greater than the preset door beam projection length.

7. The method according to claim 6, characterized in that: The preset angle threshold is set to 20 degrees, and the preset door beam projection length is equal to 35 pixels in the environment image.

8. The method according to claim 4, characterized in that: After executing step A121, the robot uses the center point of the environmental image collected at the current position to represent the current position of the robot; then the robot sets the direction from the center of the environmental image collected at the current position to the center of the door beam line as the distribution direction of the door frame relative to the current position of the robot; then the angle formed by the distribution direction relative to a coordinate axis of the image coordinate system is set as the azimuth angle formed by the door frame relative to the current position of the robot, or the angle formed by the distribution direction relative to the door beam line is set as the azimuth angle formed by the door frame relative to the current position of the robot.

9. A robot equipped with a visual sensor, characterized in that: The robot is used to execute the method for robot identifying door frames as described in any one of claims 1 to 8.

10. The robot according to claim 9, characterized in that: The visual sensor uses a camera; The camera lens of the robot is installed toward the ceiling of the indoor room where the robot is located, so that the viewing angle of the camera covers the surface of the ceiling and the space below it.

Images