A stair measurement method, system and quadruped robot based on depth vision

Abstract

The present invention belongs to the field of robot technology, and provides a stair measurement method, system and quadruped robot based on depth vision, wherein a stair measurement method based on depth vision includes: collecting RGB images and depth images of stairs; segmenting the vertical plane and / or horizontal plane of the stairs in the RGB image, selecting a vertical plane and / or a horizontal plane to establish a circumscribed rectangular frame; selecting two relative points on the upper and lower boundary lines of the circumscribed rectangular frame of the vertical plane as the first point pair, and / or, selecting two relative points on the upper and lower boundary lines of the circumscribed rectangular frame of the horizontal plane as the second point pair; obtaining the Euclidean distance of the first point pair and / or the second point pair based on the RGB image and the depth image, the Euclidean distance of the first point pair is the height of the vertical plane of the stairs, and the Euclidean distance of the second point pair is the width of the horizontal plane of the stairs. The present invention solves the problems of interference and low precision in the width and height measurement process of stairs in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of robots, and in particular relates to a stair measurement method and system based on depth vision and a quadruped robot. Background Art

[0002] In some high-risk industries, in order to ensure the safety of facilities and equipment and the operating environment, and to prevent workers from being in high-risk environments for a long time, companies use advanced robot technology to replace workers for safety inspections. Common inspection robots include wheeled robots, tracked robots, and quadruped robots. Due to the multi-layered and complex environmental terrain in some high-risk industries, wheeled and tracked robots cannot traverse all terrains. In comparison, the advantage of quadruped robots is that they can climb stairs. In order to ensure that quadruped robots can climb stairs smoothly, the modeling and accurate measurement of stairs are particularly important.

[0003] In the prior art, the construction and measurement of stairs adopts a method combining RGB and depth maps. The side lines of the stairs are identified by straight line extraction from the RGB image, and the side difference and side slope of each step are obtained through the side lines. The width and height of the stairs are measured in combination with the depth map to provide visual assistance for the robot to climb the stairs. This measurement method requires the accumulation of the sides of multiple stairs to improve the measurement accuracy. However, when there are too many stairs, the higher stairs are farther away from the robot, and the light reflection and the image texture of the step surface have a great influence on the detection of the depth value, resulting in interference in the depth map and reduced measurement accuracy of the width and height of the stairs. Summary of the invention

[0004] The present invention aims to at least solve the technical problems existing in the prior art, and provide a stair measurement method, system and quadruped robot based on depth vision to solve the problems of interference and low precision in the width and height measurement process of stairs in the prior art.

[0005] In order to achieve the above-mentioned purpose of the present invention, according to a first aspect of the present invention, the present invention provides a stair measurement method based on depth vision, including: collecting RGB images and depth images of the stairs; segmenting the vertical plane and / or horizontal plane of the stairs in the RGB image, selecting a vertical plane and / or a horizontal plane to establish a circumscribed rectangular frame; selecting two relative points on the upper and lower boundary lines of the circumscribed rectangular frame of the vertical plane as a first point pair, and / or, selecting two relative points on the upper and lower boundary lines of the circumscribed rectangular frame of the horizontal plane as a second point pair; based on the RGB image and the depth image, obtaining the Euclidean distance of the first point pair and / or the second point pair, the Euclidean distance of the first point pair is the height of the vertical plane of the stairs, and the Euclidean distance of the second point pair is the width of the horizontal plane of the stairs.

[0006] Further, when the depth value of the first point pair or the second point pair is not 0, the Euclidean distance of the first point pair or the second point pair is obtained based on the RGB image and the depth image; when the depth value of the first point pair is 0, the Euclidean distance between two first reference points on the line connecting the first point pair whose depth values ​​are not 0 is obtained, and the Euclidean distance of the first point pair is obtained based on the Euclidean distance of the first reference point pair; or, when the depth value of the second point pair is 0, the Euclidean distance between two second reference points on the line connecting the second point pair whose depth values ​​are not 0 is obtained, and the Euclidean distance of the second point pair is obtained based on the Euclidean distance of the second reference point pair.

[0007] Further, the step of obtaining the Euclidean distance of the first point pair based on the Euclidean distance of the first reference point pair is specifically as follows: based on the RGB image, obtaining the ratio of the plane coordinate distance of the first reference point pair to the plane coordinate distance of the first point pair, defining the ratio as a first scaling ratio, and obtaining the Euclidean distance of the first point pair based on the Euclidean distance of the first reference point pair and the first scaling ratio; the step of obtaining the Euclidean distance of the second point pair based on the Euclidean distance of the second reference point pair is specifically as follows: based on the RGB image, obtaining the ratio of the plane coordinate distance of the second reference point pair to the plane coordinate distance of the second point pair, defining the ratio as a second scaling ratio, and obtaining the Euclidean distance of the second point pair based on the Euclidean distance of the second reference point pair and the second scaling ratio.

[0008] Further, the step of obtaining the first reference point pair includes: traversing all points on the line connecting the first point pair, and selecting the first point and the last point on the line connecting the first point pair whose depth value is not 0 as the first reference point pair; the step of obtaining the second reference point pair includes: traversing all points on the line connecting the second point pair, and selecting the first point and the last point on the line connecting the second point pair whose depth value is not 0 as the second reference point pair.

[0009] Furthermore, the first point pair and / or the second point pair are midpoints of upper and lower boundaries of the circumscribed rectangular frame.

[0010] Furthermore, the specific steps of selecting a horizontal plane to establish a circumscribed rectangular frame are: selecting a complete horizontal plane closest to the camera to establish a circumscribed rectangular frame.

[0011] Furthermore, the step of segmenting the vertical surface and / or horizontal surface of the stairs in the RGB image is specifically as follows: segmenting the vertical surface and / or horizontal surface in the RGB image using an image segmentation model, wherein the image segmentation model is trained by an initial segmentation model using a sample training set.

[0012] Furthermore, the number of samples in the sample training set is not less than 500.

[0013] In order to achieve the above-mentioned purpose of the present invention, according to the second aspect of the present invention, the present invention provides a stair measurement system based on depth vision, using any one of the above-mentioned stair measurement methods based on depth vision, the system includes an acquisition module, a segmentation module, a selection module and a calculation module: the acquisition module is used to obtain the RGB image and depth image of the stairs; the segmentation module is used to segment the vertical plane and / or horizontal plane in the RGB image; the selection module is used to select a vertical plane and / or a horizontal plane to establish a circumscribed rectangular frame, and select a first point pair and / or a second point pair on the upper and lower boundary lines of the circumscribed rectangular frame of the vertical plane and / or the horizontal plane; the calculation module is used to obtain the Euclidean distance of the first point pair and / or the second point pair according to the RGB image and the depth image.

[0014] In order to achieve the above-mentioned purpose of the present invention, according to the third aspect of the present invention, the present invention provides a quadruped robot, including a depth camera and a controller, the lens of the depth camera is aimed at the front of the quadruped robot, the depth camera is connected to the controller, and after the controller receives the RGB image and depth image output by the depth camera, the controller measures the stair parameters according to any one of the above-mentioned depth vision-based stair measurement methods.

[0015] The technical principle of the present invention is as follows: the upper boundary point and the lower boundary point are selected at the same ratio on the upper boundary and the lower boundary of the circumscribed rectangular frame, so the Euclidean distance between the upper boundary points is the absolute width of the circumscribed rectangular frame, that is, the absolute width of the stair image segmentation surface; the segmentation surface of the stair image includes a vertical segmentation surface and a horizontal segmentation surface. When the selected segmentation surface is a vertical surface, the calculated Euclidean distance between the upper boundary point and the lower boundary point is the height of the stairs. When the segmentation surface of the stairs is a horizontal segmentation surface, the calculated Euclidean distance between the upper boundary point and the lower boundary point is the width of the stairs. The length and width of the stairs can be successfully calculated in the above manner.

[0016] The beneficial effects of the present invention are as follows: by accurately segmenting the staircase image and calculating the absolute width of the segmentation surface through the plane coordinates of the four corners of the circumscribed rectangular frame. Compared with the staircase measurement technology in the prior art, the present invention relies on a segmentation surface of the staircase image during the calculation of staircase data. When the number of stairs is small, the calculation result of the staircase data is not affected, thereby ensuring the calculation accuracy of the staircase data. In addition, the present invention has better robustness. When the staircase image is affected by light reflection and the image texture of the step surface, the segmentation surface with less influence can be selected for calculation, thereby reducing the influence of external factors on the calculation of staircase data and improving the calculation accuracy of staircase data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the steps of a staircase measurement method based on depth vision of the present invention;

[0018] Figure 2is a schematic diagram of a staircase image of the present invention;

[0019] Figure 3 is a schematic diagram of segmentation of the vertical and horizontal planes of the stairs in the stair image of the present invention;

[0020] Figure 4 is a schematic diagram of a circumscribed rectangular frame of a vertical surface of the present invention;

[0021] Figure 5 is a schematic diagram of a circumscribed rectangular frame of a horizontal plane of the present invention;

[0022] Figure 6 is a schematic diagram of the structure of the second point pair of the present invention;

[0023] Figure 7 It is a structural schematic diagram of the second point-to-line connection of the present invention;

[0024] Figure 8 is a schematic structural diagram of a second reference point pair on a second point pair connection line of the present invention;

[0025] Fig. 9 It is a structural schematic diagram of a stair measurement system based on depth vision of the present invention.

[0026] Fig.10 The figure is a schematic diagram of the appearance structure of a quadruped robot of the present invention.

[0027] Reference numeral: Depth camera 1 . DETAILED DESCRIPTION

[0028] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0029] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0030] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0031] As attached Figure 1 As shown, the present invention provides a staircase measurement method based on depth vision, comprising:

[0032] S1: Collect RGB images and depth images of the stairs. The RGB image of the stairs is as shown in the attached figure. Figure 2 As shown; specifically, a depth camera 1 is used to collect an RGB image and a depth image of the stairs; the model of the depth camera 1 is preferably an Intel Realsense D435i depth camera.

[0033] S2: As attached Figure 3 As shown in FIG. 1 , the vertical and / or horizontal surfaces of the stairs in the RGB image are segmented; as shown in FIG. Figure 4 and attached Figure 5 As shown, a vertical plane and / or a horizontal plane are selected to establish a circumscribed rectangular frame; specifically, the RGB image is placed in an image segmentation model for segmentation, a vertical plane and / or a horizontal plane are selected from the segmented RGB image to establish a circumscribed rectangular frame, and the plane coordinates of the four corners of the circumscribed rectangular frame are obtained.

[0034] S3: selecting two relative points on the upper and lower boundary lines of the circumscribed rectangular frame of the vertical plane as a first point pair, and / or selecting two relative points on the upper and lower boundary lines of the circumscribed rectangular frame of the horizontal plane as a second point pair;

[0035] As attached Figure 6 As shown, two relative points are selected on the upper and lower boundary lines of the circumscribed rectangular frame of the horizontal plane as the second point pair, and the plane coordinates of the second point pair are obtained through the plane coordinates of the four corners of the circumscribed rectangular frame; preferably, two relative points are selected on the upper and lower boundary lines as the second point pair according to the ratio value of 1 / 2, that is, the midpoint of the upper boundary and the lower boundary is selected as the second point pair. In other implementations, the ratio value can be 1 / 3, 2 / 3, 1 / 4 or other numbers less than 1. The selection process of the first point pair refers to the second point pair, and will not be repeated here.

[0036] S4: Acquire the Euclidean distance of the first point pair and / or the second point pair based on the RGB image and the depth image, wherein the Euclidean distance of the first point pair is the height of the vertical surface of the stairs, and the Euclidean distance of the second point pair is the width of the horizontal surface of the stairs.

[0037] Preferably, step S4 is specifically: when the depth value of the first point pair or the second point pair is not 0, obtaining the Euclidean distance of the first point pair or the second point pair based on the RGB image and the depth image; specifically, the Euclidean distance calculation process of the second point pair is as follows:

[0038] Assume that the four-corner plane coordinates of the circumscribed rectangular box of the horizontal plane include the upper left corner plane coordinates (x1, y1), the upper right corner plane coordinates (x2, y2), the lower left corner plane coordinates (x3, y3), and the lower right corner plane coordinates (x4, y4).

[0039] Assume the ratio is 1 / 2, the second point pair includes the upper boundary point p u and the lower boundary point p d , the upper boundary point p u The plane coordinates are (x u ,y u ), the lower boundary point p d The plane coordinates are (x d ,y d ), then the upper boundary point p u The plane coordinate value of Lower boundary point p d The plane coordinate value of

[0040] Where min(y2,y1) is the minimum value of (y2,y1), min(y3,y4) is the minimum value of (y3,y4), for The absolute value of for The absolute value of .

[0041] If the ratio is 2 / 3, then the upper boundary point p u The plane coordinate value of Lower boundary point p d The plane coordinate value of The remaining ratio values ​​are similar.

[0042] Get the upper boundary point p u The depth value z u With the lower boundary point p d The depth value z d , construct the upper boundary point p u The three-dimensional coordinates p u =(x u ,y u ,z u ), the lower boundary point p d The three-dimensional coordinates p d=(x d ,y d ,z d ), then the upper boundary point p u With the lower boundary point p d The Euclidean distance D ud for:

[0043]

[0044] D ud is the width of the horizontal plane of the stairs. The Euclidean distance of the first point pair refers to the Euclidean distance of the second point pair, which will not be repeated here.

[0045] Ideally, the depth camera 1 can successfully obtain the depth value of the first point pair and establish the three-dimensional coordinates of the first point pair; in the implementation process, the depth camera 1 is affected by the accuracy of the depth sensor and cannot guarantee that it can successfully obtain the depth value of the first point pair every time. Therefore, the line traversal method is used to first select the points with depth values ​​on the line connecting the first point pair, and then calculate the Euclidean distance of the first point pair.

[0046] Preferably, step S4 further comprises: when the depth value of the first point pair is 0, obtaining the Euclidean distance between two first reference points on the line connecting the first point pair whose depth values ​​are not 0, and obtaining the Euclidean distance of the first point pair based on the Euclidean distance of the first reference point pair;

[0047] Alternatively, when the depth value of the second point pair is 0, the Euclidean distance between two second reference points on the line connecting the second point pair whose depth values ​​are not 0 is obtained, and the Euclidean distance of the second point pair is obtained based on the Euclidean distance of the second reference point pair.

[0048] Specifically, the step of obtaining the Euclidean distance of the second reference point pair includes:

[0049] As attached Figure 7 As shown, connect the upper boundary point p u With the lower boundary point p d , forming a second point pair line, traversing the points on the second point pair line, obtaining two points with non-zero depth values ​​on the second point pair line based on the depth image as the second reference point pair, and u and the lower boundary point p d The plane coordinates of the second reference point pair are obtained by the following process:

[0050] According to the upper boundary point p u With the lower boundary point p d Calculate the slope k and intercept b of the line connecting the second pair of points using the plane coordinates of b=y u -kx u ;

[0051] Traverse the points p on the line connecting the second point pair, remove the point p with a depth value of 0, select two points from the remaining points as the second reference point pair, and calculate the plane coordinates of the second reference point pair according to the slope k of the line connecting the second point pair, the intercept b and the ordinate of the second reference point pair; since the points p in the staircase image are all pixel points, the ordinate y of the point p is p for y u To d For example, if y u is 20, y d is 15, then the ordinate y of point p is p is 15, 16, 17, 18, 19 or 20, then the abscissa of point p is

[0052] As attached Figure 8 As shown, point p1 and point p2 are selected as the second reference point pair, and the plane coordinates of p1 are calculated according to the plane coordinate calculation method of the second reference point pair as (x p1 ,y p1 ), the plane coordinates of p2 are (x p2 ,y p2 ). Get the depth value z of point p1 through the depth image p1 and the depth value z of point p2 p2 , according to the depth value z of point p1 p1 The three-dimensional coordinates (x p1 ,y p1 ,z p1 ), according to the depth value z of point p2 p2 and the plane coordinates to form the three-dimensional coordinates of point p2 (x p2 ,y p2 ,z p2 ), obtain the Euclidean distance D between point p1 and point p2 through the three-dimensional coordinates of point p1 and point p2 12 :

[0053]

[0054] The process of obtaining the Euclidean distance of the first reference point pair refers to the process of obtaining the Euclidean distance of the second reference point pair, which will not be repeated here.

[0055] Preferably, the step of acquiring the Euclidean distance of the first point pair based on the Euclidean distance of the first reference point pair is specifically: acquiring the ratio of the plane coordinate distance of the first reference point pair to the plane coordinate distance of the first point pair based on the RGB image, defining the ratio as a first scaling ratio, and acquiring the Euclidean distance of the first point pair based on the Euclidean distance of the first reference point pair and the first scaling ratio;

[0056] The step of obtaining the Euclidean distance of the second point pair based on the Euclidean distance of the second reference point pair is specifically as follows: based on the RGB image, obtaining the ratio of the plane coordinate distance of the second reference point pair to the plane coordinate distance of the second point pair, defining the ratio as a second scaling ratio, and obtaining the Euclidean distance of the second point pair based on the Euclidean distance of the second reference point pair and the second scaling ratio.

[0057] Specifically, the process of obtaining the Euclidean distance of the second point pair is as follows:

[0058] First, obtain the second scaling ratio: According to the plane coordinates (x p1 ,y p1 ) and the plane coordinates of point p2 (x p2 ,y p2 ) ; Calculate the plane coordinate distance of the second reference point pair

[0059] According to the upper boundary point p u With the lower boundary point p d Plane coordinates of the second point pair, get the plane coordinate distance The plane coordinate distance D' passing through the second reference point pair 12 , the plane coordinate distance D' of the second point pair ud Get the second scaling factor k':

[0060]

[0061] Then according to the Euclidean distance D of the second reference point pair 12 and the second scaling factor k' yields the Euclidean distance D of the second pair of points ud :

[0062]

[0063] The process of obtaining the Euclidean distance of the first point pair refers to the process of obtaining the Euclidean distance of the second point pair, which will not be repeated here.

[0064] As attached Figure 5 As shown, in the staircase image, the higher the stairs, the smaller the horizontal plane, the closer the distance between the upper and lower boundaries of the circumscribed rectangular frame, the shorter the line connecting the second point pair, and the higher the stairs, the farther away from the depth camera 1, the lower the accuracy of the detection result of the depth camera 1, and the fewer points with non-zero depth values, which is not conducive to the selection of the second reference point pair. Since the width of each step of the stairs is the same, when selecting the horizontal plane to calculate the width of the horizontal plane of the stairs, the complete horizontal plane in the staircase image that is closest to the camera head of the depth camera 1 is preferred according to the ordinate of the staircase image. As shown in the attached figure, Figure 4As shown, the vertical plane in the stair image changes little, so when selecting a vertical plane to calculate the height of the vertical plane of the stair, it is sufficient to select a vertical plane at an appropriate position.

[0065] Preferably, in order to improve the segmentation accuracy of the image segmentation model, the step of segmenting the vertical surface and / or horizontal surface of the stairs in the RGB image is specifically as follows: using the image segmentation model to segment the vertical surface and / or horizontal surface in the RGB image, the image segmentation model is trained by the initial segmentation model through the sample training set; the number of samples in the sample training set is not less than 500.

[0066] As attached Fig. 9 As shown, the present invention also provides a stair measurement system based on depth vision, which uses any of the above-mentioned stair measurement methods based on depth vision during operation. The system includes an acquisition module, a segmentation module, a selection module and a calculation module:

[0067] The acquisition module is used to obtain the RGB image and depth image of the stairs;

[0068] The segmentation module is used to segment the vertical plane and / or horizontal plane in the RGB image;

[0069] The selection module is used to select a vertical plane and / or a horizontal plane to establish a circumscribed rectangular frame, and select a first point pair and / or a second point pair on the upper and lower boundary lines of the circumscribed rectangular frame of the vertical plane and / or the horizontal plane;

[0070] The calculation module is used to obtain the Euclidean distance of the first point pair and / or the second point pair according to the RGB image and the depth image.

[0071] As attached Fig.10 As shown, the present invention also provides a quadruped robot, including a depth camera 1 and a controller, wherein the lens of the depth camera 1 is aimed at the front of the quadruped robot, and the depth camera 1 is connected to the controller.

[0072] After receiving the RGB image and the depth image output by the depth camera 1 , the controller measures the stair parameters according to the stair measurement method based on depth vision provided above.

[0073] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0074] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A staircase measurement method based on depth vision, characterized in that: include: Collect RGB images and depth images of the stairs; Segment the vertical surface and / or horizontal surface of the stairs in the RGB image, and select a vertical surface and / or a horizontal surface to establish an external rectangular frame; Selecting two relative points on the upper and lower boundary lines of the circumscribed rectangular frame of the vertical plane as a first point pair, and / or selecting two relative points on the upper and lower boundary lines of the circumscribed rectangular frame of the horizontal plane as a second point pair; Acquire the Euclidean distance of the first point pair and / or the second point pair based on the RGB image and the depth image, wherein the Euclidean distance of the first point pair is the height of the vertical surface of the stairs, and the Euclidean distance of the second point pair is the width of the horizontal surface of the stairs; When the depth value of the first point pair or the second point pair is not 0, obtaining the Euclidean distance of the first point pair or the second point pair based on the RGB image and the depth image; When the depth value of the first point pair is 0, the Euclidean distance between two first reference points whose depth values ​​are not 0 on the line connecting the first point pair is obtained, and the Euclidean distance of the first point pair is obtained based on the Euclidean distance of the first reference point pair; Alternatively, when the depth value of the second point pair is 0, the Euclidean distance between two second reference points on the line connecting the second point pair whose depth values ​​are not 0 is obtained, and the Euclidean distance of the second point pair is obtained based on the Euclidean distance of the second reference point pair; The step of obtaining the Euclidean distance of the first point pair based on the Euclidean distance of the first reference point pair is specifically as follows: obtaining the ratio of the plane coordinate distance of the first reference point pair to the plane coordinate distance of the first point pair based on the RGB image, defining the ratio as a first scaling ratio, and obtaining the Euclidean distance of the first point pair based on the Euclidean distance of the first reference point pair and the first scaling ratio; Among them, the step of obtaining the Euclidean distance of the second point pair based on the Euclidean distance of the second reference point pair is specifically: obtaining the ratio of the plane coordinate distance of the second reference point pair to the plane coordinate distance of the second point pair based on the RGB image, defining the ratio as a second scaling ratio, and obtaining the Euclidean distance of the second point pair based on the Euclidean distance of the second reference point pair and the second scaling ratio.

2. A method for measuring stairs based on depth vision as claimed in claim 1, characterized in that: The step of obtaining the first reference point pair includes: traversing all points on the line connecting the first point pair, and selecting the first point and the last point on the line connecting the first point pair whose depth value is not 0 as the first reference point pair; The step of obtaining the second reference point pair includes: traversing all points on the second point pair line, and selecting the first point and the last point on the second point pair line with a depth value other than 0 as the second reference point pair.

3. A method for measuring stairs based on depth vision as claimed in claim 1 or 2, characterized in that: The first point pair and / or the second point pair are midpoints of upper and lower boundaries of the circumscribed rectangular frame.

4. A method for measuring stairs based on depth vision as claimed in claim 1 or 2, characterized in that: The specific steps of selecting a horizontal plane to establish a circumscribed rectangular frame are: selecting a complete horizontal plane closest to the camera to establish a circumscribed rectangular frame.

5. A method for measuring stairs based on depth vision as claimed in claim 1 or 2, characterized in that: The step of segmenting the vertical surface and / or horizontal surface of the stairs in the RGB image is specifically as follows: segmenting the vertical surface and / or horizontal surface in the RGB image using an image segmentation model, wherein the image segmentation model is trained by an initial segmentation model using a sample training set.

6. A method for measuring stairs based on depth vision as claimed in claim 5, characterized in that: The number of samples in the sample training set is not less than 500.

7. A staircase measurement system based on depth vision, characterized in that: A staircase measurement method based on depth vision as described in any one of claims 1 to 6 is used, and the system includes an acquisition module, a segmentation module, a selection module and a calculation module: The acquisition module is used to obtain the RGB image and depth image of the stairs; The segmentation module is used to segment the vertical plane and / or horizontal plane in the RGB image; The selection module is used to select a vertical plane and / or a horizontal plane to establish a circumscribed rectangular frame, and select two relative points on the upper and lower boundary lines of the circumscribed rectangular frame of the vertical plane and / or the horizontal plane as a first point pair and / or a second point pair; The calculation module is used to obtain the Euclidean distance of the first point pair and / or the second point pair according to the RGB image and the depth image; The Euclidean distance of the first pair of points is the height of the vertical plane of the stairs, and the Euclidean distance of the second pair of points is the width of the horizontal plane of the stairs; When the depth value of the first point pair or the second point pair is not 0, obtaining the Euclidean distance of the first point pair or the second point pair based on the RGB image and the depth image; When the depth value of the first point pair is 0, the Euclidean distance between two first reference points whose depth values ​​are not 0 on the line connecting the first point pair is obtained, and the Euclidean distance of the first point pair is obtained based on the Euclidean distance of the first reference point pair; Alternatively, when the depth value of the second point pair is 0, the Euclidean distance between two second reference points on the line connecting the second point pair whose depth values ​​are not 0 is obtained, and the Euclidean distance of the second point pair is obtained based on the Euclidean distance of the second reference point pair; The step of obtaining the Euclidean distance of the first point pair based on the Euclidean distance of the first reference point pair is specifically as follows: obtaining the ratio of the plane coordinate distance of the first reference point pair to the plane coordinate distance of the first point pair based on the RGB image, defining the ratio as a first scaling ratio, and obtaining the Euclidean distance of the first point pair based on the Euclidean distance of the first reference point pair and the first scaling ratio; Among them, the step of obtaining the Euclidean distance of the second point pair based on the Euclidean distance of the second reference point pair is specifically: obtaining the ratio of the plane coordinate distance of the second reference point pair to the plane coordinate distance of the second point pair based on the RGB image, defining the ratio as a second scaling ratio, and obtaining the Euclidean distance of the second point pair based on the Euclidean distance of the second reference point pair and the second scaling ratio.

8. A quadruped robot, characterized in that: The invention comprises a depth camera and a controller, wherein the lens of the depth camera is aimed at the front of the quadruped robot, the depth camera is connected to the controller, and after receiving the RGB image and the depth image output by the depth camera, the controller measures the stair parameters according to the stair measurement method based on depth vision described in one of claims 1 to 6.

Images