A method and system for detecting stair height based on depth images

Abstract

The application discloses a kind of based on depth image's stair height detection method and system, belong to computer vision technical field, method includes: obtaining depth map and carrying out filling-in preprocessing to it;After standardization to the depth map after filling-in, project to one-dimensional imaging plane;Gradient detection is carried out to the depth map after projection, obtain depth local maximum and depth local minimum;Select the two points of the same horizontal coordinate in depth map, vertical coordinate is depth local maximum and depth local minimum respectively as two edge key points, the three-dimensional coordinates of two edge key points are calculated by camera projection algorithm, the distance of two edge key points is calculated according to two three-dimensional coordinates, obtain the height of stair;Combine pixel information and depth information, convert pixel coordinate system and camera coordinate system, obtain the height of stair, overcome the defects that existing conversion method has error, large amount of calculation, coordinate conversion is complex, to improve detection speed.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of based on depth image's stair height detection method and system, belong to computer vision technical field. BACKGROUND

[0002] Three-dimensional information acquisition of object is an important issue in computer vision field, which aims to detect the two-dimensional information of object from image, and project it to three-dimensional space, finally calculate its three-dimensional information;Three-dimensional information of object can more fully, truly reflect objective object than two-dimensional information, and provide more information quantity for people.

[0003] At present, the method for obtaining three-dimensional information of object is roughly divided into the following two kinds:

[0004] The first kind is contact measurement method, its principle is that the probe of sampling head contacts model surface, collects the data of a profile point, then moves a pitch horizontally, collects the data of adjacent profile point, finally constructs the wireframe model of entire surface;This method has higher accuracy and reliability, but also has problems such as unable to measure flexible object, slow measurement speed, limited to fixed environment, etc.

[0005] The second kind is three-dimensional sensor based on non-structured light, its principle is to determine distance information from two-dimensional image obtained by one or more camera systems, form three-dimensional surface data;This method needs a large number of related matching operations, when the structure information of measured target is too simple or too complex, or the reflectivity of each point on the measured object has no obvious difference, such related matching operation will become very complex and difficult.

[0006] Therefore, in the current technology for detecting stair height, there are the following defects: (1) slow detection speed, limited to fixed environment;(2) matching operation is extremely complex, difficult to calculate. SUMMARY

[0007] The purpose of the present application is to provide a kind of based on depth image's stair height detection method and system, solve the problems such as slow detection speed, serious environment limitation, complex and difficult calculation in prior art.

[0008] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0009] In a first aspect, the present application provides a kind of based on depth image's stair height detection method, comprising:

[0010] obtain depth map and carry out filling-in pre-processing to it;

[0011] after filling-in the depth map, standardization is carried out and then projected to one-dimensional imaging plane;

[0012] Gradient detection is performed on the projected depth map to obtain a depth local maximum and a depth local minimum;

[0013] Two edge key points are selected from the depth map, which have the same horizontal coordinate and different vertical coordinates of the depth local maximum and the depth local minimum, and three-dimensional coordinates of the two edge key points are calculated by using a camera projection algorithm, a distance between the two edge key points is calculated according to the two three-dimensional coordinates, and a stair height is obtained.

[0014] With reference to the first aspect, further, the supplementing preprocessing comprises:

[0015] The depth value interpolation is performed on the depth value missing area in the depth map.

[0016] With reference to the first aspect, further, the depth map is normalized by using a range standardization method.

[0017] With reference to the first aspect, further, the gradient detection comprises:

[0018] The CV2.sobel function is used to perform gradient detection on the projected depth map to obtain a gradient map, and a horizontal coordinate direction mean value of the gradient map is calculated to obtain a one-dimensional array, and the depth local maximum and the depth local minimum are found in the one-dimensional array.

[0019] With reference to the first aspect, further, an expression of the camera projection algorithm is as follows:

[0020]

[0021] wherein, u and v are horizontal and vertical coordinates of the edge key points, is a camera intrinsic matrix, f x and f y respectively represent focal lengths of the camera in x and y directions, c x and c y respectively represent offsets of the camera pixels in x and y directions, X c , Y c and Z c respectively represent three-dimensional coordinate values corresponding to the edge key points.

[0022] With reference to the first aspect, further, the distance between the two edge key points is calculated according to the two three-dimensional coordinates by using the Pythagorean theorem.

[0023] With reference to the first aspect, further, the depth map is obtained by using a RealSense image acquisition module.

[0024] In the second aspect, the present application further provides a stair height detection system based on a depth image, comprising:

[0025] Depth map acquisition module: for acquiring a depth map and pre-processing it for filling in the gaps;

[0026] Projection module: for projecting the depth map after filling in the gaps to a one-dimensional imaging plane after standardization;

[0027] Gradient detection module: for gradient detection on the projected depth map to obtain depth local maxima and depth local minima;

[0028] Stair height calculation module: for selecting two points with the same horizontal coordinate and vertical coordinates of depth local maxima and depth local minima in the depth map as two edge key points, calculating the three-dimensional coordinates of the two edge key points through a camera projection algorithm, and calculating the distance between the two edge key points to obtain the stair height.

[0029] Compared with the prior art, the present application has the following advantages:

[0030] The stair height detection method and system based on a depth image provided by the present application perform gradient detection on the projected depth map to obtain depth local maxima and depth local minima, obtain the boundary information of the stairs, select edge key points using the depth local maxima and depth local minima, and calculate the three-dimensional coordinates of the edge key points through a camera projection algorithm, i.e., convert the pixel coordinate system and the camera coordinate system by combining the pixel information and the depth information, and finally obtain the height of the stairs, effectively overcoming the problems of errors in the conversion method, large amount of calculation, and complex coordinate conversion in the prior art, thereby improving the detection speed and not requiring fixed configuration environment coding, and having high practical value and social value in tasks such as computer-aided design and manufacturing (CAD / CAM), reverse engineering (RE), rapid prototyping (RP), and virtual reality (VR). BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a flowchart of a stair height detection method based on a depth image provided by an embodiment of the present application. DETAILED DESCRIPTION

[0032] The present application will be further described below with reference to the accompanying drawings, and the following examples are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0033] Example 1

[0034] As shown in the figure, the stair height detection method based on a depth image provided by an embodiment of the present application comprises: Figure 1

[0035] S1, acquire a depth map and pre-process it for filling in the gaps. ​

[0036] The depth map of the stairs is acquired by a RealSense image acquisition module, and the depth value interpolation completion is performed on the missing area of the depth value in the depth map to complete the missing pre-processing of the depth map.

[0037] S2, projecting the depth map after the missing to a one-dimensional imaging plane after standardization.

[0038] The depth map after the completion of the depth value is standardized, and the specific method is range standardization method, and the purpose of standardization is that the current image is 16-bit int16, in the interval of 0-25536, the current pixel value represents the depth value, but the CV2.sobel function used for gradient detection is suitable for 8-bit int8, in the interval of 0-255.

[0039] The standardized depth map is projected to the interval of 0-255, which is used for subsequent gradient detection.

[0040] S3, gradient detection is performed on the projected depth map to obtain depth local maximum and depth local minimum.

[0041] The CV2.sobel function built in CV2 is used to perform gradient detection on the projected depth map, because the pixel value in the depth map represents the depth information, so the local extreme value of the depth change in the depth map can be detected.

[0042] The CV2.sobel function is used to perform gradient detection on the projected depth map to obtain a gradient map, and the mean value of the horizontal coordinate direction of the gradient map is obtained to obtain a one-dimensional array, and the depth local maximum and the depth local minimum are found in the one-dimensional array.

[0043] S4, selecting two points with the same horizontal coordinate and the vertical coordinates of the depth local maximum and the depth local minimum in the depth map as two edge key points, calculating the three-dimensional coordinates of the two edge key points through the camera projection algorithm, and calculating the distance between the two edge key points according to the two three-dimensional coordinates to obtain the height of the stairs.

[0044] After obtaining the depth local maximum and the depth local minimum, the depth image pixel coordinates are projected to three-dimensional coordinates, and the projection is performed as follows:

[0045] Selecting two points with the same horizontal coordinate and the vertical coordinates of the depth local maximum and the depth local minimum in the depth map as a group of edge key points.

[0046] The camera intrinsic matrix is obtained, and the specific form is:

[0047]

[0048] Where, f x and fy f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively. x f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively. y f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively.

[0049] Then the three-dimensional coordinates of each edge key point in the group are calculated by the camera projection algorithm:

[0050]

[0051] Wherein, u and v are the horizontal and vertical coordinates of the edge key point respectively, is the camera intrinsic matrix, f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively. x f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively. y f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively. x f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively. y f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively. c f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively. c f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively. c f x and f y represent the focal length of the camera in x and y direction respectively, c x and c y represent the offset of the camera pixel in x and y direction respectively.

[0052] After obtaining two three-dimensional coordinates in the group of edge key points, the distance between the two points is calculated by the Pythagorean theorem, that is, the height of the stairs is obtained:

[0053]

[0054] Wherein, D represents the distance between the two points, (X1, Y1, Z1) and (X2, Y2, Z2) represent the three-dimensional coordinates of the two points respectively.

[0055] In this embodiment, a plurality of groups of edge key points are selected for calculation, and the average value of all stair heights is obtained after the stair heights are obtained, so that the detection result of the final stair height is obtained.

[0056] Embodiment 2

[0057] The stair height detection system based on a depth image provided by the embodiment of the application comprises:

[0058] A depth map acquisition module is configured to acquire a depth map and perform a gap filling preprocessing on the depth map.

[0059] A projection module is configured to project the gap filled depth map to a one-dimensional imaging plane after standardization.

[0060] A gradient detection module is configured to perform gradient detection on the projected depth map to obtain a depth local maximum value and a depth local minimum value.

[0061] The stair height calculation module is configured to select two points with the same horizontal coordinate and different local maximum and local minimum depth as two edge key points in the depth image, calculate the three-dimensional coordinates of the two edge key points by using a camera projection algorithm, and calculate the distance between the two edge key points according to the two three-dimensional coordinates to obtain the stair height.

[0062] Those skilled in the art will understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0063] The present application is described with reference to the flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the functions specified in the flowchart

[0064] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the functions specified in the flowchart

[0065] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the functions specified in the flowchart

[0066] The above merely describes the preferred embodiments of the present application, and it should be pointed out that, for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A method for detecting a stair height based on a depth image, characterized by, The method comprises the following steps: obtaining a depth map and performing a filling-in preprocessing on the depth map; projecting the filling-in preprocessed depth map to a one-dimensional imaging plane after standardization; performing gradient detection on the projected depth map to obtain a depth local maximum value and a depth local minimum value; selecting two points with the same horizontal coordinate and the depth local maximum value and the depth local minimum value as two edge key points in the depth map, calculating three-dimensional coordinates of the two edge key points through a camera projection algorithm, and calculating a distance between the two edge key points according to the two three-dimensional coordinates to obtain a stair height; the gradient detection comprises the following steps: performing gradient detection on the projected depth map by using a CV2.sobel function to obtain a gradient map, calculating a mean value in a horizontal coordinate direction of the gradient map to obtain a one-dimensional array, and finding the depth local maximum value and the depth local minimum value in the one-dimensional array.

2. The method of claim 1, wherein, the filling-in preprocessing comprises the following steps: performing interpolation completion on a depth value in a depth value missing area in the depth map.

3. The method of claim 1, wherein, the depth map is standardized by a range standardization method.

4. The method of claim 1, wherein, an expression of the camera projection algorithm is as follows: ； wherein, and are the horizontal and vertical coordinates of the edge key point, respectively, is a camera intrinsic matrix, and are the focal lengths of the camera in the and directions, respectively, and are the offsets of the camera pixel in the and directions, respectively, , and are the three-dimensional coordinate values corresponding to the edge key point, respectively.

5. The method of claim 1, wherein, the distance between the two edge key points is calculated according to the two three-dimensional coordinates through the Pythagorean theorem.

6. The method of claim 1, wherein, the depth map is obtained by a RealSense image acquisition module.

7. A depth image based stair height detection system, characterized by, The method comprises the following steps: a depth map acquisition module is configured to obtain a depth map and perform a filling-in preprocessing on the depth map; a projection module is configured to project the filling-in preprocessed depth map to a one-dimensional imaging plane after standardization; a gradient detection module is configured to perform gradient detection on the projected depth map to obtain a depth local maximum value and a depth local minimum value; a stair height calculation module is configured to select two points with the same horizontal coordinate and the depth local maximum value and the depth local minimum value as two edge key points in the depth map, calculate three-dimensional coordinates of the two edge key points through a camera projection algorithm, and calculate a distance between the two edge key points according to the two three-dimensional coordinates to obtain a stair height; wherein the gradient detection comprises the following steps: performing gradient detection on the projected depth map by using a CV2.sobel function to obtain a gradient map, calculating a mean value in a horizontal coordinate direction of the gradient map to obtain a one-dimensional array, and finding the depth local maximum value and the depth local minimum value in the one-dimensional array.

Images