Evaluation methods and related equipment for the depth performance of depth cameras

Abstract

This disclosure provides a method, apparatus, computer device, readable storage medium, and program for evaluating the depth performance of a depth camera, relating to the field of depth camera technology. The method includes: acquiring a depth image of a stepped test object and corresponding point cloud data using a depth camera; wherein the stepped test object includes N planes, where N is an integer greater than or equal to 2; acquiring the region of interest (ROI) for each of the N planes in the depth image; acquiring the average point cloud depth of the ROI in each of the N planes; and determining the depth performance of the depth camera based on the average point cloud depth of the ROIs between the N planes. The method provided by this disclosure can evaluate the depth performance of a depth camera.

Description

Technical Field

[0001] This disclosure relates to the field of depth camera technology, and in particular to a method, apparatus, computer device, readable storage medium, and program for evaluating the depth performance of a depth camera. Background Technology

[0002] Depth cameras acquire depth information of target objects and scenes through visual imaging, enabling 3D reconstruction, target recognition, and depth measurement. Utilizing depth information from depth cameras can enrich the functionality of devices and improve the performance of the vision system in products or systems, such as facial recognition, navigation, and AI-powered motion sensing. The accuracy of depth information plays a crucial role in the application of depth cameras. If the accuracy of the depth information is too weak, the application of functions such as scene reconstruction, target recognition, and obstacle avoidance will be significantly reduced, or even completely rendered unusable, making practical applications impossible. Summary of the Invention

[0003] This disclosure provides a method, apparatus, computer device, readable storage medium, and program for evaluating the depth performance of a depth camera, relating to the field of depth camera technology. This method can evaluate the depth performance of a depth camera.

[0004] This disclosure provides a method for evaluating the depth performance of a depth camera, comprising: acquiring a depth image of a stepped test body and point cloud data corresponding to the depth image using the depth camera; wherein the stepped test body includes N planes, where N is an integer greater than or equal to 2; acquiring the region of interest (ROI) of each of the N planes in the depth image; acquiring the average point cloud depth of the ROI of each of the N planes; and determining the depth performance of the depth camera based on the average point cloud depth of the ROIs between the N planes.

[0005] In one embodiment, obtaining the average depth of the point cloud in the region of interest of each of the N planes includes: obtaining the pixel coordinates and depth value of each point in the point cloud of the region of interest of each of the N planes; obtaining the world coordinate value of the corresponding point based on the pixel coordinates and depth value of each point; and obtaining the average depth of the point cloud in the region of interest of each plane based on the world coordinate value of each point in the point cloud of the region of interest of each of the N planes.

[0006] In one embodiment, obtaining the world coordinates of a point based on its pixel coordinates and depth value includes: using the depth-to-point-cloud formula, and assuming that the origin of the world coordinate system coincides with the origin of the camera coordinate system, to obtain the conversion formula between the pixel coordinates and depth value of each point and the world coordinates of the corresponding point.

[0007] In one embodiment, obtaining the average point cloud depth of the region of interest of each plane based on the world coordinates of each point in the point cloud of the region of interest of each of the N planes includes: obtaining the average point cloud depth of the region of interest of each plane based on the world coordinates of each point in the point cloud of the region of interest of each of the N planes using the least squares method.

[0008] In one embodiment, obtaining the average depth of the point cloud in the region of interest of each plane using the least squares method based on the world coordinates of each point in the point cloud of the region of interest of each of the N planes includes: performing plane fitting based on the world coordinates of each point in the point cloud of the first region of interest to obtain the plane equation of the plane formed by the point cloud in the first region of interest, wherein the first region of interest is the region of interest of any one of the N planes; substituting the X-axis and Y-axis coordinates of the world coordinates of each point in the point cloud of the first region of interest into the plane equation of the first region of interest to obtain the updated Z-axis coordinate value; and taking the average of the updated Z-axis coordinates of the world coordinates of all points in the point cloud of the first region of interest as the average depth of the point cloud of the first region of interest.

[0009] In one embodiment, determining the depth performance of the depth camera based on the average point cloud depth of the region of interest between the N planes includes: comparing the difference between the average point cloud depth of the region of interest of the first plane and the average point cloud depth of the region of interest of the second plane in the depth image with the vertical distance between the first plane and the second plane of the physical object of the stepped test body to determine the depth accuracy of the depth camera, wherein the first plane and the second plane are any two planes among the N planes.

[0010] In one embodiment, the vertical distance between the planes of the stepped test body includes at least one or more of the following: nanometer, micrometer, centimeter, filament, millimeter, centimeter, or decimeter.

[0011] This disclosure provides an evaluation device for the depth performance of a depth camera, comprising: a first acquisition unit, configured to acquire a depth image of a stepped test object and point cloud data corresponding to the depth image using the depth camera; wherein the stepped test object includes N planes, where N is an integer greater than or equal to 2; a second acquisition unit, configured to acquire the region of interest (ROI) of each of the N planes in the depth image; a third acquisition unit, configured to acquire the average point cloud depth of the ROI of each of the N planes; and a judgment unit, configured to judge the depth performance of the depth camera based on the average point cloud depth of the ROI between the N planes.

[0012] This disclosure provides a computer device including a processor, a memory, and an input / output interface; the processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive data and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device performs the method as described in any of the above embodiments.

[0013] This disclosure provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, such that a computer device having the processor performs the method described in any of the above embodiments.

[0014] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the above embodiments.

[0015] The method for evaluating the depth performance of a depth camera disclosed herein involves acquiring a depth image of a stepped test object and corresponding point cloud data using the depth camera; wherein the stepped test object comprises N planes, where N is an integer greater than or equal to 2; obtaining the region of interest (ROI) of each of the N planes in the depth image; obtaining the average point cloud depth of the ROI of each of the N planes; and determining the depth performance of the depth camera based on the average point cloud depth of the ROIs between the N planes, thereby enabling the evaluation of the depth performance of the depth camera. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram of an exemplary system architecture for evaluating the depth performance of a depth camera to which embodiments of the present disclosure can be applied is shown;

[0018] Figure 2 This is a three-dimensional structural schematic diagram of a stepped test body provided in an embodiment of this disclosure;

[0019] Figure 3 This is a side view schematic diagram of the structure of a stepped test body provided in an embodiment of this disclosure;

[0020] Figure 4This is a flowchart of a method for evaluating the depth performance of a depth camera according to an embodiment of this disclosure;

[0021] Figure 5 This is a schematic diagram of an embodiment of the present disclosure providing a method for acquiring a depth image of a stepped test object and the corresponding point cloud data using a depth camera;

[0022] Figure 6 This is a schematic diagram of the region of interest for each of N planes in a depth image provided by an embodiment of this disclosure;

[0023] Figure 7 This is a flowchart of a method for obtaining the average point cloud depth in the region of interest of each of the N planes, according to an embodiment of this disclosure;

[0024] Figure 8 This is a diagram illustrating the mapping process of camera world coordinates according to an embodiment of this disclosure;

[0025] Figure 9 This is a flowchart of a method provided in this disclosure for obtaining the average depth of the point cloud in the region of interest of each of the N planes by using the least squares method based on the world coordinates of each point in the point cloud in the region of interest of each plane;

[0026] Figure 10 This is a flowchart of a method for determining the depth performance of a depth camera based on the average point cloud depth of the region of interest between the N planes, according to an embodiment of this disclosure.

[0027] Figure 11 This is a schematic diagram of the structure of a depth camera depth performance evaluation device provided in an embodiment of this disclosure;

[0028] Figure 12 This is a schematic diagram of the structure of a computer device that implements the method for evaluating the depth performance of a depth camera provided in the embodiments of this disclosure. Detailed Implementation

[0029] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0030] In this embodiment of the disclosure, a depth image of a stepped test object and point cloud data corresponding to the depth image can be acquired by a depth camera; wherein, the stepped test object includes N planes, where N is an integer greater than or equal to 2; the region of interest of each of the N planes in the depth image is acquired; the average point cloud depth of the region of interest of each of the N planes is acquired; the depth performance of the depth camera is determined based on the average point cloud depth of the region of interest between the N planes, thereby enabling the evaluation of the depth performance of the depth camera.

[0031] The following is a brief explanation of some of the terms used in this disclosure:

[0032] Depth cameras (3D cameras) can detect distance information in the shooting space, which is the biggest difference between them and ordinary cameras. Ordinary color cameras capture and record all objects within their field of view, but the recorded data does not include the distances of these objects to the camera. They can only determine which objects are far away and which are near through semantic analysis of the image, without precise data. 3D cameras solve this problem. The data acquired by a 3D camera accurately determines the distance of each point in the image from the camera. Adding this to the two-dimensional coordinates of that point in the 2D image, the three-dimensional spatial coordinates of each point in the image can be obtained. These three-dimensional coordinates can then be used to reconstruct the real scene, enabling applications such as scene modeling.

[0033] Structured light is a system consisting of a projector and a camera. Specific light information is projected onto the surface of an object and its background by the projector, and then captured by the camera. Based on the changes in the light signal caused by the object, information such as the object's position and depth is calculated, thus reconstructing the entire three-dimensional space.

[0034] The solutions provided in this disclosure involve technologies such as depth cameras and structured light.

[0035] Figure 1 A schematic diagram of an exemplary system architecture 100 to which the depth performance evaluation method of the depth camera of the present disclosure can be applied is shown.

[0036] like Figure 1 As shown, system architecture 100 may include one or more of terminals 101, 102, and 103, a network 104, and a server 105. Network 104 is the medium used to provide a communication link between terminals 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0037] It should be understood that Figure 1The number of terminals, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminals, networks, and servers. For example, server 105 could be a server cluster consisting of multiple servers.

[0038] Terminals 101, 102, and 103 interact with server 105 via network 104, and can receive or send messages, etc. Terminals 101, 102, and 103 can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers, etc.

[0039] Server 105 can be a server providing various services. For example, after terminal 103 (or terminal 101 or 102) sends a request to server 105 to evaluate the depth performance of a depth camera, server 105 can acquire a depth image of a stepped test body and the corresponding point cloud data through the depth camera; wherein, the stepped test body includes N planes, where N is an integer greater than or equal to 2; the region of interest of each of the N planes in the depth image is acquired; the average point cloud depth of the region of interest of each of the N planes is acquired; and the depth performance of the depth camera is judged based on the average point cloud depth of the region of interest between the N planes, thereby realizing the evaluation of the depth performance of the depth camera.

[0040] The terminal can be a mobile phone (such as terminal 101), a tablet computer (such as terminal 102), or a desktop computer (such as terminal 101), etc., without limitation. The terminal can display an application, which could be an application evaluating the depth performance of the depth camera, etc. Figure 1 The terminals mentioned are only a portion of the devices exemplified; in this disclosure, terminals are not limited to these. Figure 1 The equipment listed in the text.

[0041] The field of depth camera technology has a wide range of applications, and the requirements for depth accuracy of depth cameras are not the same. The evaluation method of depth camera depth accuracy requires a variety of scenarios to evaluate the performance quality of depth cameras.

[0042] Figure 2 This is a three-dimensional structural schematic diagram of a stepped test body provided in an embodiment of this disclosure.

[0043] Figure 3 This is a side view schematic diagram of the structure of a stepped test body provided in an embodiment of this disclosure.

[0044] refer to Figure 2 and Figure 3The stepped test body of this application comprises N planes, where N is an integer greater than or equal to 2. The vertical distance between the planes of the stepped test body includes at least one or more of the following: nanometer, micrometer, centimeter, filament, millimeter, centimeter, or decimeter. For example, a distance of 1 millimeter between plane 1 and plane 2 can be used to test the accuracy of a depth camera at a depth of millimeters.

[0045] Figure 4 This is a flowchart illustrating a method for evaluating the depth performance of a depth camera according to an embodiment of this disclosure. The method provided in this embodiment can be... Figure 1 The methods described in this embodiment are executed by a terminal or server, or by interaction between a terminal and a server. However, this disclosure is not limited thereto; the methods of this disclosure can be executed by any processor with computing power.

[0046] like Figure 4 As shown, the method provided in this disclosure embodiment may include the following steps.

[0047] In step S410, a depth image of the stepped test body and the corresponding point cloud data are acquired by a depth camera; wherein the stepped test body includes N planes, and N is an integer greater than or equal to 2.

[0048] In this step, the terminal or server acquires the depth image of the stepped test object and the corresponding point cloud data through a depth camera; wherein, the stepped test object includes N planes, where N is an integer greater than or equal to 2.

[0049] Figure 5 This is a schematic diagram of an embodiment of the present disclosure that uses a depth camera to acquire a depth image of a stepped test object and the corresponding point cloud data of the depth image.

[0050] refer to Figure 5 The depth camera faces the stepped test object, which is centered in the depth camera's field of view (FOV). The depth camera's optical axis (Zc) is perpendicular to the Xs and Ys directions of the stepped test object, and parallel to the Zs direction of the stepped test object. The perpendicular distance between the depth camera and the surface of the nearest step N can be, for example, 300mm or 500mm (the distance can be customized as needed). A static depth map and corresponding point cloud data are acquired using the depth camera.

[0051] In step S420, the region of interest for each of the N planes in the depth image is obtained.

[0052] In this step, the terminal or server acquires the region of interest for each of the N planes in the depth image.

[0053] Figure 6This is a schematic diagram of the region of interest for each of N planes in a depth image provided in an embodiment of this disclosure.

[0054] refer to Figure 6 For point cloud data of a stepped test body with N planes, according to Figure 6 For the stepped test surface shown in the top view (1, 2, 3...N), select N regions of interest (ROIs) of equal length and width in the u and v directions. On the depth image, select the four vertices 11, 12, 13, 14 of the first selected plane to form a rectangle and obtain its coordinates. Project these coordinates onto the corresponding points in the point cloud. Using the coordinate results, calculate the length of the point cloud in the u direction: u12 - u11, and the width in the v direction: v14 - v11. Then, select ROIs of equal length and width on the N stepped surfaces.

[0055] In step S430, the average point cloud depth of the region of interest in each of the N planes is obtained.

[0056] In this step, the terminal or server obtains the average point cloud depth in the region of interest of each of the N planes.

[0057] In step S440, the depth performance of the depth camera is determined based on the average point cloud depth of the region of interest between the N planes.

[0058] In this step, the terminal or server determines the depth performance of the depth camera based on the average point cloud depth of the region of interest between the N planes.

[0059] Figure 4 The method for evaluating the depth performance of a depth camera, as shown, involves acquiring a depth image of a stepped test object and corresponding point cloud data using the depth camera. The stepped test object comprises N planes, where N is an integer greater than or equal to 2. The method obtains the region of interest (ROI) for each of the N planes in the depth image. It also obtains the average point cloud depth of the ROI for each of the N planes. Finally, it determines the depth performance of the depth camera based on the average point cloud depth of the ROIs between the N planes, thus enabling the evaluation of the depth camera's depth performance.

[0060] Figure 7 This is a flowchart illustrating a method for obtaining the average point cloud depth in the region of interest of each of the N planes, as provided in this disclosure embodiment. The method provided in this disclosure embodiment can be... Figure 1 The methods described in this embodiment are executed by a terminal or server, or by interaction between a terminal and a server. However, this disclosure is not limited thereto; the methods of this disclosure can be executed by any processor with computing power.

[0061] like Figure 7 As shown, the method provided in this disclosure embodiment may include the following steps.

[0062] In step S710, the pixel coordinates and depth values ​​of each point in the point cloud of the region of interest of each of the N planes are obtained.

[0063] In this step, the terminal or server obtains the pixel coordinates and depth value of each point in the point cloud of the region of interest of each of the N planes.

[0064] In step S720, the world coordinates of the corresponding point are obtained based on the pixel coordinates and depth value of each point.

[0065] In this step, the terminal or server obtains the world coordinates of the corresponding point based on the pixel coordinates and depth value of each point.

[0066] Figure 8 This is a diagram illustrating the mapping process of camera world coordinates provided in an embodiment of this disclosure.

[0067] refer to Figure 8 The four coordinate systems represented are the world coordinate system, the camera coordinate system, the image pixel coordinate system, and the image plane coordinate system.

[0068] The system comprises four coordinate systems: world coordinate system (Xw, Yw, Zw), camera coordinate system (Xc, Yc, Zc), image pixel coordinate system (uv), and image plane coordinate system (Xp, Yp). The world coordinate system (Xw, Yw, Zw) describes the coordinate positions of objects and the camera in three-dimensional space. The camera coordinate system (Xc, Yc, Zc) uses the camera's optical center as its origin, with the Zc axis coinciding with the optical axis and perpendicular to the imaging plane, taking the shooting direction as the positive direction. The Xc and Yc axes are parallel to the x and y axes of the image plane coordinate system, and the distance between Oc and Op is the focal length f. The image pixel coordinate system (uv) uses the upper left corner of the image as its origin and is measured in pixels. The image plane coordinate system (Xp, Yp) uses the intersection of the optical axis and the image as its origin and is measured in millimeters. M represents a point in space, and m is the image point of M on the image.

[0069] Where u and v are arbitrary coordinate points in the image pixel coordinate system, u0 and v0 are the center coordinates of the image in the image pixel coordinate system, Xw, Yw and Zw represent three-dimensional coordinate points in the world coordinate system, and Zc represents the Z-axis value of the camera coordinate system, that is, the distance (depth value) from the target to the camera.

[0070] Where R and T are the 3x3 rotation matrix and 3x1 translation matrix of the extrinsic parameter matrix, respectively, f is the focal length, and dx and dy are the intrinsic parameters of the camera. These parameters can be obtained after calibration and are known.

[0071] In one embodiment, obtaining the world coordinates of a point based on its pixel coordinates and depth value includes: using the depth-to-point-cloud formula, and assuming that the origin of the world coordinate system coincides with the origin of the camera coordinate system, to obtain the conversion formula between the pixel coordinates and depth value of each point and the world coordinates of the corresponding point.

[0072] The formula for converting depth to point cloud is as follows: (1)

[0073]

[0074] In formula (1), Zc represents the distance (depth value) from the target to the camera, u and v are any coordinate points in the image pixel coordinate system, u0 and v0 are the center coordinates of the image in the image pixel coordinate system, Xw, Yw and Zw represent the three-dimensional coordinate points in the world coordinate system, R and T are the 3x3 rotation matrix and 3x1 translation matrix of the extrinsic parameter matrix, respectively, f is the focal length, and dx and dy are the intrinsic parameters of the camera.

[0075] When the origin of the world coordinate system coincides with the origin of the camera coordinate system, formula (1) can be transformed into formula (2):

[0076]

[0077] According to formula (2), the conversion formula (3) between pixel coordinates and depth values ​​and the world coordinates of the corresponding points can be obtained:

[0078]

[0079] In step S730, the average depth of the point cloud in the region of interest of each plane is obtained based on the world coordinates of each point in the point cloud in the region of interest of each of the N planes.

[0080] In this step, the terminal or server obtains the average depth of the point cloud in the region of interest of each plane based on the world coordinates of each point in the point cloud in the region of interest of each of the N planes.

[0081] In one embodiment, the average depth of the point cloud in the region of interest of each plane is obtained by using the least squares method, based on the world coordinates of each point in the point cloud in the region of interest of each of the N planes.

[0082] Figure 9 This is a flowchart illustrating a method for obtaining the average depth of the point cloud in the region of interest of each of the N planes using the least squares method, based on the world coordinates of each point in the point cloud within the region of interest of each plane. The method provided in this disclosure can be... Figure 1The methods described in this embodiment are executed by a terminal or server, or by interaction between a terminal and a server. However, this disclosure is not limited thereto; the methods of this disclosure can be executed by any processor with computing power.

[0083] like Figure 9 As shown, the method provided in this disclosure embodiment may include the following steps.

[0084] In step S910, a plane fitting is performed based on the world coordinates of each point in the point cloud of the first region of interest to obtain the plane equation of the plane formed by the point cloud in the first region of interest, wherein the first region of interest is the region of interest of any one of the N planes.

[0085] In this step, the terminal or server performs plane fitting based on the world coordinates of each point in the point cloud of the first region of interest to obtain the plane equation of the plane formed by the point cloud in the first region of interest, wherein the first region of interest is the region of interest of any one of the N planes.

[0086] Among them, the least squares method is used to perform planar fitting on the first region of interest. The matrix form of the least squares method is given by formula (4):

[0087] Em = f (4)

[0088] E: an n*k matrix, m: a k*1 column vector, and f: an n*1 column vector.

[0089] The general expression for the plane equation is formula (5):

[0090] Ax + By + Cz + D = 0 (C ≠ 0) (5)

[0091] The transformation is shown in the following formula (6):

[0092]

[0093] make: Formula (7) can be obtained:

[0094] z = a0x + a1y + a2 (7)

[0095] The corresponding least squares method yields formula (8):

[0096]

[0097] After substituting the world coordinates of each point in the point cloud of the first region of interest into formula (8), the solution can be obtained:

[0098] a0, a1, a2.

[0099] In step S920, the X-axis and Y-axis coordinates of each point in the point cloud of the first region of interest are substituted into the plane equation of the first region of interest to obtain the updated Z-axis coordinate value.

[0100] In this step, the terminal or server substitutes the X-axis and Y-axis coordinates of each point in the point cloud of the first region of interest into the plane equation (7) of the first region of interest to obtain the updated Z-axis coordinate value.

[0101] In step S930, the average value of the updated Z-axis coordinates of all points in the point cloud of the first region of interest is taken as the average depth of the point cloud of the first region of interest.

[0102] In this step, the terminal or server uses the average of the updated Z-axis coordinates of all points in the point cloud of the first region of interest as the average depth of the point cloud of the first region of interest.

[0103] Figure 10 This is a flowchart illustrating a method for determining the depth performance of a depth camera based on the average point cloud depth of a region of interest among N planes, as provided in this disclosure. The method provided in this disclosure can be used by... Figure 1 The methods described in this embodiment are executed by a terminal or server, or by interaction between a terminal and a server. However, this disclosure is not limited thereto; the methods of this disclosure can be executed by any processor with computing power.

[0104] like Figure 10 As shown, the method provided in this disclosure embodiment may include the following steps.

[0105] In step S1010, the difference between the average point cloud depth of the region of interest in the first plane and the average point cloud depth of the region of interest in the second plane in the depth image is compared with the vertical distance between the first plane and the second plane of the physical object of the stepped test body to determine the depth accuracy of the depth camera, wherein the first plane and the second plane are any two planes among the N planes.

[0106] In this step, the terminal or server compares the difference between the average point cloud depth of the region of interest in the first plane and the average point cloud depth of the region of interest in the second plane in the depth image with the vertical distance between the first plane and the second plane of the physical object of the stepped test body to determine the depth accuracy of the depth camera, wherein the first plane and the second plane are any two planes among the N planes.

[0107] Figure 11This is a schematic diagram of the structure of a depth camera depth performance evaluation device provided in an embodiment of this disclosure.

[0108] like Figure 11 As shown, the depth performance evaluation device 1100 for a depth camera provided in this embodiment may include:

[0109] The first acquisition unit 1110 is used to acquire a depth image of the stepped test body and the point cloud data corresponding to the depth image through a depth camera; wherein, the stepped test body includes N planes, and N is an integer greater than or equal to 2;

[0110] The second acquisition unit 1120 is used to acquire the region of interest of each of the N planes in the depth image;

[0111] The third acquisition unit 1130 is used to acquire the average point cloud depth in the region of interest of each of the N planes;

[0112] The judgment unit 1140 is used to judge the depth performance of the depth camera based on the average point cloud depth of the region of interest between the N planes.

[0113] Figure 11 An evaluation device for the depth performance of a depth camera includes a first acquisition unit for acquiring a depth image of a stepped test object and corresponding point cloud data using the depth camera; wherein the stepped test object comprises N planes, where N is an integer greater than or equal to 2; a second acquisition unit for acquiring the region of interest (ROI) of each of the N planes in the depth image; a third acquisition unit for acquiring the average point cloud depth of the ROI of each of the N planes; and a judgment unit for judging the depth performance of the depth camera based on the average point cloud depth of the ROI between the N planes, thereby enabling the evaluation of the depth performance of the depth camera.

[0114] In one embodiment, the third acquisition unit 1130 is further configured to acquire the pixel coordinates and depth values ​​of each point in the point cloud of the region of interest of each of the N planes; acquire the world coordinate values ​​of the corresponding point based on the pixel coordinates and depth values ​​of each point; and acquire the average depth of the point cloud in the region of interest of each plane based on the world coordinate values ​​of each point in the point cloud of the region of interest of each of the N planes.

[0115] In one embodiment, the third acquisition unit 1130 is further configured to obtain the conversion formula between the pixel coordinates and depth values ​​of each point and the world coordinate values ​​of the corresponding point, based on the depth-to-point-cloud formula and assuming that the origin of the world coordinate system and the origin of the camera coordinate system coincide.

[0116] In one embodiment, the third acquisition unit 1130 is further configured to obtain the average depth of the point cloud in the region of interest of each plane by using the least squares method, based on the world coordinates of each point in the point cloud in the region of interest of each of the N planes.

[0117] In one embodiment, the third acquisition unit 1130 is further configured to perform plane fitting based on the world coordinate values ​​of each point in the point cloud of the first region of interest to obtain the plane equation of the plane formed by the point cloud in the first region of interest, wherein the first region of interest is the region of interest of any one of N planes; substitute the X-axis coordinates and Y-axis coordinates of the world coordinate values ​​of each point in the point cloud of the first region of interest into the plane equation of the first region of interest to obtain the updated Z-axis coordinate value; and take the average value of the updated Z-axis coordinate values ​​of the world coordinate values ​​of all points in the point cloud of the first region of interest as the average point cloud depth of the first region of interest.

[0118] In one embodiment, the judging unit 1140 is further configured to compare the difference between the average point cloud depth of the region of interest of the first plane and the average point cloud depth of the region of interest of the second plane in the depth image with the vertical distance between the first plane and the second plane of the physical object of the stepped test body, so as to judge the depth accuracy of the depth camera, wherein the first plane and the second plane are any two planes among the N planes.

[0119] See Figure 12 , Figure 12 This is a schematic diagram of the structure of a computer device that implements the method for evaluating the depth performance of a depth camera provided in the embodiments of this disclosure.

[0120] like Figure 12 As shown, the computer device in this embodiment may include one or more processors 1201, a memory 1202, and an input / output interface 1203. The processor 1201, memory 1202, and input / output interface 1203 are connected via a bus 1204. The memory 1202 stores a computer program, which includes program instructions. The input / output interface 1203 receives and outputs data, such as for data interaction between the host machine and the computer device, or for data interaction between various virtual machines within the host machine. The processor 1201 executes the program instructions stored in the memory 1202.

[0121] The processor 1201 can perform the following operations:

[0122] A depth image of a stepped test object and corresponding point cloud data are acquired using a depth camera. The stepped test object comprises N planes, where N is an integer greater than or equal to 2. The region of interest (ROI) of each of the N planes in the depth image is obtained. The average point cloud depth of the ROI of each of the N planes is obtained. The depth performance of the depth camera is determined based on the average point cloud depth of the ROI between the N planes.

[0123] In some feasible implementations, the processor 1201 may be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0124] The memory 1202 may include read-only memory and random access memory, and provides instructions and data to the processor 1201 and input / output interface 1203. A portion of the memory 1202 may also include non-volatile random access memory. For example, the memory 1202 may also store device type information.

[0125] In practice, the computer device can execute the implementation methods provided by the steps in the above embodiments through its built-in functional modules. For details, please refer to the implementation methods provided by the steps in the above embodiments, which will not be repeated here.

[0126] This disclosure provides a computer device including a processor, an input / output interface, and a memory. The processor retrieves a computer program from the memory and executes the steps of the method shown in the above embodiments to perform a transmission operation.

[0127] This disclosure also provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and execute the methods provided in the steps of the above embodiments. Specific implementations of the steps in the above embodiments can be found therein and will not be repeated here. Furthermore, the beneficial effects of using the same method will not be repeated here either. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this disclosure, please refer to the description of the method embodiments of this disclosure. As an example, the computer program can be deployed to execute on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network.

[0128] The computer-readable storage medium can be the apparatus provided in any of the foregoing embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0129] This disclosure also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative embodiments described above.

[0130] The terms "first," "second," etc., used in the specification, claims, and drawings of this disclosure are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0132] The methods and related apparatuses provided in this disclosure are described with reference to the method flowcharts and / or structural diagrams provided in this disclosure. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions are provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable transmission device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable transmission device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable transmission device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable transmission device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0133] The above-disclosed embodiments are merely preferred embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Therefore, any equivalent variations made in accordance with the claims of this disclosure shall still fall within the scope of this disclosure.

Claims

1. A method for evaluating the depth performance of a depth camera, characterized in that, include: A depth image of the stepped test object and the corresponding point cloud data are acquired using a depth camera; wherein, the stepped test object includes N planes, where N is an integer greater than or equal to 2; Obtain the region of interest for each of the N planes in the depth image; Obtain the average point cloud depth in the region of interest of each of the N planes; The depth performance of the depth camera is determined based on the average point cloud depth of the region of interest between the N planes. The step of obtaining the average point cloud depth in the region of interest of each of the N planes includes: Obtain the pixel coordinates and depth value of each point in the point cloud of the region of interest of each of the N planes; Obtain the world coordinates of each point based on its pixel coordinates and depth value; Plane fitting is performed based on the world coordinates of each point in the point cloud of the first region of interest to obtain the plane equation of the plane formed by the point cloud of the first region of interest, wherein the first region of interest is the region of interest of any one of the N planes. Substitute the X-axis and Y-axis coordinates of each point in the point cloud of the first region of interest into the plane equation of the first region of interest to obtain the updated Z-axis coordinate value. The average value of the updated Z-axis coordinates of all points in the point cloud of the first region of interest is taken as the average depth of the point cloud of the first region of interest.

2. The method according to claim 1, characterized in that, The world coordinates of each point are obtained based on its pixel coordinates and depth value, including: Based on the depth-to-point cloud formula, and assuming that the origin of the world coordinate system and the origin of the camera coordinate system coincide, we obtain the conversion formula between the pixel coordinates and depth values ​​of each point and the corresponding world coordinate values.

3. The method according to claim 1, characterized in that, The depth performance of the depth camera is determined based on the average point cloud depth of the region of interest between the N planes, including: The depth accuracy of the depth camera is determined by comparing the difference between the average point cloud depth of the region of interest in the first plane and the average point cloud depth of the region of interest in the second plane in the depth image with the vertical distance between the first plane and the second plane of the physical object of the stepped test body. The first plane and the second plane are any two planes among the N planes.

4. The method according to claim 1, wherein the vertical distance between the planes of the stepped test body includes at least one or more of the following: nanometer, micrometer, centimeter, filament, millimeter, centimeter, or decimeter.

5. A device for evaluating the depth performance of a depth camera, characterized in that, include: The first acquisition unit is used to acquire a depth image of the stepped test body and the point cloud data corresponding to the depth image through a depth camera; wherein, the stepped test body includes N planes, and N is an integer greater than or equal to 2; The second acquisition unit is used to acquire the region of interest for each of the N planes in the depth image; The third acquisition unit is used to acquire the average point cloud depth in the region of interest of each of the N planes; The judgment unit is used to judge the depth performance of the depth camera based on the average point cloud depth of the region of interest between the N planes. The third acquisition unit is further configured to acquire the pixel coordinates and depth values ​​of each point in the point cloud of the region of interest of each of the N planes; acquire the world coordinate values ​​of the corresponding point based on the pixel coordinates and depth values ​​of each point; perform plane fitting based on the world coordinate values ​​of each point in the point cloud of the first region of interest to obtain the plane equation of the plane formed by the point cloud of the first region of interest, wherein the first region of interest is the region of interest of any one of the N planes; substitute the X-axis coordinates and Y-axis coordinates of the world coordinate values ​​of each point in the point cloud of the first region of interest into the plane equation of the first region of interest to obtain the updated Z-axis coordinate value; and take the average of the updated Z-axis coordinate values ​​of the world coordinate values ​​of all points in the point cloud of the first region of interest as the average depth of the point cloud of the first region of interest.

6. A computer device, characterized in that, Includes processor, memory, and input / output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive data and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device executes the method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the method described in any one of claims 1-4.

Images