A method, apparatus, device and medium for precision testing of a depth camera

Abstract

The application discloses a precision test method, device and equipment of a depth camera and a medium, relates to the technical field of camera testing, and comprises the following steps: collecting a set of image pairs captured by the depth camera during movement of a calibration board at a plurality of different positions in the imaging range of the depth camera; for each image pair, determining a first central two-dimensional coordinate of the identification point in the color image pixel coordinate system and a FOV reference range of the identification point in the color image; taking the depth of the first central two-dimensional coordinate of the identification point in the depth map as an observed value of the distance of the identification point; calculating the coordinates of the identification point in the camera coordinate system based on the extrinsic matrix of the depth camera in the calibration board coordinate system, i.e. the true value of the distance of the identification point; and determining the precision test results under different measurement distances and different FOV reference ranges according to the observed value and the true value of the distance of the identification point obtained from each image pair. The application can improve the test efficiency while reducing the test cost.

Description

Technical Field

[0001] This application relates to the field of camera testing technology, and in particular to a method, apparatus, device and medium for testing the accuracy of a depth camera. Background Technology

[0002] As a core sensor in fields such as 3D perception, robot navigation, and human-computer interaction, the quantitative evaluation of the ranging accuracy of depth cameras has become a key issue in machine vision research. The core purpose of depth camera ranging accuracy testing is to evaluate the ranging accuracy and stability of depth cameras in different scenarios and to obtain their error characteristics (such as systematic errors and random errors), thereby providing reliable data support for subsequent tasks such as 3D reconstruction, target detection, and obstacle avoidance.

[0003] Currently, traditional methods for testing the accuracy of depth cameras mainly rely on mechanical devices and manual operation. The specific implementation is as follows: The depth camera is mounted on a guide rail, and the distance between the depth camera and the target is adjusted by sliding the guide rail. At different distances, the distance between the depth camera and the target must be manually measured using tools such as a laser rangefinder or a measuring tape, and the measurement result is used as a reference value for the actual distance. This method is not only cumbersome and inefficient, but also prone to introducing human error, making it difficult to meet the requirements of high-precision and high-efficiency testing.

[0004] To overcome the above problems, several technical solutions have emerged in recent years in this field that directly calculate the distance between the depth camera and the target based on image data and camera parameters, reducing the dependence on external measurement tools. However, these solutions still have limitations such as low calculation accuracy and limited testing flexibility and efficiency.

[0005] The main limitations of the relevant technologies are the shortcomings of traditional manual distance measurement methods and the limitations of existing direct calculation methods.

[0006] 1. The shortcomings of traditional manual distance measurement methods.

[0007] (1) System errors are difficult to completely eliminate: Due to the difficulty in achieving precise registration of the spatial pose between the target and the depth camera, and the inherent assembly deviations of the core components such as the mechanical support structure of the optical module, the lens group, and the imaging sensor, the manual ranging results cannot accurately represent the actual distance between the target and the depth camera. The inherent spatial pose offset and assembly deviation of the hardware system reduce the accuracy of the test data.

[0008] (2) Low testing efficiency: Traditional testing schemes are usually limited to accuracy evaluation at a single distance, which cannot fully reflect the performance of depth cameras at different distances. This not only reduces testing efficiency, but also makes it difficult to meet the needs of multi-distance accuracy testing in actual application scenarios.

[0009] 2. Limitations of existing direct calculation methods.

[0010] (1) Unreliable true distance values: Existing methods calculate the true distance to the target by using the three-dimensional coordinates of the marker in the depth image and the equation of the target plane. The three-dimensional coordinates of the marker in the depth image are measured by the depth camera. Therefore, the true value calculated using the measured values ​​will inevitably have errors, which will affect the accuracy of the calculated true value and reduce the reliability of the test results.

[0011] (2) Neglecting the impact of FOV on ranging accuracy: The ranging accuracy of depth cameras often varies significantly under different FOV conditions, and the selection of the FOV range is one of the important prerequisites affecting ranging accuracy. However, existing methods have failed to fully consider the impact of FOV in their design and implementation, and also lack a mechanism for systematically evaluating ranging accuracy under different FOVs. This limitation makes it difficult for the test results to fully reflect the performance of depth cameras in actual application scenarios, reducing the practicality and reliability of the test results.

[0012] (3) Limited testing flexibility and efficiency: Existing methods require extracting the target region image corresponding to the target calibration board from the color image. This process is based on a pre-configured calibration board group file, which records the regions of interest corresponding to all calibration boards. Even if the region of interest is slightly larger than the actual imaging range of the calibration board, this method still limits the relative position of the calibration board and the camera to a fixed range, resulting in reduced testing flexibility and limited efficiency.

[0013] (4) Limited improvement in testing efficiency: By using multiple calibration plates, existing methods can simultaneously obtain the accuracy test results of the depth camera at multiple distances in a single test. However, this improvement in efficiency is limited by the number of calibration plates in the image, and the optimization effect is limited, making it difficult to meet the demand for higher efficiency.

[0014] (5) Increased testing costs: In order to adapt to the pinhole imaging characteristics of depth cameras (near targets appear larger in the image, while distant targets appear smaller), existing methods require that calibration boards closer to the camera be small in size, while calibration boards farther from the camera be large in size. This means that different sizes of calibration boards need to be customized for accuracy testing at different distances, which not only increases the complexity of testing, but also significantly increases the testing costs. Summary of the Invention

[0015] The purpose of this application is to provide a method, apparatus, device, and medium for testing the accuracy of a depth camera, which improves testing efficiency and reduces testing costs.

[0016] To achieve the above objectives, this application provides the following solution:

[0017] Firstly, this application provides a method for testing the accuracy of a depth camera, including:

[0018] The acquisition calibration plate moves to multiple different positions within the imaging range of the depth camera, acquiring a set of image pairs captured by the depth camera, each image pair including a color image and a depth image;

[0019] For each image pair, extract the calibration board markers from the color image, determine the first center two-dimensional coordinates of the markers in the pixel coordinate system of the color image, and the FOV reference range of the markers in the color image;

[0020] The depth of the depth map corresponding to the first center two-dimensional coordinates of the marker point is used as the observed value of the marker point distance, which is the distance from the marker point to the depth camera;

[0021] Based on the first center two-dimensional coordinates of the marker point and the third center three-dimensional coordinates of the calibration point in the calibration plate coordinate system, calculate the extrinsic parameter matrix of the depth camera in the calibration plate coordinate system.

[0022] Based on the third center three-dimensional coordinates and extrinsic parameter matrix of the marker point in the calibration board coordinate system, calculate the fourth center three-dimensional coordinates of the marker point in the camera coordinate system; the depth coordinate in the fourth center three-dimensional coordinates is the true value of the distance to the marker point.

[0023] Based on the observed and actual values ​​of the distance to the marker points obtained for each image pair, the accuracy test results are determined under different measurement distances and different FOV reference ranges; the measured distance is the actual value of the distance to the marker points.

[0024] Optionally, the calibration board's marker points are extracted from the color image, and the first central two-dimensional coordinates of the marker points in the pixel coordinate system of the color image and the FOV reference range of the marker points in the color image are determined, specifically including:

[0025] A color image is preprocessed to obtain a preprocessed color image; the preprocessing includes grayscale conversion and filtering.

[0026] Based on the calibration board type, extract the calibration board marker points from the preprocessed color image and determine the first center two-dimensional coordinates of the marker points in the pixel coordinate system of the color image;

[0027] The FOV reference range of the marker points is determined based on the first center two-dimensional coordinates of the marker points in the preprocessed color image.

[0028] Optionally, the FOV reference range of the marker point is determined based on the first center two-dimensional coordinates of the marker point in the preprocessed color image, specifically including:

[0029] The FOV reference range with the smallest area among the multiple preset FOV reference ranges where the first center two-dimensional coordinates of the marker point are located is taken as the FOV reference range of the marker point.

[0030] Optionally, the calibration plate type includes a checkerboard calibration plate, a circular grid calibration plate, and an ArUco marker calibration plate.

[0031] Optionally, based on the first center two-dimensional coordinates of the marker point and the third center three-dimensional coordinates of the calibration point in the calibration plate coordinate system, the extrinsic parameter matrix of the depth camera in the calibration plate coordinate system is calculated, specifically including:

[0032] The formula for calculating the extrinsic parameter matrix of the depth camera in the calibration plate coordinate system is expressed as follows:

[0033] ;

[0034] in, For the first center two-dimensional coordinate, The x-axis is... The vertical axis is , For the third center three-dimensional coordinates, This is the extrinsic parameter matrix.

[0035] Optionally, the formula for calculating the fourth center three-dimensional coordinates of the marker point in the camera coordinate system is:

[0036] ;

[0037] in, It is an extrinsic parameter matrix. For the third center three-dimensional coordinates, The fourth center's three-dimensional coordinates.

[0038] Optionally, the accuracy test results include absolute error, relative error, root mean square error, standard deviation, and signal-to-noise ratio.

[0039] Secondly, this application provides a depth camera accuracy testing device and a depth camera accuracy testing method, wherein the depth camera accuracy testing device includes:

[0040] The image acquisition module is used to acquire a set of image pairs captured by the depth camera during the movement of the calibration board at multiple different positions within the imaging range of the depth camera. Each image pair includes a color image and a depth image.

[0041] The marker point extraction and FOV reference range determination module is used to extract the marker points of the calibration board from the color image for each image pair, and determine the first center two-dimensional coordinates of the marker points in the pixel coordinate system of the color image and the FOV reference range of the marker points in the color image.

[0042] The camera coordinate system coordinate determination module is used to take the depth of the depth map corresponding to the first center two-dimensional coordinates of the marker point as the observed value of the marker point distance, wherein the marker point distance is the distance from the marker point to the depth camera;

[0043] The extrinsic parameter matrix determination module is used to calculate the extrinsic parameter matrix of the depth camera in the calibration plate coordinate system based on the first center two-dimensional coordinates of the marker point and the third center three-dimensional coordinates of the calibration point in the calibration plate coordinate system.

[0044] The camera coordinate system coordinate determination module is used to calculate the fourth center three-dimensional coordinates of the marker point in the camera coordinate system based on the third center three-dimensional coordinates and the extrinsic parameter matrix of the marker point in the calibration plate coordinate system; the depth coordinate in the fourth center three-dimensional coordinates is the true value of the distance to the marker point;

[0045] The accuracy test result determination module is used to determine the accuracy test results under different measurement distances and different FOV reference ranges based on the observed and true values ​​of the distance to the marker points obtained for each image pair; the measured distance is the true value of the distance to the marker points.

[0046] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the depth camera accuracy testing method described in any one of the above.

[0047] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the depth camera accuracy testing method described in any one of the above descriptions.

[0048] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a method, apparatus, device, and medium for testing the accuracy of a depth camera. It collects a set of image pairs captured by the depth camera during the movement of a calibration board at multiple different positions within the imaging range of the depth camera. Each image pair includes a color image and a depth image. The calibration board's marker points are extracted from the color images. The depth of the depth image is used as the observed distance value of the marker point. Based on the third-center three-dimensional coordinates and extrinsic parameter matrix of the marker point in the calibration board coordinate system, the fourth-center three-dimensional coordinates of the marker point in the camera coordinate system are calculated. The depth coordinates in the fourth-center three-dimensional coordinates represent the true distance to the marker point. Since the true value is only related to the position of the marker point in the calibration board and the camera's extrinsic parameters, and is not affected by camera hardware errors or the depth camera's ranging accuracy, the obtained true value is more accurate. Furthermore, this application obtains the test accuracy under different fields of view by determining the field of view (FOV) reference range of the marker point. The depth camera test can be completed using only a single calibration board, and the testing tools are simple and readily available, significantly reducing testing costs. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating a method for testing the accuracy of a depth camera according to an embodiment of this application.

[0051] Figure 2 This is a detailed flowchart illustrating a method for testing the accuracy of a depth camera, as provided in one embodiment of this application.

[0052] Figure 3 This is a schematic diagram of a preset FOV reference range provided in an embodiment of this application.

[0053] Figure 4 This application provides a functional module diagram of a depth camera accuracy testing device according to one embodiment.

[0054] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0056] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] In one exemplary embodiment, this application provides a method for testing the accuracy of a depth camera, such as... Figure 1 and Figure 2 As shown, the accuracy testing method for the depth camera includes steps 101-106.

[0058] Step 101: Collect a set of image pairs captured by the depth camera during the movement of the calibration plate at multiple different positions within the imaging range of the depth camera. Each image pair includes a color image and a depth image.

[0059] Step 102: For each image pair, extract the calibration board markers from the color image, determine the first center two-dimensional coordinates of the markers in the pixel coordinate system of the color image, and the FOV reference range of the markers in the color image.

[0060] Step 103: The depth of the depth map corresponding to the first center two-dimensional coordinates of the marker point is used as the observed value of the marker point distance, which is the distance from the marker point to the depth camera.

[0061] Step 104: Calculate the extrinsic parameter matrix of the depth camera in the calibration plate coordinate system based on the first center two-dimensional coordinates of the marker point and the third center three-dimensional coordinates of the calibration point in the calibration plate coordinate system.

[0062] Step 105: Calculate the third center three-dimensional coordinates of the marker point in the camera coordinate system based on the third center three-dimensional coordinates and the extrinsic parameter matrix of the marker point in the calibration plate coordinate system; the depth coordinates in the fourth center three-dimensional coordinates are the true values ​​of the distance to the marker point.

[0063] Step 106: Based on the observed and true values ​​of the distance to the marker points obtained for each image pair, determine the accuracy test results under different measurement distances and different FOV reference ranges; the measured distance is the true value of the distance to the marker points.

[0064] The core idea of ​​this application is to determine the ranging accuracy of the depth camera under different fields of view (FOV) and distances by capturing multiple images or videos of the calibration board at different distances from the camera's optical axis and phase plane. Through systematic image acquisition, marker point extraction, coordinate calculation, and error analysis, the ranging accuracy of the depth camera can be efficiently and accurately evaluated. This method can adapt to different field of view and distance conditions, and significantly improves testing efficiency and result reliability through automated processes.

[0065] In one exemplary embodiment, acquiring a set of image pairs captured by a depth camera specifically includes: fixing the depth camera to be tested, keeping a calibration board approximately parallel to the image plane and moving it within the camera's imaging range. During data acquisition, it is necessary to ensure that the calibration board appears completely within the depth camera's imaging range at different positions, avoiding any partial displacement of the calibration board out of the field of view. To ensure data diversity, the calibration board should cover the camera's field of view boundary in both the horizontal and vertical directions as much as possible. The distance between the calibration board and the camera needs to meet the following conditions: the details of the calibration board (such as corners and patterns) can be clearly distinguished by the camera, and the size of the calibration board in the image is appropriate, neither too far away causing loss of details nor too close causing the calibration board to exceed the field of view.

[0066] The specific implementation of the calibration plate position change includes: within the field of view of the depth camera, moving the calibration plate to different positions on the image plane (such as upper left, upper right, lower left, lower right, center, etc.) to ensure coverage of the entire imaging area; repeating the above operation at different distances (such as near, middle, far) to ensure that the calibration data is valid at different depths, where the distance refers to the distance from the calibration plate to the depth camera.

[0067] This application uses a depth camera to capture photos or videos of the calibration board at different positions and distances. The photos include color images and depth images, and each frame of the video is also composed of color images and depth images.

[0068] Each color image and its corresponding depth image constitute an image pair.

[0069] In an exemplary embodiment, step 102 involves extracting the calibration board's marker points from the color image, determining the first central two-dimensional coordinates of the marker points in the pixel coordinate system of the color image, and the FOV reference range of the marker points in the color image, specifically including steps 201-203.

[0070] Step 201: Preprocess the color image to obtain a preprocessed color image; the preprocessing includes grayscale conversion and filtering.

[0071] This application performs grayscale conversion and filtering preprocessing on each frame of image or photograph: grayscale conversion can use, but is not limited to, the OpenCV function `cv2.cvtColor()` to convert the color image from the blue-green-red (BGR) color space to the grayscale space. Filtering can use, but is not limited to, Gaussian filtering, and can employ the function `cv2.GaussianBlur()`. These methods ensure data quality during image acquisition and preprocessing, providing a reliable foundation for subsequent camera calibration.

[0072] Step 202: Extract the calibration board markers from the preprocessed color image according to the calibration board type, and determine the first center two-dimensional coordinates of the markers in the pixel coordinate system of the color image.

[0073] The calibration plate types include checkerboard calibration plates, circular grid calibration plates, and ArUco marker calibration plates.

[0074] The methods for extracting marker points vary depending on the type of calibration board. For example, if the calibration board is a checkerboard calibration board, the cv2.findChessboardCorners() method can be used to detect its corner points; if the calibration board is a circular grid calibration board, calibration is performed by detecting the position of the circle center, and OpenCV provides the cv2.findCirclesGrid() method; if the calibration board is an ArUco marker calibration board, OpenCV provides the cv2.aruco.detectMarkers() method for detection.

[0075] Step 203: Determine the FOV reference range of the marker points based on the first center two-dimensional coordinates of the marker points in the preprocessed color image.

[0076] In an exemplary embodiment, step 203 specifically includes: taking the preset FOV reference range with the smallest area among multiple preset FOV reference ranges where the first center two-dimensional coordinates of the marker point are located as the FOV reference range of the marker point.

[0077] This application divides the depth map into multiple FOV reference ranges based on the positional relationship between the pixel coordinates of the marker points and the principal point coordinates. After obtaining the first center two-dimensional coordinates of the marker points, it is necessary to determine the minimum FOV reference range of the marker points in the image: for example, the preset FOV reference range is 81%. FOV refers to the remaining 81% of the center of the depth map after cropping by 5% from the top, bottom, left, and right sides. Figure 3 As shown, Figure 3 It includes three preset FOV reference ranges: 100% FOV, 81% FOV, and 64% FOV.

[0078] This application focuses on how to accurately and quickly obtain the true value of the distance between the calibration board marker point and the camera, and how to quickly obtain a large number of test results under different FOVs and distances by using at least one calibration board and recording only one video or multiple photos.

[0079] In an exemplary embodiment, the accuracy testing method for the depth camera further includes: obtaining the second center three-dimensional coordinates of the marker point in the camera coordinate system by using the matrix transformation formula of the marker point in the camera coordinate system and the pixel coordinate system based on the first center two-dimensional coordinates and the depth of the corresponding depth map.

[0080] The matrix transformation formula for the marker point in the camera coordinate system and the pixel coordinate system is expressed as:

[0081] ;

[0082] The depth of the marker point is obtained from the first center two-dimensional coordinates (u, v) of the marker point on the calibration board and the depth image. The second center 3D coordinates of the marker point in the camera coordinate system can be obtained using the following formula, where, The value is the observed distance to the marker point.

[0083] ;

[0084] ;

[0085] in, This indicates the depth of the depth map. For the first center two-dimensional coordinate, The x-axis is... The vertical axis is , This represents the focal length of the depth camera in the horizontal direction on the image plane. This represents the focal length of the depth camera in the vertical direction on the image plane. This represents the horizontal coordinates of the principal point on the image plane. This represents the coordinates of the principal point on the image plane in the vertical direction. The second center's three-dimensional coordinates. , , and The intrinsic parameter matrix that constitutes the depth camera.

[0086] In an exemplary embodiment, step 104 specifically includes: calculating the extrinsic parameter matrix of the depth camera in the calibration plate coordinate system based on the first center two-dimensional coordinates of the marker point and the third center three-dimensional coordinates of the calibration point in the calibration plate coordinate system, using the matrix transformation formula of the marker point in the calibration plate coordinate system and the pixel coordinate system.

[0087] Solving for the extrinsic parameter matrix of the calibration board: The formula for the matrix transformation of the marker point in the calibration board coordinate system and the pixel coordinate system is expressed as:

[0088] .

[0089] in, For the first center two-dimensional coordinate, The x-axis is... The vertical axis is , For the third center three-dimensional coordinates, It is an extrinsic parameter matrix. This represents the focal length of the depth camera in the horizontal direction on the image plane. This represents the focal length of the depth camera in the vertical direction on the image plane. This represents the horizontal coordinates of the principal point on the image plane. This represents the coordinates of the principal point on the image plane in the vertical direction. Represents the rotation matrix. Represents the translation matrix. and Together they constitute the extrinsic parameter matrix of the depth camera.

[0090] In an exemplary embodiment, in step 105, the formula for calculating the fourth center three-dimensional coordinates of the marker point in the camera coordinate system is: .

[0091] in, It is an extrinsic parameter matrix. For the third center three-dimensional coordinates, The fourth center's three-dimensional coordinates.

[0092] In one exemplary embodiment, the accuracy test results include absolute error, relative error, root mean square error, standard deviation, and signal-to-noise ratio.

[0093] Repeat steps 102-105 for each frame of all photos or videos acquired in step 101, compare the actual values ​​with the measured values ​​of all marker points, and record the actual distance between the marker points and the image plane. Measuring distance In addition to the FOV reference range, step 106 statistically analyzes the absolute error, relative error, root mean square error, standard deviation, and signal-to-noise ratio of the marker points at different distances and within the FOV reference range, ultimately generating a quantitative evaluation result of the depth camera ranging accuracy.

[0094] The technical solution of this application, through automated image processing, coordinate transformation, and error analysis, can quickly and accurately evaluate the ranging accuracy of depth cameras under different field-of-view ranges and distances. Compared with previous methods, this method significantly improves testing efficiency and result reliability, while reducing testing costs and complexity, providing strong technical support for the research, development, calibration, and optimization of depth cameras. Specific beneficial effects are as follows.

[0095] 1. High-precision true value acquisition: This application calculates the true distance between the marker point and the camera by using the coordinates of the marker point on the calibration board in the calibration board coordinate system and the camera extrinsic parameter matrix. Since the true value is only related to the position of the marker point on the calibration board and the camera extrinsic parameters, it is not affected by camera hardware errors and the ranging accuracy of the depth camera, thus the obtained true value is more accurate.

[0096] 2. High-efficiency data acquisition: This application can acquire images of the calibration board in all ranging ranges at once by recording video, thereby obtaining the ranging accuracy in all ranging ranges. It is not restricted by external factors such as the type and quantity of calibration boards. The selection of marker points and accuracy calculation are realized through automatic feature recognition. No human-computer interaction or prior knowledge is required, which significantly improves the testing efficiency.

[0097] 3. FOV Range Evaluation: This application calculates the FOV range of the marker point in the image by identifying the coordinates of the marker point in the pixel coordinate system, and systematically evaluates the ranging accuracy under different FOVs, comprehensively reflecting the performance of the depth camera under different field of view conditions, and improving the practicality and reliability of the test results.

[0098] 4. Low-cost testing: This application requires only one calibration board to complete the camera accuracy test. The testing tools are simple and easy to obtain, which significantly reduces the testing cost and is suitable for large-scale application scenarios.

[0099] Based on the same inventive concept, this application also provides a depth camera accuracy testing apparatus for implementing the depth camera accuracy testing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more depth camera accuracy testing apparatus embodiments provided below can be found in the limitations of the depth camera accuracy testing method described above, and will not be repeated here.

[0100] In one exemplary embodiment, such as Figure 4 As shown, a depth camera accuracy testing device is provided. The depth camera accuracy testing device applies the depth camera accuracy testing method described above, and the depth camera accuracy testing device includes the following modules.

[0101] The image acquisition module is used to acquire a set of image pairs captured by the depth camera during the movement of the calibration board at multiple different positions within the imaging range of the depth camera. Each image pair includes a color image and a depth image.

[0102] The marker point extraction and FOV reference range determination module is used to extract the marker points of the calibration board from the color image for each image pair, and determine the first center two-dimensional coordinates of the marker points in the pixel coordinate system of the color image and the FOV reference range of the marker points in the color image.

[0103] The camera coordinate system coordinate determination module is used to take the depth of the depth map corresponding to the first center two-dimensional coordinates of the marker point as the observed value of the marker point distance, wherein the marker point distance is the distance from the marker point to the depth camera.

[0104] The extrinsic parameter matrix determination module is used to calculate the extrinsic parameter matrix of the depth camera in the calibration plate coordinate system based on the first center two-dimensional coordinates of the marker point and the third center three-dimensional coordinates of the calibration point in the calibration plate coordinate system.

[0105] The camera coordinate system coordinate determination module is used to calculate the fourth center three-dimensional coordinates of the marker point in the camera coordinate system based on the third center three-dimensional coordinates and the extrinsic parameter matrix of the marker point in the calibration plate coordinate system; the depth coordinates in the fourth center three-dimensional coordinates are the true values ​​of the distance to the marker point.

[0106] The accuracy test result determination module is used to determine the accuracy test results under different measurement distances and different FOV reference ranges based on the observed and true values ​​of the distance to the marker points obtained for each image pair; the measured distance is the true value of the distance to the marker points.

[0107] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 5 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores accuracy test data from the depth camera. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for accuracy testing of a depth camera.

[0108] Those skilled in the art will understand that Figure 5 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0109] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0110] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0111] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0112] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0113] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units, etc., and are not limited to these.

[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method of precision testing of a depth camera, characterized in that, The precision test method of the depth camera comprises: Collecting, during movement of a calibration board at a plurality of different positions within an imaging range of a depth camera, a set of image pairs captured by the depth camera, each image pair comprising a color image and a depth image; the position change of the calibration board comprising: fixing the depth camera to be tested, keeping the calibration board parallel to the image plane and moving the calibration board within the imaging range of the depth camera; moving the calibration board at different positions on the image plane within the field of view of the depth camera to cover the entire imaging area; repeating the above operation at different distances; the different positions on the image plane comprising upper left, upper right, lower left, lower right and center; For each image pair, extracting an identification point of the calibration board from the color image, determining a first central two-dimensional coordinate of the identification point in a pixel coordinate system of the color image and a FOV reference range of the identification point in the color image; Taking the depth of the first central two-dimensional coordinate of the identification point corresponding to the depth map as an observed value of the identification point distance, the identification point distance being the distance from the identification point to the depth camera; According to the first central two-dimensional coordinate of the identification point and a third central three-dimensional coordinate of the identification point in a calibration board coordinate system, calculating an extrinsic matrix of the depth camera in the calibration board coordinate system; According to the third central three-dimensional coordinate of the identification point in the calibration board coordinate system and the extrinsic matrix, calculating a fourth central three-dimensional coordinate of the identification point in a camera coordinate system; the depth coordinate in the fourth central three-dimensional coordinate being a true value of the identification point distance; According to the observed value and the true value of the identification point distance obtained from each image pair, determining precision test results under different measurement distances and different FOV reference range conditions; the measurement distance being the true value of the identification point distance.

2. The precision testing method of a depth camera according to claim 1, wherein, Extracting an identification point of the calibration board from the color image, determining a first central two-dimensional coordinate of the identification point in a pixel coordinate system of the color image and a FOV reference range of the identification point in the color image, specifically comprising: Preprocessing the color image to obtain a preprocessed color image; the preprocessing comprising grayscale processing and filtering processing; According to the type of the calibration board, extracting the identification point of the calibration board from the preprocessed color image, and determining a first central two-dimensional coordinate of the identification point in a pixel coordinate system of the color image; According to the first central two-dimensional coordinate of the identification point in the preprocessed color image, determining the FOV reference range of the identification point.

3. The method of precision testing of a depth camera according to claim 2, wherein, According to the first central two-dimensional coordinate of the identification point in the preprocessed color image, determining the FOV reference range of the identification point, specifically comprising: Taking the preset FOV reference range with the smallest area in a plurality of preset FOV reference ranges of the first central two-dimensional coordinate of the identification point as the FOV reference range of the identification point.

4. The method of claim 2, wherein, The type of the calibration board comprises a checkerboard calibration board, a circular grid calibration board and an ArUco marker calibration board.

5. The method of claim 1, wherein, According to the first central two-dimensional coordinate of the identification point and a third central three-dimensional coordinate of the identification point in a calibration board coordinate system, calculating an extrinsic matrix of the depth camera in the calibration board coordinate system, specifically comprising: The formula for calculating the extrinsic matrix of the depth camera in the calibration board coordinate system is: ； wherein, is a first center two-dimensional coordinate, is a horizontal coordinate, is a vertical coordinate, is a third center three-dimensional coordinate, denotes the focal length of the depth camera in the horizontal direction of the image plane, denotes the focal length of the depth camera in the vertical direction of the image plane, denotes the coordinate of the principal point on the image plane in the horizontal direction, denotes the coordinate of the principal point on the image plane in the vertical direction, denotes a rotation matrix, denotes a translation matrix, is an extrinsic matrix.

6. The method of precision testing of a depth camera according to claim 1, wherein, The formula for calculating the fourth central three-dimensional coordinate of the identification point in the camera coordinate system is: ； wherein, is an extrinsic matrix, is a third center three-dimensional coordinate, is a fourth center three-dimensional coordinate.

7. The method of precision testing of a depth camera according to claim 1, wherein, The precision test result includes absolute error, relative error, root mean square error, standard deviation and signal-to-noise ratio.

8. An accuracy testing device for a depth camera, characterized by The precision test device of the depth camera applies the precision test method of the depth camera in any one of claims 1-7, and the precision test device of the depth camera comprises: An image acquisition module is configured to acquire a set of images taken by the depth camera during movement of the calibration board at a plurality of different positions within the imaging range of the depth camera, each image pair comprising a color image and a depth image; the calibration board position change comprises fixing the depth camera to be tested, keeping the calibration board parallel to the image plane and moving the calibration board within the imaging range of the depth camera; moving the calibration board at different positions on the image plane within the field of view of the depth camera to cover the entire imaging area; repeating the above operation at different distances; the different positions on the image plane include the upper left, upper right, lower left, lower right and center; An identification point extraction and FOV reference range determination module is configured to extract the identification points of the calibration board from the color image for each image pair, and determine the first central two-dimensional coordinates of the identification points in the pixel coordinate system of the color image and the FOV reference range of the identification points in the color image; A camera coordinate system coordinate determination module is configured to take the depth of the identification points in the depth map as the observed value of the identification point distance, which is the distance from the identification points to the depth camera; An extrinsic parameter matrix determination module is configured to calculate the extrinsic parameter matrix of the depth camera in the calibration board coordinate system according to the first central two-dimensional coordinates of the identification points and the third central three-dimensional coordinates of the identification points in the calibration board coordinate system; A camera coordinate system coordinate determination module is configured to calculate the fourth central three-dimensional coordinates of the identification points in the camera coordinate system according to the third central three-dimensional coordinates of the identification points in the calibration board coordinate system and the extrinsic parameter matrix; the depth coordinate in the fourth central three-dimensional coordinates is the true value of the identification point distance; A precision test result determination module is configured to determine the precision test result under different measurement distances and different FOV reference range conditions according to the observed value and the true value of the identification point distance obtained from each image pair; the measurement distance is the true value of the identification point distance.

9. A computer device comprising: A memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the precision test method of the depth camera in any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the precision test method of the depth camera in any one of claims 1-7.

Images