Precision testing method, device and equipment of depth camera and medium

Abstract

The invention discloses a precision testing method, device and equipment for a depth camera and a medium, and relates to the technical field of camera testing, and the method comprises the steps: collecting an image pair set shot by the depth camera when a calibration plate moves at a plurality of different positions in the imaging range of the depth camera; for each image pair, determining a first center two-dimensional coordinate of the identification point in a color image pixel coordinate system and an FOV reference range of the identification point in the color image; taking the depth of the depth map corresponding to the first center two-dimensional coordinate of the identification point as an observation value of the distance of the identification point; calculating the coordinates of the identification points in the camera coordinate system based on the external parameter matrix of the depth camera in the calibration plate coordinate system, namely the true value of the distance of the identification points; and determining precision test results under the conditions of different measurement distances and different FOV reference ranges according to observation values and true values of the obtained identification point distances of each image pair. The test efficiency can be improved, and the test cost can be reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera testing, in particular to a precision testing method and device for a depth camera, equipment and a medium. BACKGROUND

[0002] As a core sensor in the fields of three-dimensional perception, robot navigation and human-computer interaction, the ranging accuracy of a depth camera has become a key issue in machine vision research. The core purpose of depth camera ranging accuracy testing is to evaluate the ranging accuracy and stability of the depth camera in different scenes and obtain its error characteristics (such as systematic error, random error, etc.), so as to provide reliable data support for subsequent three-dimensional reconstruction, target detection and obstacle avoidance tasks.

[0003] At present, the traditional method of depth camera precision testing mainly relies on mechanical devices and manual operation, and the specific implementation is as follows: the depth camera is installed on a guide rail, and the distance between the depth camera and the target to be photographed is adjusted by sliding the guide rail. At different distances, the distance between the depth camera and the target is measured manually with the help of a laser range finder or a tape measure, and the measurement result is taken as the reference value of the true distance. This method is not only tedious and inefficient, but also prone to human error, making it difficult to meet the testing needs of high precision and high efficiency.

[0004] In order to overcome the above problems, in recent years, a number of technical solutions based on image data and camera parameters have emerged to directly calculate the distance between the depth camera and the measured target, reducing the dependence on external measuring tools, but there are still limitations such as low calculation accuracy, limited testing flexibility and efficiency.

[0005] The related art mainly has the shortcomings of traditional manual ranging method and the limitations of existing direct calculation method.

[0006] 1. The shortcomings of the traditional manual ranging method.

[0007] (1) Systematic error is difficult to completely eliminate: due to the difficulty in accurately registering the spatial pose between the measured target and the depth camera, and the inherent assembly deviation of the core components such as the mechanical support structure, lens group and imaging sensor of the optical module, the manual ranging result cannot accurately represent the actual distance between the measured target and the depth camera. The inherent spatial pose offset and assembly deviation of the hardware system reduce the accuracy of the test data.

[0008] (2) Low testing efficiency: traditional testing schemes are usually limited to accuracy evaluation at a single distance, which cannot fully reflect the performance of the depth camera at different distances, which not only reduces the testing efficiency, but also makes it difficult to meet the needs of multi-distance precision testing in actual application scenarios.

[0009] 2. Limitations of existing direct calculation methods.

[0010] (1) Distance true value is unreliable: Existing methods calculate the true value of the target distance by the three-dimensional coordinates of the marker pattern in the depth image and the target plane equation. The three-dimensional coordinates of the marker pattern in the depth image are measured by the depth camera, so the true value calculated by the measurement value must have errors, which will affect the accuracy of the calculated true value and reduce the reliability of the test results.

[0011] (2) Ignoring the influence of FOV on the ranging accuracy: The ranging accuracy of the depth camera under different FOV conditions often has significant differences, and the selection of the FOV range is one of the important prerequisite conditions that affect the ranging accuracy. However, the existing methods fail to fully consider the influence of FOV during design and implementation, and lack a mechanism for systematically evaluating the ranging accuracy under different FOVs. This limitation makes it difficult for the test results to fully reflect the performance of the depth camera in actual application scenarios, reducing the practicality and reliability of the test results.

[0012] (3) Limited test flexibility and efficiency: The existing method requires to extract the target region image corresponding to the target calibration board from the color image, and this process is based on a pre-configured calibration board group file, which records the regions of interest corresponding to all calibration boards. Even if the region of interest is slightly larger than the actual imaging range of the calibration board, this method still limits the relative position of the calibration board and the camera to a fixed range, resulting in reduced test flexibility and limited efficiency.

[0013] (4) Limited test efficiency improvement: By using multiple calibration boards, the existing method can obtain the accuracy test results of the depth camera at multiple distances in a single test. However, this improvement in efficiency is limited by the number of calibration boards in the picture, and the optimization effect is limited, making it difficult to meet the demand for higher efficiency.

[0014] (5) Increased test cost: In order to adapt to the pinhole imaging characteristics of the depth camera (near targets are larger in the image, and distant targets are smaller), the existing method requires that the calibration boards closer to the camera in the calibration board group use small sizes, and the calibration boards farther from the camera use large sizes. This means that different sizes of calibration boards need to be customized for accuracy testing at different distances, which not only increases the complexity of the test, but also significantly increases the test cost. SUMMARY

[0015] The purpose of the present application is to provide a depth camera accuracy testing method, device, equipment and medium, which improves the test efficiency while reducing the test cost.

[0016] To achieve the above purpose, the present application provides the following solutions: In a first aspect, the present application provides a depth camera accuracy testing method, comprising: A set of image pairs captured by a depth camera during movement of a calibration board at a plurality of different positions within an imaging range of the depth camera, each image pair including a color image and a depth image; For each image pair, extracting an identification point of the calibration board from the color image, determining a first central two-dimensional coordinate of the identification point in a pixel coordinate system of the color image and a FOV reference range of the identification point in the color image; Corresponding the first central two-dimensional coordinate of the identification point to a depth of the depth map as an observed value of an identification point distance, the identification point distance being a distance from the identification point to the depth camera; According to the first central two-dimensional coordinate of the identification point and a third central three-dimensional coordinate of the calibration point in a calibration board coordinate system, calculating an extrinsic matrix of the depth camera in the calibration board coordinate system; According to the third central three-dimensional coordinate of the identification point in the calibration board coordinate system and the extrinsic matrix, calculating a fourth central three-dimensional coordinate of the identification point in a camera coordinate system; the depth coordinate in the fourth central three-dimensional coordinate being a true value of the identification point distance; According to the observed value and the true value of the identification point distance obtained from each image pair, determining accuracy test results under different measurement distances and different FOV reference range conditions; the measurement distance being the true value of the identification point distance.

[0017] Optionally, extracting an identification point of the calibration board from the color image, determining a first central two-dimensional coordinate of the identification point in a pixel coordinate system of the color image and a FOV reference range of the identification point in the color image, specifically comprising: Preprocessing the color image to obtain a preprocessed color image; the preprocessing including grayscale processing and filtering processing; According to the calibration board type, extracting an identification point of the calibration board from the preprocessed color image, determining a first central two-dimensional coordinate of the identification point in a pixel coordinate system of the color image; According to the first central two-dimensional coordinate of the identification point in the preprocessed color image, determining a FOV reference range of the identification point.

[0018] Optionally, according to the first central two-dimensional coordinate of the identification point in the preprocessed color image, determining a FOV reference range of the identification point, specifically comprising: Taking a preset FOV reference range with the smallest area among a plurality of preset FOV reference ranges of the first central two-dimensional coordinate of the identification point as the FOV reference range of the identification point.

[0019] Optionally, the calibration board type includes a checkerboard calibration board, a circular grid calibration board, and an ArUco marker calibration board.

[0020] Optionally, the extrinsic matrix of the depth camera in the calibration board coordinate system is calculated according to the first central two-dimensional coordinates of the mark point and the third central three-dimensional coordinates of the mark point in the calibration board coordinate system, and specifically includes: The formula for calculating the extrinsic matrix of the depth camera in the calibration board coordinate system is represented as: ; Wherein, is the first central two-dimensional coordinates, is the horizontal coordinate, is the vertical coordinate, is the third central three-dimensional coordinates, is the extrinsic matrix.

[0021] Optionally, the formula for calculating the fourth central three-dimensional coordinates of the mark point in the camera coordinate system is: ; Wherein, is the extrinsic matrix, is the third central three-dimensional coordinates, is the fourth central three-dimensional coordinates.

[0022] Optionally, the precision test result includes absolute error, relative error, root mean square error, standard deviation and signal-to-noise ratio.

[0023] In a second aspect, the present application provides a depth camera precision testing device, the depth camera precision testing method of the depth camera precision testing device, and the depth camera precision testing device includes: An image acquisition module is configured to acquire a set of images taken by the depth camera during movement of the calibration board at a plurality of different positions within the imaging range of the depth camera, wherein each image pair includes a color image and a depth image. A mark point extraction and FOV reference range determination module is configured to extract the mark point of the calibration board from the color image and determine the first central two-dimensional coordinates of the mark point in the pixel coordinate system of the color image and the FOV reference range of the mark point in the color image for each image pair. A camera coordinate system coordinate determination module is configured to use the depth of the mark point in the depth map as the observed value of the distance of the mark point from the depth camera, wherein the distance of the mark point from the depth camera is the distance of the mark point to the depth camera. An extrinsic matrix determination module is configured to calculate the extrinsic matrix of the depth camera in the calibration board coordinate system according to the first central two-dimensional coordinates of the mark point and the third central three-dimensional coordinates of the mark point in the calibration board coordinate system. The coordinate determination module in the camera coordinate system is configured to calculate fourth central three-dimensional coordinates of the identification point in the camera coordinate system according to the third central three-dimensional coordinates of the identification point in the calibration board coordinate system and the extrinsic parameter matrix; the depth coordinate in the fourth central three-dimensional coordinates is a real value of the identification point distance; The precision test result determination module is configured to determine precision test results under different measurement distances and different FOV reference ranges according to the observation value and the real value of the identification point distance obtained by each image pair; the measurement distance is the real value of the identification point distance.

[0024] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of the depth camera precision test method in any one of the above.

[0025] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the depth camera precision test method in any one of the above.

[0026] According to the embodiments provided in the present application, the present application discloses the following technical effects: the present application provides a depth camera precision test method, device, equipment and medium, a set of image pairs of a calibration board in a depth camera imaging range are collected during movement of the calibration board at a plurality of different positions, each image pair includes a color image and a depth image, identification points of the calibration board are extracted from the color image, the depth of the depth image is taken as an observation value of the identification point distance, fourth central three-dimensional coordinates of the identification point in a camera coordinate system are calculated according to third central three-dimensional coordinates of the identification point in a calibration board coordinate system and an extrinsic parameter matrix; the depth coordinate in the fourth central three-dimensional coordinates is a real value of the identification point distance, since the real value is only related to the position of the identification point in the calibration board and the camera extrinsic parameter, the real value is not affected by the camera hardware error and the depth camera ranging accuracy, and therefore the real value obtained is more accurate. In addition, the present application determines a field of view (FOV) reference range of the identification point to obtain test precision under different fields of view. The test of the depth camera can be completed by using only one calibration board, the test tool is simple and easy to obtain, and the test cost is significantly reduced. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0028] Figure 1A flowchart of a precision testing method of a depth camera is provided in an embodiment of the present application.

[0029] Figure 2 A detailed flowchart of a precision testing method of a depth camera is provided in an embodiment of the present application.

[0030] Figure 3 A preset FOV reference range diagram is provided in an embodiment of the present application.

[0031] Figure 4 A functional module diagram of a precision testing device of a depth camera is provided in an embodiment of the present application.

[0032] Figure 5 A structural diagram of a computer device is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0034] The above-mentioned purposes, features and advantages of the present application can be more obvious and easy to understand. The present application will be described in further detail below with reference to the drawings and specific embodiments.

[0035] In an exemplary embodiment, the present application provides a precision testing method of a depth camera, as shown in Figure 1 and Figure 2 The precision testing method of the depth camera includes steps 101-106.

[0036] Step 101: Collecting a set of image pairs captured by the depth camera during movement of a calibration board at a plurality of different positions within an imaging range of the depth camera, each image pair including a color image and a depth image.

[0037] Step 102: For each image pair, extracting an identification point of the calibration board from the color image, determining a first central two-dimensional coordinate of the identification point in a pixel coordinate system of the color image, and determining a FOV reference range of the identification point in the color image.

[0038] Step 103: Taking the depth of the first central two-dimensional coordinate of the identification point in the depth image as an observed value of the distance of the identification point, the distance of the identification point being the distance from the identification point to the depth camera.

[0039] Step 104: According to the first center two-dimensional coordinates of the mark point and the third center three-dimensional coordinates of the calibration point in the calibration board coordinate system, the extrinsic matrix of the depth camera in the calibration board coordinate system is calculated.

[0040] Step 105: According to the third center three-dimensional coordinates of the mark point in the calibration board coordinate system and the extrinsic matrix, the fourth center three-dimensional coordinates of the mark point in the camera coordinate system are calculated; the depth coordinate in the fourth center three-dimensional coordinates is the true value of the mark point distance.

[0041] Step 106: According to the observation value and the true value of the mark point distance obtained from each image pair, the accuracy test results under different measurement distances and different FOV reference range conditions are determined; the measurement distance is the true value of the mark point distance.

[0042] The core idea of the present application is to shoot multiple pictures or videos of the calibration board at different distances from the camera optical axis and the image plane, to solve the ranging accuracy of the camera under different FOV and distances, and to efficiently and accurately evaluate the ranging accuracy of the depth camera through systematic image acquisition, mark point extraction, coordinate calculation and error analysis. This method can adapt to different field of view ranges and distance conditions, and significantly improve the test efficiency and result reliability through automatic process.

[0043] In an exemplary embodiment, a set of image pairs shot by a depth camera is collected, specifically including: fixing the depth camera to be tested, keeping the calibration board approximately parallel to the image plane and moving it within the imaging range of the camera, and ensuring that the calibration board can appear completely in the imaging range of the depth camera at different positions during data collection, avoiding partial movement of the calibration board out of the field of view. In order to ensure the diversity of data, the calibration board should cover the boundaries of the camera's field of view as much as possible in both horizontal and vertical directions. The distance between the calibration board and the camera should meet the following conditions: the details (such as corners, patterns) of the calibration board can be clearly distinguished by the camera, and the size of the calibration board in the image is moderate, neither too far away to lose details nor too close to exceed the field of view.

[0044] The specific implementation of the position change of the calibration board includes: moving the calibration board at different positions (such as upper left, upper right, lower left, lower right, center, etc.) in the image plane within the field of view of the depth camera to ensure that the entire imaging area is covered; repeating the above operation at different distances (such as near, medium, far) to ensure that the calibration data is effective at different depths. Here, the distance refers to the distance between the calibration board and the depth camera.

[0045] The present application uses a depth camera to shoot pictures or videos of a calibration board at different positions and distances, the pictures including color images and depth images, and each frame of the video is also composed of color images and depth images.

[0046] Each color image and the corresponding depth image constitute an image pair.

[0047] In an exemplary embodiment, the step 102 of extracting the identification points of the calibration board from the color image, determining the first central two-dimensional coordinates of the identification points in the pixel coordinate system of the color image and the FOV reference range of the identification points in the color image, specifically comprises steps 201-203.

[0048] Step 201: Preprocessing the color image to obtain a preprocessed color image; the preprocessing includes grayscale processing and filtering processing.

[0049] The present application performs grayscale and filtering preprocessing on each frame of image or photo: grayscale processing can use but is not limited to the cv2.cvtColor() function of OpenCV to convert the color image from the blue-green-red (BGR) color space to the grayscale space. Filtering can use but is not limited to Gaussian filtering, which can use the cv2.GaussianBlur() function. Through the above method, the data quality in the image acquisition and preprocessing stage can be ensured, providing a reliable basis for subsequent camera calibration.

[0050] Step 202: Extracting the identification points of the calibration board from the preprocessed color image according to the type of the calibration board, and determining the first central two-dimensional coordinates of the identification points in the pixel coordinate system of the color image.

[0051] The type of the calibration board includes a checkerboard calibration board, a circular grid calibration board and an ArUco marker calibration board.

[0052] According to the different types of the calibration board, the method of extracting the identification points also differs. For example, if the calibration board is a checkerboard calibration board, the cv2.findChessboardCorners() method can be used to detect its corner points; if the calibration board is a circular grid calibration board, the position of the center of the circle is detected for calibration, and OpenCV provides the cv2.findCirclesGrid() method; if the calibration board is an ArUco marker calibration board, OpenCV provides the cv2.aruco.detectMarkers() method for detection.

[0053] Step 203: Determining the FOV reference range of the identification points according to the first central two-dimensional coordinates of the identification points in the preprocessed color image.

[0054] In an exemplary embodiment, the step 203 specifically comprises: taking the preset FOV reference range with the smallest area among the multiple preset FOV reference ranges where the first central two-dimensional coordinates of the identification points are located as the FOV reference range of the identification points.

[0055] This application divides the depth map into multiple FOV reference ranges according to the positional relationship between the pixel coordinates of the marker points and the coordinates of the principal point. After obtaining the first center 2D coordinates of the marker point, it is necessary to determine the minimum FOV reference range of the marker point in the image: for example, the preset FOV reference range is 81%. FOV refers to the remaining center 81% of the depth map after cropping 5% from the top, bottom, left, and right of the depth map, such as Figure 3 As shown, Figure 3 There are three preset FOV reference ranges, namely 100% FOV, 81% FOV and 64% FOV.

[0056] This application mainly focuses on how to accurately and quickly obtain the true value of the distance between the calibration plate marker and the camera, and how to quickly obtain a large number of test results at different FOVs and distances by using at least one calibration plate and only recording one video or multiple photos.

[0057] In an exemplary embodiment, the accuracy testing method of the depth camera also includes: according to the first center two-dimensional coordinates and the depth of the corresponding depth map, using the matrix transformation formula of the identification point in the camera coordinate system and the pixel coordinate system to obtain the second center three-dimensional coordinates of the identification point in the camera coordinate system.

[0058] The matrix transformation formula of the identification point in the camera coordinate system and the pixel coordinate system is expressed as: ; The depth of the marker point obtained based on the first center two-dimensional coordinates (u, v) of the calibration plate marker point and the depth image , the second center three-dimensional coordinates of the identification point in the camera coordinate system can be obtained by the following formula, where: The value is the observed value of the distance to the identified point.

[0059] ; ; in, represents the depth of the depth map, is the first center two-dimensional coordinate, is the horizontal axis, is the vertical axis, Represents the horizontal focal length of the depth camera on the image plane, Indicates the vertical focal length of the depth camera on the image plane, represents the horizontal coordinate of the principal point on the image plane, represents the vertical coordinate of the principal point on the image plane, is the three-dimensional coordinate of the second center. 、 、 and The intrinsic matrix of the depth camera.

[0060] In an exemplary embodiment, the step 104 specifically comprises: according to the first central two-dimensional coordinates of the marker points and the third central three-dimensional coordinates of the calibration points in the calibration board coordinate system, using the matrix transformation formula of the marker points between the calibration board coordinate system and the pixel coordinate system, calculating the extrinsic matrix of the depth camera in the calibration board coordinate system.

[0061] The calibration board extrinsic matrix solution: the matrix transformation formula of the marker points between the calibration board coordinate system and the pixel coordinate system is expressed as: .

[0062] Wherein, the first central two-dimensional coordinates, the horizontal coordinate, the vertical coordinate, the third central three-dimensional coordinates, the extrinsic matrix, denotes the focal length of the depth camera in the horizontal direction of the image plane, denotes the focal length of the depth camera in the vertical direction of the image plane, denotes the coordinate of the principal point in the horizontal direction of the image plane, denotes the coordinate of the principal point in the vertical direction of the image plane, denotes the rotation matrix, denotes the translation matrix, and together constitute the extrinsic matrix of the depth camera.

[0063] In an exemplary embodiment, in the step 105, the formula for calculating the fourth central three-dimensional coordinates of the marker points in the camera coordinate system is: .

[0064] Wherein, the extrinsic matrix, the third central three-dimensional coordinates, the fourth central three-dimensional coordinates.

[0065] In an exemplary embodiment, the precision test result includes absolute error, relative error, root mean square error, standard deviation and signal-to-noise ratio.

[0066] Repeat the above steps 102-105 for each frame of all photos or videos collected in step 101, compare the true value and the measured value of all marker points, and record the true distance , the measured distance and FOV reference range, the absolute error, relative error, root mean square error, standard deviation, signal-to-noise ratio and other evaluation indexes of the identified points in different distances and FOV reference range are counted and identified by step 106, and finally the quantitative evaluation results of the ranging accuracy of the depth camera are generated.

[0067] The technical solution of the present application can quickly and accurately evaluate the ranging accuracy of the depth camera under different field of view ranges and distances through automatic image processing, coordinate conversion and error analysis. Compared with the previous method, the method significantly improves the test efficiency and result reliability, while reducing the test cost and complexity, and provides strong technical support for the research, calibration and optimization of the depth camera. The specific beneficial effects are as follows.

[0068] 1. High-precision true value acquisition: The present application calculates the true value of the distance between the identified point and the camera through the coordinates of the identified point on the calibration board in the calibration board coordinate system and the camera extrinsic parameter matrix. Since the true value is only related to the position of the identified point in the calibration board and the camera extrinsic parameter, it is not affected by the camera hardware error and the ranging accuracy of the depth camera, so the obtained true value is more accurate.

[0069] 2. Efficient data acquisition: The present application can obtain the imaging photos of the calibration board in all ranging ranges at one time through video recording, thereby obtaining the ranging accuracy in all ranging ranges, without being restricted by external factors such as the type and number of calibration boards. The selection of the identified point and the accuracy calculation are realized through automatic feature recognition, without human-computer interaction or prior knowledge, which significantly improves the test efficiency.

[0070] 3. FOV range evaluation: The present application calculates the FOV range of the identified point in the photo through the identification of the coordinates of the identified point in the pixel coordinate system, and systematically evaluates the ranging accuracy under different FOVs, which fully reflects the performance of the depth camera under different field of view conditions, and improves the practicality and reliability of the test results.

[0071] 4. Low-cost testing: The present application only needs to use one calibration board to complete the camera accuracy test, the test tool is simple and easy to obtain, which significantly reduces the test cost and is suitable for large-scale application scenarios.

[0072] Based on the same inventive concept, the embodiments of the present application also provide a depth camera precision testing device for implementing the depth camera precision testing method described above. The implementation scheme of the problem solving provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more depth camera precision testing device embodiments provided below can be referred to the limitations of the depth camera precision testing method in the above text, which will not be repeated here.

[0073] In one exemplary embodiment, as Figure 4As shown, a depth camera precision testing device is provided, which applies the depth camera precision testing method, and the depth camera precision testing device comprises the following modules.

[0074] An image acquisition module is configured to acquire a plurality of image pairs of the calibration board in the process of moving at different positions in the imaging range of the depth camera, each image pair comprising a color image and a depth image.

[0075] An identification point extraction and FOV reference range determination module is configured to extract identification points of the calibration board from the color image for each image pair, and determine a first central two-dimensional coordinate of the identification points in the pixel coordinate system of the color image and a FOV reference range of the identification points in the color image.

[0076] A camera coordinate system coordinate determination module is configured to take the depth of the first central two-dimensional coordinate of the identification points in the depth map as an observation value of the identification point distance, the identification point distance being the distance from the identification point to the depth camera.

[0077] An extrinsic parameter matrix determination module is configured to calculate an extrinsic parameter matrix of the depth camera in the calibration board coordinate system according to the first central two-dimensional coordinate of the identification points and the third central three-dimensional coordinate of the calibration points in the calibration board coordinate system.

[0078] A camera coordinate system coordinate determination module is configured to calculate a fourth central three-dimensional coordinate of the identification points in the camera coordinate system according to the third central three-dimensional coordinate of the identification points in the calibration board coordinate system and the extrinsic parameter matrix, the depth coordinate in the fourth central three-dimensional coordinate being a true value of the identification point distance.

[0079] A precision testing result determination module is configured to determine the precision testing results under different measurement distances and different FOV reference ranges according to the observation value and the true value of the identification point distance obtained for each image pair, the measurement distance being the true value of the identification point distance.

[0080] In an exemplary embodiment, a computer device, which can be a server or a terminal, is provided, and an internal structure diagram of the computer device can be as shown in Figure 5As shown in the figure. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the precision test data of the depth camera. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a precision test method of a depth camera.

[0081] Those skilled in the art can understand that, Figure 5 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above method embodiments.

[0082] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, which is executed by a processor to implement the steps in the above method embodiments.

[0083] In an exemplary embodiment, a computer program product is provided, including a computer program, which is executed by a processor to implement the steps in the above method embodiments.

[0084] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0085] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0086] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, data processing logic of programmable logic devices, and the like.

[0087] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method of precision testing of a depth camera, characterized in that, The precision test method of the depth camera comprises: Collecting, during movement of the calibration board at a plurality of different positions within the imaging range of the depth camera, a set of image pairs captured by the depth camera, each image pair comprising a color image and a depth image; For each image pair, extracting an identification point of the calibration board from the color image, determining a first central two-dimensional coordinate of the identification point in a pixel coordinate system of the color image, and a FOV reference range of the identification point in the color image; Taking the depth of the first central two-dimensional coordinate of the identification point in the depth map as an observed value of the identification point distance, the identification point distance being the distance from the identification point to the depth camera; According to the first central two-dimensional coordinate of the identification point and a third central three-dimensional coordinate of the identification point in a calibration board coordinate system, calculating an extrinsic matrix of the depth camera in the calibration board coordinate system; According to the third central three-dimensional coordinate of the identification point in the calibration board coordinate system and the extrinsic matrix, calculating a fourth central three-dimensional coordinate of the identification point in a camera coordinate system, the depth coordinate in the fourth central three-dimensional coordinate being a true value of the identification point distance; According to the observed value and the true value of the identification point distance obtained from each image pair, determining precision test results under different measurement distances and different FOV reference ranges, the measurement distance being the true value of the identification point distance.

2. The precision testing method of a depth camera according to claim 1, wherein, The method comprises the following steps: Preprocessing the color image to obtain a preprocessed color image; the preprocessing comprises grayscale processing and filtering processing; According to the type of the calibration board, extracting the identification point of the calibration board from the preprocessed color image, and determining a first central two-dimensional coordinate of the identification point in a pixel coordinate system of the color image; According to the first central two-dimensional coordinate of the identification point in the preprocessed color image, determining a FOV reference range of the identification point.

3. The method of precision testing of a depth camera according to claim 2, wherein, The method comprises the following steps: The FOV reference range of the identification point is determined according to the first central two-dimensional coordinate of the identification point in the preprocessed color image, specifically comprising:

4. The method of claim 2, wherein, The FOV reference range of the identification point is determined according to the first central two-dimensional coordinate of the identification point in the preprocessed color image, specifically comprising:

5. The method of claim 1, wherein, The FOV reference range of the identification point is determined according to the first central two-dimensional coordinate of the identification point in the preprocessed color image, specifically comprising: The FOV reference range of the identification point is determined according to the first central two-dimensional coordinate of the identification point in the preprocessed color image, specifically comprising: ； wherein, is a first center two-dimensional coordinate, is a horizontal coordinate, is a vertical coordinate, is a third center three-dimensional coordinate, denotes the focal length of the depth camera in the horizontal direction of the image plane, denotes the focal length of the depth camera in the vertical direction of the image plane, denotes the coordinate of the principal point on the image plane in the horizontal direction, denotes the coordinate of the principal point on the image plane in the vertical direction, denotes a rotation matrix, denotes a translation matrix, is an extrinsic matrix.

6. The method of precision testing of a depth camera according to claim 1, wherein, The type of the calibration board comprises a chessboard calibration board, a circular grid calibration board, and an ArUco marker calibration board. ； wherein, is an extrinsic matrix, is a third center three-dimensional coordinate, is a fourth center three-dimensional coordinate.

7. The method of precision testing of a depth camera according to claim 1, wherein, The method comprises the following steps:

8. An accuracy testing device for a depth camera, characterized by The formula for calculating the extrinsic matrix of the depth camera in the calibration board coordinate system is as follows: The formula for calculating the fourth central three-dimensional coordinate of the identification point in the camera coordinate system is as follows: The precision test results comprise absolute error, relative error, root mean square error, standard deviation, and signal-to-noise ratio. The precision test device of the depth camera applies the precision test method of the depth camera according to any one of claims 1-7, and the precision test device of the depth camera comprises: An image acquisition module is configured to acquire a plurality of image pairs of the calibration board in different positions within the imaging range of the depth camera, each of the image pairs including a color image and a depth image; An identification point extraction and FOV reference range determination module is configured to extract identification points of the calibration board from the color image, determine a first central two-dimensional coordinate of the identification points in a pixel coordinate system of the color image, and determine a FOV reference range of the identification points in the color image for each of the image pairs; A camera coordinate system coordinate determination module is configured to take the depth of the first central two-dimensional coordinate of the identification points corresponding to the depth image as an observation value of the identification point distance, the identification point distance being the distance from the identification point to the depth camera; An extrinsic parameter matrix determination module is configured to calculate an extrinsic parameter matrix of the depth camera in the calibration board coordinate system according to the first central two-dimensional coordinate of the identification points and a third central three-dimensional coordinate of the calibration points in the calibration board coordinate system; A camera coordinate system coordinate determination module is configured to calculate a fourth central three-dimensional coordinate of the identification points in the camera coordinate system according to the third central three-dimensional coordinate of the identification points in the calibration board coordinate system and the extrinsic parameter matrix, the depth coordinate in the fourth central three-dimensional coordinate being a true value of the identification point distance; An accuracy test result determination module is configured to determine accuracy test results under different measurement distances and different FOV reference range conditions according to the observation value and the true value of the identification point distance obtained for each of the image pairs, the measurement distance being the true value of the identification point distance.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the depth camera accuracy test method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the depth camera accuracy test method of any one of claims 1-7.

Images