Low-resolution tof area array intrinsic calibration method, device, equipment and medium

Abstract

The application provides a low-resolution TOF surface array internal parameter calibration method, device, equipment and medium, comprising: arranging a TOF camera module to be calibrated and a white wall in parallel, and emitting light to the white wall; the TOF camera module collects a depth image at a set position, and obtains a measured distance dm of a set pixel point in the depth image; a least square method is used to fit a quadratic function formula, and an extreme point of the function is calculated; according to a conversion relationship between a polar coordinate system and a Cartesian coordinate system, coordinates of the set pixel point in the Cartesian coordinate system are obtained; and a focal length of the TOF camera module is calculated. The application does not need a high-resolution image, has great advantages for low-resolution application occasions, and does not need to shoot characteristic pictures in multiple postures to improve the robustness of an algorithm, but only needs to shoot at a set position, so that high-precision calculation can be completed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of depth camera, in particular to a low-resolution TOF area array intrinsic parameter calibration method, device, equipment and medium. BACKGROUND

[0002] TOF (Time of Flight) is the abbreviation of Time of flight, which means flight time. TOF imaging is to emit modulated light to the target, receive the light returned from the object by the sensor, and calculate the time difference to obtain the target distance. The depth image reflects the distance from the object to the camera sensor, which can be converted into point cloud through the intrinsic parameter matrix.

[0003] In many application scenarios, point cloud needs to be used, so the accuracy of TOF camera intrinsic parameter solving is very important. However, in actual application, the current calibration method mainly has the following problems:

[0004] (1) The current calibration method is basically based on image features. Image features are extracted, which often needs high-resolution images, so that they can be better extracted. For low-resolution images, it is difficult to extract accurately, resulting in large error. In addition, the number of features that can be extracted from low-resolution images is small, which cannot meet the required number for fitting.

[0005] (2) In practice, dozens of calibration board pictures with different placement positions and attitudes need to be taken. This method needs mechanical movement, which consumes a lot of time and reduces the production efficiency. SUMMARY

[0006] In view of the above problems, the present application is proposed to provide a face detection method, device, equipment and medium which can overcome the above problems or at least partially solve the above problems.

[0007] In order to solve the above problems, the present application discloses a low-resolution TOF area array intrinsic parameter calibration method, which comprises: arranging a TOF camera module to be calibrated and a white wall in parallel, and emitting light to the white wall; the TOF camera module collects a depth image at a set position, and obtains a measured distance dm of a set pixel point in the depth image; a least square method is used to fit a quadratic function formula: dm=a(i-cy)2+b(j-cx)2+c, and the extreme point of the function is calculated, that is, the optical center coordinates (cx, cy) of the TOF camera module, wherein (i, j) is the coordinates of the set pixel point; according to the conversion relationship between the polar coordinate system and the Cartesian coordinate system, the coordinates (x, y, z) of the set pixel point in the Cartesian coordinate system are obtained, Wherein, (i, j) is the coordinate position of the set pixel, (cx, cy) is the optical center coordinate, fx and fy are the focal length of the set pixel in the x direction and y direction respectively; the focal length of the TOF camera module is calculated, specifically, through the following formula: derr = dm / dep, ds = derr*derr-1, beta = (i-cy)*(i-cy)+(j-cx)*(j-cx); the values of derr, dsq and beta are obtained, wherein, dm is the distance from the optical center to the set pixel point, dep is the depth of the TOF camera module in the Cartesian coordinate system, then the least square method is used to fit the straight line equation formula: beta = k*dsq+b, the slope is k, and the focal length is f = sqrt(k).

[0008] Optionally, the TOF camera module collects a depth image at the set position, and obtains the measured distance d m of the set pixel point in the depth image. In the step, the TOF camera module collects a set number of first depth images at the first set position, and obtains the first measured distance d m1 of the set pixel point in the first depth image. The TOF camera module collects a set number of second depth images at the second set position, and obtains the second measured distance d m2 of the set pixel point in the second depth image. The difference between the first measured distance and the second measured distance is calculated to obtain the measured distance d m of the depth image. m1 -d m2 .

[0009] The embodiment of the application further discloses a low-resolution TOF area array intrinsic parameter calibration device, which comprises: a transmission control module: used for arranging a TOF camera module to be calibrated and a white wall in parallel, and transmitting light to the white wall; a distance measurement module: used for the TOF camera module to collect a depth image at a set position, and obtain the measured distance d m of the set pixel point in the depth image; an optical center coordinate calculation module: used for fitting a quadratic function formula d m =a(i-cy) 2 +b(j-cx) 2 +c by using the least square method, and calculating the extreme point of the function, i.e. the optical center coordinate (cx, cy) of the TOF camera module, wherein (i, j) is the coordinate of the set pixel point; a coordinate conversion module, used for obtaining the coordinates (x, y, z) of the set pixel point in the Cartesian coordinate system according to the conversion relationship between the polar coordinate system and the Cartesian coordinate system,

[0010]

[0011] Wherein, (i, j) is the coordinate position of the set pixel, (cx, cy) is the optical center coordinate, fx and fy are the focal length of the set pixel in the x direction and the y direction respectively; the focal length calculation module: for calculating the focal length of the TOF camera module, specifically, through the following formula: derr = dm / dep, ds = derr*derr-1, beta = (i-cy)*(i-cy) + (j-cx)*(j-cx); get the values of derr, dsq, beta, wherein, dm is the distance from the optical center to the set pixel point, dep is the depth of the TOF camera module in the Cartesian coordinate system, then the least square method is used to fit the linear equation formula: beta = k*dsq + b, the slope is k, and the focal length is f = sqrt(k).

[0012] The embodiment of the application further discloses an electronic device, including a processor, a storage device, and a computer program stored on the storage device and capable of running on the processor, and the computer program is executed by the processor to realize the method described above.

[0013] The embodiment of the application further discloses a non-volatile readable storage medium, and the non-volatile readable storage medium stores a computer program, and the computer program is executed by a processor to realize the method in any one of claims 1 to 2.

[0014] The embodiment of the application includes the following advantages:

[0015] In the embodiment of the application, a quadratic function equation is fitted according to the relationship between the optical center coordinate and the set pixel coordinate point, then the optical center coordinate is calculated through the extreme value, and the focal length can be obtained through the linear equation converted from the coordinates of the set pixel point in the Cartesian coordinate system. The application does not need to extract image features, so it does not need high-resolution images, and has great advantages for low-resolution application occasions. Moreover, the application does not need to shoot feature pictures in multiple poses to improve the robustness of the algorithm, but only needs to shoot at the set position, so that high-precision calculation can be completed. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a step schematic diagram of the low-resolution TOF area array intrinsic parameter calibration method embodiment of the application;

[0017] Figure 2 is a schematic diagram of step S20 in Figure 1

[0018] Figure 3 is a schematic diagram of the low-resolution TOF area array intrinsic parameter calibration device of the application. DETAILED DESCRIPTION

[0019] ​In order to make the above objectives, characteristics and advantages of the present application more apparent, more comprehensible, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0020] Since the existing TOF camera intrinsic parameter calibration method is based on image features, for low-resolution images, accurate extraction cannot be performed, which is prone to cause large errors, and a very large number of pictures need to be obtained, which consumes time and reduces production efficiency. Based on this, an embodiment of a low-resolution TOF area array intrinsic parameter calibration method is proposed, as shown in Figures 1 to 2 The steps are as follows:

[0021] Step S10: The TOF camera module to be calibrated is arranged in parallel with a white wall, and light is emitted to the white wall.

[0022] In this embodiment, the white wall is selected because the white wall has a 90% reflectivity to light, which can more easily obtain a depth image.

[0023] Step S20: The TOF camera module acquires a depth image at a set position and obtains a measured distance d m of a set pixel point in the depth image.

[0024] In this embodiment, step S20 includes:

[0025] Step S201: The TOF camera module acquires a set number of first depth images at a first set position and obtains a first measured distance d m1 of a set pixel point in the first depth image.

[0026] Step S202: The TOF camera module acquires a set number of second depth images at a second set position and obtains a second measured distance d m2 of a set pixel point in the second depth image.

[0027] Step S203: The difference between the first measured distance and the second measured distance is calculated to obtain a measured distance d m of the depth image: d m1 = d m2 .

[0028] In this embodiment, the first set position is closer to the position of the white wall than the second set position.

[0029] In this embodiment, because there is no depth calibration of the TOF camera module, there is an error between the depth information directly obtained from the TOF camera module and the real information. For example, the depth obtained by the TOF camera module without calibration is 90 cm or other distance when the real distance is 50 cm. The TOF camera module without calibration cannot be used to calculate the intrinsic parameters. Therefore, the error can be eliminated by subtracting the values of two positions.

[0030] Step S30: A quadratic function formula d m = a(i-cy) 2 + b(j-cx) 2 + c is fitted by using the least square method, and the extreme point of the function is calculated, that is, the optical center coordinates (cx, cy) of the TOF camera module, wherein (i, j) is the coordinate of the set pixel point.

[0031] In this embodiment, the distribution of d m is that the value of d m at the optical center is the smallest, and the value is larger as the edge is farther. Therefore, the position of the optical center can be obtained by the extreme value.

[0032] Step S40: The coordinates (x, y, z) of the set pixel point in the Cartesian coordinate system are obtained according to the conversion relationship between the polar coordinate system and the Cartesian coordinate system.

[0033] In this embodiment, y = z*(i-cy) / fy x = z*(j-cx) / fx, wherein (i, j) is the coordinate position of the set pixel in the imaging plane, (cx, cy) is the optical center coordinates, fx and fy are the focal lengths of the set pixel in the x direction and the y direction respectively.

[0034] The formula of step S50 can be obtained by the above formula (1), and the relationship among the optical center coordinates (cx, cy), fx and fy is known.

[0035] Step S50: The focal length of the TOF camera module is calculated. Specifically, because most of the pixels of the chip are square, fx = fy is set, and the following formula is used: d err = d m / dep; d sq = d err * d err -1; beta = (i-cy)*(i-cy)+(j-cx)*(j-cx), d err , d sq, the value of beta. Wherein, dm is the distance from the optical center to the set pixel point, dep is the depth difference of the TOF camera module in the Cartesian coordinate system of the first set position and the second set position, and then a straight line equation formula is fitted by using the least square method:

[0036] beta = k * d sq + b, the slope is k, and the focal length f is sqrt(k).

[0037] The derivation process of beta = k * d sq + b is described below.

[0038] Assuming that the distance from the optical center to the set pixel point (that is, the imaging point) is d m , the angle θ between the camera internal focal length f and d m is calculated as follows:

[0039] And dist = (i-cy) 2 + (j-cx) 2 + f 2 , so we get:

[0040]

[0041] Transforming the formula, we get:

[0042]

[0043] Let fx = fy = f, we get:

[0044] Let beta = (i-cy) 2 + (j-cx) 2 , k = f 2 , The relationship between beta, k, and d sq is beta = k * d sq + b.

[0045] It should be noted that for the method embodiment, in order to simply describe, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the order of the described actions, because according to the embodiments of the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0046] Referring to Figure 2, show a structure block diagram of a low resolution TOF area array inner parameter calibration device embodiment of the application, which can include the following modules:

[0047] The emission control module is configured to arrange the TOF camera module to be calibrated in parallel with the white wall and emit light to the white wall.

[0048] The distance measurement module is configured to collect a depth image at a set position by the TOF camera module and obtain a measurement distance d of a set pixel point in the depth image. m

[0049] The optical center coordinate calculation module is configured to fit a quadratic function formula d = a(i-cy) + b(j-cx) + c by using the least square method, calculate the extreme point of the function, i.e., the optical center coordinate (cx, cy) of the TOF camera module, and obtain the coordinates (x, y, z) of the set pixel point in the Cartesian coordinate system according to the conversion relationship between the polar coordinate system and the Cartesian coordinate system. m 2 2

[0050] The coordinate conversion module is configured to obtain the coordinates (x, y, z) of the set pixel point in the Cartesian coordinate system according to the conversion relationship between the polar coordinate system and the Cartesian coordinate system. y = z*(i-cy) / fy, x = z*((j-cx) / fx, wherein (i, j) is the coordinate position of the set pixel, (cx, cy) is the optical center coordinate, fx and fy are the focal lengths of the set pixel in the x direction and the y direction, respectively.

[0051] The focal length calculation module is configured to calculate the focal length of the TOF camera module, specifically, obtain the values of derr, dsq, and beta by using the following formulas: derr = dm / dep, dsq = derr*derr-1, and beta = (i-cy)*(i-cy)+(j-cx)*(j-cx), wherein dm is the distance from the optical center to the set pixel point, and dep is the depth of the TOF camera module in the Cartesian coordinate system, and then fit a straight line equation formula: beta = k*dsq+b by using the least square method, wherein k is the slope, and f = sqrt(k) is the focal length.

[0052] For the device embodiment, it is basically similar to the method embodiment, so the description is relatively simple, and the relevant parts are described in the method embodiment.

[0053] The application also provides an electronic device, which can include a processor, a memory, and a computer program stored on the memory and capable of running on the processor, and the computer program is executed by the processor to implement the low resolution TOF area array inner parameter calibration method as described above.

[0054] ​​​​The embodiment of the present application further provides a nonvolatile readable storage medium, and the nonvolatile readable storage medium stores a computer program. The computer program is executed by a processor to implement the low-resolution TOF area array inner parameter calibration method.

[0055] The embodiment of the present application has the following advantages: in the embodiment of the present application, a quadratic function equation is fitted according to the relationship between the optical center coordinates and the set pixel coordinate points, the optical center coordinates are calculated through extreme value, and the focal length can be obtained through setting the pixel point coordinates in the Cartesian coordinate system and converting into a linear equation. The present application does not need to extract image features, and thus does not need high-resolution images. The present application has great advantages for low-resolution application occasions. Moreover, the present application only needs to take pictures at set positions, and high-precision calculation can be completed.

[0056] Each of the embodiments in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other.

[0057] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0058] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the method, terminal device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The device for implementing the functions specified in one or more flows and / or blocks. Figure 1 The device for implementing the functions specified in one or more flows and / or blocks.

[0059] These computer program instructions can also be stored in a computer readable storage medium, which can guide the computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams.Figure 1 one or more processes and / or blocks Figure 1 the function specified in the one or more blocks or one or more blocks.

[0060] These computer program instructions can also be loaded into computer or other programmable data processing terminal devices, so that a series of operational steps are performed on the computer or other programmable terminal devices to generate a computer-implemented process, thus the instructions executed on the computer or other programmable terminal devices provide a process for implementing the functions specified in the flowchart Figure 1 one or more processes and / or blocks Figure 1 the function specified in the one or more blocks or one or more blocks.

[0061] Although the preferred embodiments of the present application have been described, those skilled in the art who have acquired the basic inventive concept can make further changes and modifications to the embodiments. Therefore, the appended claims are intended to include the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0062] Finally, it should also be noted that, in this paper, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or terminal device. Without more limitations, the element defined by the statement "including a…" does not exclude the presence of additional identical elements in the process, method, article or terminal device including the element.

[0063] The above provides a method, device, equipment and medium for face detection, and the principle and implementation mode of the present application are described by applying specific examples. The above embodiment description is only used to help understand the method and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description should not be understood as a limitation of the present application.

Claims

1. A method for calibrating intrinsic parameters of a low-resolution TOF area array, characterized in that, The method comprises: The TOF camera module to be calibrated is arranged in parallel with a white wall, and light is emitted to the white wall; The TOF camera module acquires a depth image at a set position, and obtains a measured distance d of a set pixel point of the TOF camera module in the depth image m ; Using the least squares method, we can fit the quadratic function formula: d m =a(i-cy) 2 +b(j-cx) 2 +c, calculate the extreme point of the function, and obtain the optical center coordinates (cx, cy) of the TOF camera module, where (i, j) is the coordinate of the set pixel point; According to the conversion relationship between the polar coordinate system and the Cartesian coordinate system, the coordinates (x, y, z) of the set pixel point in the Cartesian coordinate system are obtained. y=z*(i-cy) / fy, x=z*(j-cx) / fx, where (i, j) is the coordinate position of the set pixel, (cx, cy) is the optical center coordinate, fx and fy are the focal lengths of the set pixel in the x and y directions, respectively; The focal length of the TOF camera module is calculated, specifically, by the following formula: derr = dm / dep, dsq = derr*derr-1, beta = (i-cy)*(i-cy)+(j-cx)*(j-cx); values of derr, dsq and beta are obtained, wherein dm is the distance from the optical center to the set pixel point, dep is the depth of the TOF camera module in the Cartesian coordinate system, then a straight line equation formula is fitted by using the least square method: beta = k*dsq+b, the slope is k, and the focal length size is f = sqrt(k). The device comprises:

2. The method of claim 1, wherein, In the TOF camera module in the set position to collect depth image, and obtain the set pixel in the depth image in the measured distance d m In step, comprising: The TOF camera module acquires a set number of first depth images at a first set position, and obtains a first measured distance d of a set pixel point in the first depth image m1 ; The TOF camera module acquires a second depth image of a set number at a second set position, and obtains a second measured distance d of the set pixel point in the second depth image m2 ; calculating a difference between the first measured distance and the second measured distance, obtaining a measured distance d of the depth image m = d m1 - d m2 .

3. A low-resolution TOF area array intrinsic calibration device, characterized in that, The emission control module is configured to arrange the TOF camera module to be calibrated in parallel with a white wall, and emit light to the white wall; The focal length calculation module is configured to calculate the focal length of the TOF camera module, specifically, by the following formula: derr = dm / dep, ds = derr*derr-1, beta = (i-cy)*(i-cy)+(j-cx)*(j-cx); values of derr, dsq and beta are obtained, wherein dm is the distance from the optical center to the set pixel point, dep is the depth of the TOF camera module in the Cartesian coordinate system, then a straight line equation formula is fitted by using the least square method: beta = k*dsq+b, the slope is k, and the focal length size is f = sqrt(k). Distance measurement module: for the TOF camera module to collect depth image at a set position, and obtain the measured distance d of a set pixel point in the depth image m ; Optical center coordinate calculation module: used to fit the quadratic function formula using the least squares method: d m =a(i-cy) 2 +b(j-cx) 2 +c, calculate the extreme point of the function, that is, the optical center coordinates (cx, cy) of the TOF camera module, where (i, j) is the coordinate of the set pixel point; The coordinate conversion module is configured to obtain coordinates (x, y, z) of the set pixel point in the Cartesian coordinate system according to a conversion relationship between the polar coordinate system and the Cartesian coordinate system, y = z * (i - cy) / fy, x = z * (j - cx) / fx, wherein (i, j) is a coordinate position of the set pixel, (cx, cy) is the optical center coordinate, fx and fy are focal lengths of the set pixel in the x direction and the y direction, respectively. The non-volatile readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1 to 2.

4. An electronic device, comprising: The non-volatile readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1 to 2.

5. A non-volatile readable storage medium, characterized by ​

Images