Camera calibration method, camera, and computer-readable storage medium

Abstract

The present invention relates to a camera calibration method, a camera and a computer-readable storage medium. The method includes: obtaining a first output signal matrix of the array during the process of the camera capturing an intensity map under lightless conditions, where the first output signal matrix includes first output signal values of each pixel in the array; obtaining a second output signal matrix of the array during the process of the camera capturing a preset plane under a first preset light intensity condition, where the second output signal matrix includes second output signal values of each pixel in the array; obtaining a detection efficiency matrix of the array according to the first output signal matrix and the second output signal matrix, where the detection efficiency matrix includes detection efficiency values of each pixel in the array; calibrating the real-time intensity map of the camera according to the first output signal matrix and the detection efficiency matrix to obtain a calibrated intensity map of the camera. The present application realizes intelligent and accurate calibration of the internal parameters of the camera lens.

Description

Technical Field

[0001] The present invention relates to the technical field of cameras, and particularly to a camera calibration method, a camera and a computer-readable storage medium. Background Art

[0002] A Time of Flight (ToF) camera system is a range imaging camera system that uses the time-of-flight technology to measure the distance information between the camera and the object by measuring the round-trip time of artificial light. Compared with other systems, the ToF camera has many advantages such as simple and compact structure, wide application range, high speed and high precision, and is widely used in technical fields such as intelligent detection, ranging and image recognition.

[0003] Due to factors such as the position deviation of the sensor array, the actual working error of the sensor, and the parameter error of the camera lens, the ToF camera needs to be calibrated before use to improve the measurement accuracy of the ToF camera.

[0004] However, the traditional ToF camera calibration method requires removing the camera lens first and calibrating the offset of each pixel in the sensor of the camera one by one. The calibration steps are cumbersome, the efficiency is low, and the calibration accuracy depends highly on the operator, which greatly increases the mass production cost of the camera. Summary of the Invention

[0005] Based on this, in view of the technical problems in the above-mentioned background art that the traditional camera calibration method has cumbersome steps, low efficiency and high dependence on the operator, resulting in an increase in the mass production cost of the camera, it is necessary to provide an intelligent camera calibration method, a camera and a computer-readable storage medium, which can significantly improve the efficiency and accuracy of camera calibration and effectively reduce the mass production cost of the camera.

[0006] To achieve the above object and other objects, a first aspect of the present application provides a camera calibration method, where the camera includes a photoelectric sensor array, and the method includes:

[0007] Obtaining a first output signal matrix of the array during the process of the camera capturing an intensity map under lightless conditions, where the first output signal matrix includes first output signal values of each pixel in the array;

[0008] Obtaining a second output signal matrix of the array during the process of the camera capturing a preset plane under a first preset light intensity condition, where the second output signal matrix includes second output signal values of each pixel in the array;

[0009] Obtaining a detection efficiency matrix of the array according to the first output signal matrix and the second output signal matrix, where the detection efficiency matrix includes detection efficiency values of each pixel in the array;

[0010] Calibrate the real-time intensity map of the camera according to the first output signal matrix and the detection efficiency matrix to obtain the calibrated intensity map of the camera.

[0011] In the camera calibration method in the above embodiment, first, obtain the first output signal matrix of the array during the process of the camera capturing the intensity map under lightless conditions, where the first output signal matrix includes the first output signal values of each pixel in the array; then obtain the second output signal matrix of the array during the process of the camera capturing a preset plane under the first preset light intensity condition, where the second output signal matrix includes the second output signal values of each pixel in the array; and based on the first output signal matrix and the second output signal matrix, obtain the detection efficiency matrix of the array, where the detection efficiency matrix includes the detection efficiency values of each pixel in the array. Furthermore, calibrate the real-time intensity map of the camera according to the first output signal matrix and the detection efficiency matrix to obtain the calibrated intensity map of the camera, so as to achieve intelligent and accurate calibration of the internal parameters of the camera lens, significantly improve the efficiency and accuracy of camera calibration, and effectively reduce the mass production cost of the camera.

[0012] In one of the embodiments, the step of obtaining the detection efficiency matrix of the array includes:

[0013] Calculate the difference M1 - M0 between the second output signal matrix M1 and the first output signal matrix M0;

[0014] Normalize the second output signal values of each pixel in the array to obtain the median of the second output signal matrix;

[0015] Obtain the detection efficiency matrix PDE according to the ratio of the difference M1 - M0 to the median of the second output signal matrix;

[0016] PDE = (M1 - M0) / median(M1);

[0017] where median(M1) is the median of the second output signal matrix.

[0018] In the camera calibration method of the above embodiments, during the process of obtaining the intensity map captured by the camera under lightless conditions, the first output signal matrix M0 of the array is acquired to obtain the dark count rate (DCR) distribution matrix M0 of the photoelectric sensor array in the camera; during the process of obtaining the second output signal matrix M1 of the array when the camera captures a preset plane, such as a white wall plane, under the first preset light intensity condition, the difference M1 - M0 between the second output signal matrix M1 and the first output signal matrix M0 is calculated, and the second output signal values of each pixel in the array are normalized according to the difference M1 - M0 to obtain the detection efficiency matrix PDE. Thus, the real-time intensity map of the camera is calibrated according to the first output signal matrix and the detection efficiency matrix to obtain the calibrated intensity map of the camera, realizing intelligent and precise calibration of the internal parameters of the camera lens, significantly improving the efficiency and accuracy of camera calibration, and effectively reducing the mass production cost of the camera.

[0019] In one of the embodiments, the step of obtaining the calibrated intensity map of the camera includes:

[0020] Obtaining the real-time output signal matrix K of the array during the process of the camera obtaining the real-time intensity map, where the real-time output signal matrix K includes the real-time output signal values of each pixel in the array;

[0021] Calculating the calibrated matrix M according to the following formula, where the calibrated matrix M includes the calibrated output signal values of each pixel in the array:

[0022] M = (K – M0) / PDE;

[0023] Obtaining the calibrated intensity map according to the calibrated matrix M.

[0024] In the camera calibration method of the above embodiments, after obtaining the detection efficiency matrix PDE, during the process of the camera obtaining the real-time intensity map, the real-time output signal matrix K of the array is acquired, and the real-time intensity map of the camera is calibrated according to the DCR distribution matrix M0 and the detection efficiency matrix PDE to obtain the calibrated intensity map of the camera, realizing intelligent and precise calibration of the internal parameters of the camera lens, significantly improving the efficiency and accuracy of camera calibration, and effectively reducing the mass production cost of the camera.

[0025] In one of the embodiments, the camera calibration method further includes:

[0026] Obtaining a pixel coordinate matrix including the coordinate values of each pixel and the optical center coordinate;

[0027] Calculate a compensation coefficient matrix of the array according to the optical center coordinates and the pixel coordinate matrix, where the compensation coefficient matrix includes compensation coefficient values of each pixel in the array;

[0028] Obtain an offset matrix of the array according to the compensation coefficient matrix, where the offset matrix includes optical center offset values of each pixel in the array;

[0029] Calibrate a real-time depth map of the camera according to the compensation coefficient matrix and the offset matrix to obtain a calibrated depth map of the camera.

[0030] In the camera calibration method in the above embodiment, first obtain a pixel coordinate matrix including coordinate values of each pixel and an optical center coordinate; calculate a compensation coefficient matrix of the array according to the optical center coordinate and the pixel coordinate matrix, where the compensation coefficient matrix includes compensation coefficient values of each pixel in the array; obtain an offset matrix of the array according to the compensation coefficient matrix, where the offset matrix includes optical center offset values of each pixel in the array; and then calibrate a real-time depth map of the camera according to the compensation coefficient matrix and the offset matrix to obtain a calibrated depth map of the camera, so as to avoid depth measurement errors caused by position offsets of pixels in the photoelectric sensor array and reduce the accuracy of camera depth information measurement.

[0031] In one embodiment, the obtaining the optical center coordinates includes:

[0032] Obtain a test intensity map of an object including feature points captured by the camera, where the test intensity map is captured with the feature points located on the optical axis of the camera lens;

[0033] Obtain the optical center coordinates based on the test intensity map.

[0034] In one embodiment, the obtaining the optical center coordinates includes:

[0035] Based on the camera capturing a plane of a planar object at a first distance from the surface of the camera lens to obtain a first calibrated depth information map, where the surface of the camera lens is parallel to the plane;

[0036] Obtain a second calibrated depth information map when the camera captures the plane at a second distance from the surface of the camera lens, where the first distance is not equal to the second distance;

[0037] Determine the optical center coordinates according to the first calibrated depth information map and the second calibrated depth information map.

[0038] In one embodiment, the step of calculating the compensation coefficient matrix of the array according to the optical center coordinates and the pixel coordinate matrix includes:

[0039] Determine that the optical center coordinates are (Cx, Cy) and the pixel coordinate values in the pixel coordinate matrix are (Px i , Py j );

[0040] For the compensation coefficient value factor of the pixel in the i-th row and j-th column of the array ij , it is calculated according to the following formula;

[0041]

[0042] where L is the number of rows of the array, N is the number of columns of the array, i is a positive integer, j is a positive integer, and EFL is the equivalent focal length of the camera lens.

[0043] In one embodiment, the step of obtaining the offset matrix of the array according to the compensation coefficient matrix includes:

[0044] Obtain the real-time depth map of the preset plane captured by the camera to obtain the real-time depth value matrix S LN , the lens surface of the camera is parallel to the plane, and the real-time depth value matrix S LN includes the real-time depth value S of each pixel in the array capturing the plane ij ;

[0045] Obtain the minimum distance value T0 between the lens surface and the plane;

[0046] According to the real-time depth value matrix S LN and the minimum distance value T0, calculate the offset matrix offset of the array using the following formula LN ;

[0047] offset ij = S ij - T0 / factor ij .

[0048] The second aspect of the present application provides a camera, including a photoelectric sensor array, a memory, a processor, and a computer program stored on the memory and executable on the processor. The photoelectric sensor array is connected to the processor, and when the processor executes the computer program, it implements the steps of any method in the embodiments of the present application.

[0049] In one embodiment, the photoelectric sensor array is a single-photon avalanche diode array.

[0050] The third aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of any method in the embodiments of the present application.

[0051] In the camera or computer-readable storage medium in the above embodiments, first, during the process of obtaining the intensity map captured by the camera under lightless conditions, the first output signal matrix of the array is obtained, and the first output signal matrix includes the first output signal values of the pixels in the array; then, during the process of the camera capturing a preset plane under the first preset light intensity condition, the second output signal matrix of the array is obtained, and the second output signal matrix includes the second output signal values of the pixels in the array; to obtain the detection efficiency matrix of the array according to the first output signal matrix and the second output signal matrix, the detection efficiency matrix includes the detection efficiency values of the pixels in the array, and then calibrate the real-time intensity map of the camera according to the first output signal matrix and the detection efficiency matrix to obtain the calibrated intensity map of the camera, so as to realize the intelligent and accurate calibration of the internal parameters of the camera lens, significantly improve the efficiency and accuracy of camera calibration, and effectively reduce the mass production cost of the camera. Brief Description of the Drawings

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0053] Figure 1 It is a schematic flowchart of a camera calibration method provided in an embodiment of the present application;

[0054] Figure 2 It is a schematic structural diagram of a camera provided in an embodiment of the present application;

[0055] Figure 3 It is a schematic flowchart of a camera calibration method provided in another embodiment of the present application;

[0056] Figure 4 It is a schematic structural diagram of a camera provided in another embodiment of the present application. Detailed Description of the Embodiments

[0057] To facilitate the understanding of the present application, the present application will be described more comprehensively below with reference to the relevant drawings. The preferred embodiments of the present application are shown in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure content of the present application more thorough and comprehensive.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0059] In the case of using "including", "having", and "comprising" described herein, unless explicit limiting terms are used, such as "only", "consisting of", etc., another component may also be added. Unless otherwise mentioned, terms in the singular form may include the plural form and should not be construed as having a quantity of one.

[0060] It should be understood that although terms such as "first" and "second" may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element.

[0061] In the description of this application, it should be noted that unless otherwise clearly specified and limited, the terms "installed", "connected", and "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a direct connection or an indirect connection through an intermediate medium, and it may be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0062] After the direct Time of Flight (dToF) camera is installed, due to the working parameter errors of each component inside the camera and the calibration parameter errors of the camera itself, it is necessary to calibrate the dToF camera before it actually leaves the factory or before the camera takes pictures to avoid the adverse effects brought by the parameter errors of the camera itself.

[0063] Please refer to Figure 1 , in an embodiment of this application, a camera calibration method is provided. The camera includes a photoelectric sensor array, and the method includes the following steps:

[0064] Step 202: Obtain a first output signal matrix of the array during the process of the camera capturing an intensity map under lightless conditions. The first output signal matrix includes first output signal values of each pixel in the array;

[0065] Step 204: Obtain a second output signal matrix of the array during the process of the camera capturing a preset plane under a first preset light intensity condition, where the second output signal matrix includes second output signal values of each pixel in the array;

[0066] Step 206: Obtain a detection efficiency matrix of the array according to the first output signal matrix and the second output signal matrix, where the detection efficiency matrix includes detection efficiency values of each pixel in the array;

[0067] Step 208: Calibrate a real-time intensity map of the camera according to the first output signal matrix and the detection efficiency matrix to obtain a calibrated intensity map of the camera.

[0068] As an example, please refer to Figure 2 , in camera 10, there is a photoelectric sensor array 11. The pixels in the photoelectric sensor array 11 may include at least one of a photo-diode (PD), an ambient light sensor (ALS), or a single photon avalanche diode (SPAD). A photoelectric sensor may contain several pixels. For example, the photoelectric sensor array 11 may include an L-row and N-column SPAD array 111 and a time-to-digital converter (TDC) 112. Among them, the SPAD array is communicatively connected to the time-to-digital converter 112 for calculating the number of times the SPAD array is triggered; the time-to-digital converter 112 is connected to the processor 12 for transmitting the obtained number of times the SPAD array is triggered to the processor 12. SPAD is a binary device. It biases a PN junction under a bias voltage close to avalanche. A small number of carriers excited by a weak optical signal pass through the field region close to avalanche, and the number doubles due to impact ionization, thus obtaining a larger electrical signal. Therefore, SPAD has only two states: "having an output signal" and "not having an output signal". In this embodiment, the time-to-digital converter 112 is used to record the number of times the SPAD array is triggered to indirectly measure the intensity of incident light. For example, when the ambient light intensity value is 10 kLux, the time-to-digital converter can be used to record the number of times the SPAD array is triggered by ambient light.

[0069] As an example, please continue to refer to Figure 1, first, by obtaining the first output signal matrix of the array during the process of the camera capturing the intensity map under lightless conditions, the first output signal matrix includes the first output signal values of each pixel in the array; then obtaining the second output signal matrix of the array during the process of the camera capturing a preset plane under a first preset light intensity condition, the second output signal matrix includes the second output signal values of each pixel in the array; to obtain the detection efficiency matrix of the array according to the first output signal matrix and the second output signal matrix, the detection efficiency matrix includes the detection efficiency values of each pixel in the array, and further calibrating the real-time intensity map of the camera according to the first output signal matrix and the detection efficiency matrix to obtain the calibrated intensity map of the camera, realizing intelligent and precise calibration of the internal parameters of the camera lens, significantly improving the efficiency and accuracy of camera calibration, and effectively reducing the mass production cost of the camera.

[0070] Further, in an embodiment of the present application, the step of obtaining the detection efficiency matrix of the array includes:

[0071] Calculating the difference M1 - M0 between the second output signal matrix M1 and the first output signal matrix M0;

[0072] Normalizing the second output signal values of each pixel in the array to obtain the median of the second output signal matrix;

[0073] Obtaining the detection efficiency matrix PDE according to the ratio of the difference M1 - M0 to the median of the second output signal matrix;

[0074] PDE = (M1 - M0) / median(M1);

[0075] Wherein, median(M1) is the median of the second output signal matrix, and the median function is a computer function that can return the median of a given set of values. The median is the value in the middle of a set of values. If the parameter set contains an even number of numbers, the median function will return the average of the two middle numbers.

[0076] As an example, the dark count rate (DCR) distribution matrix M0 of the photoelectric sensor array in the camera can be obtained by acquiring the first output signal matrix M0 of the array during the process of the camera capturing the intensity map under lightless conditions; the second output signal matrix M1 of the array can be obtained by acquiring the camera capturing a preset plane, such as a white wall plane, under the first preset light intensity condition, and the difference M1 - M0 between the second output signal matrix M1 and the first output signal matrix M0 can be calculated. The second output signal values of each pixel in the array are normalized to obtain the median of the second output signal matrix; the detection efficiency matrix PDE is obtained according to the ratio of the difference M1 - M0 to the median of the second output signal matrix. Thus, the real-time intensity map of the camera is calibrated according to the first output signal matrix and the detection efficiency matrix, and the calibrated intensity map of the camera is obtained, realizing intelligent and precise calibration of the internal parameters of the camera lens, significantly improving the efficiency and accuracy of camera calibration, and effectively reducing the mass production cost of the camera.

[0077] Further, in an embodiment of the present application, the step of obtaining the calibrated intensity map of the camera includes:

[0078] Acquire the real-time output signal matrix K of the array during the process of the camera acquiring the real-time intensity map, where the real-time output signal matrix K includes the real-time output signal values of each pixel in the array;

[0079] Calculate the calibrated matrix M according to the following formula, where the calibrated matrix M includes the calibrated output signal values of each pixel in the array:

[0080] M = (K – M0) / PDE;

[0081] Obtain the calibrated intensity map according to the calibrated matrix M.

[0082] Specifically, after obtaining the detection efficiency matrix PDE, acquire the real-time output signal matrix K of the array during the process of the camera acquiring the real-time intensity map, and calibrate the real-time intensity map of the camera according to the DCR distribution matrix M0 and the detection efficiency matrix PDE to obtain the calibrated intensity map of the camera, realizing intelligent and precise calibration of the internal parameters of the camera lens, significantly improving the efficiency and accuracy of camera calibration, and effectively reducing the mass production cost of the camera.

[0083] Please refer to Figure 3 , in an embodiment of the present application, a camera calibration method is provided. After calibrating the intensity map, the following steps are further included:

[0084] Step 2092: Acquire the pixel coordinate matrix including the coordinate values of each pixel and the optical center coordinate.

[0085] Step 2094: Calculate a compensation coefficient matrix of the array according to the optical center coordinates and the pixel coordinate matrix, where the compensation coefficient matrix includes compensation coefficient values of each pixel in the array.

[0086] Step 2096: Obtain an offset matrix of the array according to the compensation coefficient matrix, where the offset matrix includes optical center offset values of each pixel in the array.

[0087] Step 2098: Calibrate a real-time depth map of the camera according to the compensation coefficient matrix and the offset matrix to obtain a calibrated depth map of the camera.

[0088] As an example, please refer to Figure 2 and Figure 3 , since there may be position deviations of each pixel in the photoelectric sensor array 11, resulting in inaccurate depth information obtained from the depth map captured by the camera 10. To avoid the adverse effects of such position deviations of the internal pixels of the camera on the measurement accuracy of the depth information of the camera, after calibrating the intensity map, an object including feature points is set as the shooting target object for obtaining the test intensity map, and the feature points of the shooting target object are set to be on the optical axis of the camera lens, and then the target object is shot to obtain the test intensity map. Based on the test intensity map, a pixel coordinate matrix including coordinate values of each pixel is obtained. One photoelectric sensor may include several pixels; a compensation coefficient matrix of the array is calculated according to the optical center coordinates and the pixel coordinate matrix, where the compensation coefficient matrix includes compensation coefficient values of each pixel in the array; an offset matrix of the array is obtained according to the compensation coefficient matrix, where the offset matrix includes optical center offset values of each pixel in the array; thereby, the real-time depth map of the camera is calibrated according to the compensation coefficient matrix and the offset matrix to obtain a calibrated depth map of the camera, avoiding depth measurement errors caused by position offsets of the pixels in the photoelectric sensor array and reducing the accuracy of the camera depth information measurement.

[0089] As an example, in an embodiment of the present application, the obtaining of the optical center coordinates may include the following steps:

[0090] Obtain a test intensity map of an object including feature points captured by the camera, where the test intensity map is captured with the feature points on the optical axis of the camera lens;

[0091] Obtain the optical center coordinates based on the test intensity map.

[0092] As an example, in an embodiment of the present application, the obtaining of the optical center coordinates includes:

[0093] Based on the camera photographing a plane object at a first distance from the surface of the camera lens to obtain a first calibrated depth information map, the surface of the camera lens is parallel to the plane;

[0094] Obtain a second calibrated depth information map when the camera photographs the plane at a second distance from the surface of the camera lens, and the first distance is not equal to the second distance;

[0095] Determine the optical center coordinates according to the first calibrated depth information map and the second calibrated depth information map.

[0096] In the above embodiments, two methods for obtaining the optical center coordinates of the camera are exemplarily given. Other methods can also be used in this application to obtain the optical center coordinates, which will not be elaborated here.

[0097] Further, in an embodiment of the present application, the step of calculating the compensation coefficient matrix of the array according to the optical center coordinates and the pixel coordinate matrix includes:

[0098] Determine that the optical center coordinates are (Cx, Cy) and the pixel coordinate values in the pixel coordinate matrix are (Px i , Py j );

[0099] For the compensation coefficient value factor of the pixel in the i-th row and j-th column of the array ij , it is calculated according to the following formula;

[0100]

[0101] where L is the number of rows of the array, N is the number of columns of the array, i is a positive integer, j is a positive integer, and EFL is the equivalent focal length of the camera lens. Preferably, L≥2; N≥2.

[0102] Further, in an embodiment of the present application, the step of obtaining the offset matrix of the array according to the compensation coefficient matrix includes:

[0103] Obtain a real-time depth map of the camera photographing a preset plane to obtain a real-time depth value matrix S LN , the surface of the camera lens is parallel to the plane, and the real-time depth value matrix S LN includes the real-time depth value S of each pixel in the array photographing the plane ij ;

[0104] Obtain the minimum distance value T0 between the surface of the lens and the plane;

[0105] According to the real-time depth value matrix S LNAnd the minimum distance value T0 is used to calculate the offset matrix offset of the array by the following formula LN ;

[0106] offset ij = S ij - T0 / factor ij ,

[0107] In the above formula, i ∈ [1, L], j ∈ [1, N], L is the number of rows of the photoelectric sensor array, N is the number of columns of the photoelectric sensor array, i is a positive integer, and j is a positive integer.

[0108] Specifically, after the camera obtains the real-time depth map, according to the real-time depth value S measured by the pixels in the i-th row and j-th column of the photoelectric sensor array ij , the obtained compensation coefficient matrix factor ij and the offset matrix offset ij , the calibrated depth value T can be calculated by the following formula

[0109] T = (S ij - offset ij ) × factor ij ;

[0110] For the real-time depth value of any pixel in the camera, its optical center offset value can be subtracted from it, and the difference obtained is multiplied by the compensation coefficient of the pixel to obtain the calibrated depth value of the pixel.

[0111] In an embodiment of the present application, the first preset light intensity is set to 1 klux - 10 klux. For example, the first preset light intensity can be 1 klux, 3 klux, 5 klux, 7 klux, 9 klux or 10 klux.

[0112] In an embodiment of the present application, the first distance and the second distance are set to 0.3 m - 1.0 m. For example, the first distance can be 0.3 m, 0.5 m, 0.7 m, 0.9 m or 1.0 m; the second distance can be 0.3 m, 0.5 m, 0.7 m, 0.9 m or 1.0 m.

[0113] Please refer to Figure 4 , in an embodiment of the present application, a camera 20 is provided, including a photoelectric sensor array 11, a memory 21, a processor 12, and a computer program stored on the memory and executable on the processor. The photoelectric sensor array 11 is connected to the processor 12, and when the processor executes the computer program, the steps of the method described in any embodiment of the present application are implemented.

[0114] In one embodiment of the present application, the optoelectronic sensor array is a single-photon avalanche diode array.

[0115] Those skilled in the art can understand that Figure 4 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the camera to which the solution of the present application is applied. The specific camera may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0116] Further, in one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any embodiment of the present application are implemented.

[0117] In the camera or computer-readable storage medium in the above embodiment, first, by obtaining the first output signal matrix of the array during the process of the camera capturing an intensity map under lightless conditions, the first output signal matrix includes the first output signal values of each pixel in the array; then obtaining the second output signal matrix of the array during the process of the camera capturing a preset plane under the first preset light intensity condition, the second output signal matrix includes the second output signal values of each pixel in the array; so as to obtain the detection efficiency matrix of the array according to the first output signal matrix and the second output signal matrix, the detection efficiency matrix includes the detection efficiency values of each pixel in the array, and further calibrate the real-time intensity map of the camera according to the first output signal matrix and the detection efficiency matrix to obtain the calibrated intensity map of the camera, realizing intelligent and precise calibration of the internal parameters of the camera lens, significantly improving the efficiency and accuracy of camera calibration, and effectively reducing the mass production cost of the camera.

[0118] It should be understood that although Figure 1-2 the steps in the flowchart of are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figure 1-2 at least a part of the steps in may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0119] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0120] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0121] The above embodiments merely represent several implementation manners of the present invention. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the invention patent should be subject to the appended claims.

Claims

1. A camera calibration method, characterized in that, The camera includes a photoelectric sensor array, and the method includes: Obtaining a first output signal matrix of the array during the process of the camera capturing an intensity map under lightless conditions, where the first output signal matrix includes first output signal values of each pixel in the array; Obtaining a second output signal matrix of the array during the process of the camera capturing a preset plane under a first preset light intensity condition, where the second output signal matrix includes second output signal values of each pixel in the array; Obtaining a detection efficiency matrix of the array according to the first output signal matrix and the second output signal matrix, where the detection efficiency matrix includes detection efficiency values of each pixel in the array; Calibrating a real-time intensity map of the camera according to the first output signal matrix and the detection efficiency matrix to obtain a calibrated intensity map of the camera; Obtaining a pixel coordinate matrix including coordinate values of each pixel and a pixel of the optical center coordinate; Calculating a compensation coefficient matrix of the array according to the optical center coordinate and the pixel coordinate matrix, where the compensation coefficient matrix includes compensation coefficient values of each pixel in the array; Obtaining an offset matrix of the array according to the compensation coefficient matrix, where the offset matrix includes optical center offset values of each pixel in the array; Calibrating a real-time depth map of the camera according to the compensation coefficient matrix and the offset matrix to obtain a calibrated depth map of the camera.

2. The camera calibration method according to claim 1, wherein The step of obtaining the detection efficiency matrix of the array includes: Calculating a difference M1 - M0 between the second output signal matrix M1 and the first output signal matrix M0; Performing normalization processing on the second output signal values of each pixel in the array to obtain a median value of the second output signal matrix; Obtaining a detection efficiency matrix PDE according to a ratio of the difference M1 - M0 to the median value of the second output signal matrix; PDE = (M1 - M0) / median(M1); where median(M1) is the median value of the second output signal matrix.

3. The camera calibration method according to claim 2, wherein, The step of obtaining the calibrated intensity map of the camera includes: Obtaining a real-time output signal matrix K of the array during the process of the camera obtaining a real-time intensity map, where the real-time output signal matrix K includes real-time output signal values of each pixel in the array; Calculating a calibrated matrix M according to the following formula, where the calibrated matrix M includes calibrated output signal values of each pixel in the array: M = (K – M0) / PDE; Obtaining the calibrated intensity map according to the calibrated matrix M.

4. The camera calibration method according to any one of claims 1-3, characterized in that The obtaining of the optical center coordinate includes: Obtaining a test intensity map of the camera capturing an object including feature points, where the test intensity map is captured with the feature points located on the optical axis of the camera lens; Obtaining the optical center coordinate based on the test intensity map.

5. The camera calibration method according to any one of claims 1 to 3, characterized in that, The obtaining of the optical center coordinate includes: Based on the camera capturing a plane object at a first distance from the surface of the camera lens to obtain a first calibrated depth information map, where the surface of the camera lens is parallel to the plane; Obtaining a second calibrated depth information map when the camera captures the plane at a second distance from the surface of the camera lens, where the first distance is not equal to the second distance; Determine the optical center coordinates according to the first calibrated depth information map and the second calibrated depth information map.

6. The camera calibration method according to any one of claims 1-3, characterized in that, The step of calculating the compensation coefficient matrix of the array according to the optical center coordinates and the pixel coordinate matrix includes: Determine that the optical center coordinates are (Cx, Cy) and the pixel coordinate values in the pixel coordinate matrix are (Px i , Py j ); The compensation coefficient value factor for the pixel at the i-th row and j-th column in the array ij is calculated according to the following formula; i ∈ [1, L], j ∈ [1, N]; where L is the number of rows of the array, N is the number of columns of the array, i is a positive integer, j is a positive integer, and EFL is the equivalent focal length of the camera lens.

7. The camera calibration method according to claim 6, wherein The step of obtaining the offset matrix of the array according to the compensation coefficient matrix includes: Obtain a real-time depth map of a preset plane captured by the camera to obtain a real-time depth value matrix S LN , the lens surface of the camera is parallel to the plane, and the real-time depth value matrix S LN includes the real-time depth value S of each pixel in the array capturing the plane ij ; Obtain the minimum distance value T0 between the lens surface and the plane; According to the real-time depth value matrix S LN and the minimum distance value T0, calculate the offset matrix offset of the array using the following formula LN ; offset ij = S ij - T0 / factor ij 。 8. A camera, characterized in that, It includes a photoelectric sensor array, a memory, a processor, and a computer program stored on the memory and executable on the processor. The photoelectric sensor array is connected to the processor, and when the processor executes the computer program, the steps of the method according to any one of claims 1-7 are implemented.

9. The camera according to claim 8, characterized in that, The photoelectric sensor array is a single-photon avalanche diode array.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1-7 are implemented.

Images