A method, apparatus, and device for determining a fixed phase deviation

Abstract

An embodiment of this specification provides a method, apparatus, and device for determining a fixed phase deviation. The method includes: obtaining a plurality of sample images and the measured distance values of each pixel point in a preset area of each sample image; calibrating a target camera based on the plurality of sample images to obtain a calibration parameter set of the target camera; determining the coordinate positions of each pixel point in the preset area in the camera coordinate system according to the calibration parameter set and the coordinate positions of each pixel point in the pixel coordinate system in the preset area of each sample image; calculating the true distance value corresponding to each pixel point; and determining the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance value and the measured distance value corresponding to each pixel point in the preset area. In the embodiment of this specification, the true distance value corresponding to each pixel point can be accurately determined based on the calibration parameters, thereby effectively improving the accuracy of the determined fixed phase deviation.

Description

Technical Field

[0001] The embodiments of this specification relate to the technical field of camera calibration, and particularly to a method, apparatus, and device for determining a fixed phase deviation. Background Art

[0002] With the rapid development of optics and electronics, image sensors using processes such as germanium-silicon and pure silicon have received increasing attention. Due to design or production differences, there will be slight differences between pixels in the sensor, that is, a fixed pattern phase noise (FPPN) is generated at each pixel point. Therefore, it is necessary to determine the fixed phase deviation of each pixel point to calibrate each pixel point.

[0003] In the prior art, usually the distance between the camera and the target board is fixed, the camera is used to capture the target board to obtain a distance map, the distance error of each pixel point is obtained by calculating the difference between the measured distance and the real distance, and the fixed phase deviation is determined according to the distance error. Among them, the real distance between the camera and the target board is determined by measuring the distance from the optical center of the camera to the target board (usually attached to the wall). During the data acquisition process, it is necessary to strictly ensure that the main optical axis of the camera is perpendicular to the wall. If the main optical axis is tilted, the real distance at each pixel point will have a measurement error, resulting in an error in the FPPN calculation result. Thus, it can be seen that the technical solution in the prior art cannot accurately determine the fixed phase deviation generated at each pixel point.

[0004] For the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] The embodiments of this specification provide a method, apparatus, and device for determining a fixed phase deviation to solve the problem in the prior art that the fixed phase deviation generated at each pixel point cannot be accurately determined.

[0006] An embodiment of this specification provides a method for determining a fixed phase deviation, including: obtaining a set of sample image information; wherein, the set of sample image information includes a plurality of sample images obtained by photographing a target board using a target camera, and the measured distance values of each pixel point in a preset area of each sample image; calibrating the target camera based on the plurality of sample images to obtain a set of calibration parameters of the target camera; determining the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system according to the set of calibration parameters of the target camera and the coordinate positions of each pixel point in the preset area of each sample image in the pixel coordinate system; calculating the true distance values corresponding to each pixel point in the preset area of each sample image according to the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system; and determining the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance values and the measured distance values corresponding to each pixel point in the preset area of each sample image.

[0007] An embodiment of this specification further provides a device for determining a fixed phase deviation, including: an obtaining module, configured to obtain a set of sample image information; wherein, the set of sample image information includes a plurality of sample images obtained by photographing a target board using a target camera, and the measured distance values of each pixel point in a preset area of each sample image; a calibration module, configured to calibrate the target camera based on the plurality of sample images to obtain a set of calibration parameters of the target camera; a first determination module, configured to determine the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system according to the set of calibration parameters of the target camera and the coordinate positions of each pixel point in the preset area of each sample image in the pixel coordinate system; a calculation module, configured to calculate the true distance values corresponding to each pixel point in the preset area of each sample image according to the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system; and a second determination module, configured to determine the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance values and the measured distance values corresponding to each pixel point in the preset area of each sample image.

[0008] An embodiment of this specification further provides a device for determining a fixed phase deviation, including a processor and a memory for storing processor-executable instructions, and when the processor executes the instructions, the steps of any one of the method embodiments in the embodiments of this specification are implemented.

[0009] An embodiment of this specification further provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed, the steps of any one of the method embodiments in the embodiments of this specification are implemented.

[0010] The embodiment of this specification provides a method for determining a fixed phase deviation. It is possible to obtain a sample image information set including multiple sample images obtained by photographing a target board using a target camera and the measured distance values of each pixel point in a preset area of each sample image. Based on the multiple sample images, the target camera can be calibrated to obtain a calibration parameter set of the target camera. According to the calibration parameter set of the target camera and the coordinate positions of each pixel point in the pixel coordinate system in the preset area of each sample image, the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system can be determined. Since the coordinate positions of the points corresponding to each pixel point of the sample image in the captured image in the camera coordinate system are known, by calculating the distance between the coordinate position in the camera coordinate system and the origin of the camera coordinate system, the true distance value corresponding to each pixel point in the preset area of the sample image can be obtained, and thus the true distance between the target board and the camera can be accurately determined. Further, based on the true distance values and the measured distance values corresponding to each pixel point in the preset area of each sample image, the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera can be determined. The above implementation manner of the present invention uses the calibration parameter set of the target camera obtained by calibration and the coordinate positions of each pixel point on the sample image used for calibrating the camera in the pixel coordinate system to determine the coordinate positions of each pixel point in the camera coordinate system, and then further calculates the true distance value corresponding to each pixel point. Such a calculation does not need to consider the pose of the target camera when photographing the target board. The present invention can obtain target camera parameters related to the pose during calibration and participate in the calculation of the true distance value corresponding to each pixel point, thereby eliminating the influence of the shooting pose. Therefore, when ensuring that the target camera can clearly capture the target board, without considering the influence of the tilt of the optical axis of the target camera, the position of the target board can be placed arbitrarily, the image acquisition process is simple, the true distance value corresponding to each pixel point in the preset area of each sample image can be accurately obtained, the measurement accuracy is greatly improved, and thus the accuracy of the fixed phase deviation (FPPN) is greatly improved. Brief Description of the Drawings

[0011] The drawings described herein are used to provide a further understanding of the embodiments of this specification, form a part of the embodiments of this specification, and do not limit the embodiments of this specification. In the drawings:

[0012] Figure 1 is a schematic diagram of the steps of the method for determining the fixed phase deviation provided by the embodiment of this specification;

[0013] Figure 2 is a schematic diagram of the relationship between the pixel coordinate system, the image physical coordinate system, the camera coordinate system, and the world coordinate system provided by the embodiment of this specification;

[0014] Figure 3It is a schematic structural diagram of a device for determining a fixed phase deviation provided according to an embodiment of this specification;

[0015] Figure 4 It is a schematic structural diagram of a device for determining a fixed phase deviation provided according to an embodiment of this specification. Specific embodiments

[0016] Next, the principles and spirit of the embodiments of this specification will be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and then implement the embodiments of this specification, and do not limit the scope of the embodiments of this specification in any way. On the contrary, these embodiments are provided to make the disclosure of the embodiments of this specification more thorough and complete, and to be able to fully convey the scope of this disclosure to those skilled in the art.

[0017] Those skilled in the art know that the embodiments of the embodiments of this specification can be implemented as a system, a device, a method, or a computer program product. Therefore, the disclosure of the embodiments of this specification can be specifically implemented in the following forms, namely: completely hardware, completely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0018] Although the following described processes include multiple operations that occur in a specific order, it should be clearly understood that these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel (for example, using a parallel processor or a multi-threaded environment).

[0019] Please refer to Figure 1 , this embodiment can provide a method for determining a fixed phase deviation. The method for determining the fixed phase deviation can be used to accurately determine the true distance values corresponding to each pixel point in the preset area of the sample image by using the calibration parameter set obtained by calibrating the target camera and the coordinate positions of each pixel point in the pixel coordinate system, thereby effectively improving the accuracy of the fixed phase deviation determined for each pixel point. The above method for determining the fixed phase deviation may include the following steps.

[0020] S101: Obtain a sample image information set; wherein, the sample image information set includes a plurality of sample images obtained by using a target camera to photograph a target board, and the measured distance values of each pixel point in the preset area of each sample image.

[0021] In this embodiment, a sample image information set can be obtained. The above sample image information set can be obtained in real time during the shooting process by the target camera, or the sample images taken by the target camera can be pre-stored at a preset position and read from the preset position when needed. Specifically, it can be determined according to the actual situation, and this specification does not limit this.

[0022] In this embodiment, the sample image information set may include multiple sample images obtained by photographing a target board using a target camera, and the measured distance values of each pixel point in the preset area of each sample image. Among them, the above-mentioned measured distance values can be obtained by analyzing the raw data generated when photographing the image and through relevant calculations. The above-mentioned raw data can be a video or picture with original information photographed and recorded by the camera, including information such as sensor metadata, sensor size, color attributes, configuration files, and so on.

[0023] In this embodiment, the measured distance values of each pixel point in the preset area of each sample image may not be obtained and saved in the sample image information set in advance, and can be obtained by analysis when it is necessary to determine the fixed phase deviation.

[0024] In this embodiment, the sample image information set may include more or less data. For example, it may also include the shooting time information, etc. Specifically, it can be determined according to the actual situation, and the embodiments of this specification do not limit this.

[0025] In this embodiment, multiple sample images are obtained by photographing a target board using a target camera. The shooting angles and postures of different sample images can be different. That is to say, within the FOV of the target camera, on the premise of ensuring that the target camera can clearly capture the target board, the optical axis of the target camera does not have to be perpendicular to the target board, and the posture and position of the target board can be set arbitrarily. The size of the above-mentioned target board can be determined according to the FOV of the camera. FOV refers to the range that the lens can cover. Objects beyond this range will not be captured by the lens. The size of the above-mentioned target board can be 128 pixels × 128 pixels. The number of obtained sample images can be greater than or equal to 3. For example: 3, 8, 10, etc.; specifically, the size of the above-mentioned target board and the number of sample images can be determined according to the actual situation, and the embodiments of this specification do not limit this.

[0026] In this embodiment, the above-mentioned target camera can be a depth camera. For example: TOF camera, structured light camera, etc. The image obtained by the depth camera shooting can represent depth data. The above-mentioned TOF camera can be a measurement system based on the time-of-flight (TOF) principle, and can immediately obtain the complete three-dimensional information of the target object by emitting modulated light and detecting the round-trip time of its reflected light, so as to analyze the depth information of the target object. Of course, it can be understood that the above-mentioned target camera can also be other types of cameras, specifically, it can be determined according to the actual situation, and the embodiments of this specification do not limit this.

[0027] In this embodiment, the above target camera can be the camera to be calibrated, or can be a sample camera randomly selected from multiple cameras to be calibrated. The fixed phase deviation of the sample camera can be used to calibrate other cameras to be calibrated, and the other cameras to be calibrated can be cameras of the same production batch or cameras with image sensors using the same process. Specifically, it can be determined according to the actual situation, and the embodiments of this specification do not limit this.

[0028] S102: Calibrate the target camera based on multiple sample images to obtain a set of calibration parameters for the target camera.

[0029] In this embodiment, in order to accurately determine the true distance corresponding to each pixel point in the sample image, the target camera can be calibrated based on the above multiple sample images using the Zhang Zhengyou calibration method to obtain a set of calibration parameters for the target camera. It should be noted that the Zhang Zhengyou calibration method is only an example, and the present invention does not limit the specific calibration method, and other calibration methods that can obtain the set of calibration parameters of the target camera (including internal parameters and external parameters) can also be used. Among them, camera calibration refers to establishing the relationship between the pixel positions of the camera image and the positions of the scene points. According to the camera imaging model, from the corresponding relationship between the coordinates of the feature points in the image and the world coordinates, the parameters of the camera model are solved. The model parameters that the camera needs to be calibrated include internal parameters, external parameters, and distortion coefficients.

[0030] In this embodiment, calibrating the target camera based on multiple sample images using the Zhang Zhengyou calibration method can include: detecting the feature points in each sample image, solving the internal and external parameters of the camera in the case of ideal undistortion and improving the accuracy using maximum likelihood estimation, applying the least squares method to obtain the actual radial distortion coefficients, combining the internal parameters, external parameters, and distortion coefficients, and using the maximum likelihood method to optimize the estimation and improve the estimation accuracy, so as to obtain a set of calibration parameters for the target camera.

[0031] In this embodiment, the images for calibration can be multiple sample images in the sample image information set, or can be multiple images re - photographed from multiple angles using the target camera. Among them, the multiple images for calibration can be checkerboard images. Specifically, it can be determined according to the actual situation, and the embodiments of this specification do not limit this.

[0032] In this embodiment, the above set of calibration parameters can include camera internal parameters, camera external parameters, and distortion coefficients, etc. In some embodiments, the camera external parameters of the above set of calibration parameters specifically include the rotation matrix R and the translation matrix T, and the camera internal parameters of the above set of calibration parameters include the x - direction and y - direction coordinate values of the optical center of the target camera in the pixel coordinate system, that is, the principal point coordinate values (c x , c y ), and the distance from the optical center of the target camera to the imaging plane, that is, the focal length (fx , f y ). Specifically, it can be determined according to the actual situation, and the embodiments of this specification do not limit this.

[0033] S103: Determine the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system according to the calibration parameter set of the target camera and the coordinate positions of each pixel point in the pixel coordinate system in the preset area of each sample image.

[0034] In this embodiment, the coordinates of each pixel point in the image correspond to the position in the pixel coordinate system. To determine the true distance corresponding to each pixel point, the coordinate positions of each pixel point in the pixel coordinate system can be converted into the coordinate positions in the camera coordinate system through the calibration parameter set of the target camera obtained in step S102. That is, the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system can be determined according to the calibration parameter set of the target camera and the coordinate positions of each pixel point in the pixel coordinate system.

[0035] In this embodiment, the above-mentioned preset area can be the white area in the image after corner detection, or the central area of the sample image, or a spiral-shaped area, etc. Of course, the preset area is not limited to the above examples. Those skilled in the art may make other changes under the inspiration of the technical essence of the embodiments of this specification, but as long as the functions and effects achieved are the same or similar to those of the embodiments of this specification, they should all be covered within the protection scope of the embodiments of this specification.

[0036] In this embodiment, four coordinate systems are involved in the whole conversion process: pixel coordinate system (u, v), image physical coordinate system (x, y), camera coordinate system (X c , Y c , Z c ) and world coordinate system (X w , Y w , Z w ). The calibration parameter set of the above-mentioned target camera includes: camera internal parameters, where the camera internal parameters include internal focal lengths (f x , f y ), the x-direction and y-direction coordinate values of the optical axis of the target camera in the pixel coordinate system (i.e., the principal point coordinate values) (c x , c y ); camera external parameters, where the camera external parameters include rotation matrix R and translation matrix T; distortion coefficients (k1, k2, p1, p2, k3).

[0037] In this embodiment, the relationship between the pixel coordinate system, the image physical coordinate system, the camera coordinate system and the world coordinate system can be as Figure 2As shown in the figure, the world coordinate system is a physical coordinate system with the upper left corner of the target board as the origin ( Figure 2 not shown); the camera coordinate system has the optical center O of the target camera as the origin, the x direction is Figure 2 the Xc axis shown, the y direction is Figure 2 the Yc axis shown, and the z direction is Figure 2 the Zc axis shown; the pixel coordinate system has the upper left corner pixel of the sample image as the origin, the x direction is Figure 2 the u axis shown, and the y direction is Figure 2 the v axis shown; the image physical coordinate system has the projection point (principal point) of the optical center of the target camera (i.e., the origin O of the camera coordinate system) on the imaging plane as the origin, the x direction is Figure 2 the x axis shown, and the y direction is the y axis; Figure 2 In the figure, the principal point is the projection point of the optical center of the target camera (the origin O of the camera coordinate system) on the imaging plane, and its coordinate value in the pixel coordinate system is (i.e., the principal point coordinate value) (c x , c y ). Figure 2 The P point in the figure is a point on the target board. Since the sample image is obtained by using the target camera to photograph the target board, the P point on the target board can correspond to (u, v) in the pixel coordinate system. The coordinates of a certain pixel point (i, j) in the camera coordinate system are (X c , Y c , Z c ), the coordinates in the world coordinate system are (X w , Y w , Z w ), the coordinates in the image coordinate system are (x, y), and the coordinates in the pixel coordinate system are (u, v).

[0038] In this embodiment, the conversion relationship between the world coordinate system and the pixel coordinate system can be shown as follows:

[0039]

[0040] Among them, Z c is the depth value corresponding to the pixel point (i, j) in the sample image, that is, the coordinate value in the z direction in the camera coordinate system; (u, v) is the coordinate position of the pixel point (i, j) in the pixel coordinate system in the sample image; M c is the camera internal parameter matrix R is the rotation matrix; T is the translation matrix; (X w , Y w , Z w ) is the coordinate position of the point corresponding to the pixel point (i, j) on the photographed target board in the world coordinate system.

[0041] In this embodiment, only Z in the above formula c is unknown, and M c , R, and T can be determined according to the calibration parameter set. When the pixel point (i, j) is determined, (u, v) and (X w , Y w , Z w ) can all be directly determined through the self-defined coordinate system. In some embodiments, the upper left corner of the image can be used as the origin of the pixel coordinate system for convenience of calculation. Of course, the determination method of the pixel coordinate system is not limited to the above example. Those skilled in the art may make other changes under the inspiration of the technical essence of the embodiments of this specification, but as long as the functions and effects achieved are the same or similar to those of the embodiments of this specification, they should all be covered within the protection scope of the embodiments of this specification.

[0042] In this embodiment, from P -1 P = E, it can be obtained that:

[0043]

[0044] That is:

[0045]

[0046] In this embodiment, since there is only one unknown Z in the above formula c , therefore, the solution process of (X c , Y c , Z c ) can be as follows:

[0047]

[0048] To simplify the formula, let:

[0049]

[0050] Then, from the third row of [u, v, 1] T being 1, it can be known that:

[0051] Z c ·M1[2] = Z w + M2[2]

[0052] Among them, since the index of the matrix can be 0, 1, 2, M1[2] can represent the third element in M1; M2[2] can represent the third element in M2; the corresponding Z w is also the third element in.

[0053] According to the above formula, it can be determined that:

[0054] Zc =(Z w +M2[2]) / M1[2]

[0055] Since M2 = R -1 T, Z w are constants. Therefore, in the above embodiments of this specification, the internal parameter matrix M c , rotation matrix R, and translation matrix T of the target camera in the calibration parameter set of the target camera obtained according to step S102 can be used to calculate the z-direction coordinate value Zc of each pixel point in the camera coordinate system from the x-direction and y-direction coordinate values (u, v) of each pixel point in the pixel coordinate system.

[0056] Meanwhile, it can also be obtained that:

[0057]

[0058] Among them, the above (u, v) are the coordinate positions of the pixel point (i, j) in the sample image in the pixel coordinate system, and can be directly determined by the pixel coordinate system when the pixel point (i, j) is determined. That is to say, in the above embodiments of the present invention, the principal point coordinate values (c x , c y ) and the internal parameter focal lengths (f x , f y ) of the target camera in the calibration parameter set of the target camera obtained according to step S102 can be used to calculate the x-direction and y-direction coordinate values Xc and Yc of each pixel point in the camera coordinate system from the z-direction coordinate value Zc of each pixel point calculated above and the x-direction and y-direction coordinate values (u, v) of each pixel point in the pixel coordinate system.

[0059] In this embodiment, by repeating the above steps for each pixel point in the preset area of each sample image, the coordinate positions (X c , Y c , Z c ) of each pixel point in the preset area of each sample image in the camera coordinate system can be obtained.

[0060] S104: Calculate the true distance value corresponding to each pixel point in the preset area of each sample image according to the coordinate position of each pixel point in the preset area of each sample image in the camera coordinate system.

[0061] In this embodiment, the true distance value corresponding to each pixel point in the preset area of each sample image can be calculated according to the coordinate position of each pixel point in the preset area of each sample image in the camera coordinate system.

[0062] In this embodiment, since the coordinate position (X c , Y c , Z c ) of the point corresponding to the pixel point (i, j) in the target board to be photographed in the camera coordinate system has been obtained, therefore, by calculating the distance between the coordinate position in the camera coordinate system and the origin O of the camera coordinate system, the true distance value corresponding to the pixel point (i, j) can be obtained. Among them, the true distance corresponding to the above pixel point (i, j) can be the actual distance between the camera and the point corresponding to the pixel point (i, j) in the target board when using the camera to photograph the target board, and can also be called the actual distance between the photographing device and the object to be photographed.

[0063] In this embodiment, the origin of the above camera coordinate system can be at the optical center of the camera. In the paraxial light rays of the convex lens, the incident ray and the corresponding and parallel outgoing ray form conjugate light rays, and the intersection point of the line connecting the incident point and the outgoing point with the principal optical axis is called the focal point of the convex lens, and the point located at the center of the lens is called the optical center. Therefore, the above true distance can be the actual distance between the optical center of the camera and the point corresponding to the pixel point (i, j) in the target board.

[0064] In this embodiment, the true distance value corresponding to each pixel point in the preset area of each sample image can be calculated according to the following formula based on the coordinate position of each pixel point in the preset area of each sample image in the camera coordinate system:

[0065]

[0066] Among them, D real (i, j) is the true distance value corresponding to the pixel point (i, j); (X c , Y c , Z c ) is the coordinate position of the point corresponding to the pixel point (i, j) in the target board to be photographed in the camera coordinate system.

[0067] S105: Determine the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance value and the measured distance value corresponding to each pixel point in the preset area of each sample image.

[0068] In this embodiment, the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera can be determined based on the true distance value and the measured distance value corresponding to each pixel point in the preset area of each sample image.

[0069] In this embodiment, the difference between the measured distance value and the true distance value corresponding to a certain pixel point can be used as the fixed phase deviation of the pixel point, so that the fixed phase deviation corresponding to each pixel point in the preset area of each sample image can be obtained.

[0070] In this embodiment, the FOV refers to the range that the lens can cover, and objects beyond this range will not be captured by the lens. Therefore, the target camera can be calibrated by using the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera.

[0071] In this embodiment, since the fixed phase deviation is determined only for the preset region of each sample image, there will be some pixel points with multiple fixed phase deviation values and some pixel points without corresponding fixed phase deviation values. Therefore, the fixed phase deviation of the pixel points with multiple fixed phase deviation values can be determined by taking the mean, maximum value, or minimum value, and the fixed phase deviation of the pixel points without corresponding fixed phase deviation values can be determined by methods such as linear interpolation or Lagrange interpolation. Specifically, the mean value is calculated based on the multiple fixed phase deviation values corresponding to each pixel point in the preset region (such as the white region of the checkerboard) of each sample image, and this mean value is used as the fixed phase deviation of the target pixel point. Then, based on the fixed phase deviation of each pixel point in the preset region (such as the white region of the checkerboard target board), the fixed phase deviation of the pixel points outside the preset region in the image (such as the black region of the checkerboard target board, because the reflectivity of the black region is lower and the data accuracy obtained is relatively poor) can be determined by using linear interpolation. Of course, those skilled in the art may make other changes under the inspiration of the technical essence of the embodiments of this specification, but as long as the functions and effects achieved are the same or similar to those of the embodiments of this specification, they should be covered within the protection scope of the embodiments of this specification.

[0072] From the above description, it can be seen that the embodiments of this specification achieve the following technical effects: It is possible to obtain a sample image information set including multiple sample images obtained by photographing a target board using a target camera and the measured distance values of each pixel point in the preset area of each sample image. Based on the multiple sample images, the target camera can be calibrated to obtain a calibration parameter set of the target camera. According to the calibration parameter set of the target camera and the coordinate positions of each pixel point in the preset area of each sample image in the pixel coordinate system, the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system can be determined. Since the coordinate positions of the points corresponding to each pixel point of the sample image in the captured image in the camera coordinate system are known, therefore, by calculating the distance between the coordinate position in the camera coordinate system and the origin of the camera coordinate system, the true distance value corresponding to each pixel point in the preset area of the sample image can be obtained, so that the true distance between the target board and the camera can be accurately determined. Further, based on the true distance values and the measured distance values corresponding to each pixel point in the preset area of each sample image, the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera can be determined. Thus, it is possible to accurately determine the true distance value corresponding to each pixel point in the preset area of the sample image based on the calibration parameter set and the coordinate positions of each pixel point in the preset area of each sample image in the pixel coordinate system. Such a calculation does not need to consider the pose of the target camera when photographing the target board. The present invention can obtain target camera parameters related to the pose during calibration and participate in the calculation of the true distance value corresponding to each pixel point, thereby eliminating the influence of the shooting pose. Therefore, when ensuring that the target camera can clearly capture the target board, there is no need to consider the influence of the tilt of the optical axis of the target camera, and the position of the target board can be placed arbitrarily. The image acquisition process is simple, and the true distance value corresponding to each pixel point in the preset area of each sample image can be accurately obtained, effectively improving the measurement accuracy, and further greatly improving the accuracy of the fixed phase deviation (FPPN).

[0073] In one embodiment, obtaining the sample image information set may include: using the target camera to photograph the target board to obtain multiple sample images, and obtaining the raw data of each sample image. And according to the raw data of each sample image, determine the measured distance value of each pixel point in the preset area of each sample image. Further, a sample image information set can be generated according to the multiple sample images and the measured distance values of each pixel point in the preset area of each sample image.

[0074] In this embodiment, the distance between the target camera and the target board to be photographed can be fixed, and multiple sample images are obtained by photographing the target board with the target camera. The photographing angles of different sample images can be different. The size of the above-mentioned target board can be determined according to the FOV of the camera. FOV refers to the range that the lens can cover. Objects beyond this range will not be captured by the lens. The size of the above-mentioned target board can be 128 pixels × 128 pixels. The number of the obtained sample images can be greater than or equal to 3, for example: 3, 8, 10, etc.; specifically, the size of the above-mentioned target board and the number of sample images can be determined according to the actual situation, and the embodiments of this specification do not limit this.

[0075] In this embodiment, the raw data generated when photographing an image can be obtained. By analyzing the raw data and through relevant calculations, the measured distance values of each pixel point in the preset area of each sample image can be determined. The above-mentioned raw data can be a video or a picture with original information photographed and recorded by the camera, including information such as sensor metadata, sensor size, color attributes, configuration files, and so on.

[0076] In this embodiment, when the above-mentioned target camera is a TOF, the image obtained by photographing with the target camera can be a phase image. According to the TOF ranging principle, a depth image can be calculated based on the phase image, so as to determine the measured distance values of each pixel point in the preset area of each sample image. Of course, the determination method of the measured distance value is not limited to the above example. Those skilled in the art may make other changes under the inspiration of the technical essence of the embodiments of this specification, but as long as the functions and effects achieved are the same or similar to those of the embodiments of this specification, they should all be covered within the protection scope of the embodiments of this specification.

[0077] In this embodiment, the above-mentioned sample image information set can contain more or less data. For example: it can also contain the shooting time information, raw data, or only contain multiple sample images, etc. Specifically, it can be determined according to the actual situation, and the embodiments of this specification do not limit this.

[0078] In one embodiment, the target board is a photographing board with a checkerboard image.

[0079] Currently, it is usually to fix the distance between the TOF camera and the white target board, use the TOF camera to obtain a distance map, and obtain the distance error of each pixel point by calculating the difference between the measured distance and the real distance. In the case of using a white plane as the measurement target and needing to measure the distance from the optical center of the camera to the white plane to obtain the real distance of each pixel point, it is necessary to strictly ensure that the main optical axis of the camera is perpendicular to the placement of the white plane during the data acquisition process. Otherwise, the inclination of the optical axis will cause a deviation in the real distance measurement at each pixel point, resulting in an error in the calculation result of the fixed phase deviation.

[0080] Therefore, in this embodiment, the target board is a shooting board with a checkerboard image, and the multiple sample images obtained by shooting can be checkerboard images. Due to the characteristics of the checkerboard image itself and the method of calculating the true distance in the specification embodiments, within the FOV of the target camera, when ensuring that the target camera can clearly capture the shooting board with the checkerboard image, there is no need to consider the influence of the optical axis tilt. The position of the shooting board with the checkerboard image can be placed arbitrarily, and there is no need to pay attention to the attitude and position of the shooting board with the checkerboard image relative to the camera.

[0081] In this embodiment, thus, the camera calibration can be performed using the checkerboard image to obtain calibration parameters such as the internal parameters and external parameters of the camera, and the coordinate positions of each pixel point in the pixel coordinate system in the preset area of each sample image. Furthermore, the true distance value corresponding to each pixel can be accurately calculated.

[0082] In one embodiment, the preset area is the white area in the checkerboard image.

[0083] In this embodiment, since the reflectivity of the white area in the checkerboard image is high and the reflectivity of the black area is low, therefore, in order to better identify the preset area and improve the accuracy of the result, preferably, the preset area can be set as the white area in the image.

[0084] In this embodiment, before calibrating the target camera using the Zhang Zhengyou calibration method, corner detection can also be performed on each sample image. Corners are points where the brightness of a two-dimensional image changes violently or points with a maximum curvature on the image edge curve. By corner detection, the black-and-white boundary area of the checkerboard image can be accurately identified, and the image obtained after corner detection can be a black-and-white image. Corner detection algorithms can be classified into three categories: corner detection based on grayscale images, corner detection based on binary images, and corner detection based on contour curves.

[0085] In this embodiment, since there is no need to pay attention to the attitude and position of the shooting board with the checkerboard image relative to the camera, therefore, the sample images are obtained by shooting from multiple angles. Among them, using multiple sample images obtained by shooting from multiple angles to determine the fixed phase deviation can improve the accuracy of the determined fixed phase deviation.

[0086] In this embodiment, there is no corresponding fixed phase deviation for the pixel points in the black area of the multiple sample images obtained by shooting from multiple angles. Therefore, based on the fixed phase deviation at each pixel point in the white area, the fixed phase deviation at each pixel point in the black area can be determined by linear fitting.

[0087] In one embodiment, the pixel coordinate system has the upper left corner pixel of each sample image as the origin, and the camera coordinate system has the optical center O of the target camera as the origin. The step of determining the coordinate positions of each pixel point in the camera coordinate system according to the calibration parameter set of the target camera and the coordinate positions of each pixel point in the pixel coordinate system further includes: Based on the calibration parameter set of the target camera and the x - direction and y - direction coordinate values (u, v) of each pixel point in the pixel coordinate system, calculating the z - direction coordinate value Zc of each pixel point in the camera coordinate system. In one embodiment, the following specific calculation formula is adopted:

[0088] Z c =(Z w +M2[2]) / M1[2]

[0089] Wherein, M2 = R -1 T, Z w is a constant.

[0090] Then, based on the calculated z - direction coordinate value Zc of each pixel point in the camera coordinate system, the x - direction and y - direction coordinate values (u, v) of each pixel point in the pixel coordinate system, and the calibration parameter set of the target camera, calculate the x - direction and y - direction coordinate values Xc and Yc of each pixel point in the camera coordinate system. In one embodiment, the following specific calculation formula is adopted:

[0091]

[0092] In the above - mentioned embodiment, the calibration parameters in the calibration parameter set on which the step of calculating Zc is based include: the internal parameter matrix M c of the target camera, the rotation matrix R, and the translation matrix T; the calibration parameters in the calibration parameter set on which the steps of calculating Xc and Yc are based include: the principal point coordinate values (c x , c y ) of the target camera and the internal parameter focal lengths (f x , f y ) of the target camera.

[0093] In one embodiment, determining the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance value and the measured distance value corresponding to each pixel point in the preset region of each sample image may include: calculating the fixed phase deviation corresponding to each pixel point in the preset region of each sample image according to the true distance value and the measured distance value corresponding to each pixel point in the preset region of each sample image. The fixed phase deviations corresponding to each pixel point in the preset region of each sample image may be stitched together to obtain the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera. Further, the fixed phase deviations of each pixel point within the full FOV measurement range of the target camera may be recorded in a preset order.

[0094] In this embodiment, the fixed phase deviation corresponding to each pixel point in the preset region of each sample image may be calculated according to the following formula based on the true distance value and the measured distance value corresponding to each pixel point in the preset region of each sample image:

[0095] D fppn (i,j) = D cal (i,j) - D real (i,j)

[0096] where D fppn (i,j) is the fixed phase deviation of the pixel point (i,j); D cal (i,j) is the measured distance value corresponding to the pixel point (i,j); D real (i,j) is the true distance value corresponding to the pixel point (i,j).

[0097] In the current scheme for determining the fixed phase deviation, after obtaining the error between the true distance and the measured distance at each pixel point, it is also necessary to calculate the average distance error as a sample point by selecting a sample region, and perform a least squares surface fitting on the sample points to obtain the fixed phase deviation. This calculation process undoubtedly increases the amount of calculation and the calculation time. In this embodiment, after determining the fixed phase deviation corresponding to each pixel point in the preset region according to the difference between the true distance value and the measured distance value corresponding to each pixel point in the preset region of each sample image, the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera can be obtained by stitching, effectively improving the calculation efficiency.

[0098] In this embodiment, since the fixed phase deviation is determined only for the preset region of each sample image, there will be some pixel points with multiple fixed phase deviation values at the same time, and there will also be some pixel points without corresponding fixed phase deviation values. Therefore, the stitching process can include stitching of the same pixel points and stitching of different pixel points. Among them, the mean value, the maximum value, or the minimum value can be used to determine the fixed phase deviation of the pixel points with multiple fixed phase deviation values at the same time, and linear interpolation, Lagrange interpolation, etc. can be used to determine the fixed phase deviation of the pixel points without corresponding fixed phase deviation values. For the stitching of different pixel points, the stitching can be performed in the order of the positions of the pixel points in the image, so as to obtain the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera.

[0099] In this embodiment, after determining the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera, it can also be saved for subsequent calibration. Therefore, the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera can be recorded in a preset order.

[0100] In this embodiment, the above preset order can be the order of each pixel point in the image from left to right and from top to bottom, or from right to left and from top to bottom, etc. Of course, the preset order is not limited to the above examples. Those skilled in the art may make other changes under the inspiration of the technical essence of the embodiments of this specification, but as long as the functions and effects achieved are the same or similar to those of the embodiments of this specification, they should all be covered within the protection scope of the embodiments of this specification.

[0101] In one embodiment, stitching the fixed phase deviations corresponding to each pixel point in the preset region of each sample image to obtain the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera may include: when it is determined that there are multiple fixed phase deviation values at the target pixel point according to the fixed phase deviations corresponding to each pixel point in the preset region of each sample image, calculating the mean value of the multiple fixed phase deviations corresponding to the target pixel point. The mean value of the multiple fixed phase deviations corresponding to the target pixel point can be used as the fixed phase deviation of the target pixel point to obtain the fixed phase deviation of each pixel point in the preset region. Based on the fixed phase deviation of each pixel point in the preset region, linear interpolation can be used to determine the fixed phase deviation of the pixel points outside the preset region in the image. Further, the fixed phase deviation of each pixel point in the preset region and the fixed phase deviation of the pixel points outside the preset region in the image can be stitched to obtain the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera.

[0102] In this embodiment, since the fixed phase deviation is determined only for the preset region of each sample image, there will be some pixel points with multiple fixed phase deviation values and some pixel points without corresponding fixed phase deviation values. Therefore, when it is determined that there are multiple fixed phase deviation values at the target pixel point, the mean value of the multiple fixed phase deviations corresponding to the target pixel point can be calculated, and the mean value of the multiple fixed phase deviations corresponding to the target pixel point is used as the fixed phase deviation of the target pixel point. Repeat the above steps for each pixel point for which the fixed phase deviation has been calculated, so that the fixed phase deviations of each pixel point in the preset region can be obtained.

[0103] In this embodiment, when it is determined that there is no fixed phase deviation at the target pixel point, linear interpolation can be performed using other pixel points with existing fixed phase deviations, so that the fixed phase deviations of the pixel points outside the preset region in the image can be obtained.

[0104] In this embodiment, the fixed phase deviations of each pixel point in the preset region and the fixed phase deviations of the pixel points outside the preset region in the image can be spliced according to the position order of the pixel points in the image, so as to obtain the fixed phase deviations of each pixel point within the full FOV measurement range of the target camera.

[0105] In one embodiment, after determining the fixed phase deviations of each pixel point within the full FOV measurement range of the target camera based on the true distance values and the measured distance values corresponding to each pixel point in the preset region of each sample image, it may further include: using the fixed phase deviations of each pixel point within the full FOV measurement range of the target camera to calibrate the fixed phase deviation of the camera to be calibrated; wherein, the camera to be calibrated has the same process as the image sensor in the target camera.

[0106] In this embodiment, the target camera can be a sample camera randomly selected from multiple cameras to be calibrated. The fixed phase deviations of each pixel point within the full FOV measurement range of the sample camera can be used to calibrate the camera to be calibrated, and the camera to be calibrated can be a camera in the same production batch or a camera with an image sensor using the same process.

[0107] In this embodiment, the fixed phase deviations of each pixel point within the full FOV measurement range of the above target camera can be saved in the camera module in the form of a file, so that it can be obtained in time for fixed phase deviation calibration during shooting.

[0108] In this embodiment, compared with sensors made of other materials, the germanium-silicon sensor has better immunity to background light and has the potential to achieve higher pixel resolution. The image sensor in the target camera is preferably a germanium-silicon image sensor. Of course, it can be understood that it can also be an image sensor of other processes, such as: a pure silicon image sensor, etc., which can be specifically determined according to the actual situation, and the embodiments of this specification do not limit this.

[0109] In one embodiment, the calibration parameter set of the target camera may include: the camera internal parameters, the camera external parameters, and the distortion coefficients. In this embodiment, the camera internal parameters include the principal point coordinate values (c x , c y ) of the target camera and the internal focal lengths (f x , f y ) of the target camera. The camera external parameters include the rotation matrix R and the translation matrix T. It can be specifically determined according to the actual situation, and the embodiments of this specification do not limit this.

[0110] The above method will be described below in conjunction with a specific embodiment. However, it should be noted that this specific embodiment is only for better explaining the embodiments of this specification and does not constitute an improper limitation to the embodiments of this specification.

[0111] (1) Use the target camera to collect m (m≥3) checkerboard images from multiple angles.

[0112] (2) Detect the corner points of each checkerboard image and perform camera calibration using the Zhang Zhengyou calibration method to obtain the calibration parameters: the internal focal lengths (f x , f y ) of the camera, the principal point coordinates (c x , c y ) of the camera, the external parameters (R, T) of the camera, and the distortion coefficients (k1, k2, p1, p2, k3).

[0113] (3) Use the calibration parameters to calculate the coordinate positions (X c , Y c , Z c ) in the camera coordinate system corresponding to the pixel points (i, j) in the white area of each checkerboard image. Among them, Z c is the depth value corresponding to the pixel point (i, j).

[0114] (4) According to the following formula, calculate the true distance value corresponding to the pixel point (i, j) based on the coordinate position in the camera coordinate system:

[0115]

[0116] Among them, D real(i, j) is the true distance value corresponding to the pixel point (i, j); (X c , Y c , Z c ) is the coordinate position in the camera coordinate system of the point corresponding to the pixel point (i, j) on the target board being photographed.

[0117] (5) In a single checkerboard image, to avoid inaccurate measurement of the black area, only the white area of the checkerboard is selected for calculation. According to the following formula, calculate the FPPN of each pixel point in the white area of multiple checkerboard images:

[0118] D fppn (i, j) = D cal (i, j) - D real (i, j)

[0119] where D fppn (i, j) is the fixed phase deviation of the pixel point (i, j); D cal (i, j) is the measured distance value corresponding to the pixel point (i, j); D real (i, j) is the true distance value corresponding to the pixel point (i, j).

[0120] (6) Stitch the FPPN calculated from multiple checkerboard images. Since there may be pixel point overlaps in the white areas of multiple sample images obtained by multi-angle shooting, making one pixel point correspond to multiple fixed phase deviation values, and the white areas in multiple sample images obtained by multi-angle shooting cannot completely cover the full FOV measurement range of the target camera, therefore, there may be some pixel points without corresponding fixed phase deviation.

[0121] During the stitching process, if there are n FPPN values at the pixel point (i, j), calculate according to the following formula:

[0122]

[0123] where D fppn (i, j) is the fixed phase deviation of the pixel point (i, j); D cal (i, j) k is the k-th measured distance value corresponding to the pixel point (i, j); D real (i, j) k is the k-th true distance value corresponding to the pixel point (i, j); n is the total number of FPPN values at the pixel point (i, j).

[0124] (7) For the pixel points in the image that did not participate in the calculation, use linear interpolation to determine their corresponding FPPN values, so as to obtain the FPPN values of the full FOV measurement range of the target camera.

[0125] (8) Save the FPPN values at each pixel point as a file in the order of the pixel positions in the image to obtain the FPPN lookup table file.

[0126] In the embodiments of this specification, m checkerboard images (at least 3 images) can be collected for calibration (for example, using Zhang Zhengyou calibration method) to obtain calibration parameters (camera internal parameters, camera external parameters, and distortion coefficients). The true distance values corresponding to each pixel point in the white area of the m pieces of data can be calculated using the calibration parameters. In addition, the measured distance values corresponding to each pixel point in the white area of the m checkerboard images can be obtained by parsing the original data generated when the camera takes pictures. By calculating the difference between the measured distance and the true distance in the m checkerboard images and calculating the average value, the FPPN at each pixel point in the white area can be obtained. Finally, the FPPN at each pixel point in the black area can be obtained through linear fitting. The beneficial effects obtained by using the above embodiments may include: simple operation process. In the full FOV measurement range of the camera, when ensuring that the camera can clearly capture the checkerboard, without considering the influence of the optical axis tilt, the checkerboard position can be placed arbitrarily, and there is no need to pay attention to the pose and position of the checkerboard relative to the camera; less time-consuming, the image acquisition process is simple, and only at least three images need to be collected; high precision. Through the camera internal parameter data, external parameter data, etc., the true distance values corresponding to each pixel point in the white area of each image can be accurately obtained, effectively improving the measurement accuracy, and thus improving the accuracy of the determined FPPN.

[0127] Based on the same inventive concept, an apparatus for determining a fixed phase deviation is also provided in the embodiments of this specification, as described in the following embodiments. Since the principle of solving problems by the apparatus for determining a fixed phase deviation is similar to that of the method for determining a fixed phase deviation, the implementation of the apparatus for determining a fixed phase deviation can refer to the implementation of the method for determining a fixed phase deviation, and the repeated parts will not be described again. As used below, the term "unit" or "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the apparatuses described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated. Figure 3 is a structural block diagram of the apparatus for determining a fixed phase deviation according to the embodiments of this specification, as Figure 3 shown, and may include: an acquisition module 301, a calibration module 302, a first determination module 303, a calculation module 304, and a second determination module 305. The following describes this structure.

[0128] The acquisition module 301 can be used to acquire a sample image information set; wherein, the sample image information set includes a plurality of sample images obtained by using a target camera to photograph a target board, and the measured distance values of each pixel point in a preset area of each sample image;

[0129] The calibration module 302 can be used to calibrate the target camera based on the multiple sample images to obtain the calibration parameter set of the target camera;

[0130] The first determination module 303 can be used to determine the coordinate positions of the respective pixel points in the preset area of each sample image in the camera coordinate system according to the calibration parameter set of the target camera and the coordinate positions of the respective pixel points in the pixel coordinate system in the preset area of each sample image;

[0131] The calculation module 304 can be used to calculate the true distance values corresponding to the respective pixel points in the preset area of each sample image according to the coordinate positions of the respective pixel points in the preset area of each sample image in the camera coordinate system;

[0132] The second determination module 305 can be used to determine the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance values and the measured distance values corresponding to the respective pixel points in the preset area of each sample image.

[0133] The embodiment of the present specification also provides an electronic device, which can specifically refer to Figure 4 the schematic structural diagram of the electronic device composed of the method for determining the fixed phase deviation provided by the embodiment of the present specification. The electronic device can specifically include an input device 41, a processor 42, and a memory 43. Among them, the input device 41 can specifically be used to input the sample image information set. The processor 42 can specifically be used to obtain the sample image information set; wherein, the sample image information set includes multiple sample images obtained by using the target camera to photograph the target board, and the measured distance values of the respective pixel points in the preset area of each sample image; calibrate the target camera based on the multiple sample images to obtain the calibration parameter set of the target camera; determine the coordinate positions of the respective pixel points in the preset area of each sample image in the camera coordinate system according to the calibration parameter set of the target camera and the coordinate positions of the respective pixel points in the pixel coordinate system in the preset area of each sample image; calculate the true distance values corresponding to the respective pixel points in the preset area of each sample image according to the coordinate positions of the respective pixel points in the preset area of each sample image in the camera coordinate system; determine the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance values and the measured distance values corresponding to the respective pixel points in the preset area of each sample image. The memory 43 can specifically be used to store data such as the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera.

[0134] In this embodiment, the input device may specifically be one of the main devices for information exchange between a user and a computer system. The input device may include a keyboard, a mouse, a camera, a scanner, a light pen, a handwriting input board, a voice input device, etc.; the input device is used to input raw data and programs for processing these data into the computer. The input device may also acquire data transmitted from other modules, units, and devices. The processor may be implemented in any suitable manner. For example, the processor may take the form of, for example, a microprocessor or a processor, a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and a form embedded with a microcontroller, and so on. The memory may specifically be a memory device for storing information in modern information technology. The memory may include multiple levels. In a digital system, anything that can store binary data can be a memory; in an integrated circuit, a circuit with a storage function without a physical form is also called a memory, such as RAM, FIFO, etc.; in a system, a storage device with a physical form is also called a memory, such as a memory stick, a TF card, etc.

[0135] In this embodiment, the functions and effects specifically implemented by the electronic device may be explained by comparison with other embodiments and will not be elaborated here.

[0136] An embodiment of this specification also provides a computer storage medium for a determination method based on a fixed phase deviation. The computer storage medium stores computer program instructions, and when the computer program instructions are executed, the following can be achieved: acquiring a sample image information set; wherein, the sample image information set includes multiple sample images obtained by using a target camera to photograph a target board, and the measured distance values of each pixel point in a preset area of each sample image; calibrating the target camera based on the multiple sample images to obtain a calibration parameter set of the target camera; determining the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system according to the calibration parameter set of the target camera and the coordinate positions of each pixel point in the pixel coordinate system in the preset area of each sample image; calculating the true distance values corresponding to each pixel point in the preset area of each sample image according to the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system; and determining the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance values and the measured distance values corresponding to each pixel point in the preset area of each sample image.

[0137] In this embodiment, the storage medium includes, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), a Cache, a Hard Disk Drive (HDD), or a Memory Card. The memory can be used to store computer program instructions. The network communication unit can be set according to the standards specified by the communication protocol and is used for the interface of network connection communication.

[0138] In this embodiment, the functions and effects specifically implemented by the program instructions stored in the computer storage medium can be explained by comparison with other embodiments and will not be elaborated here.

[0139] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the embodiments of this specification can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device, so that they can be stored in the storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order than here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. In this way, the embodiments of this specification are not limited to any specific combination of hardware and software.

[0140] Although the embodiments of this specification provide method operation steps as described in the above embodiments or flowcharts, based on routine or non-creative labor, more or fewer operation steps can be included in the method. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of this specification. When the method is executed by an actual device or terminal product, it can be executed in the method order shown in the embodiments or the drawings or executed in parallel (for example, in an environment of a parallel processor or multi-threaded processing).

[0141] It should be understood that the above description is for illustrative purposes rather than for limitation. Many embodiments and many applications other than the examples provided will be obvious to those skilled in the art after reading the above description. Therefore, the scope of the embodiments of this specification should not be determined by referring to the above description, but should be determined by referring to the full scope of the foregoing claims and the equivalents of these claims.

[0142] The foregoing is only the preferred embodiment of the embodiments of this specification and is not intended to limit the embodiments of this specification. For those skilled in the art, various modifications and variations can be made to the embodiments of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of this specification shall be included within the protection scope of the embodiments of this specification.

Claims

1. A method for determining a fixed phase deviation, characterized in that Including: Obtain a sample image information set; wherein, the sample image information set includes a plurality of sample images obtained by photographing a target board using a target camera, and the measured distance values of each pixel point in a preset area of each sample image. Based on the plurality of sample images, calibrate the target camera to obtain a calibration parameter set of the target camera. According to the calibration parameter set of the target camera and the coordinate positions of each pixel point in the preset area of each sample image in the pixel coordinate system, determine the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system. According to the coordinate positions of each pixel point in the preset area of each sample image in the camera coordinate system, calculate the true distance values corresponding to each pixel point in the preset area of each sample image. Based on the true distance values and the measured distance values corresponding to each pixel point in the preset area of each sample image, determine the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera.

2. The method according to claim 1, wherein Obtaining the sample image information set includes: Use a target camera to photograph a target board to obtain a plurality of sample images. Obtain the original data of each sample image. According to the original data of each sample image, determine the measured distance values of each pixel point in the preset area of each sample image. Generate the sample image information set according to the plurality of sample images and the measured distance values of each pixel point in the preset area of each sample image.

3. The method according to claim 1, wherein The target board is a photographing board with a checkerboard image, and the preset area is the white area in the image.

4. The method according to claim 1, wherein The pixel coordinate system takes the upper left corner pixel of each sample image as the origin, and the camera coordinate system takes the optical center of the target camera as the origin. Among them, according to the calibration parameter set of the target camera and the coordinate positions of each pixel point in the pixel coordinate system, determining the coordinate positions of each pixel point in the camera coordinate system includes: According to the calibration parameter set of the target camera and the x-direction and y-direction coordinate values of each pixel point in the pixel coordinate system, calculate the z-direction coordinate value of each pixel point in the camera coordinate system: According to the z-direction coordinate value of each pixel point in the camera coordinate system, the x-direction and y-direction coordinate values of each pixel point in the pixel coordinate system, and the calibration parameter set of the target camera, calculate the x-direction and y-direction coordinate values of each pixel point in the camera coordinate system.

5. The method according to claim 4, characterized in that, Wherein: The calibration parameter set of the target camera on which the step of calculating the z-direction coordinate value of each pixel point in the camera coordinate system is based includes the internal parameter matrix Mc, rotation matrix R, and translation matrix T of the target camera. The calibration parameter set of the target camera on which the step of calculating the x-direction and y-direction coordinate values of each pixel point in the camera coordinate system is based includes the principal point coordinate value of the target camera and the internal focal length of the target camera.

6. The method according to claim 1, wherein Determining the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance value and the measured distance value corresponding to each pixel point in the preset region of each sample image, includes: Calculating the fixed phase deviation corresponding to each pixel point in the preset region of each sample image according to the true distance value and the measured distance value corresponding to each pixel point in the preset region of each sample image; Stitching the fixed phase deviations corresponding to each pixel point in the preset region of each sample image to obtain the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera; Recording the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera in a preset order.

7. The method according to claim 6, wherein Stitching the fixed phase deviations corresponding to each pixel point in the preset region of each sample image to obtain the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera, includes: When it is determined that there are multiple fixed phase deviation values at the target pixel point based on the fixed phase deviations corresponding to each pixel point in the preset region of each sample image, calculating the mean value of the multiple fixed phase deviations corresponding to the target pixel point; Taking the mean value of the multiple fixed phase deviations corresponding to the target pixel point as the fixed phase deviation of the target pixel point to obtain the fixed phase deviation of each pixel point in the preset region; Based on the fixed phase deviation of each pixel point in the preset region, determining the fixed phase deviation of the pixel points outside the preset region in the image by using linear interpolation; Stitching the fixed phase deviation of each pixel point in the preset region and the fixed phase deviation of the pixel points outside the preset region in the image to obtain the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera.

8. The method according to claim 1, characterized in that, After determining the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance value and the measured distance value corresponding to each pixel point in the preset region of each sample image, it further includes: Calibrating the fixed phase deviation of the camera to be calibrated by using the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera; wherein, the camera to be calibrated has the same process as the image sensor in the target camera.

9. The method according to claim 1, wherein The calibration parameter set of the target camera includes: camera internal parameters, camera external parameters, and distortion coefficients. Among them, the camera internal parameters include the principal point coordinate values (c x , c y ) and the internal focal length (f x , f y ) of the target camera. The camera external parameters include the rotation matrix R and the translation matrix T.

10. A device for determining a fixed phase deviation, characterized in that, Includes: An acquisition module, configured to acquire a sample image information set; wherein, the sample image information set contains multiple sample images obtained by using a target camera to photograph a target board, and the measured distance value of each pixel point in the preset region of each sample image; A calibration module, configured to calibrate the target camera based on the multiple sample images to obtain a calibration parameter set of the target camera; A first determination module, configured to determine the coordinate position of each pixel point in the preset region of each sample image in the camera coordinate system according to the calibration parameter set of the target camera and the coordinate position of each pixel point in the preset region of each sample image in the pixel coordinate system; A calculation module, configured to calculate the true distance value corresponding to each pixel point in the preset region of each sample image according to the coordinate positions of the pixel points in the preset region of each sample image in the camera coordinate system; A second determination module, configured to determine the fixed phase deviation of each pixel point within the full FOV measurement range of the target camera based on the true distance value and the measured distance value corresponding to each pixel point in the preset region of each sample image.

11. A determining device for fixed phase deviation, characterized in that, It includes a processor and a memory for storing instructions executable by the processor. When the processor executes the instructions, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that, Computer instructions are stored thereon. When the instructions are executed, the steps of the method according to any one of claims 1 to 9 are implemented.

Images