Parameter calibration method and device

Abstract

The present application provides a parameter calibration method and device. It can be applied to the calibration of the internal parameters and external parameters of a camera device. The method and device, when the position of the calibration pattern is fixed, accurately adjust the position and angle of the camera device relative to the calibration pattern by changing the posture parameters of the camera device, so that the camera device accurately shoots the calibration pattern at each predetermined position and angle relative to the calibration pattern, and obtains at least one shooting result. The above shooting result can accurately correspond to the coordinates of the pixel coordinate system and the world coordinate system, so that the accuracy of the corresponding calculated internal parameters and external parameters is higher, and the calibration accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of optical technology, and in particular to a parameter calibration method and device. Background Art

[0002] In the practical application of machine vision / computer vision and other technologies, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image captured by the camera, a geometric model of camera imaging must be established. The parameters of these geometric models can be summarized as camera parameters (or camera parameters). In most cases, camera parameters must be solved through experiments and calculations. The process of solving camera parameters through experiments and calculations is called camera calibration (or camera calibration).

[0003] The current camera calibration method fixes the camera to be calibrated, then manually holds a calibration plate in the camera's field of view and moves the calibration plate to change its angle and position, so that the camera can capture images of the calibration plate at different angles and positions; finally, the internal and external parameters of the camera are solved by jointly determining the real position of the feature points of the calibration plate in the three-dimensional world and the position in the image formed by the camera.

[0004] The current calibration plates are printed on various types of plates, and their flatness and printing accuracy will affect the accuracy of the world coordinate system. When the flatness and printing accuracy are low, the accuracy of the world coordinate system is low, which affects the calibration accuracy. In addition, the manual movement of the calibration plate makes it impossible to accurately determine the position of the calibration plate, which also affects the calibration accuracy. Summary of the invention

[0005] The present application provides a parameter calibration method and device to improve the calibration accuracy of a camera device.

[0006] In the first aspect, the present application provides a parameter calibration method. The method includes: generating a first calibration pattern; sending a first instruction, the first instruction is used to indicate at least one set of posture parameters of at least one camera device in a first time period; sending a second instruction to at least one camera device, the second instruction is used to indicate at least one of the shooting timing and the number of shots of at least one camera device in the first time period; obtaining at least one shooting result from at least one camera device; determining at least one internal parameter of at least one camera device based on part or all of the at least one shooting result, any of the part or all of the shooting results containing part or all of the first calibration pattern.

[0007] The parameter calibration method provided in the present application can be applied to the calibration of the internal parameters and external parameters of the camera device. When the position of the calibration pattern is fixed, the method accurately adjusts the position and angle of the camera device relative to the calibration pattern by changing the posture parameters of the camera device, so that the camera device can accurately shoot the calibration pattern at each predetermined position and angle relative to the calibration pattern to obtain at least one shooting result. The above shooting result can accurately correspond to the coordinates of the pixel coordinate system and the world coordinate system, so that the corresponding calculated internal parameters and external parameters are more accurate, thereby improving the calibration accuracy.

[0008] In one implementation, sending the first instruction includes: sending the first instruction to a motion device, the motion device carrying at least one camera device, and at least one set of posture parameters used by the motion device to control the posture of at least one camera device. Thus, the motion device can accurately control the posture of the camera device according to the posture parameters, and can achieve posture reproduction, which is conducive to improving calibration accuracy.

[0009] In one implementation, the shooting result includes at least one photo and / or at least one video taken in response to the second instruction from at least one camera device. Thus, the image used to calculate the internal parameters of the camera device can be selected from the photo or from the video frame of the video, which is flexible.

[0010] In one implementation, sending a first instruction includes: sending multiple first instructions within a first time period; sending a second instruction to at least one camera device includes: sending multiple second instructions to at least one camera device within the first time period; wherein the multiple first instructions and the multiple second instructions are sent alternately. Thus, the camera device can be instructed to take different photos or videos in different positions according to actual needs through the first instruction and the second instruction, for example, so that the first calibration pattern appears at various positions in the field of view of the camera device, thereby improving the calibration accuracy.

[0011] In one implementation, at least one of the internal parameters has a corresponding error threshold, and when the repetition accuracy of at least one of the internal parameters is greater than the corresponding error threshold, the method further includes: adjusting the first calibration pattern to a second calibration pattern, and / or adjusting the number of feature points corresponding to each field of view of at least one camera device extracted from at least one shooting result, and / or adjusting the image quality of the first calibration pattern. Thus, the present application can perform multiple calibrations on the camera device, so that the internal parameter matrix and the external parameter matrix of the camera device are continuously iteratively updated, and the calibration accuracy is continuously improved until the value of the calibration accuracy is less than the corresponding threshold.

[0012] In one implementation, the number and / or size of markers in the first calibration pattern and the second calibration pattern are different.

[0013] In one implementation, adjusting the number of feature points corresponding to each field of view of at least one camera device extracted from at least one shooting result includes: determining at least one sub-region included in the first calibration pattern, and uniformly adjusting the number of feature points in the at least one sub-region. Thus, the number of feature points in each sub-region is uniform, so as to improve the calibration accuracy.

[0014] In one implementation, the first calibration pattern is a preset grayscale pattern, and the internal parameters include a white balance parameter of at least one camera device.

[0015] In one implementation, the first calibration pattern is a preset monochrome pattern, and the internal parameters include a vignetting parameter of at least one camera device.

[0016] In one implementation, the first calibration pattern is a preset color card pattern containing a plurality of standard colors, and the internal parameters include a color difference parameter of at least one camera device.

[0017] In one implementation, the first calibration pattern is a preset target pattern containing lines at different angles, and the internal parameters include at least one modulation transfer function parameter of the camera device.

[0018] In the second aspect, the present application provides a parameter calibration device. The device includes: a generating unit for generating a first calibration pattern; a first sending unit for sending a first instruction, the first instruction is used to indicate at least one set of posture parameters of at least one camera device in a first time period; a second sending unit for sending a second instruction to the camera device, the second instruction is used to indicate at least one of the shooting timing and the number of shooting times of at least one camera device in the first time period; an acquiring unit for acquiring at least one shooting result from at least one camera device; a processing unit for determining at least one internal parameter of at least one camera device based on part or all of the shooting results in at least one shooting result, and any of the part or all of the shooting results contains part or all of the first calibration pattern.

[0019] The parameter calibration device provided in the present application can be applied to the calibration of the internal parameters and external parameters of the camera device. When the position of the calibration pattern is fixed, the method accurately adjusts the position and angle of the camera device relative to the calibration pattern by changing the posture parameters of the camera device, so that the camera device can accurately shoot the calibration pattern at each predetermined position and angle relative to the calibration pattern to obtain at least one shooting result. The above shooting result can accurately correspond to the coordinates of the pixel coordinate system and the world coordinate system, so that the corresponding calculated internal parameters and external parameters are more accurate, thereby improving the calibration accuracy.

[0020] In one implementation, the first sending unit is specifically used to send a first instruction to a motion device, the motion device carries at least one camera device, and at least one set of posture parameters is used by the motion device to control the posture of at least one camera parameter. Thus, the motion device can accurately control the posture of the camera device according to the posture parameters, and can achieve the reproduction of the posture, which is conducive to improving the calibration accuracy.

[0021] In one implementation, the shooting result includes at least one photo and / or at least one video taken in response to the second instruction from at least one camera device. Thus, the image used to calculate the internal parameters of the camera device can be selected from the photo or from the video frame of the video, which is flexible.

[0022] In one implementation, the first sending unit is used to send multiple first instructions within a first time period; the second sending unit is used to send multiple second instructions to at least one camera device within the first time period; wherein the multiple first instructions and the multiple second instructions are sent alternately. Thus, the camera device can be instructed to take different photos or videos in different positions according to requirements through the first instruction and the second instruction, for example, so that the first calibration pattern appears at various positions in the field of view of the camera device, thereby improving the calibration accuracy.

[0023] In one implementation, at least one of the internal parameters has a corresponding error threshold; a processing unit is used to determine whether the repetition accuracy of at least one of the internal parameters is greater than the corresponding error threshold; a generating unit is used to adjust the first calibration pattern to the second calibration pattern when the repetition accuracy is greater than the corresponding error threshold, and / or adjust the image quality of the first calibration pattern; a processing unit is used to adjust the number of feature points corresponding to each field of view of at least one camera device extracted from at least one shooting result when the repetition accuracy is greater than the corresponding error threshold. Thus, the present application can perform multiple calibrations on the camera device, so that the internal parameter matrix and the external parameter matrix of the camera device are continuously iteratively updated, and the calibration accuracy is continuously improved until the value of the calibration accuracy is less than the corresponding threshold.

[0024] In one implementation, the number and / or size of markers in the first calibration pattern and the second calibration pattern are different.

[0025] In one implementation, the processing unit is used to determine at least one sub-region included in the first calibration pattern, and uniformly adjust the number of feature points in the at least one sub-region, thereby making the number of feature points in each sub-region uniform to improve calibration accuracy.

[0026] In one implementation, the first calibration pattern is a preset grayscale pattern, and the internal parameters include a white balance parameter of at least one camera device.

[0027] In one implementation, the first calibration pattern is a preset monochrome pattern, and the internal parameters include a vignetting parameter of at least one camera device.

[0028] In one implementation, the first calibration pattern is a preset color card pattern containing a plurality of standard colors, and the internal parameters include a color difference parameter of at least one camera device.

[0029] In one implementation, the first calibration pattern is a preset target pattern containing lines at different angles, and the internal parameters include at least one modulation transfer function parameter of the camera device.

[0030] In a third aspect, the present application provides a parameter calibration device. The device includes: a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program codes, the computer program codes include computer instructions, and when the processor executes the computer instructions, the device executes the method in the first aspect and any implementation thereof.

[0031] In a fourth aspect, the present application provides a parameter calibration system. The system includes: a control device for generating a first calibration pattern; a display screen coupled to the control device for displaying the first calibration pattern; a motion device for carrying at least one camera device so that the field of view of at least one camera device faces the display screen, and the motion device is coupled to the control device; the control device is also used to send a first instruction to the motion device, the first instruction is used to indicate at least one set of posture parameters of at least one camera device in a first time period; the motion device is also used to control the posture of at least one camera device according to at least one set of posture parameters; the control device is also used to send a second instruction to at least one camera device, the second instruction is used to indicate at least one of the shooting timing and the number of shooting times of at least one camera device in the first time period; and, obtain at least one shooting result from at least one camera device; and, according to part or all of the shooting results in at least one shooting result, determine at least one internal parameter of at least one camera device, and any of the part or all of the shooting results contains part or all of the pattern of the first calibration pattern.

[0032] In a fifth aspect, the present application further provides a computer storage medium, which is a computer instruction, and when the computer instruction is executed on a control device, the control device executes the method in the first aspect and any implementation thereof.

[0033] In a sixth aspect, the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect and any implementation manner thereof.

[0034] In the seventh aspect, the present application also provides a chip system, which includes a processor for supporting the above-mentioned device or equipment to implement the functions involved in the above-mentioned first aspect and any implementation method thereof, for example, generating or processing the information involved in the above-mentioned method. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a basic structural diagram of a camera;

[0036] Figure 2 It is a schematic diagram of the mapping of objects from a three-dimensional scene to a two-dimensional image;

[0037] Figure 3 It is a schematic diagram of the transformation relationship of each coordinate system;

[0038] Figure 4 It is a schematic diagram of the camera distortion;

[0039] Figure 5 is a schematic diagram of a camera calibration method;

[0040] Figure 6 is a schematic diagram of some calibration plates shown in the embodiments of the present application;

[0041] Figure 7 is a schematic diagram of a hardware system for parameter calibration provided in an embodiment of the present application;

[0042] Figure 8 is a schematic diagram of a hardware system for parameter calibration provided in an embodiment of the present application;

[0043] Fig. 9 is a schematic diagram of the structure of a control device shown in an embodiment of the present application;

[0044] Fig.10 is a schematic diagram of the structure of a sports device shown in an embodiment of the present application;

[0045] Fig.11 is a flow chart of a parameter calibration method provided in an embodiment of the present application;

[0046] Fig.12 is a schematic diagram of adjusting the posture of a camera device provided in an embodiment of the present application;

[0047] Fig.13 is a schematic diagram of adjusting the posture of a camera device provided in an embodiment of the present application;

[0048] Fig.14 is a schematic diagram of shooting by a camera device provided in an embodiment of the present application;

[0049] Fig.15 is a schematic diagram of shooting by a camera device provided in an embodiment of the present application;

[0050] Fig.16 is a schematic diagram of shooting by a camera device provided in an embodiment of the present application;

[0051] Fig.17 is a schematic diagram of shooting by a camera device provided in an embodiment of the present application;

[0052] Fig.18 It is a schematic diagram of feature points;

[0053] Fig.19 It is a schematic diagram of a control device adjusting a calibration pattern;

[0054] Fig. 20 It is a schematic diagram of a control device adjusting a calibration pattern;

[0055] Fig.21 It is a schematic diagram of a control device adjusting a calibration pattern;

[0056] Fig. 22 is a schematic diagram of other calibration patterns provided in the embodiments of the present application;

[0057] Fig.23 It is a schematic diagram of a parameter calibration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In the technical fields of machine vision / computer vision, cameras are usually used to image objects and scenes in the objective world, so as to reconstruct the objective objects and scenes in three dimensions based on the camera's imaging, and realize the three-dimensional recognition of the objective world by machines, such as the shape, size, distance from the observation point, texture and motion characteristics of the object. Machine vision / computer vision technology can be applied to virtual reality, advanced driver assistance systems, drone cruise systems, visual detection, biometric recognition, distance measurement, three-dimensional reconstruction and other fields. In recent years, with the increasing popularity of products such as smartphones, tablets, virtual reality devices, cars equipped with driver assistance systems, drones, etc., machine vision / computer vision technology has been continuously developed.

[0059] In the practical application of machine vision / computer vision and other technologies, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image captured by the camera, a geometric model of camera imaging must be established. These geometric model parameters can be summarized as camera parameters (or camera parameters). In most cases, camera parameters must be solved through experiments and calculations. The process of solving camera parameters through experiments and calculations is called camera calibration (or camera calibration). Camera calibration is an important link in the fields of machine vision / computer vision. The accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the image reasoning results. Therefore, doing a good job of camera calibration is the prerequisite for doing a good job of subsequent work. Quickly and accurately completing camera calibration is the basis for improving the efficiency of subsequent work.

[0060] The basic principles of camera calibration and the main technical concepts involved are explained in detail below with some drawings and texts. It should be noted that in different expressions and habits, some technical concepts may be expressed differently, for example, expressing a concept as multiple sub-concepts, or generalizing several concepts into one or more other concepts by splitting or combining. These expressions are only changes in habits and are not enough to constitute a substantial difference in the embodiments of the present application.

[0061] Figure 1 This is a basic structural diagram of a camera. Figure 1 As shown, the camera includes an optical lens 10 and a photoelectric sensor 20. The optical lens 10 is used to project the optical image of the space object onto the photoelectric sensor 20, and the photoelectric sensor 20 is used to convert the optical signal projected by the optical lens 10 into an electrical signal, and finally forms an image with the help of other devices such as an image signal sensor. Generally speaking, the photoelectric sensor 20 may include a charge-coupled device (CCD) or a complementary metal oxide semiconductor active pixel sensor (CMOS active pixel sensor).

[0062] like Figure 2As shown, objects in real space are three-dimensional, while objects in their corresponding images are two-dimensional. Therefore, it can be considered that there is a three-dimensional to two-dimensional mapping model between objects in a three-dimensional scene and their corresponding two-dimensional images. This mapping model enables the objects in a three-dimensional scene to form a conversion from three-dimensional to two-dimensional or two-dimensional to three-dimensional. Then, it is easy to understand that when a camera takes an image of an object in a three-dimensional scene, the camera can be considered as this mapping model, and the camera parameters are the parameters of this mapping model. Therefore, as long as the camera parameters are obtained, the world coordinates of the object in space can be inferred from the pixel coordinates of the object in the image, thereby realizing functions such as visual detection, biometric recognition, distance measurement, and three-dimensional reconstruction.

[0063] Generally speaking, the following concepts are involved in camera calibration:

[0064] Internal parameters of the camera (referred to as camera internal parameters): refers to parameters related to the characteristics of the camera itself, which can be determined by the position of the optical lens and photoelectric sensor installed in the camera module. The camera internal parameters may include at least one of the following parameters: focal length, pixel size, optical distortion, white balance, resolution, contrast, vignetting and dark corners.

[0065] The external parameters of the camera (referred to as camera external parameters) can include at least one of the following:

[0066] World coordinate system: also known as measurement coordinate system, is a three-dimensional rectangular coordinate system that can be used as a reference to describe the spatial position of the camera and the object to be measured in the objective world. The position of the world coordinate system can be freely determined according to the actual situation.

[0067] The external parameters of a camera and the internal parameters of a camera are collectively referred to as camera parameters.

[0068] Camera coordinate system: It is a three-dimensional rectangular coordinate system with the origin at the optical center of the optical lens (where the optical center refers to the center point of the lens of the optical lens), the x and y axes are parallel to the two sides of the image plane of the camera, and the z axis is the optical axis of the optical lens, which is perpendicular to the imaging plane.

[0069] Pixel coordinate system: It is a two-dimensional rectangular coordinate system that reflects the arrangement of pixels in the camera CCD / CMOS chip. The origin of the pixel coordinate system is located in the upper left corner of the image, and the u-axis and v-axis are parallel to the two sides of the image plane. The unit of the coordinate axis in the pixel coordinate system is pixel (integer).

[0070] Image coordinate system: It is a two-dimensional rectangular coordinate system. The units of its coordinate axes are usually millimeters (mm). The origin is the intersection of the optical axis of the camera and the image plane (called the principal point), that is, the center point of the image. The X-axis and Y-axis are parallel to the u-axis and v-axis of the pixel coordinate system respectively. There is a translation relationship between the image coordinate system and the pixel coordinate system, that is, it can be obtained by translation.

[0071] Figure 3 This is a schematic diagram of the transformation relationship between various coordinate systems. Figure 3 The conversion relationship between the various coordinate systems is introduced, and then the camera parameters that need to be calibrated in the embodiment of the present application are specifically described:

[0072] 1. The conversion of the world coordinate system to the camera coordinate system can be achieved through the following matrix transformation:

[0073]

[0074] Among them, R is a 3*3 rotation matrix, t is a 3*1 translation vector, (x c ,y c , z c , 1) T is the coordinate of the camera coordinate system, (x w ,y w , z w , 1) T is the coordinate of the world coordinate system. Therefore, the conversion from the world coordinate system to the camera coordinate system actually includes a rotation process and a translation process.

[0075] 2. The conversion of image coordinate system to pixel coordinate system can be achieved through the following matrix transformation:

[0076]

[0077] Among them, (X, Y, 1) T is the coordinate of the image coordinate system, (u, v, 1) T is the coordinate of the image coordinate system, dX and dY are the physical sizes of the pixel in the X and Y axis directions respectively, and u0 and v0 are the coordinates of the optical center.

[0078] 3. The camera coordinate system is converted to the image coordinate system as follows Figure 3 The following is a perspective projection process:

[0079] For example, the line connecting any point P in space and the optical center C of the camera is CP, and the intersection point p of CP and the image plane is the projection of the space point P on the image plane, which can be realized by the following matrix transformation:

[0080]

[0081] Among them, S is the scale factor (S is not 0), f is the effective focal length of the camera (the distance from the optical center to the image plane), (x c ,y c , z c , 1) T is the coordinate of the spatial point P in the camera coordinate system, (X, Y, 1) T is the coordinate of the image point p in the image coordinate system.

[0082] 4. Convert the world coordinate system to the pixel coordinate system:

[0083] According to the conversion matrices between the coordinate systems shown in 1 to 3 above, the world coordinate system can be converted to the pixel coordinate system through the following matrix transformation:

[0084]

[0085] in:

[0086]

[0087] In the above matrix, M is the parameter matrix of the camera, i.e., the camera parameters; M1 is the intrinsic parameter matrix of the camera, i.e., the internal parameters of the camera; M2 is the extrinsic parameter matrix of the camera, i.e., the external parameters of the camera. Camera calibration includes solving the above intrinsic parameter matrix M1 and / or extrinsic parameter matrix M2. For example: solving the pixel size 1 / dX and 1 / dY of the camera, the focal length f of the camera, the optical center position (v0, u0) of the camera, the rotation matrix R, and the translation variable and other parameters.

[0088] It should be noted that the actual image of the camera is usually distorted, which can be called distortion. Figure 4 As shown, the distortion can be specifically divided into tangential distortion and radial distortion. Among them, radial distortion comes from the lens of the camera itself, which comes from the influence of "fisheye" and "barrel shape", and is more obvious at the edge of the lens imaging; tangential distortion is caused by defects in lens manufacturing or deviations in installation, which make the lens itself not parallel to the image plane. When considering tangential distortion and radial distortion, the matrix transformation shown in 1 to 3 above can also be expanded to:

[0089] x′ c =x c / z c

[0090] y′ c =y c / z c

[0091]

[0092]

[0093]

[0094] Among them, k1, k2, k3, k4, k5 and k6 are radial distortion coefficients, and p1 and p2 are tangential distortion coefficients. Therefore, the camera calibration process can also determine the distortion parameters of the camera.

[0095] In order to solve the camera parameters, in a laboratory environment, a camera calibration method is used. Figure 5 As shown in the figure, first, the camera to be calibrated is fixed at a certain position in the laboratory space environment; then, a calibration plate is manually held in the camera's field of view, and the calibration plate is moved to change the angle and position of the calibration plate, so that the camera can capture images of the calibration plate in different postures relative to the calibration plate; finally, the internal and external parameters of the camera are solved by the real position of the feature points of the calibration plate in the three-dimensional world (i.e., the coordinates of the feature points in the world coordinate system) and the position in the image imaged by the camera (i.e., the coordinates of the feature points in the pixel coordinate system of the image). The calibration of industrial-grade cameras generally requires the use of a large robotic arm, on which the calibration plate is hoisted, and the angle and position of the calibration plate are changed by the swinging action of the robotic arm. However, the swinging of the robotic arm and the stabilization of the posture after the swinging of the arm both consume a lot of time, resulting in low efficiency of camera calibration and difficulty in large-scale application.

[0096] Figure 6 Schematic diagram of some calibration plates shown in the embodiments of the present application. Figure 6 As shown, a checkerboard pattern, a dot pattern, a QR code pattern or other specific patterns can be printed on the calibration plate. For the calibration plate, any pattern unit, such as any square of the checkerboard pattern or any dot of the dot pattern, can be called a marker. Some feature points can be determined from the marker to calculate the camera parameters. Figure 6 In addition to the calibration plates shown in the figure, other forms of calibration plates are also used to calibrate other parameters of the camera. For example, a calibration plate containing a color card is used to calibrate the white balance parameters of the camera; a calibration plate made of uniform diffuse reflection material is used to calibrate the vignette parameters of the camera with a uniformly illuminated light source; and a calibration plate containing a specially designed pattern is used to calibrate the optical resolution modulation transfer function (MTF) of the camera.

[0097] Combination Figure 5 and Figure 6 The parameter calibration method shown may have the following problems in practical application:

[0098] Since the calibration plate is printed on various types of plates, the flatness and printing accuracy of the calibration plate will affect the accuracy of the world coordinate system. When the flatness and printing accuracy are low, the accuracy of the world coordinate system is low, which affects the calibration accuracy. In addition, the method of manually moving the calibration plate makes it impossible to accurately determine the position of the calibration plate, which also affects the calibration accuracy. To calibrate different parameters, calibration plates with different patterns need to be printed on different plates. The calibration plates are independent of each other, and the lighting conditions and shape and size of the calibration plates when calibrating different camera parameters are different, making it impossible to reuse the calibration plate. In addition, during the camera calibration process, in order to obtain the camera's internal and external parameters and other collective parameters, as well as the camera's lens distortion and other parameters, the calibration plate needs to be moved to various local positions in the camera's field of view. The method of manually moving the calibration plate or the method of moving the calibration plate by a robotic arm is time-consuming and labor-intensive, reducing the calibration efficiency. In addition, the method of manually moving the calibration plate or the method of moving the calibration plate by a robotic arm to achieve complete repetition of the calibration process makes it difficult to reproduce the problems that occur during the calibration process.

[0099] In order to solve the above technical problems, the embodiments of the present application provide a parameter calibration method and device. The method and device can be applied to the calibration of various consumer-grade cameras and various industrial-grade cameras to obtain the internal parameters, external parameters, lens distortion parameters, white balance parameters, color difference and MTF parameters of the camera. Among them, the above-mentioned consumer-grade cameras may include, for example, various cameras or camera arrays for installation in mobile phones, tablet computers, cameras, virtual / mixed / augmented reality devices, drones with cameras, and camera-based assisted driving devices / systems; industrial-grade cameras may include, for example, surveillance cameras, ranging / speed measurement cameras used in industrial scenes, and cameras or camera arrays contained in various industrial cameras.

[0100] Figure 7 An exemplary hardware system is provided. Figure 7As shown, the hardware system may include, for example: a control device 100, a display screen 200, a camera device 300 and a motion device 400, wherein the control device 100 establishes a communication connection with the display screen 200, the camera device 300 and the motion device 400. Among them, the display screen 200 can be used to display various calibration patterns, calibration results and other information in the embodiment of the present application. The motion device 400 includes at least one installation station 410 for installing a camera, each installation station 410 can be used to install at least one camera device 300 to be calibrated, each camera device 300 includes at least one camera to be calibrated, and when the camera device 300 includes two or more cameras, the hardware system can be used to perform binocular or multi-target calibration on two or more cameras, and calibrate a camera array including multiple cameras. The motion device 400 can drive the installation station to produce displacement and rotation, thereby changing the posture of the camera. In addition, the display screen 200 and the motion device 400 in the hardware system should have the following positional relationship: the display screen 200 is arranged facing the field of view 310 of the camera, so that when the motion device 400 drives the posture of the camera to change, the display screen 200 can appear at various positions in the field of view 310 of the camera.

[0101] It is necessary to add that Figure 7 The hardware structure shown is only used to illustrate the feasibility of realizing the position change of the camera device relative to the display screen, and does not constitute a specific limitation on the method and hardware system of the embodiment of the present application. Figure 8 As shown, it may include multiple control devices 100, and / or multiple display screens 200, and / or multiple motion devices 400. The embodiment of the present application does not limit the specific implementation of the above hardware structure.

[0102] Fig. 9 1 is a schematic diagram of the structure of a control device 100 shown in an embodiment of the present application. The control device 100 may include, for example, a computer, a mobile phone, a tablet computer, an embedded system, and the like. Fig. 9As shown, the control device 100 may include: a processor 110, a memory 120, an interface unit 130 and an input unit 140. The processor 110, the memory 120, the interface unit 130 and the input unit 140 are coupled and connected. The memory 120 stores readable program instructions, and the processor 110 is used to read and execute the program instructions in the memory 120 to execute the relevant steps of the parameter calibration method of this embodiment, for example: sending instructions to the motion device through the interface unit 130 so that the motion device drives the camera device to generate displacement and rotation, etc., or sending instructions to the camera device through the interface unit 130 so that the camera device captures images or records videos, etc. The memory 120 may also store document materials corresponding to the calibration pattern, and the processor 110 may read and decode the above document materials and send them to the display screen for display. The storage device 120 may also be used for result files, intermediate result files, logs, debugging information and other information generated during the calibration process.

[0103] In the embodiment of the present application, the processor 110 of the control device 100 may include one or more processing units, such as a system on a chip (SoC), a central processing unit (CPU), a microcontroller (MCU), a storage controller, etc. Among them, different processing units may be independent devices or integrated into one or more processors 110.

[0104] In the embodiment of the present application, the memory 120 of the control device 100 may include one or more storage units, for example, it may include volatile memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), etc.; it may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive HDD and solid state drive SSD, etc. Among them, different storage units may be independent devices, or they may be integrated or packaged in one or more processors 110 or interface units 130, becoming a part of the processor 110 or the interface unit 130.

[0105] In the embodiment of the present application, the interface unit 130 of the control device 100 may include, for example, a network adapter, a network interface card, a LAN adapter, a network interface controller (NIC), a modem, an antenna, etc. The interface unit 130 may be an independent device, or may be partially or completely integrated or packaged in the processor 110 to become a part of the processor 110.

[0106] In the embodiment of the present application, the input unit 140 may include, for example, a keyboard, a mouse, a touch display screen, a joystick controller, a touchpad, etc. coupled to the processor 110.

[0107] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the control device 100. In other embodiments of the present application, the control device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0108] Fig.10 4 is a schematic diagram of the structure of the sports device 400 shown in the embodiment of the present application. Fig.10 As shown, the motion device includes a three-axis mobile platform 420, a two-axis rotating platform 430 and an installation station 410, wherein the two-axis rotating platform 430 is installed on the three-axis mobile platform 420, and the installation station 410 is fixed to the two-axis rotating platform 430, and is installed with the camera device 300 to be calibrated. The three-axis mobile platform 420 can generate displacement along the X-axis, Y-axis and Z-axis directions in the figure to change the position of the camera device 300 in the world coordinate system (i.e., real space), and the two-axis rotating platform 430 can generate rotation with the Y-axis and Z-axis as the rotation axes to change the pitch angle and horizontal angle of the camera device 300 in the world coordinate system.

[0109] In one embodiment, the displacement and rotation of the motion device 400 can be achieved by a motor drive. Fig.10 The motion device 400 shown may include at least one motor for driving the three-axis mobile platform 420 to generate displacement and at least one motor for driving the two-axis rotating platform 430 to generate rotation. Adaptively, the motion device 400 may also include a programmable controller, which may be coupled to the interface unit 130 of the motor and the control device 100, and is used to drive the motor to operate according to the instructions issued by the control device 100. It is understandable that the displacement and rotation of the motion device 400 may also be achieved by other means, such as hydraulic drive, electromagnetic drive, etc., which are not specifically limited here.

[0110] It should be noted that the sports device 400 provided in the embodiment of the present application can have a variety of different forms and can be implemented in a variety of ways. The structure of the sports device 400 discussed above is only some preferred implementations, which are used to illustrate the feasibility of the method and structure described in the embodiment of the present application, and do not limit the protection scope of the embodiment of the present application. The sports device 400 of the embodiment of the present application implemented by other methods or sequences is also within the protection scope of the embodiment of the present application.

[0111] The present application provides a parameter calibration method, which can be applied to Figure 7-Figure 10 Any of the control devices shown. Based on Figure 7-Figure 10 The hardware system shown in the figure, the camera device in the method may include a camera device (for example Figure 7 ), and may also include multiple camera devices (e.g. Figure 8 ). Therefore, the “camera device” in the implementation of this application can also be described as “at least one camera device”.

[0112] like Fig.11 As shown, the method steps S101 to S106:

[0113] Step S101, generating a first calibration pattern.

[0114] In a specific implementation, the first calibration pattern may be generated in a programming manner, and the first calibration pattern may be dynamically changed according to the calibration result of the calibration pattern previously used in this embodiment and the calibration requirement.

[0115] In one implementation, when the first calibration pattern is generated in a programming manner, the control device can obtain configuration parameters corresponding to the first calibration pattern, generate the first calibration pattern according to the configuration parameters of the first calibration pattern, and send an image signal corresponding to the first calibration pattern to a display screen so that the display screen displays the first calibration pattern.

[0116] To achieve the above purpose, the control device may pre-store configuration parameters of multiple calibration patterns, and select configuration parameters of the first calibration pattern from the above multiple configuration parameters in accordance with a preset calibration pattern generation order, or in response to an input action performed by a user on an input unit, to generate a calibration pattern. Alternatively, the control device may display an interactive interface on a display screen for guiding the user to input configuration parameters, so that the user can input and debug configuration parameters according to the calibration requirements of the camera device; the control device may also generate a corresponding first calibration pattern on a display screen in real time based on the configuration parameters already input by the user, and refresh the first calibration pattern in real time according to the user's adjustment of the configuration parameters until the first calibration pattern meets the user's requirements. This application does not make any specific limitation on the method for generating the calibration pattern.

[0117] It can be understood that when the first calibration patterns are different, the configuration parameters of the calibration patterns are also different.

[0118] For example, when the first calibration pattern is a checkerboard pattern or a dot pattern, the configuration parameters may include: the distribution area of ​​the marker (for example: distributed in the full screen, center, upper left corner, lower left corner, upper right corner and lower right corner of the display screen, etc.), the density of the marker (for example: how many checkerboards / dots are included in a display area of ​​a unit size of the display screen), the total number of markers, the size of the checkerboard / dots (for example: the length and width of each square of the checkerboard, the diameter of the dot, etc., the above length, width and diameter can be determined by the number of pixels) and the horizontal spacing and vertical spacing between markers, etc. (the above horizontal spacing and vertical spacing can be determined by the number of pixels).

[0119] In other examples, when the first calibration pattern is a grayscale pattern, the configuration parameters may include, for example, the number, position, size, grayscale value and color of each color card and other parameters.

[0120] In a specific implementation, the first calibration pattern can be generated according to a corresponding image file, and the memory of the control device can also pre-store at least one image file, each image file containing a calibration pattern. The above image file can, for example, contain a checkerboard pattern, a dot matrix pattern, a grayscale pattern, a stripe pattern, a solid color pattern, etc.

[0121] In one implementation, when the memory stores an image file, the control device can select a first image file corresponding to a first calibration pattern from the at least one image file according to a preset calibration pattern generation order, decode the first image file to generate an image signal, and output the image signal to a display screen for display, so that the display screen displays the first calibration pattern.

[0122] In another implementation, when the memory stores image files, the display screen can display a graphical operation interface for browsing and selecting image files, so that the user can use the input unit to perform various input actions based on the graphical operation interface. The control device responds to the user's input action on the input unit, selects a first image file from at least one image file, generates a preview of the corresponding first calibration pattern and sends it to the display screen for display. When the user further performs an input action to confirm the selection, the control device causes the display screen to display the first calibration pattern.

[0123] It should be understood that the calibration pattern of the embodiment of the present application can also be implemented in a variety of ways. The above-mentioned implementation methods are merely some preferred implementation methods and do not limit the protection scope of the embodiment of the present application. Other methods of generating calibration patterns that can be applied in the embodiment of the present application are also within the protection scope of the embodiment of the present application.

[0124] It should be noted that the display screen in the embodiment of the present application can preferably use a self-luminous display screen with an active light source to improve the quality of the calibration pattern, thereby improving the calibration quality of the camera device. Among them, the self-luminous display can include, for example: an organic light-emitting diode (OLED) display screen, an active-matrix organic light-emitting diode (AMOLED) display screen, a quantum dot display screen, etc. Compared with the traditional printed calibration plate, the use of a self-luminous display screen to display the calibration pattern has the following advantages:

[0125] 1. The calibration pattern displayed on the display screen is controlled by the control device and has the ability to be reproduced and dynamically adjusted.

[0126] 2. The calibration patterns at various positions of the display screen are not affected by changes in ambient light sources and differences in lighting conditions, and have high consistency, which is beneficial to improving the quality of images taken by the camera device, avoiding problems such as feature point detection failure and edge detection failure in subsequent image processing, and is beneficial to improving the calibration quality.

[0127] 3. During the printing process of traditional calibration plates, the color mode will be converted from other modes to the printing mode (for example: CMYK), causing some colors to be lost or changed, so it cannot be used to calibrate the color difference, white balance and other parameters of the camera device. The display screen can use the same color mode and color gamut as the image taken by the camera device, which can realize the calibration of the color difference and white balance and other parameters of the camera device.

[0128] Step S102: sending a first instruction, where the first instruction is used to indicate at least one set of posture parameters of the camera device within a first time period.

[0129] In an embodiment of the present application, the posture of the camera device relative to the control device can be determined by a set of posture parameters, and each set of posture parameters can determine a posture of the camera device. In a specific implementation, a set of posture parameters may include at least one position parameter, and / or at least one angle parameter. Among them, the position parameters are used to determine the position of the camera device in the world coordinate system. For example, the position parameters may include part or all of the coordinate values ​​of the X-axis, Y-axis and Z-axis coordinates of the camera device in the world coordinate system, such as (X1, Y1, Z1), (X2, Y2), (X3, Z3) or (X4), etc. It can be selected that when a set of posture parameters contains some position parameters, it indicates that the coordinate values ​​corresponding to the missing position parameters of the camera device do not change. The angle parameters are used to determine the angles of the camera device relative to the various coordinate axes of the world coordinate system. For example, the angle parameters may include part or all of the angle values ​​of the camera device relative to the X-axis, Y-axis and Z-axis in the world coordinate system, such as (α1, β1, γ1), (α2, γ2), (β3, γ3) or (β4), etc. Optionally, when a set of posture parameters includes some angle parameters, it indicates that the coordinate values ​​of the camera device corresponding to the missing angle parameters do not change.

[0130] In the specific implementation, based on Figure 7 In the hardware system shown, the control device can send a first instruction to the motion device, and the motion device can carry at least one camera device, and drive the camera device to translate and rotate according to at least one set of posture parameters indicated by the first instruction to control the posture of at least one camera device.

[0131] Fig.12 Schematic diagram of adjusting the posture of the camera device provided in the embodiment of the present application. Fig.12 As shown, in one implementation, the control device can send multiple first instructions to the motion device within a first time period, each first instruction indicates a set of posture parameters, and the posture parameters indicated by different first instructions are different or the same. Thus, the motion device can receive multiple first instructions within the first time period, and each time a first instruction is received, the motion device will perform a translation and rotation according to the posture parameters indicated by the first instruction, and adjust the camera device to the corresponding posture. Optionally, the sending of multiple first instructions within the above-mentioned first time period can be performed in sequence at different times.

[0132] For example, Fig.12As shown, at N (N is a natural number) moments in the first time period, the control device sends the first instruction 1 to the first instruction N to the motion device, and the first instruction 1 to the first instruction N indicate the first group of posture parameters to the Nth group of posture parameters in sequence. Then, at time t1, the control device sends the first instruction 1 to the motion device to adjust the posture of the camera device from the initial posture 0 to posture 1; at time t2, the control device sends the first instruction 2 to the motion device to adjust the posture of the camera device from posture 1 to posture 2; at time t3, the control device sends the first instruction 3 to the motion device to adjust the posture of the camera device from posture 2 to posture 3; and so on..., at time tN, the control device sends the first instruction N to the motion device to adjust the posture of the camera device from posture N-1 to posture N.

[0133] Fig.13 Schematic diagram of adjusting the posture of the camera device provided in the embodiment of the present application. Figure 3 As shown, in one implementation, the control device can send only one first instruction in a first time period, and the first instruction includes multiple different posture parameters, which can be connected into a moving trajectory, as well as the angle change trend of the camera device. Therefore, after receiving the first instruction, the motion device can drive the camera to move along the moving trajectory indicated by the first instruction, and adjust the angle of the camera device during the movement, so that the camera device has different postures in the first time period.

[0134] For example, Fig.13 As shown, at a certain moment in the first time period, the control device sends a first instruction 1 to the motion device, and the first instruction 1 indicates a moving trajectory L1 through a plurality of different first group posture parameters-Mth group posture parameters (M is a natural number). In response to the first instruction 1, the motion device moves the camera device along the trajectory L1, so that the posture of the camera device changes from posture 0 to posture M. During the change process, if the motion device receives a new first instruction 2, the motion device can interrupt the movement process along the trajectory L1, and continue to adjust the posture of the camera device along the trajectory L2 indicated by the new first instruction 2; or, if the motion device receives a new first instruction 2, the motion device can continue to move along the trajectory L1, and after completing the movement along the trajectory L1, continue to adjust the posture of the camera device along the trajectory L2 indicated by the new first instruction 2.

[0135] In one implementation, based on Fig.10For the motion device shown, the first instruction indicating the posture parameters of the camera device can be implemented in the following manner: the first instruction can include at least one control parameter of the motion device, for example: the displacement distance, linear velocity and / or displacement time of the three-axis mobile platform along the X-axis, Y-axis and Z-axis, the angle, angular velocity and / or rotation time of the two-axis rotating platform around the Y-axis and Z-axis, etc. The above control parameters can be sent to the controller of the motion device, so that the controller determines the rotation direction, rotation speed and rotation time of the motor according to the above control parameters, and then drives the motor to operate, so that the three-axis mobile platform and the two-axis rotating platform generate displacement or rotation corresponding to the above control parameters, and then adjust the camera device to the corresponding posture.

[0136] Step S103: Send a second instruction to the camera device, where the second instruction is used to instruct the camera device at least one of a shooting timing and a shooting number within the first time period.

[0137] In a specific implementation, the camera device performs at least one shooting including taking a photo and / or taking a video.

[0138] In one implementation, when the camera device takes photos, the control device can alternately send multiple first instructions and multiple second instructions to the camera device within a first time period. Each time the camera device receives a second instruction, it takes a corresponding photo. Since the posture parameters of the camera device are changing within the first time period, the camera device can take multiple photos with different posture parameters. In the above multiple photos, the calibration pattern of the display screen may appear in whole or in part in the photos, and the location where the calibration pattern appears may also be different in different photos.

[0139] For example, Fig.14As shown, at time t0 in the first time period, the camera device is located at the upper left of the display screen, and the posture parameter is 0. At N (N is a natural number) moments in the first time period, the control device sequentially sends the first instruction 1 to the first instruction N to the motion device, and the first instruction 1 to the first instruction N sequentially indicate the first group of posture parameters to the Nth group of posture parameters, and, alternating with the N first instructions, the control device also sequentially sends multiple (for example, N) second instructions to the motion device, so that the camera device takes photos under different posture parameters. For example: at time t1, the control device sends a first instruction 1 to the motion device to adjust the posture of the camera device from the initial posture 0 to the posture 1, and then the control device sends a second instruction 1 to the camera device, so that the camera device takes at least one photo at posture 1; at time t2, the control device sends a first instruction 2 to the motion device to adjust the posture of the camera device from posture 1 to posture 2, and then the control device sends a second instruction 2 to the camera device, so that the camera device takes at least one photo at posture 2; at time tN, the control device sends a first instruction N to the motion device to adjust the posture of the camera device from posture N-1 to posture N, and then the control device sends a second instruction N to the camera device, so that the camera device takes at least one photo at the Nth set of posture parameters.

[0140] In one implementation, when the camera device takes a photo, the second instruction can be specifically used to indicate the time interval ΔT for the camera device to take photos. Therefore, the control device may only need to send the second instruction to the camera device once. This allows the camera device to take at least one photo every ΔT during the change of posture parameters. The first instruction and the second instruction may be sent at the same time or at different times, for example, the first instruction may be sent first and then the second instruction, or the second instruction may be sent first and then the first instruction.

[0141] For example, Fig.15 As shown, at time t0, the control device sends a first instruction to the motion device, and the first instruction indicates a trajectory L1 through a plurality of different first group posture parameters-Mth group posture parameters (M is a natural number). In response to the first instruction, the motion device moves the camera device along the trajectory L1, so that the posture of the camera device changes from posture 0 to posture M. During the movement of the camera device along the trajectory L1, the control device can send a second instruction to the camera device, and the second instruction indicates the time interval ΔT, so that the camera device takes a photo every ΔT until the camera device completes its movement along the trajectory L1.

[0142] In one implementation, when the camera device shoots a video, the control device may send the second instruction to the camera device only once within the first time period. After receiving the second instruction, the camera device starts shooting the video. As the position of the camera device changes, the calibration pattern of the display screen may appear in the video in whole or in part, and the position where the calibration pattern appears may also be different in different video frames. The first instruction and the second instruction may be sent at the same time or at different times, for example, the first instruction may be sent first and then the second instruction, or the second instruction may be sent first and then the first instruction.

[0143] Optionally, after the control device sends the second instruction to the camera apparatus, the control device may also send a termination instruction to the camera apparatus so that the camera apparatus stops shooting video.

[0144] For example, Fig.16 As shown, at time t0, the control device sends a first instruction to the motion device, and the first instruction indicates a trajectory L1 through a plurality of different first group posture parameters-Mth group posture parameters (M is a natural number). In response to the first instruction, the motion device moves the camera device along the trajectory L1, so that the posture of the camera device changes from posture 1 to posture M. In the process of the camera device moving along the trajectory L1, the control device can send a second instruction to the camera device, so that the camera device starts to change its posture while shooting video. At a certain time tM thereafter, for example, after the camera device finishes moving along the trajectory L1, the control device sends a termination instruction to the camera device, so that the camera device stops shooting video.

[0145] Optionally, when the second instruction is used to instruct the camera to shoot a video, the control device may send the second instruction and the termination instruction alternately multiple times, so that the camera shoots multiple video clips during the posture change process.

[0146] For example, Fig.17As shown, in the process of the camera device moving along the trajectory L1, the control device sends the first second instruction at time t11 and sends the first termination instruction at time t12, so that the camera device shoots the video segment 1 at time t11-t12; the control device sends the second second instruction at time t21 and sends the second termination instruction at time t22, so that the camera device shoots the video segment 2 at time t21-t22; the control device sends the Pth second instruction at time tP1 (P is a natural number) and sends the Pth termination instruction at time tP2, so that the camera device shoots the video segment P at time tP1-tP2. Among them, the above-mentioned time t11, for example, can be before the camera device moves to position 1, and the time t12, for example, can be after the camera device moves to position 1, so that video segment 1 includes video frames taken by the camera device at position 1; the time t21, for example, can be before the camera device moves to position 2, and the time t22, for example, can be after the camera device moves to position 2, so that video segment 2 includes video frames taken by the camera device at position 2; the time t31, for example, can be before the camera device moves to position 3, and the time t32, for example, can be after the camera device moves to position 3, so that video segment 3 includes video frames taken by the camera device at position 3.

[0147] Step S104: obtaining at least one shooting result from the camera device.

[0148] In a specific implementation, the above-mentioned shooting results include photos and / or videos shot by the camera device. The camera device can send the photos and videos shot back to the control device, and the control device saves the photos and videos sent back by the camera device in a memory for use in calculating the internal parameters of the camera device in subsequent steps. For example, the camera device can send analog signals of photos and videos to a capture card, which performs analog-to-digital conversion of the signals, and then sends the converted digital signals of photos and videos to the control device for storage.

[0149] Step S105: determining at least one internal parameter of the camera device according to a part or all of the at least one shooting result.

[0150] Among them, any shooting result in the above-mentioned "partial or entire shooting result" shall include part or all of the first calibration pattern. Further optionally, the "partial or entire shooting result" makes the first calibration pattern appear at various positions in the photo or video (i.e., various positions in the field of view of the camera device), especially at positions with severe distortion.

[0151] In a specific implementation, when the shooting result is a photo, the control device can detect and obtain feature points from the above "part or all" of the photo. Each feature point has a pixel coordinate system coordinate in the photo and a world coordinate system coordinate in the display screen. According to the pixel coordinate system coordinate and the world coordinate system coordinate of the above feature point, the internal parameters such as M1 and M2 in the above "converting the world coordinate system to the pixel coordinate system" formula (i.e., formula (1)) can be solved.

[0152] In a specific implementation, when the shooting result is a video, the control device can extract at least one video frame from the video, and each extracted video frame contains part or all of the first calibration pattern. Then, the control device can detect and obtain feature points from the above at least one video frame. Each feature point has a pixel coordinate system coordinate in the video frame and a world coordinate system coordinate in the display screen. According to the pixel coordinate system coordinate and the world coordinate system coordinate of the above feature point, the internal parameters such as M1 and M2 in the above formula of "converting the world coordinate system to the pixel coordinate system" (i.e., formula (1)) can be solved.

[0153] like Fig.18 As shown, the feature points can be points that are easily identified from the first calibration pattern. For example, the feature points of the checkerboard calibration pattern can be the corner points of each square, and the feature points of the dot calibration pattern can be the center points of each dot. The feature points can be obtained by binarizing the photo or video frame and performing edge detection, which will not be described here.

[0154] Step S106, determine whether the error of at least one target internal parameter among at least one internal parameter satisfies its corresponding error threshold. If the error of any target internal parameter does not satisfy its corresponding error threshold, return to step S102; if all target internal parameters satisfy their corresponding error thresholds, output the calibration result.

[0155] In specific implementation scenarios, technicians may have different requirements for the calibration of the camera device. For example, in some implementation scenarios, the technician hopes to calibrate all the internal parameters of the camera device, and in other implementation scenarios, the technician hopes to calibrate only part of the internal parameters of the camera device. In order to distinguish between the internal parameters that need to be calibrated and the internal parameters that do not need to be calibrated, the internal parameters that need to be calibrated are referred to as target internal parameters. It is understandable that there can be one or more target internal parameters, and the target internal parameters can be all or part of the parameters of the above-mentioned "at least one internal parameter".

[0156] In the embodiment of the present application, as shown in formula (1), the internal parameters are partially or completely included in the internal parameter matrix, so the process of solving the internal parameters can actually be the process of solving the internal parameter matrix. When all internal parameters or target internal parameters meet their corresponding error thresholds, it can be considered that the internal parameter matrix is ​​accurate and meets the output condition. Therefore, in step S106, if all target internal parameters meet their respective corresponding error thresholds, the calibration result is output. If the error of any target internal parameter does not meet its corresponding error threshold, it is necessary to return to step S102 to continue calibration, so that the internal parameter matrix can be iteratively updated to obtain more accurate internal parameters.

[0157] In the embodiment of the present application, the error of the internal parameters may include, for example, repeatability, reprojection error, etc. In the embodiment of the present application, it can be assumed that the internal parameters obtained by the first calibration are inaccurate, so that after the first calibration, the process returns to step S102 to perform the second calibration and subsequent repeated calibrations, so as to calculate the calibration error.

[0158] Among them, repeatability refers to the deviation between multiple internal parameters obtained by repeatedly performing multiple calibrations under the same calibration environment. The smaller the deviation, the smaller the error of the internal parameter; the larger the deviation, the larger the error of the internal parameter. The above-mentioned same calibration environment means that the calibration pattern used for at least two calibrations, the posture change trajectory of the camera device, the brightness and contrast of the display screen, the number of feature points, the extraction accuracy, the number of photos or video frames used to calculate the calibration results, the posture taken, etc. are partially or completely the same.

[0159] In a specific implementation, at least one parameter among the internal parameters may have a corresponding error threshold, and the error threshold may correspond to the repeatability accuracy or the reprojection error, for example, a focal length threshold, an optical center threshold, a pixel size threshold, an optical distortion threshold, a white balance threshold, etc. Then, as an example, when calculating the repeatability accuracy, if the deviation of the focal lengths calibrated at least twice is greater than the focal length threshold, and / or the deviation of the optical centers calibrated twice is greater than the optical center threshold, and / or the deviation of the pixel sizes calibrated twice is greater than the pixel size threshold, and / or the deviation of the optical distortion calibrated twice is greater than the optical distortion threshold, and / or the deviation of the white balance calibrated twice is greater than the white balance threshold, then it is necessary to return to step S102 to continue calibration.

[0160] In addition, by using the internal and external parameters of the camera device obtained by calibration, the coordinate point in the world coordinate system is projected and calculated, and the projection value of the coordinate point in the pixel coordinate system can be obtained. At the same time, the coordinate point in the shooting result of the camera device also corresponds to an observation value in the pixel coordinate system. The error between the projection value and the observation value is the reprojection error. The smaller the reprojection error, the smaller the error of the internal parameter, and the larger the reprojection error, the larger the error of the internal parameter.

[0161] In a specific implementation, for example, repeated calibration can be performed in the following manner:

[0162] Method 1: You can change the calibration pattern.

[0163] For example, Fig.19 As shown, when a checkerboard pattern or a dot pattern (e.g., a first calibration pattern) is used to calibrate the internal parameters of the camera device, the calibration pattern is fixed during a calibration process, that is, the number and size of the markers remain unchanged. In this case, if the calibration accuracy does not meet the accuracy requirements, it may be that the number of feature points is insufficient. Therefore, in the next calibration, a new calibration pattern (e.g., a second calibration pattern) can be obtained by reducing the size of the markers and increasing the number of markers to increase the number of feature points extracted from the shooting results by the control device in step S105 to improve the calibration accuracy.

[0164] For example, Fig. 20 As shown, when a checkerboard pattern or a dot pattern (e.g., a first calibration pattern) is used to calibrate the internal parameters of the camera device, if the calibration accuracy does not meet the accuracy requirements, it may be that the calibration pattern is not clear enough, resulting in poor feature point extraction accuracy. Therefore, when performing repeated calibration, a new calibration pattern (e.g., a second calibration pattern) can be obtained by increasing the size of the marker to improve the feature point extraction accuracy.

[0165] Method 2: adjusting the number of feature points corresponding to each field of view of the camera device extracted from at least one shooting result.

[0166] For example, Fig.21 As shown in the figure, when the internal parameters of the camera device are calibrated using a checkerboard pattern or a dot pattern, if the calibration accuracy does not meet the accuracy requirements, it may be that the feature points are not evenly distributed in various areas of the camera device's field of view, for example: the feature points are concentrated in a certain area, or there are no feature points distributed in a certain area or only a very small number of feature points are distributed, which affects the weight of the internal parameter matrix algorithm. Therefore, when the feature points are concentrated in a certain area, some feature points in the area can be deleted or feature points in other areas can be added when performing repeated calibration.

[0167] In the specific implementation, Fig.21As shown, the field of view of the camera device can be divided into K sub-areas, for example, into sub-areas 1 to sub-areas 9 through a nine-square grid. After the control device extracts feature points from the shooting results, the number of feature points in sub-areas 1 to sub-areas 9 can be counted respectively. If the number of feature points in a certain sub-area, such as sub-area 5, is significantly more than that in other sub-areas (for example: the number of feature points in sub-area 5 is more than that in other sub-areas by a preset percentage), some nodes in sub-area 5 are deleted so that the number of feature points in sub-area 5 is close to the number of feature points in any other sub-area. If the number of feature points in a certain sub-area, such as sub-area 1, is significantly less than that in other sub-areas, the position of the camera device can be moved so that the calibration pattern is located in sub-area 1, and the video or photo is taken again, and the feature points are extracted from the video frame or photo taken again to supplement the feature points of sub-area 1. It should be supplemented that in one adjustment, the control device can dynamically uniformly adjust the feature points of each sub-area so that the number of feature points in each sub-area tends to be uniform, so as to improve the calibration accuracy.

[0168] Method 3: You can adjust the image quality of the calibration pattern.

[0169] For example, when a checkerboard pattern or a dot pattern is used to calibrate the internal parameters of a camera device, if the calibration accuracy does not meet the accuracy requirements, it may be that the brightness and contrast of the display screen and other parameters that affect the image quality are set unreasonably, or the thresholds used for binarization and edge detection of the shooting results are unreasonably. Therefore, when performing repeated calibration, the brightness and contrast of the display screen can also be changed, or the thresholds related to binarization and edge detection can be adjusted to improve the extraction accuracy of feature points.

[0170] It can be understood that based on the repeated calibration method of the above example, the embodiment of the present application can perform multiple calibrations on the camera device, so that the intrinsic parameter matrix and the extrinsic parameter matrix of the camera device are continuously iteratively updated, and the calibration accuracy is continuously improved until the value of the calibration accuracy is less than the corresponding threshold.

[0171] It should be noted that when a checkerboard pattern or a dot pattern is used to calibrate a camera device, the field of view (FOV) and depth of field of different cameras are quite different, which will have different effects on the accuracy of calibration. In order to improve the calibration accuracy of cameras with various field of view and depth of field, the method of the embodiment of the present application can dynamically adjust the calibration parameters according to the field of view and depth of field of the camera device during specific implementation. For example: when the field of view of the camera device is small, the number of markers in the calibration pattern can be appropriately reduced to increase the number of feature points in the field of view; when the field of view of the camera device is large, the number of markers in the calibration pattern can be appropriately increased to reduce the number of feature points in the field of view and improve the recognition accuracy of the feature points. In addition, in the process of changing the posture of the camera device, the calibration pattern should be made to appear at various positions in the field of view of the camera device, especially at the edge of the field of view of the camera device (the edge of the field of view is often a severely distorted area), so that the feature points extracted subsequently can be evenly distributed in the field of view of the camera device, thereby improving the calibration accuracy.

[0172] It can be seen from the above technical scheme that the parameter calibration method provided in the embodiment of the present application can be applied to the calibration of the internal parameters and external parameters of the camera device. When the position of the calibration pattern is fixed, the method accurately adjusts the position and angle of the camera device relative to the calibration pattern by changing the posture parameters of the camera device, so that the camera device can accurately shoot the calibration pattern at each predetermined position and angle relative to the calibration pattern to obtain at least one shooting result. The above shooting result can accurately correspond to the coordinates of the pixel coordinate system and the world coordinate system, so that the corresponding calculated internal parameters and external parameters are more accurate, thereby improving the calibration accuracy.

[0173] It can be understood that when the first calibration pattern is different, the method of the embodiment of the present application can also be used to calibrate different parameters.

[0174] For example, Fig. 22As shown, when the first calibration pattern is a grayscale plate pattern containing different grayscales and different colors, the method of the embodiment of the present application can be used to calibrate the white balance parameters of the camera device. When the first calibration pattern is a monochrome pattern, by adjusting the posture parameters of the camera device, the monochrome pattern appears at various positions in the field of view of the camera device and taking corresponding photos or videos, the vignette parameters of the camera device can be calculated according to the shooting results. When the first calibration pattern is a standard plate pattern containing lines of different angles, by adjusting the posture parameters of the camera device, the lines appear at various positions in the field of view of the camera device and taking corresponding photos or videos, the optical resolution modulation transfer function MTF of the camera device can be calibrated. When the first calibration pattern is a color card pattern containing multiple standard colors, the color difference parameters of the camera device can be calibrated, and the above-mentioned various calibration patterns can be preset in the control device to be generated when needed.

[0175] In the embodiments provided by the present application, the data forwarding method and the data caching method provided by the present application are introduced from the perspective of the device itself and the interaction between the devices. It is understandable that each device, such as the above-mentioned network device and storage device, includes a hardware structure and / or software module corresponding to each function in order to realize the above-mentioned functions. It should be easily appreciated by those skilled in the art that the present application can be implemented in the form of hardware or a combination of hardware and computer software in combination with the units and algorithm steps of each example described in the embodiments disclosed herein. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0176] In the embodiments provided by the present application, the various schemes of the parameter calibration method provided by the present application are introduced from the perspective of the device / device itself and from the perspective of the interaction between the devices / devices. It is understandable that each device / device, such as the above-mentioned control device and the motion device, includes a hardware structure and / or software module corresponding to each function in order to realize the above-mentioned functions. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0177] For example, the control device implements corresponding functions through software modules. The control device may include, for example, a computer, a mobile phone, a tablet computer, an embedded system, and the like.

[0178] In one embodiment, Fig.23 As shown, the parameter calibration device for realizing the function of controlling the behavior of the above-mentioned device includes: a generating unit 501, used to generate a first calibration pattern; a first sending unit 502, used to send a first instruction, wherein the first instruction is used to indicate at least one set of posture parameters of at least one camera device within a first time period; a second sending unit 503, used to send a second instruction to the camera device, wherein the second instruction is used to indicate the shooting time and number of shots of the at least one camera device within the first time period; an acquiring unit 504, used to acquire at least one shooting result from the at least one camera device; a processing unit 505, used to determine the internal parameters of the at least one camera device according to part or all of the shooting results of the at least one shooting result, wherein any of the part or all of the shooting results contains part or all of the first calibration pattern.

[0179] The parameter calibration device provided in the present application can be applied to the calibration of the internal parameters and external parameters of the camera device. When the position of the calibration pattern is fixed, the method accurately adjusts the position and angle of the camera device relative to the calibration pattern by changing the posture parameters of the camera device, so that the camera device can accurately shoot the calibration pattern at each predetermined position and angle relative to the calibration pattern to obtain at least one shooting result. The above shooting result can accurately correspond to the coordinates of the pixel coordinate system and the world coordinate system, so that the corresponding calculated internal parameters and external parameters are more accurate, thereby improving the calibration accuracy.

[0180] Optionally, the first sending unit 502 is specifically used to send a first instruction to a motion device, the motion device carries the at least one camera device, and the motion device is used to control the posture of the at least one camera parameter according to the at least one set of posture parameters. Thus, the motion device can accurately control the posture of the camera device according to the posture parameters, and can achieve the reproduction of the posture, which is conducive to improving the calibration accuracy.

[0181] Optionally, the shooting result includes at least one photo and / or at least one video shot by the at least one camera device in response to the second instruction. Thus, the images involved in calculating the internal parameters of the camera device can be selected from photos or from video frames of the video, which is flexible.

[0182] Optionally, the first time period includes a plurality of the first instructions and a plurality of the second instructions, and the first sending unit 502 and the second sending unit 503 are specifically used to send the plurality of the first instructions and the plurality of the second instructions alternately. Thus, the first instruction and the second instruction can be used to instruct the camera device to take different photos or videos in different postures according to needs, for example, so that the first calibration pattern appears at various positions in the field of view of the camera device, thereby improving the calibration accuracy.

[0183] Optionally, at least one of the internal parameters has a corresponding error threshold; the processing unit 505 is used to determine whether the repetition accuracy of at least one of the internal parameters is greater than the corresponding error threshold; the generation unit 501 is used to adjust the first calibration pattern to the second calibration pattern when the repetition accuracy is greater than the corresponding error threshold, and / or adjust the image quality of the first calibration pattern; the processing unit 505 is used to adjust the number of feature points extracted from the first calibration pattern when the repetition accuracy is greater than the corresponding error threshold. Thus, the present application can perform multiple calibrations on the camera device, so that the internal parameter matrix and the external parameter matrix of the camera device are continuously iteratively updated, and the calibration accuracy is continuously improved until the value of the calibration accuracy is less than the corresponding threshold.

[0184] Optionally, the number and / or size of markers in the first calibration pattern and the second calibration pattern are different.

[0185] Optionally, the processing unit 505 is configured to divide the first calibration pattern into at least one sub-region, and uniformly adjust the number of the feature points in the at least one sub-region, thereby making the number of feature points in each sub-region uniform, so as to improve the calibration accuracy.

[0186] Optionally, the first calibration pattern is a preset grayscale pattern, and the internal parameters include white balance parameters of the at least one camera device.

[0187] Optionally, the first calibration pattern is a preset monochrome pattern, and the internal parameters include vignetting parameters of the at least one camera device.

[0188] Optionally, the first calibration pattern is a preset color card pattern containing multiple standard colors, and the internal parameters include color difference parameters of the at least one camera device.

[0189] Optionally, the first calibration pattern is a preset target pattern including lines at different angles, and the internal parameters include modulation transfer function parameters of the at least one camera device.

[0190] In addition, for example, the above-mentioned devices implement corresponding functions through hardware modules.

[0191] In one embodiment, the above functions of the control device can be controlled by Fig. 9 The structure shown is implemented. For example, the interface unit 130 is used to send a first instruction to the motion device, and to send a second instruction to at least one camera device, and to obtain at least one shooting result from at least one camera device; the memory 120 is used to store computer program code, and the computer program code includes computer instructions; when the processor 110 executes the computer instructions, the control device executes the methods involved in the above embodiments, for example: generating a first calibration pattern; sending a first instruction, the first instruction is used to indicate at least one set of posture parameters of at least one camera device in a first time period; sending a second instruction to the at least one camera device, the second instruction is used to indicate the shooting time and number of shots of the at least one camera device in the first time period; obtaining at least one shooting result from the at least one camera device; determining the internal parameters of the at least one camera device according to part or all of the shooting results in the at least one shooting result, and any of the part or all of the shooting results contains part or all of the first calibration pattern.

[0192] The present application also provides a parameter calibration system. Figure 7 As shown, the parameter calibration system includes: a control device 100, used to generate a first calibration pattern; a display screen 200, coupled to the control device 100, used to display the first calibration pattern; a motion device 400, used to carry at least one camera device 300, so that the field of view of at least one camera device 300 faces the display screen 200, and the motion device 400 is coupled to the control device 100; the control device 100 is also used to send a first instruction to the motion device 400, the first instruction is used to indicate at least one set of posture parameters of the at least one camera device 300 in a first time period; the motion device 400 is also used to Control the posture of at least one camera device 300 according to at least one set of posture parameters; the control device 100 is also used to send a second instruction to at least one camera device 300, the second instruction is used to indicate the shooting time and number of shots of at least one camera device 300 within a first time period; and, obtain at least one shooting result from at least one camera device 300; and, determine the internal parameters of at least one camera device 300 according to part or all of the at least one shooting result, any of the part or all of the shooting results contains part or all of the first calibration pattern.

[0193] The embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer storage medium is run on a computer, the computer is enabled to execute the above-mentioned methods.

[0194] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the above-mentioned methods.

[0195] The present application also provides a chip system. The chip system includes a processor for supporting the above-mentioned device or equipment to implement the functions involved in the above-mentioned aspects, for example, generating or processing the information involved in the above-mentioned method. In a possible design, the chip system also includes a memory for storing the necessary program instructions and data of the above-mentioned device or equipment. The chip system can be composed of a chip, or it can include a chip and other discrete devices.

[0196] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A parameter calibration method, characterized in that: Based on the internal parameters to be determined, generating a corresponding first calibration pattern; Repeat the following steps to perform parameter calibration at least twice: Sending a first instruction to a motion device, the motion device carrying at least one camera device, the first instruction being used to indicate at least one set of posture parameters of the at least one camera device within a first time period, the at least one set of posture parameters being used by the motion device to control the posture of the at least one camera device; Sending a second instruction to the at least one camera device, wherein the second instruction is used to instruct the at least one camera device to take pictures at least one of a shooting time and a shooting number within a first time period; Acquire at least one shooting result from the at least one camera device, wherein the at least one shooting result includes a shooting result of the camera device under each of the posture parameters; determining at least one internal parameter of the at least one camera device according to part or all of the at least one shooting result, wherein any of the part or all of the shooting results includes part or all of the first calibration pattern; Wherein, at least one of the following information used in the at least two parameter calibrations is different: a calibration pattern, an image quality of the calibration pattern, and a number of feature points.

2. The method according to claim 1, characterized in that The shooting result includes at least one photo and / or at least one video taken from the at least one camera device in response to the second instruction.

3. The method according to claim 1, characterized in that The sending of the first instruction comprises: sending a plurality of the first instructions within the first time period; The sending the second instruction to the at least one camera device comprises: sending a plurality of second instructions to the at least one camera device within a first time period; Wherein, a plurality of the first instructions and a plurality of the second instructions are sent alternately.

4. The method according to claim 1, characterized in that: At least one of the internal parameters has a corresponding error threshold. When the repetition accuracy of at least one of the internal parameters is greater than the corresponding error threshold, when performing repeated calibration, the method also includes: adjusting the first calibration pattern to a second calibration pattern, and / or adjusting the number of feature points corresponding to each field of view of the at least one camera device extracted from the at least one shooting result, and / or adjusting the image quality of the first calibration pattern.

5. The method according to claim 4, characterized in that The number and / or size of markers in the first calibration pattern and the second calibration pattern are different.

6. The method according to claim 4 or 5, characterized in that: The adjusting the number of feature points corresponding to each field of view of the at least one camera device extracted from the at least one shooting result includes: determining at least one sub-region included in the first calibration pattern, and uniformly adjusting the number of the feature points in the at least one sub-region.

7. The method according to claim 1, characterized in that The first calibration pattern is a preset grayscale pattern, and the internal parameters include white balance parameters of the at least one camera device.

8. The method according to claim 1, characterized in that The first calibration pattern is a preset monochrome pattern, and the internal parameters include vignetting parameters of the at least one camera device.

9. The method according to claim 1, characterized in that: The first calibration pattern is a preset color card pattern containing a plurality of standard colors, and the internal parameters include color difference parameters of the at least one camera device.

10. The method according to claim 1, characterized in that The first calibration pattern is a preset target pattern including lines at different angles, and the internal parameters include modulation transfer function parameters of the at least one camera device.

11. A parameter calibration device, characterized in that: include: A generating unit, configured to generate a corresponding first calibration pattern based on the internal parameter to be determined; A first sending unit, configured to send a first instruction to a motion device, the motion device carrying at least one camera device, the first instruction being used to indicate at least one set of posture parameters of the at least one camera device within a first time period, the at least one set of posture parameters being used by the motion device to control the posture of the at least one camera device; A second sending unit, used to send a second instruction to the camera device, wherein the second instruction is used to instruct at least one of a shooting timing and a shooting number of the at least one camera device within a first time period; An acquiring unit, configured to acquire at least one shooting result from the at least one camera device, wherein the at least one shooting result includes a shooting result of the camera device under each of the posture parameters; a processing unit, configured to determine at least one internal parameter of the at least one camera device according to part or all of the at least one shooting result, wherein any of the part or all of the shooting results includes part or all of the first calibration pattern; Among them, the first sending unit, the second sending unit, the acquisition unit, and the processing unit repeatedly execute the above steps to perform parameter calibration at least twice, and at least one of the following information used in the at least two parameter calibrations is different: calibration pattern, image quality of the calibration pattern, and the number of feature points.

12. The device according to claim 11, characterized in that The shooting result includes at least one photo and / or at least one video taken from the at least one camera device in response to the second instruction.

13. The device according to claim 11, characterized in that The first sending unit is used to send a plurality of the first instructions within the first time period; The second sending unit is used to send a plurality of second instructions to the at least one camera device within a first time period; Wherein, a plurality of the first instructions and a plurality of the second instructions are sent alternately.

14. The device according to claim 11, characterized in that At least one of the internal parameters has a corresponding error threshold; The processing unit is used to determine whether the repetition accuracy of at least one parameter among the internal parameters is greater than the corresponding error threshold; The generating unit is configured to adjust the first calibration pattern to a second calibration pattern and / or adjust the image quality of the first calibration pattern when performing repeated calibration when the repetition accuracy is greater than the corresponding error threshold; The processing unit is used to adjust the number of feature points corresponding to each field of view of the at least one camera device extracted from the at least one shooting result when performing repeated calibration when the repetition accuracy is greater than the corresponding error threshold.

15. The device according to claim 14, characterized in that The number and / or size of markers in the first calibration pattern and the second calibration pattern are different.

16. The device according to claim 14 or 15, characterized in that The processing unit is used to determine at least one sub-region included in the first calibration pattern, and uniformly adjust the number of the feature points in the at least one sub-region.

17. The device according to claim 11, characterized in that The first calibration pattern is a preset grayscale pattern, and the internal parameters include white balance parameters of the at least one camera device.

18. The device according to claim 11, characterized in that The first calibration pattern is a preset monochrome pattern, and the internal parameters include vignetting parameters of the at least one camera device.

19. The device according to claim 11, characterized in that The first calibration pattern is a preset color card pattern containing a plurality of standard colors, and the internal parameters include color difference parameters of the at least one camera device.

20. The device according to claim 11, characterized in that The first calibration pattern is a preset target pattern including lines at different angles, and the internal parameters include modulation transfer function parameters of the at least one camera device.

21. A parameter calibration device, characterized in that: include: A memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the device executes the method as described in any one of claims 1-10.

22. A parameter calibration system, characterized in that: include: A control device, configured to generate a corresponding first calibration pattern based on the internal parameter to be determined; a display screen, coupled to the control device, and configured to display the first calibration pattern; A motion device, used for carrying at least one camera device so that the field of view of the at least one camera device faces the display screen, and the motion device is coupled to the control device; The control device is further used to send a first instruction to the motion device, wherein the first instruction is used to indicate at least one set of posture parameters of at least one camera device in a first time period, and the at least one set of posture parameters is used by the motion device to control the posture of the at least one camera device; The motion device is further used to control the posture of the at least one camera device according to the at least one set of posture parameters; The control device is further used to send a second instruction to the at least one camera device, wherein the second instruction is used to instruct the at least one camera device to take pictures during the first period of time. as well as, Acquire at least one shooting result from the at least one camera device, the at least one shooting result comprising a shooting result of the camera device under each of the posture parameters; and determining at least one internal parameter of the at least one camera device according to part or all of the at least one shooting result, wherein any of the part or all of the shooting results includes part or all of the first calibration pattern; The control device repeatedly executes the above steps to perform parameter calibration at least twice, and at least one of the following information used in the at least two parameter calibrations is different: calibration pattern, image quality of the calibration pattern, and the number of feature points.

23. A computer storage medium, characterized in that The method comprises computer instructions, and when the computer instructions are executed on a control device, the control device is caused to execute the method according to any one of claims 1 to 10.

24. A computer program product, characterized in that The method comprises computer instructions, and when the computer instructions are executed on a control device, the control device is caused to execute the method according to any one of claims 1 to 10.

25. A chip system, characterized in that: The method comprises a processor and a memory, wherein the memory stores program instructions, and when the program instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 10.

Images