A camera calibration method, calibration device, and computer-readable storage medium

Abstract

The present application provides a camera calibration method, calibration device, and computer-readable storage medium. The camera calibration method includes: establishing a deep learning model, determining a training target set and a training sample set for the deep learning model, and training the deep learning model; after the deep learning model is trained, obtaining a first calibration plate image, extracting first pixel coordinates of feature points in the first calibration plate image and first calibration plate coordinates corresponding to the first pixel coordinates; and analyzing the mapping relationship between the first pixel coordinates and the first calibration plate coordinates using the trained deep learning model to output camera parameters. The camera calibration method, calibration device, and computer-readable storage medium provided in the present application can effectively improve the convenience and accuracy of camera calibration.

Description

Technical Field

[0001] The present application belongs to the field of machine vision calibration technology, and in particular relates to a camera calibration method, a calibration device, and a computer-readable storage medium. Background Art

[0002] With the advancement of hardware and algorithms, machine vision has found increasing applications in the industrial sector. Camera calibration is fundamental to these applications, and numerous researchers have conducted research on this topic. Camera calibration involves using a series of feature points to establish the relationship between the calibration plate coordinate system and the image coordinate system, thereby determining the camera's intrinsic and extrinsic parameters. The accuracy of these parameters directly determines the accuracy of the resulting image depth calculation. Compared to general industrial scenarios, medical surgical robots require higher positioning accuracy, posing a significant challenge to camera calibration.

[0003] In his 2000 paper, "A flexible new technique for camera calibration," published in IEEE Transactions on Pattern Analysis and Machine Intelligence, Zhang Zhengyou proposed a calibration method that has gained widespread industry recognition. Numerous researchers are interested in further optimizing Zhang's method to improve camera calibration accuracy. For example, P. Zhao proposed using a circular calibration plate to replace the checkerboard calibration plate in his paper “Camera Calibration Technology Based on Circular Points for Binocular Stereovision System” published at the Berlin Conference in 2011. The circle is detected on the calibration plate and the center of the circle is used to construct the calibration plate coordinate system and the image coordinate system. This method can effectively improve the accuracy of feature point detection and ultimately improve the accuracy of depth calculation. X. Liu pointed out in his paper “Precise and robust binocular camera calibration based on multiple constraints” published in Applied Optics in 2018 that the calibration method proposed by Zhang Zhengyou only optimizes the reprojection error and it is difficult to obtain accurate camera internal and external parameters. X. Liu et al. proposed that more accurate calibration parameters can be obtained by optimizing the combined errors such as reprojection error and epipolar error.

[0004] However, current optimizations of Zhang Zhengyou's calibration method have only achieved limited improvements. Existing camera calibration methods still suffer from the following problems: 1. To obtain more accurate camera parameters, a large number of calibration plate images must be captured in advance. 2. Errors exist in feature point positioning, which may be caused by the feature point detection algorithm or by systematic errors introduced during optical imaging. These errors can be collectively referred to as noise. 3. The equation solution often falls into the problem of local optimal solutions, resulting in certain errors in the solved camera parameters. For example, the circular calibration plate proposed by P. Zhao et al., while easier to detect feature points than the checkerboard calibration plate, is still affected by noise in feature point positioning. The multi-error optimization strategy proposed by X. Liu et al. cannot ensure that the combination of multiple errors will result in a globally optimal solution for camera parameters. Improving the convenience and accuracy of camera calibration remains an urgent problem. Summary of the Invention

[0005] In response to the above technical problems, the present application provides a camera calibration method, a calibration device, and a computer-readable storage medium to improve the convenience and accuracy of camera calibration.

[0006] The present application provides a camera calibration method, including: establishing a deep learning model, determining a training target set and a training sample set for the deep learning model, and training the deep learning model; after the deep learning model training is completed, obtaining a first calibration plate image, extracting first pixel coordinates of feature points in the first calibration plate image and first calibration plate coordinates corresponding to the first pixel coordinates; analyzing the mapping relationship between the first pixel coordinates and the first calibration plate coordinates through the trained deep learning model, and outputting camera parameters.

[0007] In one embodiment, the deep learning model includes a feature point optimization module and a camera parameter regression module; after the deep learning model training is completed, the feature point optimization module is used to optimize the first pixel coordinates; the camera parameter regression module is used to analyze the mapping relationship between the first calibration plate coordinates and the optimized first pixel coordinates, and output the camera parameters.

[0008] In one embodiment, determining the training target set of the deep learning model includes: obtaining different second calibration plate images, extracting second pixel coordinates of feature points in the different second calibration plate images and second calibration plate coordinates corresponding to the second pixel coordinates; and determining the training target set of the deep learning model based on multiple sets of mapping relationships between the second pixel coordinates and the second calibration plate coordinates.

[0009] In one embodiment, the step of determining a training target set for the deep learning model based on multiple sets of mapping relationships between the second pixel coordinates and the second calibration plate coordinates includes: analyzing the multiple sets of mapping relationships between the second pixel coordinates and the second calibration plate coordinates through Zhang's calibration method to determine multiple sets of camera parameters; determining a value range of the corresponding parameter based on the distribution of parameter values of the same parameter in the multiple sets of camera parameters; and selecting a preset number of parameter values within the value range of each parameter and randomly combining them to generate multiple sets of target camera parameters, wherein the multiple sets of target camera parameters constitute the training target set for the deep learning model.

[0010] In one embodiment, the step of determining the value range of the corresponding parameter based on the distribution of parameter values of the same parameter in the multiple groups of camera parameters includes: obtaining the average value and variance of the parameter values of the same parameter in the multiple groups of camera parameters; and determining the value range based on the average value and variance of the parameter values.

[0011] In one embodiment, the step of determining the value range based on the mean value and variance of the parameter value includes: determining the value range as (mean-3*std, mean+3*std) based on the mean value and variance of the parameter value, wherein mean represents the mean value of the parameter value; std represents the variance of the parameter value.

[0012] In one embodiment, determining a training sample set for the deep learning model includes: obtaining third calibration plate coordinates of feature points in a selected calibration plate; determining multiple sets of third pixel coordinates corresponding to the third calibration plate coordinates based on the third calibration plate coordinates and multiple sets of target camera parameters in the training target set, in combination with Zhang's calibration method; adding noise to the multiple sets of third pixel coordinates to obtain multiple sets of fourth pixel coordinates; and using a set consisting of the multiple sets of fourth pixel coordinates as the training sample set for the deep learning model; wherein the noise includes at least one of Gaussian noise and salt and pepper noise.

[0013] In one embodiment, training the deep learning model includes: inputting the third calibration plate coordinates and the fourth pixel coordinates into the deep learning model; training the deep learning model through a loss function; outputting, through the deep learning model, camera parameters whose error with the target camera parameters is less than or equal to the target error, and updating the model parameters of the deep learning model.

[0014] In one embodiment, the step of training the deep learning model using a loss function includes:

[0015] The loss function is calculated using the following formula:

[0016]

[0017] During the training process, ensuring that the function value of the loss function is minimized;

[0018] Wherein, loss is the loss function, para is the true value of the camera parameter, is the estimated value of the camera parameter, point is the true value of the fourth pixel coordinate, is the estimated value of the fourth pixel coordinate, and α is a weight factor.

[0019] In one embodiment, during the training process of the deep learning model, the feature point optimization module in the deep learning model is used to optimize the fourth pixel coordinates; the camera parameter regression module in the deep learning model is used to analyze the mapping relationship between the third calibration plate coordinates and the optimized fourth pixel coordinates, and output camera parameters whose error with the target camera parameters is less than or equal to the target error.

[0020] The present application also provides a camera calibration device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned calibration method when executing the computer program.

[0021] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned calibration method are implemented.

[0022] The present application provides a camera calibration method, calibration device, and computer-readable storage medium, which have the following beneficial effects:

[0023] 1. Determine the training sample set and training target set of the deep learning model by using all possible parameter values of the camera parameters in real application scenarios, and train the deep learning model with this training data. This ensures that the trained deep learning model can obtain the global optimal solution of the camera parameters in actual applications, thereby improving the accuracy of camera calibration.

[0024] 2. The deep learning model learns from a large amount of training data during the training process, so that the trained deep learning model can obtain accurate camera parameters in actual application with only a small number of calibration plate images, thereby improving the convenience of camera calibration.

[0025] 3. By adding noise to the training samples and training the deep learning model with the noisy training samples, the trained deep learning model can eliminate the influence of noise on camera parameters in practical applications, making it more robust to noise and improving the accuracy of camera calibration;

[0026] 4. The pixel coordinates of the feature points are optimized through the feature point optimization module of the deep learning model, and the constraint conditions of the feature point pixel coordinates are added to the loss function to train the feature point optimization module. This ensures that the trained feature point optimization module can output more accurate feature point pixel coordinates in actual applications, thereby improving the accuracy of camera calibration.

[0027] 5. The camera parameters are directly regressed through the camera parameter regression module of the deep learning model without calculating the reprojection or epipolar error. The camera parameter constraints are added to the loss function to train the camera parameter regression module. The trained camera parameter regression module can output more accurate camera parameters in actual applications, thereby improving the accuracy of camera calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of a calibration plate provided in an embodiment of the present application;

[0029] Figure 2 1 is a flowchart of the camera calibration method provided in Example 1 of the present application;

[0030] Figure 3 This is a schematic diagram of the structure of the deep learning model provided in Example 1 of the present application;

[0031] Figure 4 This is a schematic diagram of a specific process of the camera calibration method provided in Example 2 of the present application;

[0032] Figure 5 It is a structural diagram of the camera calibration device provided in Example 3 of the present application. DETAILED DESCRIPTION

[0033] The technical solution of this application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. As used herein, "and / or" includes any and all combinations of one or more of the relevant listed items.

[0034] Figure 1 Schematic diagram of the calibration plate provided in the embodiment of the present application. Figure 1 As shown, Figure 1 (a) represents a checkerboard calibration plate, where the characteristic points of the checkerboard calibration plate are the corners where black and white squares intersect; Figure 1 (b) represents a circular calibration plate, where the feature point of the circular calibration plate is the center of the circular pattern. Optionally, the feature points in the calibration plate image are extracted using a corner detection algorithm or an edge detection algorithm.

[0035] First embodiment

[0036] This embodiment provides a method for determining a training sample set and a training target set of a deep learning model, a training process of the deep learning model, and a method for calibrating a camera in practical applications after the deep learning model training is completed.

[0037] Figure 2 This is a flow chart of the camera calibration method provided in Example 1 of this application. Figure 2 As shown, the calibration method of the present application may include the following steps:

[0038] Step S101: Establish a deep learning model, determine a training target set and a training sample set for the deep learning model, and train the deep learning model;

[0039] Among them, the deep learning model includes a feature point optimization module and a camera parameter regression module; optionally, the deep learning model adopts a general network structure as the backbone structure, and the general network structure includes a fully connected, fully convolutional, or recurrent convolutional network, and combines residual networks, batch normalization and other technologies to optimize the network structure.

[0040] In one embodiment, determining the training target set of the deep learning model in step S101 includes:

[0041] Different second calibration plate images are obtained, and second pixel coordinates of feature points in the different second calibration plate images and second calibration plate coordinates corresponding to the second pixel coordinates are extracted; wherein the different second calibration plate images include images obtained by photographing the same second calibration plate with a camera at different shooting angles and different relative positions.

[0042] Using Zhang's calibration method, multiple mapping relationships between the second pixel coordinates and the second calibration plate coordinates are analyzed to determine multiple sets of camera parameters. The formula used in Zhang's calibration method is as follows:

[0043]

[0044] Among them, (u, v) is the pixel coordinate of the feature point in the calibration plate image, (U, V, W) is the calibration plate coordinate corresponding to the pixel coordinate, and Z is the scale factor; the camera parameters include 12 parameters: image distance (f), the physical length of a pixel on the photosensitive plate (dX, dY), the pixel coordinates of the center of the photosensitive plate in the pixel coordinate system (u0, v0), the angle between the horizontal and vertical edges of the photosensitive plate (θ), the camera's rotation matrix (R), and the camera's translation matrix (T); among them, the camera's rotation matrix (R) includes 3 rotation parameters, and the camera's rotation matrix (T) includes 3 translation parameters.

[0045] Determine the value range of the corresponding parameter based on the distribution of parameter values of the same parameter in multiple sets of camera parameters; optionally, in a preferred embodiment, the parameter values of each camera parameter obey a normal distribution, and the mean and variance of the parameter values are used to characterize the distribution of the parameter values; obtain the mean and variance of the parameter values of the same parameter in multiple sets of camera parameters; determine the value range of the corresponding parameter based on the mean and variance of the parameter values; preferably, the value range of each parameter must satisfy a normal distribution probability of the parameter value of each parameter greater than 97%;

[0046] Optionally, based on the mean and variance of the parameter values, the value range of the corresponding parameter is determined as:

[0047] (mean-3*std, mean+3*std)

[0048] Among them, mean represents the average value of the parameter; std represents the variance of the parameter.

[0049] A preset number of parameter values are randomly combined within the value range of each parameter to generate multiple sets of target camera parameters. For example, 100 parameter values are selected within the value range of each camera parameter to be combined, and 100 12 A set of target camera parameters, multiple sets of target camera parameters constitute the training target set of the deep learning model.

[0050] In one embodiment, determining a training sample set for a deep learning model in step S101 includes:

[0051] Obtain third calibration plate coordinates of feature points in the selected calibration plate; determine multiple sets of third pixel coordinates corresponding to the third calibration plate coordinates based on the third calibration plate coordinates and multiple sets of target camera parameters in combination with Zhang's calibration method; add noise to the multiple sets of third pixel coordinates to obtain multiple sets of fourth pixel coordinates; and use the set of the multiple sets of fourth pixel coordinates as a training sample set for a deep learning model.

[0052] Among them, the selected calibration plate may be the calibration plate corresponding to the first calibration plate image, the calibration plate corresponding to the second calibration plate image, or other calibration plates. The calibration plate corresponding to the first calibration plate image and the calibration plate corresponding to the second calibration plate image may be the same calibration plate or different calibration plates; the noise includes at least one of Gaussian noise and salt and pepper noise.

[0053] Specifically, Gaussian noise refers to a type of noise whose probability density function follows a Gaussian distribution. That is, the number of noise points is the largest at a certain intensity, and the farther away from this intensity, the fewer the noise points are, and this pattern follows a Gaussian distribution. Salt and pepper noise, also known as impulse noise, is a type of randomly appearing white dots (salt noise) or black dots (pepper noise).

[0054] In one embodiment, the deep learning model is trained in step S101, including:

[0055] The coordinates of the third calibration plate and the fourth pixel coordinates are input into the deep learning model; the deep learning model is trained through the loss function; the deep learning model outputs camera parameters whose errors with the target camera parameters are less than or equal to the target errors, and the model parameters of the deep learning model are updated.

[0056] Step S102: After the deep learning model training is completed, a first calibration plate image is obtained, and first pixel coordinates of feature points in the first calibration plate image and first calibration plate coordinates corresponding to the first pixel coordinates are extracted;

[0057] Optionally, the calibration plate is photographed by a camera to obtain a calibration plate image; the pixel coordinates of the feature points in the calibration plate image are extracted with the help of the OpenCV function library. In addition, the positions of the feature points in the calibration plate are pre-set, and a calibration plate coordinate system is established with a feature point in the calibration plate as the origin. According to the row and column where the feature point is located, the feature point in the calibration plate image is matched with the feature point in the calibration plate to determine the calibration plate coordinates corresponding to the pixel coordinates of the feature point in the calibration plate image.

[0058] Step S103: Analyze the mapping relationship between the first pixel coordinates and the first calibration plate coordinates through the trained deep learning model, and output camera parameters.

[0059] Optionally, the structure of the deep learning model is as follows Figure 3 After the deep learning model training is completed, the feature point optimization module is used to optimize the first pixel coordinate A; the camera parameter regression module is used to analyze the mapping relationship between the first calibration plate coordinate B and the optimized first pixel coordinate, and output the camera parameter C.

[0060] Taking the binocular camera as an example, the first pixel coordinate A includes the first pixel coordinate A1 of the left camera and the first pixel coordinate A2 of the right camera; the target camera parameters include the left camera parameter C1 and the right camera parameter C2. C1=(f l ,dX l ,dY l ,u 0l ,v 0l ,θ l ,R l ,T l ), C2=(f r ,dX r ,dY r ,u 0r ,v 0r ,θ r ,R r ,T r ); If a checkerboard with 8 rows and 6 columns of corner points is selected as the calibration board, then m=48.

[0061] Among them, the pixel coordinates are the coordinates of the feature point in the calibration plate image in the pixel coordinate system. The pixel coordinate system is a two-dimensional coordinate system with its origin at the upper left corner of the calibration plate image. The pixel coordinates of the feature point are (m, n), which means that the feature point is m pixels away from the origin in the horizontal coordinate direction and n pixels away from the origin in the vertical coordinate direction.

[0062] The calibration plate coordinates are the coordinates of the feature points in the calibration plate in the calibration plate coordinate system. The calibration plate coordinate system is a three-dimensional coordinate system with its origin at the upper left corner of the calibration plate. The calibration plate coordinates of the feature point are (m, n, k), which means that the feature point is m mm away from the coordinate origin in the horizontal coordinate direction, n mm away from the coordinate origin in the vertical coordinate direction, and k mm away from the coordinate origin in the direction perpendicular to the horizontal and vertical coordinate planes.

[0063] The first calibration plate image, first pixel coordinates, and first calibration plate coordinates are the images and coordinates obtained for calibrating camera parameters in actual application after the deep learning model training is completed; the second calibration plate image, second pixel coordinates, and second calibration plate coordinates are the images and coordinates obtained for determining the training target set of the deep learning model before training the deep learning model; the third pixel coordinates, fourth pixel coordinates, and third calibration plate coordinates are the coordinates obtained for determining the training sample set of the deep learning model before training the deep learning model.

[0064] The camera calibration method provided in Example 1 of the present application determines a training sample set and a training target set of a deep learning model based on all possible parameter values of camera parameters in real application scenarios, and trains the deep learning model through these training data, so that the trained deep learning model can obtain the global optimal solution of camera parameters in actual application, thereby improving the accuracy of camera calibration; the deep learning model learns a large amount of training data during the training process, so that the trained deep learning model only needs a small number of calibration plate images to obtain accurate camera parameters in actual application, thereby improving the convenience of camera calibration; noise is added to the training samples, and the deep learning model is trained with the noisy training samples, so that the trained deep learning model can eliminate the influence of noise on camera parameters in actual application, is more robust to noise, and improves the accuracy of camera calibration.

[0065] Example 2

[0066] This embodiment provides a specific training process for a deep learning model. The difference from the first embodiment is the specific role played by the feature point optimization module and the camera parameter regression module in the training process of the deep learning model, as well as the constraining role of the loss function in the training process of the feature point optimization module and the camera parameter regression module.

[0067] Figure 4 This is a schematic diagram of the specific flow of the camera calibration method provided in Example 2 of this application. Figure 4 As shown, the camera calibration method provided by this application includes:

[0068] Step S201: obtaining the third calibration plate coordinates and the fourth pixel coordinates;

[0069] Step S202: inputting the fourth pixel coordinate into the feature point optimization module for optimization;

[0070] Step S203: inputting the third calibration plate coordinates and the optimized fourth pixel coordinates into a camera parameter regression module for mapping relationship analysis, and outputting the camera parameters;

[0071] Step S204: determining whether the error between the output camera parameters and the target camera parameters is less than or equal to the target error;

[0072] If the error between the output camera parameters and the target camera parameters is greater than the target error, executing step S205: updating the model parameters of the deep learning model to the process parameters, and returning to executing step S202 to continue training the deep learning model;

[0073] If the error between the output camera parameters and the target camera parameters is less than or equal to the target error, step S206 is executed: the model parameters of the deep learning model are updated to the final parameters, and the deep learning model training is completed.

[0074] Optionally, the process parameters are model parameters determined when the error between the output camera parameters and the target camera parameters is greater than the target error; the final parameters are model parameters determined when the error between the output camera parameters and the target camera parameters is less than or equal to the target error.

[0075] During the training process of the deep learning model, the feature point optimization module is used to optimize the fourth pixel coordinates; the camera parameter regression module is used to analyze the mapping relationship between the third calibration plate coordinates and the optimized fourth pixel coordinates, and output camera parameters whose error with the target camera parameters is less than or equal to the target error.

[0076] It is worth mentioning that during the training process of the deep learning model, the deep learning model is trained through a loss function. Optionally, the loss function is calculated using the following formula:

[0077]

[0078] During the training process, ensure that the function value of the loss function is minimized;

[0079] Among them, loss is the loss function, para is the true value of the camera parameter, is the estimated value of the camera parameter, point is the true value of the fourth pixel coordinate, is the estimated value of the fourth pixel coordinate, and α is the weight factor.

[0080] Specifically, the first part of the loss function optimizes the camera parameters, and the second part optimizes the pixel coordinates of the feature points. The two parts of the loss are combined by the weight α. Optionally, the value range of α is (0, 1); when optimizing the camera parameters, the relative error is used as the loss function to overcome the problem of inconsistent dimensions between different camera parameters; when optimizing the pixel coordinates of the feature points, the absolute error is used as the loss function.

[0081] The specific implementation method of this embodiment is referred to Example 1 and will not be repeated here.

[0082] The camera calibration method provided in Example 2 of the present application optimizes the pixel coordinates of feature points through a feature point optimization module of a deep learning model, and adds constraints on the pixel coordinates of feature points to the loss function to train the feature point optimization module, so that the trained feature point optimization module can output more accurate pixel coordinates of feature points in actual applications; directly regresses the camera parameters through a camera parameter regression module of a deep learning model without calculating reprojection or epipolar errors, and adds constraints on the camera parameters to the loss function to train the camera parameter regression module, so that the trained camera parameter regression module can output more accurate camera parameters in actual applications, effectively improving the accuracy of camera calibration.

[0083] Example 3

[0084] This embodiment provides a camera calibration device, which differs from the first and second embodiments in the specific structure of the camera calibration device and the process of implementing the camera calibration method through the camera calibration device.

[0085] Figure 5 This is a schematic diagram of the structure of the camera calibration device provided in Example 3 of this application. Figure 5 As shown, the calibration device of this embodiment includes: a processor 110, a memory 111, and a computer program 112 stored in the memory 111 and executable on the processor 110. When the processor 110 executes the computer program 112, the steps of the above-mentioned calibration method embodiments are implemented, for example Figure 2 Steps S101 to S103 are shown.

[0086] The calibration device may include, but is not limited to, a processor 110 and a memory 111. Those skilled in the art will understand that Figure 5 It is only an example of a calibration device and does not constitute a limitation of the calibration device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the calibration device may also include input and output devices, network access devices, buses, etc.

[0087] The processor 110 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0088] The memory 111 may be an internal storage unit of the calibration device, such as a hard disk or memory of the calibration device. The memory 111 may also be an external storage device of the calibration device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the calibration device. Furthermore, the memory 111 may also include both an internal storage unit and an external storage device of the calibration device. The memory 111 is used to store the computer program and other programs and data required by the calibration device. The memory 111 may also be used to temporarily store data that has been output or is to be output.

[0089] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the calibration method described above are implemented.

[0090] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] As used herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion of elements other than the listed elements and may also include additional elements not specifically listed.

[0092] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A camera calibration method, characterized in that: include: Establishing a deep learning model and determining a training target set for the deep learning model; Obtaining the third calibration plate coordinates of the feature points in the selected calibration plate; Determining, based on the third calibration plate coordinates and the multiple sets of target camera parameters in the training target set, multiple sets of third pixel coordinates corresponding to the third calibration plate coordinates in combination with Zhang's calibration method; adding noise to the plurality of sets of third pixel coordinates to obtain a plurality of sets of fourth pixel coordinates; wherein the noise includes at least one of Gaussian noise and salt and pepper noise; Using the set consisting of the multiple groups of fourth pixel coordinates as a training sample set for the deep learning model; The deep learning model is trained; during the training process of the deep learning model, a feature point optimization module in the deep learning model is used to optimize the fourth pixel coordinates; a camera parameter regression module in the deep learning model is used to analyze the mapping relationship between the third calibration plate coordinates and the optimized fourth pixel coordinates, and output camera parameters whose error with the target camera parameters is less than or equal to the target error; After the deep learning model training is completed, obtaining a first calibration plate image, extracting first pixel coordinates of feature points in the first calibration plate image and first calibration plate coordinates corresponding to the first pixel coordinates; The mapping relationship between the first pixel coordinates and the first calibration plate coordinates is analyzed by the trained deep learning model to output camera parameters.

2. The calibration method according to claim 1, wherein: The deep learning model includes a feature point optimization module and a camera parameter regression module; The feature point optimization module is used to optimize the first pixel coordinates; the camera parameter regression module is used to analyze the mapping relationship between the first calibration plate coordinates and the optimized first pixel coordinates, and output the camera parameters.

3. The calibration method according to claim 1, wherein: Determining a training target set for the deep learning model includes: Acquire a different second calibration plate image, extract second pixel coordinates of feature points in the different second calibration plate image and second calibration plate coordinates corresponding to the second pixel coordinates; The training target set of the deep learning model is determined based on multiple groups of mapping relationships between the second pixel coordinates and the second calibration plate coordinates.

4. The calibration method according to claim 3, wherein: The step of determining the training target set of the deep learning model according to the multiple sets of mapping relationships between the second pixel coordinates and the second calibration plate coordinates includes: Analyzing multiple sets of mapping relationships between the second pixel coordinates and the second calibration plate coordinates using Zhang's calibration method to determine multiple sets of camera parameters; Determining a value range of a corresponding parameter according to a distribution of parameter values of the same parameter in the multiple groups of camera parameters; A preset number of parameter values are selected within the value range of each parameter and randomly combined to generate multiple groups of target camera parameters, and the multiple groups of target camera parameters constitute the training target set of the deep learning model.

5. The calibration method according to claim 4, wherein: The step of determining the value range of the corresponding parameter according to the distribution of parameter values of the same parameter in the multiple groups of camera parameters includes: Obtaining an average value and a variance of the parameter values of the same parameter in the multiple sets of camera parameters; The value range is determined according to the mean value and variance of the parameter value.

6. The calibration method according to claim 5, wherein: The step of determining the value range according to the mean value and the variance of the parameter value includes: According to the mean and variance of the parameter values, the value range is determined as: in, represents the average value of the parameter value; represents the variance of the parameter values.

7. The calibration method according to claim 1, wherein: The training of the deep learning model includes: Inputting the third calibration plate coordinates and the fourth pixel coordinates into the deep learning model; Training the deep learning model using a loss function; The deep learning model outputs camera parameters whose errors with the target camera parameters are less than or equal to the target errors, and updates the model parameters of the deep learning model.

8. The calibration method according to claim 7, wherein: The step of training the deep learning model using a loss function includes: The loss function is calculated using the following formula: During the training process, ensuring that the function value of the loss function is minimized; in, is the loss function, is the true value of the camera parameter, are the estimated values of the camera parameters, is the true value of the fourth pixel coordinate, is the estimated value of the fourth pixel coordinate, is the weight factor.

9. A camera calibration device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the calibration method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the steps of the calibration method according to any one of claims 1 to 8.

Images