Parameter calibration method and device, electronic equipment and computer readable storage medium

Abstract

Embodiments of the present application disclose a parameter calibration method and device, electronic equipment and a computer readable storage medium. The method comprises: acquiring a target image by a camera to be calibrated; inputting the target image into a trained distortion classification model to obtain a distortion category of the target image; determining a distortion parameter interval of the camera to be calibrated according to the distortion category, wherein the distortion parameter interval corresponds to a value range of a distortion parameter of the camera to be calibrated; determining a target distortion correction parameter of the camera to be calibrated according to the distortion parameter interval; and calibrating the camera to be calibrated according to the target distortion correction parameter. In the embodiments of the present application, the determination of the distortion parameter of the camera to be calibrated is realized by a model, and the camera is calibrated according to the determined target distortion correction parameter. Compared with manual calibration of the parameter by a special parameter calibration device, the efficiency of the camera parameter calibration is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, specifically to a parameter calibration method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the development of computer vision technology, corresponding camera products are constantly emerging, and cameras require parameter calibration. Currently, camera parameter calibration still relies on dedicated parameter calibration equipment on the production line. However, this parameter calibration equipment depends on manual operation, resulting in low efficiency in camera parameter calibration. Summary of the Invention

[0003] This application provides a parameter calibration method, apparatus, electronic device, and computer-readable storage medium, which can improve the efficiency of camera parameter calibration.

[0004] In a first aspect, embodiments of this application provide a parameter calibration method, including:

[0005] The target image is obtained by acquiring images through the camera to be calibrated;

[0006] The target image is input into a trained distortion classification model to obtain the distortion category of the target image;

[0007] Based on the distortion category, the distortion parameter range of the camera to be calibrated is determined, and the distortion parameter range is used to correspond to the value range of the distortion parameters of the camera to be calibrated.

[0008] The target distortion correction parameters of the camera to be calibrated are determined based on the distortion parameter range.

[0009] The camera to be calibrated is calibrated according to the target distortion correction parameters.

[0010] Secondly, embodiments of this application also provide a parameter calibration device, including:

[0011] The acquisition module is used to acquire images from the camera to be calibrated to obtain the target image;

[0012] The acquisition module is used to input the target image into a trained distortion classification model to obtain the distortion category of the target image;

[0013] The determination module is used to determine the distortion parameter range of the camera to be calibrated based on the distortion category, wherein the distortion parameter range corresponds to the value range of the distortion parameters of the camera to be calibrated.

[0014] The selection module is used to determine the target distortion correction parameters of the camera to be calibrated based on the distortion parameter range.

[0015] The calibration module is used to calibrate the camera to be calibrated according to the target distortion correction parameters.

[0016] Thirdly, embodiments of this application also provide an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the parameter calibration method described above.

[0017] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the parameter calibration method described above.

[0018] This application embodiment inputs images captured by the camera to be calibrated into a trained distortion classification model. The model then outputs the distortion category of the image. After determining the distortion parameter range of the camera to be calibrated based on the distortion category, a parameter can be selected from this range as the distortion correction parameter for the camera. This achieves the localization of the distortion parameters of the camera to be calibrated, facilitating camera parameter calibration. Specifically, this application embodiment determines the distortion parameters of the camera to be calibrated through a model, which improves the efficiency of camera parameter calibration compared to manual calibration using dedicated parameter calibration equipment. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of a scenario illustrating the parameter calibration method provided in an embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating the parameter calibration method provided in the embodiments of this application;

[0022] Figure 3 This is a schematic diagram of the output flow of the distortion classification model of the parameter calibration method provided in the embodiments of this application;

[0023] Figure 4 This is a schematic diagram of the parameter calibration device provided in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0025] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] This application provides a parameter calibration method, apparatus, electronic device, and computer-readable storage medium. Specifically, this application provides a parameter calibration apparatus suitable for electronic devices, wherein the electronic device can be a terminal or a server, etc. The terminal can be a mobile phone, camera, camcorder, high-definition camera, or Automated Optical Inspection (AOI) camera, etc., with image acquisition capabilities. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The server can be directly or indirectly connected via wired or wireless communication.

[0027] For example, please see Figure 1 This application embodiment takes the parameter calibration method jointly executed by a terminal and a server as an example. In addition to the terminal and server, other devices can be added to assist in completing the parameter calibration method; the type of other devices is not limited here. The terminal and server are connected via a network, such as a wired or wireless network. The specific execution process is as follows:

[0028] The terminal device 10 to be calibrated starts the image acquisition function, obtains the acquired image, and then sends the acquired image to the server 11;

[0029] After receiving the image sent by the terminal device 10, the server 11 inputs the image into the pre-trained distortion classification model inside the server 11 and outputs the distortion category corresponding to the image. Then, based on the distortion category, the distortion parameter range of the terminal device 10 to be calibrated is determined, where the distortion parameter range corresponds to the value range of the distortion parameters of the terminal device 10 to be calibrated. Then, a distortion parameter value is selected from the distortion parameter range and used as the distortion correction parameter of the terminal device 10 to be calibrated.

[0030] In this embodiment, a distortion parameter value can be randomly selected from the distortion parameter range as the distortion correction parameter of the terminal device 10, or the median value of the distortion parameter range can be used as the distortion correction parameter of the terminal device 10. The specific selection method can be determined according to actual needs. In this embodiment, the specific implementation method and selection conditions for selecting the distortion parameter value from the distortion parameter range are not limited.

[0031] In this embodiment of the application, the parameters of the terminal device 10 can be calibrated according to the selected distortion correction parameters to achieve rapid calibration of the terminal device 10 to be calibrated.

[0032] The parameter calibration method provided in this application relates to machine learning in the field of artificial intelligence. This application embodiment can obtain parameter information of the camera capturing the image through image analysis, improving the accuracy of camera parameter information acquisition and achieving accurate calibration of camera parameters.

[0033] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0034] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0035] This application embodiment inputs images captured by the camera to be calibrated into a trained distortion classification model. The model then outputs the distortion category of the image. After determining the distortion parameter range of the camera to be calibrated based on the distortion category, a parameter can be selected from this range as the distortion correction parameter for the camera. This achieves automatic localization of the distortion parameters of the camera to be calibrated, facilitating camera parameter calibration. Specifically, this application embodiment determines the distortion parameters of the camera to be calibrated through a model, which improves the efficiency of camera parameter calibration compared to manual calibration using dedicated parameter calibration equipment.

[0036] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0037] Please see Figure 2 , Figure 2 This is a flowchart illustrating the parameter calibration method provided in an embodiment of this application. The specific flow of this parameter calibration method can be as follows:

[0038] 101. Acquire the target image by using the camera to be calibrated.

[0039] In this embodiment of the application, the camera to be calibrated may include a camera with unknown parameter information, such as a camera whose parameter information is unclear after manufacturing. In this embodiment of the application, the camera mainly includes a mobile phone camera.

[0040] In this embodiment, the image is rendered based on the parameters of the camera that captured the image; therefore, the parameter information of the camera can be obtained from the image. Based on this, in this application embodiment, the parameter information of the camera that captured the image can be obtained by analyzing the captured image, thereby achieving the localization and acquisition of camera parameters.

[0041] 102. Input the target image into the trained distortion classification model to obtain the distortion category of the target image.

[0042] In this embodiment, the distortion category is determined based on the relative focal length and distortion parameters corresponding to the image. For example, the calculation result of the relative focal length and distortion parameters corresponding to the image is assigned to a distortion category, and then the relative focal length and distortion parameters corresponding to the target image are determined in reverse based on the distortion category output by the model.

[0043] By utilizing a trained distortion classification model, the distortion categories of an image can be classified, thus obtaining the corresponding distortion category. Using a trained distortion classification model improves the accuracy of prediction results compared to directly obtaining distortion parameters.

[0044] In this embodiment, the feature information of the image can be extracted through the backbone network of the distortion classification model. Then, the distortion category corresponding to the feature information of the output image is predicted based on the fully connected layers (branch parts) of the distortion classification model. That is, optionally, in this embodiment, the distortion classification model includes a backbone network and a distortion category classification branch. The step "inputting the target image into the trained distortion classification model to obtain the distortion category of the target image" includes:

[0045] The target image is input into the backbone network to obtain the image feature information of the target image;

[0046] The image feature information is input into the distortion category classification branch to obtain the distortion category of the target image.

[0047] In this embodiment, the distortion classification model is based on EfficientNet (backbone network). By extracting image feature information and outputting the distortion category to which the feature information belongs through a fully connected layer, the distortion category corresponding to the image can be obtained.

[0048] In this embodiment of the application, a trained distortion classification model can be obtained by training the model with sample images, and the training process can improve the accuracy of the model in predicting the distortion category of the image.

[0049] Since images are acquired through camera capture, and the number of cameras and distortion categories are limited, the sample data for model training is also limited. Therefore, in this embodiment, distortion processing can be applied to the acquired original images to enrich the sample images for model training, thereby improving the accuracy of model prediction. That is, in some embodiments of this application, before the step "inputting the target image into the trained distortion classification model to obtain the distortion category of the target image", the method further includes:

[0050] Acquire several original images with different relative focal lengths;

[0051] For each original image, the target distortion category of the original image is determined, and the target distortion parameter range of the original image is determined according to the correspondence between the distortion category and the distortion parameter range;

[0052] For each original image, the target distortion parameters of the original image are determined according to the target distortion parameter range of the original image, and the original image is distorted according to the target distortion parameters to obtain the sample distorted image corresponding to the original image;

[0053] The preset classification model is trained based on the sample distorted image, the distortion category corresponding to the sample distorted image, and the sample relative focal length until the preset stopping condition is met, thus obtaining the trained distortion classification model.

[0054] In this embodiment, the original image is a distortion-corrected image captured by the sample camera; that is, the original image is a distortion-free image. Since the distortion parameters of the sample camera are known, distortion correction can be performed on the captured distorted image based on the known distortion parameters to obtain a distortion-free image after correction.

[0055] By determining the distortion category and distortion parameters of the original image, and performing distortion processing on the original image according to the distortion parameters, a single original undistorted image can generate at least one distorted image (i.e., a sample distorted image), which enriches the sample data used for model training and improves the accuracy of the model's predictions after training.

[0056] In this embodiment of the application, the distortion category of the original image can be randomly selected from various preset distortion categories. The random method can improve the diversity and irregularity of image distortion and enhance the generalization ability of the trained model.

[0057] The camera's parameters mainly include distortion parameters and relative focal length. Therefore, the model can be trained based on sample images and the distortion category and relative focal length information corresponding to the sample images. This allows the trained model to output not only the distortion category of the image but also the corresponding relative focal length.

[0058] In order to improve the model's ability to accurately output the relative focal length corresponding to the image, in this embodiment of the application, the image can also be preprocessed to enrich the relative focal length in the training samples. That is, in some embodiments of this application, the step "training the preset classification model based on the sample distorted image, the distortion category corresponding to the sample distorted image, and the sample relative focal length until the preset stopping condition is met, to obtain the trained distortion classification model" includes:

[0059] For each sample distorted image, the sample distorted image is cropped according to a random cropping ratio to obtain the cropped image corresponding to each sample distorted image;

[0060] For each cropped image, the cropped image is scaled according to the random cropping ratio corresponding to the cropped image to obtain a target sample distortion image. The target relative focal length of the target sample distortion image is determined according to the relative focal length corresponding to the cropped image and the random cropping ratio.

[0061] Based on the sample distorted image, its distortion category, and relative focal length, as well as the target sample distorted image, its distortion category, and target relative focal length, the preset classification model is trained until a preset stopping condition is met, thus obtaining the trained distortion classification model.

[0062] By cropping the image, for example, by cropping at different ratios, at least one cropped image with a different field of view can be obtained. The different field of view can result in images with different relative focal lengths, which enriches the diversity of the relative focal lengths of the images during model training and improves the accuracy of the model's prediction of the relative focal lengths of the images after training.

[0063] In this embodiment, the relative focal length of the cropped image can be calculated based on the image cropping ratio and the relative focal length of the original image. For example, the original image size is w1,h1, and the relative focal length is f. x The image is cropped to dimensions w² and h². Since the image is cropped proportionally in both width and height, the relative focal length of the sample image is f. x ′=f x *w1 / w2.

[0064] In this embodiment of the application, the cropped image is scaled so that the scaled image can maintain the same size as the original image, which facilitates model training.

[0065] In this embodiment, the range of the random size is preferably between the original image size and two-thirds of the original image size. For example, the input image is randomly cropped, and the cropped size is between w1*2 / 3, h1*2 / 3 and w1, h1. Then, it is enlarged back to the original size w1, h1, and then processed by f. x ′=f x *w1 / w2 calculates the processed relative focal length, and the size of this relative focal length is within f. x to f x The random size range between 3 / 2 ensures the clarity of the sample image after scaling, which facilitates feature extraction and improves the accuracy of feature extraction.

[0066] In this embodiment, the same original image can be cropped multiple times at random sizes (for example, the same original image can be input multiple times to obtain multiple sample images cropped at random sizes) to increase the number of sample images used for model training, which facilitates subsequent model training. In this embodiment, both the sample images and the original images can be used as sample data for model training.

[0067] In this embodiment of the application, the original images can be collected from different mobile phone models. For example, 10 mobile phones can be used, each collecting 500 sets of images of different scenes. Each mobile phone has two cameras, a main camera and a secondary camera, for a total of 10,000 images. Then, a calibration tool (e.g., a calibration box) is used to calibrate the main and secondary cameras of the ten mobile phones to determine the main and secondary camera parameters, thereby obtaining the distortion parameters and relative focal length of each image.

[0068] In this embodiment, since the camera parameters include distortion parameters and relative focal length, a new attribute value for analyzing camera parameters can be obtained by calculating the camera's distortion parameters and relative focal length. That is, the new attribute value is related to the camera's distortion parameters and relative focal length, and each camera corresponds to a unique new attribute value. Therefore, it is possible to predict the new attribute value of the camera using a model to determine the new attribute value of the camera. Then, the camera's distortion parameters can be determined based on the new attribute value of the camera and the relative focal length, or the camera's relative focal length can be determined based on the new attribute value of the camera and the distortion parameters.

[0069] However, since the model has difficulty in regressing when predicting new attribute values, that is, the model has a poor ability to accurately predict a specific value, in this embodiment of the application, a classification method can be used to predict new attribute values.

[0070] In this embodiment, by sampling the distortion parameters and relative focal length of the sample camera, a fixed range of values ​​corresponding to the new attribute value of the camera can be determined. When predicting the new attribute value using a classification method, the above-mentioned range of values ​​can be divided into multiple sub-intervals, and the distortion category of each sub-interval can be determined. Therefore, in the classification prediction process, the distortion category corresponding to the image can be predicted. Then, the specific new attribute value of the image can be determined according to the numerical range corresponding to the distortion category. That is, optionally, in some embodiments of this application, before the step "determining the target distortion category of the original image", the method further includes:

[0071] The range of parameter values ​​is determined based on the distortion parameters and relative focal length of the original image;

[0072] The range of parameter value changes is divided into several sub-ranges of parameter value changes;

[0073] Assign a distortion category to each sub-interval of parameter value change to obtain several preset distortion categories.

[0074] The parameter value variation range corresponds to the value range of the new attribute mentioned above. By assigning a distortion category to each parameter value variation sub-range, the distortion category corresponding to each parameter value variation sub-range can be obtained. Similarly, after obtaining the distortion category to which the image belongs, the specific parameter value (new attribute value) corresponding to the image can be determined according to the correspondence between the distortion category and the parameter value variation sub-range.

[0075] In the process of constructing training samples, when determining the target distortion category for each original distortion-free image, it is possible to randomly select from multiple distortion categories. That is, the step "determining the target distortion category of the original image" includes:

[0076] One distortion category is randomly selected from the several preset distortion categories as the target distortion category.

[0077] The randomization method enhances the diversity of image distortion categories and improves the generalization ability of the trained model. After determining the target distortion category corresponding to the original image, distortion parameters can be selected from the parameter value variation sub-interval corresponding to the target distortion category to perform distortion processing on the image. In this embodiment, the target distortion parameters can be randomly selected from the parameter value variation sub-interval to perform distortion processing on the original image, resulting in a distorted sample image.

[0078] When a camera captures an image, there is radial distortion and tangential distortion. Since tangential distortion is generally small, this embodiment only considers radial distortion. Radial distortion generally includes three distortion parameter values: a first distortion parameter k1, a second distortion parameter k2, and a third distortion parameter k3. The influence of the third distortion parameter k3 is small and can be ignored. However, in this embodiment, for ease of calculation, only the first distortion parameter k1 and the second distortion parameter k2 are considered.

[0079] The distortion formula is as follows: Let the original coordinates of the image be (u, v), and the distorted coordinates be (u, v). d ,v d ), then x = (uc) x ) / f x ,y=(vc y ) / f y , x'=x*(1+k1r 2 +k2r 4 ), y'=y*(1+k1r 2 +k2r 4 ), u d =f x *x'+c x ,v d =fy *y'+c y , where r 2 =x 2 +y 2 f x =f y c x ,c y Let K be the coordinates of the center point of the image. Statistically, the distortion parameters k1 and k2 satisfy the relationship: k2 = 0.018k1 + 0.805k1 2 Therefore, when performing distortion processing on an image, after determining the first distortion parameter k1, the second distortion parameter k2 can be determined using the above formula.

[0080] Therefore, to simplify the complexity of data processing, in this embodiment, the new attribute value of the camera can be determined by the first distortion parameter k1 and the relative focal length. For example, the quotient of the first distortion parameter k1 and the square of the relative focal length can be determined as the new attribute value of the camera, that is, the new attribute value k1' = k1 / f x 2 According to sample statistics, the range of values ​​for k1', i.e., the range of new attribute values ​​(i.e., the range of parameter value changes), is within [-5×10]. -7 5×10 -7 ]between.

[0081] Therefore, in this embodiment, the parameter value variation range can be divided into sub-intervals, and each sub-interval can be categorized. Then, the prediction of camera parameter information is transformed into a camera parameter type classification problem, improving the accuracy of camera parameter prediction. For example, in this embodiment, the parameter value variation range is divided into 40 sub-intervals, each assigned a distortion category. Each sub-interval can be distinguished by a category identifier C, denoted as category C∈[0,39]. The corresponding value range for each sub-interval is [-5*10]. -7 +c*2.5*10 -8 -5*10 -7 +(c+1)*2.5*10 -8 Then, the model is trained to classify the distortion category of the target image. Once the distortion category of the target image is determined, the corresponding parameter value variation sub-interval is determined based on the correspondence between the distortion category and the parameter value variation sub-interval. Subsequently, according to... Furthermore, the first distortion parameter k1 and the second distortion parameter k2 can be calculated from the relative focal length of the image output by the model.

[0082] 103. Based on the distortion category, determine the distortion parameter range of the camera to be calibrated, wherein the distortion parameter range corresponds to the value range of the distortion parameters of the camera to be calibrated.

[0083] In this process, by determining the distortion category corresponding to the image, the distortion parameter range of the camera that captured the image can be determined based on the distortion category of the image.

[0084] In this embodiment, since there is a computational relationship between the camera's distortion category, relative focal length, and distortion parameters, the camera's distortion parameters can be determined based on the camera's new attribute values ​​and relative focal length, or the camera's relative focal length can be determined based on the camera's new attribute values ​​and distortion parameters. Because different relative focal lengths result in different field-of-view ranges in the image, predicting the relative focal length information corresponding to the image using a regression model is more accurate. Optionally, in some embodiments of this application, the distortion classification model further includes a focal length information prediction branch. The step "determining the distortion parameter range of the camera to be calibrated based on the distortion category" includes:

[0085] The image feature information is input into the focal length information prediction branch to obtain the relative focal length of the target image;

[0086] The distortion parameter range of the target image is determined based on the distortion category and the relative focal length.

[0087] Specifically, the relative focal length of the output image is predicted by using focal length information. Then, the distortion parameter range of the image can be obtained based on the distortion category and relative focal length of the image.

[0088] For example, please see Figure 3 , Figure 3 This is a schematic diagram of the output process of the distortion classification model. During the training of the EfficientNet network model, two branches can be output in the globally fully connected layer. One branch predicts the parameter value label type through the classification model, and the other branch predicts the relative focal length through the regression model. Then, the distortion parameter information of the camera is determined by the relative focal length and the parameter value label type. Accordingly, the EfficientNet network model requires two loss functions. For the classification model, the cross-entropy loss function can be used, and for the regression model, the L2 (least squared error) loss function can be used. In this embodiment, the learning rate of the model is 0.001, and it is iterated until the model converges.

[0089] Since each distortion category of an image corresponds to a sub-interval of parameter value changes, after predicting the distortion category of each image, the corresponding sub-interval of parameter value changes can be obtained. Furthermore, after determining the relative focal length of the image, the distortion parameter interval corresponding to the image can be determined based on the sub-interval of parameter value changes and the relative focal length. That is, the step "determine the distortion parameter interval of the target image based on the distortion category and the relative focal length" includes:

[0090] Determine the sub-interval of parameter value changes corresponding to the distortion category;

[0091] The distortion parameter range of the target image is determined based on the sub-range of parameter value changes and the relative focal length.

[0092] By calculating the relationship between relative focal length, distortion parameters, and new attribute values, the distortion parameter range corresponding to the distortion category can be determined.

[0093] Since the image may have black borders after distortion processing, and these black borders affect model training, in this embodiment, the black borders of the image are cropped to ensure the predictive ability of the model. Optionally, in some embodiments of this application, the step "training a preset classification model based on the sample distorted image, the distortion category corresponding to the sample distorted image, and the sample relative focal length until a preset stopping condition is met, to obtain the trained distortion classification model" includes:

[0094] The distorted sample image is cropped according to the target cropping size to obtain a distorted sample cropped image.

[0095] The preset classification model is trained based on the cropped image of the distorted sample and the corresponding distortion category and relative focal length of the cropped image of the distorted sample until the preset stopping condition is met, thus obtaining the trained distortion classification model.

[0096] In this process, cropping the distorted sample images can remove potential black borders, improving the accuracy of model training and prediction. Furthermore, uniform cropping ensures that all training images are the same size, facilitating model training. For example, all images can be cropped from 784x588 to 640x480.

[0097] 104. Determine the target distortion correction parameters of the camera to be calibrated based on the distortion parameter range.

[0098] Each distortion category corresponds to a distortion parameter range, and each distortion parameter range corresponds to multiple specific distortion parameters. Therefore, in this embodiment, a distortion parameter can be randomly selected from the distortion parameter range as the target distortion correction parameter for the camera to be calibrated. In this embodiment, the median value of the distortion parameter range can also be used as the target distortion correction parameter for the camera to be calibrated, thereby enabling the location and acquisition of camera distortion parameters and facilitating the calibration of camera parameters.

[0099] 105. Calibrate the camera to be calibrated according to the target distortion correction parameters.

[0100] Among them, the calibration of the camera to be calibrated can be achieved by using the target distortion correction parameters; and the efficiency of camera parameter calibration is improved by obtaining the distortion correction parameters of the camera through model prediction.

[0101] This application embodiment inputs images captured by the camera to be calibrated into a trained distortion classification model. The model then outputs the distortion category of the image. After determining the distortion parameter range of the camera to be calibrated based on the distortion category, a parameter can be selected from this range as the distortion correction parameter for the camera. This achieves the localization of the distortion parameters of the camera to be calibrated, facilitating camera parameter calibration. Specifically, this application embodiment determines the distortion parameters of the camera to be calibrated through a model, which improves the efficiency of camera parameter calibration compared to manual calibration using dedicated parameter calibration equipment.

[0102] By cropping and scaling the original image, and by applying random distortion to the distortion-free image, the number of sample images available for training is increased, thereby improving the accuracy of model training.

[0103] To facilitate better implementation of the parameter calibration method of this application, this application also provides a parameter calibration device based on the above parameter calibration method. The meanings of the terms used are the same as in the above parameter calibration method, and specific implementation details can be found in the description of the method embodiments.

[0104] Please see Figure 4 , Figure 4 The schematic diagram of the parameter calibration device provided in this application shows that the parameter calibration device may include a data acquisition module 201, an acquisition module 202, a determination module 203, a selection module 204, and a calibration module 205, as follows:

[0105] The acquisition module 201 is used to acquire images through the camera to be calibrated to obtain the target image.

[0106] The acquisition module 202 is used to input the target image into a trained distortion classification model to obtain the distortion category of the target image.

[0107] The distortion classification model includes a backbone network, a distortion category classification branch, and a focal length information prediction branch. The acquisition module 202 includes:

[0108] The first acquisition unit is used to input the target image into the backbone network to obtain the image feature information of the target image;

[0109] The second acquisition unit is used to input the image feature information into the distortion category classification branch to obtain the distortion category of the target image.

[0110] The determining module 203 is used to determine the distortion parameter range of the camera to be calibrated according to the distortion category, wherein the distortion parameter range corresponds to the value range of the distortion parameters of the camera to be calibrated.

[0111] The determining module 203 includes:

[0112] The third acquisition unit is used to input the image feature information into the focal length information prediction branch to obtain the relative focal length of the target image;

[0113] A determining unit is configured to determine the distortion parameter range of the target image based on the distortion category and the relative focal length.

[0114] The device includes a training unit, which includes:

[0115] The acquisition sub-unit is used to acquire several original images with different relative focal lengths;

[0116] The first determining subunit is used to determine the target distortion category of each original image, and to determine the target distortion parameter range of the original image according to the correspondence between the distortion category and the distortion parameter range.

[0117] The distortion subunit is used to determine the target distortion parameters of the original image based on the target distortion parameter range of the original image for each original image, and to perform distortion processing on the original image based on the target distortion parameters to obtain the sample distortion image corresponding to the original image.

[0118] The training subunit is used to train a preset classification model based on the sample distorted image, the distortion category corresponding to the sample distorted image, and the sample relative focal length, until a preset stopping condition is met, thereby obtaining the trained distortion classification model.

[0119] The device further includes a type determination unit, which includes:

[0120] The second determining subunit is used to determine the range of parameter value changes based on the distortion parameters and relative focal length of the original image;

[0121] Divide into sub-units, used to divide the range of parameter value changes into several sub-ranges of parameter value changes;

[0122] The allocation sub-unit is used to assign a distortion category to each parameter value variation sub-interval, resulting in several preset distortion categories;

[0123] Specifically, the first determining subunit is used for:

[0124] One distortion category is randomly selected from the several preset distortion categories as the target distortion category.

[0125] The determining unit includes:

[0126] The third determining subunit is used to determine the parameter value change sub-interval corresponding to the distortion category;

[0127] The fourth determining subunit is used to determine the distortion parameter range of the target image based on the parameter value change sub-range and the relative focal length.

[0128] Specifically, the training subunit is used for:

[0129] For each sample distorted image, the sample distorted image is cropped according to a random cropping ratio to obtain the cropped image corresponding to each sample distorted image;

[0130] For each cropped image, the cropped image is scaled according to the random cropping ratio corresponding to the cropped image to obtain a target sample distortion image. The target relative focal length of the target sample distortion image is determined according to the relative focal length corresponding to the cropped image and the random cropping ratio.

[0131] Based on the sample distorted image, its distortion category, and relative focal length, as well as the target sample distorted image, its distortion category, and target relative focal length, the preset classification model is trained until a preset stopping condition is met, thus obtaining the trained distortion classification model.

[0132] The training subunit is also specifically used for:

[0133] The distorted sample image is cropped according to the target cropping size to obtain a distorted sample cropped image.

[0134] The preset classification model is trained based on the cropped image of the distorted sample and the corresponding distortion category and relative focal length of the cropped image of the distorted sample until the preset stopping condition is met, thus obtaining the trained distortion classification model.

[0135] The selection module 204 is used to determine the target distortion correction parameters of the camera to be calibrated based on the distortion parameter range.

[0136] The calibration module 205 is used to calibrate the camera to be calibrated according to the target distortion correction parameters.

[0137] In this embodiment, the acquisition module 201 first acquires images captured by the camera to be calibrated to obtain a target image. Next, the acquisition module 202 inputs the target image acquired by the acquisition module 201 into a trained distortion classification model to obtain the distortion category corresponding to the target image. Then, the determination module 203 determines the distortion parameter range of the camera to be calibrated based on the distortion category. Subsequently, the selection module 204 selects the target distortion correction parameters of the camera to be calibrated from the distortion parameter range. Finally, the calibration module 205 calibrates the camera to be calibrated based on the target distortion correction parameters selected by the selection module 204.

[0138] In this embodiment, images captured by the camera to be calibrated are input into a trained distortion classification model. This model outputs the distortion category of the image, and after determining the distortion parameter range of the camera to be calibrated based on the distortion category, a parameter can be selected from this range as the distortion correction parameter. This achieves the localization of the distortion parameters of the camera to be calibrated, facilitating camera parameter calibration. This embodiment uses a model to determine the distortion parameters of the camera to be calibrated, which improves the efficiency of camera parameter calibration compared to manual calibration using dedicated calibration equipment.

[0139] In addition, this application also provides an electronic device, such as Figure 5 As shown, it illustrates the structural diagram of the electronic device involved in this application, specifically:

[0140] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0141] The processor 401 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0142] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and information retrieval by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0143] The electronic device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0144] The electronic device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0145] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402, thereby implementing the steps in any of the parameter calibration methods provided in the embodiments of this application.

[0146] This application embodiment inputs images captured by the camera to be calibrated into a trained distortion classification model. The model then outputs the distortion category of the image. After determining the distortion parameter range of the camera to be calibrated based on the distortion category, a parameter can be selected from this range as the distortion correction parameter for the camera. This achieves the localization of the distortion parameters of the camera to be calibrated, facilitating camera parameter calibration. Specifically, this application embodiment determines the distortion parameters of the camera to be calibrated through a model, which improves the efficiency of camera parameter calibration compared to manual calibration using dedicated parameter calibration equipment.

[0147] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0148] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute the steps in any of the parameter calibration methods provided in this application.

[0149] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0150] Since the instructions stored in the storage medium can execute any of the information acquisition steps provided in this application, the beneficial effects that any parameter calibration method provided in this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0151] The foregoing has provided a detailed description of a parameter calibration method, apparatus, electronic device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A parameter calibration method, characterized in that, include: The target image is obtained by acquiring images through the camera to be calibrated; The target image is input into a trained distortion classification model to obtain the distortion category of the target image; Based on the distortion category, the distortion parameter range of the camera to be calibrated is determined, and the distortion parameter range is used to correspond to the value range of the distortion parameters of the camera to be calibrated. The target distortion correction parameters of the camera to be calibrated are determined based on the distortion parameter range. The camera to be calibrated is calibrated according to the target distortion correction parameters; The distortion classification model includes a backbone network, a distortion category classification branch, and a focal length information prediction branch. The step of inputting the target image into the trained distortion classification model to obtain the distortion category of the target image includes: The target image is input into the backbone network to obtain the image feature information of the target image; The image feature information is input into the distortion category classification branch to obtain the distortion category of the target image; The step of determining the distortion parameter range of the camera to be calibrated based on the distortion category includes: The image feature information is input into the focal length information prediction branch to obtain the relative focal length of the target image; The distortion parameter range of the target image is determined based on the parameter value change sub-range corresponding to the distortion category and the relative focal length; wherein, the parameter value change sub-range is one of multiple parameter value change sub-ranges obtained by dividing the parameter value change range, and the distortion category corresponds one-to-one with the parameter value change sub-range.

2. The method according to claim 1, characterized in that, Before inputting the target image into the trained distortion classification model to obtain the distortion category of the target image, the method further includes: Acquire several original images with different relative focal lengths; For each original image, the target distortion category of the original image is determined, and the target distortion parameter range of the original image is determined according to the correspondence between the distortion category and the distortion parameter range; For each original image, the target distortion parameters of the original image are determined according to the target distortion parameter range of the original image, and the original image is distorted according to the target distortion parameters to obtain the sample distorted image corresponding to the original image; The preset classification model is trained based on the sample distorted image, the distortion category corresponding to the sample distorted image, and the sample relative focal length until the preset stopping condition is met, thus obtaining the trained distortion classification model.

3. The method according to claim 2, characterized in that, Before determining the target distortion category of the original image, the method further includes: The range of parameter values ​​is determined based on the distortion parameters and relative focal length of the original image; The range of parameter value changes is divided into several sub-ranges of parameter value changes; Assign a distortion category to each parameter value variation sub-interval to obtain several preset distortion categories; Determining the target distortion category of the original image includes: One distortion category is randomly selected from the several preset distortion categories as the target distortion category.

4. The method according to claim 2, characterized in that, The step of training a preset classification model based on the sample distorted image, the distortion category corresponding to the sample distorted image, and the sample relative focal length until a preset stopping condition is met, to obtain the trained distortion classification model, includes: For each sample distorted image, the sample distorted image is cropped according to a random cropping ratio to obtain the cropped image corresponding to each sample distorted image; For each cropped image, the cropped image is scaled according to the random cropping ratio corresponding to the cropped image to obtain a target sample distortion image. The target relative focal length of the target sample distortion image is determined according to the relative focal length corresponding to the cropped image and the random cropping ratio. Based on the sample distorted image, its distortion category, and relative focal length, as well as the target sample distorted image, its distortion category, and target relative focal length, the preset classification model is trained until a preset stopping condition is met, thus obtaining the trained distortion classification model.

5. The method according to claim 2, characterized in that, The step of training a preset classification model based on the sample distorted image, the distortion category corresponding to the sample distorted image, and the sample relative focal length until a preset stopping condition is met, to obtain the trained distortion classification model, includes: The distorted sample image is cropped according to the target cropping size to obtain a distorted sample cropped image. The preset classification model is trained based on the cropped image of the distorted sample and the corresponding distortion category and relative focal length of the cropped image of the distorted sample until the preset stopping condition is met, thus obtaining the trained distortion classification model.

6. A parameter calibration device, characterized in that, include: The acquisition module is used to acquire images from the camera to be calibrated to obtain the target image; The acquisition module is used to input the target image into a trained distortion classification model to obtain the distortion category of the target image; The determination module is used to determine the distortion parameter range of the camera to be calibrated based on the distortion category, wherein the distortion parameter range corresponds to the value range of the distortion parameters of the camera to be calibrated. The selection module is used to determine the target distortion correction parameters of the camera to be calibrated based on the distortion parameter range. A calibration module is used to calibrate the camera to be calibrated according to the target distortion correction parameters; The distortion classification model includes a backbone network, a distortion category classification branch, and a focal length information prediction branch. The acquisition module includes: The first acquisition unit is used to input the target image into the backbone network to obtain the image feature information of the target image; The second acquisition unit is used to input the image feature information into the distortion category classification branch to obtain the distortion category of the target image; The determining module includes: The third acquisition unit is used to input the image feature information into the focal length information prediction branch to obtain the relative focal length of the target image; The determining unit is used to determine the distortion parameter range of the target image based on the parameter value change sub-range corresponding to the distortion category and the relative focal length; wherein, the parameter value change sub-range is one of a plurality of parameter value change sub-ranges divided by the parameter value change range, and the distortion category corresponds one-to-one with the parameter value change sub-range.

7. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the parameter calibration method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the parameter calibration method as described in any one of claims 1-5.

Images