Data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure disclose a data processing method and device, electronic equipment and storage medium. The method comprises: obtaining at least two matching features between an image and a three-dimensional model; wherein the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment; determining initial device parameters of the image collection device based on the at least two matching features; and optimizing the initial device parameters based on the two matching features to obtain target device parameters. This technical solution can match a single image obtained by an image collection device with a three-dimensional model in a collection environment, and optimize the initial device parameters by comprehensively considering at least two matching features between the image and the three-dimensional model, thereby improving the accuracy of the device parameters and calibrating more device parameters with a smaller amount of matching features.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image technology, and in particular, to a data processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] At present, image acquisition devices are widely used in various scenes. With the wide application of image acquisition devices, the demand for image acquisition devices is no longer limited to simple picture acquisition, but tends to obtain more information from it, such as three-dimensional space information. There are schemes in the prior art that can recover part of the information in the three-dimensional space based on a single image, but such schemes mostly rely on some assumptions, such as image imaging field of view angle, height, camera distortion size, etc. However, in some real-world scenarios, such as image acquisition devices with high installation height, large field of view angle, and large image distortion, do not meet the above assumptions. In addition, due to the variety of image acquisition devices, it is also difficult to calibrate the system for image acquisition devices that have been put into use. Therefore, how to simplify the parameter calibration method of the image acquisition device is one of the technical problems to be solved in the field. SUMMARY

[0003] The present disclosure provides a data processing method and device, electronic equipment and computer readable storage medium.

[0004] In a first aspect, a data processing method is provided in the embodiments of the present disclosure, comprising:

[0005] obtaining at least two matching features between an image and a three-dimensional model; wherein the image is an image collected by an image acquisition device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment;

[0006] determining initial device parameters of the image acquisition device based on the at least two matching features;

[0007] optimizing the initial device parameters based on the two matching features to obtain target device parameters.

[0008] Further, the three-dimensional model is a three-dimensional model in a world geodetic coordinate system.

[0009] Further, the at least two matching features include matching features between points on the image and points on the three-dimensional model, and / or matching features between lines on the image and lines on the three-dimensional model.

[0010] Further, determining the initial device parameters of the image acquisition device based on the at least two matching features comprises:

[0011] According to the at least two matching features, at least two sets of device parameters in at least two camera calibration modes are obtained.

[0012] An initial device parameter is determined from the at least two sets of device parameters.

[0013] Further, the initial device parameter is determined from the at least two sets of device parameters, including:

[0014] According to the re-projection error corresponding to each matching feature in each set of device parameters, the re-projection error corresponding to each matching feature is obtained.

[0015] For the same camera calibration mode, according to the re-projection error corresponding to each matching feature, a total re-projection error corresponding to the camera calibration mode is determined.

[0016] According to the size of the total re-projection error, the initial device parameter is determined.

[0017] Further, according to the re-projection error corresponding to each matching feature in each set of device parameters, the re-projection error corresponding to each matching feature is obtained, including:

[0018] The matching feature is corrected for distortion by using the device parameter.

[0019] The re-projection error corresponding to the matching feature after distortion correction is obtained.

[0020] In a second aspect, a data processing method is provided in the embodiments of the present disclosure, including:

[0021] At least two sets of device parameters of an image acquisition device in at least two camera calibration modes are obtained.

[0022] An initial device parameter is determined from the at least two sets of device parameters.

[0023] The initial device parameter is optimized to obtain a target device parameter.

[0024] Further, the initial device parameter is determined from the at least two sets of device parameters, including:

[0025] A re-projection error under the device parameter is obtained.

[0026] According to the size of the re-projection error, the initial device parameter is determined.

[0027] Further, the re-projection error under the device parameter is obtained, including:

[0028] A matching feature between an image and a three-dimensional model is corrected for distortion by using the device parameter; wherein the image is an image collected by an image acquisition device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment.

[0029] obtaining the reprojection error of the matched feature under the device parameter after distortion correction.

[0030] Further, the three-dimensional model is a three-dimensional model in a world geodetic coordinate system.

[0031] Further, the matched feature includes a matched feature between a point on the image and a point on the three-dimensional model, and / or a matched feature between a line on the image and a line on the three-dimensional model.

[0032] In a third aspect, a data processing method is provided in the embodiments of the present disclosure, including:

[0033] obtaining input data; the input data includes an image collected by an image collection device from a collection environment and a three-dimensional model in the collection environment;

[0034] calling a preset service interface, so as to obtain at least two matched features between the image and the three-dimensional model by the preset service interface, determine an initial device parameter of the image collection device based on the at least two matched features, and optimize the initial device parameter based on the two matched features to obtain a target device parameter;

[0035] outputting the target device parameter.

[0036] In a fourth aspect, a data processing method is provided in the embodiments of the present disclosure, including:

[0037] obtaining input data; the input data includes at least two camera calibration modes;

[0038] calling a preset service interface, so as to obtain at least two groups of device parameters of an image collection device in at least two camera calibration modes by the preset service interface, determine an initial device parameter from the at least two groups of device parameters, and optimize the initial device parameter to obtain a target device parameter;

[0039] outputting the target device parameter.

[0040] In a fifth aspect, a data processing method is provided in the embodiments of the present disclosure, including:

[0041] receiving a camera calibration request; the camera calibration request includes at least two camera calibration modes and configuration parameters of at least two matched features;

[0042] obtaining the at least two matched features between the image and the three-dimensional model in the at least two camera calibration modes; the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment;

[0043] determine at least two sets of device parameters in the at least two camera calibration modes based on the at least two matching features;

[0044] determine initial device parameters of the image acquisition device from the at least two sets of device parameters;

[0045] optimize the initial device parameters based on the two matching features to obtain target device parameters.

[0046] In a sixth aspect, an embodiment of the present disclosure provides a data processing apparatus, comprising:

[0047] a first obtaining module configured to obtain at least two matching features between an image and a three-dimensional model; wherein the image is an image collected by an image acquisition device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment;

[0048] a first determining module configured to determine initial device parameters of the image acquisition device based on the at least two matching features;

[0049] a first optimizing module configured to optimize the initial device parameters based on the two matching features to obtain target device parameters.

[0050] In a seventh aspect, an embodiment of the present disclosure provides a data processing apparatus, comprising:

[0051] a second obtaining module configured to obtain at least two sets of device parameters of an image acquisition device in at least two camera calibration modes;

[0052] a second determining module configured to determine initial device parameters from the at least two sets of device parameters;

[0053] a second optimizing module configured to optimize the initial device parameters to obtain target device parameters.

[0054] In an eighth aspect, an embodiment of the present disclosure provides a data processing apparatus, comprising:

[0055] a third obtaining module configured to obtain input data; the input data comprising an image collected by an image acquisition device from a collection environment and a three-dimensional model in the collection environment;

[0056] a first calling module configured to call a preset service interface, so as to obtain, by the preset service interface, at least two matching features between an image and a three-dimensional model, determine initial device parameters of the image acquisition device based on the at least two matching features, and optimize the initial device parameters based on the two matching features to obtain target device parameters;

[0057] The first output module is configured to output the target device parameter.

[0058] In a ninth aspect, the embodiments of the present disclosure provide a data processing apparatus, comprising:

[0059] The fourth acquisition module is configured to acquire input data; wherein the input data comprises at least two camera calibration modes;

[0060] The second calling module is configured to call a preset service interface, so as to acquire at least two groups of device parameters of an image acquisition device in at least two camera calibration modes by the preset service interface, determine an initial device parameter from the at least two groups of device parameters, and optimize the initial device parameter to obtain a target device parameter;

[0061] The second output module is configured to output the target device parameter.

[0062] In a tenth aspect, the embodiments of the present disclosure provide a data processing apparatus, comprising:

[0063] The receiving module is configured to receive a camera calibration request; wherein the camera calibration request comprises at least two camera calibration modes and configuration parameters of at least two matching features;

[0064] The fifth acquisition module is configured to acquire the at least two matching features between an image and a three-dimensional model in the at least two camera calibration modes; the image is an image collected by an image acquisition device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment;

[0065] The third determination module is configured to determine at least two groups of device parameters in the at least two camera calibration modes based on the at least two matching features;

[0066] The fourth determination module is configured to determine an initial device parameter of the image acquisition device from the at least two groups of device parameters;

[0067] The third optimization module is configured to optimize the initial device parameter based on the two matching features to obtain a target device parameter.

[0068] The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software comprises one or more modules corresponding to the above functions.

[0069] In a possible design, the apparatus includes a memory and a processor. The memory is configured to store one or more pieces of computer instructions for supporting the apparatus to perform the corresponding method. The processor is configured to execute the computer instructions stored in the memory. The apparatus can further include a communication interface configured to enable the apparatus to communicate with other devices or communication networks.

[0070] In an eleventh aspect, an electronic device is provided, including a memory and a processor. The memory is configured to store one or more pieces of computer instructions. The one or more pieces of computer instructions are executed by the processor to implement the method in any of the preceding aspects.

[0071] In a twelfth aspect, a computer-readable storage medium is provided, configured to store computer instructions for the apparatus in any of the preceding aspects, and the computer instructions are used to implement the method in any of the preceding aspects.

[0072] The technical solutions provided by the embodiments of the present disclosure can have the following beneficial effects.

[0073] In the process of calibrating the camera parameters, at least two matching features between the image and the three-dimensional space are obtained, and the initial device parameters are obtained based on the at least two matching features, and then the initial device parameters are optimized based on the at least two matching features to obtain accurate device parameters. This way, by matching a single image obtained by an image acquisition device with a three-dimensional model in a collection environment, and by comprehensively matching at least two features between the image and the three-dimensional model, the initial device parameters are optimized, which can improve the accuracy of the device parameters, and more device parameters can be calibrated by a smaller amount of matching features.

[0074] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0075] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of the non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0076] Figure 1 A flowchart of a data processing method according to an embodiment of the present disclosure is shown;

[0077] Figure 2 An application schematic diagram in a calibration scenario of camera parameters according to an embodiment of the present disclosure is shown;

[0078] Figure 3 A flowchart of a data processing method according to another embodiment of the present disclosure is shown;

[0079] Figure 4 An application scenario diagram showing camera parameter calibration according to an embodiment of the present disclosure is shown;

[0080] Figure 5 A flowchart of a data processing method according to another embodiment of the present disclosure is shown;

[0081] Figure 6 A flowchart of a data processing method according to another embodiment of the present disclosure is shown;

[0082] Figure 7 A flowchart of a data processing method according to another embodiment of the present disclosure is shown;

[0083] Figure 8 A structural diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0084] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, portions irrelevant to the description of the exemplary embodiments are omitted in the accompanying drawings for the sake of clarity.

[0085] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate that there are features, numbers, steps, actions, components, parts or combinations thereof disclosed in the specification, and do not exclude the possibility that one or more other features, numbers, steps, actions, components, parts or combinations thereof exist or are added.

[0086] It should also be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0087] The details of the embodiments of the present disclosure will be described in detail below through specific embodiments.

[0088] Figure 1 A flowchart of a data processing method according to an embodiment of the present disclosure is shown. As shown in the figure, the data processing method includes the following steps: Figure 1

[0089] In step S101, at least two matching features between an image and a three-dimensional model are acquired; wherein the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment;

[0090] In step S102, initial device parameters of the image collection device are determined based on the at least two matching features;

[0091] ​In step S103, the initial device parameters are optimized based on the two matched features to obtain target device parameters.

[0092] In this embodiment, the device parameters can be calibrated for the image acquisition device. The image acquisition device can be an image acquisition device already installed at a fixed position, for example, the image acquisition device can be an image acquisition device installed at a high position.

[0093] The device parameters to be calibrated can include but are not limited to camera intrinsic parameters, camera extrinsic parameters, and / or camera distortion coefficients. The camera intrinsic parameters can include camera focal length, principal point position, and skew coefficient; the camera extrinsic parameters can include the position and attitude of the camera in the world geodetic coordinate system; the camera distortion coefficients can include radial distortion coefficients and tangential distortion coefficients; due to the deviation of lens manufacturing precision and assembly process, the camera can be distorted during imaging, which can be divided into radial distortion and tangential distortion, the radial distortion is caused by the lens shape on the camera lens, and the tangential distortion is caused by the non-parallelism between the camera lens itself and the imaging plane.

[0094] The existing parameter calibration schemes of image acquisition devices mainly include: a calibration method based on a known calibration object and a calibration method based on image information itself.

[0095] Among them, Zhang's calibration method is a relatively common calibration method based on a known calibration object. In this method, the image acquisition device obtains multiple images by shooting a planar calibration board from multiple angles, and then calibrates and solves the camera parameters based on the homography matrix method. However, this scheme is not suitable for all scenarios, for example, for image acquisition devices installed at a high position, it is difficult to shoot images of the planar calibration board from multiple angles, and the labor cost is high. Moreover, this method can only calibrate the camera intrinsic parameters and the camera distortion coefficients, and cannot calibrate the camera extrinsic parameters in the world geodetic coordinate system (such as the WGS84 coordinate system).

[0096] The vanishing point-based camera calibration method is a relatively common calibration method based on image information itself. This method does not require a calibration object, but only uses vanishing point information on the image to calibrate the camera. In three-dimensional space, parallel lines are projected onto the imaging plane and intersect at a point, which is the vanishing point. This method requires objects with parallel lines to calculate the vanishing point (such as highway lane lines, indoor corridor corner lines, and building outline lines), so the applicable scenarios are limited. In addition, the calibration effect of the vanishing point calibration method depends on the calculation accuracy of the vanishing point position, and the calculated vanishing point position usually has a large error, so the camera parameters obtained by this method have poor accuracy. Furthermore, this method cannot calibrate parameters such as camera distortion coefficients.

[0097] Another common calibration method based on image information itself is self-calibration motion structure from motion method. This method needs to obtain multiple images co-visible with the images collected by the image collection device, and then uses the method of multi-view geometry to calibrate the camera parameters corresponding to all images while optimizing the loss function. However, this method needs to manually shoot a set of images with similar view angles to the images collected by the image collection device on site, which has high labor cost. In addition, if the installation position of the image collection device is high, the view angle difference between the images shot on the ground is too large, which can easily lead to failure of image feature point matching. Moreover, this method can only calibrate the camera intrinsic parameters and camera distortion coefficients, and cannot calibrate the camera extrinsic parameters in the world geodetic coordinate system (such as the WGS84 coordinate system).

[0098] In the embodiments of the present disclosure, the image collection device can be a monocular camera, and in the parameter calibration process of the image collection device, a single image in the collection environment can be obtained by using the image collection device. The collection environment is the parameter calibration environment of the image collection device. That is, the single image is used to calibrate the monocular camera in the embodiments of the present disclosure. The three-dimensional model in the embodiments of the present disclosure can be obtained by using an image-based data reconstruction method or a laser scanning reconstruction method. When collecting data of the collection environment, a handheld device, an unmanned vehicle or a drone can be used for collection based on the needs of the environment. The reconstruction method of the three-dimensional model can refer to the known technology, which is not limited here.

[0099] The matching features can be matching features between the image and the three-dimensional model, which can include but are not limited to point features, line features, curve features, point-to-plane features, semantic features, etc. The matching point features are matching features between a first point on the image and a second point on the three-dimensional model. The matching line features are matching features between a straight line on the image and a corresponding straight line on the three-dimensional model. The at least two kinds of matching features can be two or more different features selected from the point features, the line features, the curve features, the point-to-plane features and the semantic features, etc.

[0100] In some embodiments, one of the at least two matching features can be matching point features and / or matching line features. Thus the at least two matching features can include a combination of matching point features and / or matching line features with one or more other features. In some embodiments, a plurality of the matching features can be obtained for each type of matching feature between the image and the three-dimensional model. For example, a plurality of point features can be extracted from the image using known point feature extraction methods (such as ORB point features, SIFT point features, SURF point features, etc.), and then a plurality of point features can be obtained from the three-dimensional model using automatic matching or manual annotation to obtain matching point features. For another example, a plurality of line features can be extracted from the image using known line feature extraction methods (such as the LSD method), and then a plurality of line features can be obtained from the three-dimensional model using automatic matching or manual annotation to obtain matching line features. It can be understood that the matching features in the embodiments of the present disclosure include matching features (such as point features, line features, surface features, etc.) in the image and the three-dimensional model.

[0101] In some embodiments, the initial device parameters of the image acquisition device can be determined based on the at least two matching features. The initial device parameters can include camera parameters obtained based on the at least two matching features and known calibration methods. In some embodiments, a homography matrix can be calculated from the at least two matching features using a known calibration method such as the homography matrix method, and then the camera parameters can be calculated based on the homography matrix to determine the initial device parameters. In other embodiments, corresponding homography matrices or projection matrices can be calculated from the at least two matching features using multiple known calibration methods such as the homography matrix method or the projection matrix method, and then multiple sets of camera parameters can be calculated based on the homography matrices or the projection matrices, and one of the sets of camera parameters can be determined as the initial device parameters.

[0102] After the initial device parameters are determined, the initial device parameters can be optimized based on the at least two matching features to obtain target device parameters. The optimization method can use a non-linear optimization method such as a bundle adjustment optimization method. The bundle adjustment optimization method can use the initial device parameters as initial values, construct a loss function using the re-projection errors of the at least two matching features, and iteratively optimize the loss function to finally solve more accurate camera parameters. In the optimization process, the re-projection error corresponding to each matching feature is calculated, and the sum of the re-projection errors corresponding to each matching feature is used as the loss for iterative optimization.

[0103] In the process of calibrating camera parameters, the embodiments of the present disclosure acquire at least two matching features between an image and a three-dimensional space, and obtain initial device parameters based on the at least two matching features, and then optimize the initial device parameters based on the at least two matching features to obtain accurate device parameters. This way, by matching a single image obtained by an image acquisition device and a three-dimensional model in a collection environment, and by optimizing the initial device parameters based on at least two matching features between the image and the three-dimensional model, the accuracy of the device parameters can be improved, and more device parameters can be calibrated by fewer matching features.

[0104] In an optional implementation of the present embodiment, the three-dimensional model is a three-dimensional model in a world geodetic coordinate system.

[0105] In this optional implementation, after establishing the three-dimensional model for the collection environment in which the image acquisition device is located, the position information of the three-dimensional model in the world geodetic coordinate system can be assigned to the points on the three-dimensional model. For example, the data in the collection environment can be collected by a UAV, and the UAV can acquire the position information (such as GPS information) of the data in the world geodetic coordinate system while collecting the data. The three-dimensional model established by the data collected by the UAV can be directly aligned to the world geodetic coordinate system. Since the three-dimensional model is a three-dimensional model in the world geodetic coordinate system, the camera extrinsic parameters calibrated by the matching features between the image collected by the image acquisition device and the three-dimensional model are also relative to the values in the world geodetic coordinate system.

[0106] Figure 2 An application schematic diagram in the calibration scenario of camera parameters according to an embodiment of the present disclosure is shown. As shown in Figure 2 The image acquisition device 201 is installed in the high-altitude collection environment 202.

[0107] 1. Three-dimensional model reconstruction.

[0108] The data in the high-altitude collection environment 202 is acquired by the UAV 203 (for example, the image, video and / or point cloud data in the high-altitude collection environment can be collected), and then the three-dimensional model of the high-altitude collection environment 202 is reconstructed according to the data collected by the UAV 203. The three-dimensional model can be a three-dimensional model in the world geodetic coordinate system. The points on the three-dimensional model have world geodetic coordinates.

[0109] 2. Matching feature extraction.

[0110] An image in the high-altitude collection environment 202 is collected by the image collection device 201, and ORB (Oriented FAST and rotated BRIEF), SURF (Speeded Up Robust Features) or SIFT (Scale-Invariant Feature Transform) feature points in the image are extracted, and line features are extracted from the image by using a line detection algorithm LSD (Line Segment Detector).

[0111] 3. Determine initial device parameters.

[0112] The matching points and matching lines matched with the point features and line features in the three-dimensional model are obtained by automatic matching or manual annotation in the three-dimensional model reconstructed from the data collected by the unmanned aerial vehicle 203. The matching points and matching lines in the image and the three-dimensional model are taken as the matching results, and the matching results are taken as the input of the calibration algorithm to obtain the camera parameters of the image collection device 201. For example, a homography matrix corresponding to the matching results can be obtained by using a homography matrix method, and then the camera parameters of the image collection device 201, including the camera intrinsic parameters and the camera extrinsic parameters, are calculated from the homography matrix by using the orthogonality of the rotation matrix. In the case that the principal point of the image collection device 201 is located at the image center and the skew factor is 0, only the camera focal length can be calibrated.

[0113] 4. Optimize the initial device parameters to obtain target device parameters.

[0114] The camera parameters obtained by using the homography matrix calibration are taken as the initial device parameters, and the initial device parameters are further optimized based on the matching results of the point features and the line features. In the optimization process, the bundle adjustment optimization method can be used for optimization.

[0115] The bundle adjustment optimization method: a loss function of the re-projection error of the point features and the re-projection error of the line features is established, and the re-projection error of the point features and the re-projection error of the line features are iteratively optimized based on the loss function. The re-projection error of the point features and the re-projection error of the line features are both obtained by projecting the points or lines on the three-dimensional model to the imaging plane through the projection matrix, and the projection matrix can be adjusted by continuously optimizing the loss of the re-projection error of the point features and the re-projection error of the line features, so as to finally obtain the projection matrix that minimizes the loss of the re-projection error of the point features and the re-projection error of the line features, and then the optimized camera parameters can be calculated according to the projection matrix.

[0116] In an optional implementation of the embodiment, the step S102, i.e., the step of determining the initial device parameters of the image acquisition device based on the at least two matching features, further comprises the following steps:

[0117] obtaining at least two sets of device parameters in at least two camera calibration modes according to the at least two matching features;

[0118] determining the initial device parameters from the at least two sets of device parameters.

[0119] In the optional implementation, the device parameters can be camera parameters of the image acquisition device. Multiple camera calibration modes can be selected to calibrate the camera parameters of the image acquisition device. In each camera calibration mode, at least two sets of camera parameters are obtained by using at least two matching features.

[0120] In some embodiments, one of the at least two camera calibration modes can be a camera calibration mode suitable for a planar scene, and the other can be a camera calibration mode suitable for a non-planar scene.

[0121] The above method of the embodiment of the present disclosure is illustrated below by way of example.

[0122] Suppose that two camera calibration modes, i.e., homography method and projection matrix method, are adopted; and suppose that N pairs of matching features are obtained from the image and the three-dimensional model, the N pairs of matching features including matching features between a first point on the image and a second point on the three-dimensional model, and matching features between a first line on the image and a second line on the three-dimensional model.

[0123] 1. The process of obtaining camera parameters from the N pairs of matching features by using the homography method is as follows:

[0124] The point feature x on the image and the point feature X=[X1 Y1 0] on the plane in the three-dimensional scene satisfy the following homographic relationship:

[0125]

[0126] where P is the projection matrix of the camera, K is the intrinsic parameter matrix of the camera, R=[r1 r2 r3] is the rotation matrix of the camera, t is the translation vector of the camera, and H is the homography matrix.

[0127] The line feature l p on the image and the line feature l w on the plane in the three-dimensional scene (where the line feature l w corresponds to any end point X on the straight line) satisfy the following homographic relationship:

[0128]

[0129] Based on the aforementioned N pairs of matching features (including point-to-point and point-to-line matching features), the homography matrix can be obtained using a normalized direct linear transformation. Then, utilizing the orthogonality property of the rotation matrix, the camera intrinsic and extrinsic parameters can be obtained. Assuming the principal point of the image acquisition device is located at the image center and the tangent coefficient is 0, the camera intrinsic parameters can be obtained simply by calibrating the camera focal length.

[0130] 2. The process of obtaining camera parameters from N pairs of matching features using the projection matrix method is as follows:

[0131] A point feature x on an image and a point feature X on a non-planar surface in a 3D scene satisfy the following homography relation:

[0132] x=PX=K[R|t]X (3)

[0133] Line features on the image p and line features on a plane in a 3D scene w (Denote the characteristic of this line l) w For any endpoint of the corresponding straight line (X), the following homography relation is satisfied:

[0134]

[0135] Based on the aforementioned N pairs of matching features (including point-to-point and point-to-line matching features), the projection matrix P can be obtained using normalized direct linear transformation. Based on the orthogonality property of rotation matrices, and using matrix operations such as RQ decomposition, the camera intrinsic and extrinsic parameters can be decomposed from the projection matrix P.

[0136] 3. Using the homography matrix method and projection matrix method described above, two sets of camera parameters can be obtained. One of the two sets of camera parameters can be selected as the initial equipment parameters based on the magnitude of the error.

[0137] In an optional implementation of this embodiment, the step of determining the initial device parameters from the at least two sets of device parameters further includes the following steps:

[0138] The reprojection error corresponding to each matching feature is obtained based on each set of device parameters;

[0139] For the same camera calibration mode, the total reprojection error corresponding to the camera calibration mode is determined based on the reprojection error corresponding to each of the matching features;

[0140] The initial equipment parameters are determined based on the magnitude of the total weight projection error.

[0141] In the optional implementation, after a set of device parameters is determined according to each camera calibration mode, the re-projection error of each matching feature (including at least two different matching features) can be determined according to the set of device parameters, and the total projection error in the camera calibration mode can be obtained according to the re-projection error corresponding to each matching feature. In some embodiments, the total projection error can be the sum of the re-projection errors corresponding to all matching features.

[0142] After the total projection error corresponding to each camera calibration mode is determined, one set of camera parameters can be selected as the initial device parameters according to the size of the total projection error. The smaller the projection error is, the more accurate the camera parameters are, and therefore in some embodiments, the set of camera parameters corresponding to the smaller total projection error can be determined as the initial device parameters.

[0143] The following still takes the above-mentioned camera parameters obtained from point features and line features by using the homography matrix method and the projection matrix method as an example for illustration.

[0144] The point feature re-projection error and the line feature re-projection error are calculated respectively according to the camera parameters obtained by using the homography matrix method and the projection matrix method.

[0145] 1. Point feature re-projection error calculation process.

[0146] The projection coordinate of the point feature X in the three-dimensional model after being projected to the imaging plane by the camera parameters (obtained by using the homography matrix method or the projection matrix method) is PX, and the difference between the projection coordinate and the observed pixel coordinate x on the image is the point feature re-projection error, and the calculation formula is as follows:

[0147] e point = x - PX (5)

[0148] 2. Line feature re-projection error calculation process.

[0149] The projection coordinate of any endpoint X on the spatial line feature l w in the three-dimensional model after being projected to the imaging plane by the camera parameters (obtained by using the homography matrix method or the projection matrix method) is PX. Let the corresponding line feature observed on the image be l p = [l1 l2 l3] T Then the line feature re-projection error can be represented as the distance from the projection point PX to l p , and the calculation formula is as follows:

[0150]

[0151] 3. The point feature re-projection error e point and the line feature re-projection error e lineThe summation can obtain total projection error of the camera parameters.

[0152] In an optional implementation of the embodiment, the step of obtaining the re-projection error corresponding to each of the matching features according to each set of the device parameters further includes the following steps:

[0153] correcting the matching features for distortion by using the device parameters;

[0154] obtaining the re-projection error corresponding to the matching features after the distortion correction.

[0155] In the optional implementation, for each camera calibration mode, the matching features can be corrected for distortion by using the distortion coefficients in the previously obtained camera parameters, and then the re-projection error is determined according to the corrected matching features. The determination of the re-projection error can refer to the description in the foregoing, which will not be described here again.

[0156] The following continues to take the above-mentioned camera parameter acquisition from point features and line features by using the homography matrix method and the projection matrix method as an example to describe the process of distortion correction.

[0157] For an image with large distortion, a straight line on the image can be curved, thereby causing the re-projection error of the line feature to lack geometric meaning. Therefore, the point features and the line features observed from the image can be corrected for distortion first, and then the re-projection error is calculated.

[0158] The point feature x is denoted as x after being corrected for distortion by using a distortion model (the distortion coefficients are obtained from the camera parameters obtained by the homography matrix method or the projection matrix method) unidistort , and the distortion correction process is as follows:

[0159] x ′ = K -1 x (7)

[0160]

[0161] wherein k1 and k2 are radial distortion coefficients, and p1 and p1 are tangential distortion coefficients, r 2 = x ′2 +y ′2 . Therefore, the re-projection error of the point feature with distortion can be expressed as follows:

[0162] e point = x undistort -PX (9)

[0163] And the re-projection error of the line feature with distortion can be expressed as follows:

[0164]

[0165] wherein, l p_undistort =[l1 l2 l3] T is the line feature after distortion correction, the two end points of the original line feature are corrected for distortion in the manner described above for the point feature, and the line feature after distortion correction is calculated from the two end points after distortion correction.

[0166] Figure 3 A flowchart illustrating a data processing method according to another embodiment of the present disclosure is shown. As shown in the flowchart, the data processing method includes the following steps: Figure 3

[0167] In step S301, at least two sets of device parameters of an image acquisition device in at least two camera calibration modes are acquired;

[0168] In step S302, initial device parameters are determined from the at least two sets of device parameters;

[0169] In step S303, the initial device parameters are optimized to obtain target device parameters.

[0170] In this embodiment, the device parameters of the image acquisition device can be calibrated. The image acquisition device can be an image acquisition device already installed at a fixed position, for example, an image acquisition device installed at a high place.

[0171] The device parameters to be calibrated can include, but are not limited to, camera intrinsic parameters, camera extrinsic parameters, and / or camera distortion coefficients. The camera intrinsic parameters can include camera focal length, principal point position, and skew coefficient; the camera extrinsic parameters can include the position and attitude of the camera in the world geodetic coordinate system; the camera distortion coefficients can include radial distortion coefficients and tangential distortion coefficients; due to the deviation of lens manufacturing precision and assembly process, the camera can be distorted during imaging, which can be divided into radial distortion and tangential distortion, the radial distortion is caused by the lens shape on the camera lens, and the tangential distortion is caused by the non-parallelism between the camera lens itself and the imaging plane.

[0172] In the embodiments of the present disclosure, the image acquisition device can be a monocular camera, and during the parameter calibration of the image acquisition device, a single image in the acquisition environment can be acquired by using the image acquisition device. The acquisition environment is the parameter calibration environment of the image acquisition device. That is, the single image is used to calibrate the monocular camera in the embodiments of the present disclosure. The three-dimensional model in the embodiments of the present disclosure can be obtained by using an image-based data reconstruction method or a laser scanning reconstruction method. When data acquisition is performed on the acquisition environment, a handheld device, an unmanned vehicle, or a drone can be used for acquisition based on the needs of the environment. The reconstruction method of the three-dimensional model can refer to known technologies, which are not limited herein.

[0173] ​In some embodiments, multiple camera calibration modes can be selected to calibrate the camera parameters of the image acquisition device, and a set of device parameters can be obtained in each camera calibration mode. The device parameters can be camera parameters of the image acquisition device. For example, the camera parameters of the image acquisition device can be calibrated by using a homography matrix method or a projection matrix method.

[0174] In some embodiments, one of the at least two camera calibration modes can be a camera calibration mode suitable for a planar scene, and the other can be a camera calibration mode suitable for a non-planar scene.

[0175] After obtaining multiple sets of device parameters in multiple camera calibration modes, one of the device parameters can be selected as the initial device parameter according to the re-projection error, for example, the device parameter with the smallest re-projection error can be selected as the initial device parameter. The re-projection error is the re-projection error of the matching features between the image acquired by the image acquisition device and the three-dimensional model of the acquisition environment.

[0176] After the initial device parameter is determined, the initial device parameter can also be optimized by using an optimization method of the device parameter, and finally the optimized target device parameter is obtained. The optimization method can use a non-linear optimization method such as a bundle adjustment optimization method. The bundle adjustment optimization method can use the initial device parameter as the initial value, and use the re-projection error of the matching features between the image acquired by the image acquisition device and the three-dimensional model of the acquisition environment to construct a loss function, and iteratively optimize the loss function to finally solve more accurate camera parameters.

[0177] In the process of calibrating the camera parameters, the embodiments of the present disclosure obtain multiple sets of device parameters of the image acquisition device by using at least two camera calibration modes, and then select one of the multiple sets of device parameters as the initial device parameter, and then optimize the initial device parameter to obtain accurate device parameters. This way can improve the accuracy of the device parameters by matching the single image obtained by the image acquisition device with the three-dimensional model in the acquisition environment, and comprehensively obtaining the initial device parameters of the image acquisition device by using multiple camera calibration modes, and then optimizing the initial device parameters.

[0178] In an optional implementation of the present embodiment, determining the initial device parameter from the at least two sets of device parameters comprises:

[0179] Obtaining the re-projection error under the device parameter;

[0180] Determining the initial device parameter according to the size of the re-projection error.

[0181] In the optional implementation, the re-projection error can be determined for each set of device parameters. The re-projection error includes the re-projection error between a feature (which can include but is not limited to a point feature, a line feature, a semantic feature, a plane feature, etc.) on the image and a feature on the three-dimensional model. The image can be an image collected by the image collection device in the collection environment, and the three-dimensional model can be a three-dimensional model of the collection environment established in advance.

[0182] After the three-dimensional model of the collection environment in which the image collection device is located is established, the position information of a point on the three-dimensional model in the world geodetic coordinate system can be assigned. For example, the data in the collection environment can be collected by a drone, and the drone can obtain the position information (such as GPS information) of the data in the world geodetic coordinate system while collecting the data. The three-dimensional model established by the data collected by the drone can be directly aligned to the world geodetic coordinate system. Since the three-dimensional model is a three-dimensional model in the world geodetic coordinate system, the camera extrinsic parameter calibrated by matching the features between the image collected by the image collection device and the three-dimensional model is also a value relative to the world geodetic coordinate system.

[0183] For each set of device parameters, the feature (for example, the observed point) on the three-dimensional model matched with the feature on the image is projected by the device parameters to obtain the projected feature (for example, the projected point calculated by the device parameters), and the re-projection error corresponding to the set of device parameters can be determined by calculating the error between the observed feature on the image and the projected feature. The calculation of the re-projection error of the point feature and the line feature can refer to the formulas (5) and (6) in the above description, respectively.

[0184] The matching features between the image and the three-dimensional model can include but are not limited to point features, line features, curve features, point-to-plane features, semantic features, etc. The matching point feature is the matching feature between a first point on the image and a second point on the three-dimensional model; and the matching line feature is the matching feature between a straight line on the image and a corresponding straight line on the three-dimensional model.

[0185] In some embodiments, at least two matching features can be used to calculate the initial device parameters in each camera calibration mode, and the at least two matching features can include a combination of point features and one or more other features. In some embodiments, a plurality of matching features between the image and the three-dimensional model can be obtained for each matching feature. For example, a plurality of point features (such as ORB point features, SIFT point features, SURF point features, etc.) can be extracted from the image using a known point feature extraction method, and then point features matching the point features can be obtained from the three-dimensional model using automatic matching or manual annotation, to finally obtain the matching point features. For another example, a plurality of line features can be extracted from the image using a known line feature extraction method (such as the LSD method), and then line features matching the line features can be obtained from the three-dimensional model using automatic matching or manual annotation, to finally obtain the matching line features. It can be understood that the matching features in the embodiments of the present disclosure include matching features (such as point features, line features, surface features, etc.) in the image and the three-dimensional model.

[0186] Since a plurality of matching features can be obtained for each set of camera calibration modes, a plurality of re-projection errors can also be obtained accordingly. Therefore, the total projection error corresponding to the device parameters of the set can be calculated according to the plurality of re-projection errors, for example, the total projection error can be the sum of the plurality of re-projection errors.

[0187] The total projection error corresponding to each set of device parameters can be obtained, and in some embodiments, the device parameters with the smallest total projection error can be determined as the initial device parameters. In this way, the initial device parameters obtained are more accurate.

[0188] In an optional implementation of the present embodiment, obtaining the re-projection error under the device parameters comprises:

[0189] Correcting distortion of the matching features between the image and the three-dimensional model using the device parameters, wherein the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model of the collection environment;

[0190] Obtaining the re-projection error of the matching features after distortion correction under the device parameters.

[0191] In the optional implementation, for each camera calibration mode, the distortion coefficients in the previously obtained camera parameters can be used to correct the distortion of the matching features, and then the re-projection error can be determined according to the corrected matching features. The distortion correction of the matching features can be referred to formulas (7) and (8) in the foregoing, and the calculation of the re-projection error of the point features and the line features after distortion correction can be referred to formulas (9) and (10) in the foregoing, which will not be described herein again.

[0192] Figure 4 An application scenario of camera parameter calibration according to an embodiment of the present disclosure is shown. As shown in the figure, for an image acquisition device 401 located in a high-altitude environment, the calibration process of camera parameters is as follows. Figure 4

[0193] 1. Three-dimensional model reconstruction.

[0194] Data in the high-altitude acquisition environment 402 is acquired by the unmanned aerial vehicle 403 (for example, images, videos and / or point cloud data in the high-altitude acquisition environment can be acquired), and a three-dimensional model of the high-altitude acquisition environment 402 is reconstructed according to the data acquired by the unmanned aerial vehicle 403. The three-dimensional model can be a three-dimensional model in a world geodetic coordinate system. Points on the three-dimensional model have world geodetic coordinates.

[0195] 2. Matching feature extraction.

[0196] An image in the high-altitude acquisition environment 402 is acquired by the image acquisition device 401, and ORB (Oriented FAST and rotated BRIEF), SURF (Speeded Up Robust Features) or SIFT (Scale-Invariant Feature Transform) feature points in the image are extracted, and line features are extracted from the image by using a line detection algorithm LSD (Line Segment Detector).

[0197] 3. Determining initial device parameters.

[0198] From the three-dimensional model reconstructed according to the data acquired by the unmanned aerial vehicle 403, point features and line features matched with the above-mentioned point features and line features are obtained by automatic matching or manual annotation. The matched point features and matched line features in the image and the three-dimensional model are taken as a matching result, and the matching result is taken as an input of a calibration algorithm to obtain camera parameters of the image acquisition device 401. In this example, two camera calibration modes are selected: a homography matrix method and a projection matrix method.

[0199] 3.1. The homography matrix corresponding to the matching result is obtained by using the homography matrix method, and the camera parameters of the image acquisition device 401, including camera intrinsic parameters and camera extrinsic parameters, are calculated from the homography matrix by using the orthogonal property of a rotation matrix. In the case that the principal point of the image acquisition device 401 is located at the center of the image and the skew factor is 0, only the camera focal length can be calibrated. The camera calibration method of the homography matrix method is described in the above formula (1), (2) and related text description, and will not be repeated here.

[0200] ​3.2, obtain the projection matrix corresponding to the matching result above by using the projection matrix method, and then according to the orthogonal property of the rotation matrix, the camera intrinsic parameters and the camera extrinsic parameters can be decomposed from the projection matrix P by using the matrix operation such as RQ decomposition. The camera calibration method of the projection matrix method can be referred to formula (3), (4) and related text description in the above, and will not be repeated here.

[0201] 3.3, calculate the re-projection error corresponding to the camera parameters obtained by the homography matrix method and the re-projection error corresponding to the camera parameters obtained by the projection matrix method. The re-projection error calculation of point features can be referred to formula (5) and related text description in the above, and the re-projection error calculation of line features can be referred to formula (6) and related text description in the above, and will not be repeated here.

[0202] 3.4, based on the camera parameters obtained by the homography matrix method, the total projection error of the camera parameters obtained by the homography matrix method is calculated by calculating the total projection error of the camera parameters obtained by the homography matrix method.

[0203] 3.5, based on the camera parameters obtained by the projection matrix method, the total projection error of the camera parameters obtained by the projection matrix method is calculated by calculating the total projection error of the camera parameters obtained by the projection matrix method.

[0204] 3.6, the camera parameters corresponding to the smaller total projection error are determined as the initial device parameters.

[0205] 4, the initial device parameters are optimized to obtain the target device parameters.

[0206] The camera parameters obtained by the homography matrix calibration above are used as the initial device parameters, and then the initial device parameters are optimized based on the matching results of the point features and the line features. In the optimization process, the bundle adjustment optimization method can be used for optimization.

[0207] Bundle adjustment optimization method: establish the loss function of the re-projection error of point features and the re-projection error of line features, and based on the loss function, the re-projection error of point features and the re-projection error of line features are iteratively optimized.

[0208] The loss function can be expressed as follows:

[0209]

[0210] In the formula, C represents the loss of the re-projection error of point features and the re-projection error of line features, ρ(·) represents a robust kernel function, The inverse matrix of the covariance matrix of the re-projection error e point of point features, which is used to weight e point . reprojection error e of a line feature line inverse of a covariance matrix of e line weighted.

[0211] In the iterative optimization process, the initial device parameters can be optimized by continuously reducing the loss by using gradient descent method and the like, and finally the optimized target device parameters can be obtained.

[0212] Figure 5 A flowchart of a data processing method according to another embodiment of the present disclosure is shown. As shown in the flowchart, the data processing method comprises the following steps: Figure 5

[0213] In step S501, input data is obtained; wherein the input data comprises an image collected by an image collection device from a collection environment and a three-dimensional model in the collection environment;

[0214] In step S502, a preset service interface is called so as to obtain at least two matching features between the image and the three-dimensional model by the preset service interface, and determine initial device parameters of the image collection device based on the at least two matching features, and optimize the initial device parameters based on the two matching features to obtain target device parameters;

[0215] In step S503, the target device parameters are output.

[0216] In this embodiment, the device parameters of the image collection device can be calibrated. The image collection device can be an image collection device already installed at a fixed position, for example, the image collection device can be an image collection device installed at a high place.

[0217] The device parameters to be calibrated can include but are not limited to camera intrinsic parameters, camera extrinsic parameters and / or camera distortion coefficients. The camera intrinsic parameters can include camera focal length, principal point position, skew coefficient; the camera extrinsic parameters can include the position and attitude of the camera in the world geodetic coordinate system; the camera distortion coefficients can include radial distortion coefficients and tangential distortion coefficients; due to the deviation of lens manufacturing precision and assembly process, the camera may

[0218] ​In the embodiments of the present disclosure, the data processing method can be executed in the cloud. A preset service interface can be pre-deployed in the cloud. The preset service interface can be a Saas (Software-as-a-Service) interface. The demander can pre-obtain the use right of the preset service interface, and can call the preset service interface to calibrate the camera of the image acquisition device when needed, so as to obtain the camera parameters of the image acquisition device.

[0219] In the embodiments, the demander can pre-obtain the image collected by the image acquisition device in the collection environment and the three-dimensional model of the collection environment, and provide the image and the three-dimensional model as the calling parameters to the preset service interface.

[0220] The image acquisition device can be a monocular camera. In the parameter calibration process of the image acquisition device, a single image in the collection environment can be obtained by using the image acquisition device. The collection environment is the parameter calibration environment of the image acquisition device. That is, the single image is used to calibrate the monocular camera in the embodiments of the present disclosure. The three-dimensional model in the embodiments of the present disclosure can be obtained by using an image-based data reconstruction method or a laser scanning reconstruction method. When collecting data of the collection environment, a handheld device, an unmanned vehicle or an unmanned aerial vehicle can be used based on the environment. The reconstruction method of the three-dimensional model can refer to the known technology, and is not limited herein.

[0221] The matching features can be matching features between the image and the three-dimensional model, which can include but are not limited to point features, line features, curve features, point-to-plane features, semantic features, etc. The matching point features are matching features between a first point on the image and a second point on the three-dimensional model. The matching line features are matching features between a straight line on the image and a corresponding straight line on the three-dimensional model. The at least two matching features can be two or more different features selected from the point features, the line features, the curve features, the point-to-plane features and the semantic features.

[0222] In some embodiments, one of the at least two matching features can be point features. Thus the at least two matching features can include a combination of point features and one or more other features. In some embodiments, a plurality of matching features of each type can be obtained between the image and the 3D model for each type of matching feature. For example, a plurality of point features can be extracted from the image using known point feature extraction methods (such as ORB point features, SIFT point features, SURF point features, etc.), and then a plurality of matching point features can be obtained from the 3D model using automatic matching or manual annotation, to finally obtain the matching point features. For another example, a plurality of line features can be extracted from the image using known line feature extraction methods (such as the LSD method), and then a plurality of matching line features can be obtained from the 3D model using automatic matching or manual annotation, to finally obtain the matching line features. It can be understood that the matching features in the embodiments of the present disclosure include matching features (such as point features, line features, surface features, etc.) in the image and the 3D model.

[0223] In some embodiments, the initial device parameters of the image acquisition device can be determined based on the at least two matching features. The initial device parameters can include camera parameters obtained based on the at least two matching features and known calibration methods. In some embodiments, a homography matrix can be calculated from the at least two matching features using a known calibration method such as the homography matrix method, and then the camera parameters can be solved according to the homography matrix, which can be determined as the initial device parameters. In other embodiments, corresponding homography matrices or projection matrices can be calculated from the at least two matching features using multiple known calibration methods such as the homography matrix method or the projection matrix method, and then multiple sets of camera parameters can be solved according to the homography matrices or the projection matrices, and one of the sets of camera parameters can be determined as the initial device parameters.

[0224] After the initial device parameters are determined, the initial device parameters can be optimized to obtain target device parameters based on the at least two matching features. The optimization method can use a non-linear optimization method such as the bundle adjustment optimization method. The bundle adjustment optimization method can use the initial device parameters as initial values, construct a loss function using the re-projection errors of the at least two matching features, and iteratively optimize the loss function to finally solve more accurate camera parameters. In the optimization process, the re-projection error corresponding to each matching feature is calculated, and the sum of the re-projection errors corresponding to each matching feature is used as the loss for iterative optimization.

[0225] In the process of calibrating the camera parameters, at least two matching features between the image and the three-dimensional space are acquired, and initial device parameters are acquired based on the at least two matching features, and then the initial device parameters are optimized based on the at least two matching features to obtain accurate device parameters. In this way, a single image obtained by matching the image acquisition device and the three-dimensional model in the collection environment, and at least two matching features between the image and the three-dimensional model are integrated, and the initial device parameters are optimized, which can improve the accuracy of the device parameters, and more device parameters can be calibrated by fewer matching features.

[0226] Other details in the embodiment can be referred to the related description in the above, which will not be repeated here.

[0227] Figure 6 A flowchart of a data processing method according to another embodiment of the present disclosure is shown. As shown in the figure, the data processing method comprises the following steps: Figure 6

[0228] In step S601, input data is acquired; wherein the input data comprises at least two camera calibration modes;

[0229] In step S602, a preset service interface is called, so that at least two sets of device parameters of the image acquisition device in at least two camera calibration modes are acquired by the preset service interface, and initial device parameters are determined from the at least two sets of device parameters, and the initial device parameters are optimized to obtain target device parameters;

[0230] In step S603, the target device parameters are output.

[0231] In the embodiment, the data processing method can be executed in the cloud. The preset service interface can be pre-deployed in the cloud. The preset service interface can be a Saas (Software-as-a-service, Software-as-a-service) interface. The demand side can obtain the use right of the preset service interface in advance, and can calibrate the camera of the image acquisition device by calling the preset service interface when needed, so as to obtain the camera parameters of the image acquisition device.

[0232] In the embodiment, the demand side can specify at least two camera calibration modes, and the preset service interface can calibrate the camera parameters of the image acquisition device according to the received at least two camera calibration modes.

[0233] In the embodiment, the device parameters of the image acquisition device can be calibrated. The image acquisition device can be an image acquisition device already installed at a fixed position, for example, the image acquisition device can be an image acquisition device installed at a high place.

[0234] ​The device parameters to be calibrated can include, but are not limited to, camera intrinsic parameters, camera extrinsic parameters, and / or camera distortion coefficients. The camera intrinsic parameters can include a camera focal length, a principal point position, and a skew coefficient. The camera extrinsic parameters can include a position and an attitude of the camera in a world geodetic coordinate system. The camera distortion coefficients can include radial distortion coefficients and tangential distortion coefficients. Due to manufacturing precision and assembly process deviations of a lens, the camera can have distortion during imaging. The distortion can be divided into radial distortion and tangential distortion. The radial distortion is caused by the shape of a lens on the camera lens, and the tangential distortion is caused by the fact that the lens itself on the camera lens is not parallel to the imaging plane.

[0235] In the embodiments of the present disclosure, the image acquisition device can be a monocular camera. During the parameter calibration of the image acquisition device, a single image in the acquisition environment can be acquired by using the image acquisition device. The acquisition environment is the parameter calibration environment of the image acquisition device. That is, the single image is used to calibrate the monocular camera in the embodiments of the present disclosure. The three-dimensional model in the embodiments of the present disclosure can be obtained by using an image-based data reconstruction method or a laser scanning reconstruction method. When data is acquired in the acquisition environment, a handheld device, an unmanned vehicle, or an unmanned aerial vehicle can be used for acquisition based on the needs of the environment. The reconstruction method of the three-dimensional model can refer to known technologies, which are not limited herein.

[0236] In some embodiments, a plurality of camera calibration modes can be selected to calibrate the camera parameters of the image acquisition device, and a set of device parameters can be obtained in each camera calibration mode. The device parameters can be the camera parameters of the image acquisition device. For example, a homography matrix method or a projection matrix method can be used to calibrate the camera parameters of the image acquisition device.

[0237] In some embodiments, one of the at least two camera calibration modes can be a camera calibration mode suitable for a planar scene, and the other can be a camera calibration mode suitable for a non-planar scene.

[0238] After a plurality of sets of device parameters in a plurality of camera calibration modes are obtained, one of the device parameters in each set can be selected as an initial device parameter according to a re-projection error. For example, the device parameter with the minimum re-projection error can be selected as the initial device parameter. The re-projection error is the re-projection error of matching features between an image acquired by the image acquisition device and a three-dimensional model of the acquisition environment.

[0239] After the initial device parameters are determined, the initial device parameters can also be optimized by using a device parameter optimization method, and finally the optimized target device parameters are obtained. The optimization method can use a bundle adjustment optimization method or other nonlinear optimization methods. The bundle adjustment optimization method can use the initial device parameters as initial values, construct a loss function using the re-projection error of the matching features between the image collected by the image collection device and the three-dimensional model in the collection environment, and iteratively optimize the loss function to finally solve more accurate camera parameters.

[0240] In the process of calibrating the camera parameters, the embodiment of the present disclosure obtains multiple sets of device parameters of the image collection device by using at least two camera calibration modes, and then selects one set of device parameters from the multiple sets of device parameters as the initial device parameters, and then optimizes the initial device parameters to obtain accurate device parameters. This way can improve the accuracy of the device parameters by matching a single image obtained by the image collection device with a three-dimensional model in the collection environment, and obtaining the initial device parameters of the image collection device by using multiple camera calibration modes, and then optimizing the initial device parameters.

[0241] Other details in the embodiment can be referred to the related description in the foregoing, which will not be described here.

[0242] Figure 7 A flowchart of a data processing method according to another embodiment of the present disclosure is shown. As shown in Figure 7 The data processing method includes the following steps:

[0243] In step S701, a camera calibration request is received; wherein the camera calibration request includes at least two camera calibration modes and configuration parameters of at least two matching features;

[0244] In step S702, the at least two matching features between an image and a three-dimensional model are obtained in the at least two camera calibration modes; the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment;

[0245] In step S703, at least two sets of device parameters in the at least two camera calibration modes are determined based on the at least two matching features;

[0246] In step S704, initial device parameters of the image collection device are determined from the at least two sets of device parameters;

[0247] In step S705, the initial device parameters are optimized based on the two matching features to obtain target device parameters.

[0248] In this embodiment, the data processing method can be executed on the server side. When camera calibration is performed on the image acquisition device, the user can configure at least two camera calibration modes and the identification of at least two matching features through the configuration parameters provided on the page on the client side, and the user can initiate a request for camera parameter calibration of the image acquisition device through the request button provided on the page to the server side.

[0249] In some embodiments, the user-initiated request can include configuration parameters of at least two camera calibration modes and at least two matching features. In other embodiments, it can also include configuration parameters of the image acquisition device that needs to be calibrated; the configuration parameters of the image acquisition device can include but are not limited to the identification, network address, etc. of the image acquisition device.

[0250] In some embodiments, after the server side receives the request of the user, the image acquisition device can be triggered to acquire the image in the current acquisition environment, and / or the three-dimensional model in the current acquisition environment can be obtained through some preset manner.

[0251] In other embodiments, the user can obtain the image acquired by the image acquisition device in the current acquisition environment and the three-dimensional model of the current acquisition environment in advance, and provide the image and the three-dimensional model to the server side when sending the request.

[0252] In some embodiments, after the server side obtains the image and the three-dimensional model, at least two matching features between the image and the three-dimensional model can be obtained. Then, for the at least two matching features, the corresponding device parameters can be obtained by using each camera calibration mode, and finally at least two sets of device parameters (i.e. one set of device parameters corresponding to each camera calibration mode) can be obtained. The re-projection error, positive depth limit, etc. can be used as a measurement method to select one set of device parameters as the initial device parameters.

[0253] For the initial device parameters, the initial device parameters can be iteratively optimized by using the bundle adjustment optimization method, etc. to obtain the optimized target device parameters.

[0254] The specific details in this embodiment can be referred to the related description in the above, which will not be repeated here.

[0255] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure.

[0256] The data processing apparatus according to an embodiment of the present disclosure can be realized as part or all of an electronic device through software, hardware or a combination of both. The data processing apparatus includes:

[0257] The first obtaining module is configured to obtain at least two matching features between an image and a three-dimensional model; the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model of the collection environment;

[0258] The first determining module is configured to determine initial device parameters of the image collection device based on the at least two matching features.

[0259] The first optimization module is configured to optimize the initial device parameters based on the two matching features to obtain target device parameters.

[0260] In an optional implementation of the embodiment, the three-dimensional model is a three-dimensional model in a world geodetic coordinate system.

[0261] In an optional implementation of the embodiment, the at least two matching features include matching features between points on the image and points on the three-dimensional model, and / or matching features between lines on the image and lines on the three-dimensional model.

[0262] In an optional implementation of the embodiment, the first determining module includes:

[0263] The first obtaining submodule is configured to obtain at least two groups of device parameters in at least two camera calibration modes according to the at least two matching features.

[0264] The first determining submodule is configured to determine initial device parameters from the at least two groups of device parameters.

[0265] In an optional implementation of the embodiment, the first determining submodule includes:

[0266] The second obtaining submodule is configured to obtain the re-projection error corresponding to each matching feature according to each group of device parameters.

[0267] The second determining submodule is configured to determine, for the same camera calibration mode, a total re-projection error corresponding to the camera calibration mode according to the re-projection error corresponding to each matching feature.

[0268] The third determining submodule is configured to determine the initial device parameters according to the size of the total re-projection error.

[0269] In an optional implementation of the embodiment, the second obtaining submodule includes:

[0270] The first correction submodule is configured to perform distortion correction on the matching features by using the device parameters.

[0271] The third obtaining sub-module is configured to obtain the re-projection error corresponding to the matched feature after distortion correction.

[0272] The data processing apparatus in the embodiment and the data processing method in the embodiment and related embodiments correspond to each other, and specific details can be referred to the description of the data processing method in the embodiment and related embodiments above. Figure 1 The data processing method in the embodiment and related embodiments corresponds to the data processing apparatus in the embodiment and related embodiments, and specific details can be referred to the description of the data processing apparatus in the embodiment and related embodiments above. Figure 1 The data processing method in the embodiment and related embodiments corresponds to the data processing apparatus in the embodiment and related embodiments, and specific details can be referred to the description of the data processing apparatus in the embodiment and related embodiments above.

[0273] The data processing apparatus according to the embodiment of the present disclosure can be realized by software, hardware or combination of the two to become part or all of an electronic device. The data processing apparatus includes:

[0274] The second obtaining module is configured to obtain at least two groups of device parameters of an image acquisition device in at least two camera calibration modes.

[0275] The second determining module is configured to determine initial device parameters from the at least two groups of device parameters.

[0276] The second optimization module is configured to optimize the initial device parameters to obtain target device parameters.

[0277] In an optional implementation of the embodiment, the second determining module includes:

[0278] The fourth obtaining sub-module is configured to obtain a re-projection error under the device parameters.

[0279] The fourth determining sub-module is configured to determine the initial device parameters according to the size of the re-projection error.

[0280] In an optional implementation of the embodiment, the fourth obtaining sub-module includes:

[0281] The second correction sub-module is configured to correct distortion of matched features between an image and a three-dimensional model by using the device parameters, wherein the image is an image collected by an image acquisition device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment.

[0282] The fifth obtaining sub-module is configured to obtain the re-projection error of the matched feature after distortion correction under the device parameters.

[0283] In an optional implementation of the embodiment, the three-dimensional model is a three-dimensional model in a world geodetic coordinate system.

[0284] In an optional implementation of the embodiment, the at least two matching features include matching features between a point on the image and a point on the three-dimensional model, and / or matching features between a line on the image and a line on the three-dimensional model.

[0285] The data processing apparatus in the embodiment and the data processing method in the embodiment correspond to each other, and specific details can be referred to the description of the data processing method in the embodiment and the related embodiments above. Figure 3 The data processing method in the embodiment and the related embodiments corresponds to the data processing apparatus in the embodiment, and specific details can be referred to the description of the data processing apparatus in the embodiment and the related embodiments above. Figure 3 The data processing method in the embodiment and the related embodiments corresponds to the data processing apparatus in the embodiment, and specific details can be referred to the description of the data processing apparatus in the embodiment and the related embodiments above.

[0286] The data processing apparatus according to an embodiment of the present disclosure can be realized as part or all of an electronic device through software, hardware or a combination of both. The data processing apparatus includes:

[0287] The third obtaining module is configured to obtain input data, wherein the input data includes an image collected by an image collection device from a collection environment and a three-dimensional model in the collection environment.

[0288] The first calling module is configured to call a preset service interface, so as to obtain, by the preset service interface, at least two matching features between the image and the three-dimensional model, determine an initial device parameter of the image collection device based on the at least two matching features, and optimize the initial device parameter based on the two matching features to obtain a target device parameter.

[0289] The first output module is configured to output the target device parameter.

[0290] The data processing apparatus in the embodiment and the data processing method in the embodiment correspond to each other, and specific details can be referred to the description of the data processing method in the embodiment and the related embodiments above. Figure 5 The data processing method in the embodiment and the related embodiments corresponds to the data processing apparatus in the embodiment, and specific details can be referred to the description of the data processing apparatus in the embodiment and the related embodiments above. Figure 5 The data processing method in the embodiment and the related embodiments corresponds to the data processing apparatus in the embodiment, and specific details can be referred to the description of the data processing apparatus in the embodiment and the related embodiments above.

[0291] The data processing apparatus according to an embodiment of the present disclosure can be realized as part or all of an electronic device through software, hardware or a combination of both. The data processing apparatus includes:

[0292] The fourth obtaining module is configured to obtain input data, wherein the input data includes at least two camera calibration modes.

[0293] The second calling module is configured to call a preset service interface, so as to obtain, by the preset service interface, at least two groups of device parameters of an image collection device in at least two camera calibration modes, determine an initial device parameter from the at least two groups of device parameters, and optimize the initial device parameter to obtain a target device parameter.

[0294] The second output module is configured to output the target device parameters.

[0295] The data processing device in this embodiment and Figure 6 The data processing methods in the illustrated embodiments and related embodiments are consistent; for details, please refer to the above description. Figure 6 The descriptions of the data processing methods in the illustrated embodiments and related embodiments are not repeated here.

[0296] According to an embodiment of the present disclosure, the data processing apparatus can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The data processing apparatus includes:

[0297] The receiving module is configured to receive a camera calibration request; wherein the camera calibration request includes configuration parameters for at least two camera calibration modes and at least two matching features;

[0298] The fifth acquisition module is configured to acquire at least two matching features between an image and a 3D model under the at least two camera calibration modes; the image is an image acquired by an image acquisition device from an acquisition environment, and the 3D model is a 3D model under the acquisition environment;

[0299] The third determining module is configured to determine at least two sets of device parameters under the at least two camera calibration modes based on the at least two matching features;

[0300] The fourth determining module is configured to determine the initial device parameters of the image acquisition device from the at least two sets of device parameters;

[0301] The third optimization module is configured to optimize the initial device parameters based on the two matching features to obtain the target device parameters.

[0302] The data processing device in this embodiment and Figure 7 The data processing methods in the illustrated embodiments and related embodiments are consistent; for details, please refer to the above description. Figure 7 The descriptions of the data processing methods in the illustrated embodiments and related embodiments are not repeated here.

[0303] Figure 8 This is a schematic diagram of the structure of an electronic device suitable for implementing the data processing method according to embodiments of the present disclosure.

[0304] like Figure 8As shown, the electronic device 800 includes a processing unit 801, which can be implemented as a CPU, a GPU, a FPGA, a NPU, or the like. The processing unit 801 can perform various processes in the embodiments of any of the methods of the present disclosure according to a program stored in a read only memory (ROM) 802 or a program loaded into a random access memory (RAM) 803 from the storage section 808. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0305] Connected to the I / O interface 805 are an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a display device such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a loudspeaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as necessary. A removable recording medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 810 as necessary, so that a computer program read therefrom is installed into the storage section 808 as necessary.

[0306] In particular, according to embodiments of the present disclosure, the above-mentioned reference to any of the methods in the embodiments of the present disclosure can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a non-transitory computer readable medium, the computer program comprising program code for executing any of the methods of the embodiments of the present disclosure. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from the removable recording medium 811.

[0307] The flow and block diagrams in the drawings represent possible architectural, functional, and operational architectures of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0308] The units or modules described in the embodiments of the present disclosure can be implemented by software, or by hardware. The described units or modules can also be arranged in a processor, and the name of the unit or module does not constitute a limitation on the unit or module itself in some cases.

[0309] As another aspect, the present disclosure also provides a computer readable storage medium, which can be the computer readable storage medium included in the apparatus described in the above embodiments, or can exist separately from the apparatus and not be assembled into the apparatus. The computer readable storage medium stores one or more programs for execution by one or more processors to perform the methods described in the present disclosure.

[0310] The above description is merely preferred embodiments of the present disclosure and a description of principles of applied technologies. It should be understood by those skilled in the art that the scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also includes other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the inventive concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions.

Claims

1. A data processing method, wherein, The method comprises: obtaining at least two matching features between an image and a three-dimensional model, wherein the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model of the collection environment; determining initial device parameters of the image collection device in at least two camera calibration modes based on the at least two matching features, wherein the at least two camera calibration modes comprise a camera calibration mode suitable for a planar scene and a camera calibration mode suitable for a non-planar scene; optimizing the initial device parameters based on the at least two matching features to obtain target device parameters.

2. The method of claim 1, wherein, The three-dimensional model is a three-dimensional model in a world geodetic coordinate system.

3. The method of claim 1 or 2, wherein, The at least two matching features comprise matching features between points on the image and points on the three-dimensional model, and / or matching features between lines on the image and lines on the three-dimensional model.

4. The method of claim 1 or 2, wherein, Determining the initial device parameters of the image collection device in at least two camera calibration modes based on the at least two matching features comprises: obtaining at least two groups of device parameters in the at least two camera calibration modes according to the at least two matching features; determining the initial device parameters from the at least two groups of device parameters, comprising: selecting one of each group of device parameters as the initial device parameters according to a re-projection error.

5. The method of claim 4, wherein, Determining the initial device parameters from the at least two groups of device parameters comprises: obtaining the re-projection error corresponding to each matching feature according to each group of device parameters; for the same camera calibration mode, determining a total re-projection error corresponding to the camera calibration mode according to the re-projection error corresponding to each matching feature; determining the initial device parameters according to the size of the total re-projection error.

6. The method of claim 5, wherein, Obtaining the re-projection error corresponding to each matching feature according to each group of device parameters comprises: correcting the distortion of the matching feature by using the device parameters; and obtaining the re-projection error corresponding to the matching feature after distortion correction.

7. A data processing method, wherein, The method comprises: obtaining at least two groups of device parameters of an image collection device in at least two camera calibration modes; determining initial device parameters in the at least two camera calibration modes from the at least two groups of device parameters, comprising: selecting one of each group of device parameters as the initial device parameters according to a re-projection error, wherein the at least two camera calibration modes comprise a camera calibration mode suitable for a planar scene and a camera calibration mode suitable for a non-planar scene; optimizing the initial device parameters to obtain target device parameters.

8. The method of claim 7, wherein, Determining the initial device parameters in the at least two camera calibration modes from the at least two groups of device parameters comprises: obtaining a re-projection error under the device parameters; determining the initial device parameters according to the size of the re-projection error.

9. The method of claim 8, wherein, Obtaining the re-projection error under the device parameters comprises: correcting the distortion of matching features between an image and a three-dimensional model by using the device parameters, wherein the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model of the collection environment; acquire the re-projection error of the matched feature after distortion correction under the device parameter.

10. The method of claim 9, wherein, The three-dimensional model is a three-dimensional model in a world geodetic coordinate system.

11. The method of claim 9 or 10, wherein, The matched feature includes a matched feature between a point on the image and a point on the three-dimensional model, and / or a matched feature between a line on the image and a line on the three-dimensional model.

12. A data processing method, wherein, Comprising: acquiring input data; the input data includes images collected by an image collection device from a collection environment and a three-dimensional model in the collection environment; calling a preset service interface so as to acquire at least two matched features between the image and the three-dimensional model by the preset service interface, and determine initial device parameters of the image collection device under at least two camera calibration modes based on the at least two matched features, wherein the at least two camera calibration modes include a camera calibration mode suitable for a planar scene and a camera calibration mode suitable for a non-planar scene, and the initial device parameters are optimized based on the at least two matched features to obtain target device parameters; outputting the target device parameters.

13. A data processing method, wherein, Comprising: acquiring input data; wherein the input data includes at least two camera calibration modes, the at least two camera calibration modes including a camera calibration mode suitable for a planar scene and a camera calibration mode suitable for a non-planar scene; calling a preset service interface so as to acquire at least two sets of device parameters of an image collection device under at least two camera calibration modes by the preset service interface, and determine initial device parameters from the at least two sets of device parameters and optimize the initial device parameters to obtain target device parameters; wherein determining the initial device parameters from the at least two sets of device parameters includes selecting one from each set of device parameters as the initial device parameters according to re-projection error; outputting the target device parameters.

14. A data processing method, wherein, Comprising: receiving a camera calibration request; wherein the camera calibration request includes at least two camera calibration modes and configuration parameters of at least two matched features; acquiring the at least two matched features between the image and the three-dimensional model under the at least two camera calibration modes; the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment; determining at least two sets of device parameters under the at least two camera calibration modes based on the at least two matched features, wherein the at least two camera calibration modes include a camera calibration mode suitable for a planar scene and a camera calibration mode suitable for a non-planar scene; determining initial device parameters of the image collection device from the at least two sets of device parameters includes selecting one from each set of device parameters as the initial device parameters according to re-projection error; optimizing the initial device parameters based on the at least two matched features to obtain target device parameters.

15. A data processing apparatus, wherein, Comprising: The first obtaining module is configured to obtain at least two matching features between an image and a three-dimensional model; the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model of the collection environment; The first determining module is configured to determine initial device parameters of the image collection device in at least two camera calibration modes based on the at least two matching features, wherein the at least two camera calibration modes include a camera calibration mode suitable for a planar scene and a camera calibration mode suitable for a non-planar scene; The first optimization module is configured to optimize the initial device parameters based on the at least two matching features to obtain target device parameters.

16. A data processing apparatus, wherein, Comprising: The second obtaining module is configured to obtain at least two groups of device parameters of an image collection device in at least two camera calibration modes, wherein the at least two camera calibration modes include a camera calibration mode suitable for a planar scene and a camera calibration mode suitable for a non-planar scene; The second determining module is configured to determine initial device parameters from the at least two groups of device parameters, including selecting one from each group of device parameters as the initial device parameters according to a reprojection error; The second optimization module is configured to optimize the initial device parameters to obtain target device parameters.

17. A data processing apparatus, wherein, Comprising: The third obtaining module is configured to obtain input data; The input data includes an image collected by an image collection device from a collection environment and a three-dimensional model of the collection environment; The first calling module is configured to call a preset service interface, so as to obtain, by the preset service interface, at least two matching features between an image and a three-dimensional model, and determine initial device parameters of the image collection device in at least two camera calibration modes based on the at least two matching features, wherein the at least two camera calibration modes include a camera calibration mode suitable for a planar scene and a camera calibration mode suitable for a non-planar scene, and optimize the initial device parameters based on the at least two matching features to obtain target device parameters; The first output module is configured to output the target device parameters.

18. A data processing apparatus, wherein, Comprising: The fourth obtaining module is configured to obtain input data; wherein the input data includes at least two camera calibration modes, and the at least two camera calibration modes include a camera calibration mode suitable for a planar scene and a camera calibration mode suitable for a non-planar scene; The second calling module is configured to call a preset service interface, so as to obtain, by the preset service interface, at least two groups of device parameters of an image collection device in at least two camera calibration modes, determine initial device parameters from the at least two groups of device parameters, and optimize the initial device parameters to obtain target device parameters; wherein determining the initial device parameters from the at least two groups of device parameters includes selecting one from each group of device parameters as the initial device parameters according to a reprojection error; The second output module is configured to output the target device parameters.

19. A data processing apparatus, wherein, Comprising: The receiving module is configured to receive a camera calibration request; wherein the camera calibration request comprises at least two camera calibration modes and configuration parameters of at least two matching features, the at least two camera calibration modes comprising: a camera calibration mode suitable for a planar scene, and a camera calibration mode suitable for a non-planar scene; The fifth obtaining module is configured to obtain the at least two matching features between an image and a three-dimensional model in the at least two camera calibration modes; the image is an image collected by an image collection device from a collection environment, and the three-dimensional model is a three-dimensional model in the collection environment; The third determining module is configured to determine at least two groups of device parameters in the at least two camera calibration modes based on the at least two matching features; The fourth determining module is configured to determine initial device parameters of the image collection device from the at least two groups of device parameters, comprising: selecting one from each group of device parameters as the initial device parameters according to a re-projection error; The third optimization module is configured to optimize the initial device parameters based on the at least two matching features to obtain target device parameters.

20. An electronic device, comprising: comprise a memory and a processor; wherein The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any one of claims 1-14.

21. A computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are executed by the processor to implement the method of any one of claims 1-14.

Images