Camera model data processing method and device based on two-layer perceptron, equipment and medium

Abstract

The invention discloses a camera model data processing method and device based on a two-layer perceptron, equipment and a medium. The method comprises the following steps: acquiring three-dimensional data of a shooting scene; inputting the three-dimensional data into a pinhole camera model to obtain an image result corresponding to the shooting scene; wherein the pinhole camera model converts the three-dimensional data into camera normalized data, and through a distortion conversion model, the camera normalized data is converted into an image result corresponding to the shooting scene; training learning parameters of a two-layer perceptron model by taking the distortion conversion model as output and taking feature point pairs of camera normalized calibration data and image calibration data of a calibration scene as input; the learning parameters of the two-layer perceptron are external parameters, internal parameters and distortion coefficients of the pinhole camera model. According to the invention, the precision of camera parameters is improved, and image shooting errors caused by shooting are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a camera model data processing method, device, equipment and medium based on a two-layer perceptron. Background Art

[0002] The theories of image 3D reconstruction technology and photogrammetry technology are based on camera models. The main problem in the use of camera models is how to obtain accurate camera model parameters. The most widely used camera model in engineering practice is the pinhole camera model. The pinhole camera model parameter acquisition method is to make a calibration plate and take pictures to obtain the image data of the calibration plate, then extract the corner points of the calibration plate in the image by image processing, and finally obtain the camera parameters and distortion coefficients.

[0003] In this way, the pinhole camera model considers the linear part and the nonlinear part separately. The linear transformation part mainly models the collinear equations of the projection process, that is, it mainly realizes the transformation from the world three-dimensional coordinate system to the camera normalized coordinate system; the nonlinear transformation part mainly models the radial distortion and tangential distortion parts of the projection process, that is, it realizes the transformation from the camera normalized coordinate system to the image pixel coordinate system. The linear changes commonly used in the pinhole camera model are accurate, but the nonlinear changes of the pinhole camera model are modeled using binary high-order polynomials. The transformation relationship between the camera normalized coordinates and the image pixel coordinates cannot be obtained because the inverse function of this binary high-order polynomial cannot be solved. Therefore, the nonlinear transformation model cannot give the transformation from the distorted coordinates to the non-distorted coordinates, and cannot realize the transformation of normalized data into image data, which leads to low accuracy of the camera parameters of the pinhole camera model and large image shooting errors.

[0004] Therefore, there is an urgent need for a camera model data processing strategy to solve the problem of large image capture errors caused by the low accuracy of camera parameters of the pinhole camera model. Summary of the invention

[0005] The embodiments of the present invention provide a camera model data processing method, apparatus, device and medium based on a two-layer perceptron to solve the problem of large image shooting errors caused by low camera parameter accuracy of a pinhole camera model.

[0006] In order to solve the above problem, an embodiment of the present invention provides a camera model data processing method based on a two-layer perceptron, comprising:

[0007] Acquire three-dimensional data of the shooting scene;

[0008] The three-dimensional data is input into a pinhole camera model to obtain an image result corresponding to the shooting scene; wherein the pinhole camera model converts the three-dimensional data into camera normalized data, and converts the camera normalized data into an image result corresponding to the shooting scene through a distortion transformation model; with the distortion transformation model as output and feature point pairs of camera normalized calibration data and image calibration data of a calibration scene as input, the learning parameters of a two-layer perceptron model are trained; the learning parameters of the two-layer perceptron are external parameters, internal parameters and distortion coefficients of the pinhole camera model.

[0009] As an improvement of the above solution, the pinhole camera model converts the three-dimensional data into camera normalized data, including:

[0010] The pinhole camera model substitutes the three-dimensional data into the camera three-dimensional coordinate system transformation formula to obtain the camera three-dimensional data; wherein the camera three-dimensional coordinate system transformation formula is specifically:

[0011]

[0012] In the formula, X w , Y w and Z w are the values ​​of the x-axis, y-axis, and z-axis of the three-dimensional data respectively; c , Y c and Z c are the values ​​of the x-axis, y-axis, and z-axis of the camera's three-dimensional data; R is a 3x3 rotation matrix; T is a 3x1 translation vector; O T is a 3x1 zero vector;

[0013] The pinhole camera model substitutes the camera 3D data into the camera normalized coordinate system transformation formula to obtain camera normalized data; wherein the camera normalized coordinate system transformation formula is specifically:

[0014]

[0015] Where x, y and z are the values ​​of the x-axis, y-axis and z-axis of the camera normalized data; z c is the value of the z-axis of the camera's three-dimensional data corresponding to the camera coordinate system, and f is the camera's focal length.

[0016] As an improvement of the above solution, the training of the distortion conversion model includes:

[0017] Acquire feature point pairs between a number of camera normalized calibration data and image calibration data of the calibration scene;

[0018] According to each pair of feature points, the model training operation is repeatedly performed on the two-layer perceptron model until the loss function of the two-layer perceptron model converges, thereby obtaining a distortion conversion model;

[0019] The model training operation is specifically as follows:

[0020] Selecting an untrained feature point pair from a plurality of feature point pairs as a target feature point pair; wherein the target feature point pair includes: target camera normalization calibration data and target image calibration data;

[0021] Input the target camera normalized calibration data into the two-layer perceptron model to obtain the image projection data corresponding to the target camera normalized calibration data;

[0022] Substitute the image projection data and the target image calibration data into the loss function to calculate the loss value;

[0023] Based on the loss value, the learning parameters of the two-layer perceptron model are optimized by a preset gradient descent optimization algorithm, and the optimized learning parameters are updated as the learning parameters of the two-layer perceptron model in the next model training operation.

[0024] As an improvement of the above solution, the distortion conversion model satisfies the following conditions:

[0025]

[0026] Where w1 and b1 represent the transformation parameters and bias of the first linear layer of the distortion transformation model, respectively; w2 and b2 represent the transformation parameters and bias of the second linear layer of the distortion transformation model, respectively; x is the value of the x-axis of the camera normalized data, and y is the value of the y-axis of the camera normalized data; x distorted is the value of the x-axis of the camera normalized data after distortion, distorted is the value of the y-axis of the camera normalized data after distortion, _u is the value of the x-axis of the image result, _v is the value of the y-axis of the image result, c x is the x coordinate of the center point of the image coordinate system, c y is the y coordinate of the center point of the image coordinate system.

[0027] As an improvement of the above solution, the loss function satisfies the following conditions:

[0028]

[0029] In the formula, loss is the loss value, _u i is the value of the x-axis of the image projection data, _v i is the value of the y-axis of the image projection data, u i Calibrate the x-axis value of the target image data, v i is the value of the y-axis of the target image calibration data, and N is the number of observed data points.

[0030] As an improvement of the above solution, the step of obtaining feature point pairs between a plurality of camera normalized calibration data and image calibration data of the calibration scene includes:

[0031] Acquire three-dimensional point cloud data and image point cloud data of a calibration scene; wherein the image point cloud data is acquired in the calibration scene;

[0032] Converting the three-dimensional point cloud data into camera normalized calibration data;

[0033] Based on the SIFT image feature point extraction algorithm, feature points are extracted from the image point cloud data to obtain image calibration data;

[0034] Summarize each pair of camera normalization calibration data and image calibration data with the same name to obtain several feature point pairs.

[0035] As an improvement of the above solution, the acquisition of the three-dimensional point cloud data is specifically as follows:

[0036] Acquire a number of laser point cloud data generated during the inspection process of the power transmission line in the calibration scene;

[0037] Calculate the first three-dimensional point cloud data corresponding to each of the laser point cloud data based on the point cloud volume density calculation formula;

[0038] Determine a minimum point cloud x-coordinate value, a minimum point cloud y-coordinate value, and a minimum point cloud z-coordinate value in the plurality of first three-dimensional point cloud data;

[0039] Preprocess each of the first three-dimensional point cloud data to obtain second three-dimensional point cloud data, and use the second three-dimensional point cloud data as three-dimensional point cloud data; wherein the preprocessing is specifically: subtracting the minimum point cloud x coordinate value from the x coordinate axis of the first three-dimensional point cloud data, subtracting the minimum point cloud y coordinate value from the y coordinate axis of the first three-dimensional point cloud data, and subtracting the minimum point cloud z coordinate value from the z coordinate axis of the first three-dimensional point cloud data.

[0040] Correspondingly, an embodiment of the present invention further provides a camera model data processing device based on a two-layer perceptron, comprising: a data acquisition module and a result generation module;

[0041] The data acquisition module is used to acquire three-dimensional data of the shooting scene;

[0042] The result generation module is used to input the three-dimensional data into the pinhole camera model to obtain the image result corresponding to the shooting scene; wherein the pinhole camera model converts the three-dimensional data into camera normalized data, and converts the camera normalized data into the image result corresponding to the shooting scene through a distortion transformation model; with the distortion transformation model as output and the feature point pairs of the camera normalized calibration data and the image calibration data of the calibration scene as input, the learning parameters of the two-layer perceptron model are trained; the learning parameters of the two-layer perceptron are the external parameters, internal parameters and distortion coefficients of the pinhole camera model.

[0043] Correspondingly, an embodiment of the present invention also provides a computer terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements a camera model data processing method based on a two-layer perceptron as described in the present invention.

[0044] Correspondingly, an embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a camera model data processing method based on a two-layer perceptron as described in the present invention.

[0045] As can be seen from the above, the present invention has the following beneficial effects:

[0046] The present invention provides a camera model data processing method based on a two-layer perceptron, which obtains three-dimensional data of a shooting scene; inputs the three-dimensional data into a pinhole camera model to obtain an image result corresponding to the shooting scene; wherein the pinhole camera model converts the three-dimensional data into camera normalized data, and converts the camera normalized data into an image result corresponding to the shooting scene through a distortion transformation model; takes the distortion transformation model as output, takes feature point pairs of camera normalized calibration data and image calibration data of a calibration scene as input, and trains the learning parameters of a two-layer perceptron model; the learning parameters of the two-layer perceptron are external parameters, internal parameters and distortion coefficients of the pinhole camera model. After acquiring three-dimensional data, the present invention first converts the three-dimensional data into camera normalized data, and then converts the camera normalized data into image results through a two-layer perceptron model. The nonlinear problem of converting the camera normalized data into image results is optimized based on deep learning of the two-layer perceptron model, and the problem that binary high-order polynomials cannot convert camera normalized data into image results is solved, so that the camera parameters of the pinhole camera model are adjusted based on the converted results, the accuracy of the camera parameters is improved, and the image shooting errors caused by shooting are avoided.

[0047] Furthermore, the present invention uses a two-layer perceptron to replace the binary high-order polynomial to model the nonlinear transformation part of the pinhole camera model. The pinhole camera model is used to perform a projection transformation on the three-dimensional data corresponding to the world coordinate point, and the image projection data of the corresponding point is obtained by subtracting the image calibration data actually observed to construct the minimum mean square error problem, and then the camera internal parameters, external parameters and distortion coefficients are solved by the gradient descent method, which greatly improves the accuracy of the camera parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flowchart of a camera model data processing method based on a two-layer perceptron provided by an embodiment of the present invention;

[0049] Figure 2 It is a structural schematic diagram of a camera model data processing device based on a two-layer perceptron provided in one embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the structure of a terminal device provided by an embodiment of the present invention;

[0051] Figure 4 This is a comparison diagram of three-dimensional point projection errors provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Embodiment 1

[0054] See also Figure 1 , Figure 1 is a flowchart of a camera model data processing method based on a two-layer perceptron provided by an embodiment of the present invention, such as Figure 1 As shown, this embodiment includes step 101 to step 102, and each step is specifically as follows:

[0055] Step 101: Acquire three-dimensional data of a shooting scene.

[0056] In this embodiment, a scene is photographed by a camera to obtain three-dimensional data.

[0057] Step 102: input the three-dimensional data into a pinhole camera model to obtain an image result corresponding to the shooting scene; wherein the pinhole camera model converts the three-dimensional data into camera normalized data, and converts the camera normalized data into an image result corresponding to the shooting scene through a distortion transformation model; with the distortion transformation model as output and the feature point pairs of the camera normalized calibration data and the image calibration data of the calibration scene as input, the learning parameters of the two-layer perceptron model are trained; the learning parameters of the two-layer perceptron are the external parameters, internal parameters and distortion coefficients of the pinhole camera model.

[0058] In a specific embodiment, the processing flow of the camera model is: converting the world three-dimensional coordinates (i.e., the three-dimensional data described in the present invention) into camera three-dimensional coordinates (i.e., the camera three-dimensional data described in the present invention), converting the camera three-dimensional coordinates into camera normalized coordinates (i.e., the camera normalized data described in the present invention), converting the camera normalized coordinates into distorted camera normalized coordinates (i.e., the distorted camera normalized data described in the present invention), and converting the distorted camera normalized coordinates into image pixel coordinates (i.e., the image result described in the present invention).

[0059] In this embodiment, the pinhole camera model converts the three-dimensional data into camera normalized data, including:

[0060] The pinhole camera model substitutes the three-dimensional data into the camera three-dimensional coordinate system transformation formula to obtain the camera three-dimensional data; wherein the camera three-dimensional coordinate system transformation formula is specifically:

[0061]

[0062] In the formula, X w , Y w and Z w are the values ​​of the x-axis, y-axis, and z-axis of the three-dimensional data respectively; c , Y c and Z c are the values ​​of the x-axis, y-axis, and z-axis of the camera's three-dimensional data; R is a 3x3 rotation matrix; T is a 3x1 translation vector; O T is a 3x1 zero vector;

[0063] The pinhole camera model substitutes the camera 3D data into the camera normalized coordinate system transformation formula to obtain camera normalized data; wherein the camera normalized coordinate system transformation formula is specifically:

[0064]

[0065] Where x, y and z are the values ​​of the x-axis, y-axis and z-axis of the camera normalized data; z cis the value of the z-axis of the camera's three-dimensional data corresponding to the camera coordinate system, and f is the camera's focal length.

[0066] In this embodiment, the training of the distortion conversion model includes:

[0067] Acquire feature point pairs between a number of camera normalized calibration data and image calibration data of the calibration scene;

[0068] According to each pair of feature points, the model training operation is repeatedly performed on the two-layer perceptron model until the loss function of the two-layer perceptron model converges, thereby obtaining a distortion conversion model;

[0069] The model training operation is specifically as follows:

[0070] Selecting an untrained feature point pair from a plurality of feature point pairs as a target feature point pair; wherein the target feature point pair includes: target camera normalization calibration data and target image calibration data;

[0071] Input the target camera normalized calibration data into the two-layer perceptron model to obtain the image projection data corresponding to the target camera normalized calibration data;

[0072] Substitute the image projection data and the target image calibration data into the loss function to calculate the loss value;

[0073] Based on the loss value, the learning parameters of the two-layer perceptron model are optimized by a preset gradient descent optimization algorithm, and the optimized learning parameters are updated as the learning parameters of the two-layer perceptron model in the next model training operation.

[0074] In this embodiment, the distortion conversion model satisfies the following conditions:

[0075]

[0076] Where w1 and b1 represent the transformation parameters and bias of the first linear layer of the distortion transformation model, respectively; w2 and b2 represent the transformation parameters and bias of the second linear layer of the distortion transformation model, respectively; x is the value of the x-axis of the camera normalized data, and y is the value of the y-axis of the camera normalized data; x distorted is the value of the x-axis of the camera normalized data after distortion, distorted is the value of the y-axis of the camera normalized data after distortion, _u is the value of the x-axis of the image result, _v is the value of the y-axis of the image result, c x is the x coordinate of the center point of the image coordinate system, c y is the y coordinate of the center point of the image coordinate system.

[0077] In this embodiment, the loss function satisfies the following conditions:

[0078]

[0079] In the formula, loss is the loss value, _u i is the value of the x-axis of the image projection data, _v i is the value of the y-axis of the image projection data, u i Calibrate the x-axis value of the target image data, v i is the value of the y-axis of the target image calibration data, and N is the number of observed data points.

[0080] In a specific implementation, the two-layer perceptron model includes two linear layers plus a layer with a sigmoid activation function to model radial distortion and tangential distortion. The neural network structure and parameters for modeling the distortion part are shown in Table 3.

[0081] Table 3

[0082]

[0083] It is understandable that in computer vision, the PnP method is often used to estimate the camera's projection matrix. In the PnP solution process, only the linear transformation part of the camera model can be processed, and the nonlinear distortion part of the camera cannot be modeled. The usual processing method is to assume that the camera's intrinsic parameter matrix is ​​known, and use the camera's intrinsic parameter matrix parameters to convert the image pixel coordinates to the camera's normalized coordinates. Finally, the PnP method is used to solve the projection matrix from the world coordinates to the camera's normalized coordinates (the matrix is ​​a linear transformation). From the actual camera calibration work, the camera's intrinsic parameters will change due to the distance between the calibration plate and the camera. Therefore, if a fixed camera intrinsic parameter matrix parameter is used to transform the image pixel coordinates into the camera's normalized coordinates, it will inevitably introduce new system errors in the photogrammetry system, resulting in larger errors in the measurement results. This embodiment starts from the perspective of photogrammetric adjustment theory, comprehensively considers the linear transformation and nonlinear transformation in the camera projection model, uses two linear layers plus a sigmoid activation function to describe the nonlinear distortion of the camera, and uniformly considers the radial distortion and tangential distortion factors of the camera. With the goal of minimizing the camera projection error, the external parameters, internal parameters and distortion parameters in the camera model are uniformly optimized to achieve the adjustment between the parameters of each part in the camera projection process.

[0084] In a specific embodiment, the two-layer perceptron model simultaneously completes the modeling of the camera's external parameters (6), distortion coefficients (17), and internal parameters (4), and the model contains a total of 27 parameters. Assuming that the real three-dimensional coordinates (world coordinates) of the spatial point A are (X, Y, Z), the corresponding real observed image pixel coordinates are (u, v), and the projection coordinates of the camera model are (_u, _v), the difference between the real observed value and the projection value can be calculated, and the square of the error in the u and v directions is calculated and then summed to obtain the loss function. Obviously, this is a nonlinear optimization problem, and the optimization variables are the 27 parameters of the camera model. After constructing the loss function, the nonlinear least squares optimization tool ceres (i.e., the gradient descent optimization algorithm described in the present invention) is used for optimization.

[0085] In this embodiment, the step of obtaining feature point pairs between a plurality of camera normalized calibration data and image calibration data of the calibration scene includes:

[0086] Acquire three-dimensional point cloud data and image point cloud data of a calibration scene; wherein the image point cloud data is acquired in the calibration scene;

[0087] Converting the three-dimensional point cloud data into camera normalized calibration data;

[0088] Based on the SIFT image feature point extraction algorithm, feature points are extracted from the image point cloud data to obtain image calibration data;

[0089] Summarize each pair of camera normalization calibration data and image calibration data with the same name to obtain several feature point pairs.

[0090] In a specific embodiment, in order to determine the parameters of the pinhole camera model, it is necessary to select corresponding points of the same name from the image point cloud data taken by the calibration scene camera. In this patent, the sift image feature point extraction algorithm is used to extract features from the image point cloud data to obtain a feature point set S (i.e., the image calibration data described in the present invention), and finally, the points of the same name corresponding to the three-dimensional point cloud in Section 1.1 are selected from the image two-dimensional feature point set S. In this way, we obtain a set of points of the same name from the three-dimensional world coordinate points and the two-dimensional image pixel coordinate points, as shown in Table 2. (Selecting three-dimensional point cloud and two-dimensional image point pairs through manual observation)

[0091] Table 2

[0092]

[0093] In this embodiment, the acquisition of the three-dimensional point cloud data is specifically as follows:

[0094] Acquire a number of laser point cloud data generated during the inspection process of the power transmission line in the calibration scene;

[0095] Calculate the first three-dimensional point cloud data corresponding to each of the laser point cloud data based on the point cloud volume density calculation formula;

[0096] Determine a minimum point cloud x-coordinate value, a minimum point cloud y-coordinate value, and a minimum point cloud z-coordinate value in the plurality of first three-dimensional point cloud data;

[0097] Preprocess each of the first three-dimensional point cloud data to obtain second three-dimensional point cloud data, and use the second three-dimensional point cloud data as three-dimensional point cloud data; wherein the preprocessing is specifically: subtracting the minimum point cloud x coordinate value from the x coordinate axis of the first three-dimensional point cloud data, subtracting the minimum point cloud y coordinate value from the y coordinate axis of the first three-dimensional point cloud data, and subtracting the minimum point cloud z coordinate value from the z coordinate axis of the first three-dimensional point cloud data.

[0098] In a specific embodiment, the laser point cloud data generated during the inspection of the transmission line is used as the data source. Since the laser point cloud data of the transmission channel is relatively sparse, the feature points are discretely sampled, and the distances between points are not equal, it is difficult to accurately click on the coordinates of the ideal point when selecting the feature points. Based on the concept of point cloud volume density ρ, for example: search for the K points closest to A near the specified three-dimensional point A (x, y, z), in the present invention K = 9, so that the circumscribed cube surrounding the K points can be obtained, and then the length, width and height of the cube are calculated, and the point cloud density of A in the x, y, and z directions is calculated respectively. The calculation method is as follows:

[0099]

[0100] In the formula, l x is the length of the circumscribed cube in the x direction, l y is the length of the circumscribed cube in the y direction, l z is the length of the circumscribed cube in the z direction, ρ x is the density of the circumscribed cube in the x direction, ρ y is the density of the circumscribed cube in the y direction, ρ z is the density of the circumscribed cube in the z direction.

[0101] After obtaining the point cloud density of the point cloud data near point A, the following formula is used to calculate the coordinates of the corresponding point of A in the point cloud data.

[0102]

[0103] In the formula, x, y, z, xminmin min It is the coordinate of point A in the point cloud annotation software.

[0104] In a specific implementation, the data values ​​of the laser point cloud data are quite different. The data comes from a 110kV transmission line with a span of about 150m. This phenomenon will cause the error of the camera projection matrix to become larger or the solution to fail. Therefore, the data needs to be preprocessed before solving the camera matrix. After preprocessing, the magnitude difference between the data is reduced, which is conducive to the algorithm to solve the accurate parameters of the camera model, see Table 3.

[0105] Table 3

[0106]

[0107] In order to make the optimization algorithm converge quickly to the global optimal solution and prevent overfitting, only 27 parameters of the camera are modeled in this scheme, that is, 6 external parameters and 17 distortion parameters are randomly initialized using standard normal distribution, and the coordinates of the two principal points of the camera and the focal lengths in the x and y directions are initialized by the length and width of the photo taken by the camera. The optimization algorithm converged after 982 iterations. Table 4 shows the pixel error of the three-dimensional point projected through the camera model after the minimum mean square error optimization. The average error in the u and v directions is less than 2.4 pixels. Figure 3 It is the effect of projecting the three-dimensional point coordinates to the two-dimensional image pixel coordinates, where the black points are the real feature point coordinates and the white points are the projected coordinate points.

[0108] Table 4

[0109]

[0110] See also Figure 2 , Figure 2 It is a structural schematic diagram of a camera model data processing device based on a two-layer perceptron provided in an embodiment of the present invention, comprising: a data acquisition module 201 and a result generation module 202;

[0111] The data acquisition module 201 is used to acquire three-dimensional data of the shooting scene;

[0112] The result generation module 202 is used to input the three-dimensional data into the pinhole camera model to obtain the image result corresponding to the shooting scene; wherein the pinhole camera model converts the three-dimensional data into camera normalized data, and converts the camera normalized data into the image result corresponding to the shooting scene through a distortion transformation model; with the distortion transformation model as output and the feature point pairs of the camera normalized calibration data and the image calibration data of the calibration scene as input, the learning parameters of the two-layer perceptron model are trained; the learning parameters of the two-layer perceptron are the external parameters, internal parameters and distortion coefficients of the pinhole camera model.

[0113] It can be understood that the above-mentioned system item embodiment corresponds to the method item embodiment of the present invention, which can implement the camera model data processing method based on the two-layer perceptron provided by any one of the above-mentioned method item embodiments of the present invention.

[0114] Beneficial Effects

[0115] This embodiment obtains three-dimensional data of a shooting scene; inputs the three-dimensional data into a pinhole camera model to obtain an image result corresponding to the shooting scene; wherein the pinhole camera model converts the three-dimensional data into camera normalized data, and converts the camera normalized data into an image result corresponding to the shooting scene through a distortion transformation model; takes the distortion transformation model as output, and takes feature point pairs of camera normalized calibration data and image calibration data of a calibration scene as input, to train learning parameters of a two-layer perceptron model; the learning parameters of the two-layer perceptron are external parameters, internal parameters and distortion coefficients of the pinhole camera model. After acquiring three-dimensional data, the present invention first converts the three-dimensional data into camera normalized data, and then converts the camera normalized data into image results through a two-layer perceptron model. The nonlinear problem of converting the camera normalized data into image results is optimized based on deep learning of the two-layer perceptron model, and the problem that binary high-order polynomials cannot convert camera normalized data into image results is solved, so that the camera parameters of the pinhole camera model are adjusted based on the converted results, the accuracy of the camera parameters is improved, and the image shooting errors caused by shooting are avoided.

[0116] Embodiment 2

[0117] See also Figure 3 , Figure 3 It is a schematic diagram of the structure of a terminal device provided in one embodiment of the present invention.

[0118] A terminal device of this embodiment includes: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, the steps of the camera model data processing method based on the two-layer perceptron in the embodiment are implemented, for example: Figure 1 All steps of the camera model data processing method based on the two-layer perceptron shown. Alternatively, when the processor executes the computer program, the functions of each module in the above-mentioned device embodiments are realized, for example: Figure 2 All modules of the camera model data processing device based on the two-layer perceptron are shown.

[0119] In addition, an embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the camera model data processing method based on the two-layer perceptron as described in any of the above embodiments.

[0120] Those skilled in the art will understand that the schematic diagram is merely an example of a terminal device and does not constitute a limitation on the terminal device. The terminal device may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0121] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 301 is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0122] The memory 302 can be used to store the computer program and / or module. The processor 301 implements various functions of the terminal device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory 302. The memory 302 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0123] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0124] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0125] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A camera model data processing method based on a two-layer perceptron, characterized in that: include: Acquire three-dimensional data of the shooting scene; The three-dimensional data is input into a pinhole camera model to obtain an image result corresponding to the shooting scene; wherein the pinhole camera model converts the three-dimensional data into camera normalized data, and converts the camera normalized data into an image result corresponding to the shooting scene through a distortion transformation model; with the distortion transformation model as output and feature point pairs of camera normalized calibration data and image calibration data of a calibration scene as input, the learning parameters of a two-layer perceptron model are trained; the learning parameters of the two-layer perceptron are external parameters, internal parameters and distortion coefficients of the pinhole camera model.

2. The camera model data processing method based on a two-layer perceptron according to claim 1, characterized in that: The pinhole camera model converts the three-dimensional data into camera normalized data, including: The pinhole camera model substitutes the three-dimensional data into the camera three-dimensional coordinate system transformation formula to obtain the camera three-dimensional data; wherein the camera three-dimensional coordinate system transformation formula is specifically: Where, X w , Y w and Z w are the values ​​of the x-axis, y-axis, and z-axis of the three-dimensional data respectively; c , Y c and Z c are the values ​​of the x-axis, y-axis, and z-axis of the camera's 3D data corresponding to the world coordinate system; R is a 3x3 rotation matrix; T is a 3x1 translation vector; O T is a 3x1 zero vector; The pinhole camera model substitutes the camera 3D data into the camera normalized coordinate system transformation formula to obtain camera normalized data; wherein the camera normalized coordinate system transformation formula is specifically: Where x, y and 1 are the values ​​of the x-axis, y-axis and z-axis of the camera normalized data; z c is the value of the z-axis of the camera's three-dimensional data corresponding to the camera coordinate system, and f is the camera's focal length.

3. The camera model data processing method based on a two-layer perceptron according to claim 1, characterized in that: The training of the distortion conversion model includes: Acquire feature point pairs between a number of camera normalized calibration data and image calibration data of the calibration scene; According to each pair of feature points, the model training operation is repeatedly performed on the two-layer perceptron model until the loss function of the two-layer perceptron model converges, thereby obtaining a distortion conversion model; The model training operation is specifically as follows: Selecting an untrained feature point pair from a plurality of feature point pairs as a target feature point pair; wherein the target feature point pair includes: target camera normalization calibration data and target image calibration data; Input the target camera normalized calibration data into the two-layer perceptron model to obtain the image projection data corresponding to the target camera normalized calibration data; Substitute the image projection data and the target image calibration data into the loss function to calculate the loss value; Based on the loss value, the learning parameters of the two-layer perceptron model are optimized by a preset gradient descent optimization algorithm, and the optimized learning parameters are updated as the learning parameters of the two-layer perceptron model in the next model training operation.

4. The camera model data processing method based on a two-layer perceptron according to claim 3 is characterized in that: The distortion conversion model satisfies the following conditions: Where w1 and b1 represent the transformation parameters and bias of the first linear layer of the distortion transformation model, respectively; w2 and b2 represent the transformation parameters and bias of the second linear layer of the distortion transformation model, respectively; x is the value of the x-axis of the camera normalized data, and y is the value of the y-axis of the camera normalized data; x distorted is the value of the x-axis of the camera normalized data after distortion, distorted is the value of the y-axis of the camera normalized data after distortion, _u is the value of the x-axis of the image result, _v is the value of the y-axis of the image result, c x is the x coordinate of the center point of the image coordinate system, c y is the y coordinate of the center point of the image coordinate system.

5. The camera model data processing method based on a two-layer perceptron according to claim 3 is characterized in that: The loss function satisfies the following conditions: In the formula, loss is the loss value, _u i is the value of the x-axis of the image projection data, _v i is the value of the y-axis of the image projection data, u i Calibrate the x-axis value of the target image data, v i is the value of the y-axis of the target image calibration data, and N is the number of observed data points.

6. The camera model data processing method based on a two-layer perceptron according to claim 3 is characterized in that: The step of obtaining feature point pairs between a plurality of camera normalized calibration data and image calibration data of the calibration scene includes: Acquire three-dimensional point cloud data and image point cloud data of a calibration scene; wherein the image point cloud data is acquired in the calibration scene; Converting the three-dimensional point cloud data into camera normalized calibration data; Based on the SIFT image feature point extraction algorithm, feature points are extracted from the image point cloud data to obtain image calibration data; Summarize each pair of camera normalization calibration data and image calibration data with the same name to obtain several feature point pairs.

7. The camera model data processing method based on a two-layer perceptron according to claim 6 is characterized in that: The acquisition of the three-dimensional point cloud data is specifically as follows: Acquire a number of laser point cloud data generated during the inspection process of the power transmission line in the calibration scene; Calculate the first three-dimensional point cloud data corresponding to each of the laser point cloud data based on the point cloud volume density calculation formula; Determine a minimum point cloud x-coordinate value, a minimum point cloud y-coordinate value, and a minimum point cloud z-coordinate value in the plurality of first three-dimensional point cloud data; Preprocess each of the first three-dimensional point cloud data to obtain second three-dimensional point cloud data, and use the second three-dimensional point cloud data as three-dimensional point cloud data; wherein the preprocessing is specifically: subtracting the minimum point cloud x coordinate value from the x coordinate axis of the first three-dimensional point cloud data, subtracting the minimum point cloud y coordinate value from the y coordinate axis of the first three-dimensional point cloud data, and subtracting the minimum point cloud z coordinate value from the z coordinate axis of the first three-dimensional point cloud data.

8. A camera model data processing device based on a two-layer perceptron, characterized in that: include: Data acquisition module and result generation module; The data acquisition module is used to acquire three-dimensional data of the shooting scene; The result generation module is used to input the three-dimensional data into the pinhole camera model to obtain the image result corresponding to the shooting scene; wherein the pinhole camera model converts the three-dimensional data into camera normalized data, and converts the camera normalized data into the image result corresponding to the shooting scene through a distortion transformation model; with the distortion transformation model as output and the feature point pairs of the camera normalized calibration data and the image calibration data of the calibration scene as input, the learning parameters of the two-layer perceptron model are trained; the learning parameters of the two-layer perceptron are the external parameters, internal parameters and distortion coefficients of the pinhole camera model.

9. A computer terminal device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a camera model data processing method based on a two-layer perceptron as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute a camera model data processing method based on a two-layer perceptron as described in any one of claims 1 to 7.

Images