Image conversion apparatus and method

Abstract

The embodiment of the invention provides an image conversion device and method, and the device comprises a first image processing device which carries out the first image processing of an original image according to the value of a first parameter of the original image, and obtains an intermediate image; the iteration device is used for iteratively executing first image processing on the intermediate image according to the value of the first parameter of the intermediate image to obtain a final intermediate image; the estimation device is used for estimating an estimation value of the first parameter according to a conversion relation between a distorted image and a corresponding perspective image and the original image and the final intermediate image; and the second image processing device is used for performing second image processing on the original image according to the estimated value of the first parameter to obtain a converted image.

Description

Technical Field

[0001] The present application relates to the field of image processing. Background Art

[0002] Computer vision technology has been increasingly widely used in all aspects of production and life, and camera calibration is an important part of computer vision technology. Camera calibration methods can be roughly divided into internal camera parameter calibration methods and external camera parameter calibration methods. Among them, the internal camera parameter calibration technology also includes the nonlinear distortion coefficient caused by the camera lens.

[0003] Traditional camera calibration methods require the use of calibration objects of known size. By establishing the correspondence between points on the calibration object with known coordinates and their image points, a certain algorithm is used to obtain the internal and external parameters of the camera model. Since calibration objects are always required during the calibration process, the manufacturing accuracy of the calibration objects will affect the calibration results. In addition, this method will be limited in some situations where it is not suitable to place calibration objects.

[0004] Therefore, people have proposed camera calibration methods that do not require calibration objects. For example, based on deep learning methods for camera correction, RGB images are used to estimate camera internal parameters, thereby broadening the application scope of camera calibration.

[0005] It should be noted that the above introduction to the technical background is only for the convenience of providing a clear and complete description of the technical solutions of the present application and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art simply because they are described in the background technology section of this document. Summary of the invention

[0006] The inventors found that in the prior art, when the image distortion is small or does not exist, the accuracy of the camera parameters obtained by the deep learning method is not high. When the image has large distortion, the parameters estimated by the deep learning method only perform the image dedistortion process once, and the dedistorted image obtained still has many distortion components. Using the parameters to perform multiple dedistortion processes will result in a decrease in the quality of the dedistorted image.

[0007] In response to at least one of the above problems, an embodiment of the present application provides an image conversion device and method.

[0008] According to a first aspect of an embodiment of the present application, there is provided an image conversion device, the device comprising:

[0009] A first image processing device, which performs a first image processing on the original image according to a value of a first parameter of the original image to obtain an intermediate image;

[0010] an iterative device, which iteratively performs a first image processing on the intermediate image according to the value of the first parameter of the intermediate image to obtain a final intermediate image;

[0011] an estimation device for estimating an estimated value of the first parameter based on a conversion relationship between the distorted image and the corresponding perspective image, the original image and the final intermediate image;

[0012] The second image processing device performs second image processing on the original image according to the estimated value of the first parameter to obtain a converted image.

[0013] According to a second aspect of an embodiment of the present application, there is provided an image conversion method, the method comprising:

[0014] Performing a first image processing on the original image according to a value of a first parameter of the original image to obtain an intermediate image;

[0015] Iteratively performing first image processing on the intermediate image according to the value of the first parameter of the intermediate image to obtain a final intermediate image;

[0016] estimating an estimated value of the first parameter according to a conversion relationship between the distorted image and the corresponding perspective image, the original image, and the final intermediate image; and

[0017] The second image processing is performed on the original image according to the estimated value of the first parameter to obtain a converted image.

[0018] One of the beneficial effects of the embodiments of the present application is that: by performing multiple image processing on the original image to obtain multiple intermediate images, the original image and the multiple intermediate images are used as training data to estimate the estimated value of the first parameter, and a high-precision first parameter can be obtained, so that the accuracy of image dedistortion can be improved when the original image is dedistorted, and the original image is processed according to the high-precision parameter to obtain an image with good dedistortion effect and high image quality, and when the original image is distorted, a distorted image close to the actual situation can be obtained, thereby meeting the application in different scenarios.

[0019] With reference to the following description and accompanying drawings, the specific embodiments of the present application are disclosed in detail, indicating the way in which the principles of the present application can be adopted. It should be understood that the embodiments of the present application are not limited in scope. Within the scope of the spirit and clauses of the appended claims, the embodiments of the present application include many changes, modifications and equivalents.

[0020] Features described and / or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0021] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, integers, steps or components, but does not exclude the presence or addition of one or more other features, integers, steps or components. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the implementation methods of the present application, and together with the text description, explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0023] Figure 1 is a schematic diagram of an image conversion method according to an embodiment of the present application;

[0024] Figure 2 It is a schematic diagram for explaining the image distortion caused by the camera distortion in spherical projection;

[0025] Figure 3 is a schematic diagram of an implementation of the first image processing of an embodiment of the present application;

[0026] Figure 4 It is a schematic diagram for explaining the termination condition of iterative execution;

[0027] Figure 5 is another schematic diagram for explaining the termination condition of iterative execution;

[0028] Figure 6 is a schematic diagram of another implementation of the first image processing of the embodiment of the present application;

[0029] Figure 7 is a schematic diagram of an image conversion device according to an embodiment of the present application;

[0030] Figure 8 A schematic diagram of an electronic device according to an embodiment of the present application;

[0031] Fig. 9 It is a schematic block diagram of the system structure of the electronic device according to the embodiment of the present application. DETAILED DESCRIPTION

[0032] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements in terms of title, but do not indicate the spatial arrangement or temporal order of these elements, etc., and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the presence of the stated features, elements, components or components, but do not exclude the presence or addition of one or more other features, elements, components or components.

[0033] In the embodiments of the present application, the singular forms "a", "the", etc. include plural forms and should be broadly understood as "a kind" or "a type" rather than being limited to the meaning of "one"; in addition, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least in part according to...", and the term "based on" should be understood as "at least in part based on...", unless the context clearly indicates otherwise.

[0034] With reference to the accompanying drawings, the above and other features of the present application will become apparent through the following description. In the description and the accompanying drawings, specific embodiments of the present application are specifically disclosed, which show some embodiments in which the principles of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the present application includes all modifications, variations and equivalents falling within the scope of the attached claims.

[0035] Embodiments of the first aspect

[0036] An embodiment of the present application provides an image conversion method. Figure 1 FIG. 1 is a schematic diagram of an image conversion method according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0037] 101: Performing first image processing on the original image according to a value of a first parameter of the original image to obtain an intermediate image;

[0038] 102: Iteratively perform first image processing on the intermediate image according to the value of the first parameter of the intermediate image to obtain a final intermediate image;

[0039] 103: estimating an estimated value of the first parameter according to a conversion relationship between the distorted image and the corresponding perspective image, the original image, and the final intermediate image; and

[0040] 104: Perform a second image processing on the original image according to the estimated value of the first parameter to obtain a converted image.

[0041] Therefore, by performing multiple image processing on the original image to obtain multiple intermediate images, and using the original image and the multiple intermediate images as training data to estimate the estimated value of the first parameter, a high-precision first parameter can be obtained.

[0042] In some embodiments, the image conversion method of the embodiments of the present application can be applied to a scenario where a distorted image is dedistorted. For example, the original image is a distorted image (hereinafter referred to as the "distorted image"), and the first parameter of the original image includes the focal length f and the distortion coefficient ξ of the camera corresponding to the original image. For the convenience of explanation below, the first parameter of the distorted image is sometimes referred to as a "distortion parameter."

[0043] For example, in step 101, a first image processing is performed on the distorted image according to the value of the distortion parameter of the distorted image to obtain an intermediate image; in step 102, a first image processing is iteratively performed on the intermediate image according to the value of the distortion parameter of the intermediate image to obtain a final intermediate image; in step 103, an estimated value of the distortion parameter is estimated according to a conversion relationship between the distorted image and the final intermediate image and the value of the distortion parameter of the distorted image or the value of the distortion parameter of the final intermediate image; in step 104, a second image processing is performed on the distorted image according to the estimated value of the distortion parameter to obtain a dedistorted image.

[0044] Therefore, by performing multiple dedistortion processing on the distorted image to obtain multiple intermediate images, and using the distorted image and the multiple intermediate images as training data to estimate the estimated values ​​of the distortion parameters, high-precision distortion parameters can be obtained, thereby improving the accuracy of image dedistortion.

[0045] Figure 2 This is a schematic diagram to illustrate the image distortion caused by camera distortion during spherical projection. Figure 2 In the figure, for convenience of explanation, it is assumed that the focal length f=1 when no distortion occurs.

[0046] Below Figure 2 Take the image distortion as an example to illustrate.

[0047] like Figure 2 As shown in (a), in the absence of distortion, the distance between the projection of a point (X, Y, Z) in the world coordinate system on the imaging plane and the center point of the imaging plane is r, but due to distortion, the distance between the projection of a point in the world coordinate system on the imaging plane and the center point of the imaging plane is rd, causing the focus of the image to be distorted from point Op to Od. The distance between Op and Od is recorded as "distortion coefficient ξ". The relationship between the above points is as follows: Figure 2 As shown in (b), the distortion coefficient ξ and the viewing angle θ satisfy formula (1):

[0048]

[0049] Then the distance rd and the viewing angle θ can be expressed by the distortion coefficient ξ:

[0050]

[0051]

[0052] In addition, the focal length f of the image is also distorted. In the embodiment of the present application, the image dedistortion process is described by taking the “focal length f” and “distortion coefficient ξ” of the image as the distortion parameters of the image, but the embodiment of the present application is not limited thereto, and other parameters may also be used as the distortion parameters of the image.

[0053] In the embodiment of the present application, in order to facilitate calculation, the "distortion coefficient ξ" can be normalized. The value range of the normalized "distortion coefficient ξ" is 0 to 1. When the distortion coefficient ξ is 0, it means that the image is not distorted. When the distortion coefficient ξ is 1, it means that the image is distorted. The larger the distortion coefficient ξ, the greater the degree of distortion; and vice versa. In the description of the following embodiments, the value of the predefined distortion coefficient can be any number between 0 and 1. When the image is a distortion-free image or a perspective image, its distortion coefficient can be predefined as 0. When the image is a distorted image, its distortion coefficient can be predefined as 1, and vice versa.

[0054] Figure 3 FIG. 1 is a schematic diagram of an implementation of the first image processing of an embodiment of the present application. Figure 3 In the example shown, the original image F is distorted, hereinafter referred to as the “distorted image F”. Figure 3 The process of performing the first image processing on the distorted image F is shown.

[0055] In the above example, the first image processing is to dedistort the distorted image to obtain an intermediate image, such as Figure 3 As shown, the process of performing the first image processing on the distorted image F may include:

[0056] 301: Expand the canvas according to the estimated expansion ratio of the size of the distorted image;

[0057] 302: Perform bilinear interpolation processing on the extended canvas according to the value of the distortion parameter of the distorted image to obtain a first interpolated image;

[0058] 303: Crop the first interpolated image to obtain a first cropped image having a ratio of length to width equal to that of the distorted image;

[0059] 304: Scale the first cropped image to obtain an intermediate image of the same size as the distorted image.

[0060] like Figure 3 As shown, image F represents the original distorted image, images F1 to FN-1 represent intermediate images obtained in the dedistortion process, image FN represents the final intermediate image, and N is a natural number greater than 1. The process of performing the first image processing on the distorted image F is as follows: the expansion ratio is estimated according to the size of the distorted image F (length is W, width is H) to obtain the expanded canvas P, bilinear interpolation processing is performed on the expanded canvas P according to the distortion parameters f and ξ of the distorted image F to obtain the first interpolated image P1, the first interpolated image P1 is cropped to obtain the first cropped image P11 proportional to the length and width ratio of the distorted image F, and the first cropped image is scaled to obtain the intermediate image F1 of the same size as the distorted image F.

[0061] In the above example, the expansion ratio of the canvas P can be predefined, for example, 1.5 times, or the position of at least one of the four vertices of the distorted image F in the canvas can be calculated, and then the expansion ratio of the canvas can be determined according to the position. The implementation method of expanding the canvas can refer to the relevant technology, and the embodiment of the present application is not limited to this.

[0062] In the above example, in the first interpolation image P1 obtained by bilinear interpolation, there is a largest rectangular area A1, the midpoints of the four sides of A1 correspond to the midpoints of the four sides of the distorted image F, and the largest range is cut out in A1 according to the aspect ratio of the distorted image F, for example, a first cropped image P11 with a length of W1 and a width of H1 is cut out, H1 / W1=H / W. In addition, the implementation of bilinear interpolation and proportional scaling can refer to the relevant technology, and the embodiment of the present application does not limit this.

[0063] In the embodiment of the present application, the process of performing the first image processing on the intermediate image is the same as the process of performing the first image processing on the distorted image, that is, the canvas is expanded according to the estimated expansion ratio of the size of the intermediate image F1; the extended canvas is bilinearly interpolated according to the distortion parameters f1 and ξ1 of the intermediate image F1 to obtain an interpolated image; the interpolated image is cropped to obtain a cropped image with a ratio of length to width equal to that of the intermediate image F1; the cropped image is scaled to obtain an intermediate image F2 with the same size as that of the intermediate image F1. In addition, the above process is iteratively performed on the intermediate images F2 to FN-1 until the final intermediate image FN is obtained.

[0064] In the above example, at least three intermediate images are obtained as an example, but the embodiments of the present application are not limited to this. The number of intermediate images and the selection of the final intermediate image are related to the termination condition of the iterative execution.

[0065] In some embodiments, the termination condition of the iterative execution includes that the change trend of the value of the first parameter begins to reverse or reaches a predetermined number of iterations.

[0066] When performing iterative dedistortion processing on the distorted image, the first parameter (ie, the distortion parameter) is the focal length focal and the distortion coefficient ξ of the camera corresponding to the distorted image, and the termination condition of the iterative execution includes that the value of the distortion parameter starts to increase or reaches a predetermined number of iterations.

[0067] The following explains them separately.

[0068] In some embodiments, when the value of the distortion parameter begins to increase during the iteration, the iteration is terminated.

[0069] For an image, the relationship between the image's field of view angle θ, size H, and focal length focal can be expressed as:

[0070] Tan(θ)=(0.5H) / focal (4)

[0071] After the image is dedistorted, the image size H is kept unchanged, so the field of view θ of the dedistorted image becomes smaller. Therefore, when the dedistortion operation is repeated, the focal length focal needs to be reduced to ensure that the size of the field of view θ corresponding to the size H remains unchanged and balance the reduction factor of the initial field of view θ.

[0072] When the dedistortion operation is iteratively performed on the image, if the predicted value of the focal length f or the distortion coefficient ξ is greater than the previous value, it can be considered that the image is over-corrected and further correction iterations should be stopped.

[0073] Figure 4 and Figure 5 They are schematic diagrams illustrating the termination conditions of iterative execution. Figure 4 The figure shows the change of focal length when 10 iterations are performed on a distorted image A. Figure 5 The change of focal length when another distorted image B is subjected to 10 iterations is shown.

[0074] like Figure 4 and Figure 5 As shown, the initial focal length of the distorted image is f0, the focal length of the image after the first first image processing is f1, the focal length of the image after the second first image processing is f2, the focal length of the image after the third first image processing is f3, the focal length of the image after the fourth first image processing is f4, the focal length of the image after the fifth first image processing is f5, the focal length of the image after the sixth first image processing is f6, the focal length of the image after the seventh first image processing is f7, the focal length of the image after the eighth first image processing is f8, and the focal length of the image after the ninth first image processing is f9, and then the tenth first image processing can be performed according to the focal length f9.

[0075] like Figure 4 As shown, the focal lengths f0 and f4 decrease in sequence, and f4 and f5 increase in sequence. The iteration terminates after the fifth first image processing is performed (for example, Figure 4 The step 401 shown in FIG. 4 is performed, and the intermediate image after the fourth first image processing is performed is output as a final intermediate image (for example, Figure 4 Step 402 shown).

[0076] That is, in some embodiments, when the iteration is terminated when the value of the distortion parameter begins to increase, the intermediate image obtained after the previous execution of the first image processing is used as the final intermediate image. In addition, when the value of the distortion parameter of the intermediate image obtained after the first image processing is performed on the distorted image once is greater than the value of the distortion parameter of the distorted image, the distorted image is used as the final intermediate image.

[0077] For example, Figure 5 As shown, the focal length increases from f0 to f1, and the iteration is terminated after the first image processing is performed (for example, Figure 5 The step 501 shown in FIG. 5 is performed, and the distorted image B is output as a final intermediate image (eg, Figure 5 Step 502 shown).

[0078] The focal length is used as an example for explanation above. The distortion coefficient ξ is similar and will not be explained here one by one.

[0079] In some embodiments, the iteration is terminated after reaching a predetermined number of iterations. In the embodiment of the present application, when the iteration is terminated after reaching a predetermined number of iterations, the intermediate image obtained after the last first image processing is performed is used as the final intermediate image. The predetermined number of iterations can be set as needed, and the embodiment of the present application does not limit the predetermined number of iterations.

[0080] In the embodiments of the present application, Figure 1 As shown, the method may also include:

[0081] 105: Obtain the value of the first parameter of the original image and the value of the first parameter of the intermediate image.

[0082] When the original image is a distorted image, the original image and the intermediate image can be considered to be distorted, and the first parameter thereof is the focal length and distortion coefficient of the camera corresponding to the image. Therefore, the first parameter of the distorted image is also referred to as a "distortion parameter". In step 105, the values ​​of the distortion parameters of the original image and the values ​​of the distortion parameters of the intermediate image are obtained. For example, the distortion parameters of the image can be predicted based on a deep learning model, for example, the focal length and distortion coefficient of the image are predicted using a DeepCalib model. The specific implementation of the DeepCalib model can refer to related technologies, for example, the paper "DeepCalib: A Deep Learning Approach for Automatic Intrinsic Calibration of Wide Field-of-View Cameras" by Bogdan O, Eckstein V, Rameau F, etc. can be referred to. The specific implementation method can refer to related technologies, and the embodiments of the present application are not limited to this.

[0083] The "value of the distortion parameter of the original image" obtained in step 105 can be used in step 101. For example, the method can be used Figure 3 The distortion parameter f,ξ is obtained for the distorted image F. The "value of the distortion parameter of the intermediate image" obtained in step 105 can be used in step 102. For example, this method can be used Figure 3 The distortion parameters fn, ξn are obtained for the intermediate images F1 to FN-1, where n is a natural number greater than or equal to 1 and less than or equal to N-1, and N is a natural number greater than 1.

[0084] In an embodiment of the present application, an estimated value of the distortion parameter is obtained according to the predicted value of the distortion parameter, and the image is dedistorted according to the estimated value. Compared with a dedistorted image obtained by repeatedly performing the dedistortion process on the distorted image, a dedistorted image obtained by performing the dedistortion process on the distorted image once according to the estimated value has higher clarity and better dedistortion effect.

[0085] In an embodiment of the present application, a forward transformation function may be used to estimate the estimated value of the first parameter, or a reverse transformation function may be used to estimate the estimated value of the first parameter. Forward Transform (FT) refers to the point coordinate transformation of a perspective image to a distorted image, and can be used in scenarios where the focal length and distortion coefficient (also called "distortion parameter") of the original image are estimated when the original image is distorted. Backward Transform (BT) refers to the point coordinate transformation of a distorted image to a perspective image, and can be used in scenarios where the focal length of the original image is estimated when the original image is not distorted.

[0086] The following explains them separately.

[0087] In some embodiments, when the original image is distorted, the forward transformation function FT-Mf may be used to estimate the estimated value of the distortion parameter of the original image, and the difference function used for estimation is the residual function of the forward transformation function.

[0088] The forward transformation function FT-Mf can be expressed as:

[0089] xqi=Mfx(f,ξ;xpi,ypi)

[0090] yqi = Mfy(f,ξ;xpi,ypi) (5)

[0091] The residual function can be expressed as:

[0092] dxi=Mfx(xpi,ypi)-xqi

[0093] dyi = Mfy(xpi,ypi) – yqi (6)

[0094] Among them, Mf represents the functional expression of the forward transformation, Mfx corresponds to the expression in the x direction (for example, the width direction of the image), Mfy corresponds to the expression in the y direction (for example, the height direction of the image), xqi and yqi represent the x-direction coordinates and y-direction coordinates of each pixel point qi in the distorted image, xpi and ypi represent the x-direction coordinates and y-direction coordinates of each pixel point pi in the perspective image, f includes the focal length of the perspective image and the focal length of the distorted image, ξ represents the distortion coefficient of the distorted image, dxi and dyi represent the residuals between the distorted image and the distorted image corresponding to the perspective image.

[0095] When estimating the estimated value of the distortion parameter using the forward transformation function FT-Mf, the image coordinates FN (xpi, ypi) of the final intermediate image FN are input into the function FT-Mf to obtain the image coordinates Fn (xqi, yqi) of the theoretical distorted image Fn corresponding to the final intermediate image FN. That is, the final intermediate image is input into the forward transformation function FT-Mf as a perspective image. The output result of the function is a distorted image corresponding to the final intermediate image. In addition, the focal length fn of the final intermediate image FN can also be input into the forward transformation function FT-Mf. In addition, if the distortion coefficient ξ of the perspective image (i.e., the final intermediate image) needs to be input, the distortion coefficient ξ can be pre-set to 0.

[0096] In the above example, the residual function is used to calculate the residuals dxi and dyi between the image coordinates Fn(xqi, yqi) of the distorted image corresponding to the final intermediate image and the image coordinates F(xqi, yqi) of the original image F (i.e., the distorted image). When the residuals dxi and dyi are the smallest, it indicates that the perspective image corresponding to the final intermediate image is most similar to the distorted image, which means that the value of the distortion coefficient at this time is closest to the true value.

[0097] In addition, the above example uses the example of inputting the pixel coordinates of the image into the function FT-Mf, but the embodiments of the present application are not limited to this. In practical applications, the corresponding parameters can be input according to the actual construction of the function FT-Mf. For example, the function FT-Mf can also involve the field of view angle θ, rotation angle φ, etc. of the image, so the field of view angle θ, rotation angle φ, etc. of the image can also be input into the function FT-Mf. In addition, the normalized pixel coordinates can also be used when constructing the function FT-Mf, so the pixel coordinates of the input image can also be normalized before subsequent processing. The embodiments of the present application do not limit the specific expression of the function FT-Mf, and specific reference can be made to the relevant technology.

[0098] In some embodiments, the first parameter (e.g., distortion parameter) may be estimated using the least squares method (LSM). For example, the total residual S may be expressed as S=(1 / 2)*SUM(dxi^2+dyi^2), and the value of the distortion parameter f, ξ corresponding to the minimum total residual S is estimated using the least squares method, which is the estimated value f*, ξ* of the distortion parameter.

[0099] In the above example, in step 104, the second image processing is performed on the original distorted image according to the estimated value f*,ξ* of the distortion parameter to obtain a dedistorted converted image. The process of performing the second image processing on the original image is roughly the same as the process of performing the first image processing on the original image, except that the distortion parameter used in the second image processing is the estimated value of the distortion parameter.

[0100] That is, the second image processing may include: expanding the canvas according to the estimated expansion ratio of the size of the original image F; performing bilinear interpolation processing on the expanded canvas according to the estimated values ​​of the first parameters f*,ξ* to obtain a second interpolated image; cropping the second interpolated image to obtain a second cropped image that is proportional to the aspect ratio of the original image; and scaling the second cropped image to obtain the converted image, i.e., the dedistorted image, of the same size as the original image. Thus, the original distorted image is subjected to a relatively small number of second image processings (e.g., one second image processing) according to the estimated high-precision distortion parameters, so that a dedistorted image with high image quality can be obtained. Therefore, the present application can obtain a dedistorted image with good dedistortion effect and high image quality.

[0101] The above describes the process of dedistorting the original image when the original image is distorted, but the embodiments of the present application are not limited thereto. The embodiments of the present application can also be applied to the scene where the original image is distorted when the original image is not distorted or the distortion of the original image is within an acceptable degree or range. For example, it is hoped to obtain a more realistic distorted image based on the original non-distorted image, and then use the transformed distorted image to conduct further research or use the transformed image as learning data for machine learning. The embodiments of the present application are not limited to this.

[0102] The process of adding distortion to the original image is described below.

[0103] Figure 6 It is a schematic diagram of another implementation of the first image processing of an embodiment of the present application.

[0104] exist Figure 6 In the example shown, the original image F' has no distortion, or it can be said that the distortion of the original image F' is within an acceptable degree or range (hereinafter, for the convenience of explanation, the original image F' is sometimes referred to as the distortion-free image F'). Figure 6 The process of performing the first image processing on the original image F' is shown.

[0105] In the above example, the first image processing is to distort the undistorted image to obtain an intermediate image. When the undistorted image is iteratively distorted, the first parameter is the focal length f of the camera corresponding to the undistorted image, and the distortion coefficient ξ used when adding distortion is pre-set, for example, the distortion coefficient ξ is pre-set to any value between 0 and 1. Figure 6 As shown, the process of performing the first image processing on the original image F' may include:

[0106] 601: Expand the canvas according to the estimated expansion ratio of the size of the original image;

[0107] 602: Perform distortion processing on the extended canvas according to the value of the first parameter of the original image and a predetermined distortion coefficient to obtain a first distorted image;

[0108] 603: Crop the first distorted image to obtain a first cropped image having a ratio of length to width equal to that of the original image;

[0109] 604 Scale the first cropped image to obtain the intermediate image of the same size as the original image.

[0110] like Figure 6As shown, image F' represents the original undistorted image, images F1' to FN-1' represent intermediate images obtained in the distortion process, image FN' represents the final intermediate image, and N is a natural number greater than 1. The process of performing the first image processing on the undistorted image F' is as follows: according to the size of the undistorted image F' (length W, width H), the expansion ratio is estimated to obtain the expanded canvas P', according to the focal length f of the undistorted image F' and the predetermined distortion coefficient ξ0, the expanded canvas P' is subjected to distortion processing to obtain the first distorted image P1', the first distorted image P1' is cropped to obtain the first cropped image P11' which is proportional to the length and width ratio of the undistorted image F', and the first cropped image P11' is scaled to obtain the intermediate image F1' which is the same size as the undistorted image F'.

[0111] In the above example, the expansion ratio of the canvas P' can be predefined, for example, 1.5 times, or the position of at least one of the four vertices of the undistorted image F' in the canvas under the predetermined distortion coefficient ξ0 can be calculated, and then the expansion ratio of the canvas is determined according to the position. The predetermined distortion coefficient ξ0 can take any value between 0 and 1. The implementation method of expanding the canvas can refer to the relevant technology, and the embodiment of the present application is not limited to this.

[0112] In the above example, in the first distorted image P1' obtained by the distortion processing, there is a largest rectangular area A1', the midpoints of the four sides of A1' correspond to the midpoints of the four sides of the distorted image F, and the largest range is cut out in A1 according to the length-width ratio of the undistorted image F', for example, a first cropped image P11' with a length of W1 and a width of H1 is cut out, H1 / W1=H / W. In addition, the implementation methods of the distortion processing and proportional scaling can refer to the relevant technology, and the embodiments of the present application are not limited thereto.

[0113] In the embodiment of the present application, the process of performing the first image processing on the intermediate images F1' to FN-1' is the same as the process of performing the first image processing on the distortion-free image F', that is, the canvas is expanded according to the estimated expansion ratio of the size of the intermediate image F1'; the expanded canvas is distorted according to the focal length f1 of the intermediate image F1' and the predetermined distortion coefficient ξ1 to obtain the intermediate distorted image; the intermediate distorted image is cropped to obtain a cropped image with the same length-to-width ratio as the intermediate image F1'; the cropped image is scaled to obtain an intermediate image F2' with the same size as the intermediate image F1'. In addition, the above process is iteratively performed on the intermediate images F2' to FN-1' until the final intermediate image FN' is obtained.

[0114] In the above example, the termination condition of the iterative execution includes that the focal length f starts to decrease or reaches a predetermined number of iterations.

[0115] In the embodiment of the present application, when the iteration is terminated when the focal length f begins to decrease, the intermediate image obtained after the previous first image processing is used as the final intermediate image. In addition, when the focal length f of the intermediate image obtained after the first image processing is performed once on the distortion-free image is smaller than the focal length f of the distortion-free image, the distortion-free image is used as the final intermediate image.

[0116] In addition, when the iteration is terminated after reaching the predetermined number of iterations, the intermediate image obtained after the last first image processing is performed is used as the final intermediate image. The predetermined number of iterations can be set as needed, and the embodiment of the present application does not limit the predetermined number.

[0117] In the above example, in step 105, only the focal length value of the camera corresponding to the original image and the focal length value of the camera corresponding to the intermediate image can be obtained. The implementation method of obtaining the focal length value of the camera corresponding to the image can refer to the relevant technology, and the embodiment of the present application does not limit this.

[0118] In some embodiments, when there is no distortion in the original image, or when the distortion of the original image is within an acceptable degree or range, the inverse transformation function BT-Mf can be used to estimate the focal length of the original image, and the difference function used for estimation is the residual function of the inverse transformation function.

[0119] The inverse transformation function BT-Mf can be expressed as:

[0120] xpi=Mfx(f,ξ;xqi,yqi)

[0121] ypi = Mfy(f,ξ;xqi,yqi) (7)

[0122] The residual function can be expressed as:

[0123] dxi=Mfx(xqi,yqi)-xpi

[0124] dyi = Mfy(xqi,yqi) – ypi (8)

[0125] Among them, Mf represents the functional expression of the inverse transformation, Mfx corresponds to the expression in the x direction (for example, the width direction of the image), Mfy corresponds to the expression in the y direction (for example, the height direction of the image), xpi and ypi represent the x-direction coordinate and y-direction coordinate of each pixel point pi of the perspective image, xqi and yqi represent the x-direction coordinate and y-direction coordinate of each pixel point qi of the distorted image, f includes the focal length of the perspective image and the focal length of the distorted image, ξ represents the distortion coefficient of the distorted image, dxi and dyi represent the residuals between the perspective image and the perspective image corresponding to the distorted image.

[0126] The estimation of the focal length f of the camera using the reverse transformation function BT-Mf is similar to the estimation of the focal length f and the distortion coefficient ξ using the forward transformation function FT-Mf. The image coordinates Fn'(xqi, yqi) of the final intermediate image FN' are input into the function BT-Mf to obtain the image coordinates Fn'(xpi, ypi) of the theoretical non-distorted image Fn' corresponding to the final intermediate image FN', that is, the image coordinates Fn'(xpi, ypi) of the perspective image corresponding to the final intermediate image FN'. That is, the final intermediate image is input as a distorted image into the reverse transformation function BT-Mf, and the output result of the function is the non-distorted perspective image corresponding to the final intermediate image. In addition, the focal length fn and the distortion coefficient ξn-1 of the final intermediate image FN can also be input into the reverse transformation function BT-Mf.

[0127] In the above example, the residual function is used to calculate the residuals dxi and dyi between the image coordinates Fn'(xpi, ypi) of the perspective image corresponding to the final intermediate image FN' and the image coordinates F'(xpi, ypi) of the original image F'. When the residuals dxi and dyi are the smallest, it indicates that the perspective image (theoretical distortion-free image) corresponding to the final intermediate image is most similar to the original distortion-free image, which means that the focal length value at this time is closest to the true value.

[0128] In addition, similar to the forward function FT-Mf, in practical applications, corresponding parameters can be input according to the actual construction of the function BT-Mf. The above description only takes the input pixel coordinates as an example, but the embodiment of the present application is not limited thereto. The field of view angle θ of the image and the rotation angle θ can also be input. In addition, the pixel coordinates may be normalized before subsequent processing, which is not limited in the embodiments of the present application.

[0129] In the above example, the least squares method can also be used to estimate the distortion parameters. For example, the total residual S can be expressed as S = (1 / 2) * SUM (dxi^2 + dyi^2). The least squares method is used to estimate the value of the focal length f corresponding to the minimum total residual S. This value is the estimated value f* of the focal length f.

[0130] In addition, in the embodiment of the present application, other estimation methods may be used to estimate the distortion parameters, such as maximum likelihood estimation, etc. For details, please refer to the relevant technology and will not be described one by one here.

[0131] In addition, in the embodiment of the present application, the forward transformation function FT-Mf and / or the reverse transformation function BT-Mf may be a linear function of the distortion parameter f,ξ after one optimization, which is not limited in the embodiment of the present application.

[0132] The above only describes the steps or processes related to the present application, but the present application is not limited thereto. The image conversion method may also include other steps or processes, and the specific contents of these steps or processes may refer to the prior art. In addition, the above only uses some structures of the model used in the image conversion method as an example to exemplify the embodiments of the present application, but the present application is not limited to these structures, and these structures may be appropriately modified, and the implementation methods of these modifications should all be included in the scope of the embodiments of the present application.

[0133] The above embodiments are merely exemplary of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0134] It can be seen from the above embodiments that by performing multiple image processing on the original image to obtain multiple intermediate images, and using the original image and the multiple intermediate images as training data to estimate the estimated value of the first parameter, a high-precision first parameter can be obtained, thereby improving the accuracy of image dedistortion when the original image is dedistorted, and by performing image processing on the original image according to the high-precision parameter, an image with good dedistortion effect and high image quality can be obtained, and when the original image is distorted, a distorted image close to the actual situation can be obtained, thereby meeting applications in different scenarios.

[0135] Embodiments of the second aspect

[0136] An embodiment of the present application also provides an image conversion device, which corresponds to the image conversion method of the first aspect embodiment. Figure 7 Schematic diagram of an image conversion device according to an embodiment of the present application. Figure 7 As shown, the image conversion device 700 includes:

[0137] A first image processing device 701, which performs a first image processing on the original image according to the value of the first parameter of the original image to obtain an intermediate image;

[0138] Iteration device 702, which iteratively performs first image processing on the intermediate image according to the value of the first parameter of the intermediate image to obtain a final intermediate image;

[0139] An estimation device 703, which estimates an estimated value of the first parameter according to a conversion relationship between the distorted image and the corresponding perspective image, the original image and the final intermediate image;

[0140] The second image processing device 704 performs second image processing on the original image according to the estimated value of the first parameter to obtain a converted image.

[0141] In some embodiments, the termination condition for the iteration performed by the iteration device 702 includes that the change trend of the value of the first parameter starts to reverse or reaches a predetermined number of iterations.

[0142] In some embodiments, the conversion relationship includes a first conversion relationship for converting from a perspective image to a distorted image, and the estimation device 703 converts the final intermediate image into a corresponding distorted image according to the first conversion relationship, and obtains an estimated value of the first parameter when the value of a first difference function is minimized based on the distorted image corresponding to the final intermediate image and the original image, wherein the first difference function represents the difference between the distorted image corresponding to the final intermediate image and the original image.

[0143] In some embodiments, the conversion relationship includes a second conversion relationship for converting from a distorted image to a perspective image. The estimation device 703 converts the final intermediate image into a corresponding perspective image according to the second conversion relationship, and obtains an estimated value of the first parameter when the value of a second difference function is minimized based on the perspective image corresponding to the final intermediate image and the original image. The second difference function represents the difference between the perspective image corresponding to the final intermediate image and the original image.

[0144] In some embodiments, the estimation device 703 calculates the estimated value of the first parameter using a least squares method.

[0145] In some embodiments, the first parameter includes a focal length and / or a distortion coefficient of a camera corresponding to the image.

[0146] In some embodiments, the first image processing device 701 performs the first image processing including:

[0147] Expanding the canvas according to the estimated expansion ratio of the size of the original image;

[0148] Performing bilinear interpolation processing on the extended canvas according to the value of the first parameter of the original image to obtain a first interpolated image;

[0149] Cropping the first interpolated image to obtain a first cropped image having a length-to-width ratio equal to that of the original image;

[0150] The first cropped image is scaled to obtain the intermediate image having the same size as the original image.

[0151] In some embodiments, the first image processing device 701 performs the first image processing including:

[0152] Expanding the canvas according to the estimated expansion ratio of the size of the original image;

[0153] Performing distortion processing on the extended canvas according to the value of the first parameter of the original image and a predetermined distortion coefficient to obtain a first distorted image;

[0154] Cropping the first distorted image to obtain a first cropped image having a length-to-width ratio equal to that of the original image; and

[0155] The first cropped image is scaled to obtain the intermediate image having the same size as the original image.

[0156] In some embodiments, the second image processing device 704 performs the second image processing including:

[0157] Expanding the canvas according to the estimated expansion ratio of the size of the original image;

[0158] Performing bilinear interpolation processing on the extended canvas according to the estimated value of the first parameter to obtain a second interpolated image;

[0159] Cropping the second interpolated image to obtain a second cropped image having a ratio of length to width equal to that of the original image;

[0160] The second cropped image is scaled to obtain the converted image having the same size as the original image.

[0161] In some embodiments, the second image processing device 704 performs the second image processing including:

[0162] Expanding the canvas according to the estimated expansion ratio of the size of the original image;

[0163] Performing distortion processing on the extended canvas according to the estimated value of the first parameter and a predetermined distortion coefficient to obtain a second interpolated image;

[0164] Cropping the second interpolated image to obtain a second cropped image having a ratio of length to width equal to that of the original image; and

[0165] The second cropped image is scaled to obtain the converted image having the same size as the original image.

[0166] In some embodiments, Figure 7 As shown, the image conversion device 700 also includes:

[0167] The parameter acquisition device 705 acquires the value of the first parameter of the original image and the value of the first parameter of the intermediate image.

[0168] In this embodiment, the specific implementation methods of the above-mentioned devices are the same as those described in the embodiment of the first aspect and will not be repeated here.

[0169] It is worth noting that the above only describes the components or modules related to the present application, but the present application is not limited thereto. The image conversion device 700 may also include other components or modules, and the specific contents of these components or modules may refer to the relevant technology.

[0170] To keep it simple, Figure 7 The connection relationship or signal direction between various components or modules is only exemplified, but it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The above-mentioned components or modules can be implemented by hardware facilities such as processors and memories; the embodiments of the present application are not limited to this.

[0171] The above embodiments are merely exemplary of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0172] It can be seen from the above embodiments that by performing multiple image processing on the original image to obtain multiple intermediate images, the original image and the multiple intermediate images are used as training data to estimate the estimated value of the first parameter, and a high-precision first parameter can be obtained, so that the accuracy of image dedistortion can be improved when the original image is dedistorted, and the original image is processed according to the high-precision parameter to obtain an image with good dedistortion effect and high image quality, and when the original image is distorted, a distorted image close to the actual situation can be obtained, thereby meeting the application in different scenarios. .

[0173] Embodiments of the third aspect

[0174] The embodiment of the present application also provides an electronic device, Figure 8 A schematic diagram of an electronic device according to an embodiment of the present application. Figure 8 As shown, the electronic device 800 includes an image conversion device 801. The structure and function of the image conversion device 801 are the same as those of the image conversion device 700 in the embodiment of the second aspect. For specific contents, please refer to the description in the embodiment of the second aspect, and will not be repeated here.

[0175] Fig. 9 1 is a schematic block diagram of the system structure of the electronic device according to the embodiment of the present application. Fig. 9 As shown, the electronic device 900 may include a processor 901 and a memory 902; the memory 902 is coupled to the processor 901. This figure is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0176] In some embodiments, processor 901 includes at least one of a central processing unit (CPU) and a graphics processing unit (GPU).

[0177] like Fig. 9 As shown, the electronic device 900 may further include: an input device 903 , a display 904 , and a power supply 905 .

[0178] In one embodiment, the function of the image conversion device can be integrated into the processor 901. The processor 901 can be configured to: perform a first image processing on the distorted image according to the value of the distortion parameter of the distorted image to obtain an intermediate image; iteratively perform a first image processing on the intermediate image according to the value of the distortion parameter of the intermediate image to obtain a final intermediate image; estimate the estimated value of the distortion parameter according to the conversion relationship between the distorted image and the final intermediate image and the value of the distortion parameter of the distorted image or the value of the distortion parameter of the final intermediate image; perform a second image processing on the distorted image according to the estimated value of the distortion parameter to obtain a dedistorted image. For other embodiments, specific reference may be made to the embodiment of the first aspect, which will not be repeated here.

[0179] In another embodiment, the image conversion device may be configured separately from the processor 901 . For example, the image conversion device may be configured as a chip connected to the processor 901 , and the functions of the image conversion device are implemented under the control of the processor 901 .

[0180] In this embodiment, the electronic device 900 does not necessarily include Fig. 9 All parts shown in .

[0181] like Fig. 9 As shown, the processor 901 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices. The processor 901 receives inputs and controls the operations of various components of the electronic device 900 .

[0182] The memory 902 may be, for example, a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The processor 901 may execute the program stored in the memory 902 to implement information storage or processing. The functions of other components are similar to those of the prior art and are not described in detail herein. The components of the electronic device 900 may be implemented by dedicated hardware, firmware, software or a combination thereof without departing from the scope of the present application.

[0183] It can be seen from the above embodiments that by performing multiple image processing on the original image to obtain multiple intermediate images, and using the original image and the multiple intermediate images as training data to estimate the estimated value of the first parameter, a high-precision first parameter can be obtained, thereby improving the accuracy of image dedistortion when the original image is dedistorted, and by performing image processing on the original image according to the high-precision parameter, an image with good dedistortion effect and high image quality can be obtained, and when the original image is distorted, a distorted image close to the actual situation can be obtained, thereby meeting applications in different scenarios.

[0184] An embodiment of the present application also provides a computer-readable program, wherein when the program is executed in an image conversion device or an electronic device, the program enables a computer to execute the image conversion method described in the first aspect embodiment in the image conversion device or the electronic device.

[0185] An embodiment of the present application also provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to execute the image conversion method described in the embodiment of the first aspect in an image conversion device or an electronic device.

[0186] The image conversion device or the image conversion method executed in the electronic device described in the embodiments of the present application can be directly embodied as hardware, a software module executed by a processor, or a combination of the two. Figure 7 One or more of the functional block diagrams shown in and / or one or more combinations of the functional block diagrams may correspond to various software modules of the computer program flow or to various hardware modules. These software modules may correspond to Figure 1 These hardware modules can be implemented by solidifying these software modules using, for example, a field programmable gate array (FPGA).

[0187] The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in a memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the electronic device uses a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0188] against Figure 7One or more of the functional block diagrams and / or one or more combinations of the functional block diagrams described herein may be implemented as a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or any appropriate combination thereof for performing the functions described in the present application. Figure 7 One or more of the described functional block diagrams and / or one or more combinations of the functional block diagrams may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0189] The present application is described above in conjunction with specific implementation methods, but it should be clear to those skilled in the art that these descriptions are exemplary and are not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the spirit and principles of the present application, and these modifications and variations are also within the scope of the present application.

[0190] This application also provides the following notes:

[0191] 1. An image conversion method, comprising:

[0192] S1: performing a first image processing on the original image according to a value of a first parameter of the original image to obtain an intermediate image;

[0193] S2: iteratively performing a first image processing on the intermediate image according to the value of the first parameter of the intermediate image to obtain a final intermediate image;

[0194] S3: estimating an estimated value of the first parameter according to a conversion relationship between the distorted image and the corresponding perspective image, the original image, and the final intermediate image;

[0195] S4: Performing second image processing on the original image according to the estimated value of the first parameter to obtain a converted image.

[0196] 2. The method described in Note 1, wherein the termination condition for the iterative execution in step S2 includes that the change trend of the value of the first parameter begins to reverse or reaches a predetermined number of iterations.

[0197] 3. The method described in Note 2, wherein when the iteration is terminated when the trend of the change in the value of the first parameter begins to reverse, the intermediate image obtained after the previous execution of the first image processing is used as the final intermediate image.

[0198] 4. The method according to Note 1, wherein:

[0199] The conversion relationship includes a first conversion relationship for converting from a perspective image to a distorted image.

[0200] Step S3 includes: converting the final intermediate image into a corresponding distorted image according to the first conversion relationship, and obtaining an estimated value of the first parameter when the value of a first difference function is minimized based on the distorted image corresponding to the final intermediate image and the original image, wherein the first difference function represents the difference between the distorted image corresponding to the final intermediate image and the original image.

[0201] 5. The method according to Note 1, wherein:

[0202] The transformation relationship includes a second transformation relationship for transforming from a distorted image to a perspective image,

[0203] Step S3 includes: converting the final intermediate image into a corresponding perspective image according to the second conversion relationship, and obtaining an estimated value of the first parameter when the value of a second difference function is minimized based on the perspective image corresponding to the final intermediate image and the original image, wherein the second difference function represents the difference between the perspective image corresponding to the final intermediate image and the original image.

[0204] 6. The method according to Note 4 or 5, wherein:

[0205] The estimates of the distortion parameters are calculated using the least squares method.

[0206] 7. The method according to Note 1, wherein:

[0207] The first parameters are the focal length and distortion coefficient of the image.

[0208] 8. The method according to Note 4, wherein step S1 comprises:

[0209] Expand the canvas according to the estimated expansion ratio of the original image size;

[0210] Performing bilinear interpolation processing on the extended canvas according to the value of the first parameter of the original image to obtain a first interpolated image;

[0211] Cropping the first interpolated image to obtain a first cropped image having a ratio of length to width equal to that of the original image;

[0212] The first cropped image is scaled to obtain an intermediate image of the same size as the original image.

[0213] 9. The method according to Note 5, wherein step S1 comprises:

[0214] Expand the canvas according to the estimated expansion ratio of the original image size;

[0215] Performing distortion processing on the extended canvas according to the value of the first parameter of the original image and a predetermined distortion coefficient to obtain a first distorted image;

[0216] Cropping the first distorted image to obtain a first cropped image having a ratio of length to width equal to that of the original image; and

[0217] The first cropped image is scaled to obtain an intermediate image of the same size as the original image.

[0218] 10. The method according to Note 4, wherein step S4 comprises:

[0219] Expand the canvas according to the estimated expansion ratio of the original image size;

[0220] Performing bilinear interpolation processing on the extended canvas according to the estimated value of the first parameter to obtain a second interpolated image;

[0221] Cropping the second interpolated image to obtain a second cropped image having a length-to-width ratio equal to that of the original image;

[0222] The second cropped image is scaled to obtain a converted image of the same size as the original image.

[0223] 11. The method according to Note 5, wherein step S4 comprises:

[0224] Expand the canvas according to the estimated expansion ratio of the original image size;

[0225] Performing distortion processing on the extended canvas according to the estimated value of the first parameter and the predetermined distortion coefficient to obtain a second interpolated image;

[0226] Cropping the second interpolated image to obtain a second cropped image having a length-to-width ratio equal to that of the original image; and

[0227] The second cropped image is scaled to obtain a converted image of the same size as the original image.

[0228] 12. A storage medium storing a computer-readable program, wherein the computer-readable program causes a processor coupled to the storage medium to execute the following method:

[0229] Performing a first image processing on the original image according to a value of a first parameter of the original image to obtain an intermediate image;

[0230] Iteratively performing first image processing on the intermediate image according to the value of the first parameter of the intermediate image to obtain a final intermediate image;

[0231] estimating an estimated value of the first parameter according to a conversion relationship between the distorted image and the corresponding perspective image, the original image, and the final intermediate image; and

[0232] The second image processing is performed on the original image according to the estimated value of the first parameter to obtain a converted image.

Claims

1. An image conversion device, comprising: A first image processing device, which performs a first image processing on the original image according to a value of a first parameter of the original image to obtain an intermediate image; an iterative device, which iteratively performs a first image processing on the intermediate image according to the value of the first parameter of the intermediate image to obtain a final intermediate image; an estimation device for estimating an estimated value of the first parameter based on a conversion relationship between the distorted image and the corresponding perspective image, the original image and the final intermediate image; The second image processing device performs second image processing on the original image according to the estimated value of the first parameter to obtain a converted image.

2. The device according to claim 1, in, The termination conditions for the iteration executed by the iteration device include that the change trend of the value of the first parameter starts to reverse or reaches a predetermined number of iterations.

3. The device according to claim 1, in, The conversion relationship includes a first conversion relationship for converting from a perspective image to a distorted image. The estimation device converts the final intermediate image into a corresponding distorted image according to the first conversion relationship, and obtains an estimated value of the first parameter when the value of a first difference function is minimized based on the distorted image corresponding to the final intermediate image and the original image, wherein the first difference function represents the difference between the distorted image corresponding to the final intermediate image and the original image.

4. The device according to claim 1, in, The transformation relationship includes a second transformation relationship for transforming from a distorted image to a perspective image, The estimation device converts the final intermediate image into a corresponding perspective image according to the second conversion relationship, and obtains an estimated value of the first parameter when the value of a second difference function is minimized based on the perspective image corresponding to the final intermediate image and the original image, wherein the second difference function represents the difference between the perspective image corresponding to the final intermediate image and the original image.

5. The device according to claim 3 or 4, in, The estimation means calculates the estimated value of the first parameter using a least squares method.

6. The device according to claim 1, in, The first parameter includes the focal length and / or distortion coefficient of the camera corresponding to the image.

7. The device according to claim 3, in, The first image processing device performing the first image processing includes: Expanding the canvas according to the estimated expansion ratio of the size of the original image; Performing bilinear interpolation processing on the extended canvas according to the value of the first parameter of the original image to obtain a first interpolated image; Cropping the first interpolated image to obtain a first cropped image having a ratio of length to width equal to that of the original image; and The first cropped image is scaled to obtain the intermediate image having the same size as the original image.

8. The device according to claim 3, in, The second image processing device performing the second image processing includes: Expanding the canvas according to the estimated expansion ratio of the size of the original image; Performing bilinear interpolation processing on the extended canvas according to the estimated value of the first parameter to obtain a second interpolated image; Cropping the second interpolated image to obtain a second cropped image having a ratio of length to width equal to that of the original image; and The second cropped image is scaled to obtain the converted image having the same size as the original image.

9. The device according to claim 1, in, The device also includes: A parameter acquisition device is used to acquire the value of the first parameter of the original image and the value of the first parameter of the intermediate image.

10. An image conversion method, the method comprising: Performing a first image processing on the original image according to a value of a first parameter of the original image to obtain an intermediate image; Iteratively performing first image processing on the intermediate image according to the value of the first parameter of the intermediate image to obtain a final intermediate image; estimating an estimated value of the first parameter according to a conversion relationship between the distorted image and the corresponding perspective image, the original image, and the final intermediate image; as well as The second image processing is performed on the original image according to the estimated value of the first parameter to obtain a converted image.

Images