Image processing method and device, electronic equipment, readable storage medium

Abstract

The application relates to an image processing method and device, electronic equipment and computer readable storage medium. A first intermediate image is obtained by performing projection transformation on a to-be-processed image. The projection transformation is first projection transformation or second projection transformation. A second intermediate image is obtained by performing first projection transformation on a first part image of the to-be-processed image and performing second projection transformation on a second part image of the to-be-processed image. The first part image comprises a head region, and the second part image comprises a body region. A deformed image is obtained according to the to-be-processed image, the first intermediate image and the second intermediate image. In the image, the head region and the body region are both subjected to deformed processing, and the fusion effect is good, realistic and natural.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method and device, an electronic device, and a readable storage medium. Background Art

[0002] As electronic devices become increasingly powerful in terms of camera functionality, they are equipped with multiple cameras for users to take photos. Generally, in addition to a standard camera, electronic devices also include a telephoto camera or a wide-angle camera. Wide-angle cameras use lenses with shorter focal lengths than standard lenses, but with a wider field of view. Therefore, using a wide-angle camera to capture images with a wider field of view can produce images, but the images will be severely distorted.

[0003] In order to compensate for the severe distortion of the image, it is necessary to perform distortion correction on the image. However, traditional distortion correction methods cannot effectively eliminate the distortion. Summary of the Invention

[0004] The embodiments of the present application provide an image processing method and device, an electronic device, and a readable storage medium, which can effectively eliminate distortion in an image.

[0005] An image processing method, comprising:

[0006] Performing a projection transformation on the image to be processed to obtain a first intermediate image; the projection transformation is a first projection transformation or a second projection transformation;

[0007] performing the first projective transformation on a first partial image of the image to be processed, and performing the second projective transformation on a second partial image of the image to be processed, to obtain a second intermediate image; the first partial image includes a head region, and the second partial image includes a body region;

[0008] A dedistorted image is obtained according to the image to be processed, the first intermediate image, and the second intermediate image.

[0009] An image processing device, comprising:

[0010] a first intermediate image generation module, configured to perform a projective transformation on the image to be processed to obtain a first intermediate image; the projective transformation being a first projective transformation or a second projective transformation; and a second intermediate image generation module, configured to perform a first projective transformation on a first partial image of the image to be processed and a second projective transformation on a second partial image of the image to be processed to obtain a second intermediate image; the first partial image including a head region and the second partial image including a body region.

[0011] The image generation module is configured to generate a de-distorted image according to the to-be-processed image, the first intermediate image and the second intermediate image.

[0012] An electronic device includes a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the Bluetooth communication method.

[0013] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the Bluetooth communication method.

[0014] The image processing method and device, the electronic device and the computer readable storage medium project the to-be-processed image to obtain the first intermediate image; the projection transformation is the first projection transformation or the second projection transformation. The first part of the to-be-processed image is subjected to the first projection transformation, and the second part of the to-be-processed image is subjected to the second projection transformation to obtain the second intermediate image; the first part of the to-be-processed image includes the head region, and the second part of the to-be-processed image includes the body region. The de-distorted image is obtained according to the to-be-processed image, the first intermediate image and the second intermediate image.

[0015] Because the projection transformation is the first projection transformation or the second projection transformation, the de-distortion effect of the head region subjected to the first projection transformation is better, and the de-distortion effect of the body region subjected to the second projection transformation is better. Therefore, the projection transformation of the to-be-processed image can eliminate the distortion of the head region or the body region in the to-be-processed image. The first part of the to-be-processed image is subjected to the first projection transformation, and the second part of the to-be-processed image is subjected to the second projection transformation to obtain the second intermediate image. That is, the head region or the body region in the second intermediate image is also subjected to the same projection transformation, so that not only the head region or the body region in the first intermediate image which has not been eliminated can be eliminated in the second intermediate image, but also the second intermediate image can be well fused with the first intermediate image. Finally, according to the to-be-processed image, the first intermediate image and the second intermediate image, the image subjected to the de-distortion of the head region and the body region can be obtained, and the fusion effect is good and natural. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 An application environment diagram of the image processing method in one embodiment;

[0018] Figure 2 A flowchart of the image processing method in one embodiment;

[0019] Figure 3 For Figure 2 A flowchart of the method for generating a second intermediate image in one embodiment;

[0020] Figure 4 For Figure 2 A flowchart of another method for generating a second intermediate image in one embodiment;

[0021] Figure 5 For Figure 2 A flowchart of the method for generating a de-distorted image in one embodiment;

[0022] Figure 6 A schematic diagram of the image processing method in one embodiment;

[0023] Figure 7 A schematic diagram of the method for dividing a to-be-processed image into a first partial image and a second partial image in one embodiment;

[0024] Figure 8 A schematic diagram of the method for dividing a to-be-processed image into a first partial image and a second partial image in another embodiment;

[0025] Figure 9 A schematic diagram of the image processing method in one specific embodiment;

[0026] Figure 10 A block diagram of the structure of the image processing device in one embodiment;

[0027] Figure 11 A schematic diagram of the internal structure of the electronic device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0029] With the increasingly powerful photographing function on electronic devices, multiple cameras are configured on electronic devices for users to take pictures. Generally, in addition to a standard camera, a long-focus camera or a wide-angle camera is also configured on an electronic device. Taking pictures with a wide-angle camera can obtain an image with a larger field of view, but the edges of the image will be severely distorted, especially for a portrait at the edge of the image.

[0030] Among them, a traditional image processing method can eliminate the distortion of portraits by applying spherical projection transformation to the image, but it cannot effectively eliminate the distortion of the background area, so that straight lines in the background area appear obviously curved and part of the field of view angle is lost.

[0031] Another traditional image processing method, using cylindrical projection, can eliminate distortion along the horizontal axis of the portrait, but distortion still exists in other directions. The background area also experiences noticeable vertical curvature, which also results in a loss of field of view.

[0032] Figure 1 FIG. 1 is an application scenario diagram of an image processing method in an embodiment. Figure 1 As shown, the application environment includes an electronic device 120, which performs a projection transformation on the image to be processed to obtain a first intermediate image; the projection transformation is a first projection transformation or a second projection transformation; a first projection transformation is performed on a first portion of the image to be processed, and a second projection transformation is performed on a second portion of the image to be processed to obtain a second intermediate image; the first portion of the image includes a head area, and the second portion of the image includes a body area; a dedistorted image is obtained based on the image to be processed, the first intermediate image, and the second intermediate image. Here, the electronic device 120 can be any terminal device with a camera function, such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a wearable device (smart bracelet, smart watch, smart glasses), a smart home, or the like.

[0033] Figure 2 FIG. 1 is a flow chart of an image processing method in one embodiment. The image processing method in this embodiment is executed on Figure 1 The electronic device 120 in FIG. 1 is used as an example for description. The image processing method includes:

[0034] Step 220 , performing a projective transformation on the image to be processed to obtain a first intermediate image; the projective transformation is a first projective transformation or a second projective transformation.

[0035] The electronic device uses a wide-angle camera to capture the original image. Due to the wide-angle camera's large field of view, the edges of the image will be distorted, especially for portraits at the edge of the image, the distortion is more obvious. The image to be processed is an image obtained by pre-processing the image to be processed. Specifically, the image to be processed can be obtained by optically dedistorting the original image, or by denoising, enhancing, and other processing on the original image. This application does not limit this. The distortion of the portrait at the edge of the image on the image to be processed is still very obvious.

[0036] Therefore, it is necessary to perform the distortion removal on the portrait region on the to-be-processed image, and the portrait region on the to-be-processed image includes a head region and a body region. The head region includes a head region and a neck region, and the body region includes a region corresponding to a limb and a trunk. Of course, this is only one way of dividing the portrait region into a head region and a neck region, and in other embodiments, the portrait region can be divided into a head region and a body region according to other division manners, which is not limited in the present application.

[0037] In order to remove the distortion of the portrait region at the edge of the to-be-processed image, first, the to-be-processed image is subjected to a projection transformation to obtain a first intermediate image. The projection transformation is mainly used to generate new coordinates by projecting the coordinates of the pixel points in the image. The distortion in the image can be eliminated by the projection transformation. Here, the projection transformation is a first projection transformation or a second projection transformation, and the distortion removal effect of the first projection transformation on the head region is better, and the distortion removal effect of the second projection transformation on the body region is better. That is, the projection transformation can be the first projection transformation or the second projection transformation. In other words, when the to-be-processed image is subjected to the projection transformation to obtain the first intermediate image, the to-be-processed image can be subjected to the first projection transformation to obtain the first intermediate image, or the to-be-processed image can be subjected to the second projection transformation to obtain the second intermediate image. The present application does not make any limitation, and one of the above two manners can be selected. If the to-be-processed image is subjected to the first projection transformation to obtain the first intermediate image, the head region in the first intermediate image will have a better distortion removal effect. If the to-be-processed image is subjected to the second projection transformation to obtain the first intermediate image, the body region in the first intermediate image will have a better distortion removal effect.

[0038] In step 240, the first part image of the to-be-processed image is subjected to the first projection transformation, and the second part image of the to-be-processed image is subjected to the second projection transformation to obtain a second intermediate image. The first part image includes a head region, and the second part image includes a body region.

[0039] Specifically, the electronic device can divide the to-be-processed image into a first part image and a second part image, wherein the first part image includes a head region, and the second part image includes a body region. Then, different projection transformations are performed on the first part image and the second part image, respectively, that is, the first part image of the to-be-processed image is subjected to the first projection transformation, and the second part image of the to-be-processed image is subjected to the second projection transformation to obtain a second intermediate image. In this way, the head region in the first part image can be better removed from the distortion, and the body region in the second part image can be better removed from the distortion.

[0040] Meanwhile, the first projection transformation or the second projection transformation is performed on the to-be-processed image to obtain a first intermediate image, and then the first projection transformation is performed on the first partial image of the to-be-processed image and the second projection transformation is performed on the second partial image of the to-be-processed image to obtain a second intermediate image. Therefore, the head region in the first partial image or the body region in the second partial image has undergone the same projection transformation as the projection transformation, and thus a better fusion effect can be achieved when the first intermediate image and the second intermediate image are fused.

[0041] In step 260, a de-distorted image is obtained according to the to-be-processed image, the first intermediate image and the second intermediate image.

[0042] The first intermediate image is obtained by performing the first projection transformation or the second projection transformation on the to-be-processed image, and thus the head region or the body region in the to-be-processed image can be de-distorted.

[0043] The second intermediate image is obtained by performing the first projection transformation on the first partial image of the to-be-processed image and performing the second projection transformation on the second partial image of the to-be-processed image, and thus the head region or the body region in the first intermediate image that has not been de-distorted can be de-distorted in the second intermediate image.

[0044] Therefore, the electronic device obtains a de-distorted image according to the to-be-processed image, the first intermediate image and the second intermediate image. Specifically, the corresponding region in the to-be-processed image can be replaced by the de-distorted head region or body region in the first intermediate image. For example, if the de-distorted region in the first intermediate image is the head region, the head region in the to-be-processed image is replaced by the de-distorted head region in the first intermediate image. Then, the body region in the to-be-processed image is replaced by the body region in the second intermediate image. Thus, after the to-be-processed image is replaced twice, a de-distorted image can be obtained.

[0045] If the de-distorted region in the first intermediate image is the body region, the body region in the to-be-processed image is replaced by the de-distorted body region in the first intermediate image. Then, the head region in the to-be-processed image is replaced by the head region in the second intermediate image. Similarly, after the to-be-processed image is replaced twice, a de-distorted image can be obtained.

[0046] Moreover, in the process of replacing the to-be-processed image twice, because the head region in the first partial image or the body region in the second partial image has undergone the same projection transformation as the projection transformation, when the region in the to-be-processed image is replaced by the partial region in the first intermediate image and the second intermediate image and fused, a better fusion effect can be achieved.

[0047] In the embodiments of the present application, because the projection transformation is the first projection transformation or the second projection transformation, the electronic device can eliminate the distortion of the head region or the body region in the to-be-processed image by performing the projection transformation on the to-be-processed image. The first projection transformation is performed on the first part of the to-be-processed image, and the second projection transformation is performed on the second part of the to-be-processed image, to obtain a second intermediate image. That is, the head region or the body region in the second intermediate image is also subjected to the same projection transformation, so that the distortion of the head region or the body region in the first intermediate image that has not been eliminated can be eliminated in the second intermediate image, and the second intermediate image can be better fused with the first intermediate image. Finally, according to the to-be-processed image, the first intermediate image and the second intermediate image, an image in which the head region and the body region are subjected to the distortion elimination processing and has a better fusion effect and is more realistic and natural can be obtained.

[0048] In one embodiment, the first projection transformation is a spherical projection transformation, and the second projection transformation is a cylindrical projection transformation.

[0049] Specifically, the projection transformation is mainly used to perform projection transformation on the coordinates of the pixel points in the original image to generate new coordinates, and the distortion in the original image can be eliminated through the projection transformation. The spherical projection transformation refers to the process of coordinate transformation between the spherical coordinate system and the standard Euclidean coordinate system of a point in the Euclidean space. The cylindrical projection transformation refers to the coordinate transformation process of projecting a two-dimensional image onto a three-dimensional cylinder.

[0050] The function (formula) corresponding to the spherical projection transformation is as follows:

[0051] r0=d / (2.0*tan(0.5*atan(d / (2.0*f))));

[0052] rp=sqrt(x^2+y^2);

[0053] ru=r0*tan(0.5*atan(rp / f));

[0054] Theta=tan(y / x);

[0055] x’=ru*sin(theta);

[0056] y’=ru*cos(theta);

[0057] Wherein, d = min(W, H), W is the width of the image, H is the height of the image; r0 is the scaling factor. x is the horizontal coordinate of the pixel point on the image, y is the vertical coordinate of the pixel point on the image, x' is the new X coordinate generated after the spherical projection transformation of the image, y' is the new Y coordinate generated after the spherical projection transformation of the image. rp is the radial distance between any point on the image and the center of the image, ru is the radial distance between any point on the image and the center of the image after the spherical projection transformation; f is the distortion correction strength.

[0058] Wherein, the cylindrical projection transformation corresponds to the function (formula) of the cylindrical projection transformation, the formula of the cylindrical projection transformation in the horizontal axis direction (X direction) is as follows:

[0059] x1 = r0 (tan (0.5 * atan (x / f))) ;

[0060] Wherein, r0 = d / (2 * tan (0.5 * atan (d / f))).

[0061] Wherein, the function of the cylindrical projection transformation in the vertical axis direction (Y direction) is:

[0062] y1 = r0 (tan (0.5 * atan (y / f))).

[0063] Wherein, r0 = d / (2 * tan (0.5 * atan (d / f)).

[0064] Wherein, d is the short side length of the image, r0 is the scaling factor, x is the horizontal coordinate of the pixel point on the image, y is the vertical coordinate of the pixel point on the image, x1 is the new X coordinate generated after the cylindrical projection transformation of the image, y1 is the new Y coordinate generated after the cylindrical projection transformation of the image. r0 can make the short side length consistent before and after the cylindrical projection transformation of the image. f is the correction strength coefficient, f can be set according to the lens parameters, or can be fine-tuned according to the user's effect of distortion removal.

[0065] In the embodiment of the application, if the first projection transformation is the spherical projection transformation and the second projection transformation is the cylindrical projection transformation, in step 220, the electronic device performs the spherical projection transformation or the cylindrical projection transformation on the to-be-processed image. In step 240, the electronic device performs the spherical projection transformation on the first part of the to-be-processed image and performs the cylindrical projection transformation on the second part of the to-be-processed image to obtain the second intermediate image. Because the spherical projection transformation can achieve better distortion removal for the head region, and the cylindrical projection transformation can achieve better distortion removal for the body region. Therefore, finally, the electronic device can obtain the image in which the head region and the body region are both subjected to distortion removal according to the to-be-processed image, the first intermediate image and the second intermediate image.

[0066] In the previous embodiment, the first projection transformation is a spherical projection transformation, and the second projection transformation is a cylindrical projection transformation. As shown in FIG. 2, in step 240, the first projection transformation is performed on the first part of the image to be processed, and the second projection transformation is performed on the second part of the image to be processed to obtain a second intermediate image, including: Figure 3

[0067] In step 242, the spherical projection transformation is performed on the first part of the image to be processed to obtain the first part of the image after the spherical projection transformation.

[0068] Specifically, the electronic device can divide the image to be processed into the first part of the image and the second part of the image, where the first part of the image includes the head region, and the second part of the image includes the body region. Then, different projection transformations are performed on the first part of the image and the second part of the image, that is, the spherical projection transformation is performed on the first part of the image to be processed to obtain the first part of the image after the spherical projection transformation. In this way, the head region in the first part of the image can be better eliminated from distortion.

[0069] In step 244, the cylindrical projection transformation is performed on the second part of the image to be processed to obtain the second part of the image after the cylindrical projection transformation.

[0070] The electronic device can perform the cylindrical projection transformation on the second part of the image to be processed to obtain the second intermediate image. In this way, the body region in the second part of the image can be better eliminated from distortion. Specifically, the formula of the cylindrical projection transformation can be used to perform the cylindrical projection transformation on the second part of the image to be processed. The formula of the cylindrical projection transformation in the horizontal axis direction (X direction) can be used to perform the cylindrical projection transformation on the second part of the image to be processed in the horizontal axis direction. The formula of the cylindrical projection transformation in the vertical axis direction (Y direction) can be used to perform the cylindrical projection transformation on the second part of the image to be processed in the vertical axis direction. The cylindrical projection transformation can also be performed on the second part of the image to be processed in the horizontal axis direction and the vertical axis direction. The present application does not limit this.

[0071] In step 246, the second intermediate image is obtained based on the first part of the image after the spherical projection transformation and the second part of the image after the cylindrical projection transformation.

[0072] After obtaining the first part of the image after the spherical projection transformation and the second part of the image after the cylindrical projection transformation, the electronic device combines the first part of the image after the spherical projection transformation and the second part of the image after the cylindrical projection transformation to obtain the second intermediate image. Specifically, the first part of the image after the spherical projection transformation and the second part of the image after the cylindrical projection transformation can be spliced to obtain the second intermediate image.

[0073] ​In the embodiments of the present application, the electronic device divides the to-be-processed image into a first partial image and a second partial image, wherein the first partial image includes a head region, and the second partial image includes a body region. Then, different projection transformations are respectively performed on the first partial image and the second partial image, that is, spherical projection transformation is performed on the first partial image of the to-be-processed image, and cylindrical projection transformation is performed on the second partial image of the to-be-processed image to obtain a second intermediate image. In this way, the head region in the first partial image can be better eliminated from distortion, and the body region in the second partial image can be better eliminated from distortion. Finally, the first partial image after the spherical projection transformation and the second partial image after the cylindrical projection transformation are combined to obtain the second intermediate image.

[0074] However, when the to-be-processed image is divided into the first partial image and the second partial image, a straight line division method is generally used. If the to-be-processed image includes multiple portrait regions, the straight line division method cannot guarantee that all head regions in the to-be-processed image are divided into the first partial image, and cannot guarantee that all body regions are divided into the second partial image. Therefore, if the head region in the first intermediate image is eliminated from distortion, the head region in the to-be-processed image is replaced by the head region in the first intermediate image that is eliminated from distortion. At this time, the body region in the to-be-processed image is replaced by the body region in the second intermediate image. Thus, after the to-be-processed image is replaced twice, a deformed image can be obtained.

[0075] If the body region in the first intermediate image is eliminated from distortion, the body region in the to-be-processed image is replaced by the body region in the first intermediate image that is eliminated from distortion. At this time, the head region in the to-be-processed image is replaced by the head region in the second intermediate image. Similarly, after the to-be-processed image is replaced twice, a deformed image can be obtained.

[0076] Moreover, in the process of replacing the to-be-processed image twice, because the head region in the first partial image or the body region in the second partial image has undergone the same projection transformation as the projection transformation, when the electronic device replaces the region in the to-be-processed image with the partial region in the first intermediate image and the second intermediate image and performs fusion, a better fusion effect can be obtained.

[0077] In one embodiment, before the cylindrical projection transformation is performed on the second partial image of the to-be-processed image to obtain the second partial image after the cylindrical projection transformation, the method comprises:

[0078] performing spherical projection transformation on the second partial image of the to-be-processed image to obtain the second partial image after the spherical projection transformation.

[0079] For example, Figure 4As shown, the first part of the image to be processed is subjected to a first projection transformation, and the second part of the image to be processed is subjected to a second projection transformation to obtain a second intermediate image, comprising:

[0080] In step 420, the first part of the image to be processed is subjected to a spherical projection transformation;

[0081] In step 440, the second part of the image to be processed is subjected to a spherical projection transformation to obtain a second part of the image after the spherical projection transformation;

[0082] In step 460, the second part of the image after the spherical projection transformation is subjected to a cylindrical projection transformation to obtain a second part of the image after the cylindrical projection transformation;

[0083] In step 480, the second intermediate image is obtained based on the first part of the image after the spherical projection transformation and the second part of the image after the cylindrical projection transformation.

[0084] Specifically, if the first projection transformation is a spherical projection transformation and the second projection transformation is a cylindrical projection transformation, then when the electronic device subjects the first part of the image to be processed to the first projection transformation and subjects the second part of the image to be processed to the second projection transformation to obtain the second intermediate image, first, the first part of the image to be processed is subjected to a spherical projection transformation; second, the second part of the image to be processed is subjected to a spherical projection transformation to obtain a second part of the image after the spherical projection transformation; third, the second part of the image after the spherical projection transformation is subjected to a cylindrical projection transformation in the horizontal axis direction to obtain a second part of the image after the cylindrical projection transformation; and finally, the second intermediate image is obtained based on the first part of the image after the spherical projection transformation and the second part of the image after the cylindrical projection transformation.

[0085] The electronic device adds a step of subjecting the second part of the image to be processed to a spherical projection transformation to obtain a second part of the image after the spherical projection transformation before subjecting the second part of the image after the spherical projection transformation to a cylindrical projection transformation in the horizontal axis direction to obtain a second part of the image after the cylindrical projection transformation. This added step can be used to also subject the second part of the image to a spherical projection transformation to eliminate distortion of the second part of the image in the vertical axis direction. Subjecting the second part of the image after the spherical projection transformation to a cylindrical projection transformation in the horizontal axis direction can eliminate distortion of the second part of the image in the horizontal direction. Thus, for the second part of the image, distortion in the horizontal axis direction and the vertical axis direction can be eliminated.

[0086] In the embodiments of the present application, the electronic device increases the spherical projection transformation on the second part of the image to be processed before performing the cylindrical projection transformation on the second part of the image after the spherical projection transformation, to obtain the second part of the image after the cylindrical projection transformation. For the second part of the image, the spherical projection transformation is used to eliminate the distortion in the vertical axis direction, and the cylindrical projection transformation is used to eliminate the distortion in the horizontal axis direction. Thus, the distortion in the horizontal axis direction and the vertical axis direction of the second part of the image can be eliminated.

[0087] In one embodiment, step 260, obtaining the de-distorted image according to the image to be processed, the first intermediate image and the second intermediate image, comprises:

[0088] The first region is obtained from the first intermediate image, and the second region is obtained from the second intermediate image; the first region and the second region constitute the portrait region.

[0089] The de-distorted image is obtained according to the image to be processed, the first region and the second region.

[0090] The first intermediate image is an image obtained by performing the first projection transformation or the second projection transformation on the image to be processed, which can eliminate the distortion of the head region or the body region in the image to be processed.

[0091] The second intermediate image is an image obtained by performing the first projection transformation on the first part of the image to be processed and performing the second projection transformation on the second part of the image to be processed. In the second intermediate image, the distortion of the head region or the body region that has not been eliminated in the first intermediate image can be eliminated.

[0092] The electronic device divides the portrait region in the image to be processed into the first region and the second region, i.e., the first region and the second region constitute the portrait region. For example, the first region is the head region, and the second region is the body region; or the first region is the body region, and the second region is the head region. Therefore, the de-distorted image is obtained according to the image to be processed, the first intermediate image and the second intermediate image, specifically, the first region obtained by the projection transformation is obtained from the first intermediate image, and the second region is obtained from the second intermediate image. The first region and the second region constitute the portrait region, so the de-distorted image can be obtained according to the image to be processed, the first region and the second region.

[0093] In the embodiment of the present application, the electronic device obtains the first region from the first intermediate image and the second region from the second intermediate image when obtaining the de-distorted image according to the to-be-processed image, the first intermediate image and the second intermediate image. Then, the de-distorted image is obtained according to the to-be-processed image, the first region and the second region. The first region and the second region are both de-distorted regions, and the first region and the second region jointly constitute the portrait region. Therefore, the de-distorted image can be obtained according to the to-be-processed image, the first region and the second region. The image includes a complete de-distorted portrait region, and the de-distortion processing is simultaneously performed on the head region and the body region.

[0094] According to the previous embodiment, the de-distorted image is obtained according to the to-be-processed image, the first region and the second region, including:

[0095] The region corresponding to the first region in the to-be-processed image is replaced by the first region, and the region corresponding to the second region in the to-be-processed image is replaced by the second region, to obtain the de-distorted image.

[0096] Specifically, when obtaining the de-distorted image according to the to-be-processed image, the first region and the second region, the electronic device replaces the region corresponding to the first region in the to-be-processed image by the first region, and replaces the region corresponding to the second region in the to-be-processed image by the second region, to obtain the de-distorted image.

[0097] Because the first region and the second region are both de-distorted regions, and the first region and the second region jointly constitute the portrait region, the region corresponding to the first region in the to-be-processed image can be replaced by the first region, and the region corresponding to the second region in the to-be-processed image can be replaced by the second region, to obtain the de-distorted image. The image includes a complete de-distorted portrait region, and the de-distortion processing is simultaneously performed on the head region and the body region.

[0098] In the embodiment of the present application, the electronic device replaces the region corresponding to the first region in the to-be-processed image by the first region, and replaces the region corresponding to the second region in the to-be-processed image by the second region, to obtain the de-distorted image. In this way, the first region in the image is a de-distorted region, and the second region in the image is also a de-distorted region. Therefore, the de-distortion processing is simultaneously performed on the portrait region, and because the first region or the second image has undergone the same projection transformation as the projection transformation, a good fusion effect can be obtained when the regions in the to-be-processed image are replaced and fused.

[0099] In one embodiment, as Figure 5As shown in the figure, the region corresponding to the first region in the to-be-processed image is replaced with the first region, and the region corresponding to the second region in the to-be-processed image is replaced with the second region, to obtain a de-distorted image, including:

[0100] In step 520, the to-be-processed image, the first intermediate image, and the second intermediate image are respectively divided to obtain grid maps; the grid map includes grid points.

[0101] In combination Figure 6 As shown in the figure, it is a schematic diagram of an image processing method in an embodiment. Figure 6 In (a), the to-be-processed image obtained and the grid map divided for the to-be-processed image are shown. Specifically, the to-be-processed image is uniformly divided into grid points to obtain a grid map. Assuming that the pixel resolution of the to-be-processed image is 3000*4000, the to-be-processed image is uniformly divided into 31*41 grid points to obtain a grid map, and the size of each grid in the grid map is 10*10.

[0102] Figure 6 In (b), the grid map of the first intermediate image is obtained according to the grid map of the to-be-processed image. Here, the first intermediate image is an image obtained by performing spherical projection transformation on the to-be-processed image. From Figure 6 In the figure corresponding to (b), it can be seen that the grids in the grid map of the first intermediate image are not uniformly distributed.

[0103] Similarly, the second intermediate image is divided into grid points. The specific schematic diagram of dividing the second intermediate image into grid points is not shown in the figure.

[0104] In step 540, the coordinates of the grid points of the region corresponding to the first region in the to-be-processed image are updated to the coordinates of the grid points of the first region, and the coordinates of the grid points of the region corresponding to the second region in the to-be-processed image are updated to the coordinates of the grid points of the second region, to obtain a new grid map of the to-be-processed image.

[0105] Specifically, the electronic device uses an image recognition algorithm to perform image segmentation on the to-be-processed image to obtain a head region and a body region. The coordinates of the original grid points corresponding to the head region or the coordinates of the original grid points corresponding to the body region are obtained from the grid map of the to-be-processed image. Then, the coordinates of the grid points of the first region are obtained from the grid map of the first intermediate image. The coordinates of the original grid points corresponding to the head region or the body region in the to-be-processed image are updated to the coordinates of the grid points of the first region, to obtain an updated first grid map of the to-be-processed image. That is, as shown in Figure 6 In (c).

[0106] The coordinates of the original grid points corresponding to the head region or the coordinates of the original grid points corresponding to the body region are obtained from the first grid map of the to-be-processed image. The coordinates of the grid points of the second region are obtained from the grid map of the second intermediate image. The coordinates of the original grid points corresponding to the head region or the body region in the to-be-processed image are updated to the coordinates of the grid points of the second region, to obtain an updated second grid map of the to-be-processed image. Here, the updated second grid map is the new grid map of the to-be-processed image. The schematic diagram for obtaining the updated second grid map is not shown in the figure.

[0107] In step 560, interpolation operation is performed on the pixels corresponding to the new grid map to obtain the deformed image.

[0108] In the new grid map of the to-be-processed image, the grid points of the first region and the grid points of the second region may not have corresponding pixel points in the to-be-processed image. Therefore, it is necessary to perform interpolation calculation based on the pixels in the to-be-processed image and the new grid points to generate the pixels corresponding to the grid points of the first region and the grid points of the second region. Finally, the deformed image is generated according to the pixels corresponding to the new grid map and the pixel values thereof. The interpolation calculation method includes any one of the nearest neighbor interpolation method, the bilinear interpolation method, and the cubic polynomial interpolation method. The present application does not limit this.

[0109] In the embodiment of the present application, the electronic device replaces in the manner of the grid map, which is accurate and easy to operate. Thus, the coordinates of the grid points of the region corresponding to the first region in the to-be-processed image are updated to the coordinates of the grid points of the first region, and the coordinates of the grid points of the region corresponding to the second region in the to-be-processed image are updated to the coordinates of the grid points of the second region, to obtain the new grid map of the to-be-processed image. Moreover, in the process of replacing the to-be-processed image twice, because the head region in the first part of the image or the body region in the second part of the image has undergone the same projection transformation as the projection transformation, when the region in the to-be-processed image is replaced by the part of the region in the first intermediate image and the second intermediate image and is fused, a good fusion effect can be achieved.

[0110] In one embodiment, if the projection transformation is the first projection transformation, the first region is the head region, and the second region is the body region.

[0111] The region corresponding to the first region in the to-be-processed image is replaced by the first region, and the region corresponding to the second region in the to-be-processed image is replaced by the second region, to obtain the deformed image, including:

[0112] The head region in the to-be-processed image is replaced by the head region in the first intermediate image, and the body region in the to-be-processed image is replaced by the body region in the second intermediate image, to obtain the deformed image.

[0113] Specifically, the projection transformation is a first projection transformation, wherein the first projection transformation is a spherical projection transformation, and the second projection transformation is a cylindrical projection transformation. Because the first intermediate image is obtained by performing a spherical projection transformation on the image to be processed, distortion of the head region in the first intermediate image is effectively eliminated. Therefore, the first region obtained from the first intermediate image is the head region, and the second region obtained from the second intermediate image is the body region. When obtaining a dedistorted image based on the image to be processed, the first region, and the second region, the head region in the image to be processed is replaced with the head region in the first intermediate image, and the body region in the image to be processed is replaced with the body region in the second intermediate image, thereby obtaining the dedistorted image.

[0114] In the embodiments of the present application, it is known that the head region in an image can be dedistorted by spherical projection transformation, and the non-head region in an image can be dedistorted by cylindrical projection transformation. If the projection transformation is a first projection transformation, and the first projection transformation is a spherical projection transformation, and the second projection transformation is a cylindrical projection transformation, then a spherical projection transformation is performed on the image to be processed to obtain a first intermediate image, in which the head region is effectively dedistorted. A spherical projection transformation is performed on the first portion of the image to be processed, and a cylindrical projection transformation is performed on the second portion of the image to be processed to obtain a second intermediate image; the first portion includes the head region, and the second portion includes the body region. Then, in the second intermediate image, the body region in the second portion is effectively dedistorted. Therefore, the head region in the image to be processed is replaced with the head region in the first intermediate image, and the body region in the image to be processed is replaced with the body region in the second intermediate image, to obtain a dedistorted image, thereby achieving simultaneous dedistortion of both the head and body regions in the image to be processed.

[0115] In one embodiment, if the projection transformation is the second projection transformation, the first region is the body region, and the second region is the head region;

[0116] Replacing a region corresponding to the first region in the image to be processed with the first region, and replacing a region corresponding to the second region in the image to be processed with the second region, to obtain a dedistorted image, including:

[0117] The body region in the image to be processed is replaced by the body region in the first intermediate image, and the head region in the image to be processed is replaced by the head region in the second intermediate image, to obtain a dedistorted image.

[0118] Specifically, the projection transformation is the second projection transformation, and the first projection transformation is a spherical projection transformation, and the second projection transformation is a cylindrical projection transformation. Because the cylindrical projection transformation is performed on the to-be-processed image to obtain the first intermediate image, the body region in the first intermediate image is well deformed. Therefore, the first region obtained from the first intermediate image is the body region, and the second region obtained from the second intermediate image is the head region. When the deformed image is obtained according to the to-be-processed image, the first region and the second region, the body region in the to-be-processed image is replaced by the body region in the first intermediate image, and the head region in the to-be-processed image is replaced by the head region in the second intermediate image, to obtain the deformed image.

[0119] In the embodiment of the present application, it is known that the spherical projection transformation can realize the deforming of the head region in the image, and the cylindrical projection transformation can realize the deforming of the non-head region in the image. If the projection transformation is the second projection transformation, and the first projection transformation is the spherical projection transformation, and the second projection transformation is the cylindrical projection transformation. Then, the cylindrical projection transformation is performed on the to-be-processed image to obtain the first intermediate image, and the body region in the first intermediate image is well deformed. The spherical projection transformation is performed on the first part of the to-be-processed image, and the cylindrical projection transformation is performed on the second part of the to-be-processed image to obtain the second intermediate image; the first part of the image includes the head region, and the second part of the image includes the body region. Then, the head region in the first part of the image in the second intermediate image is well deformed. Therefore, the body region in the to-be-processed image is replaced by the body region in the first intermediate image, and the head region in the to-be-processed image is replaced by the head region in the second intermediate image to obtain the deformed image, which realizes the deforming of the head region and the body region in the to-be-processed image.

[0120] In one embodiment, if the projection transformation is the first projection transformation, before the first projection transformation is performed on the first part of the to-be-processed image, and the second projection transformation is performed on the second part of the to-be-processed image to obtain the second intermediate image, it includes:

[0121] The to-be-processed image is divided into a first part of the image and a second part of the image; the first part of the image includes the head region and part of the body region connected with the head region, and the second part of the image includes the other body region except the part of the body region.

[0122] Specifically, if the projection transformation is the first projection transformation, and the first projection transformation is a spherical projection transformation, the second projection transformation is a cylindrical projection transformation. Because the spherical projection transformation is performed on the to-be-processed image to obtain the first intermediate image, the head region is well eliminated from distortion in the first intermediate image. Then, the to-be-processed image is divided into a first partial image and a second partial image, and a straight line is generally used for division. If the to-be-processed image includes multiple portrait regions, the straight line division method cannot guarantee that all head regions in the to-be-processed image are divided into the first partial image, and cannot guarantee that all body regions are divided into the second partial image. For example, the first partial image includes the head region and part of the body region (for example, the shoulder region) connected with the head region, and the second partial image includes the other body regions except the part of the body region. As shown in FIG. 7, Figure 7 It can be seen from the figure that the image is divided into the first partial image 720 and the second partial image 740 by using a horizontal straight line. Figure 7 The head region of the first person on the left and the head region of the person on the right are both divided into the first partial image.

[0123] In this way, the electronic device performs the spherical projection transformation on the head region and part of the body region connected with the head region in the to-be-processed image. The cylindrical projection transformation is performed on the other body regions except the part of the body region in the to-be-processed image. Although the effect of removing distortion by performing the spherical projection transformation on the part of the body region connected with the head region is not as good as the effect of removing distortion by performing the cylindrical projection transformation, because the head region in the first intermediate image and the part of the body region connected with the head region in the second intermediate image are both obtained by using the spherical projection transformation, when the head region in the to-be-processed image is replaced by the head region in the first intermediate image, and the body region in the to-be-processed image is replaced by the body region (the part of the body region connected with the head region and the other body regions) in the second intermediate image, the image after removing distortion is obtained, and the fusion effect is good. The reason is that the part of the body region connected with the head region in the second intermediate image is obtained by using the spherical projection transformation, and the head region in the first intermediate image is also obtained by using the spherical projection transformation. Because the same kind of projection transformation is used, when the part of the body region connected with the head region is connected with the head region in the to-be-processed image, the fusion effect is good.

[0124] In the embodiments of the present application, if the projection transformation is the first projection transformation and the first projection transformation is a spherical projection transformation, the second projection transformation is a cylindrical projection transformation. The electronic device performs a spherical projection transformation on the to-be-processed image to obtain a first intermediate image, in which the distortion of the head region is well eliminated. Then, the to-be-processed image is divided into a first partial image and a second partial image; the first partial image includes the head region and a part of the body region connected with the head region, and the second partial image includes other body regions except the part of the body region. Next, the first projection transformation is performed on the first partial image of the to-be-processed image, and the second projection transformation is performed on the second partial image of the to-be-processed image to obtain a second intermediate image. In this way, the distortion of the other body regions in the second partial image can be well eliminated. When the part of the body region connected with the head region is connected with the head region in the to-be-processed image, the fusion effect is good.

[0125] In one embodiment, if the projection transformation is the second projection transformation, before the first projection transformation is performed on the first partial image of the to-be-processed image and the second projection transformation is performed on the second partial image of the to-be-processed image to obtain a second intermediate image, the method comprises:

[0126] dividing the to-be-processed image into a first partial image and a second partial image; the second partial image includes a body region and a part of a head region connected with the body region, and the first partial image includes other head regions except the part of the head region.

[0127] Specifically, if the projection transformation is the second projection transformation and the first projection transformation is a spherical projection transformation, the second projection transformation is a cylindrical projection transformation. Because the spherical projection transformation is performed on the to-be-processed image to obtain a first intermediate image, the distortion of the body region in the first intermediate image is well eliminated. Then, the to-be-processed image is divided into a first partial image and a second partial image, and a straight line division method is generally used. If the to-be-processed image includes multiple portrait regions, the straight line division method cannot guarantee that all the head regions in the to-be-processed image are divided into the first partial image, and cannot guarantee that all the body regions are divided into the second partial image. For example, the divided second partial image includes a body region and a part of a head region connected with the body region, and the first partial image includes other head regions except the part of the head region. As shown in FIG. 8, Figure 8 the image is divided into a first partial image 820 and a second partial image 840 by using a horizontal straight line. As can be seen from the figure, Figure 8 the body region of the second person on the left side and the part of the head region connected with the body region are both divided into the first partial image.

[0128] Thus, the electronic device performs cylindrical projection transformation on the body region and the partial head region connected with the body region in the to-be-processed image. The electronic device performs spherical projection transformation on the head region other than the partial head region in the to-be-processed image. Although the cylindrical projection transformation on the partial head region connected with the body region does not have as good a de-distortion effect as the spherical projection transformation, since the body region in the first intermediate image and the partial head region connected with the body region in the second intermediate image are both obtained by using the spherical projection transformation, when the body region in the to-be-processed image is replaced by the body region in the first intermediate image and the head region in the to-be-processed image is replaced by the head region (the partial head region connected with the body region and the head region other than the partial head region) in the second intermediate image, the obtained de-distorted image has a better fusion effect. The reason is that the partial head region connected with the body region in the second intermediate image is obtained by using the cylindrical projection transformation, and the body region in the first intermediate image is also obtained by using the cylindrical projection transformation. Since the same kind of projection transformation is used, when the partial head region connected with the body region in the to-be-processed image is connected with the body region, the fusion effect is better.

[0129] In the embodiment, if the projection transformation is the second projection transformation and the first projection transformation is the spherical projection transformation, the second projection transformation is the cylindrical projection transformation. The electronic device performs cylindrical projection transformation on the to-be-processed image to obtain the first intermediate image, and the body region in the first intermediate image is better de-distorted. Then, the to-be-processed image is divided into a first partial image and a second partial image. The second partial image includes the body region and the partial head region connected with the body region, and the first partial image includes the head region other than the partial head region. Next, the first projection transformation is performed on the first partial image of the to-be-processed image, and the second projection transformation is performed on the second partial image of the to-be-processed image to obtain the second intermediate image. Thus, the head region other than the partial head region in the second partial image can be better de-distorted. When the partial head region connected with the body region in the to-be-processed image is connected with the body region, the fusion effect is better.

[0130] In one embodiment, before the interpolation operation is performed on the pixels corresponding to the new mesh map to obtain the de-distorted image, the following operations are included:

[0131] The new mesh map is optimized to obtain an optimized mesh map.

[0132] The interpolation operation is performed on the pixels corresponding to the new mesh map to obtain the de-distorted image.

[0133] The interpolation operation is performed on the pixels corresponding to the optimized mesh map to obtain the de-distorted image.

[0134] Specifically, the electronic device updates the coordinates of the grid points of the region in the to-be-processed image corresponding to the first region to the coordinates of the grid points of the first region, and updates the coordinates of the grid points of the region in the to-be-processed image corresponding to the second region to the coordinates of the grid points of the second region, to obtain a new grid map of the to-be-processed image.

[0135] Then, the electronic device performs optimization processing on the new grid map to obtain an optimized grid map. The optimization processing can be performed in any one or more of the following manners: for example, performing smoothing processing on the new grid map to obtain the optimized grid map; performing continuity processing on the lines of the same object region in the new grid map to obtain the optimized grid map; controlling a difference between the stretching parameters of the background region in the new grid map in the horizontal axis direction and the vertical axis direction to be less than a second preset difference threshold; and controlling the grid points of the edge region in the new grid map to remain straight lines. The optimized grid map can achieve effects such as keeping the grid in the image smooth, keeping the straight lines continuous, and preventing the background region from collapsing.

[0136] Therefore, an image after distortion correction can be generated according to the pixels and pixel values corresponding to the optimized grid map.

[0137] In the embodiments of the present application, the electronic device performs optimization processing on the new grid map, which can achieve effects such as keeping the grid in the image smooth, keeping the straight lines continuous, and preventing the background region from collapsing. Therefore, the image after distortion correction generated according to the pixels and pixel values corresponding to the optimized grid map has a better distortion correction effect.

[0138] In one embodiment, the optimization processing on the new grid map includes at least one of the following manners: performing smoothing processing on the new grid map to obtain the optimized grid map; performing continuity processing on the lines of the same object region in the new grid map to obtain the optimized grid map; controlling a difference between the stretching parameters of the background region in the new grid map in the horizontal axis direction and the vertical axis direction to be less than a second preset difference threshold; and controlling the grid points of the edge region in the new grid map to remain straight lines.

[0139] In the embodiments of the present application, the electronic device performs optimization processing on the new grid map in at least one of the above manners, which can achieve effects such as keeping the grid in the image smooth, keeping the straight lines continuous, and preventing the background region from collapsing. Therefore, the image after distortion correction generated according to the pixels and pixel values corresponding to the optimized grid map has a better distortion correction effect.

[0140] In one embodiment, before the projection transformation is performed on the to-be-processed image to obtain the first intermediate image, the following steps are included:

[0141] The optical distortion correction is performed on the original image to obtain the to-be-processed image.

[0142] The original image is an image obtained by using a wide-angle camera to take a picture. Due to a large field of view of the wide-angle camera, the edge of the image is distorted, especially the portrait at the edge of the image is more obviously distorted. Optical distortion refers to a distortion degree of an image formed by an optical system relative to an object itself. The optical distortion refers to a deformation degree obtained by optical calculation. The optical distortion includes barrel distortion, pincushion distortion and linear distortion. The barrel distortion, also referred to as barrel distortion, is a distortion phenomenon that the image formed by the optical system is barrel-shaped and expanded. The barrel distortion is common in imaging of a photographic lens, especially a wide-angle lens. The pincushion distortion, also referred to as pincushion distortion, is a phenomenon that the image formed by the optical system is contracted to the middle. The pincushion distortion is common in imaging of a long-focus lens.

[0143] In the embodiment of the application, when the electronic device corrects the image, first, the barrel distortion, the pincushion distortion and the linear distortion on the original image are corrected by optical distortion correction to obtain a to-be-processed image. Then, the to-be-processed image is projected and transformed to obtain a first intermediate image. The first part of the to-be-processed image is projected and transformed to obtain a second intermediate image. The to-be-processed image, the first intermediate image and the second intermediate image are used to obtain a de-distorted image. Thus, the optical distortion on the original image is also eliminated, and the de-distortion effect of the image is improved.

[0144] In one specific embodiment, as shown in FIG. 1, Figure 9 An image processing method is provided, including:

[0145] In step 902, the original image is optically distorted to obtain a to-be-processed image.

[0146] In step 904, the to-be-processed image is spherically projected and transformed to obtain a first intermediate image.

[0147] In step 906, the to-be-processed image is divided into an upper half image and a lower half image from a head region at the lowest position in the vertical axis direction.

[0148] In step 908, the upper half image of the to-be-processed image is spherically projected and transformed to obtain a spherically projected and transformed upper half image.

[0149] In step 910, the lower half image of the to-be-processed image is spherically projected and transformed to obtain a spherically projected and transformed lower half image.

[0150] At step 912, the lower half image after the spherical projection transformation is subjected to a cylindrical projection transformation to obtain a lower half image after the cylindrical projection transformation.

[0151] At step 914, a second intermediate image is obtained based on the upper half image after the spherical projection transformation and the lower half image after the cylindrical projection transformation.

[0152] At step 916, a grid map is obtained by dividing the to-be-processed image, the first intermediate image and the second intermediate image respectively; the grid map includes grid points.

[0153] At step 918, the coordinates of the grid points of the head region in the to-be-processed image are updated to the coordinates of the grid points of the head region in the first intermediate image, and the coordinates of the grid points of the body region in the to-be-processed image are updated to the coordinates of the grid points of the body region in the second intermediate image to obtain a new grid map of the to-be-processed image.

[0154] At step 920, the new grid map is subjected to an optimization process to obtain an optimized grid map.

[0155] At step 922, an interpolation operation is performed on the pixels corresponding to the optimized grid map to obtain a de-distorted image.

[0156] In the embodiments of the present application, it is known that the head region in the image can be de-distorted by spherical projection transformation, and the non-head region in the image can be de-distorted by cylindrical projection transformation. Therefore, the electronic device subjects the to-be-processed image to spherical projection transformation to obtain a first intermediate image, and the head region in the first intermediate image is better de-distorted. The upper half image of the to-be-processed image is subjected to spherical projection transformation, and the lower half image of the to-be-processed image is subjected to spherical projection transformation and cylindrical projection transformation in sequence to obtain a second intermediate image; the upper half image includes the head region, and the lower half image includes the body region. Therefore, the body region in the lower half image in the second intermediate image is better de-distorted. Therefore, the head region in the to-be-processed image is replaced by the head region in the first intermediate image, and the body region in the to-be-processed image is replaced by the body region in the second intermediate image to obtain a de-distorted image, which realizes de-distortion of the head region and the body region in the to-be-processed image at the same time. And when the body region and the head region in the to-be-processed image are connected, the fusion effect is better.

[0157] In one embodiment, as shown in Figure 10 an image processing apparatus 1000 is provided, which comprises:

[0158] A first intermediate image generation module 1020 is configured to subject the to-be-processed image to a projection transformation to obtain a first intermediate image; the projection transformation is a first projection transformation or a second projection transformation.

[0159] The second intermediate image generation module 1040 is configured to perform first projection transformation on the first partial image of the to-be-processed image, perform second projection transformation on the second partial image of the to-be-processed image, and obtain a second intermediate image; the first partial image includes a head region, and the second partial image includes a body region.

[0160] The image generation module 1060 is configured to obtain a de-warping image according to the to-be-processed image, the first intermediate image, and the second intermediate image.

[0161] In an embodiment, the first projection transformation is spherical projection transformation, and the second projection transformation is cylindrical projection transformation. The second intermediate image generation module 1040 is configured to perform spherical projection transformation on the first partial image of the to-be-processed image to obtain a first partial image after spherical projection transformation; perform cylindrical projection transformation on the second partial image of the to-be-processed image to obtain a second partial image after cylindrical projection transformation; and obtain the second intermediate image based on the first partial image after spherical projection transformation and the second partial image after cylindrical projection transformation.

[0162] In an embodiment, the second intermediate image generation module 1040 is configured to, before performing cylindrical projection transformation on the second partial image of the to-be-processed image to obtain the second partial image after cylindrical projection transformation, perform spherical projection transformation on the second partial image of the to-be-processed image to obtain a second partial image after spherical projection transformation.

[0163] In an embodiment, the image generation module 1060 includes:

[0164] The region acquisition unit is configured to acquire a first region from the first intermediate image and acquire a second region from the second intermediate image; the first region and the second region constitute a portrait region.

[0165] The de-warping unit is configured to obtain a de-warping image according to the to-be-processed image, the first region, and the second region.

[0166] In an embodiment, the de-warping unit is further configured to replace a region corresponding to the first region in the to-be-processed image with the first region, and replace a region corresponding to the second region in the to-be-processed image with the second region, to obtain the de-warping image.

[0167] In one embodiment, the de-warping unit is further configured to divide the to-be-processed image, the first intermediate image and the second intermediate image into grid maps respectively; the grid map comprises grid points; coordinates of the grid points in a region corresponding to the first region in the to-be-processed image are updated to coordinates of the grid points in the first region, and coordinates of the grid points in a region corresponding to the second region in the to-be-processed image are updated to coordinates of the grid points in the second region, to obtain a new grid map of the to-be-processed image; and interpolation operation is performed on pixels corresponding to the new grid map to obtain the de-warping image.

[0168] In one embodiment, if the projection transformation is the first projection transformation, the first region is a head region and the second region is a body region; the de-warping unit is further configured to replace the head region in the to-be-processed image with the head region in the first intermediate image and replace the body region in the to-be-processed image with the body region in the second intermediate image to obtain the de-warping image.

[0169] In one embodiment, if the projection transformation is the second projection transformation, the first region is a body region and the second region is a head region; the de-warping unit is further configured to replace the body region in the to-be-processed image with the body region in the first intermediate image and replace the head region in the to-be-processed image with the head region in the second intermediate image to obtain the de-warping image.

[0170] In one embodiment, an image processing apparatus is further provided, and the image processing apparatus comprises:

[0171] The first image division module is configured to divide the to-be-processed image into a first partial image and a second partial image; the first partial image comprises a head region and a partial body region connected to the head region, and the second partial image comprises other body regions except the partial body region.

[0172] In one embodiment, an image processing apparatus is further provided, and the image processing apparatus comprises:

[0173] The second image division module is configured to divide the to-be-processed image into a first partial image and a second partial image; the second partial image comprises a body region and a partial head region connected to the body region, and the first partial image comprises other head regions except the first partial head region.

[0174] In one embodiment, the de-warping unit is further configured to, before performing interpolation operation on pixels corresponding to the new grid map to obtain the de-warping image, comprise: performing optimization processing on the new grid map to obtain an optimized grid map; and the de-warping unit is further configured to perform interpolation operation on pixels corresponding to the optimized grid map to obtain the de-warping image.

[0175] In an embodiment, the de-distortion unit is further configured to perform an optimization process on the new mesh to obtain an optimized mesh, including at least one of the following manners:

[0176] performing a smoothing process on the new mesh to obtain an optimized mesh;

[0177] performing a continuity process on lines of the same object region in the new mesh to obtain an optimized mesh;

[0178] controlling a difference between stretching parameters of the background region in the new mesh in the horizontal axis direction and the vertical axis direction to be less than a second preset difference threshold;

[0179] controlling the grid points of the edge region in the new mesh to remain straight lines.

[0180] In an embodiment, an image processing apparatus is further provided, and the image processing apparatus further includes:

[0181] an optical distortion correction module configured to perform optical distortion correction on the original image to obtain a to-be-processed image.

[0182] It should be understood that, although each step in the flowchart in the above figure is displayed in sequence according to the indication of the arrow, these steps are not necessarily executed in sequence according to the indication of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the above figure can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0183] The division of each module in the above image processing apparatus is only used for illustration, and in other embodiments, the image processing apparatus can be divided into different modules as needed to complete all or part of the functions of the above image processing apparatus.

[0184] For specific limitations of the image processing apparatus, refer to the limitations of the image processing method in the above text, which will not be repeated here. Each module in the above image processing apparatus can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each of the above modules.

[0185] In one embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of an image processing method provided in the above embodiments.

[0186] Figure 11 FIG. 1 is a schematic diagram of the internal structure of an electronic device in one embodiment. Figure 11 As shown, the electronic device includes a processor and a memory connected via a system bus. The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement an image processing method provided in each of the above embodiments. The internal memory provides a high-speed cache operating environment for the operating system computer program in the non-volatile storage medium. The electronic device can be any terminal device such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a vehicle-mounted computer, a wearable device, etc.

[0187] The various modules in the image processing apparatus provided in the embodiments of the present application may be implemented in the form of a computer program. The computer program may be run on an electronic device or an electronic device. The program modules comprising the computer program may be stored in the electronic device or a memory of the electronic device. When the computer program is executed by a processor, the steps of the method described in the embodiments of the present application are implemented.

[0188] The present application also provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of the image processing method.

[0189] A computer program product comprising instructions which, when run on a computer, cause the computer to perform an image processing method.

[0190] Any reference to storage, memory, database or other medium herein can include non-volatile and / or volatile storage. Suitable non-volatile storage can include read-only memory (ROM), programmable ROM (PROM), electronically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile storage can include random access memory (RAM), which acts as external cache. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct memory access (RDMA), and Rambus in-memory direct computer architecture (RIM). The embodiments of the present application can be implemented as computer programs or program modules. As used herein, the terms "computer program medium" and "computer usable medium" are used to generally refer to media such as removable storage drive 820, a hard disk installed in hard disk drive 810, and signals. These computer program products are means for providing software to program or code for the various processes of the embodiments of the present application. The software can be embodied in one or more computer programs that can be executed a processing unit 810.

[0191] The above image processing embodiments are merely descriptive and illustrative of several ways to make and use the present application and do not limit the scope of the application. It is to be understood that numerous other modifications and changes can be devised by those skilled in the art which will reduce to practice without departing from the spirit of the application as disclosed herein. It is therefore desired that the scope of the application be measured only by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Performing a projection transformation on the image to be processed to obtain a first intermediate image; the projection transformation is a first projection transformation or a second projection transformation; performing the first projective transformation on a first partial image of the image to be processed, and performing the second projective transformation on a second partial image of the image to be processed, to obtain a second intermediate image; the first partial image includes a head region, and the second partial image includes a body region; Obtaining a dedistorted image based on the image to be processed, the first intermediate image, and the second intermediate image; The step of obtaining a dedistorted image based on the image to be processed, the first intermediate image, and the second intermediate image includes: Acquire a first region from the first intermediate image, and acquire a second region from the second intermediate image; the first region and the second region constitute a portrait region; replacing the area corresponding to the first area in the image to be processed with the first area, and replacing the area corresponding to the second area in the image to be processed with the second area, to obtain the dedistorted image; The replacing the area corresponding to the first area in the image to be processed with the first area, and replacing the area corresponding to the second area in the image to be processed with the second area, to obtain the dedistorted image, includes: Dividing the image to be processed, the first intermediate image, and the second intermediate image into grid maps respectively; the grid maps include grid points; updating the coordinates of the grid points in the area of ​​the image to be processed corresponding to the first area with the coordinates of the grid points in the first area, and updating the coordinates of the grid points in the area of ​​the image to be processed corresponding to the second area with the coordinates of the grid points in the second area, to obtain a new grid map of the image to be processed; An interpolation operation is performed based on the pixels corresponding to the new grid image to obtain the dedistorted image.

2. The image processing method according to claim 1, wherein: The first projection transformation is a spherical projection transformation, and the second projection transformation is a cylindrical projection transformation; performing the first projection transformation on the first portion of the image to be processed and performing the second projection transformation on the second portion of the image to be processed to obtain a second intermediate image includes: Performing the spherical projection transformation on the first partial image of the image to be processed to obtain the first partial image after the spherical projection transformation; Performing the cylindrical projection transformation on the second partial image of the image to be processed to obtain the second partial image after the cylindrical projection transformation; A second intermediate image is obtained based on the first partial image after the spherical projection transformation and the second partial image after the cylindrical projection transformation.

3. The image processing method according to claim 2, wherein: Before performing the cylindrical projection transformation on the second partial image of the image to be processed to obtain the second partial image after the cylindrical projection transformation, the method includes: The spherical projection transformation is performed on the second partial image of the image to be processed to obtain the second partial image after the spherical projection transformation.

4. The image processing method according to claim 1, wherein: If the projection transformation is the first projection transformation, the first region is the head region, and the second region is the body region; The replacing the area corresponding to the first area in the image to be processed with the first area, and replacing the area corresponding to the second area in the image to be processed with the second area, to obtain the dedistorted image, includes: The head region in the image to be processed is replaced by the head region in the first intermediate image, and the body region in the image to be processed is replaced by the body region in the second intermediate image, to obtain the dedistorted image.

5. The image processing method according to claim 1, wherein: If the projection transformation is the second projection transformation, the first region is a body region, and the second region is a head region; The replacing the area corresponding to the first area in the image to be processed with the first area, and replacing the area corresponding to the second area in the image to be processed with the second area, to obtain a dedistorted image, includes: The body region in the image to be processed is replaced by the body region in the first intermediate image, and the head region in the image to be processed is replaced by the head region in the second intermediate image, to obtain the dedistorted image.

6. The image processing method according to claim 1, wherein: If the projective transformation is a first projective transformation, before performing the first projective transformation on the first portion of the image to be processed and performing the second projective transformation on the second portion of the image to be processed to obtain the second intermediate image, the method includes: The image to be processed is divided into a first partial image and a second partial image; the first partial image includes a head area and a partial body area connected to the head area, and the second partial image includes other body areas excluding the partial body area.

7. The image processing method according to claim 1, wherein: If the projective transformation is a second projective transformation, before performing the first projective transformation on the first portion of the image to be processed and performing the second projective transformation on the second portion of the image to be processed to obtain the second intermediate image, the method includes: The image to be processed is divided into a first partial image and a second partial image; the second partial image includes a body area and a partial head area connected to the body area, and the first partial image includes other head areas excluding the partial head area.

8. The image processing method according to claim 1, wherein: Before performing an interpolation operation according to pixels corresponding to the new grid image to obtain the dedistorted image, the method includes: Optimizing the new grid map to obtain an optimized grid map; The interpolation operation is performed according to the pixels corresponding to the new grid image to obtain the dedistorted image, including: An interpolation operation is performed based on the pixels corresponding to the optimized grid map to obtain the dedistorted image.

9. The image processing method according to claim 8, characterized in that: The optimizing the new grid map to obtain an optimized grid map includes at least one of the following methods: Smoothing the new grid map to obtain an optimized grid map; Performing continuity processing on the lines of the same object area in the new grid map to obtain an optimized grid map; Controlling the difference between the stretching parameters of the background area in the horizontal and vertical directions in the new grid image to be smaller than a second preset difference threshold; The grid points in the edge area of ​​the new grid map are controlled to maintain a straight line.

10. The image processing method according to claim 1, wherein: Before performing the projective transformation on the image to be processed to obtain the first intermediate image, the method includes: Optical distortion correction is performed on the original image to obtain the image to be processed.

11. An image processing device, characterized in that: The device comprises: A first intermediate image generation module is configured to perform a projection transformation on the image to be processed to obtain a first intermediate image; the projection transformation is a first projection transformation or a second projection transformation; a second intermediate image generating module, configured to perform a first projective transformation on a first partial image of the image to be processed, and perform a second projective transformation on a second partial image of the image to be processed, to obtain a second intermediate image; the first partial image includes a head region, and the second partial image includes a body region; an image generation module, configured to obtain a dedistorted image based on the image to be processed, the first intermediate image, and the second intermediate image; The image generation module is specifically configured to: Acquire a first region from the first intermediate image, and acquire a second region from the second intermediate image; the first region and the second region constitute a portrait region; replacing the area corresponding to the first area in the image to be processed with the first area, and replacing the area corresponding to the second area in the image to be processed with the second area, to obtain the dedistorted image; The replacing the area corresponding to the first area in the image to be processed with the first area, and replacing the area corresponding to the second area in the image to be processed with the second area, to obtain the dedistorted image, includes: Dividing the image to be processed, the first intermediate image, and the second intermediate image into grid maps respectively; the grid maps include grid points; updating the coordinates of the grid points in the area of ​​the image to be processed corresponding to the first area with the coordinates of the grid points in the first area, and updating the coordinates of the grid points in the area of ​​the image to be processed corresponding to the second area with the coordinates of the grid points in the second area, to obtain a new grid map of the image to be processed; An interpolation operation is performed based on the pixels corresponding to the new grid image to obtain the dedistorted image.

12. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the image processing method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 10 are implemented.

Images