Image processing method and device, electronic equipment and computer readable storage medium

Abstract

The present disclosure relates to an image processing method, device, electronic equipment and computer readable storage medium. The method comprises: obtaining a plurality of feature points corresponding to a target contour in an image; the plurality of feature points are used to represent the target contour; performing edge detection on the image to determine an edge detection image; determining at least one edge point based on the plurality of feature points and the edge detection image; the at least one edge point is a point outside the plurality of feature points and used to represent the target contour; fitting the plurality of feature points and the at least one edge point to determine a first contour fitting result; and correcting the first contour fitting result to determine a corrected image. The present disclosure provides a technical solution for obtaining a corrected image with better authenticity and naturalness, improving the authenticity and naturalness of the image, avoiding image distortion, and achieving better correction effect.

Description

Technical Field

[0001] This disclosure relates to image processing technology, and more particularly to image processing methods, apparatus, electronic devices, and computer-readable storage media. Background Technology

[0002] With the widespread adoption of mobile devices, photo and video recording applications have developed rapidly. To meet people's daily needs, more and more photo and video recording applications offer image enhancement functions, such as face slimming functions, which correct facial images to make the user's face appear smaller. However, in related technologies, distortion and deformation often occur during image correction, resulting in a lack of realism in the corrected image and poor correction effects. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, electronic device, and computer-readable storage medium to at least solve the problem of poor image correction results in related technologies. The technical solution of this disclosure is as follows:

[0004] According to a first aspect of the present disclosure, an image processing method is provided, comprising:

[0005] Multiple feature points corresponding to the target contour in the image are obtained; the multiple feature points are used to characterize the target contour.

[0006] Perform edge detection on the image to determine the edge-detected image;

[0007] Based on the plurality of feature points and the edge detection image, at least one edge point is determined; the at least one edge point is a point outside the plurality of feature points used to characterize the target contour;

[0008] Fit the plurality of feature points and the at least one edge point to determine the first contour fitting result;

[0009] The first contour fitting result is corrected to determine the corrected image.

[0010] Optionally, determining at least one edge point based on the plurality of feature points and the edge detection image includes:

[0011] Based on a pre-set first selection step size, at least one selection position is determined on the line connecting two adjacent feature points;

[0012] For each selected location, based on the edge detection image, determine the edge intensity value corresponding to each pixel point within the target pixel region corresponding to the selected location;

[0013] Among all pixels, the target pixel corresponding to the edge intensity value that meets the preset selection conditions is selected as the edge point.

[0014] Optionally, the step of correcting the first contour fitting result and determining the corrected image includes:

[0015] Based on the obtained correction coefficients, the first contour fitting result is adjusted to determine the second contour fitting result; the correction coefficients are used to indicate the correction direction and correction magnitude.

[0016] Based on a pre-set second selection step size, contour points are selected on the second contour fitting result to determine multiple corrected contour points of the target contour. The multiple corrected contour points are used to characterize the corrected target contour.

[0017] Based on the multiple corrected contour points, the corrected image is determined.

[0018] Optionally, determining the corrected image based on the plurality of corrected contour points includes:

[0019] Based on the multiple corrected contour points, a polygonal mesh is established;

[0020] The image is deformed based on the polygonal mesh to obtain a corrected image.

[0021] Optionally, fitting the plurality of feature points and the at least one edge point to determine the first contour fitting result includes:

[0022] Based on the encoding information corresponding to the plurality of feature points, two target feature points are selected from the plurality of feature points whose encoding information matches the preset encoding information corresponding to the target contour; the encoding information is used to identify the plurality of feature points;

[0023] Based on the line connecting the two target feature points, a first coordinate axis is determined; based on the perpendicular line connecting the two target feature points, a second coordinate axis is determined; based on the first coordinate axis and the second coordinate axis, a target coordinate system is established.

[0024] Determine the coordinate information of the plurality of feature points and the at least one edge point in the target coordinate system;

[0025] The coordinate information is fitted to determine the first contour fitting result.

[0026] Optionally, obtaining multiple feature points corresponding to the target contour in the image includes:

[0027] Perform contour detection on the acquired image to determine the object's contour points;

[0028] Determine the angle of the object in the image;

[0029] Based on the object angle, multiple feature points are selected from the object contour points to correct the target contour in the object contour.

[0030] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising:

[0031] The feature point acquisition module is configured to acquire multiple feature points corresponding to the target contour in the image; the multiple feature points are used to characterize the target contour.

[0032] An edge detection module is configured to perform edge detection on the image and determine an edge-detected image;

[0033] An edge point determination module is configured to determine at least one edge point based on the plurality of feature points and the edge detection image; the at least one edge point is a point outside the plurality of feature points used to characterize the target contour;

[0034] The fitting processing module is configured to fit the plurality of feature points and the at least one edge point to determine a first contour fitting result.

[0035] The correction processing module is configured to correct the first contour fitting result and determine the corrected image.

[0036] Optionally, the edge point determination module includes:

[0037] The position selection unit is configured to determine at least one selection position on the line connecting two adjacent feature points based on a pre-set first selection step size.

[0038] The intensity value determination unit is configured to determine, for each selected location, the edge intensity value corresponding to each pixel point within the target pixel region corresponding to the selected location based on the edge detection image;

[0039] The edge point selection unit is configured to select, from each pixel point, the target pixel point corresponding to the edge intensity value that meets the preset selection conditions as the edge point.

[0040] Optionally, the correction processing module includes:

[0041] The adjustment processing unit is configured to adjust the first contour fitting result based on the acquired correction coefficients to determine the second contour fitting result; the correction coefficients are used to indicate the correction direction and correction magnitude.

[0042] The contour point determination unit is configured to select contour points on the second contour fitting result based on a preset second selection step size, and determine multiple corrected contour points of the target contour, wherein the multiple corrected contour points are used to characterize the corrected target contour.

[0043] The result determination unit is configured to determine the corrected image based on the plurality of corrected contour points.

[0044] Optionally, the result determination unit includes:

[0045] The mesh creation sub-cells are configured to create a polygonal mesh based on the plurality of corrected contour points;

[0046] The resulting sub-unit is configured to deform the image based on the polygonal mesh to obtain a corrected image.

[0047] Optionally, the fitting processing module includes:

[0048] The matching processing unit is configured to select two target feature points from the plurality of feature points based on the encoding information corresponding to the plurality of feature points, wherein the encoding information matches the preset encoding information corresponding to the target contour; the encoding information is used to identify the plurality of feature points;

[0049] The coordinate system establishment unit is configured to determine a first coordinate axis based on the line connecting the two target feature points; determine a second coordinate axis based on the perpendicular line connecting the two target feature points; and establish a target coordinate system based on the first coordinate axis and the second coordinate axis.

[0050] An information determination unit is configured to determine the coordinate information of the plurality of feature points and the at least one edge point in the target coordinate system;

[0051] The fitting processing unit is configured to fit the coordinate information to determine the first contour fitting result.

[0052] Optionally, the feature point acquisition module includes:

[0053] The detection processing unit is configured to perform contour detection on the acquired image and determine the object contour points;

[0054] An angle determination unit is configured to determine the angle of an object in the image;

[0055] The feature point selection unit is configured to select multiple feature points from the object contour points based on the object angle for correcting the target contour in the object contour.

[0056] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0057] processor;

[0058] Memory used to store the processor's executable instructions;

[0059] The processor is configured to execute the instructions to implement the image processing method as described in the first aspect.

[0060] According to a fourth aspect of the present disclosure, a computer storage medium is provided that, when an instruction processor in the storage medium is executed, enables an electronic device to perform the image processing method as described in the first aspect.

[0061] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program or computer instructions, wherein the computer program or computer instructions, when executed by a processor, implement the image processing method described in the first aspect.

[0062] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0063] This disclosure obtains multiple feature points corresponding to the target contour in an image, performs edge detection on the image to determine an edge detection image, and then determines at least one edge point based on the obtained feature points and the determined edge detection image. This at least one edge point is a point outside the multiple feature points in the image used to represent the target contour, and it is a result of comprehensively considering both feature points and the edge detection image, resulting in high accuracy. Further, the multiple feature points and at least one edge point are fitted to determine a first contour fitting result, which corresponds to the target contour before correction. The first contour fitting result is then corrected to determine the corrected image. In the technical solution provided by this disclosure, not only are multiple feature points used to represent the target contour, but at least one edge point is also used to accurately represent the target contour, effectively increasing the number of points and obtaining a more accurate first contour fitting result. This first contour fitting result has good fit with the target contour, thus the corrected image obtained using this first contour fitting result has good realism and naturalness. The method of this disclosure improves the realism and naturalness of the image, avoids image distortion, and has a good correction effect.

[0064] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0065] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0066] Figure 1 This is a schematic diagram illustrating the application environment of an image processing method according to an exemplary embodiment;

[0067] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment;

[0068] Figure 3 This is a schematic diagram illustrating the contour detection result in an image processing method according to an exemplary embodiment;

[0069] Figure 4 This is a schematic diagram illustrating edge detection results in an image processing method according to an exemplary embodiment;

[0070] Figure 5 This is a schematic diagram illustrating a plurality of feature points and at least one edge point in an image processing method according to an exemplary embodiment;

[0071] Figure 6 This is a schematic diagram illustrating the fitting result in an image processing method according to an exemplary embodiment;

[0072] Figure 7 This is a schematic diagram illustrating a downsampling result in an image processing method according to an exemplary embodiment;

[0073] Figure 8 This is a schematic diagram of the correction result in an image processing method according to an exemplary embodiment. Figure 1 ;

[0074] Figure 9 This is a schematic diagram of the correction result in an image processing method according to an exemplary embodiment. Figure 2 ;

[0075] Figure 10 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment;

[0076] Figure 11 This is a block diagram illustrating a terminal according to an exemplary embodiment;

[0077] Figure 12 This is a block diagram illustrating a server according to an exemplary embodiment. Detailed Implementation

[0078] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0079] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0080] In one possible application scenario, please refer to Figure 1 The diagram illustrates an application environment for an image processing method according to an exemplary embodiment. The application environment may include a terminal 11 and a server 12, which are connected via a wireless network.

[0081] Terminal 11 can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these. Terminal 11 has client software such as an application (App) installed to provide human-computer interaction functions. The application can be a standalone application or a subroutine within an application. For example, the application can be a photo-taking or video-recording application.

[0082] Server 12 can be a server that provides background services for applications in terminal 11. Server 12 can be a standalone physical server, or a server cluster or distributed system consisting of multiple physical servers.

[0083] Those skilled in the art will understand that the number of terminals described above can be as few as one, dozens, hundreds, or more. This disclosure does not limit the number of terminals or the type of device.

[0084] In another possible implementation scenario, this image processing method can be directly applied to the terminal.

[0085] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment, including the following steps.

[0086] In step S21, multiple feature points corresponding to the target contour in the image are obtained; the multiple feature points are used to characterize the target contour.

[0087] In this step, the target contour is the contour to be corrected in the image. The target contour can be obtained by user specification (i.e., the user selects the contour to be corrected in the image on the terminal) or by pre-setting it, meaning the target contour is automatically corrected during image correction without user specification. Specifically, the image can be a face image, and the target contour can be the overall facial contour, a partial facial contour (such as the jawline contour, cheekbone contour), or the contour of a specific facial feature. The image can also be a pet image, and the target contour can be the pet's overall contour or facial contour; the image can also be a car image, and the target contour can be the car body contour. In this embodiment, the objects in the image are not specifically limited. It should be noted that the image processing method provided in this embodiment is particularly suitable for correcting the jawline contour of a face, i.e., obtaining multiple feature points corresponding to the jawbone region in the face image.

[0088] Multiple feature points are used to characterize the target contour, representing partial contour points of the target contour. Specifically, feature point detection is performed on the image to determine multiple feature points of the target contour. It should be noted that if the image is directly corrected using the obtained multiple feature points, it will be difficult to accurately correct the image due to the small number of feature points and deviations from the true contour points. This will result in distortion and deformation of the corrected image. For example, when correcting facial contours, some parts of the facial contour in the corrected face image may appear sunken, making the corrected result lack realism and naturalness.

[0089] In one embodiment, step S21 includes:

[0090] In step S211, contour detection is performed on the acquired image to determine the object contour points.

[0091] In this step, an image is acquired and input into a pre-trained contour detection model for contour detection, identifying multiple object contour points. For example, if the image is a face image, contour detection is performed on the face image to determine the face contour points. Figure 3 An example is shown: the contour detection result of a face image. Figure 3 The encoded information is 0 ( Figure 3 (near the middle right eye) to 18 ( Figure 3 The point near the left ear is the defined facial contour point.

[0092] In step S212, the angle of the object in the image is determined.

[0093] In this step, the position of objects in the image is detected, and the object angle is determined. The object angle indicates whether the object's outline has been deflected; if so, the direction and angle of deflection are further displayed. Different deflection directions will cause varying degrees of occlusion on the object's outline, thus determining the object angle is beneficial for accurately acquiring multiple feature points subsequently. For example, in the case of a face image, the object angle is the face angle. When the face deflects to the left or right, it will cause different degrees of occlusion on the face outline. Therefore, determining the face angle is beneficial for accurately acquiring multiple feature points subsequently.

[0094] In step S213, based on the object angle, multiple feature points are selected from the object contour points to correct the target contour in the object contour.

[0095] In this step, during image correction, different object angles affect the correction area. Different object angles result in varying degrees of occlusion of the target contour, leading to the selection of multiple feature points from the object contour points. These multiple feature points represent parts of the object contour. In this embodiment, by considering the object angle, multiple feature points are accurately determined.

[0096] For example, when correcting the jawline of a face image, if the face angle shows the left side of the face in front and the right side behind, the jawline of the right side is obscured. Therefore, the correction needs to focus on the jawline of the left side. This involves selecting multiple feature points from the face contour points to correct the left jawline. Specifically, the encoding information corresponding to the face contour points is determined, and the face contour points that match the encoding information corresponding to the left jawline are selected as multiple feature points to participate in subsequent steps. For example... Figure 3 In the shown face image, the left side of the face is in front, and the right side is behind. Therefore, multiple feature points corresponding to the left jawline are selected from the face contour points with encoding information from 0 to 18, i.e., face contour points with encoding information from 9 to 15. If the face angle in the face image shows the right side in front and the left side behind, the jawline of the left side is obscured. Therefore, the jawline contour of the right side is corrected. Thus, multiple feature points selected from the face contour points are used to correct the right jawline contour. Specifically, the encoding information corresponding to the face contour points is determined, and face contour points that match the encoding information corresponding to the right jawline are selected as multiple feature points to participate in subsequent steps.

[0097] In step S22, edge detection is performed on the image to determine the edge-detected image.

[0098] In this step, the edge detection image is used to provide the edge intensity value corresponding to each pixel in the image. For example, processing a face image using the Sobel operator in image filtering yields the following result: Figure 4 The edge detection image shown can, of course, be performed using other algorithms such as Prewitt, Roberts, and Canny.

[0099] It should be noted that since both steps S21 and S22 process the original image, the execution order of steps S21 and S22 is not specifically specified in this embodiment. Step S21 can be executed first, followed by step S22; or step S22 can be executed first, followed by step S21; or steps S21 and S22 can be executed simultaneously.

[0100] In step S23, at least one edge point is determined based on the plurality of feature points and the edge detection image; the at least one edge point is a point outside the plurality of feature points used to characterize the target contour.

[0101] In this step, after acquiring multiple feature points and edge detection images, the multiple feature points are used as an aid, combined with the edge intensity values ​​provided by the edge detection images, to determine at least one edge point. This edge point is a point used to characterize the target contour in addition to the multiple feature points, thereby increasing the number of points used to characterize the target contour and thus obtaining the edge of the target contour more accurately. Figure 5 An example is shown showing the edge of a target contour consisting of at least one edge point and multiple feature points.

[0102] In one embodiment, step 23 includes:

[0103] In step S231, based on a pre-set first selection step size, at least one selection position is determined on the line connecting two adjacent feature points.

[0104] In this step, a first selection step size is preset. For any two adjacent feature points, at least one selection position is determined on the line connecting the two adjacent feature points using the first selection step size. When there is one selection position, the pixel distance between the feature point and the selection position is the first selection step size. When there are two or more selection positions, the pixel distance between two adjacent selection positions is the first selection step size. The first selection step size affects the number of selection positions, and thus the number of edge points: the smaller the first selection step size, the more selection positions are determined, and the more edge points are determined; the larger the first selection step size, the fewer selection positions are determined, and the fewer edge points are determined.

[0105] In step S232, for each selected location, based on the edge detection image, the edge intensity value corresponding to each pixel point in the target pixel region corresponding to the selected location is determined.

[0106] In this step, the target pixel region is the pixel range used for edge point selection. A pre-defined rule for determining the target pixel region is established so that, after determining the selection location, the target pixel region corresponding to that location is obtained according to the rule. Specifically, the rule can be a fixed number of pixel distances adjacent to the selection location on both sides of the line connecting two adjacent feature points, and in the direction perpendicular to the line connecting the two adjacent feature points at the selection location; alternatively, the rule can simply be a fixed number of pixel distances adjacent to the selection location. Based on the edge intensity values ​​corresponding to each pixel point provided by the edge detection image, the edge intensity values ​​corresponding to each pixel point within the target pixel region corresponding to each selection location are determined.

[0107] In step S233, among each pixel, the target pixel corresponding to the edge intensity value that meets the preset selection conditions is selected as the edge point.

[0108] In this step, the preset selection criteria are pre-defined conditions for edge point selection. Pixels that meet the preset selection criteria are target pixels, which are the edge points participating in subsequent steps. The preset selection criteria can be the pixel with the highest edge intensity value, or the pixel with an edge intensity value greater than a set threshold and the smallest distance from the selected pixel location. Of course, other preset selection criteria may exist, but this embodiment does not specifically limit them.

[0109] For example, if the first selection step size is determined to be 10 pixels, for Figure 3 For adjacent feature points 9 and 10, on the line connecting feature points 9 and 10, starting from feature point 9 (which can be set as the starting point), a selection position is determined every 10 pixels. Assuming 100 selection positions are determined, at each selection position, the edge intensity value corresponding to each pixel in the target pixel area of ​​the selection position is determined. The target pixel area is the area on both sides of the line connecting feature points 9 and 10, in the direction perpendicular to the line at the selection position, and 10 pixels away from the selection position. That is, there is a distance of 10 pixels on each side of the line connecting feature points 9 and 10, for a total of 20 pixels. Among the pixels within the 20 pixels, the pixel with the largest edge intensity value is determined as the edge point.

[0110] In step S24, the plurality of feature points and the at least one edge point are fitted to determine the first contour fitting result.

[0111] In this step, after determining multiple feature points and at least one edge point, the feature points and at least one edge point are fitted to determine the first contour fitting result. By fitting a large number of points, the accuracy of the first contour fitting result is high. Specifically, the fitting function can be a quadratic function, and the objective function corresponding to the first contour fitting result can be y = axx + bx + c.

[0112] In one embodiment, step S24 includes:

[0113] In step S241, based on the encoding information corresponding to the plurality of feature points, two target feature points are selected from the plurality of feature points whose encoding information matches the preset encoding information corresponding to the target contour; the encoding information is used to identify the plurality of feature points.

[0114] In this step, when determining multiple feature points corresponding to the target contour, different feature points have different encoded information, such as... Figure 3 Feature points 9 and 10 are included. Specifically, the preset encoding information consists of the maximum and minimum values ​​of the encoding information corresponding to the target contour. Alternatively, the encoding information can be directly specified, such as the preset encoding information for feature points 9 and 15 corresponding to the left mandibular line. Further, from among the multiple feature points, two target feature points whose encoding information matches the preset encoding information are selected.

[0115] In step S242, a first coordinate axis is determined based on the line connecting the two target feature points; a second coordinate axis is determined based on the perpendicular line connecting the two target feature points; and a target coordinate system is established based on the first coordinate axis and the second coordinate axis.

[0116] For example, the target feature points are identified as feature point 9 and feature point 15. The line connecting feature point 9 and feature point 15 is taken as the X-axis, the center point of the line is taken as the origin, and the perpendicular line between feature point 9 and feature point 15 is taken as the Y-axis at the origin, thus establishing an XY target coordinate system.

[0117] In step S243, the coordinate information of the plurality of feature points and the at least one edge point in the target coordinate system is determined.

[0118] In this step, after establishing the target coordinates, coordinate transformation is performed on multiple feature points and at least one edge point to determine the coordinate information of multiple feature points and at least one edge point in the target coordinates.

[0119] In step S244, the coordinate information is fitted to determine the first contour fitting result.

[0120] In this step, fitting the target coordinate system allows for a more accurate first contour fitting result. Figure 6 An example is shown of the first contour fitting result obtained after fitting the coordinate information in the target coordinate system. The first contour fitting result has good smoothness and can accurately represent the target contour.

[0121] In this embodiment, when fitting multiple feature points and at least one edge point, coordinate transformation is first performed on the multiple feature points and at least one edge point. Normally, the multiple feature points and at least one edge point determined in the image correspond to the image's coordinate system. Under this coordinate system, it is difficult to mine the relationship between multiple feature points and at least one edge point. However, after performing coordinate transformation, the relationship between multiple feature points and at least one edge point is clearer in the target coordinate system. Therefore, the accuracy of the fitted first contour is higher.

[0122] In step S25, the first contour fitting result is corrected to determine the corrected image.

[0123] In this step, after obtaining the first contour fitting result, the first contour fitting result is used as the basis for correction to determine the corrected image. Because the first contour fitting result fits the target contour well, the corrected image obtained using the first contour fitting result has good realism and naturalness, improving the realism and naturalness of the image, avoiding image distortion, and achieving a good correction effect.

[0124] In one embodiment, step S25 includes:

[0125] In step S251, the first contour fitting result is adjusted based on the obtained correction coefficients to determine the second contour fitting result; the correction coefficients are used to indicate the correction direction and correction magnitude.

[0126] In this step, correction coefficients are obtained. Specifically, correction coefficients can be preset, meaning that when correcting the target contour, the preset correction coefficients are read; alternatively, correction coefficients can be determined based on received user input information, allowing users to specify correction coefficients to meet different user needs. After obtaining the correction coefficients, the first contour fitting result is adjusted using the obtained correction coefficients to determine the second contour fitting result. For example, the first contour fitting result is y = (a*x*x + b*x + c), and the correction coefficient is lambda. If lambda = 0.5, it indicates a 50% reduction in the mandibular region. The result of calculating the correction coefficient and the first contour fitting result is determined as the second contour fitting result, which is then y = (a*x*x + b*x + c) * lambda.

[0127] In step 252, contour points are selected on the second contour fitting result based on a pre-set second selection step size to determine multiple corrected contour points of the target contour. These multiple corrected contour points are used to characterize the corrected target contour.

[0128] In this step, after determining the second contour fitting result, downsampling is performed based on the second selection step size to effectively improve the calculation speed. Specifically, the second selection step size is preset, for example, 6 pixels. Contour points are selected on the second contour fitting result. Starting from one end of the second contour fitting result, a corrected contour point is selected every second selection step size. Figure 7 An example is shown of the corrected contour points obtained after downsampling, which can accurately represent the target contour.

[0129] In step 253, the corrected image is determined based on the plurality of corrected contour points.

[0130] In this step, after obtaining the corrected contour points, the image can be corrected based on these points to determine the corrected image. Therefore, in this embodiment, after obtaining an accurate second contour fitting result using correction coefficients, the second contour fitting result is downsampled, effectively improving the image processing speed and thus enabling fast and accurate image correction.

[0131] In one embodiment, step 253 includes: establishing a polygonal mesh based on the plurality of corrected contour points; and performing deformation processing on the image based on the polygonal mesh to obtain a corrected image.

[0132] In this embodiment, after determining multiple corrected contour points, a polygonal mesh is created for the corrected contour points. The image is then warped based on the polygonal mesh to obtain the corrected image. Specifically, the polygonal mesh is a triangular mesh.

[0133] For example, Figure 8 Figure a shows the uncorrected face image. Using the jawline contour in the face image as the target contour, the method provided in this embodiment is used to correct the... Figure 8 The image in section a is processed. That is, to obtain... Figure 8 In Figure a, multiple feature points corresponding to the jawline contour are shown, and... Figure 8Edge detection is performed on image a to determine the edge detection image. Then, based on multiple feature points and the edge detection image, at least one edge point is determined. Two target feature points are selected based on the encoding information corresponding to the multiple feature points, and a target coordinate system is established based on these two target feature points. Coordinate transformation is performed on the multiple feature points and at least one edge point to determine their coordinate information in the target coordinate system. The coordinate information of the multiple feature points and at least one edge point in the target coordinate system is fitted to determine the first contour fitting result. Further, correction coefficients are obtained and used to adjust the first contour fitting result to determine the second contour fitting result. The second contour fitting result is downsampled using a second selection step size to determine multiple corrected contour points of the target contour. Then, a polygonal mesh corresponding to the multiple corrected contour points is established, and the image is deformed based on this polygonal mesh to obtain the desired result. Figure 8 The corrected image shown in Figure b. Figure 8 Comparing images a and b in the image, it can be seen that the corrected face image is more natural, more realistic, and has a better sense of three-dimensionality. The corrected image does not show any distortion or deformation, and the correction effect is better.

[0134] Furthermore, Figure 9 Figure a shows the uncorrected face image. Using the jawline contour in the face image as the target contour, the method provided in this embodiment is used to correct the... Figure 9 The image in section a is processed. That is, to obtain... Figure 9 In Figure a, multiple feature points corresponding to the jawline contour are shown, and... Figure 9 Edge detection is performed on image a to determine the edge detection image. Then, based on multiple feature points and the edge detection image, at least one edge point is determined. Two target feature points are selected based on the encoding information corresponding to the multiple feature points, and a target coordinate system is established based on these two target feature points. Coordinate transformation is performed on the multiple feature points and at least one edge point to determine their coordinate information in the target coordinate system. The coordinate information of the multiple feature points and at least one edge point in the target coordinate system is fitted to determine the first contour fitting result. Further, correction coefficients are obtained and used to adjust the first contour fitting result to determine the second contour fitting result. The second contour fitting result is downsampled using a second selection step size to determine multiple corrected contour points of the target contour. Then, a polygonal mesh corresponding to the multiple corrected contour points is established, and the image is deformed based on this polygonal mesh to obtain the desired result. Figure 9 The corrected image shown in Figure b. Figure 9 Comparing images a and b in the image, it can be seen that the corrected face image is more natural, more realistic, and has a better sense of three-dimensionality. The corrected image does not show any distortion or deformation, and the correction effect is better.

[0135] In the above embodiments, multiple feature points corresponding to the target contour in the image are acquired, and edge detection is performed on the image to determine an edge detection image. Then, based on the acquired multiple feature points and the determined edge detection image, at least one edge point is determined. This at least one edge point is a point outside the multiple feature points in the image used to characterize the target contour, and it is a result of comprehensively considering both feature points and the edge detection image, resulting in high accuracy. Further, the multiple feature points and at least one edge point are fitted to determine a first contour fitting result. The first contour fitting result corresponds to the target contour before correction. The first contour fitting result is then corrected to determine the corrected image. In the technical solution provided in this embodiment, not only are multiple feature points used to characterize the target contour, but at least one edge point is also used to accurately characterize the target contour, effectively increasing the number of points and obtaining a more accurate first contour fitting result. The first contour fitting result has good fit with the target contour, thus the corrected image obtained using the first contour fitting result has good realism and naturalness. The method of this embodiment improves the realism and naturalness of the image, avoids image distortion, and has a good correction effect.

[0136] Figure 10 This is a block diagram illustrating an image processing apparatus disposed on a server according to an exemplary embodiment. The apparatus includes a feature point acquisition module 101, an edge detection module 102, an edge point determination module 103, a fitting processing module 104, and a correction processing module 105.

[0137] The feature point acquisition module 101 is configured to acquire multiple feature points corresponding to the target contour in the image; the multiple feature points are used to characterize the target contour.

[0138] The edge detection module 102 is configured to perform edge detection on the image and determine the edge-detected image;

[0139] The edge point determination module 103 is configured to determine at least one edge point based on the plurality of feature points and the edge detection image; the at least one edge point is a point outside the plurality of feature points used to characterize the target contour;

[0140] The fitting processing module 104 is configured to fit the plurality of feature points and the at least one edge point to determine a first contour fitting result.

[0141] The correction processing module 105 is configured to correct the first contour fitting result and determine the corrected image.

[0142] In an exemplary embodiment of this disclosure, the edge point determination module includes:

[0143] The position selection unit is configured to determine at least one selection position on the line connecting two adjacent feature points based on a pre-set first selection step size.

[0144] The intensity value determination unit is configured to determine, for each selected location, the edge intensity value corresponding to each pixel point within the target pixel region corresponding to the selected location based on the edge detection image;

[0145] The edge point selection unit is configured to select, from each pixel point, the target pixel point corresponding to the edge intensity value that meets the preset selection conditions as the edge point.

[0146] In an exemplary embodiment of this disclosure, the correction processing module includes:

[0147] The adjustment processing unit is configured to adjust the first contour fitting result based on the acquired correction coefficients to determine the second contour fitting result; the correction coefficients are used to indicate the correction direction and correction magnitude.

[0148] The contour point determination unit is configured to select contour points on the second contour fitting result based on a preset second selection step size, and determine multiple corrected contour points of the target contour, wherein the multiple corrected contour points are used to characterize the corrected target contour.

[0149] The result determination unit is configured to determine the corrected image based on the plurality of corrected contour points.

[0150] In an exemplary embodiment of this disclosure, the result determination unit includes:

[0151] The mesh creation sub-cells are configured to create a polygonal mesh based on the plurality of corrected contour points;

[0152] The resulting sub-unit is configured to deform the image based on the polygonal mesh to obtain a corrected image.

[0153] In an exemplary embodiment of this disclosure, the fitting processing module includes:

[0154] The matching processing unit is configured to select two target feature points from the plurality of feature points based on the encoding information corresponding to the plurality of feature points, wherein the encoding information matches the preset encoding information corresponding to the target contour; the encoding information is used to identify the plurality of feature points;

[0155] The coordinate system establishment unit is configured to determine a first coordinate axis based on the line connecting the two target feature points; determine a second coordinate axis based on the perpendicular line connecting the two target feature points; and establish a target coordinate system based on the first coordinate axis and the second coordinate axis.

[0156] An information determination unit is configured to determine the coordinate information of the plurality of feature points and the at least one edge point in the target coordinate system;

[0157] The fitting processing unit is configured to fit the coordinate information to determine the first contour fitting result.

[0158] In an exemplary embodiment of this disclosure, the feature point acquisition module includes:

[0159] The detection processing unit is configured to perform contour detection on the acquired image and determine the object contour points;

[0160] An angle determination unit is configured to determine the angle of an object in the image;

[0161] The feature point selection unit is configured to select multiple feature points from the object contour points based on the object angle for correcting the target contour in the object contour.

[0162] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0163] Figure 11 This is a block diagram illustrating an electronic device terminal according to an exemplary embodiment. Terminal 1100 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0164] Reference Figure 11 Terminal 1100 may include one or more of the following components: processing component 1102, memory 1104, power component 1106, multimedia component 1108, audio component 1110, input / output (I / O) interface 1112, sensor component 1114, and communication component 1116.

[0165] Processing component 1102 typically controls the overall operation of terminal 1100, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 1102 may include one or more processors 1120 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1102 may include one or more modules to facilitate interaction between processing component 1102 and other components. For example, processing component 1102 may include a multimedia module to facilitate interaction between multimedia component 1108 and processing component 1102.

[0166] Memory 1104 is configured to store various types of data to support operation on terminal 1100. Examples of this data include instructions for any application or method operating on terminal 1100, contact data, phonebook data, messages, pictures, videos, etc. Memory 1104 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0167] Power supply component 1106 provides power to various components of terminal 1100. Power supply component 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to terminal 1100.

[0168] Multimedia component 1108 includes a screen that provides an output interface between the terminal 1100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1108 includes a front-facing camera and / or a rear-facing camera. When the terminal 1100 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0169] Audio component 1110 is configured to output and / or input audio signals. For example, audio component 1110 includes a microphone (MIC) configured to receive external audio signals when terminal 1100 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1104 or transmitted via communication component 1116. In some embodiments, audio component 1110 also includes a speaker for outputting audio signals.

[0170] I / O interface 1112 provides an interface between processing component 1102 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0171] Sensor assembly 1114 includes one or more sensors for providing state assessments of various aspects of terminal 1100. For example, sensor assembly 1114 can detect the on / off state of terminal 1100, the relative positioning of components such as the display and keypad of terminal 1100, changes in the position of terminal 1100 or a component of terminal 1100, the presence or absence of user contact with terminal 1100, the orientation or acceleration / deceleration of terminal 1100, and temperature changes of terminal 1100. Sensor assembly 1114 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1114 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1114 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0172] Communication component 1116 is configured to facilitate wired or wireless communication between terminal 1100 and other devices. Terminal 1100 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 1116 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1116 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0173] In an exemplary embodiment, terminal 1100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image processing method described above.

[0174] Figure 12 This is a schematic diagram illustrating the structure of an electronic device server according to an exemplary embodiment. (Refer to...) Figure 12Server 1200 includes processing component 1222, which further includes one or more processors, and memory resources represented by memory 1232 for storing instructions, such as application programs, that can be executed by processing component 1222. The application programs stored in memory 1232 may include one or more modules, each corresponding to a set of instructions. Furthermore, processing component 1222 is configured to execute instructions to perform the above-described image processing implementation method.

[0175] Server 1200 may also include a power supply component 1226 configured to perform power management of server 1200, a wired or wireless network interface 1250 configured to connect server 1200 to a network, and an input / output (I / O) interface 1258. Electronic device 1200 can operate on an operating system stored in memory 1232, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0176] In an exemplary embodiment, a storage medium including instructions is also provided. When the instructions in the storage medium are executed by a processor of a server, the server is enabled to perform an image processing method applied to a server or an image processing method applied to a terminal as described in any of the above method embodiments. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0177] In an exemplary embodiment, a computer program product is also provided, comprising readable program code executable by a server's processor to perform the aforementioned image processing method applied to a server, or executable by a terminal's processor to perform the aforementioned image processing method applied to a terminal. Optionally, the program code may be stored in a storage medium of the terminal or server, which may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device. Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0178] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, include: Multiple feature points corresponding to the target contour in the image are obtained; the multiple feature points are used to characterize the target contour. Perform edge detection on the image to determine the edge-detected image; Based on the multiple feature points and the edge detection image, at least one edge point is determined; The at least one edge point is a point outside the plurality of feature points used to characterize the target contour; Fit the plurality of feature points and the at least one edge point to determine the first contour fitting result; The first contour fitting result is corrected to determine the corrected image; The step of fitting the plurality of feature points and the at least one edge point to determine the first contour fitting result includes: Based on the encoding information corresponding to the plurality of feature points, two target feature points are selected from the plurality of feature points whose encoding information matches the preset encoding information corresponding to the target contour; the encoding information is used to identify the plurality of feature points; Based on the line connecting the two target feature points, a first coordinate axis is determined; based on the perpendicular line connecting the two target feature points, a second coordinate axis is determined; based on the first coordinate axis and the second coordinate axis, a target coordinate system is established. Determine the coordinate information of the plurality of feature points and the at least one edge point in the target coordinate system; The coordinate information is fitted to determine the first contour fitting result.

2. The method according to claim 1, characterized in that, Determining at least one edge point based on the plurality of feature points and the edge detection image includes: Based on a pre-set first selection step size, at least one selection position is determined on the line connecting two adjacent feature points; For each selected location, based on the edge detection image, determine the edge intensity value corresponding to each pixel point within the target pixel region corresponding to the selected location; Among all pixels, the target pixel corresponding to the edge intensity value that meets the preset selection conditions is selected as the edge point.

3. The method according to claim 1, characterized in that, The step of correcting the first contour fitting result and determining the corrected image includes: Based on the obtained correction coefficients, the first contour fitting result is adjusted to determine the second contour fitting result; the correction coefficients are used to indicate the correction direction and correction magnitude. Based on a pre-set second selection step size, contour points are selected on the second contour fitting result to determine multiple corrected contour points of the target contour. The multiple corrected contour points are used to characterize the corrected target contour. Based on the multiple corrected contour points, the corrected image is determined.

4. The method according to claim 3, characterized in that, Determining the corrected image based on the plurality of corrected contour points includes: Based on the multiple corrected contour points, a polygonal mesh is established; The image is deformed based on the polygonal mesh to obtain a corrected image.

5. The method according to any one of claims 1-4, characterized in that, The step of obtaining multiple feature points corresponding to the target contour in the image includes: Perform contour detection on the acquired image to determine the object's contour points; Determine the angle of the object in the image; Based on the object angle, multiple feature points are selected from the object contour points to correct the target contour in the object contour.

6. An image processing apparatus, characterized in that, include: The feature point acquisition module is configured to acquire multiple feature points corresponding to the target contour in the image; The plurality of feature points are used to characterize the target contour; An edge detection module is configured to perform edge detection on the image and determine an edge-detected image; An edge point determination module is configured to determine at least one edge point based on the plurality of feature points and the edge detection image; The at least one edge point is a point outside the plurality of feature points used to characterize the target contour; The fitting processing module is configured to fit the plurality of feature points and the at least one edge point to determine a first contour fitting result. The correction processing module is configured to correct the first contour fitting result and determine the corrected image; The fitting processing module includes: The matching processing unit is configured to select two target feature points from the plurality of feature points based on the encoding information corresponding to the plurality of feature points, wherein the encoding information matches the preset encoding information corresponding to the target contour; the encoding information is used to identify the plurality of feature points; The coordinate system establishment unit is configured to determine a first coordinate axis based on the line connecting the two target feature points; determine a second coordinate axis based on the perpendicular line connecting the two target feature points; and establish a target coordinate system based on the first coordinate axis and the second coordinate axis. An information determination unit is configured to determine the coordinate information of the plurality of feature points and the at least one edge point in the target coordinate system; The fitting processing unit is configured to fit the coordinate information to determine the first contour fitting result.

7. The apparatus according to claim 6, characterized in that, The edge point determination module includes: The position determination unit is configured to determine at least one selection position on the line connecting two adjacent feature points based on a pre-set first selection step size. The intensity value determination unit is configured to determine, for each selected location, the edge intensity value corresponding to each pixel point within the target pixel region corresponding to the selected location based on the edge detection image; The edge point determination unit is configured to select, from each pixel point, the target pixel point corresponding to the edge intensity value that meets the preset selection conditions as the edge point.

8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the image processing method as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, When the instruction processor in the storage medium is executed, it enables the server to perform the image processing method as described in any one of claims 1 to 5.

Images