Adjustment method, adjustment device and electronic device

Abstract

The application discloses an adjusting method, an adjusting device and electronic equipment, and belongs to the technical field of image processing. The adjusting method comprises the following steps: acquiring a key point in the case that a target image is distorted, the key point being a bounding box point of a first target in the target image; acquiring a mutation point corresponding to the key point respectively, the mutation point being a point with the minimum distance to the key point among boundary points of a face region in the target image; determining a distortion region in the face region where distortion occurs based on the key point and the mutation point; and adjusting a pixel value of the distortion region to a target pixel value.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to an adjusting method, an adjusting device and an electronic device. BACKGROUND

[0002] In a shooting scene, transparent objects such as water cups, glasses and the like have a refraction phenomenon, which can cause other objects to be deformed in a shooting picture. A human face is the most common shooting scene. When a person takes a photo with glasses, the face contour will be deformed due to the refraction of the glasses lenses and the shooting angle, which not only affects the aesthetics of the photo, but also increases the difficulty of human face recognition. Traditional repair methods require the user to operate corresponding software to correct the deformed part after shooting, for example, correcting through Photoshop software, and the operation is relatively complex. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide an adjusting method, an adjusting device and an electronic device, which can solve the problem of deformation of objects in an image due to refraction.

[0004] In a first aspect, the embodiments of the present application provide an adjusting method, which comprises:

[0005] In the case that a target image is distorted, a key point is obtained, the key point being a bounding box point of a first target in the target image;

[0006] Corresponding mutation points of the key points are obtained respectively, the mutation points being points in a boundary point of a face region in the target image, which have the smallest distance to the key points;

[0007] A distortion region in the face region, which is distorted, is determined based on the key points and the mutation points;

[0008] Pixel values of the distortion region are adjusted to target pixel values.

[0009] In a second aspect, the embodiments of the present application provide an adjusting device, which comprises:

[0010] A first determining module is configured to, in the case that a target image is distorted, obtain a key point, the key point being a bounding box point of a first target in the target image;

[0011] A second determining module is configured to obtain corresponding mutation points of the key points respectively, the mutation points being points in a boundary point of a face region in the target image, which have the smallest distance to the key points;

[0012] A third determining module is configured to determine a distortion region in the face region, which is distorted, based on the key points and the mutation points;

[0013] The first adjusting module is configured to adjust the pixel value of the distortion region to a target pixel value.

[0014] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory stores programs or instructions executable on the processor. When the programs or instructions are executed by the processor, the adjustment method according to the first aspect is implemented.

[0015] In a fourth aspect, a readable storage medium is provided, which stores programs or instructions. When the programs or instructions are executed by a processor, the adjustment method according to the first aspect is implemented.

[0016] In a fifth aspect, a chip is provided, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run programs or instructions, and implement the adjustment method according to the first aspect.

[0017] In a sixth aspect, a computer program product is provided, which is stored in a storage medium. The program product is executed by at least one processor to implement the adjustment method according to the first aspect.

[0018] In the embodiments of the present application, by determining the key points and mutation points of the face region on the target image, the distortion region in the face region where distortion occurs is determined, and then the pixel value of the distortion region is adjusted, so as to eliminate the distortion of the face region. It can be seen that the technical solution of the present application can automatically correct the distorted face in the case of image distortion, improve the correction efficiency, and ensure the beauty of the image. Compared with the scheme of manually correcting the distortion by the user, the distortion region to be corrected can be accurately determined, so as to avoid affecting other targets in the image, and the correction accuracy is higher. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is one of the flowcharts of the adjustment method provided by the embodiments of the present application;

[0020] Figure 2 is a schematic diagram of the image to be processed in the adjustment method provided by the embodiments of the present application;

[0021] Figure 3 is a schematic diagram of the key points in the adjustment method provided by the embodiments of the present application;

[0022] Figure 4 is the second flowchart of the adjustment method provided by the embodiments of the present application;

[0023] Figure 5 is a schematic diagram of the key points and mutation points in the adjustment method provided by the embodiments of the present application;

[0024] Figure 6 Figure 3 is a flowchart of a third adjustment method according to an embodiment of the present application;

[0025] Figure 7 Figure 4 is a flowchart of a fourth adjustment method according to an embodiment of the present application;

[0026] Figure 8 Figure 5 is a schematic diagram of a target edge line in the adjustment method according to an embodiment of the present application;

[0027] Figure 9 Figure 6 is a schematic diagram of the effect of the adjustment method according to an embodiment of the present application;

[0028] Figure 10 Figure 7 is a flowchart of a fifth adjustment method according to an embodiment of the present application;

[0029] Figure 11 Figure 8 is a schematic diagram of an adjustment device according to an embodiment of the present application;

[0030] Figure 12 Figure 9 is a schematic diagram of an electronic device according to an embodiment of the present application;

[0031] Figure 13 Figure 10 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0033] The terms “first”, “second”, and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by “first”, “second”, etc. are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, “and / or” in the specification and claims indicates at least one of the connected objects, and the character “ / ” generally indicates that the objects before and after are in an “or” relationship.

[0034] The embodiments of the present application will be described in detail below with reference to the drawings and specific embodiments and their application scenarios.

[0035] The embodiment of the present application first provides an adjusting method. Exemplarily, the adjusting method can be applied to electronic devices with display functions such as mobile phones, tablet computers, personal computers (PCs), wearable electronic devices (such as smart watches), augmented reality (AR) / virtual reality (VR) devices, and vehicle-mounted devices, and the embodiment of the present application does not make any limitation in this regard.

[0036] Distortion refers to the phenomenon that an object changes in shape when imaging is affected by light. In the case where a transparent object exists in a shooting scene, the transparent object will refract light, causing other objects to be distorted. When a user needs to eliminate the distortion in the image and restore the original shape of the distorted object, the image can be processed by the technical solution provided in the embodiment.

[0037] Figure 1 A flowchart of the adjusting method provided by the embodiment of the present application is shown. As shown in Figure 1 The adjusting method can include the following steps:

[0038] Step 100: In the case where a target image is distorted, a key point is obtained, the key point being a bounding box point of a first target in the target image.

[0039] The target image can be various types of images, such as a photo currently taken by a camera, an image in a video, an image stored on an electronic device, or an image received by an application, and the embodiment does not make any special limitation in this regard.

[0040] When the target image is processed, it can be determined whether the target image includes an object that is distorted. If the target image does not include a distorted object, the target image does not need to be adjusted. The first target refers to a transparent object in the target image that will cause light refraction. For example, the first target can be glass, a cup, glasses, and the like.

[0041] If the target image includes a case where distortion occurs due to light refraction of the transparent first target, the key point on the first target where distortion occurs can be determined. The point in the target image that intersects with the first target can deviate from the original position after light refraction, that is, distortion. The key point refers to a bounding box point of the first target, for example, a point on the frame of glasses. The target image starts to refract and distort at the key point, that is, the key point or the point adjacent to the key point in the target image can deviate from the original position after refraction by the transparent object.

[0042] The distortion detection model obtained through training can determine whether the target image is distorted and the key points of the distortion. Specifically, the target image is input into the distortion detection model, and the label and key points of the target image output by the distortion detection model can be obtained. The label output by the distortion detection model can indicate whether the target image is distorted. For example, when the label is 1, it indicates that the target image is distorted; when the label is 0, it indicates that the target image is not distorted.

[0043] In an exemplary embodiment, the distortion detection model can include a feature extraction layer, a classification layer, and a key point detection layer. The target image is input into the distortion detection model, and the feature extraction layer can extract features of the target image and output the features of the target image. The features can be input into the classification layer to obtain the label output by the classification layer. The feature extraction layer is also connected to the key point detection layer, and the features of the target image output by the feature extraction layer are input into the key point detection layer to obtain the key points output by the key point detection layer. When the label output by the classification layer indicates that the target image is not distorted, the key points output by the key point detection layer are empty, i.e., there are no key points in the target image.

[0044] The distortion detection model can be obtained through training of the labeled sample images. First, a certain number of sample images are obtained, which can include images without distortion and images with distortion. Then, the label and key points of each sample image are labeled. The label is used to indicate whether the sample image is distorted. The value of the label can be set in advance, where 1 indicates that the sample image is distorted, and 0 indicates that the sample image is not distorted. Alternatively, other values of the label can also be used to represent whether there is distortion, such as a label of “yes” indicating that the sample image is distorted, and a label of “no” indicating that the sample image is not distorted.

[0045] The labeled sample images can be used as training data sets for the distortion detection model to train the distortion detection model. In each training process, a sample image is input into the distortion detection model, and the distortion detection model can output the predicted label and the predicted key points of the sample image. The loss between the predicted label and the labeled label of the sample image is calculated, and the loss between the predicted key points and the labeled key points of the sample image is calculated to adjust the parameters of the distortion detection model. The adjusted parameters are used to process the next sample image, and the training is repeated multiple times until the loss between the predicted label output by the distortion detection model and the labeled label of the sample image is less than the corresponding preset value, and the loss between the predicted key points and the labeled key points is also less than the corresponding preset value, and the training of the distortion detection model is completed.

[0046] When training the distortion detection model, various loss functions can be used to calculate the loss, and various optimization functions can be used to adjust the parameters. For example, algorithms such as cross-entropy loss function and mean squared error loss function can be used to calculate the loss between the predicted label and the labeled label, and the loss between the predicted key point and the labeled key point. Alternatively, stochastic gradient descent method and Adam optimization method can be used to adjust the parameters. This implementation method does not impose any special limitations on these methods.

[0047] The trained distortion detection model can be used to detect target images. Taking glasses as an example, when a user wears glasses while taking a picture, the resulting image can be like... Figure 2 As shown in Figure 201, image 201 can be used as the target image. Inputting image 201 into the distortion detection model yields the label and key points of image 201 output by the distortion detection model. For example, the classification layer can use the softmax function to map the features of image 201 output by the feature extraction layer to a range of 0 to 1. Then, by judging the value of the label output by the classification layer, the type of image 201 can be determined. If the label value is greater than a preset value between 0 and 1, then image 201 has distortion. For instance, when the label output by the classification layer is greater than 0.5, it can be determined that image 201 has distortion; when the label output by the classification layer is not greater than 0.5, then image 201 does not have distortion. Furthermore, the label of image 201 can also be determined using other preset values, such as whether the label is greater than 0.6, 0.7, etc., and this embodiment does not impose any special limitations on this.

[0048] If the label of image 201 indicates that image 201 is distorted, then the keypoints output by the distortion detection model are obtained. Image 201 may include multiple keypoints, and the distortion detection model can output the coordinates of each keypoint in image 201. Based on the coordinates of each keypoint, the corresponding location on image 201 can be labeled. Figure 3 As shown, there are four key points on image 201, namely key point A, key point B, key point C and key point D.

[0049] In this embodiment, machine learning methods are applied to the process of refractive distortion detection. A distortion detection model for detecting key points of refractive distortion is obtained through machine learning. This not only facilitates the widespread use of machine learning, but also avoids the problem of high errors in manual processing of refractive distortion, thereby improving the accuracy of refractive distortion detection.

[0050] Step 200: Obtain the mutation point corresponding to each key point. The mutation point is the point with the smallest distance to the key point among the boundary points of the face region in the target image.

[0051] The mutation point is a position point after distortion in the face region, that is, the mutation point is a point on the boundary of the face region that has been distorted, and is a position point presented by the face region being refracted by the transparent object. When the shape of the face is distorted, all boundary points on the contour line of the face region that are distorted, that is, the mutation points, can be determined through the contour line. Specifically, Figure 4 A flowchart for obtaining the mutation points corresponding to the respective key points is shown.

[0052] As shown in Figure 4 The method comprises the following steps:

[0053] Step 401: Determine the connected region of the face region in the target image. For example, according to the color threshold of the face region, all pixel points in the target image within the color threshold range can be obtained, and the connected region formed by these pixel points is the connected region of the face. Alternatively, the target image is first converted into a grayscale image, and then the connected region of the face is determined through the color threshold of the face in the grayscale image. Each pixel point in the grayscale image has a color component, which can reduce the amount of calculation. For example, it is determined in advance that the grayscale of the face is 100-120, and then the pixel points with pixel values between 100 and 120 can be obtained from the grayscale image of the target image, and the connected region formed by these pixel points is the connected region of the face.

[0054] In addition, the connected region of the face region can also be obtained through other algorithms, such as the Two-Pass algorithm, and the present embodiment does not make special limitations on this. For example, the target image is image 201, and the connected region of the face in image 201 can be obtained through a skin color detection algorithm. The skin color detection algorithm can divide the skin region in the image in various color spaces.

[0055] Step 402: Obtain a plurality of boundary points on the boundary curve based on the boundary curve of the connected region. For example, the boundary curve of the connected region can be obtained through an edge detection algorithm. Then, the derivative of each point on the boundary curve is calculated, and the coordinates of the pixel point with a derivative of 0 are determined as the boundary point. The boundary curve can include a plurality of boundary points.

[0056] Step 403: respectively calculate the distance between each boundary point and the key point, determine the mutation point corresponding to the key point from the plurality of boundary points according to the distance, and the key point and the mutation point are one-to-one corresponding. For example, based on the coordinates at the key point and the coordinates at the boundary point, the distance between the two points can be calculated by the cosine distance, the Euclidean distance, and other distance algorithms. For example, assuming that there are 4 boundary points on the boundary curve of the face in the image 201. Respectively calculate the distance between each boundary point and the key point A, and according to the calculation result, the boundary point with the smallest distance to the key point A is taken as the mutation point corresponding to the key point A. Similarly, the nearest boundary point to each key point is determined in turn as the mutation point corresponding to the key point, and the key point and the mutation point are one-to-one corresponding. As shown in Figure 5 , the key point A corresponds to the mutation point a, the key point B corresponds to the mutation point b, the key point C corresponds to the mutation point c, and the key point D corresponds to the mutation point d.

[0057] It should be understood that in the image 201, the distortion effect caused by the glasses (i.e., the second target) is reduced relative to the normal face shape. When the second target causes an amplification distortion effect on the first target, the mutation point can be outside the key point.

[0058] The above implementation is used to determine the mutation point on the curve. The calculation process is simple, the mutation point on the curve can be quickly extracted, the calculation efficiency is improved, and the calculation amount and calculation resources are saved.

[0059] In the example implementation, the model obtained by training can also be used to determine the mutation point. In order to distinguish from the above distortion detection model, the model used to determine the mutation point is referred to as a first model. When the first model is trained, a sample image with the labeled mutation point can be used to train the first model, so that the trained first model can output the mutation point of the target image. Using the model to determine the mutation point will not affect the accuracy of the mutation point in the case where the shape of the first target itself is very irregular. It can adapt to more complex distortion conditions and improve the accuracy of the determined mutation point.

[0060] In addition, another neural network model, such as a second model, can also be trained based on a training data set with labeled mutation points and key points. Then the second model can be used to detect the key points and mutation points in the target image at the same time, which can simplify the processing process and improve the efficiency of the model.

[0061] Next, with reference to Figure 1 , step 300: based on the key points and the mutation points, determine the distortion region in the face region where the distortion occurs.

[0062] The distortion region is the region surrounded by the key points and the mutation points. The target image can include multiple distortion regions. Referring toFigure 5 It can be seen that the distortion region in the image 201 includes the distortion region of the left face and the distortion region of the right face. Taking the distortion region of the left face as an example, the distortion region is a region surrounded by the key point A, the key point B, the abrupt point b and the abrupt point a. In order to determine the distortion region, the connectivity between the key point A and the key point B needs to be estimated. Figure 6 A flowchart for determining the distortion region is shown.

[0063] As shown in Figure 6 , the method includes the following:

[0064] Step 601: Obtain the connected key curve in the face region based on the abrupt point. Specifically, according to the above embodiment, after determining the connected region of the face region, the abrupt point on the boundary curve of the connected region can be obtained. The connected key curve in the face region is the part between the abrupt points on the above boundary curve. As with the distortion region, the face region can also include multiple connected key curves. Taking the image 201 in Figure 5 as an example, the part between the abrupt point a and the abrupt point b on the boundary curve of the face, i.e. the curve ab, is the connected key curve in the distortion region of the left face. Similarly, the boundary curve part between the abrupt point c and the abrupt point d is the connected key curve in the distortion region of the right face. Determining the connected key curve from the connected region of the face region can ensure the connectivity of the connected key curve and the face region, and further ensure the connectivity of the target edge line and the face region when the target edge line is obtained by transformation, thereby improving the smoothness of the face region.

[0065] Step 602: Obtain the target edge line of the face region based on the key point and the connected key curve. The target edge line is the estimated connected curve between the key points. When the first target refracts the face region, the refracted light rays are generally parallel to each other. Further, the connected key curve between the abrupt points after the distortion of the face region is similar in shape to the target edge line between the key points. Then, by determining the transformation mode and the transformation amount of the connected key curve, the connected key curve between the abrupt points can be transformed into the target edge line between the key points by curve transformation.

[0066] Figure 7 A flowchart for determining the target edge line in the present application is shown. As shown in Figure 7 , the method includes the following:

[0067] Step 701: Determine the offset and scaling amount of the connected key curve based on the key point and the abrupt point. According to the correspondence between the key point and the abrupt point, the target edge line corresponding to the connected key curve between the abrupt points can be determined. For example, referring to Figure 5In the image 201, the target edge line between the key point A and the key point B corresponds to the connected key curve ab. When determining the target edge line AB, the offset and the scaling of the connected key curve ab need to be calculated.

[0068] The offset can be understood as the degree to which the connected key curve needs to be moved. For example, the offset can include a horizontal offset and a vertical offset, denoted as Px and Py, respectively. For example, taking the center point of the target image (such as the image 201) as the origin of the coordinate system, the horizontal offset of the connected key curve ab between the key point A (x A , y A ), the key point B (x B , y B ), and the corresponding abrupt points a (x a , y b ) and b (x b , y b ) can be expressed as: Px = [(x a + x b ) - (x A + x B )] / 2. That is, the horizontal offset of the connected key curve ab can be the average of the difference between the horizontal coordinate values of the abrupt points a and b and the horizontal coordinate values of the key points A and B. Similarly, the vertical offset of the connected key curve ab can be expressed as: Py = [(y a + y b ) - (y A + y B )] / 2.

[0069] It should be understood that in the image 201, with the center as the origin, the offset Px = [(x a + x b ) - (x A + x B )] / 2 is positive, which indicates that the connected key curve ab needs to move in the positive direction of the horizontal direction. For the connected key curve cd, since the horizontal coordinate values of the abrupt points c and d are less than those of the key points C and D, the horizontal offset of the connected key curve cd is negative, indicating that the connected key curve cd needs to move in the negative direction of the horizontal direction. That is, the offset is a value with a direction, and the calculation method of the offset can be flexibly adjusted according to the actual coordinate system of the target image, for example, adjusting the offset to Px = [(x A + x B ) - (x a + x b )] / 2, and the present embodiment is not limited thereto.

[0070] The scaling amount refers to the degree to which the connecting key curve needs to be scaled. For example, the scaling amount of the connecting key curve can be calculated by the ratio of the distance between the mutation point a and the mutation point b to the distance between the key point A and the key point B. The distance between the mutation point a and the mutation point b can be calculated by the distance formula between two points in space. The distance between the mutation point a and the mutation point b can be expressed as: Similarly, the distance between the key point A and the key point B is Further, the scaling amount of the connecting key curve ab is:

[0071] f = d1 / d2. Similarly, the offset amount and the scaling amount of the connecting key curve between the mutation point c and the mutation point d can also be calculated according to step 701, so as to obtain the offset amount and the scaling amount of the connecting key curve of each distortion region in the image 201.

[0072] Step 702: Transforming the connecting key curve based on the offset amount and the scaling amount to obtain the target edge line of the face region. Specifically, first, the connecting key curve is translated according to the offset amount. For example, the offset amount can include (Px, Py), and after the connecting key curve ab is translated in the direction of the offset amount, the coordinates of the mutation point a(x a , y a ) after translation are (x a +Px, y a +Py), and the coordinates of the mutation point b(x b , y b ) after translation are (x b +Px, y b +Py). Then, the connecting key curve after moving according to the offset amount is scaled according to the scaling amount. The connecting key curve after moving is multiplied by the scaling amount to obtain the scaled curve. For example, the scaling amount is f, and the mutation point a(x a +Px, y a +Py) on the connecting key curve before scaling has coordinates (f*(x a +Px), (f*(y a +Py)) after scaling. In summary, the curve obtained after the connecting key curve ab is moved and scaled is the target edge line between the key point A and the key point B on the face region. Figure 8 A schematic diagram of the curve of the connecting key curve after transformation in the image 201 is shown. In combination with Figure 5 and Figure 8, the target edge line AB can be obtained by moving and scaling the connected key curve ab. Similarly, the target edge line CD can be obtained by moving and scaling the connected key curve cd according to the offset and scaling amount of the connected key curve cd, so as to determine the target edge line of each distortion of the face in the image 201.

[0073] Further, it is determined whether the two ends of the target edge line obtained by transforming the connected key curve exceed the key points, and the part exceeding the key points is deleted, so that the target edge line has the key points as the end points, which can ensure the smoothness of the target edge line in the face region.

[0074] Generally, when determining the position of a target refraction, the position of the target refraction can be calculated based on the refractive index of the transparent object and the spatial refraction process. However, in the embodiment, the degree of movement and the degree of scaling of the connected key curve are determined by the two points (abrupt point a and abrupt point b) on the connected key curve and the two points (key point A and key point B) on the target edge line, so that the normal shape is restored. Compared with the complex physical refraction process, the calculation process is simple and efficient.

[0075] With reference to the foregoing Figure 6 , step 603: obtaining a distortion region based on the connected key curve and the target edge line. After obtaining the target edge line, the region surrounded by the connected key curve and the target edge line is the distortion region. Taking the image 201 as an example, after determining the target edge line between the key point A and the key point B, the key point A, the key point B, the abrupt point a and the abrupt point b are connected to each other, and the region formed thereby is the distortion region on the left side of the face in the image 201. Similarly, the region formed by the target edge line between the key point C and the key point D and the connected key curve cd between the abrupt point c and the abrupt point d is the distortion region on the right side of the face.

[0076] In the embodiment, when it is necessary to restore the shape of the face region before distortion, the normal shape of the target edge line of the face region before distortion can be automatically restored without manual intervention, the distortion region is identified, and the efficiency is high. Moreover, by transforming the connected key curve after distortion, the target edge line closer to the normal shape can be obtained, and the accuracy of the distortion region is improved.

[0077] Step 400: adjusting the pixel value of the distortion region to a target pixel value.

[0078] In this embodiment, the pixel values of the face region can be obtained by sampling in the region of the face region. The target pixel value can include the average of the pixel values of all the pixels in the face region, or the range of pixel values between the maximum pixel value and the minimum pixel value in the face region, or the pixel values with a probability distribution greater than a preset value in the face region, such as pixel values greater than 0.9, and the like. According to the target pixel value, the pixel values of each pixel in the distortion region can be adjusted to be the same or similar to the color of the face region. For example, according to the pixel values of the pixels in the face region, a pixel value range can be determined, and the original pixel values of the pixels in the distortion region can be adjusted to be within the pixel value range. The pixels in the distortion region can be determined by using bilinear interpolation, neighborhood interpolation, or the like, which is not particularly limited in this embodiment. After the color of the face region is supplemented to the distortion region, the pixels in the distortion region can present a color similar to that of the face region, thereby eliminating the distortion region of the face region and achieving the effect of correcting distortion.

[0079] Continuing with the face in the image 201 as an example, Figure 9 a schematic diagram of the face in the image 201 after correction of distortion is shown. Referring to Figure 9 , the pixel values in the distortion region of the face in the image 201 can be adjusted to restore the normal face shape. In this embodiment, even if the user takes a picture with glasses, a normal face shape can be obtained, which can better meet the needs of the user and improve the usability of the image.

[0080] Figure 10 a flowchart of the adjustment method of the present application is shown. Taking a face image as an example, referring to Figure 10 , the adjustment method of the present application can include the following steps: step 1001: training a distortion detection model. Step 1002: inputting a face image into the distortion detection model to obtain the label and key points of the face image. In this embodiment, the target image is the face image. Step 1003: determining whether the face image has distortion; if there is distortion, step 1004 is performed. Generally, in the image with glasses, the face will be distorted due to the refraction of the glasses. Step 1004: determining a target edge line in the face image. The target edge line is the target edge line of the face region that has distortion. Step 1005: determining a distortion region. Step 1006: color adjustment is performed on the distortion region to obtain a normal face picture. For example, if the distorted region is the face, the color of the distortion region can be filled with skin color. If it is determined in step 1003 that the face image does not have distortion, the image does not need to be processed, and the next target image can be processed.

[0081] It can be understood that, in the embodiment, the face image of a person wearing glasses is taken as an example to illustrate the correction process of face distortion caused by glasses. However, the method in the embodiment can also be applied to the scene of correcting image distortion caused by other transparent objects such as a water cup, a glass, and the like, for example, distortion caused by a water cup to a background, without being limited thereto. In addition, Figure 10 Each of the steps shown has been specifically described in the above embodiment, and will not be described again here.

[0082] The adjustment method provided in the embodiment can be executed by an adjustment device. In the embodiment, the adjustment method is executed by an adjustment device as an example to illustrate the adjustment device corresponding to the adjustment method provided in the embodiment.

[0083] Figure 11 A structure diagram of the adjustment device provided in the embodiment is shown. As shown in the figure, Figure 11 The adjustment device 1100 provided in the embodiment can include a first determination module 1101, a second determination module 1102, a third determination module 1103, and a first adjustment module 1104. Specifically, the first determination module 1101 can be used to acquire a key point in the case where a target image is distorted, the key point being a bounding point of a first target in the target image. The second determination module 1102 can be used to acquire a mutation point corresponding to the key point respectively, the mutation point being a point with the smallest distance to the key point among boundary points of a face region in the target image. The third determination module 1103 can be used to determine a distortion region in the face region based on the key point and the mutation point. The first adjustment module 1104 can be used to adjust a pixel value of the distortion region to a target pixel value.

[0084] In an exemplary embodiment, the third determination module 1103 can specifically include: a first acquisition unit, configured to acquire a connected key curve in the face region based on the mutation point; a second acquisition unit, configured to acquire a target edge line of the face region based on the key point and the connected key curve; and a third acquisition unit, configured to acquire the distortion region based on the connected key curve and the target edge line.

[0085] In an exemplary embodiment, the first acquisition unit can specifically include: a first determination unit, configured to determine a connected region of the face region in the target image; and a first acquisition sub-unit, configured to acquire a connected key curve between the mutation points on a boundary curve of the connected region based on the boundary curve.

[0086] In an exemplary embodiment, the second acquisition unit can specifically include: a second determination unit, configured to determine an offset and a scaling amount of the connected key curve based on the key point and the mutation point; and a first transformation unit, configured to transform the connected key curve based on the offset and the scaling amount to obtain the target edge line of the face region.

[0087] In an example implementation, the second determining module 1102 specifically includes: a third determining unit configured to determine a connected region of the face region in the target image; a fourth obtaining unit configured to obtain a plurality of boundary points on a boundary curve of the connected region based on the boundary curve; and a fourth determining unit configured to calculate a distance between each candidate mutation point and a key point respectively, and determine a mutation point corresponding to the key point from the plurality of boundary points according to the distance, the key point and the mutation point being in one-to-one correspondence.

[0088] The adjusting device 1100 provided in this embodiment determines the key point and the mutation point of the face region in the target image, and determines the distortion region in the face region. Then, the pixel value of the distortion region is adjusted, so as to eliminate the distortion of the face region, and the effect of automatically correcting the refractive distortion can be achieved. Compared with the scheme of manually correcting the distortion by the user, the distortion region to be corrected can be accurately determined, so as to avoid affecting other targets in the image, the correction accuracy is higher, and the image is more beautiful. Moreover, no manual intervention is needed in the processing process, and the labor time cost can be saved.

[0089] The adjusting device 1100 in the embodiment of the application can be an electronic device or a component in the electronic device, for example, an integrated circuit or a chip. The electronic device can be a terminal or other devices except the terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and the like. The electronic device can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the like. The embodiment of the application is not limited in this regard.

[0090] The adjusting device 1100 in the embodiment of the application can be a device with an operating system. The operating system can be an Android operating system, an ios operating system, or other possible operating systems. The embodiment of the application is not limited in this regard.

[0091] The adjusting device 1100 provided in the embodiment of the application canFigures 1 to 10 The various processes implemented by the method embodiments in the above method are not repeated here to avoid repetition.

[0092] Optionally, as shown in Figure 12 The electronic device 1200 includes a processor 1201 and a memory 1202. The memory 1202 stores programs or instructions executable by the processor 1201, which, when executed by the processor 1201, implements the various steps of the above adjustment method embodiments and achieves the same technical effects. To avoid repetition, the details are not repeated here.

[0093] It should be noted that the electronic device in the embodiments of the present application includes the above-mentioned mobile electronic device and non-mobile electronic device.

[0094] Figure 13 A hardware structure schematic diagram of an electronic device for implementing the embodiments of the present application.

[0095] The electronic device 1300 includes, but is not limited to, a radio frequency unit 1301, a network module 1302, an audio output unit 1303, an input unit 1304, a sensor 1305, a display unit 1306, a user input unit 1307, an interface unit 1308, a memory 1309, and a processor 1310, etc.

[0096] Those skilled in the art can understand that the electronic device 1300 can also include a power supply (such as a battery) for powering various components, and the power supply can be logically connected to the processor 1310 through a power management system, so as to realize the functions of managing charging, discharging, and power consumption management through the power management system. Figure 13 The electronic device structure shown in the above is not a limitation on the electronic device, and the electronic device can include more or fewer components than the diagram, or combine certain components, or different component arrangements, which are not repeated here.

[0097] The processor 1310 is configured to, in a case where a target image is distorted, acquire a key point, the key point being a bounding box point of a first target in the target image; acquire a mutation point corresponding to the key point respectively, the mutation point being a point in a boundary point of a face region in the target image and having a smallest distance from the key point; determine a distortion region in the face region where distortion occurs based on the key point and the mutation point; and adjust a pixel value of the distortion region to a target pixel value.

[0098] In some embodiments, the processor 1310 is further configured to acquire a connected key curve in the face region based on the mutation point; acquire a target edge line of the face region based on the key point and the connected key curve; and acquire the distortion region based on the connected key curve and the target edge line.

[0099] In some embodiments, the processor 1310 is further configured to determine a connected region of the face region in the target image; and obtain a connected key curve between the abrupt points on the boundary curve of the connected region.

[0100] In some embodiments, the processor 1310 is further configured to determine an offset and a scaling of the connected key curve based on the key points and the abrupt points; and transform the connected key curve based on the offset and the scaling to obtain the target edge line of the face region.

[0101] In some embodiments, the processor 1310 is further configured to determine a connected region of the face region in the target image; obtain a plurality of boundary points on the boundary curve of the connected region based on the boundary curve; calculate a distance between each boundary point and the key point respectively, and determine the abrupt point corresponding to the key point from the plurality of boundary points according to the distance, the key point and the abrupt point being one-to-one corresponding.

[0102] It should be noted that the electronic device described above in the present embodiment can implement each process in the method embodiments of the present application and achieve the same beneficial effects. To avoid repetition, details are not described here.

[0103] It should be understood that in the present embodiment, the input unit 1304 can include a graphics processing unit (GPU) 13041 and a microphone 13042. The graphics processing unit 13041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1306 can include a display panel 13061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1307 includes at least one of a touch panel 13071 and other input devices 13072. The touch panel 13071 is also called a touch screen. The touch panel 13071 can include a touch detection device and a touch controller. The other input devices 13072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, and the like, which are not described here.

[0104] The memory 1309 can be used to store software programs and various data. The memory 1309 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 1309 can include a volatile memory or a non-volatile memory, or the memory 1309 can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 1309 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0105] The processor 1310 can include one or more processing units; optionally, the processor 1310 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1310.

[0106] The embodiments of the present application also provide a readable storage medium, the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize various processes of the above adjustment method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0107] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0108] The embodiment of the present application further provides a chip, which comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions to realize the processes of the above adjustment method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0109] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system chip, a system chip, a chip system or a system on chip, etc.

[0110] The embodiment of the present application provides a computer program product, which is stored in a storage medium, and the program product is executed by at least one processor to realize the processes of the above adjustment method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0111] It should be noted that in this paper, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to the order of the functions shown or discussed, but can also include the functions performed in a substantially simultaneous manner or in the opposite order according to the functions involved, for example, the described method can be performed in an order different from the described order, and various steps can also be added, omitted or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0112] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned example methods can be realized by means of software and a necessary general hardware platform, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product in essence or in the form of a part that contributes to the prior art, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0113] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative and not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. An adjustment method, characterized in that, The method comprises the following steps: In the case that the target image is distorted, a key point is obtained, the key point being a bounding box point of a first target in the target image; A mutation point corresponding to the key point is obtained respectively, the mutation point being a point with the minimum distance to the key point among boundary points of a face region in the target image; A distortion region in the face region where distortion occurs is determined based on the key point and the mutation point; A pixel value of the distortion region is adjusted to a target pixel value; The step of determining the distortion region in the face region where distortion occurs based on the key point and the mutation point comprises the following steps: A connected key curve in the face region is obtained based on the mutation point; A target edge line of the face region is obtained based on the key point and the connected key curve; A distortion region is obtained based on the connected key curve and the target edge line; The step of obtaining the target edge line of the face region based on the key point and the connected key curve comprises the following steps: An offset and a scaling of the connected key curve are determined based on the key point and the mutation point; The connected key curve is transformed based on the offset and the scaling, and the target edge line of the face region is obtained.

2. The adjustment method of claim 1, wherein The step of obtaining the connected key curve in the face region based on the mutation point comprises the following steps: A connected region of the face region in the target image is determined; A connected key curve between the mutation points on a boundary curve of the connected region is obtained based on the boundary curve.

3. The adjustment method of claim 1, wherein The step of obtaining the mutation point corresponding to the key point comprises the following steps: A connected region of the face region in the target image is determined; A plurality of boundary points on the boundary curve of the connected region are obtained; Distances between each boundary point and the key point are calculated respectively, and the mutation point corresponding to the key point is determined from the plurality of boundary points according to the distances, the key point and the mutation point being in one-to-one correspondence.

4. An adjusting device, characterized in that The method comprises the following steps: A first determination module is configured to obtain a key point in the case that a target image is distorted, the key point being a bounding box point of a first target in the target image; A second determination module is configured to obtain a mutation point corresponding to the key point respectively, the mutation point being a point with the minimum distance to the key point among boundary points of a face region in the target image; A third determination module is configured to determine a distortion region in the face region where distortion occurs based on the key point and the mutation point; A first adjustment module is configured to adjust a pixel value of the distortion region to a target pixel value; The third determination module comprises: A first obtaining unit is configured to obtain a connected key curve in the face region based on the mutation point; A second obtaining unit is configured to obtain a target edge line of the face region based on the key point and the connected key curve; A third obtaining unit is configured to obtain a distortion region based on the connected key curve and the target edge line; The second obtaining unit comprises: A second determination unit is configured to determine an offset and a scaling of the connected key curve based on the key point and the mutation point; A first transformation unit is configured to transform the connected key curve based on the offset and the scaling factor, to obtain a target edge line of the face region.

5. The adjustment device of claim 4, wherein The first acquisition unit comprises: A first determination unit is configured to determine a connected region of the face region in the target image; A first acquisition sub-unit is configured to acquire, based on a boundary curve of the connected region, a connected key curve between the abrupt change points on the boundary curve.

6. The adjustment device of claim 4, wherein The second determination module comprises: A third determination unit is configured to determine a connected region of the face region in the target image; A fourth acquisition unit is configured to acquire, based on a boundary curve of the connected region, a plurality of boundary points on the boundary curve; A fourth determination unit is configured to calculate a distance between each boundary point and the key point respectively, and determine an abrupt change point corresponding to the key point from the plurality of boundary points according to the distance, the key point and the abrupt change point being in one-to-one correspondence.

7. An electronic device, comprising: A processor and a memory are included, the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the adjustment method of any one of claims 1-3.

8. A readable storage medium, characterized by, The readable storage medium stores programs or instructions, and the programs or instructions are executed by the processor to implement the adjustment method of any one of claims 1-3.

Images