Portrait picture adjusting method and device, equipment and medium

Abstract

The invention discloses a portrait picture adjustment method and device, equipment and a medium, is applied to a front end and relates to the technical field of computers, and the method comprises the steps: carrying out the preprocessing of an original portrait picture, obtaining a to-be-used portrait picture, and obtaining a portrait feature map based on the to-be-used portrait picture; the portrait feature map comprises a plurality of face detection frames and confidence corresponding to each face detection frame; determining a human face area of the human face detection frame with the highest confidence in the original human image picture; obtaining area deformation operation information of the user on the face area; based on the region deformation operation information and the region information of the face region, calculating a deformation region pixel displacement field by using a radial deformation algorithm, and calculating a deformation region pixel value by using a bilinear interpolation algorithm; and obtaining a final portrait picture according to the face region and based on the deformation region pixel displacement field and the deformation region pixel value. According to the invention, the delay of face image detection can be reduced and the image quality can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly relates to a method, device, equipment and medium for adjusting portrait pictures. Background Art

[0002] Currently, the main process for face image detection is as follows: The user uploads a picture to the backend server; the server runs a face detection model to locate the face area; deformation parameters are generated through backend calculation and the image is processed; the processed picture is returned to the front end for display. The disadvantages of this face image detection are as follows: It depends on the backend, resulting in response delays, there is a risk of privacy leakage, and users cannot interactively adjust in real time.

[0003] In summary, how to reduce the latency of face image detection and improve image quality is an urgent problem to be solved currently. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, device, equipment and medium for adjusting portrait pictures, which can reduce the latency of face image detection and improve image quality. The specific solutions are as follows:

[0005] In a first aspect, the present application discloses a method for adjusting portrait pictures, which is applied to the front end and includes:

[0006] Preprocess the original portrait picture to obtain a portrait picture to be used, and obtain a portrait feature map based on the portrait picture to be used; the portrait feature map includes several face detection frames and the confidence level corresponding to each face detection frame;

[0007] Determine the face area in the original portrait picture of the face detection frame with the highest confidence level;

[0008] Obtain the regional deformation operation information of the user for the face area;

[0009] Based on the regional deformation operation information and the regional information of the face area, use the radial deformation algorithm to calculate the pixel displacement field of the deformed area, and use the bilinear interpolation algorithm to calculate the pixel values of the deformed area;

[0010] Obtain the final portrait picture based on the face area and based on the pixel displacement field of the deformed area and the pixel values of the deformed area.

[0011] Optionally, the determining the face area in the original portrait picture of the face detection frame with the highest confidence level includes:

[0012] Determine the face detection frame with the highest confidence level, and perform coordinate conversion on the face detection frame with the highest confidence level in the portrait feature map to obtain the face coordinates in the original portrait picture;

[0013] Based on the face coordinates, determine the face elliptical region of the face detection box with the highest confidence in the original portrait picture through the ellipse fitting algorithm.

[0014] Optionally, the obtaining of the region deformation operation information of the user for the face region includes:

[0015] Obtain the outer frame adjustment information corresponding to the face region sent by the user through mouse dragging and the deformation intensity parameter sent by the change of the parameter slider.

[0016] Optionally, the preprocessing of the original portrait picture to obtain the portrait picture to be used includes:

[0017] Adjust the longest side of the original portrait picture to a predetermined size, and adjust other sides based on the predetermined size to obtain a temporary portrait image; the aspect ratio of the width and height of the temporary portrait image is equal to the aspect ratio of the width and height of the original portrait image;

[0018] Perform a normalization operation on the pixel values in the temporary portrait image to obtain a normalized image, and set the background color of the normalized image to black to obtain the portrait picture to be used.

[0019] Optionally, the obtaining of the portrait feature map based on the portrait picture to be used includes:

[0020] Input the portrait picture to be used into the scrfd model to obtain a portrait feature map; the scrfd model is integrated in the front end.

[0021] In a second aspect, the present application discloses a portrait picture adjustment device, which is applied to the front end and includes:

[0022] A feature map acquisition module, configured to preprocess the original portrait picture to obtain a portrait picture to be used, and obtain a portrait feature map based on the portrait picture to be used; the portrait feature map includes a plurality of face detection boxes and the confidence corresponding to each face detection box;

[0023] A face region determination module, configured to determine the face region of the face detection box with the highest confidence in the original portrait picture;

[0024] An information acquisition module, configured to obtain the region deformation operation information of the user for the face region;

[0025] A deformation calculation module, configured to calculate the pixel displacement field of the deformation region by using the radial deformation algorithm based on the region deformation operation information and the region information of the face region, and calculate the pixel values of the deformation region by using the bilinear interpolation algorithm;

[0026] An image adjustment module, configured to obtain a final portrait image according to the face region and based on the pixel displacement field and pixel values of the deformation region.

[0027] Optionally, the face region determination module includes:

[0028] A coordinate conversion unit, configured to determine the face detection box with the highest confidence, and perform coordinate conversion on the face detection box in the portrait feature map to obtain the face coordinates in the original portrait image;

[0029] A face region determination unit, configured to determine the face elliptical region of the face detection box in the original portrait image based on the face coordinates through an ellipse fitting algorithm.

[0030] Optionally, the information acquisition module includes:

[0031] An information acquisition unit, configured to acquire the outer frame adjustment information corresponding to the face region sent by the user through mouse dragging and the deformation intensity parameter sent by the change of the parameter slider.

[0032] In a third aspect, the present application discloses an electronic device, including:

[0033] A memory, configured to store a computer program;

[0034] A processor, configured to execute the computer program to implement the portrait image adjustment method disclosed above.

[0035] In a fourth aspect, the present application discloses a computer-readable storage medium, configured to store a computer program; wherein, when the computer program is executed by a processor, the portrait image adjustment method disclosed above is implemented.

[0036] It can be seen that the present application preprocesses the original portrait picture to obtain a portrait picture to be used, and obtains a portrait feature map based on the portrait picture to be used; the portrait feature map includes a plurality of face detection frames and the confidence corresponding to each face detection frame; determines the face area of the face detection frame with the highest confidence in the original portrait picture; obtains the regional deformation operation information of the user for the face area; based on the regional deformation operation information and the regional information of the face area, uses the radial deformation algorithm to calculate the pixel displacement field of the deformed area, and uses the bilinear interpolation algorithm to calculate the pixel value of the deformed area; according to the face area, and based on the pixel displacement field of the deformed area and the pixel value of the deformed area, obtains the final portrait picture. Thus, it can be seen that the present application directly completes the adjustment of the portrait picture at the front end, completely without the participation of the back end, getting rid of the influence of the back end, which is beneficial to improving the processing efficiency, reducing the latency, and after removing the transmission process between the front and back ends, reducing the risk of privacy leakage; the present application directly obtains the regional deformation operation information sent by the user at the front end, and completes the deformation through the radial deformation algorithm and the bilinear interpolation algorithm to obtain the final portrait picture, realizing the real-time interaction between the user and the front end, improving the user experience, and the hybrid application of the radial deformation algorithm and the bilinear interpolation algorithm improves the visual effect of the image and further improves the image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0038] Figure 1 It is a flowchart of a method for adjusting a portrait picture disclosed in the present application;

[0039] Figure 2 It is a schematic structural diagram of a device for adjusting a portrait picture disclosed in the present application;

[0040] Figure 3 It is a structural diagram of an electronic device disclosed in the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0042] Currently, the main process for face image detection is as follows: The user uploads a picture to the backend server; the server runs a face detection model to locate the face area; deformation parameters are generated through backend calculations and the image is processed; the processed picture is returned to the front end for display. The disadvantages of this face image detection are as follows: It relies on the backend, resulting in response delays, there is a risk of privacy leakage, and users cannot interactively adjust in real time.

[0043] Therefore, the embodiment of this application proposes a portrait picture adjustment scheme, which can reduce the latency of face image detection and improve the image quality.

[0044] The embodiment of this application discloses a method for adjusting a portrait picture. Refer to Figure 1 as shown, applied to the front end, the method includes:

[0045] Step S11: Preprocess the original portrait picture to obtain a portrait picture to be used, and obtain a portrait feature map based on the portrait picture to be used; the portrait feature map includes a number of face detection frames and the confidence level corresponding to each face detection frame.

[0046] In this embodiment, all operations of picture processing are completely completed by the front end, and the backend is not required to participate at all, realizing pure front-end portrait head adjustment and eliminating backend dependence. Specifically, WebAssembly (a new Web technology) and ONNXRuntime-Web technology (ONNX Runtime-Web is a high-performance inference engine that supports running ONNX (Open Neural Network Exchange) models in the browser. By encapsulating ONNXRuntime-Web, the integration process of model inference in the front-end project can be simplified, improving code maintainability and reusability) in this application can run lightweight AI (Artificial Intelligence) models in the browser. The web (World Wide Web) side (that is, the front end) loads the SCRFD (Sample and Computation Reduced Face Detector) model on the cdn (Content Delivery Network) through ONNXRuntime-Web, thereby realizing front-end processing.

[0047] In this embodiment, it is first necessary to obtain the original portrait picture. Specifically, the user can upload a local file by dragging and clicking or enter the network picture address. Then, the web side loads the picture through the HTML Image Element component to obtain the picture pixel data, and displays it on the web page through the Canvas component. Among them, HTML (Hyper Text Markup Language) is the hypertext markup language.

[0048] In this embodiment, after obtaining the original portrait picture, it is necessary to adjust the picture to obtain a picture suitable for subsequent interactive adjustment operations and deformation operations. Specifically, preprocessing the original portrait picture to obtain a portrait picture to be used includes: adjusting the longest side of the original portrait picture to a predetermined size, and adjusting other sides based on the predetermined size to obtain a temporary portrait image; the aspect ratio of the width and height of the temporary portrait image is equal to the aspect ratio of the width and height of the original portrait image; performing a normalization operation on the pixel values in the temporary portrait image to obtain a normalized image, and setting the background color of the normalized image to black to obtain a portrait picture to be used.

[0049] In this embodiment, obtaining the portrait feature map based on the portrait picture to be used includes: inputting the portrait picture to be used into the scrfd model to obtain the portrait feature map; the scrfd model is integrated in the front end.

[0050] It should be noted that on the web side, the SCRFD model on the CDN is loaded through ONNXRuntime-Web, and then the pixel data of the original portrait picture is processed to obtain the face image to be used. Specifically, the picture should be scaled and drawn centered on a black canvas of 640px (pixels) × 640px (pixels) at the front end. After obtaining the pixel values on the canvas, a normalization operation is used to convert the values from 0 to 255 into values from -1 to 1 and input them into the scarfd model; the model output is as follows: 1. Feature maps of different resolutions, used to decode the detection box and key point coordinates, and calculate the confidence. 2. Face detection boxes. The detection boxes are used to mark the positions of faces in the image, usually represented in the form of rectangles, containing four coordinate values (x_min, y_min, x_max, y_max). The model will output multiple detection boxes, and each detection box corresponds to a detected face area. 3. Face key points (not used in this case). The model will also output the key point coordinates of the face, usually including positions such as eyes, the tip of the nose, and the corners of the mouth. For each face detection box, the model will output 5 key points (2 eyes, the tip of the nose, 2 corners of the mouth). 4. Confidence. The confidence of each detection box represents the probability that there is a face in the detection box, usually a value between 0 and 1. The confidence is used to measure the reliability of the detection result, and a high confidence indicates that the detected face area is more reliable. It should be noted that the model outputs multiple feature maps, and each feature map corresponds to detection results of different scales. These feature maps contain the coordinates, confidence, and key point information of the face detection boxes. By decoding the feature maps, the confidence values of each detection box are extracted. These confidence values are learned by the model during training and are used to evaluate the probability of the existence of a face in the detection box. This value is usually a floating point number between 0 and 1.

[0051] It should be noted that the aspect ratios of the pictures are diverse, including square (equal width and height) and non-square (unequal width and height). These pictures with different ratios need to be uniformly processed so as to be input into the model. Here, "640" refers to pixels. Specifically, the longest side of the picture is scaled to 640 pixels while keeping the aspect ratio of the picture unchanged. Scrfd is a deep learning model that requires the input image to have a fixed size of 640x640 for better model processing. Only the scrfd model is used in this article, and it is the most suitable to choose the scrfd model in terms of comprehensive efficiency and accuracy. The scrfd model can calculate the confidence. The confidence is based on the matching degree between the detected face area and the features learned during model training.

[0052] Step S12: Determine the face area in the original portrait picture of the face detection box with the highest confidence.

[0053] In this embodiment, the coordinates of the detection box output by the model are represented in the scale of the feature map, usually relative coordinates. Therefore, it is necessary to convert them to the original portrait picture to obtain the coordinates in the original portrait picture. The determination of the face region of the face detection box with the highest confidence in the original portrait picture includes: determining the face detection box with the highest confidence, and performing coordinate conversion on the face detection box with the highest confidence in the portrait feature map to obtain the face coordinates in the original portrait picture; based on the face coordinates, determining the face elliptical region of the face detection box with the highest confidence in the original portrait picture through an ellipse fitting algorithm.

[0054] It should be noted that the coordinates of the detection box output by the model are represented in the scale of the feature map, usually a relative coordinate. For example, the resolution of the feature map may be 20×20, 40×40, and 80×80 (in pixels). The conversion process is as follows: Calculate the center point coordinates: According to the resolution of the feature map and the offset of the detection box, calculate the center point coordinates of the detection box. For a detection box on the feature map, its center point coordinates can be calculated by the following formula: center_x = col / map_size, center_y = row / map_size; where col and row are the column and row indices of the detection box on the feature map, and map_size is the size of the feature map. According to the center point coordinates and the width and height of the detection box, calculate the boundary coordinates of the detection box:

[0055] x1 = center_x − boxDetection[0] × scale;

[0056] y1 = center_y − boxDetection[1] × scale;

[0057] x2 = center_x + boxDetection[2] × scale;

[0058] y2 = center_y + boxDetection[3] × scale;

[0059] where boxDetection is the offset of the detection box output by the model, and scale is the ratio of the feature map scale to the original image scale. Before conversion, the detection box coordinates are represented in the scale of the feature map, usually a normalized relative coordinate. After conversion, the detection box coordinates are converted to the absolute coordinates of the original image and can be directly used to draw the detection box on the original image.

[0060] Step S13: Obtain the regional deformation operation information of the user for the face region.

[0061] In this embodiment, an elliptical frame operation box is displayed on the web page according to the parameters of the calculated face elliptical region. The midpoint of the ellipse is fixed, and the outer frame of the ellipse is supported to be adjusted by mouse dragging. At the same time, a slider is displayed below to control the deformation intensity parameter. Specifically, the obtaining of the regional deformation operation information of the face region by the user includes: obtaining the outer frame adjustment information corresponding to the face region sent by the user through mouse dragging and the deformation intensity parameter sent by the change of the parameter slider. It should be noted that the parameters of the face elliptical region are the upper left corner (x1, y1) and the lower right corner (x2, y2) of the region frame, and in order to display more vividly and conform to the face shape in the front end, an ellipse drawn with this rectangular frame can be displayed.

[0062] It should be noted that the interactive elliptical adjustment mechanism: supports real-time control of the deformation area by mouse dragging and parameter slider.

[0063] Step S14: Based on the regional deformation operation information and the regional information of the face region, use the radial deformation algorithm to calculate the pixel displacement field of the deformation region, and use the bilinear interpolation algorithm to calculate the pixel values of the deformation region.

[0064] In this embodiment, the calculation process of the radial deformation algorithm is as follows:

[0065] const maxDistX = width / 2;

[0066] const maxDistY = height / 2;

[0067] const maxDistSquared = maxDistX * maxDistX + maxDistY * maxDistY;

[0068] const factor = (distSquared: number) =>

[0069] Math.exp(-distSquared / (maxDistSquared / 0.9));

[0070] for (let y = 0; y < imgHeight; y++) {

[0071] for (let x = 0; x < imgWidth; x++) {

[0072] const dx = x - centerX;

[0073] const dy = y - centerY;

[0074] const distSquared = dx * dx + dy * dy;

[0075] const f = Math.min(factor(distSquared), 1);

[0076] const displacementX = strength * f * dx;

[0077] const displacementY = strength * f * dy;

[0078] const newX =

[0079] Math.max(0, Math.min(x + displacementX, imgWidth - 1));

[0080] const newY =

[0081] Math.max(0, Math.min(y + displacementY, imgHeight - 1));

[0082] const newPixel = bilinearInterpolation(newX, newY);

[0083] const index = (y * imgWidth + x) * 4;

[0084] expandedImg.data.set(newPixel, index);

[0085] };

[0086] }。

[0087] It should be noted that centerX and centerY are the positions of the center point of the detection box, width and height are the width and height of the detection box, and strength is the deformation strength.

[0088] In this embodiment, the bilinear interpolation algorithm is as follows (bilinearInterpolation represents the bilinear interpolation calculation method):

[0089] const bilinearInterpolation = (x: number, y: number): Uint8ClampedArray => {

[0090] const x0 = Math.floor(x);

[0091] const x1 = x0 + 1;

[0092] const y0 = Math.floor(y);

[0093] const y1 = y0 + 1;

[0094] const getPixel = (x: number, y: number): Uint8ClampedArray => {

[0095] if (x < 0 || x >= imgWidth || y < 0 || y >= imgHeight) {

[0096] return new Uint8ClampedArray([0, 0, 0, 0]);

[0097] };

[0098] const index = (y * imgWidth + x) * 4;

[0099] return data.slice(index, index + 4);

[0100] };

[0101] const Ia = getPixel(x0, y0);

[0102] const Ib = getPixel(x1, y0);

[0103] const Ic = getPixel(x0, y1);

[0104] const Id = getPixel(x1, y1);

[0105] const wa = (x1 - x) * (y1 - y);

[0106] const wb = (x - x0) * (y1 - y);

[0107] const wc = (x1 - x) * (y - y0);

[0108] const wd = (x - x0) * (y - y0);

[0109] const result = new Uint8ClampedArray(4);

[0110] for (let i = 0; i < 4; i++) {

[0111] result[i] = wa * Ia[i] + wb × Ib[i] + wc × Ic[i] + wd × Id[i];

[0112] }

[0113] return result;

[0114] }

[0115] It should be noted that the bilinear interpolation calculation method calculates the value of this pixel through the four pixel points in the lower right corner. The more accurate method is only the above interaction module, which requires the user to manually operate, and there is no other automated algorithm.

[0116] It should be noted that this application uses a hybrid deformation algorithm, that is, combining radial deformation and bilinear interpolation to balance the calculation efficiency and visual effect. In addition, affine transformation can be used to replace radial deformation, but it is difficult to achieve local deformation control. The nearest neighbor interpolation algorithm can also be used to replace bilinear interpolation, but it will cause edge aliasing.

[0117] Step S15: Obtain the final portrait picture according to the face area and based on the pixel displacement field and pixel values of the deformation area.

[0118] In this embodiment, the final portrait picture is displayed on the web page in real time through the Canvas component. The picture comparison mode before and after supports two methods: long mouse press for gradient comparison and dynamic dragging of the split screen comparison line.

[0119] It should be noted that this application installs the model at the front end and realizes the accurate positioning of the head area of the portrait picture through pure Web technology, realizes interactive adjustment through mouse dragging and slider control, and realizes natural deformation processing through the radial deformation algorithm and bilinear interpolation, while ensuring the processing efficiency and visual effect.

[0120] It should be noted that the above content can be divided into five units, including a picture upload unit, a face detection unit, an interactive adjustment unit, a deformation processing unit, and an image rendering unit. The picture upload unit completes the upload of the original portrait picture. The face detection unit completes image preprocessing and obtains the face elliptical area using the SCRFD model. The interactive adjustment unit supports mouse dragging to adjust the outer frame of the ellipse; at the same time, a slider is displayed below to control the deformation intensity parameter. The deformation processing unit applies the radial deformation algorithm to calculate the pixel displacement field and uses bilinear interpolation to calculate the pixel values of the deformation area. Finally, the image rendering unit displays the image on the web page in real time through the Canvas component. The specific content has been described in the introduction of the above steps and will not be specifically introduced here.

[0121] It can be seen that the present application preprocesses the original portrait picture to obtain a portrait picture to be used, and obtains a portrait feature map based on the portrait picture to be used; the portrait feature map includes a plurality of face detection frames and the confidence corresponding to each face detection frame; determines the face area of the face detection frame with the highest confidence in the original portrait picture; obtains the area deformation operation information of the user for the face area; based on the area deformation operation information and the area information of the face area, uses the radial deformation algorithm to calculate the pixel displacement field of the deformation area, and uses the bilinear interpolation algorithm to calculate the pixel value of the deformation area; according to the face area, and based on the pixel displacement field of the deformation area and the pixel value of the deformation area, obtains the final portrait picture. Thus, the present application directly completes the adjustment of the portrait picture at the front end, completely without the participation of the back end, is free from the influence of the back end, is beneficial to improving the processing efficiency, reducing the latency, and after removing the front-end and back-end transmission process, reduces the risk of privacy leakage; the present application directly obtains the area deformation operation information sent by the user at the front end, and completes the deformation through the radial deformation algorithm and the bilinear interpolation algorithm to obtain the final portrait picture, realizes the real-time interaction between the user and the front end, improves the user experience, and the hybrid application of the radial deformation algorithm and the bilinear interpolation algorithm improves the image visual effect and further improves the image quality.

[0122] Correspondingly, the embodiment of the present application also discloses a portrait picture adjustment device, as shown in Figure 2 shown, the device includes:

[0123] A feature map acquisition module 11, configured to preprocess the original portrait picture to obtain a portrait picture to be used, and obtain a portrait feature map based on the portrait picture to be used; the portrait feature map includes a plurality of face detection frames and the confidence corresponding to each face detection frame;

[0124] A face area determination module 12, configured to determine the face area of the face detection frame with the highest confidence in the original portrait picture;

[0125] An information acquisition module 13, configured to acquire the area deformation operation information of the user for the face area;

[0126] A deformation calculation module 14, configured to calculate the pixel displacement field of the deformation area by using the radial deformation algorithm and calculate the pixel value of the deformation area by using the bilinear interpolation algorithm based on the area deformation operation information and the area information of the face area;

[0127] A picture adjustment module 15, configured to obtain the final portrait picture according to the face area and based on the pixel displacement field of the deformation area and the pixel value of the deformation area.

[0128] In some embodiments, the face area determination module includes:

[0129] A coordinate conversion unit, configured to determine the face detection box with the highest confidence, and perform coordinate conversion on the face detection box in the portrait feature map to obtain the face coordinates in the original portrait picture;

[0130] A face area determination unit, configured to determine the face elliptical area of the face detection box in the original portrait picture based on the face coordinates through an ellipse fitting algorithm.

[0131] In some embodiments, the information acquisition module includes:

[0132] An information acquisition unit, configured to acquire the outer frame adjustment information corresponding to the face area sent by the user through mouse dragging and the deformation intensity parameter sent by the change of the parameter slider.

[0133] In some embodiments, the feature map acquisition module includes:

[0134] A picture adjustment unit, configured to adjust the longest side of the original portrait picture to a predetermined size, and adjust other sides based on the predetermined size to obtain a temporary portrait image; the aspect ratio of the temporary portrait image is equal to the aspect ratio of the original portrait picture;

[0135] A portrait image to be used acquisition unit, configured to perform a normalization operation on the pixel values in the temporary portrait image to obtain a normalized image, and set the background color of the normalized image to black to obtain a portrait picture to be used.

[0136] In some embodiments, the feature map acquisition module includes:

[0137] A feature map acquisition unit, configured to input the portrait picture to be used into the scrfd model to obtain a portrait feature map; the scrfd model is integrated in the front end.

[0138] It can be seen that the present application preprocesses the original portrait image to obtain a portrait image to be used, and obtains a portrait feature map based on the portrait image to be used; the portrait feature map includes a plurality of face detection frames and the confidence level corresponding to each face detection frame; determines the face area of the face detection frame with the highest confidence level in the original portrait image; obtains the regional deformation operation information of the user for the face area; based on the regional deformation operation information and the regional information of the face area, calculates the pixel displacement field of the deformed area using the radial deformation algorithm, and calculates the pixel values of the deformed area using the bilinear interpolation algorithm; and obtains the final portrait image according to the face area and based on the pixel displacement field of the deformed area and the pixel values of the deformed area. Thus, it can be seen that the present application directly completes the adjustment of the portrait image at the front end, completely without the participation of the back end, is free from the influence of the back end, is beneficial to improving the processing efficiency, reducing the latency, and after removing the front-end and back-end transmission process, reduces the risk of privacy leakage; the present application directly obtains the regional deformation operation information sent by the user at the front end, and completes the deformation through the radial deformation algorithm and the bilinear interpolation algorithm to obtain the final portrait image, realizing the real-time interaction between the user and the front end, improving the user experience, and the hybrid application of the radial deformation algorithm and the bilinear interpolation algorithm improves the image visual effect and further improves the image quality.

[0139] Furthermore, an embodiment of the present application also provides an electronic device. Figure 3 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation on the scope of use of the present application.

[0140] Figure 3 It is a structural schematic diagram of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the portrait image adjustment method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0141] In this embodiment, the power supply 26 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is imposed on it here; the input / output interface 24 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitation is made here.

[0142] In addition, as a carrier for storing resources, the memory 22 can be a read-only memory, a random access memory, a magnetic disk, an optical disk, etc. The resources stored thereon can include a computer program 221, and the storage method can be transient storage or permanent storage. Among them, in addition to the computer program that can be used to complete the portrait picture adjustment method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 221 can further include a computer program that can be used to complete other specific tasks.

[0143] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the portrait picture adjustment method disclosed above is implemented.

[0144] For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0145] The various embodiments in this application are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts between the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference can be made to the description in the method part for related parts.

[0146] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0147] The steps of the method or algorithm described in combination with the embodiments disclosed in this article can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0148] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

[0149] The above has introduced in detail a method, apparatus, device, and storage medium for adjusting portrait pictures provided by this application. Specific examples are used in this article to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A method for adjusting portrait pictures, characterized in that, Applied to the front end, including: Preprocess the original portrait picture to obtain a portrait picture to be used, and obtain a portrait feature map based on the portrait picture to be used; the portrait feature map includes a plurality of face detection frames and the confidence corresponding to each face detection frame; Determine the face area in the original portrait picture of the face detection frame with the highest confidence; Obtain the area deformation operation information of the user for the face area; Based on the area deformation operation information and the area information of the face area, use the radial deformation algorithm to calculate the pixel displacement field of the deformed area, and use the bilinear interpolation algorithm to calculate the pixel values of the deformed area; According to the face area, obtain the final portrait picture based on the pixel displacement field of the deformed area and the pixel values of the deformed area.

2. The portrait picture adjustment method according to claim 1, wherein The determining the face area in the original portrait picture of the face detection frame with the highest confidence includes: Determine the face detection frame with the highest confidence, and perform coordinate conversion on the face detection frame with the highest confidence in the portrait feature map to obtain the face coordinates in the original portrait picture; Based on the face coordinates, use the ellipse fitting algorithm to determine the face ellipse area in the original portrait picture of the face detection frame with the highest confidence.

3. The method for adjusting a portrait picture according to claim 1, wherein The obtaining the area deformation operation information of the user for the face area includes: Obtain the outer frame adjustment information corresponding to the face area sent by the user through mouse dragging and the deformation intensity parameter sent by the change of the parameter slider.

4. The method for adjusting a portrait picture according to claim 1, wherein The preprocessing the original portrait picture to obtain a portrait picture to be used includes: Adjust the longest side of the original portrait picture to a predetermined size, and adjust other sides based on the predetermined size to obtain a temporary portrait image; the aspect ratio of the temporary portrait image is equal to the aspect ratio of the original portrait image; Perform a normalization operation on the pixel values in the temporary portrait image to obtain a normalized image, and set the background color of the normalized image to black to obtain a portrait picture to be used.

5. The method for adjusting portrait pictures according to any one of claims 1 to 4, characterized in that The obtaining the portrait feature map based on the portrait picture to be used includes: Input the portrait picture to be used into the scrfd model to obtain a portrait feature map; the scrfd model is integrated in the front end.

6. A portrait picture adjustment device, characterized in that, Applied to the front end, including: A feature map acquisition module, configured to preprocess the original portrait picture to obtain a portrait picture to be used, and obtain a portrait feature map based on the portrait picture to be used; the portrait feature map includes a plurality of face detection frames and the confidence corresponding to each face detection frame; A face area determination module, configured to determine the face area in the original portrait picture of the face detection frame with the highest confidence; An information acquisition module, configured to obtain the area deformation operation information of the user for the face area; A deformation calculation module, configured to calculate the pixel displacement field of the deformed area by using the radial deformation algorithm based on the area deformation operation information and the area information of the face area, and calculate the pixel values of the deformed area by using the bilinear interpolation algorithm; A picture adjustment module, configured to obtain the final portrait picture according to the face area and based on the pixel displacement field of the deformed area and the pixel values of the deformed area.

7. The portrait picture adjustment device according to claim 6, characterized in that, The face region determination module includes: A coordinate conversion unit, configured to determine the face detection box with the highest confidence, and perform coordinate conversion on the face detection box in the portrait feature map to obtain the face coordinates in the original portrait picture; A face region determination unit, configured to determine the face elliptical region of the face detection box in the original portrait picture based on the face coordinates through an ellipse fitting algorithm.

8. The portrait picture adjustment device according to claim 6, characterized in that The information acquisition module includes: An information acquisition unit, configured to acquire the outer frame adjustment information corresponding to the face region sent by the user through mouse dragging and the deformation intensity parameter sent by the change of the parameter slider.

9. An electronic device, characterized in that, It includes: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the portrait picture adjustment method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by the processor, the portrait picture adjustment method according to any one of claims 1 to 5 is implemented.

Images