A face frame generation method and device, electronic equipment and storage medium

Abstract

The present application provides a kind of face frame generation method, device, electronic equipment and storage medium, it is related to image processing technical field.The face frame generation method includes obtaining the first coordinate data of face area in original image;According to the first width-height ratio of original image and the second width-height ratio of the canvas in display device, determine target image, and determine the scaling ratio and first offset of target image;After target image is filled to canvas according to scaling ratio and first offset, according to the first coordinate data of face area, the scaling ratio and first offset of target image, determine the second coordinate data of face area in canvas, to generate face frame;In response to calibration instruction, calibrate face frame, and determine the calibration displacement of face frame, in combination with the first coordinate data of face area and the scaling ratio of target image, determine the third coordinate data of face area in original image.The present application improves the accuracy of face frame indicating face in image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method, apparatus, electronic device, and storage medium for generating face frames. Background Technology

[0002] A face bounding box is a rectangular box generated by a face detection algorithm to mark the location and extent of detected faces in an image. The face bounding box is typically defined by its width, height, and top-left corner coordinates, or by its top-left and bottom-right corner coordinates. It forms the basis for subsequent face recognition, image face swapping, and AI-generated avatars, and is of great significance in numerous application scenarios such as security monitoring, smart access control, and portrait beautification.

[0003] In related technologies, inaccurate face bounding boxes are often generated due to various factors such as occlusion, image blurring, and limitations of the face detection algorithm itself. Furthermore, when the aspect ratio of the original image differs from that of the display device, part of the original image is cropped or whitespace is added, resulting in inaccurate face bounding boxes generated on the display device based on the original image. Summary of the Invention

[0004] The problem addressed by this invention is how to improve the accuracy of face mapping in images.

[0005] To address the above problems, the present invention provides a method, apparatus, electronic device, and storage medium for generating face frames.

[0006] In a first aspect, the present invention provides a method for generating a face bounding box, comprising:

[0007] Obtain the first coordinate data of the face region in the original image;

[0008] Based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device, a target image is determined, and the scaling ratio and first offset of the target image are determined, wherein the original image includes the target image, and the target image includes the face region;

[0009] After the target image is filled into the canvas according to the scaling ratio and the first offset, the second coordinate data of the face region in the canvas is determined according to the first coordinate data of the face region, the scaling ratio of the target image and the first offset, so as to generate a face frame, wherein the face frame is placed on the upper layer of the target image;

[0010] In response to a calibration command, the face bounding box is calibrated, and the calibration displacement of the face bounding box is determined. Based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box, the third coordinate data of the face region in the original image is determined.

[0011] Optionally, before determining the target image based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device, the method further includes:

[0012] Obtain the third aspect ratio of the screen in the display device;

[0013] When the third aspect ratio of the screen is less than or equal to the first preset threshold, or when the third aspect ratio of the screen is greater than or equal to the second preset threshold, the canvas is set to a preset size.

[0014] When the third aspect ratio of the screen is greater than the first preset threshold and the third aspect ratio of the screen is less than the second preset threshold, the canvas size is set according to the screen size.

[0015] Optionally, determining the target image based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device includes:

[0016] When the first aspect ratio of the original image is equal to the second aspect ratio of the canvas, the original image is used as the target image;

[0017] When the first aspect ratio of the original image is greater than the second aspect ratio of the canvas, the first width of the original image is cropped in the center according to the second aspect ratio of the canvas based on the first height of the original image to obtain the target image;

[0018] When the first aspect ratio of the original image is smaller than the second aspect ratio of the canvas, the first height of the original image is cropped in the center according to the second aspect ratio of the canvas based on the first width of the original image to obtain the target image.

[0019] Optionally, determining the scaling ratio and first offset of the target image includes:

[0020] When the first width of the original image is centered and cropped, the scaling ratio and the first offset of the target image are determined based on the first width of the original image, the second width of the canvas, and the third width of the target image.

[0021] When the first height of the original image is centered and cropped, the scaling ratio and the first offset of the target image are determined based on the first height of the original image, the second height of the canvas, and the third height of the target image.

[0022] Optionally, determining the second coordinate data of the face region in the canvas based on the first coordinate data of the face region, the scaling ratio of the target image, and the first offset includes:

[0023] The first coordinate data of the face region is scaled according to the scaling ratio of the target image to obtain the fourth coordinate data of the face region, and the first offset of the target image is scaled according to the scaling ratio of the target image to obtain the second offset of the target image.

[0024] The second coordinate data of the face region is obtained based on the fourth coordinate data of the face region and the second offset of the target image.

[0025] Optionally, before calibrating the face bounding box and determining the calibration displacement of the face bounding box, the method further includes:

[0026] The face bounding box is determined to be within the range of the target image;

[0027] It is determined that the calibration displacement of the face frame is greater than a third preset threshold.

[0028] Optionally, determining the third coordinate data of the face region in the original image based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box includes:

[0029] The calibration displacement of the face frame is decomposed to obtain the first lateral displacement and the first longitudinal displacement of the face frame;

[0030] Based on the scaling ratio of the target image, the first horizontal displacement and the first vertical displacement of the face frame, the second horizontal displacement and the second vertical displacement of the face frame are determined.

[0031] The third coordinate data of the face region is determined based on the first coordinate data of the face region, the second horizontal displacement of the face frame, and the second vertical displacement.

[0032] In a second aspect, the present invention provides a face frame generation device, comprising:

[0033] The acquisition module is used to acquire the first coordinate data of the face region in the original image;

[0034] The determining module is configured to determine a target image based on a first aspect ratio of the original image and a second aspect ratio of the canvas in the display device, and to determine the scaling ratio and a first offset of the target image, wherein the original image includes the target image, and the target image includes the face region;

[0035] The generation module is configured to, after the target image is filled onto the canvas according to the scaling ratio and the first offset, determine the second coordinate data of the face region in the canvas according to the first coordinate data of the face region, the scaling ratio of the target image and the first offset, so as to generate a face frame, wherein the face frame is placed on the upper layer of the target image;

[0036] A calibration module is configured to calibrate the face bounding box in response to a calibration command, determine the calibration displacement of the face bounding box, and determine the third coordinate data of the face region in the original image based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box.

[0037] Thirdly, the present invention provides an electronic device, including a memory and a processor;

[0038] The memory is used to store computer programs;

[0039] The processor is configured to implement the face frame generation method as described in the first aspect when executing the computer program.

[0040] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the face frame generation method as described in the first aspect.

[0041] The beneficial effects of the face frame generation method, apparatus, electronic device, and storage medium of the present invention are as follows: By acquiring the first coordinate data of the face region in the original image, the position and size of the face in the original image are initially determined, providing a basis for subsequent face frame generation. When the first aspect ratio of the original image does not match the second aspect ratio of the canvas in the display device, the face frame generated directly using the first coordinate data of the face region will be offset from the actual face region because the original image is cropped and fills the canvas, or the complete original image fills the canvas and leaves blank spaces. Therefore, based on the first and second aspect ratios, it is determined whether to process the original image accordingly to obtain a target image with the second aspect ratio. Then, the scaling ratio of the target image is determined so that the size of the target image is equal to the size of the canvas, and a first offset of the target image is determined to align the target image with the canvas, thereby making the target image completely fill the canvas. Since the coordinates of the face region change with the coordinates of the original image, the changed first coordinate data of the face region is determined based on the first coordinate data of the face region, the scaling ratio of the target image, and the first offset. This is the second coordinate data of the face region in the canvas. An accurate face bounding box is then generated based on this second coordinate data. To further improve the accuracy of the face bounding box, avoiding the influence of factors such as occlusion, image blurring, and limitations of the face detection algorithm itself, the face bounding box is calibrated through user interaction. Simultaneously, the calibration displacement of the face bounding box in the canvas is determined during the calibration process. Combining the first coordinate data of the face region and the scaling ratio of the target image, the face region in the original image is calibrated in reverse, determining the third coordinate data of the face region in the original image. This third coordinate data can not only be used for further image processing (such as face recognition, image face swapping, and AI-synthesized avatars), but can also be fed back to the algorithm used to obtain the first coordinate data of the face region in the original image, thereby optimizing the algorithm and improving the accuracy of the generated face bounding box. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the face bounding box generation method according to an embodiment of the present invention;

[0043] Figure 2 This is a system architecture diagram of the face frame generation device according to an embodiment of the present invention;

[0044] Figure 3 This is a system architecture diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0045] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0046] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0047] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to"; the term "based on" means "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; and the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first," "second," etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0048] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0049] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0050] like Figure 1 As shown, an embodiment of the present invention provides a method for generating a face bounding box, comprising:

[0051] S100: Obtain the first coordinate data of the face region in the original image.

[0052] Specifically, the classification ability of each Haar feature can be evaluated using a large number of images containing and without face regions to select some Haar features with strong classification capabilities. For each selected Haar feature, a weak classifier is trained to combine them into a strong classifier. The original image is divided into multiple windows, and the Haar feature value for each window is calculated. All Haar feature values ​​are input into the trained strong classifier to detect face regions in the original image and output their coordinate data. Alternatively, a convolutional neural network (such as SSD, YOLO, or MTCNN) can be trained using a large number of images with labeled face regions to learn various facial features (such as shape, texture, or color). The original image is input into the trained convolutional neural network to detect face regions in the original image and output their coordinate data. Here, the face region is a rectangle, and its coordinate data includes the width, height, and coordinates of the four vertices.

[0053] S200: Determine a target image based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device, and determine the scaling ratio and first offset of the target image, wherein the original image includes the target image, and the target image includes the face region.

[0054] Specifically, the first aspect ratio of the original image is compared with the second aspect ratio of the canvas. If the first aspect ratio equals the second aspect ratio, the original image (i.e., the target image) is directly scaled to completely fill the canvas. If the first aspect ratio does not equal the second aspect ratio, the original image is first centered and cropped, and then the cropped original image (i.e., the target image) is scaled to completely fill the canvas. During cropping, it must be ensured that the cropped portion does not include the face region. When the cropped portion overlaps with the face region, a prompt will appear: the face region is located at the edge of the image, and a face bounding box cannot be generated. During scaling, the scaling ratio and the first offset need to be determined. The scaling ratio ensures that the size of the target image is equal to the size of the canvas, and the first offset ensures that the target image is aligned with the canvas.

[0055] In some embodiments, the original image has a size of 600×600 pixels, and the canvas has a size of 800×800 pixels. The first aspect ratio of the original image and the second aspect ratio of the canvas are both calculated to be 1. The scaling ratio of the target image (i.e., the original image) is determined, either by dividing the second width of the canvas by the first width of the target image, or by dividing the second height of the canvas by the first height of the target image; the scaling ratio of the target image is 4 / 3. A first offset of the target image is determined, and the first offset of the target image is 0.

[0056] In other embodiments, the original image is 900×800 pixels in size, and the canvas is 600×500 pixels in size. The first aspect ratio of the original image is calculated to be 9 / 8, and the second aspect ratio of the canvas is 6 / 5. After cropping, the third aspect ratio of the target image is 6 / 5, for example, the target image size is 900×750 pixels, 840×700 pixels, or 720×600 pixels. The scaling ratio of the target image is determined; when the target image size is 840×700 pixels, the scaling ratio is 5 / 7. The first offset of the target image is determined, which can be determined by determining the change in the image coordinate origin. For example, with the top left corner of the image as the coordinate origin, after cropping the original image by 20 units to the left, 40 units to the right, 60 units to the top, and 40 units to the bottom, the target image is obtained. The first offset of the target image in the width direction can be determined to be 20 units, and the first offset of the target image in the height direction can be determined to be 60 units.

[0057] S300: After the target image is filled into the canvas according to the scaling ratio and the first offset, the second coordinate data of the face region in the canvas is determined according to the first coordinate data of the face region, the scaling ratio of the target image and the first offset, so as to generate a face frame, wherein the face frame is placed on the upper layer of the target image.

[0058] Specifically, after the target image completely fills the canvas, the first coordinate data of the face region on the target image will change. To generate an accurate face bounding box, the changed first coordinate data (i.e., the second coordinate data) is calculated. This involves first scaling the first coordinate data of the face region and the first offset of the target image according to a scaling ratio, and then correcting the vertex coordinates in the scaled first coordinate data of the face region based on the scaled first offset of the target image. Alternatively, the vertex coordinates in the first coordinate data of the face region can be corrected first based on the first offset of the target image, and then the first coordinate data of the face region and the first offset of the target image can be scaled according to a scaling ratio. The face bounding box is located on top of the target image and is aligned with the target image through absolute positioning. The position property of the face bounding box can be set to absolute, and the second coordinate data of the face region can be assigned to the left, top, width, and height properties of the face bounding box. In some embodiments, an icon is set on the face bounding box to indicate "Current box is editable" or "Current box is being edited".

[0059] S400: In response to a calibration command, calibrate the face bounding box and determine the calibration displacement of the face bounding box. Based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box, determine the third coordinate data of the face region in the original image.

[0060] Specifically, if the display device has a touchscreen, the face bounding box can be bound to the touchstart, touchmove, and touchend events to calibrate the face bounding box. When the user touches the face bounding box, the touchstart event is triggered to obtain the initial touch point coordinates; when the user moves the face bounding box, the touchmove event is triggered to continuously obtain the current touch point coordinates, thereby achieving continuous monitoring of the touch point movement trajectory; when the user ends their contact with the face bounding box, the touchend event is triggered to determine the updated second coordinate data of the face area on the canvas based on the final touch point coordinates. If the display device does not have a touchscreen, the face bounding box can be bound to the mousedown, mousemove, and mouseup events to calibrate the face bounding box. When a user selects a face frame by pressing the left mouse button, the mousedown event is triggered to obtain the initial cursor coordinates. When the user moves the face frame by dragging the mouse, the mousemove event is triggered to continuously obtain the current cursor coordinates, thereby enabling continuous monitoring of the cursor movement trajectory. When the user deselects the face frame by releasing the left mouse button, the mouseup event is triggered to determine the updated second coordinate data of the face region on the canvas based on the final cursor coordinates.

[0061] Furthermore, the displacement of the touch point, i.e., the calibration displacement of the face bounding box, is determined based on the initial and final touch point coordinates. Alternatively, the displacement of the cursor, i.e., the calibration displacement of the face bounding box in the canvas, is determined based on the initial and final cursor coordinates. Based on the scaling ratio of the target image and the calibration displacement of the face bounding box in the canvas, the calibration displacement of the face region in the original image is determined. Based on the first coordinate data of the face region in the original image and the calibration displacement, the updated first coordinate data of the face region in the original image, i.e., the third coordinate data of the face region in the original image, is determined.

[0062] In this embodiment, by acquiring the first coordinate data of the face region in the original image, the position and size of the face in the original image are initially determined, providing a foundation for subsequent face bounding box generation. When the first aspect ratio of the original image does not match the second aspect ratio of the canvas in the display device, the face bounding box generated directly using the first coordinate data of the face region will be offset from the actual face region because the original image is cropped and fills the canvas, or because blank spaces appear after the complete original image fills the canvas. Therefore, based on the first and second aspect ratios, it is determined whether to process the original image accordingly to obtain a target image with the second aspect ratio. Then, the scaling ratio of the target image is determined so that the size of the target image is equal to the size of the canvas, and the first offset of the target image is determined to align the target image with the canvas, so that the target image completely fills the canvas. Since the coordinates of the face region change with the coordinates of the original image, the changed first coordinate data of the face region is determined based on the first coordinate data of the face region, the scaling ratio of the target image, and the first offset. This is the second coordinate data of the face region in the canvas. An accurate face bounding box is then generated based on this second coordinate data. To further improve the accuracy of the face bounding box, avoiding the influence of factors such as occlusion, image blurring, and limitations of the face detection algorithm itself, the face bounding box is calibrated through user interaction. Simultaneously, the calibration displacement of the face bounding box in the canvas is determined during the calibration process. Combining the first coordinate data of the face region and the scaling ratio of the target image, the face region in the original image is calibrated in reverse, determining the third coordinate data of the face region in the original image. This third coordinate data can not only be used for further image processing (such as face recognition, image face swapping, and AI-synthesized avatars), but can also be fed back to the algorithm used to obtain the first coordinate data of the face region in the original image, thereby optimizing the algorithm and improving the accuracy of the generated face bounding box.

[0063] Optionally, before determining the target image based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device, the method further includes:

[0064] Obtain the third aspect ratio of the screen in the display device;

[0065] When the third aspect ratio of the screen is less than or equal to the first preset threshold, or when the third aspect ratio of the screen is greater than or equal to the second preset threshold, the canvas is set to a preset size.

[0066] When the third aspect ratio of the screen is greater than the first preset threshold and the third aspect ratio of the screen is less than the second preset threshold, the canvas size is set according to the screen size.

[0067] Specifically, to ensure a good display effect for the target image, a canvas of appropriate size needs to be set for display devices with screens of different sizes to accommodate the target image. When a user uses a display device for the first time, the width and height of the screen are first obtained to determine the third aspect ratio. If the third aspect ratio is less than or equal to the first preset threshold (e.g., 9:19), or greater than or equal to the second preset threshold (e.g., 19:9), then the display device is considered to have an ultra-wide or ultra-high screen, and the canvas is directly set to the preset size. The preset size is determined based on the specific type of display device. When the display device is a mobile phone, the preset size is preferably 504×896 pixels; when the display device is a computer, the preset size is preferably 1440×1024 pixels. If the third aspect ratio is greater than the first preset threshold (e.g., 9:19) and less than the second preset threshold (e.g., 19:9), then the display device is considered to have a standard screen. The width of the screen is scaled according to the first preset ratio to obtain the width of the canvas, and the height of the screen is scaled according to the second preset ratio to obtain the height of the canvas. The first and second preset ratios are determined based on the specific type of display device. When the display device is a mobile phone, the first preset ratio is preferably 0.95, and the second preset ratio is preferably 0.85. Then, the position of the canvas is adjusted to ensure that the canvas is centered on the screen.

[0068] In this optional embodiment, the third aspect ratio of the screen in the current display device is compared with a first preset threshold and a second preset threshold to determine whether the screen of the display device is too wide or too tall. If so, a canvas of a preset size is used directly to avoid severely compressing the effective display area of ​​the target image, facilitating subsequent user calibration of the face frame. If not, the canvas size is calculated based on the screen size to set the canvas, achieving a positive correlation between the canvas size and the screen size. The larger the screen, the larger the canvas, making full use of the screen's physical space and ensuring the display effect of the target image.

[0069] Optionally, determining the target image based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device includes:

[0070] When the first aspect ratio of the original image is equal to the second aspect ratio of the canvas, the original image is used as the target image;

[0071] When the first aspect ratio of the original image is greater than the second aspect ratio of the canvas, the first width of the original image is cropped in the center according to the second aspect ratio of the canvas based on the first height of the original image to obtain the target image;

[0072] When the first aspect ratio of the original image is smaller than the second aspect ratio of the canvas, the first height of the original image is cropped in the center according to the second aspect ratio of the canvas based on the first width of the original image to obtain the target image.

[0073] Specifically, the width (first width) and height (first height) of the original image are obtained to determine the first aspect ratio, and the width (second width) and height (second height) of the canvas are obtained to determine the second aspect ratio. Then, the first and second aspect ratios are compared. When the first aspect ratio equals the second aspect ratio, it means the shape of the original image is completely similar to the shape of the canvas. In this case, no cropping of the original image is needed; the complete original image is the target image. When the first aspect ratio is greater than the second aspect ratio, it means the original image is longer in the width direction relative to the canvas. To make the original image match the canvas (i.e., completely similar in shape to the canvas), the simplest operation is to crop only the first width. The cropped original image is the target image, and the width (third width) of the target image is equal to the first height multiplied by the second aspect ratio, and the height (third height) of the target image is equal to the first height. For example, the original image size is 700×630 pixels, and the canvas size is 600×900 pixels. The calculated first aspect ratio is 10 / 9, and the second aspect ratio is 2 / 3. Since the first aspect ratio is greater than the second aspect ratio, cropping the first width results in a target image size of 420×630 pixels. When the first aspect ratio is less than the second aspect ratio, it means the original image is longer in the height direction relative to the canvas. To make the original image match the canvas (i.e., perfectly similar in shape to the canvas), the simplest operation is to crop only the first height. The cropped original image is the target image, where the width (third width) equals the first width, and the height (third height) equals the first width divided by the second aspect ratio. For example, the original image size is 900×800 pixels, and the canvas size is 600×500 pixels. Calculated, the first aspect ratio is 9 / 8, and the second aspect ratio is 6 / 5. Since the first aspect ratio is less than the second aspect ratio, cropping the first height results in a target image size of 900×750 pixels. Because the face area is usually located in the center of the original image, cropping the width or height of the original image in the center reduces the possibility of the face area being cropped out.

[0074] In this optional embodiment, when the aspect ratio of the original image matches the canvas, unnecessary cropping is avoided, preserving the complete information of the original image and ensuring a natural and distortion-free display effect. When the aspect ratio of the original image does not match the canvas, since the face area is usually located in the center of the original image, a center-cropping strategy is adopted. This not only solves the canvas white space problem caused by the mismatch between the original image and the canvas, ensuring that the target image can completely fill the canvas, but also reduces the possibility of the face area being cropped. To further reduce the possibility of the face area being cropped, when the first aspect ratio of the original image is greater than the second aspect ratio of the canvas, a height-cropping-width strategy is adopted; and when the first aspect ratio of the original image is less than the second aspect ratio of the canvas, a width-cropping-height strategy is adopted, in order to retain as much information as possible from the original image.

[0075] Optionally, determining the scaling ratio and first offset of the target image includes:

[0076] When the first width of the original image is centered and cropped, the scaling ratio and the first offset of the target image are determined based on the first width of the original image, the second width of the canvas, and the third width of the target image.

[0077] When the first height of the original image is centered and cropped, the scaling ratio and the first offset of the target image are determined based on the first height of the original image, the second height of the canvas, and the third height of the target image.

[0078] Specifically, after acquiring the target image, the scaling ratio and first offset of the target image can be determined to ensure that the target image completely fills the canvas. If the target image is acquired by center-cropping the width of the original image (i.e., the first width), then the scaling ratio of the target image is the ratio of the width of the canvas (i.e., the second width) to the width of the target image (i.e., the third width), and the first offset of the target image is half the difference between the width of the original image (i.e., the first width) and the width of the target image (i.e., the third width). For example, the original image is 700×630 pixels, and the canvas is 600×900 pixels. After cropping, the target image is 420×630 pixels. The scaling ratio of the target image can be determined to be 10 / 7, and the first offset to be 140. If the target image is obtained by center-cropping the height of the original image (i.e., the first height), then the scaling ratio of the target image is the ratio of the canvas height (i.e., the second height) to the target image height (i.e., the third height), and the first offset of the target image is half the difference between the height of the original image (i.e., the first height) and the height of the target image (i.e., the third height). For example, the original image is 900×800 pixels, and the canvas size is 600×500 pixels. After cropping, the target image size is 900×750 pixels. Therefore, the scaling ratio of the target image can be determined to be 2 / 3, and the first offset to be 25.

[0079] In this optional embodiment, to achieve the goal of the target image completely filling the canvas, the target image needs to be completely identical to the canvas not only in shape but also in size and position. Due to different cropping strategies, the methods for determining the scaling ratio and the first offset of the original image differ. When using a height-to-width cropping strategy, changes in the width of the original image result in changes in the horizontal coordinate data, i.e., the first offset. Therefore, the first offset is determined based on the first width of the original image and the third width of the target image. Since the target image is scaled to the canvas size, the third width and third height of the target image need to be scaled by the same scaling ratio; this can be determined by combining the third width of the target image with the second width of the canvas. When using a width-to-height cropping strategy, changes in the height of the original image result in changes in the vertical coordinate data, i.e., the first offset. Therefore, the first offset is determined based on the first height of the original image and the third height of the target image. Since the target image is scaled to the canvas size, the third width and third height of the target image need to be scaled by the same scaling ratio; this can be determined by combining the third height of the target image with the second height of the canvas.

[0080] Optionally, determining the second coordinate data of the face region in the canvas based on the first coordinate data of the face region, the scaling ratio of the target image, and the first offset includes:

[0081] The first coordinate data of the face region is scaled according to the scaling ratio of the target image to obtain the fourth coordinate data of the face region, and the first offset of the target image is scaled according to the scaling ratio of the target image to obtain the second offset of the target image.

[0082] The second coordinate data of the face region is obtained based on the fourth coordinate data of the face region and the second offset of the target image.

[0083] Specifically, since the cropped original image (i.e., the target image) completely fills the canvas, the face region located in the original image coordinate system needs to be transformed to the canvas coordinate system. The original coordinate system is established with a vertex of the original image (e.g., the top left vertex) as the origin, and its width and height directions are the coordinate axes respectively. The canvas coordinate system is established with a vertex of the canvas (e.g., the top left vertex) as the origin, and its width and height directions are the coordinate axes respectively. We can first transform the first coordinate data and the first offset to the canvas coordinate system, that is, multiply the first coordinate data and the first offset by the scaling factor to obtain the fourth coordinate data and the second offset. If the second offset is in the width direction, then subtract the second offset from the x-coordinates of the vertices in the fourth coordinate data; if the second offset is in the height direction, then subtract the second offset from the y-coordinates of the vertices in the fourth coordinate data to compensate for the coordinate data changes caused by cropping, thus obtaining the second coordinate data. The face bounding box generated by the second coordinate data corresponds to the actual face region. For example, the vertex coordinates in the first coordinate data are (20, 40) and (40, 20), with a width of 20 and a height of 20. The first offset in the width direction is 10, and the scaling ratio is 2. We can calculate that the vertex coordinates in the fourth coordinate data are (40, 80) and (80, 40), with a width of 40 and a height of 40. The second offset in the width direction is 20. Therefore, we obtain the vertex coordinates in the second coordinate data as (20, 80) and (60, 40), with a width of 40 and a height of 40.

[0084] In this optional embodiment, the first coordinate data is first scaled according to the scaling ratio of the target image so that the size of the face frame to be generated is the same as that of the actual face region. Then, the first offset (i.e. the second offset) scaled according to the scaling ratio of the target image is calculated to correct the fourth coordinate data so that the position of the face frame to be generated is the same as that of the actual face region. This avoids the situation where the first aspect ratio of the original image does not match the second aspect ratio of the canvas in the display device, and the face frame generated directly using the first coordinate data of the face region is not corrected because the original image is cropped and fills the canvas, resulting in an offset from the actual face region.

[0085] Optionally, before calibrating the face bounding box and determining the calibration displacement of the face bounding box, the method further includes:

[0086] The face bounding box is determined to be within the range of the target image;

[0087] It is determined that the calibration displacement of the face frame is greater than a third preset threshold.

[0088] Specifically, the `touchend` event is triggered when the user ends their interaction with the face bounding box; the `mouseup` event is triggered when the user deselects the face bounding box by releasing the left mouse button. The system first determines whether the current face bounding box is within the target image's range and whether the current face bounding box's displacement exceeds a third preset threshold. Then, it determines the updated second coordinate data of the face region on the canvas to calibrate the face bounding box and determine its calibration displacement on the canvas. If the current face bounding box is within the target image and its displacement is less than or equal to a third preset threshold, the user's operation is considered a mis-touch. If the current face bounding box is outside the target image and its displacement is less than or equal to the third preset threshold, the user's operation is considered a mis-touch. If the current face bounding box is within the target image and its displacement is greater than the third preset threshold, the user's operation is considered a face bounding box calibration, and its displacement is determined as the calibration displacement. If the current face bounding box is outside the target image and its displacement is greater than the third preset threshold, the user's operation is considered a face bounding box calibration, and the distance the face bounding box moves to the boundary of the target image is determined as the calibration displacement. The third preset threshold should not be set too high to reduce the possibility of misjudging face bounding box calibration operations as mis-touches, nor should it be set too low to reduce the waste of computational resources caused by mis-touches. A preferred third preset threshold is 10.

[0089] In this optional embodiment, by setting the face bounding box within the target image, invalid third coordinate data is avoided due to excessive dragging of the face bounding box by the user. This prevents further image processing (such as face recognition, image face swapping, and AI-synthesized avatars) and algorithm optimization for obtaining the first coordinate data of the face region in the original image. By setting a third preset threshold to determine the user's face bounding box calibration operation, unintentional face bounding box movement is reduced, thereby avoiding unnecessary waste of computing resources and improving the user's interactive experience.

[0090] Optionally, determining the third coordinate data of the face region in the original image based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box includes:

[0091] The calibration displacement of the face frame is decomposed to obtain the first lateral displacement and the first longitudinal displacement of the face frame;

[0092] Based on the scaling ratio of the target image, the first horizontal displacement and the first vertical displacement of the face frame, the second horizontal displacement and the second vertical displacement of the face frame are determined.

[0093] The third coordinate data of the face region is determined based on the first coordinate data of the face region, the second horizontal displacement of the face frame, and the second vertical displacement.

[0094] Specifically, since the calibration displacement of the face bounding box determined through user interaction lies in the canvas coordinate system, the calibration displacement can be decomposed in the canvas coordinate system to obtain the first horizontal displacement and the first vertical displacement. Then, the first horizontal displacement and the first vertical displacement are transformed to the coordinate system of the original image, that is, the first horizontal displacement and the first vertical displacement are divided by the scaling factor, respectively, to obtain the second horizontal displacement and the second vertical displacement. Based on the first coordinate data of the face region in the coordinate system of the original image, the second horizontal displacement, and the second vertical displacement, the third coordinate data of the calibrated face bounding box in the coordinate system of the original image can be determined. For example, if the initial touch point coordinates are (100, 150), the final touch point coordinates are (150, 150), and the scaling factor is 2, the first horizontal displacement can be calculated as 50, the first vertical displacement as 0, thus obtaining the second horizontal displacement as 25 and the second vertical displacement as 0. Since the vertex coordinates in the first coordinate data are (40, 80), the vertex coordinates in the third coordinate data are (65, 80).

[0095] In this optional embodiment, the calibration displacement is transformed from the canvas coordinate system to the original image coordinate system by scaling the target image; that is, the first horizontal displacement is transformed into the second horizontal displacement, and the first vertical displacement is transformed into the second vertical displacement. The first coordinate data of the face region is corrected using the second horizontal and second vertical displacements to obtain the third coordinate data of the face region. The generation of the third coordinate data can not only be used for further image processing (such as face recognition, image face swapping, and AI-synthesized avatars), but can also be fed back to the algorithm used to obtain the first coordinate data of the face region in the original image, thereby optimizing the algorithm and improving the accuracy of the generated face bounding box.

[0096] like Figure 2 As shown, an embodiment of the present invention provides a face frame generation device 200, comprising:

[0097] The acquisition module 210 is used to acquire the first coordinate data of the face region in the original image;

[0098] The determining module 220 is configured to determine a target image based on a first aspect ratio of the original image and a second aspect ratio of the canvas in the display device, and to determine a scaling ratio and a first offset of the target image, wherein the original image includes the target image and the target image includes the face region;

[0099] The generation module 230 is configured to, after the target image is filled onto the canvas according to the scaling ratio and the first offset, determine the second coordinate data of the face region in the canvas according to the first coordinate data of the face region, the scaling ratio of the target image and the first offset, so as to generate a face frame, wherein the face frame is placed on the upper layer of the target image;

[0100] The calibration module 240 is configured to calibrate the face bounding box in response to a calibration command, determine the calibration displacement of the face bounding box, and determine the third coordinate data of the face region in the original image based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box.

[0101] like Figure 3 As shown, an electronic device 300 provided in this embodiment of the invention includes a memory 310 and a processor 320; the memory 310 is used to store a computer program; the processor 320 is used to implement the face frame generation method as described above when the computer program is executed.

[0102] Alternatively, an electronic device 300 includes a memory 310 and a processor 320 coupled to the memory 310; the memory 310 is configured to store a computer program; and the processor 320 is configured to perform the following operations when the computer program is executed:

[0103] Obtain the first coordinate data of the face region in the original image;

[0104] Based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device, a target image is determined, and the scaling ratio and first offset of the target image are determined, wherein the original image includes the target image, and the target image includes the face region;

[0105] After the target image is filled into the canvas according to the scaling ratio and the first offset, the second coordinate data of the face region in the canvas is determined according to the first coordinate data of the face region, the scaling ratio of the target image and the first offset, so as to generate a face frame, wherein the face frame is placed on the upper layer of the target image;

[0106] In response to a calibration command, the face bounding box is calibrated, and the calibration displacement of the face bounding box is determined. Based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box, the third coordinate data of the face region in the original image is determined.

[0107] This invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the face frame generation method described above.

[0108] Alternatively, a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the following operations:

[0109] Obtain the first coordinate data of the face region in the original image;

[0110] Based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device, a target image is determined, and the scaling ratio and first offset of the target image are determined, wherein the original image includes the target image, and the target image includes the face region;

[0111] After the target image is filled into the canvas according to the scaling ratio and the first offset, the second coordinate data of the face region in the canvas is determined according to the first coordinate data of the face region, the scaling ratio of the target image and the first offset, so as to generate a face frame, wherein the face frame is placed on the upper layer of the target image;

[0112] In response to a calibration command, the face bounding box is calibrated, and the calibration displacement of the face bounding box is determined. Based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box, the third coordinate data of the face region in the original image is determined.

[0113] The present invention will now be described an electronic device 300 that can serve as a server or client of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. Electronic device 300 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 300 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0114] Electronic device 300 includes a computing unit that can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) or a computer program loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The computing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0115] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. In this application, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units.

[0116] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for generating a face bounding box, characterized in that, include: Obtain the first coordinate data of the face region in the original image; Based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device, a target image is determined, and the scaling ratio and first offset of the target image are determined, wherein the original image includes the target image, and the target image includes the face region; After the target image is filled into the canvas according to the scaling ratio and the first offset, the second coordinate data of the face region in the canvas is determined according to the first coordinate data of the face region, the scaling ratio of the target image and the first offset, so as to generate a face frame, wherein the face frame is placed on the upper layer of the target image; In response to a calibration command, the face bounding box is calibrated, and the calibration displacement of the face bounding box is determined. Based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box, the third coordinate data of the face region in the original image is determined.

2. The face bounding box generation method according to claim 1, characterized in that, Before determining the target image based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device, the method further includes: Obtain the third aspect ratio of the screen in the display device; When the third aspect ratio of the screen is less than or equal to the first preset threshold, or when the third aspect ratio of the screen is greater than or equal to the second preset threshold, the canvas is set to a preset size. When the third aspect ratio of the screen is greater than the first preset threshold and the third aspect ratio of the screen is less than the second preset threshold, the canvas size is set according to the screen size.

3. The face bounding box generation method according to claim 1, characterized in that, Determining the target image based on the first aspect ratio of the original image and the second aspect ratio of the canvas in the display device includes: When the first aspect ratio of the original image is equal to the second aspect ratio of the canvas, the original image is used as the target image; When the first aspect ratio of the original image is greater than the second aspect ratio of the canvas, the first width of the original image is cropped in the center according to the second aspect ratio of the canvas based on the first height of the original image to obtain the target image; When the first aspect ratio of the original image is smaller than the second aspect ratio of the canvas, the first height of the original image is cropped in the center according to the second aspect ratio of the canvas based on the first width of the original image to obtain the target image.

4. The face frame generation method according to claim 3, characterized in that, Determining the scaling ratio and first offset of the target image includes: When the first width of the original image is centered and cropped, the scaling ratio and the first offset of the target image are determined based on the first width of the original image, the second width of the canvas, and the third width of the target image. When the first height of the original image is centered and cropped, the scaling ratio and the first offset of the target image are determined based on the first height of the original image, the second height of the canvas, and the third height of the target image.

5. The face bounding box generation method according to claim 1, characterized in that, Determining the second coordinate data of the face region in the canvas based on the first coordinate data of the face region, the scaling ratio of the target image, and the first offset includes: The first coordinate data of the face region is scaled according to the scaling ratio of the target image to obtain the fourth coordinate data of the face region, and the first offset of the target image is scaled according to the scaling ratio of the target image to obtain the second offset of the target image. The second coordinate data of the face region is obtained based on the fourth coordinate data of the face region and the second offset of the target image.

6. The face bounding box generation method according to claim 1, characterized in that, Before calibrating the face bounding box and determining the calibration displacement of the face bounding box, the method further includes: The face bounding box is determined to be within the range of the target image; It is determined that the calibration displacement of the face frame is greater than a third preset threshold.

7. The face bounding box generation method according to claim 1, characterized in that, Determining the third coordinate data of the face region in the original image based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box includes: The calibration displacement of the face frame is decomposed to obtain the first lateral displacement and the first longitudinal displacement of the face frame; Based on the scaling ratio of the target image, the first horizontal displacement and the first vertical displacement of the face frame, the second horizontal displacement and the second vertical displacement of the face frame are determined. The third coordinate data of the face region is determined based on the first coordinate data of the face region, the second horizontal displacement of the face frame, and the second vertical displacement.

8. A face frame generation device, characterized in that, include: The acquisition module is used to acquire the first coordinate data of the face region in the original image; The determining module is configured to determine a target image based on a first aspect ratio of the original image and a second aspect ratio of the canvas in the display device, and to determine the scaling ratio and a first offset of the target image, wherein the original image includes the target image, and the target image includes the face region; The generation module is configured to, after the target image is filled onto the canvas according to the scaling ratio and the first offset, determine the second coordinate data of the face region in the canvas according to the first coordinate data of the face region, the scaling ratio of the target image and the first offset, so as to generate a face frame, wherein the face frame is placed on the upper layer of the target image; A calibration module is configured to calibrate the face bounding box in response to a calibration command, determine the calibration displacement of the face bounding box, and determine the third coordinate data of the face region in the original image based on the first coordinate data of the face region, the scaling ratio of the target image, and the calibration displacement of the face bounding box.

9. An electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement the face frame generation method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the face frame generation method as described in any one of claims 1 to 7.

Images