An image processing method, device and computer readable storage medium
By acquiring image size and facial feature information, the system automatically crops and scales images, solving the problem of low efficiency in manual image selection and editing, and achieving efficient generation of video poster images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TENCENT TECHNOLOGY (SHENZHEN) CO LTD
- Filing Date
- 2021-05-17
- Publication Date
- 2026-05-29
Smart Images
Figure CN113763242B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, specifically to an image processing method, apparatus, and computer-readable storage medium. Background Technology
[0002] With the maturity of electronic technology, more and more digital products are being produced, popularized and popularized, bringing great convenience to the general public. In daily life, users can use digital products to shoot or create their own video works at any time.
[0003] In existing technologies, in order to attract users to watch videos, it is necessary to manually select and edit images in the video works to create suitable images as video posters for distribution. However, the process of manual selection is cumbersome and inefficient, resulting in low efficiency in image processing. Summary of the Invention
[0004] This application provides an image processing method, apparatus, and computer-readable storage medium, which can improve the efficiency of image processing.
[0005] To address the aforementioned technical problems, this application provides the following technical solutions:
[0006] An image processing method, comprising:
[0007] Obtain the image to be processed and its corresponding size information;
[0008] The target center point is determined based on the location information of the facial features in the image to be processed;
[0009] The cutting dimensions are determined based on the aspect ratio between the stated dimensions and the target dimensions.
[0010] The image to be processed is cropped based on the target center point and the cropping size information to obtain a target image, wherein the aspect ratio of the target image is the same as the aspect ratio of the target size information;
[0011] The target image is scaled to the target size information.
[0012] An image processing apparatus, comprising:
[0013] The acquisition unit is used to acquire the image to be processed and the corresponding size information of the image to be processed;
[0014] The first determining unit is used to determine the target center point based on the location information of the facial feature information in the image to be processed;
[0015] The second determining unit is used to determine the cutting size information based on the aspect ratio relationship between the size information and the target size information;
[0016] A cropping unit is used to crop the image to be processed based on the target center point and the cropping size information to obtain a target image, wherein the aspect ratio of the target image is the same as the aspect ratio of the target size information;
[0017] A scaling unit is used to scale the target image to the target size information.
[0018] In some embodiments, the second determining unit further includes a fourth determining subunit, configured to:
[0019] When it is detected that the aspect ratio of the size information is less than the aspect ratio of the target size information, the first width information of the size information, the second width information of the target size information, and the second height information of the target size information are obtained.
[0020] The third height information is calculated based on the first width information, the second width information, and the second height information;
[0021] The cutting dimensions are determined based on the third height information and the first width information.
[0022] In some embodiments, the cutting unit includes:
[0023] A cropping subunit is used to crop the image to be processed according to the cropping size information to obtain a cropped image, wherein the cropped image contains the target center point;
[0024] The fifth determining subunit is used to determine the cropped image as the target image when the position of the target center point in the cropped image is detected to be in the upper half of the cropped image.
[0025] In some embodiments, the trimming unit further includes:
[0026] The acquisition subunit is used to acquire the first target identification box corresponding to the target face feature information in the image to be processed when it is detected that the position of the target center point in the cropped image is not in the upper half of the cropped image.
[0027] An expansion subunit is used to expand the first target identifier box by a preset multiple to obtain an expanded second target identifier box;
[0028] Add a sub-unit to add an additional frame to the second target identifier frame to obtain a third target identifier frame after the addition. The aspect ratio of the third target identifier frame is the same as the aspect ratio of the target size information.
[0029] The sixth determining subunit is used to determine the third target identification box as the target image.
[0030] In some embodiments, the addition of the subunit is used for:
[0031] Obtain the fourth width information, the fourth height information, the second width information, and the second height information of the target size information of the second target identification box;
[0032] The fifth height information is calculated based on the fourth width information, the second width information, and the second height information;
[0033] The sixth altitude information is determined based on the difference between the fifth altitude information and the fourth altitude information;
[0034] The size information of the additional frame is determined based on the fourth width information and the sixth height information;
[0035] The second target identifier box is merged with the additional box to obtain the third target identifier box.
[0036] In some embodiments, the apparatus further includes an adding unit for:
[0037] When the sharpness information of the scaled target image meets the preset threshold, the scaled target image is retained;
[0038] Add a preset display control to the scaled target image.
[0039] In some embodiments, the apparatus further includes a deletion unit for:
[0040] When a region in the target image is detected to not belong to the image to be processed, the target image is deleted.
[0041] A computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps in the image processing method described above.
[0042] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the image processing method described above.
[0043] A computer program product or computer program includes computer instructions stored in a storage medium. A processor of a computer device reads the computer instructions from the storage medium and executes the computer instructions to perform the steps of the aforementioned image processing method.
[0044] This application embodiment obtains the image to be processed and its corresponding size information; determines the target center point based on the position information of facial features in the image to be processed; determines the cropping size information based on the aspect ratio between the size information and the target size information; crops the image to be processed based on the target center point and the cropping size information to obtain the target image; and scales the target image to the target size information. Thus, by determining the target center point of the image based on facial features and cropping it according to the aspect ratio between the size information and the target size information, a target image with a compliant aspect ratio is obtained. The target image is then scaled to the target size information, quickly generating a thumbnail that meets the requirements. Compared to solutions that require manual selection and editing of poster images, this application embodiment eliminates the need for manual intervention, greatly improving the efficiency of image processing. Attached Figure Description
[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0046] Figure 1 This is a scene diagram of the image processing system provided in the embodiments of this application;
[0047] Figure 2 This is a schematic flowchart of the image processing method provided in the embodiments of this application;
[0048] Figure 3 This is another schematic flowchart of the image processing method provided in the embodiments of this application;
[0049] Figure 4a This is a schematic diagram of a scene for the image processing method provided in an embodiment of this application;
[0050] Figure 4b This is another schematic diagram of a scenario for the image processing method provided in the embodiments of this application;
[0051] Figure 4c This is another schematic diagram of a scenario for the image processing method provided in the embodiments of this application;
[0052] Figure 4d This is another schematic diagram of a scenario for the image processing method provided in the embodiments of this application;
[0053] Figure 4e This is another schematic diagram of a scenario for the image processing method provided in the embodiments of this application;
[0054] Figure 5This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application;
[0055] Figure 6 This is a schematic diagram of the server structure provided in an embodiment of this application. Detailed Implementation
[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0057] This application provides an image processing method, apparatus, and computer-readable storage medium.
[0058] Please see Figure 1 , Figure 1 This is a schematic diagram of an image processing system provided in an embodiment of this application, including: terminal A and a server (the image processing system may also include other terminals besides terminal A; the specific number of terminals is not limited here). Terminal A and the server can be connected via a communication network, which may include wireless networks and wired networks. The wireless network includes one or more combinations of wireless wide area networks, wireless local area networks, wireless metropolitan area networks, and wireless personal networks. The network includes network entities such as routers and gateways, which are not shown in the figure. Terminal A can interact with the server through the communication network; for example, terminal A can send an image to be processed to the server through a client.
[0059] The image processing system may include an image processing device, which can be integrated into a computer device. This computer device can be a terminal or a server, and the terminal can be a mobile phone, tablet, laptop, smart TV, wearable smart device, desktop computer, etc. Taking the image processing method executed by a server as an example, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Figure 1As shown, the server can obtain the image to be processed and its corresponding size information; determine the target center point based on the position information of the facial features in the image to be processed; determine the cropping size information based on the aspect ratio between the size information and the target size information; crop the image to be processed based on the target center point and the cropping size information to obtain the target image, the aspect ratio of which is the same as that of the target size information; and scale the target image to the target size information.
[0060] The terminal may include a client, which may be a video client, a game client, an instant messaging client, or a document editing client, etc. Terminal A can send video frames from the game client as images to be processed to the server.
[0061] It should be noted that, Figure 1 The schematic diagram of the image processing system shown is merely an example. The image processing system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of image processing systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.
[0062] The following sections will provide detailed explanations.
[0063] This application provides an image processing method, which can be executed by a terminal or a server, or by both a terminal and a server. This application uses the example of an image processing method executed by a server to illustrate the method.
[0064] Please see Figure 2 , Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this application. The image processing method includes:
[0065] In step 101, the image to be processed and its corresponding size information are obtained.
[0066] The image to be processed can be a game image from a game client, a frame from a video client, or an image from an instant messaging client, etc. The format of the image to be processed can be bitmap (bmp), JPEG (Joint Photographic Experts Group), etc., without specific limitations.
[0067] To attract users to watch videos or games related to the image being processed, a thumbnail of that image needs to be created. This thumbnail, also known as a poster image, can be used as a promotional tool for the medium. Because of the thumbnail... Figure 1 Generally, the image has a fixed size, which differs from the aspect ratio of the image to be processed. The thumbnail is a part of the image to be processed, meaning that the image to be processed needs to be cropped. Existing cropping is generally done manually, which is inefficient. Therefore, in order to achieve subsequent automatic cropping, this embodiment of the application can pre-obtain the image to be processed and its corresponding size information. This size information can be width multiplied by height, for example, 1280 pixels multiplied by 720 pixels.
[0068] In step 102, the target center point is determined based on the location information of the facial feature information in the image to be processed.
[0069] In practical applications, for images containing faces, the faces are cropped as the main subject of the image. Therefore, the face region needs to be prioritized and preserved as the target center point.
[0070] In this embodiment, facial feature information can be identified using computer vision technology. Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, where cameras and computers replace human eyes to identify, detect, and measure targets, and then perform image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision researches related theories and technologies, attempting to establish artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technology typically includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition. The facial feature information can be the characteristic expression information of the facial features. The center point of the location information of the facial feature information can be used as the target center point, thereby enabling subsequent cropping of the face as the main subject.
[0071] In some implementations, the step of determining the target center point based on the location information of facial features in the image to be processed may include:
[0072] (1) Obtain the corresponding bounding boxes for the facial features in the image to be processed;
[0073] (2) When a single bounding box is detected in the image to be processed, the target center point is determined based on the single bounding box;
[0074] (3) When multiple bounding boxes are detected in the image to be processed, the multiple bounding boxes are connected to generate a target connection box, and the target center point is determined based on the target connection box.
[0075] Among them, convolutional neural networks (CNNs) can be used to identify facial features in the image to be processed. These CNNs can be constructed by mimicking the visual perception mechanism of biological organisms, so they can be used to identify facial features in the image to be processed, and the facial features can be selected with a rectangular or circular bounding box.
[0076] Furthermore, since the image to be processed can contain one or more faces, when there are multiple faces, all faces need to be displayed together as the main subject. Therefore, it is possible to pre-detect whether the image to be processed contains one or more bounding boxes, each bounding box representing one face. When a single bounding box is detected in the image to be processed, it means that the image to be processed contains only a single face, and the center point of that single bounding box can be directly used as the target center point.
[0077] When multiple bounding boxes are detected in the image to be processed, it means that the image contains multiple faces. These multiple bounding boxes can be connected into a target bounding box region, and the center point of the target bounding box can be used as the target center point to ensure that subsequent cropping can retain as many faces as possible as the main subject.
[0078] In step 103, the cutting dimension information is determined based on the aspect ratio relationship between the dimension information and the target dimension information.
[0079] The target size information refers to the size information of the thumbnail (i.e., the poster image), which is the size information of the image that the system or user wants to obtain. In order to improve the efficiency of cropping, the aspect ratio refers to the ratio between the width and height of the image.
[0080] When the aspect ratio of the image to be processed is the same as that of the thumbnail, the size of the image to be processed can be directly scaled to the size of the thumbnail to achieve thumbnail creation. However, in actual use, the aspect ratio of the image to be processed is different from that of the thumbnail. In order to achieve thumbnail creation, the aspect ratio of the image to be processed can be converted to be the same as that of the thumbnail. For easy cropping, one side of the image to be processed can be fixed and the other side can be cropped so that the cropped image has the same aspect ratio as the thumbnail.
[0081] Specifically, when the aspect ratio of the image to be processed is greater than that of the thumbnail, the height side can be fixed and the width side can be cropped. Assuming that the width and height of the image to be processed are w and h, and the width and height of the thumbnail are w' and h', in order to make w / h=w' / h', the cropped width w1=hw' / h' can be calculated. Using w1 and h as the cropping size information to crop the image to be processed can make the aspect ratio of the cropped image the same as that of the thumbnail.
[0082] When the aspect ratio of the image to be processed is smaller than that of the thumbnail, the width side can be fixed and the height side can be cropped. Assuming that the width and height of the original image are w and h respectively, and the width and height of the cropped image are w' and h' respectively, in order to make w / h=w' / h', the cropped height h1=wh' / w' can be calculated. Using w and h1 as the cropping size information to crop the image to be processed can make the aspect ratio of the cropped image the same as that of the thumbnail.
[0083] In step 104, the image to be processed is cropped based on the target center point and cropping size information to obtain the target image.
[0084] In order to unify the aspect ratio of the image to be processed and the thumbnail, and to create a poster image with thumbnail size information, the target center point can be used as a relative reference center, that is, the target center point can be used as a cropping reference center. The image to be processed is cropped using the cropping size information to obtain a target image with the same aspect ratio as the thumbnail.
[0085] In step 105, the target image is scaled to the target size information.
[0086] Since the aspect ratio of the cropped target image is the same as that of the target image, the target image can be directly scaled to the target size information to obtain a poster image with the face as the main subject and the same size as the thumbnail for promotion. This allows operators to process a large number of images more conveniently and quickly, and display the target image scaled to the target size information, which can effectively promote the processed images.
[0087] In one implementation, custom layers or controls can be added to the scaled target image to enhance the promotional effect.
[0088] As described above, this embodiment of the application obtains the image to be processed and its corresponding size information; determines the target center point based on the position information of facial features in the image to be processed; determines the cropping size information based on the aspect ratio between the size information and the target size information; crops the image to be processed based on the target center point and the cropping size information to obtain the target image; and scales the target image to the target size information. Thus, by determining the target center point of the image based on facial features and cropping it according to the aspect ratio between the size information and the target size information, a target image with a compliant aspect ratio is obtained. The target image is then scaled to the target size information, quickly generating a thumbnail that meets the requirements. Compared to solutions that require manual selection and editing of poster images, this embodiment of the application eliminates the need for manual intervention, greatly improving the efficiency of image processing.
[0089] Based on the methods described in the above embodiments, the following examples will provide further detailed explanations.
[0090] In this embodiment, the image processing device will be specifically integrated into the server as an example for explanation. Please refer to the following description for details.
[0091] Please see Figure 3 , Figure 3 This is another schematic flowchart illustrating the image processing method provided in an embodiment of this application. The method flow may include:
[0092] In step 201, the server obtains the image to be processed and its corresponding size information.
[0093] Please refer to the following: Figure 4a As shown, the image to be processed 10 can be a frame from an XX video. In order to create a poster for the XX video for promotion, this embodiment of the application can obtain the image to be processed 10 and its corresponding size information in advance. The size information can be 1280 pixels multiplied by 720 pixels.
[0094] In step 202, the server extracts facial feature information from the image to be processed using a trained multi-task convolutional neural network model, and determines the corresponding bounding box based on the location information of the facial feature information.
[0095] The Multi-task Convolutional Neural Network (MTCNN) is a multi-task neural network model for face detection. This model primarily employs three cascaded networks, using a candidate bounding box plus classifier approach for fast and efficient face detection. For further details, please refer to... Figure 4aAs shown, the server can extract facial feature information from the image to be processed 10 through the trained MTCNN model, and then mark the identification box 11 (candidate box) and the identification box 12 according to the position information of the facial feature information. The identification box can be a rectangle and contains facial feature information of a single face.
[0096] In step 203, when the server detects that the image to be processed contains a single bounding box, the target center point is determined based on the single bounding box.
[0097] When the server detects that the image to be processed contains only a single bounding box, it can directly use the center point of the rectangle of that single bounding box as the target center point.
[0098] In step 204, when the server detects that the image to be processed contains multiple bounding boxes, it connects the multiple bounding boxes to generate a target connection box, and determines the target center point based on the target connection box.
[0099] Please refer to the following: Figure 4b As shown, when the server detects that the image 10 to be processed contains multiple bounding boxes, namely bounding box 11 and bounding box 12, it connects bounding box 11 and bounding box 12 to generate a target connection box 13 containing bounding box 11 and bounding box 12, and uses the center point of the target connection box 13 as the target center point 14 to ensure that subsequent cropping can retain as many faces as possible as the main body.
[0100] In step 205, when the server detects that the aspect ratio of the size information is greater than the aspect ratio of the target size information, it acquires the first height information of the size information, the second width information of the target size information, and the second height information of the target size information. Based on the first height information, the second width information, and the second height information, it calculates the third width information and determines the cutting size information according to the third width information and the first height information.
[0101] In practical use, the aspect ratio of the image to be processed is different from that of the thumbnail (i.e., the poster image). In order to create a thumbnail, this embodiment of the application can convert the aspect ratio of the image to be processed to be the same as that of the thumbnail. Specifically:
[0102] Please refer to the following: Figure 4cAs shown, the first width information of the image to be processed is w, and the first height information is h. The second width information of the target size information of the thumbnail is w', and the second height information is h'. When the server detects that the aspect ratio w / h of the size information is greater than the aspect ratio w' / h of the target size information of the thumbnail, the first height information h side can be fixed, and the first width information w side can be cropped. In order to make w / h=w' / h', the third width information w1 after cropping can be calculated as hw' / h'. Using w1 and h as the cropping size information to crop the image to be processed can make the aspect ratio of the cropped image the same as that of the thumbnail.
[0103] In step 206, when the server detects that the aspect ratio of the size information is less than the aspect ratio of the target size information, it acquires the first width information of the size information, the second width information of the target size information, and the second height information of the target size information. Based on the first width information, the second width information, and the second height information, it calculates the third height information and determines the cutting size information according to the third height information and the first width information.
[0104] Specifically, when the server detects that the aspect ratio w / h of the size information is less than the aspect ratio w' / h' of the target size information of the thumbnail, the first width information w side can be fixed and the first height information h side can be cropped. In order to make w / h=w' / h', the third height information h1 after cropping can be calculated as h1=wh' / w'. Using w and h1 as the cropping size information to crop the image to be processed, the cropped image can be made to have the same aspect ratio as the thumbnail.
[0105] In step 207, the server crops the image to be processed according to the cropping size information to obtain the cropped image.
[0106] Please refer to the following: Figure 4c As shown, the server can use the target center point as the cropping reference center, and crop the image to be processed according to the cropping size information to obtain the cropped image defined by the dashed frame.
[0107] In step 208, when the server detects that the target center point in the cropped image is located in the upper half of the cropped image, the cropped image is determined as the target image.
[0108] Please refer to the following: Figure 4dAs shown, the cropped image 23 contains a midline C, which divides the cropped image into an upper and lower half. In real-world scenarios, cropping when the face is in the lower half will result in the face in the thumbnail being in the lower half of the image, affecting the thumbnail's promotional effect and aesthetics. Therefore, the server can pre-detect whether the target center point of the cropped image is located in the upper half of the cropped image. When the server detects that the target center point in the cropped image is located in the upper half of the cropped image, it indicates that the face is in the upper half of the image, and the cropped image can be directly identified as the target image.
[0109] In step 209, when the server detects that the position of the target center point in the cropped image is not in the upper half of the cropped image, it obtains the first target identification box corresponding to the target face feature information in the image to be processed.
[0110] Please refer to the following: Figure 4d As shown, when the server detects that the center point of the target in the cropped image is not located in the upper half of the cropped image, it indicates that the face 21 is located in the lower half of the image. To avoid affecting the promotional effect and aesthetics of the thumbnail, this embodiment of the application can obtain the first target identification box 22 corresponding to the target face feature information in the image to be processed. The area of the first target identification box can be represented by the coordinates (x1, y1) of the upper left corner and the coordinates (x2, y2) of the lower right corner.
[0111] In step 210, the server expands the first target identifier frame by a preset multiple to obtain the expanded second target identifier frame, and obtains the fourth width information, the fourth height information, the second width information of the target size information, and the second height information of the target size information of the second target identifier frame.
[0112] The preset multiplier can be 2 or 3, etc., and can be set according to the actual image processing situation or the user. Please continue reading. Figure 4d In this embodiment of the application, the preset multiple is 2. The server can expand the first target identification box 22 by 2 times to obtain the expanded second target identification box 24. The area of the second target identification box 24 can be represented by the coordinates of the upper left corner (x3, y3) and the coordinates of the lower right corner (x4, y4). Since the aspect ratio of the second target identification box is different from that of the thumbnail, the second target identification box needs to be supplemented to obtain the fourth width information x4-x3, the fourth height information y4-y3, the second width information w' of the target size information, and the second height information h' of the target size information.
[0113] In step 211, the server calculates the fifth height information based on the fourth width information, the second width information, and the second height information. It determines the sixth height information based on the difference between the fifth height information and the fourth height information. It determines the size information of the additional frame based on the fourth width information and the sixth height information. The second target identification frame is merged with the additional frame to obtain the third target identification frame. The third target identification frame is determined as the target image.
[0114] The server can calculate the fifth height information of the second target identification frame 24 when the aspect ratio of the second target identification frame 24 is equal to that of the thumbnail, based on the fixed width of the second target identification frame 24 using the formula h'(x4-x3) / w'.
[0115] Furthermore, the sixth altitude information is determined based on the difference between the fifth and fourth altitude information. Please refer to the above information. Figure 4e As shown, the size information of the additional frame 25 connected below the second target identification frame 24 is determined according to the fourth width information and the sixth height information. The second target identification frame 24 and the additional frame 25 are merged to obtain the third target identification frame. The aspect ratio of the third target identification frame is equal to that of the thumbnail, and thus the third target identification frame can be determined as the target image.
[0116] In one embodiment, to enhance the aesthetics of the target image, the server can select color information from the intersection of the second target identifier box 24 and the additional box 25, determine the target color information with the most color information, such as purple, and then use this purple to generate the color of the additional box 25. Furthermore, to further enhance the aesthetics of the target image, a transparency gradient can be applied from the center of the target image to the segmentation position of the additional box 25.
[0117] In one embodiment, when it is detected that a region in the target image does not belong to the image to be processed, the target image is deleted.
[0118] To prevent the main body of the target image from extending beyond the image to be processed, it's necessary to detect whether a region in the target image belongs to the image to be processed. In other words, does the target image contain any parts that don't belong to the image to be processed? For example, if the center point of the target image is (x5, y5), the coordinates of the center point, along with the width and height of the target image, can be used to calculate whether the target image extends beyond the image to be processed. If a region in the target image is detected as not belonging to the image to be processed, meaning the target image extends beyond the image to be processed, then the target image is considered unqualified and cannot be cropped; it can be deleted. Conversely, if a region in the target image is detected as belonging to the image to be processed, meaning the target image does not extend beyond the area of the image to be processed, then the target image is considered qualified, and subsequent steps can be performed.
[0119] In step 212, when the server detects that the sharpness information of the scaled target image meets the preset threshold, the scaled target image is retained, a preset display control is added to the scaled target image, and the target image is scaled to the target size information.
[0120] The server can also filter the sharpness of the target image, removing blurry images to ensure image quality. This embodiment uses the Laplacian operator from OpenCV (a cross-platform computer vision and machine learning software library) to evaluate sharpness information. The principle is that a high variance in an image indicates a broad response, including edge-like and edge-less elements, representing a normal focused image. Conversely, a low variance indicates minimal response diffusion, suggesting few edges. The more blurred the image, the fewer edges it has. Therefore, the Laplacian operator can be used to detect blurriness, and for edge detection, a reasonable preset threshold can be set, such as 0.8.
[0121] When the server detects that the sharpness information of the scaled target image is greater than 0.8, meaning that the sharpness information of the target image meets the preset threshold, the scaled target image can be retained. Please continue reading. Figure 4e Furthermore, preset display controls 26 can be added to the scaled target image to enhance its promotional effect.
[0122] Finally, the target image can be scaled to the target size. Since the aspect ratio is the same, the target image can be directly scaled to the target size to generate a corresponding poster for promotion, without manual intervention, which greatly improves the efficiency of image processing.
[0123] As described above, this embodiment of the application obtains the image to be processed and its corresponding size information; determines the target center point based on the position information of facial features in the image to be processed; determines the cropping size information based on the aspect ratio between the size information and the target size information; crops the image to be processed based on the target center point and the cropping size information to obtain the target image; and scales the target image to the target size information. Thus, by determining the target center point of the image based on facial features and cropping it according to the aspect ratio between the size information and the target size information, a target image with a compliant aspect ratio is obtained. The target image is then scaled to the target size information, quickly generating a thumbnail that meets the requirements. Compared to solutions that require manual selection and editing of poster images, this embodiment of the application eliminates the need for manual intervention, greatly improving the efficiency of image processing.
[0124] Furthermore, the embodiments of this application can also generate a target image by expansion when the target center point in the cropped image is detected to be not located in the upper half of the cropped area, ensuring that the face is always in a reasonable position in the target image without the need for subsequent manual modification, thereby further improving the efficiency of image processing and ensuring the effectiveness of the publicity.
[0125] To facilitate better implementation of the image processing method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above-described image processing method. The meanings of the terms used are the same as in the image processing method described above, and specific implementation details can be found in the descriptions in the method embodiments.
[0126] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. The image processing device may include an acquisition unit 301, a first determination unit 302, a second determination unit 303, a cropping unit 304, and a scaling unit 305, etc.
[0127] The acquisition unit 301 is used to acquire the image to be processed and the corresponding size information of the image to be processed.
[0128] The first determining unit 302 is used to obtain the image to be processed and the corresponding size information of the image to be processed.
[0129] In some embodiments, the first determining unit 302 includes:
[0130] The acquisition subunit is used to acquire the corresponding bounding boxes of facial feature information in the image to be processed.
[0131] The first determining subunit is used to determine the target center point based on the single identifier box when the image to be processed is detected to contain a single identifier box.
[0132] The second determining subunit is used to connect the multiple bounding boxes when it is detected that the image to be processed contains multiple bounding boxes, generate a target connection box, and determine the target center point based on the target connection box.
[0133] In some implementations, the acquisition subunit is used for:
[0134] The facial feature information in the image to be processed is extracted by the trained multi-task convolutional neural network model.
[0135] The corresponding bounding box is determined based on the location information of the facial feature.
[0136] The second determining unit 303 is used to determine the cutting size information based on the aspect ratio relationship between the size information and the target size information.
[0137] In some embodiments, the second determining unit 303 includes a third determining subunit, used for:
[0138] When the aspect ratio of the dimension information is detected to be greater than the aspect ratio of the target dimension information, the first height information of the dimension information, the second width information of the target dimension information, and the second height information of the target dimension information are obtained.
[0139] The third width information is calculated based on the first height information, the second width information, and the second height information.
[0140] The cutting dimensions are determined based on the third width information and the first height information.
[0141] In some embodiments, the second determining unit 303 further includes a fourth determining subunit, configured to:
[0142] When it is detected that the aspect ratio of the dimension information is less than the aspect ratio of the target dimension information, the first width information of the dimension information, the second width information of the target dimension information, and the second height information of the target dimension information are obtained.
[0143] The third height information is calculated based on the first width information, the second width information, and the second height information.
[0144] The cutting dimensions are determined based on the third height information and the first width information.
[0145] The cropping unit 304 is used to crop the image to be processed based on the target center point and the cropping size information to obtain a target image, wherein the aspect ratio of the target image is the same as the aspect ratio of the target size information.
[0146] In some embodiments, the cutting unit 304 includes:
[0147] The cropping subunit is used to crop the image to be processed according to the cropping size information to obtain a cropped image, which contains the target center point;
[0148] The fifth determining subunit is used to determine the cropped image as the target image when the position of the target center point in the cropped image is detected to be in the upper half of the cropped image.
[0149] In some embodiments, the trimming unit further includes:
[0150] The acquisition subunit is used to acquire the first target identification box corresponding to the target face feature information in the image to be processed when it is detected that the position of the target center point in the cropped image is not in the upper half of the cropped image.
[0151] An expansion subunit is used to expand the first target identifier box by a preset multiple to obtain an expanded second target identifier box;
[0152] Add a sub-unit to add an additional frame to the second target identifier frame to obtain the added third target identifier frame. The aspect ratio of the third target identifier frame is the same as the aspect ratio of the target size information.
[0153] The sixth determining sub-unit is used to determine the third target identification box as the target image.
[0154] In some embodiments, the addition of a subunit is used for:
[0155] Obtain the fourth width information, fourth height information, second width information of target size information, and second height information of target size information of the second target identifier box;
[0156] The fifth height information is calculated based on the fourth width information, the second width information, and the second height information.
[0157] The sixth altitude information is determined based on the difference between the fifth altitude information and the fourth altitude information;
[0158] The dimensions of the additional frame are determined based on the fourth width information and the sixth height information.
[0159] The second target identifier box is merged with the additional box to obtain the third target identifier box.
[0160] In some embodiments, the apparatus further includes a deletion unit for:
[0161] If a region in the target image is detected to not belong to the image to be processed, the target image is deleted.
[0162] The scaling unit 305 is used to scale the target image to the target size information.
[0163] In some embodiments, the apparatus further includes an adding unit for:
[0164] When the sharpness information of the scaled target image meets the preset threshold, the scaled target image is retained.
[0165] Add a preset display control to the scaled target image.
[0166] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.
[0167] As described above, this embodiment of the application acquires the image to be processed and its corresponding size information through the acquisition unit 301; the first determining unit 302 determines the target center point based on the position information of the facial feature information in the image to be processed; the second determining unit 303 determines the cropping size information based on the aspect ratio between the size information and the target size information; the cropping unit 304 crops the image to be processed based on the target center point and the cropping size information to obtain the target image; and the scaling unit 305 scales the target image to the target size information. Thus, by determining the target center point of the image based on the facial feature information and determining the cropping size information based on the aspect ratio between the size information and the target size information for cropping, a target image with a compliant aspect ratio is obtained. The target image is then scaled to the target size information, quickly generating a thumbnail that meets the requirements. Compared to solutions that require manual selection and editing of poster images, this embodiment of the application eliminates the need for manual intervention, greatly improving the efficiency of image processing.
[0168] This application also provides a computer device, which can be a server or a terminal, such as... Figure 6 As shown, it illustrates a schematic diagram of the server structure involved in an embodiment of this application. Specifically:
[0169] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 6 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0170] Processor 401 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 402, and by calling data stored in memory 402, it performs various functions of the computer device and processes data, thereby performing overall detection of the computer device. Optionally, processor 401 may include one or more processing cores; optionally, processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into processor 401.
[0171] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.
[0172] The computer equipment also includes a power supply 403 that supplies power to the various components. Optionally, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0173] The computer device may also include an input unit 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
[0174] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402, thereby implementing the various method steps provided in the foregoing embodiments, as follows:
[0175] The process involves: acquiring the image to be processed and its corresponding size information; determining the target center point based on the location information of facial features in the image to be processed; determining the cropping size information based on the aspect ratio between the size information and the target size information; cropping the image to be processed based on the target center point and the cropping size information to obtain the target image, the aspect ratio of which is the same as that of the target size information; and scaling the target image to the target size information.
[0176] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the image processing method above, which will not be repeated here.
[0177] As described above, the computer device in this embodiment can obtain the image to be processed and its corresponding size information; determine the target center point based on the position information of facial features in the image to be processed; determine the cropping size information based on the aspect ratio between the size information and the target size information; crop the image to be processed based on the target center point and the cropping size information to obtain the target image; and scale the target image to the target size information. Thus, by determining the target center point of the image based on facial features and cropping it based on the aspect ratio between the size information and the target size information, a target image with a compliant aspect ratio is obtained. The target image is then scaled to the target size information, quickly generating a thumbnail that meets the requirements. Compared to solutions that require manual selection and editing of poster images, this embodiment requires no manual intervention, greatly improving the efficiency of image processing.
[0178] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0179] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the image processing methods provided in embodiments of this application. For example, the instructions can execute the following steps:
[0180] The process involves: acquiring the image to be processed and its corresponding size information; determining the target center point based on the location information of facial features in the image to be processed; determining the cropping size information based on the aspect ratio between the size information and the target size information; cropping the image to be processed based on the target center point and the cropping size information to obtain the target image, the aspect ratio of which is the same as that of the target size information; and scaling the target image to the target size information.
[0181] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.
[0182] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0183] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0184] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the image processing methods provided in the embodiments of this application, the beneficial effects that any of the image processing methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0185] The foregoing has provided a detailed description of an image processing method, apparatus, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. An image processing method, characterized in that, include: Obtain the image to be processed and its corresponding size information; Determining the target center point based on the location information of facial feature information in the image to be processed includes: obtaining the corresponding bounding boxes of facial feature information in the image to be processed; when multiple bounding boxes are detected in the image to be processed, connecting the multiple bounding boxes to generate a target connection box containing multiple bounding boxes, and taking the center point of the target connection box as the target center point, where each bounding box represents a face. The cutting dimensions are determined based on the aspect ratio between the stated dimensions and the target dimensions. Cropping the image to be processed based on the target center point and the cropping size information to obtain a target image includes: using the target center point as a cropping reference center, cropping the image to be processed according to the cropping size information to obtain a cropped image, the cropped image including a midline, the midline being used to divide the cropped image into an upper half region and a lower half region; when it is detected that the position of the target center point in the cropped image is not in the upper half region of the cropped image, obtaining a first target identification box corresponding to the target facial feature information in the image to be processed; expanding the first target identification box by a preset multiple to obtain an expanded second target identification box, obtaining the fourth width information, the fourth height information, the second width information, and the second height information of the target size information of the second target identification box; based on the fourth... The fifth height information is calculated from the width information, the second width information, and the second height information. The sixth height information is determined based on the difference between the fifth height information and the fourth height information. The size information of the additional frame connected below the second target identifier frame is determined based on the fourth width information and the sixth height information. The second target identifier frame and the additional frame are merged to obtain a third target identifier frame. The aspect ratio of the third target identifier frame is the same as the aspect ratio of the target size information. The third target identifier frame is determined as the target image. The aspect ratio of the target image is the same as the aspect ratio of the target size information. The target image contains the target center point, and the position of the target center point in the target image is located in the upper half of the target image. The target image is scaled to the target size information.
2. The image processing method according to claim 1, characterized in that, Determining the target center point based on the location information of facial features in the image to be processed includes: When a single bounding box is detected in the image to be processed, the target center point is determined based on the single bounding box.
3. The image processing method according to claim 2, characterized in that, The step of obtaining the corresponding bounding box for the facial feature information in the image to be processed includes: Facial feature information in the image to be processed is extracted using a trained multi-task convolutional neural network model. The corresponding bounding box is determined based on the location information of the facial feature information.
4. The image processing method according to claim 1, characterized in that, Determining the cutting size information based on the aspect ratio relationship between the size information and the target size information includes: When it is detected that the aspect ratio of the size information is greater than the aspect ratio of the target size information, the first height information of the size information, the second width information of the target size information, and the second height information of the target size information are obtained. The third width information is calculated based on the first height information, the second width information, and the second height information. The cutting dimensions are determined based on the third width information and the first height information.
5. The image processing method according to claim 1, characterized in that, Determining the cutting size information based on the aspect ratio relationship between the size information and the target size information includes: When it is detected that the aspect ratio of the size information is less than the aspect ratio of the target size information, the first width information of the size information, the second width information of the target size information, and the second height information of the target size information are obtained. The third height information is calculated based on the first width information, the second width information, and the second height information; The cutting dimensions are determined based on the third height information and the first width information.
6. The image processing method according to any one of claims 1 to 5, characterized in that, The process of cropping the image to be processed based on the target center point and the cropping size information to obtain the target image includes: When the target center point in the cropped image is detected to be located in the upper half of the cropped image, the cropped image is determined as the target image.
7. The image processing method according to claim 1, characterized in that, The method further includes: When the sharpness information of the scaled target image meets the preset threshold, the scaled target image is retained; Add a preset display control to the scaled target image.
8. The image processing method according to claim 1, characterized in that, The method further includes: When a region in the target image is detected to not belong to the image to be processed, the target image is deleted.
9. An image processing apparatus, characterized in that, include: The acquisition unit is used to acquire the image to be processed and the corresponding size information of the image to be processed; A first determining unit is configured to determine a target center point based on the location information of facial feature information in the image to be processed; wherein, the first determining unit includes: an acquisition subunit, configured to acquire corresponding bounding boxes of facial feature information in the image to be processed; and a second determining subunit, configured to connect the multiple bounding boxes to generate a target connection box containing multiple bounding boxes when it is detected that the image to be processed contains multiple bounding boxes, and to use the center point of the target connection box as the target center point, wherein each bounding box represents a face. The second determining unit is used to determine the cutting size information based on the aspect ratio relationship between the size information and the target size information; A cropping unit is used to crop the image to be processed based on the target center point and the cropping size information to obtain a target image. The cropping unit includes: using the target center point as a cropping reference center, cropping the image to be processed according to the cropping size information to obtain a cropped image, wherein the cropped image includes a midline that divides the cropped image into an upper and lower half; when it is detected that the position of the target center point in the cropped image is not in the upper half of the cropped image, acquiring a first target identifier box corresponding to the target facial feature information in the image to be processed; expanding the first target identifier box by a preset multiple to obtain an expanded second target identifier box, and acquiring fourth width information, fourth height information, and the second width of the target size information of the second target identifier box. Information, second height information; based on the fourth width information, the second width information, and the second height information, calculate the fifth height information; determine the sixth height information based on the difference between the fifth height information and the fourth height information; determine the size information of the additional frame connected below the second target identification frame based on the fourth width information and the sixth height information; merge the second target identification frame and the additional frame to obtain the third target identification frame, the aspect ratio of the third target identification frame is the same as the aspect ratio of the target size information; determine the third target identification frame as the target image, the aspect ratio of the target image is the same as the aspect ratio of the target size information, the target image contains the target center point, and the position of the target center point in the target image is located in the upper half of the target image; A scaling unit is used to scale the target image to the target size information.
10. The image processing apparatus according to claim 9, characterized in that, The first determining unit includes: The first determining subunit is used to determine the target center point based on the single identifier box when the image to be processed is detected to contain a single identifier box.
11. The image processing apparatus according to claim 10, characterized in that, The acquisition subunit is used for: Facial feature information in the image to be processed is extracted using a trained multi-task convolutional neural network model. The corresponding bounding box is determined based on the location information of the facial feature information.
12. The image processing apparatus according to claim 9, characterized in that, The second determining unit includes a third determining subunit, used for: When it is detected that the aspect ratio of the size information is greater than the aspect ratio of the target size information, the first height information of the size information, the second width information of the target size information, and the second height information of the target size information are obtained. The third width information is calculated based on the first height information, the second width information, and the second height information. The cutting dimensions are determined based on the third width information and the first height information.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the image processing method according to any one of claims 1 to 8.