Image processing method and apparatus

Abstract

The application discloses an image processing method and device. The method comprises the following steps: obtaining a to-be-processed image containing head information of a target object; performing encoding processing on the to-be-processed image through an encoding network of a segmentation model to obtain encoding features; performing decoding processing on the encoding features through a first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features; and performing decoding processing on the encoding features and the head region image through a second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features. The application solves the technical problem that the image processing resources are consumed more and the time is consumed longer when predicting the head region and the facial features in the related art.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an image processing method and apparatus. Background Technology

[0002] With the development of image processing technology, more and more character special effects designs are emerging in interactive entertainment scenarios, especially head special effects, such as hair color changing, facial area special effects, and facial feature special effects.

[0003] It should be noted that before applying head effects, the hair region, face region, and facial features need to be obtained from the given image. To achieve this, related technologies use separate modules to predict the head, face, hair, and facial features, which consumes significant image processing resources and takes a long time.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides an image processing method and apparatus to at least solve the technical problems of high image processing resource consumption and long processing time when predicting head regions and facial features in related technologies.

[0006] According to one aspect of the present invention, an image processing method is provided, comprising: acquiring an image to be processed containing head information of a target object; encoding the image to be processed through an encoding network of a segmentation model to obtain encoded features; decoding the encoded features through a first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features; and decoding the encoded features and the head region image through a second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features.

[0007] Optionally, the encoded features include multiple encoded features of different resolutions, and the head region image includes multiple head region images of different resolutions. The second decoding network of the segmentation model decodes the encoded features and the head region images to obtain the facial organ image, which includes: combining multiple encoded features of different resolutions and multiple head region images of different resolutions through the second decoding network, and decoding the combined features to obtain the facial organ image.

[0008] Optionally, before encoding the image to be processed through the encoding network of the segmentation model to obtain encoded features, the method further includes: acquiring multiple sets of first sample data, and training a preset encoding network and a preset first decoding network based on the multiple sets of first sample data to obtain a pre-trained encoding network and a pre-trained first decoding network, wherein the first sample data consists of a first sample image containing head information of the target object and a first labeled image, and the first labeled image is an image in the first sample image labeled with facial region features; acquiring multiple sets of second sample data, and training a preset second decoding network based on the multiple sets of second sample data to obtain a pre-trained second decoding network, wherein the second sample data consists of a second sample image containing head information of the target object and a second labeled image, and the second labeled image is an image in the second sample image labeled with facial organ features; and obtaining a segmentation model from the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network.

[0009] Optionally, obtaining a segmentation model from a pre-trained encoding network, a pre-trained first decoding network, and a pre-trained second decoding network includes: acquiring multiple sets of third sample data, and training the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network based on the multiple sets of third sample data to obtain the encoding network, the first decoding network, and the second decoding network. The third sample data consists of sample images containing head information of the target object and third labeled images, and the third labeled images are images in which facial region features and facial organ features are labeled. The segmentation model is composed of the encoding network, the first decoding network, and the second decoding network.

[0010] Optionally, the first labeled image is an image in which head region features, face region features, and hair region features are labeled in the first sample image. The head region image is obtained by decoding the encoded features through the first decoding network of the segmentation model. This includes: inputting the encoded features into the first decoding network for processing to obtain a first head region image, a second head region image, and a third head region image. The first head region image includes head region features, the second head region image includes face region features, and the third head region image includes hair region features.

[0011] Optionally, before acquiring the image to be processed containing the head information of the target object, the method further includes: acquiring an image containing the target object and locating the head position from the image containing the target object, wherein the human body information includes head information and torso information; and acquiring the image to be processed containing the head information of the target object from the image containing the target object based on the head position.

[0012] According to another aspect of the present invention, an image processing method is also provided, comprising: acquiring an image to be processed containing head information of a target object, and displaying the image to be processed in a user interface; displaying a head region image obtained by processing the image to be processed through a segmentation model in the user interface, wherein the head region image contains facial region features; and displaying a facial organ image obtained by processing the head region image through a segmentation model in the user interface, wherein the facial organ image contains facial organ features.

[0013] According to another aspect of the present invention, an image processing method is also provided, comprising: a cloud server receiving an image to be processed containing head information of a target object; the cloud server encoding the image to be processed using an encoding network of a segmentation model to obtain encoded features; the cloud server decoding the encoded features using a first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features; the cloud server decoding the encoded features and the head region image using a second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features; and the cloud server returning the facial organ image to a client.

[0014] According to another aspect of the present invention, an image processing apparatus is also provided, comprising: an acquisition unit for acquiring an image to be processed containing head information of a target object; an encoding unit for encoding the image to be processed through an encoding network of a segmentation model to obtain encoded features; a first decoding unit for decoding the encoded features through a first decoding network of a segmentation model to obtain a head region image, wherein the head region image contains facial region features; and a second decoding unit for decoding the encoded features and the head region image through a second decoding network of a segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features.

[0015] According to another aspect of the present invention, a storage medium is also provided, wherein the storage medium includes a stored program, wherein the program controls the device where the storage medium is located to execute the image processing method described above during runtime.

[0016] According to another aspect of the present invention, a processor is also provided, wherein the processor is configured to run a program, wherein the program executes the image processing method described above during runtime.

[0017] In this embodiment of the invention, an image to be processed containing head information of the target object is obtained; the image to be processed is encoded using the encoding network of a segmentation model to obtain encoded features; the encoded features are decoded using the first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features; the encoded features and the head region image are decoded using the second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features. By using a segmentation model containing one encoding network and two decoding networks to process the image to be processed, the goal of further processing the obtained head region image to obtain a facial organ image is achieved. This reduces the resource and time consumption when predicting the head region and facial features, thereby solving the technical problem of high image processing resource consumption and long time consumption when predicting the head region and facial features in related technologies. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1 This is a hardware structure block diagram of a computer terminal according to an embodiment of the present invention;

[0020] Figure 2 This is a flowchart of an image processing method provided according to Embodiment 1 of the present invention;

[0021] Figure 3 This is a schematic diagram of the image processing method provided in Embodiment 1 of the present invention;

[0022] Figure 4 This is a flowchart of the image processing method provided in Embodiment 2 of the present invention;

[0023] Figure 5 This is a flowchart of the image processing method provided in Embodiment 3 of the present invention;

[0024] Figure 6 This is a schematic diagram of an image processing apparatus provided according to Embodiment 4 of the present invention; and

[0025] Figure 7 This is a structural block diagram of an optional computer terminal according to an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0029] Image segmentation; given a visualization image, predict the salient foreground.

[0030] Example 1

[0031] According to an embodiment of the present invention, an image processing method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing an image processing method is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0033] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0034] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image processing method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the image processing method of the aforementioned application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0035] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0036] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0037] Under the aforementioned operating environment, this application provides the following: Figure 2 The image processing method shown. Figure 2 This is a flowchart of an image processing method according to Embodiment 1 of the present invention.

[0038] S21, Obtain the image to be processed containing the header information of the target object.

[0039] Specifically, the target object can be a human body, an animal body, or a robot. The head information includes hair information, skin information, facial features information, etc. The image to be processed can be an image containing the head information and torso information of the target object, or it can be an image containing the head information of the target object but not the torso information.

[0040] In the case where the image to be processed is an image containing head information of a target object but not torso information, the method may optionally further include, before obtaining the image to be processed containing head information of the target object: obtaining an image containing the target object and locating the head position from the image containing the target object, wherein the target object includes head information and torso information; and obtaining the image to be processed containing head information of the target object from the image containing the target object based on the head position.

[0041] Specifically, given an image containing a target object, the head position is first located, and the image to be processed, which contains the head information of the target object but does not contain the torso information, is obtained based on the head position. For example, given an image containing human body information, the image to be processed after locating the head position of the human body can be the smallest rectangular region where the head region is located. It should be noted that the head region includes the hair region and the face region.

[0042] S22, the image to be processed is encoded through the encoding network of the segmentation model to obtain encoded features.

[0043] It should be noted that the segmentation model in this embodiment includes an encoding network, a first decoding network, and a second decoding network. The encoded features obtained by the encoding network from processing the image to be processed are used as inputs to the two decoding networks, such as... Figure 3 As shown, the image to be processed, which contains the head information of the target object, can be input into the encoding network to obtain encoded features.

[0044] S23, the encoded features are decoded by the first decoding network of the segmentation model to obtain the head region image, wherein the head region image contains facial region features.

[0045] Specifically, the process of decoding the encoded features through the first decoding network is the process of predicting the head region. The prediction of the head region can include the prediction of the face region, the prediction of the hair region, and the prediction of the region composed of the face region and the hair region.

[0046] In one alternative implementation, if the image processing task is to adjust facial features, then when predicting the head region, only the facial region can be predicted.

[0047] S24, the encoded features and head region image are decoded by the second decoding network of the segmentation model to obtain the facial organ image, wherein the facial organ image contains facial organ features.

[0048] Specifically, the second decoding network's decoding process for the encoded features and the head region image involves predicting facial organs based on the head region image containing facial region features. These facial organs include one or more of the following: eyebrows, eyes, nose, and mouth. Figure 3 As shown, the encoded features and head region image are input into the second decoding network to obtain facial organ images.

[0049] In this embodiment, a segmentation model with a shared encoding network and a coexistence of a head region decoding network and a facial feature decoding network is adopted. Given an image to be processed containing head information of the target object, the model can perform head region prediction and facial feature prediction. The facial feature prediction uses the results of head region prediction, which improves the speed and accuracy of facial feature prediction.

[0050] To improve prediction performance, optionally, the encoded features include multiple encoded features of different resolutions, and the head region image includes multiple head region images of different resolutions. The encoded features and head region images are decoded by the second decoding network of the segmentation model to obtain the facial organ image. This includes: combining multiple encoded features of different resolutions and multiple head region images of different resolutions by the second decoding network, and decoding the combined features to obtain the facial organ image.

[0051] Specifically, the encoding network of the segmentation model can encode the image to be processed at different resolutions and output encoding features at different resolutions. For example, it can output encoding features at the same resolution as the image to be processed, output encoding features at 1 / 4 resolution of the image to be processed, and output encoding features at 1 / 8 resolution of the image to be processed.

[0052] Correspondingly, the first decoding network of the segmentation model decodes the encoded features at different resolutions to obtain head region images at different resolutions, for example, such as Figure 3 As shown, a head region image with the same resolution as the image to be processed, a head region image with 1 / 4 resolution of the image to be processed, and a head region image with 1 / 8 resolution of the image to be processed can be obtained.

[0053] Furthermore, after obtaining multiple head region images at different resolutions, such as Figure 3 As shown, multiple head region images of different resolutions and multiple encoded features of different resolutions are input into the second decoding network of the segmentation model for decoding processing. Specifically, the decoding processing of the second decoding network includes two parts: feature fusion and decoding, so as to continuously predict facial organs based on head region images of multiple sizes and resolutions, and finally obtain high-resolution facial organ images.

[0054] Before processing the image to be processed by the segmentation model, the model needs to be trained. Optionally, before encoding the image to be processed by the encoding network of the segmentation model to obtain encoded features, the method further includes: acquiring multiple sets of first sample data, and training a preset encoding network and a preset first decoding network based on the multiple sets of first sample data to obtain a pre-trained encoding network and a pre-trained first decoding network. The first sample data consists of a first sample image containing the head information of the target object and a first labeled image, and the first labeled image is an image in the first sample image that labels facial region features; acquiring multiple sets of second sample data, and training a preset second decoding network based on the multiple sets of second sample data to obtain a pre-trained second decoding network. The second sample data consists of a second sample image containing the head information of the target object and a second labeled image, and the second labeled image is an image in the second sample image that labels facial organ features; and obtaining the segmentation model from the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network.

[0055] Specifically, the segmentation model includes the training of an encoding network and a first decoding network, as well as the training of a second decoding network. Specifically, multiple sets of first sample images and first labeled images with facial region features are used to train the pre-built encoding network and the pre-built first decoding network. The network parameters of the trained pre-built encoding network and the pre-built first decoding network are then obtained, i.e., the pre-trained encoding network and the pre-trained first decoding network are obtained.

[0056] It should be noted that when training the encoding network and the first decoding network, the first sample image can be an image containing both the head and torso information of the target object, or an image containing the head information but not the torso information. When the first sample image contains both head and torso information, the head position needs to be located on the first sample image before the facial region features are labeled. When the first sample image contains head information but not torso information, the facial region features can be directly labeled on the first sample image to obtain the first labeled image.

[0057] Furthermore, based on the pre-trained encoding network and the pre-trained first decoding network, a pre-built preset second decoding network is trained using multiple sets of second sample images and second labeled images with facial organ features. It should be noted that when training the preset second decoding network, the network parameters of the pre-trained encoding network and the pre-trained first decoding network are given. Obtaining the network parameters of the trained preset second decoding network is equivalent to obtaining the pre-trained second decoding network.

[0058] Furthermore, it should be noted that the facial organ features annotated in the second annotated image depend on the image processing task. One or more of the following can be annotated: eyebrows, eyes, nose, and mouth. For example, if the image processing task is one-click stylization of facial features, the second annotated image is annotated with eyebrows, eyes, nose, and mouth. If the image processing task is eye magnification, the second annotated image is annotated with eyes. The embodiments of this application do not limit the number or type of facial organ features annotated. The second decoding network trained using the second sample image and the second annotated image of this application can achieve individual prediction of facial organs or overall prediction of facial organs.

[0059] To further improve the performance of the segmentation model, optionally, the segmentation model is obtained from a pre-trained encoding network, a pre-trained first decoding network, and a pre-trained second decoding network by: acquiring multiple sets of third sample data, and training the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network based on the multiple sets of third sample data to obtain the encoding network, the first decoding network, and the second decoding network. The third sample data consists of sample images containing head information of the target object and third labeled images, and the third labeled images are images in which facial region features and facial organ features are labeled. The segmentation model is composed of the encoding network, the first decoding network, and the second decoding network.

[0060] Specifically, after obtaining the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network, multiple sets of third sample images and third annotations that simultaneously annotate facial region features and facial organ features are used to train the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network. The network parameters of each pre-trained network are adjusted to obtain the encoding network, the first decoding network, and the second decoding network, that is, to obtain the segmentation model, which improves the accuracy of image prediction of the segmentation model.

[0061] In addition to predicting the face region, the encoding network and the first decoding network can also predict other regions in the head. Optionally, the first labeled image is an image in which head region features, face region features, and hair region features are labeled in the first sample image. The first decoding network of the segmentation model decodes the encoded features to obtain the head region image, which includes: inputting the encoded features into the first decoding network for processing to obtain a first head region image, a second head region image, and a third head region image. The first head region image includes head region features, the second head region image includes face region features, and the third head region image includes hair region features.

[0062] Specifically, the output of the first decoding network is related to the annotation of the first sample image and the first annotation image. When only facial region features are annotated in the first annotation image, the encoding network and the first decoding network can predict the facial region features in the image to be processed. For example... Figure 3 As shown, when the first labeled image simultaneously labels head region features, face region features, and hair region features, the encoding network and the first decoding network can predict the head region features, face region features, and hair region features in the image to be processed separately, achieving multiple outputs. Specifically, it simultaneously outputs a first head region image including head region features, a second head region image including face region features, and a third head region image including hair region features. Furthermore, when the first labeled image only labels head region features or hair region features, the encoding network and the first decoding network can predict the head region features or hair region features separately.

[0063] In this embodiment, the output of the first decoding network varies depending on the different annotation features of the first labeled image. The first decoding network can output single or multiple information depending on the task type. The various networks of the segmentation model can be decomposed and merged according to different tasks. The first decoding network can be used alone in conjunction with the encoding network to predict the head region, or the encoding network, the first decoding network and the second decoding network can be merged, and the output of the first decoding network can be used as part of the input of the second decoding network to achieve the prediction of facial features. The image prediction method is more flexible and convenient.

[0064] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0065] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0066] Example 2

[0067] In the operating environment described in Embodiment 1 above, this application provides as follows: Figure 4 The image processing method shown, Figure 4 This is a flowchart of the image processing method according to Embodiment 2 of the present invention:

[0068] S41, the cloud server receives the image to be processed, which contains the header information of the target object.

[0069] Specifically, the client sends the image to be processed, which contains the head information of the target object, to the cloud server. The target object can be a human body, an animal body, or a robot. The head information includes hair information, skin information, facial features information, etc. The image to be processed can be an image containing the head information and torso information of the target object, or it can be an image containing the head information of the target object but not the torso information.

[0070] S42, the cloud server uses the encoding network of the segmentation model to encode the image to be processed to obtain encoded features; the first decoding network of the segmentation model is used to decode the encoded features to obtain a head region image, wherein the head region image contains facial region features; the second decoding network of the segmentation model is used to decode the encoded features and the head region image to obtain a facial organ image, wherein the facial organ image contains facial organ features.

[0071] It should be noted that the cloud server stores a segmentation model, which includes an encoding network, a first decoding network, and a second decoding network. The encoded features obtained by the encoding network from processing the image to be processed are used as inputs to the two decoding networks. Specifically, the first decoding network decodes the encoded features to obtain a head region containing facial region features. The first decoding network then decodes both the encoded features and the head region containing facial region features to obtain an image of facial organs, where facial organs include one or more of the following: eyebrows, eyes, nose, and mouth.

[0072] S43, the cloud server returns facial organ images to the client.

[0073] Specifically, the cloud server encodes and decodes the image containing the head information of the target object using a stored segmentation model to obtain facial organ images, which are then sent to the client for display.

[0074] Example 3

[0075] In the operating environment described in Embodiment 1 above, this application provides as follows: Figure 5 The image processing method shown, Figure 5This is a flowchart of the image processing method according to Embodiment 3 of the present invention:

[0076] S51: Acquire the image to be processed, which contains the head information of the target object, and display the image to be processed in the user interface.

[0077] Specifically, the target object can be a human body, an animal body, or a robot. The image to be processed can be an image containing the head and torso information of the target object, or it can be an image containing the head information of the target object but not the torso information. The head information includes hair information, skin information, facial features information, etc.

[0078] The image to be processed is displayed in the user interface, which is the original image containing detailed information including all header information. This original image can be a color image.

[0079] S52, the user interface displays the head region image obtained after the image to be processed by the segmentation model, wherein the head region image contains facial region features.

[0080] It should be noted that the segmentation model includes an encoding network, a first decoding network, and a second decoding network. The encoding network processes the image to be processed to obtain encoded features, and the first decoding network decodes the encoded features to obtain the head region containing facial region features.

[0081] Specifically, the head region image displayed in the user interface can be an image that only includes facial features such as skin and facial features, but does not include hair information, and the head region image can be a binary image.

[0082] S53 displays the facial organ image obtained by processing the head region image through a segmentation model on the user interface. The facial organ image contains facial organ features.

[0083] It should be noted that the first decoding network of the segmentation model decodes the encoded features and the head region containing facial features to obtain the facial organ image.

[0084] Specifically, facial features include one or more of the following: eyebrows, eyes, nose, and mouth. The facial region image displayed on the user interface can be an image that only includes facial feature characteristics and does not include other facial features such as skin.

[0085] Example 4

[0086] According to embodiments of the present invention, an apparatus for implementing the above-described image processing method is also provided, such as... Figure 6 As shown, the device includes:

[0087] The acquisition unit 61 is used to acquire the image to be processed, which contains the header information of the target object.

[0088] The encoding unit 62 is used to encode the image to be processed through the encoding network of the segmentation model to obtain encoded features.

[0089] The first decoding unit 63 is used to decode the encoded features through the first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features.

[0090] The second decoding unit 64 is used to decode the encoded features and head region image through the second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features.

[0091] Optionally, the encoded features include multiple encoded features of different resolutions, and the head region image includes multiple head region images of different resolutions. The second decoding unit 64 is used to combine the multiple encoded features of different resolutions and the multiple head region images of different resolutions through the second decoding network, and to decode the combined features to obtain a facial organ image.

[0092] Optionally, the device further includes: a first training unit, configured to acquire multiple sets of first sample data before encoding the image to be processed through the encoding network of the segmentation model to obtain encoded features, and train a preset encoding network and a preset first decoding network based on the multiple sets of first sample data to obtain a pre-trained encoding network and a pre-trained first decoding network, wherein the first sample data consists of a first sample image containing head information of the target object and a first labeled image, and the first labeled image is an image in the first sample image labeled with facial region features; a second training unit, configured to acquire multiple sets of second sample data, and train a preset second decoding network based on the multiple sets of second sample data to obtain a pre-trained second decoding network, wherein the second sample data consists of a second sample image containing head information of the target object and a second labeled image, and the second labeled image is an image in the second sample image labeled with facial organ features; and a model building unit, configured to obtain a segmentation model from the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network.

[0093] Optionally, the model building unit includes: a training module, used to acquire multiple sets of third sample data, and train a pre-trained encoding network, a pre-trained first decoding network, and a pre-trained second decoding network based on the multiple sets of third sample data to obtain the encoding network, the first decoding network, and the second decoding network, wherein the third sample data are sample images containing head information of the target object and third labeled images, and the third labeled images are images in which facial region features and facial organ features are labeled; and a model building module, used to construct a segmentation model from the encoding network, the first decoding network, and the second decoding network.

[0094] Optionally, the first labeled image is an image in which head region features, face region features, and hair region features are labeled in the first sample image. The first decoding unit 63 includes: inputting the encoded features into the first decoding network for processing to obtain a first head region image, a second head region image, and a third head region image, wherein the first head region image includes head region features, the second head region image includes face region features, and the third head region image includes hair region features.

[0095] Optionally, the device further includes: a positioning unit, configured to acquire an image containing the target object before acquiring an image to be processed containing the head information of the target object, and to locate the head position from the image containing the target object, wherein the target object includes head information and torso information; and an acquisition unit, configured to acquire the image to be processed containing the head information of the target object from the image containing the target object based on the head position.

[0096] It should be noted that the acquisition unit 61, encoding unit 62, first decoding unit 63, and second decoding unit 64 mentioned above correspond to steps S21 to S24 in Embodiment 1. The two modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0097] Example 5

[0098] Embodiments of the present invention can provide a computer terminal, which can be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the computer terminal can also be replaced by a mobile terminal or other terminal device.

[0099] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0100] In this embodiment, the computer terminal described above can execute the program code for the following steps in the image processing method of the application: acquiring an image to be processed containing head information of the target object; encoding the image to be processed through the encoding network of the segmentation model to obtain encoded features; decoding the encoded features through the first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features; and decoding the encoded features and the head region image through the second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features.

[0101] Optionally, Figure 7 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 7 As shown, the computer terminal A may include one or more (only one is shown in the figure) processors, memory, and transmission devices.

[0102] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and apparatus in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned image processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0103] The processor can access information and application programs stored in memory via a transmission device to perform the following steps: acquiring an image to be processed containing head information of the target object; encoding the image to be processed through the encoding network of a segmentation model to obtain encoded features; decoding the encoded features through the first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features; and decoding the encoded features and the head region image through the second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features.

[0104] According to the embodiments of the present invention, an image to be processed containing head information of a target object is obtained; the image to be processed is encoded by the encoding network of a segmentation model to obtain encoded features; the encoded features are decoded by the first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features; the encoded features and the head region image are decoded by the second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features. By using a segmentation model containing one encoding network and two decoding networks to process the image to be processed, the purpose of further processing the obtained head region image to obtain a facial organ image is achieved, thereby reducing the resource consumption and time consumption when predicting the head region and facial features, and thus solving the technical problem of high image processing resource consumption and long time consumption when predicting the head region and facial features in related technologies.

[0105] Those skilled in the art will understand that Figure 7 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 7 This does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include components that are more advanced than those described above. Figure 7 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 7 The different configurations shown.

[0106] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0107] Example 6

[0108] Embodiments of the present invention also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the image processing method provided in Embodiment 1.

[0109] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0110] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring an image to be processed containing head information of a target object; encoding the image to be processed through the encoding network of a segmentation model to obtain encoded features; decoding the encoded features through the first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features; and decoding the encoded features and the head region image through the second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features.

[0111] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0112] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0114] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0116] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0117] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, include: Obtain the image to be processed, which contains the header information of the target object; The image to be processed is encoded using the encoding network of the segmentation model to obtain encoded features; The encoded features are decoded by the first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features; The encoded features and the head region image are decoded by the second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features.

2. The image processing method according to claim 1, characterized in that, The encoded features include multiple encoded features of different resolutions, and the head region image includes multiple head region images of different resolutions. The process of decoding the encoded features and the head region images using the second decoding network of the segmentation model to obtain the facial organ image includes: The second decoding network combines the encoded features of multiple different resolutions and the head region images of multiple different resolutions, and decodes the combined features to obtain the facial organ image.

3. The image processing method according to claim 1, characterized in that, Before encoding the image to be processed using the encoding network of the segmentation model to obtain encoded features, the method further includes: Multiple sets of first sample data are acquired, and a preset encoding network and a preset first decoding network are trained based on the multiple sets of first sample data to obtain a pre-trained encoding network and a pre-trained first decoding network. The first sample data consists of a first sample image containing the head information of the target object and a first labeled image. The first labeled image is an image in which the facial region features are labeled. Multiple sets of second sample data are acquired, and a preset second decoding network is trained based on the multiple sets of second sample data to obtain a pre-trained second decoding network. The second sample data consists of a second sample image containing the head information of the target object and a second labeled image. The second labeled image is an image in which the facial organ features are labeled. The segmentation model is obtained from the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network.

4. The image processing method according to claim 3, characterized in that, The segmentation model obtained from the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network includes: Multiple sets of third sample data are acquired, and the pre-trained encoding network, the pre-trained first decoding network, and the pre-trained second decoding network are trained based on the multiple sets of third sample data to obtain the encoding network, the first decoding network, and the second decoding network. The third sample data consists of sample images containing head information of the target object and third labeled images. The third labeled images are images in which facial region features and facial organ features are labeled. The segmentation model is composed of the encoding network, the first decoding network, and the second decoding network.

5. The image processing method according to claim 3, characterized in that, The first labeled image is an image in which head region features, face region features, and hair region features are labeled in the first sample image. The step of decoding the encoded features through the first decoding network of the segmentation model to obtain the head region image includes: The encoded features are input into the first decoding network for processing to obtain a first head region image, a second head region image, and a third head region image. The first head region image includes the head region features, the second head region image includes the face region features, and the third head region image includes the hair region features.

6. The image processing method according to claim 1, characterized in that, Before acquiring the image to be processed containing the head information of the target object, the method further includes: Acquire an image containing the target object, and locate the head position from the image containing the target object, wherein the target object includes the head information and torso information; The image to be processed is obtained from the image containing the target object, based on the head position, containing the head information of the target object.

7. An image processing method, characterized in that, include: The cloud server receives the image to be processed, which contains the header information of the target object; The cloud server uses an encoding network of a segmentation model to encode the image to be processed, obtaining encoded features; it uses a first decoding network of the segmentation model to decode the encoded features, obtaining a head region image, wherein the head region image includes facial region features; and it uses a second decoding network of the segmentation model to decode the encoded features and the head region image, obtaining a facial organ image, wherein the facial organ image includes facial organ features. The cloud server returns the facial organ images to the client.

8. An image processing method, characterized in that, include: Acquire an image containing the head information of the target object and display the image in the user interface; The user interface displays a head region image obtained after the image to be processed is processed by a segmentation model. The head region image contains facial region features. The head region image is obtained by decoding the encoded features through the first decoding network of the segmentation model. The encoded features are obtained by encoding the image to be processed through the encoding network of the segmentation model. The user interface displays a facial organ image obtained by processing the head region image through the segmentation model. The facial organ image contains facial organ features, and the facial organ image is obtained by decoding the encoded features and the head region image through the second decoding network of the segmentation model.

9. An image processing apparatus, characterized in that, include: The acquisition unit is used to acquire the image to be processed, which contains the header information of the target object. The encoding unit is used to encode the image to be processed through the encoding network of the segmentation model to obtain encoded features; The first decoding unit is used to decode the encoded features through the first decoding network of the segmentation model to obtain a head region image, wherein the head region image contains facial region features. The second decoding unit is used to decode the encoded features and the head region image through the second decoding network of the segmentation model to obtain a facial organ image, wherein the facial organ image contains facial organ features.

10. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to perform the image processing method according to any one of claims 1 to 6.

11. A processor, characterized in that, The processor is used to run a program, wherein the program executes the image processing method according to any one of claims 1 to 6 when it runs.

Images