Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the application discloses an image processing method and device, electronic equipment and storage medium, the method comprises: extracting the common feature, the first segmentation feature and the first depth feature of the to-be-processed image through the pre-trained image processing model respectively; the common feature is the image feature shared by portrait segmentation and depth estimation, the first segmentation feature is the image feature used for portrait segmentation, and the first depth feature is the image feature used for depth estimation; the common feature, the first depth feature and the first segmentation feature are fused through the image processing model, and portrait segmentation and depth estimation are performed on the to-be-processed image respectively, and a portrait segmentation image and a depth estimation image corresponding to the to-be-processed image are obtained. By implementing the embodiment of the application, portrait segmentation and depth estimation can be performed by using the same image processing model, and the amount of calculation can be reduced.

Description

Technical Field

[0001] This application relates to the field of imaging technology, specifically to image processing methods, apparatus, electronic devices, and storage media. Background Technology

[0002] Currently, some smartphones and other electronic devices offer image blurring capabilities, resulting in images where the subject stands out clearly while the background is blurred. The key to achieving this image blurring function lies in the electronic device's ability to segment the subject and background regions from the captured image and to accurately estimate the depth of the background region.

[0003] However, in practice, it has been found that existing portrait segmentation and depth estimation algorithms are quite complex, resulting in a large amount of computation required to achieve image blurring. Summary of the Invention

[0004] The image processing method, apparatus, electronic device, and storage medium disclosed in the embodiments of this application can perform portrait segmentation and depth estimation using the same image processing model, thereby reducing the amount of computation.

[0005] This application discloses an image processing method, the method comprising: extracting common features, a first segmentation feature, and a first depth feature from an image to be processed using an image processing model; wherein the common features are image features shared by portrait segmentation and depth estimation, the first segmentation feature is an image feature used for portrait segmentation, and the first depth feature is an image feature used for depth estimation; fusing the common features, the first segmentation feature, and the first depth feature using the image processing model, and performing portrait segmentation and depth estimation on the image to be processed respectively, to obtain a portrait segmentation map and a depth estimation map corresponding to the image to be processed.

[0006] This application discloses an image processing apparatus, comprising: an extraction module, configured to extract common features, a first segmentation feature, and a first depth feature of an image to be processed using an image processing model; wherein the common features are image features shared by portrait segmentation and depth estimation, the first segmentation feature is an image feature used for portrait segmentation, and the first depth feature is an image feature used for depth estimation; and a processing module, configured to fuse the common features, the first segmentation feature, and the first depth feature using the image processing model, and perform portrait segmentation and depth estimation on the image to be processed, respectively, to obtain a portrait segmentation map and a depth estimation map corresponding to the image to be processed.

[0007] This application discloses an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor implements any of the image processing methods disclosed in this application.

[0008] This application discloses a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements any of the image processing methods disclosed in this application.

[0009] Compared with related technologies, the embodiments of this application have the following beneficial effects:

[0010] The trained image processing model can extract common features, first segmentation features, and first depth features from the image to be processed. It can further fuse the common features, first depth features, and first segmentation features to perform portrait segmentation and depth estimation on the image to be processed, obtaining a portrait segmentation map and a depth estimation map corresponding to the image to be processed. Thus, the same image processing model can be used to complete the portrait segmentation task and the depth estimation task, without having to use two different models separately, which helps to reduce the amount of computation.

[0011] Furthermore, the image processing model integrates common features, first depth features, and first segmentation features for portrait segmentation and depth estimation. It can supervise the portrait segmentation task by using the common features shared by portrait segmentation and depth estimation, as well as the first depth features unique to depth estimation, and supervise the depth estimation task by using the common features shared by portrait segmentation and the first segmentation features unique to portrait segmentation. This can achieve high accuracy in both portrait segmentation and depth estimation tasks. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic flowchart of an image processing method disclosed in one embodiment;

[0014] Figure 2 This is an example diagram of an image processing result disclosed in one embodiment;

[0015] Figure 3 This is a schematic diagram of the structure of an image processing model disclosed in one embodiment;

[0016] Figure 4 This is a schematic flowchart of an image processing method disclosed in one embodiment;

[0017] Figure 5 This is a schematic diagram of the structure of another image processing model disclosed in one embodiment;

[0018] Figure 6 This is a schematic flowchart of another image processing method disclosed in one embodiment;

[0019] Figure 7 This is a schematic diagram of the training method for an image processing model disclosed in one embodiment;

[0020] Figure 8 This is a schematic diagram of the structure of an image processing apparatus disclosed in one embodiment;

[0021] Figure 9 This is a schematic diagram of the structure of an electronic device disclosed in one embodiment. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0024] In related technologies, two independent portrait segmentation models and depth estimation models are generally used to perform portrait segmentation and depth estimation on the image to be processed. The portrait segmentation model and the depth estimation model are completely independent in structure and need to be trained separately. For example, any network from the DeepLab series can be used as the portrait segmentation model for portrait segmentation, and a multi-scale local plane guided model (From Big to Small, BTS) or a MiDaS (MixingDatasets for Zero-shot Cross-dataset Transfer) model can be used as the depth estimation network for depth estimation.

[0025] However, the computational resources required to initialize, run, or deinitialize two different network models on a computing platform are enormous, and such excessive computational resource consumption is unbearable for small electronic devices such as smartphones.

[0026] The image processing method, apparatus, electronic device, and storage medium disclosed in this application can perform portrait segmentation and depth estimation using the same image processing model, thereby reducing the computational load. These will be described in detail below.

[0027] Please see Figure 1 , Figure 1 This is a schematic flowchart of an image processing method disclosed in one embodiment. This method can be applied to one or more electronic devices such as smartphones and tablets, and is not specifically limited thereto. Figure 1 As shown, the method may include the following steps:

[0028] 110. Extract the common features, first segmentation features, and first depth features of the image to be processed using an image processing model.

[0029] The image to be processed may include human figures. It can be an image captured by a camera device of an electronic device, or an image transmitted to the electronic device from other terminal devices or service devices; there are no specific limitations. The image to be processed can be in any format, such as a YUV image or an RGB image; there are no specific limitations. The image to be processed can be any still image, or any sequence of images from a moving video; there are no specific limitations.

[0030] In some embodiments, after acquiring the image to be processed and before inputting it into the image processing model, the electronic device may perform one or more preprocessing operations on the image to be processed as needed. These preprocessing operations may include rotation, scaling, and normalization.

[0031] Rotation operations can refer to rotating a horizontally shot image into a horizontally arranged image; or rotating a horizontally arranged image into a vertically arranged image.

[0032] The scaling operation refers to adjusting the size of the image to be processed so that the size of the image to be processed input to the image processing model is consistent with the size of the input image designed by the image processing model.

[0033] Normalization operation refers to normalizing the pixel values ​​of each pixel in the image to be processed to a preset numerical range, such as normalizing to the range of 0-255.

[0034] In the embodiments of this application, the image processing model can be any neural network model that can be used for machine learning or deep learning algorithms, and there is no specific limitation. For example, the image processing model can be a convolutional neural network (CNN), a U-Net (including encoder and decoder), a fully convolutional network (FCN), etc., and there is no specific limitation.

[0035] Image processing models can be pre-trained using sample data, enabling them to perform image feature extraction, portrait segmentation, and depth estimation.

[0036] The sample data may include sample images, as well as corresponding reference segmentation maps and reference depth maps. The reference segmentation map can be used to indicate human figures in the sample images, and the reference depth map may include manually annotated reference depth values ​​for each pixel in the sample images.

[0037] For example, the sample image can be an RGB image. The reference segmentation map can be a binary mask used to indicate the human and background regions in the sample image. It can be generated by first converting the sample image to a single-channel grayscale image, then manually annotating the human regions on the single-channel grayscale image, and finally generating the reference segmentation map using a grayscale threshold of 127.5 and the manually annotated single-channel grayscale image. The reference depth map can be a single-channel grayscale image with depth values ​​ranging from 0 to 255. The depth values ​​in the reference depth map can be measured by a ranging device, and there is no specific limitation.

[0038] Image processing models can be trained on electronic devices with stronger computing power, such as personal computers and cloud servers, and stored on electronic devices with limited computing power, such as smartphones and tablets; or, the same electronic device can be used to train and apply the image processing model, without any specific limitations.

[0039] An image processing model may include multiple processing modules and / or network layers, some of which may be used to extract one or more image features from the image to be processed. In this embodiment, the image processing model may extract common features, a first segmentation feature, and a first depth feature from the image to be processed. Wherein:

[0040] Common features are image features that can be shared by both portrait segmentation and depth estimation. Portrait segmentation refers to the image processing task of accurately distinguishing the portrait region and the background region from the image to be processed. Depth estimation refers to the image processing task of predicting the depth value corresponding to each pixel in the image to be processed. Common features can be used for both portrait segmentation and depth estimation.

[0041] The first segmentation feature can be an image feature used for portrait segmentation.

[0042] The first depth feature can be an image feature used for depth estimation.

[0043] 120. By fusing common features, first depth features, and first segmentation features through an image processing model, portrait segmentation and depth estimation are performed on the image to be processed, respectively, to obtain a portrait segmentation map and a depth estimation map corresponding to the image to be processed.

[0044] The image processing model fuses common features, first depth features, and first segmentation features. This can be achieved by merging these features through one or more operations such as addition or concatenation, and then using the fused image to perform portrait segmentation and depth estimation tasks, respectively, to obtain a portrait segmentation map and a depth estimation map. Specifically:

[0045] A human segmentation map can be used to indicate the human region in the image to be processed, as predicted by the image processing model, while a depth estimation map can include the predicted depth values ​​corresponding to each pixel in the image to be processed, as predicted by the image processing model.

[0046] In one embodiment, an electronic device can fuse a first depth feature and a common feature into a first segmentation feature using an image processing model, and then use the fused second segmentation feature to perform portrait segmentation on the image to be processed, obtaining a portrait segmentation image corresponding to the image to be processed. For example, the image processing model can perform convolution processing on the first segmentation feature, and then merge the convolutionally processed first segmentation feature, common feature, and first depth feature to obtain the second segmentation feature.

[0047] Furthermore, the electronic device can fuse the first segmentation feature and common feature into a first depth feature using an image processing model, and then use the fused second depth feature to perform portrait segmentation on the image to be processed, obtaining a portrait segmentation image corresponding to the image to be processed. For example, the image processing model can perform convolution processing on the first depth feature, and then merge the convolutionally processed first depth feature, common feature, and first segmentation feature to obtain the second depth feature.

[0048] In other words, the image processing model can integrate common features, first depth features, and first segmentation features to perform portrait segmentation on the image to be processed. It can use the depth features unique to depth estimation, as well as the common features shared by depth estimation and portrait segmentation, to supervise the portrait segmentation task and obtain a portrait segmentation map corresponding to the image to be processed.

[0049] Furthermore, the image processing model can fuse common features, first depth features, and first segmentation features to perform depth estimation on the image to be processed. It can also use segmentation features unique to human portrait segmentation, as well as common features shared by depth estimation and human portrait segmentation, to supervise the depth estimation task and obtain a depth estimation map corresponding to the image to be processed.

[0050] For example, please refer to Figure 2 , Figure 2 This is an example diagram of an image processing result disclosed in one embodiment. For example... Figure 2 As shown, after the image to be processed 210 is input into the image processing model 320, the image processing model 320 can perform portrait segmentation and depth estimation on the image to be processed 210, and obtain the portrait segmentation map 330 and the depth estimation map 340 output by the image processing model 320.

[0051] like Figure 2 As shown, the portrait segmentation map 330 can be represented by a binarized mask. In the portrait segmentation map 330, pixels with a value of 1 are displayed as white pixels, used to indicate the portrait area in the image to be processed 210; pixels with a value of 0 are displayed as black pixels, used to indicate the background area in the image to be processed 210.

[0052] The depth estimation map 340 can be represented by a grayscale image. In the depth estimation map 340, the smaller the predicted depth value corresponding to a pixel, the smaller the grayscale value corresponding to the pixel, and the closer the color of the pixel in the depth estimation map 340 is to black.

[0053] Therefore, in the foregoing embodiments, the electronic device can extract the common features, first segmentation features, and first depth features of the image to be processed using a trained image processing model, and further use the aforementioned three image features to perform portrait segmentation and depth estimation. That is, the same image processing model can be used to perform portrait segmentation and depth estimation tasks on the image to be processed, eliminating the need for two different models. Compared to the dual-model method that uses two different models to perform portrait segmentation and depth estimation tasks separately, the image processing method disclosed in this application can reduce the computational resources required for the processing model to initialize, run, or deinitialize on computing platforms such as electronic devices, thereby reducing the computational load.

[0054] Meanwhile, the image processing method disclosed in this application can utilize depth features unique to depth estimation, as well as common features shared by depth estimation and portrait segmentation, to supervise portrait segmentation tasks; and by utilizing segmentation features unique to portrait segmentation, as well as common features shared by depth estimation and portrait segmentation, to supervise depth estimation tasks, the generalization ability of the image processing model on portrait segmentation and depth estimation tasks can be improved, and the accuracy of both portrait segmentation and depth estimation can be effectively improved.

[0055] In one embodiment, after obtaining the depth estimation map and portrait segmentation map output by the image processing model, the electronic device can further calculate the area of ​​the connected components included in the portrait segmentation map. The portrait segmentation map may include one or more connected components. The electronic device can calculate the connected components in the portrait segmentation map using algorithms such as Two-Pass or Seed-Filling, without specific limitations. The area of ​​a connected component can refer to the number of pixels included in the connected component; the more pixels, the larger the area of ​​the connected component.

[0056] After calculating the area of ​​each connected component in the portrait segmentation map, the electronic device can mark connected components with areas smaller than a threshold as background regions, thus removing them from the portrait region. In other words, connected components with areas smaller than the threshold can be considered as misclassifications by the image processing model, thereby improving the accuracy of portrait segmentation.

[0057] For example, the portrait segmentation map output by the image processing model can be a binary mask, where pixels with a value of 1 can be used to indicate pixels included in the portrait region, and pixels with a value of 0 can be used to indicate pixels in the background region. The electronic device can change the pixel value of pixels in connected regions with an area smaller than a threshold from 1 to 0, thereby modifying connected regions with an area smaller than the threshold from portrait regions to background regions.

[0058] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an image processing model disclosed in one embodiment. For example... Figure 3 As shown, the image processing model 300 may include: a human face segmentation network and a depth estimation network.

[0059] The human image segmentation network and the depth estimation network can share the encoding module 310 and the first filtering module 320.

[0060] The encoding module 310 may include one or more network layers, such as downsampling layers, convolutional layers, pooling layers, etc. The encoding module 310 may include any backbone network with encoding capabilities, such as MobileNet, Residual Network (RestNet), or any network from the Xception series. The network type used by the encoding module 310 can be selected according to the actual service requirements, such as accuracy requirements and computational requirements.

[0061] The first filtering module 320 may include one or more convolutional layers capable of performing filtering processing, without being specifically limited.

[0062] like Figure 3 As shown, the human image segmentation network may further include a second filtering module 330 and a first decoding module 340. The depth estimation network may further include a third filtering module 350 and a second decoding module 360.

[0063] The second filtering module 330 and the third filtering module 350 may each include one or more convolutional layers capable of filtering. The structures of the first filtering module 320, the second filtering module 330 and the third filtering module 350 may be the same or different, and no specific limitation is made.

[0064] Both the first decoding module 340 and the second decoding module 360 ​​can be composed of one or more network layers, such as convolutional layers, deconvolutional layers, upsampling layers, batch normalization (BN) layers, and rectified linear unit (ReLU) layers. The structures of the first decoding module 340 and the second decoding module 360 ​​can be the same or different, and there is no specific limitation.

[0065] Please see Figure 4 , Figure 4 This is a schematic flowchart of an image processing method disclosed in one embodiment. Figure 4 The image processing method shown can be used by electronic devices based on, for example... Figure 3 The image processing model shown is executed. Figure 4 As shown, the method may include the following steps:

[0066] S1. Extract the basic image features of the image to be processed through the encoding module 310.

[0067] The encoding module 310 may include multiple network layers, such as downsampling layers and convolutional layers. The first network layer of the encoding module 310 can receive the input image to be processed, and through one or more network layers arranged after the first network layer, the image to be processed is processed sequentially through downsampling, convolution, etc., to extract the basic image features of the image to be processed.

[0068] S2. The basic features of the image are filtered by the first filtering module 320 to obtain the common features output by the first filtering module.

[0069] The first filtering module 320 can filter the basic features of the image through convolution. After training with the aforementioned sample data, the first filtering module 320 can extract common features from the basic features of the image that can be used for both portrait segmentation and depth estimation tasks.

[0070] S3. The image basic features are filtered by the second filtering module 330 to obtain the first segmentation features output by the second filtering module 330.

[0071] The second filtering module 330 can filter the basic image features through convolution. After training with the aforementioned sample data, the second filtering module 330 can extract the first segmentation features that can be used for portrait segmentation from the basic image features.

[0072] S4. The image basic features are filtered by the third filtering module 350 to obtain the first depth feature output by the third filtering module 350.

[0073] The third filtering module 350 can filter the basic image features through convolution. After training with the aforementioned sample data, the third filtering module 350 can extract the first depth feature that can be used for depth estimation from the basic image features.

[0074] S5. The first depth feature and common feature are fused into the first segmentation feature by the first decoding module 340, and the second segmentation feature obtained after fusion is used to perform portrait segmentation on the image to be processed, so as to obtain the portrait segmentation map corresponding to the image to be processed.

[0075] The first depth feature output by the third filtering module 350, the common feature output by the first filtering module 310, and the first segmentation feature output by the second filtering module 330 can be input to the first decoding module 340.

[0076] The first decoding module 340 uses sequentially arranged network layers to first convolve the first segmentation feature, and then merges the convolved first segmentation feature with the first depth feature and common features to obtain the second segmentation feature. Furthermore, the second segmentation feature is processed through upsampling, convolution, and other methods to obtain a portrait segmentation map corresponding to the image to be processed.

[0077] S6. The first segmentation feature and common feature are fused into the first depth feature by the second decoding module 360, and the depth of the image to be processed is estimated by using the second depth feature obtained after fusion, so as to obtain the depth estimation map corresponding to the image to be processed.

[0078] The second decoding module 360 ​​can use sequentially arranged network layers to first convolve the first depth feature, and then merge the convolved first depth feature with the first segmentation feature and common features to obtain the second depth feature. Furthermore, the second depth feature is processed through upsampling, convolution, and other methods to obtain a depth estimation map corresponding to the image to be processed.

[0079] In the foregoing embodiments, after the image processing model 200 extracts the basic image features of the image to be processed using the encoding module 310, it can first extract the common features, the first segmentation features, and the first depth features through the first filtering module 320, the second filtering module 330, and the third filtering module 350. Then, it can use the corresponding decoding modules to fuse the aforementioned three features to perform portrait segmentation and depth estimation on the image to be processed, instead of directly decoding the basic image features output by the encoding module through different decoding modules. This can further improve the accuracy of portrait segmentation and depth estimation.

[0080] In one embodiment, the aforementioned encoding module 310 may include an encoder and a pyramid pooling layer.

[0081] An encoder can be any lightweight backbone network with encoding capabilities, such as MobileNet. The encoder acquires the input image to be processed and encodes it to extract the first basic features from the image.

[0082] In one embodiment, if the image to be processed is a single still image, such as a photograph, the encoder can encode all image information included in the image to be processed. If the image to be processed is any one frame from a series of consecutive frames in a dynamic video, the encoder can encode the image information corresponding to a moving target in the image to be processed, and the encoder can determine the moving target in the image based on the input series of consecutive frames.

[0083] Pyramid pooling layers can include spatial pyramid pooling (SPP) or atrous spatial pyramid pooling (ASPP) and other pooling layers with a pyramid structure. Pyramid pooling layers resample the first basic features extracted by the encoder at a single scale at different scales to obtain multiple second basic features at different scales. By fusing these multiple second basic features, the fundamental image features of the image to be processed can be obtained.

[0084] For example, taking ASPP as an example, ASPP may include multiple parallel atrous covolutional layers, with different atrous convolutional layers corresponding to different sampling rates. Each atrous convolutional layer can resample the first basic features output by the encoder according to its corresponding sampling rate, and output second basic features of different scales after resampling. ASPP then fuses the second basic features of different scales and outputs the fused image basic features.

[0085] The addition of pyramid pooling layers enables the image processing model to adapt to images of different sizes, and achieves good accuracy in portrait segmentation and depth estimation in different regions of the image being processed.

[0086] In one embodiment, the first decoding module 340 can also be used to perform portrait segmentation on the image to be processed by fusing image information, common features, first depth features and first segmentation features from the basic image features output by the encoder 310.

[0087] Furthermore, the first decoding module 350 can also be used to perform depth estimation on the image to be processed by fusing image information, common features, first depth features and first segmentation features from the base image features output by the encoder 310.

[0088] The aforementioned first decoding module 340 may include a first decoder and a first feature pyramid module; the aforementioned second decoding module 360 ​​may include a second decoder and a second feature pyramid module.

[0089] The first feature pyramid module and the second feature pyramid module may include a Feature Pyramid Network (FPN). The first feature pyramid module may be connected to the decoder and the first decoder, respectively; the second feature pyramid module may be connected to the decoder and the second decoder, respectively.

[0090] Taking the first feature pyramid module as an example, it can further extract features from the first basic feature at different scales to obtain multiple third basic features at different scales. Similarly, the second feature pyramid module can further extract features from the first basic feature at different scales to obtain multiple fourth basic features at different scales.

[0091] It should be noted that the network structures of the first feature pyramid module and the second pyramid module can be the same or different, without any specific limitation. Furthermore, the multiple third basic features of different scales output by the first feature pyramid module after pre-training can be different from the multiple fourth basic features of different scales output by the second feature pyramid module after pre-training.

[0092] The first decoder can be used to fuse the first depth feature, common feature and multiple third basic features of different scales into the first segmentation feature, and use the second segmentation feature obtained after fusion to perform portrait segmentation on the image to be processed, so as to obtain a portrait segmentation map corresponding to the image to be processed.

[0093] The second decoder can be used to fuse the first segmentation feature, common feature and multiple fourth basic features of different scales into the first depth feature, and use the second depth feature obtained after fusion to estimate the depth of the image to be processed, so as to obtain the depth estimation map corresponding to the image to be processed.

[0094] To more clearly describe the image processing model disclosed in the embodiments of this application, please refer to... Figure 5 , Figure 5 This is a schematic diagram of the structure of another image processing model disclosed in one embodiment. For example... Figure 5 As shown, the image processing model 500 may include: a human face segmentation network and a depth estimation network.

[0095] The human image segmentation network and the depth estimation network can share the encoder 510, the dilated convolutional pyramid pooling layer (ASPP) 520 connected to the encoder 510, and the first filtering module 530.

[0096] The human image segmentation network may also include: a second filtering module 540, a first feature pyramid module 550, and a first decoder 560.

[0097] The depth estimation network may also include: a third filtering module 570, a second feature pyramid module 580, and a second decoder 590.

[0098] The first filtering module 530, the second filtering module 540 and the third filtering module 570 can be connected to the dilated convolutional pyramid pooling layer 520, respectively.

[0099] The first decoder 560 can be connected to the first filter module 530, the second filter module 540 and the third filter module 570 respectively; the second decoder 590 can also be connected to the first filter module 530, the second filter module 540 and the third filter module 570 respectively.

[0100] The first feature pyramid module 550 can be connected to the encoder 510 and the first decoder 560 respectively; the second feature pyramid module 580 can be connected to the encoder 510 and the second decoder 590 respectively.

[0101] Please see Figure 6 , Figure 6 This is a schematic diagram of the method flow of another image processing method disclosed in one embodiment. Figure 6 The image processing method shown can be used by electronic devices based on, for example... Figure 5 The image processing model shown is executed. Figure 6 As shown, the method may include the following steps:

[0102] S1. Extract the first basic features of the image to be processed using encoder 510.

[0103] S2. The first basic feature is processed by the pyramid pooling layer 520 to obtain multiple second basic features at different scales, and the multiple second basic features at different scales are fused to obtain the basic features of the image.

[0104] S3. The basic features of the image are processed by the first filtering module 530 to obtain common features.

[0105] S5. The image basic features are processed by the second filtering module 540 to obtain the first segmentation features.

[0106] S5. The basic features of the image are processed by the third filtering module 570 to obtain the first depth feature.

[0107] S6. The first basic feature is processed by the first pyramid module 550 to obtain multiple third basic features at different scales.

[0108] S7. The first basic feature is processed by the second feature pyramid module 580 to obtain multiple fourth basic features at different scales.

[0109] S8. The first depth feature, common feature and multiple third basic features of different scales are fused into the first segmentation feature by the first decoder 560, and the second segmentation feature obtained after fusion is used by the first decoder to perform portrait segmentation on the image to be processed, so as to obtain the portrait segmentation map corresponding to the image to be processed.

[0110] For example, the first decoder 560 can first convolve the first segmentation feature, and then merge the convolved first segmentation feature with the first depth feature, common features, and multiple third basic features of different scales to obtain the second segmentation feature. The second decoder 560 can further perform feature restoration on the second segmentation feature through one or more methods such as upsampling or convolution to obtain the portrait segmentation map corresponding to the image to be processed.

[0111] S9. The first segmentation feature, common feature and multiple fourth basic features of different scales are fused into the first depth feature by the second decoder 590, and the depth of the image to be processed is estimated by the second decoder using the fused second depth feature to obtain the depth estimation map corresponding to the image to be processed.

[0112] For example, the second decoder 590 can first convolve the first depth feature, and then merge the convolved first depth feature with the first segmentation feature, common features, and multiple fourth basic features of different scales to obtain the second depth feature. The second decoder 590 can further restore the second depth feature through one or more methods such as upsampling or convolution to obtain a depth estimation map corresponding to the image to be processed.

[0113] The above content introduced an image processing model that integrates a human image segmentation network and a depth estimation network, as well as image processing methods based on this model. The following content introduces the training methods for this image processing model.

[0114] Please see Figure 7 , Figure 7 This is a schematic diagram of the training method for an image processing model disclosed in one embodiment. Figure 7 The trained image processing model can be any of the image processing models disclosed in the foregoing embodiments, for example, it can be... Figure 3 or Figure 5 The image processing model shown is as follows. Figure 7 As shown, the training method may include the following steps:

[0115] 710. Use the first sample data to train the human image segmentation network in the image processing model to obtain the pre-trained human image segmentation network.

[0116] The first sample data may include: a first sample image, and a first reference segmentation map corresponding to the first sample image, the first reference segmentation map being used to indicate the human figure region in the first sample image.

[0117] The first sample data may include multiple first sample images. For each first sample image, the electronic device may input the first sample image into the image processing model to be trained, and process the first sample image through the steps and operations performed by the image processing model disclosed in the foregoing embodiments to obtain the first training segmentation map output by the image processing model's portrait segmentation network, which corresponds to the first sample image.

[0118] The electronic device can calculate the first segmentation loss corresponding to the human image segmentation network based on the segmentation loss function corresponding to the human image segmentation network, the first training segmentation map, and the first reference segmentation map, and use the calculated first segmentation loss to adjust the parameters in the human image segmentation network.

[0119] For example, the segmentation loss function can be expressed by the following formula:

[0120] L seg =CE+CE edge +IOU; Formula (1);

[0121] Among them, L seg CE can represent the loss for human image segmentation, and CE can represent the cross-entropy loss for the entire image. edge The cross-entropy loss can be represented by the distance to the portrait in the portrait region, and IOU can be represented by the Lovasz loss of the whole image; the whole image mentioned above can refer to the entire image including the first training segmentation map and the first reference segmentation map.

[0122] Electronic devices can update the parameters in the human image segmentation network by combining methods such as stochastic gradient descent (SGD) and momentum gradient descent with human image segmentation loss, without any specific limitations.

[0123] In other words, when the electronic device performs step 610 to train the human image segmentation network, it can first disable the modules or network layers specific to the depth estimation network and train the modules and network layers included in the human image segmentation network.

[0124] For example, with Figure 5 Taking the image processing model shown as an example, when the electronic device performs step 710, it can use the portrait segmentation loss to adjust the parameters in the encoder 510, the pyramid pooling layer (ASPP) 520, the first filtering module 530, the second filtering module 540, the first feature pyramid module 550 and the first decoder 560.

[0125] 720. Use the second sample data to train the depth estimation network in the image processing model to obtain the pre-trained depth estimation network.

[0126] The second sample data may include: a second sample image, and a second reference depth map corresponding to the second sample image, wherein the second reference depth map includes the reference depth value of each pixel in the second sample image.

[0127] The electronic device can input the second sample image into the image processing model to be trained, and process the second sample image through the steps and operations performed by the image processing model disclosed in the foregoing embodiments, to obtain the second training depth map output by the depth estimation network of the image processing model, which corresponds to the second sample image.

[0128] The electronic device can calculate the second depth loss corresponding to the depth estimation network based on the depth loss function corresponding to the deep network, the second training depth map, and the second reference depth map, and use the calculated second depth loss to adjust the parameters in the depth estimation network.

[0129] For example, the depth loss function can be expressed by the following formula:

[0130]

[0131]

[0132]

[0133] Among them, L depth λ can represent the depth estimation loss; 'a' can be a user-defined parameter, for example, 10; 'T' can represent the number of pixels in the second sample image, and 'i' can represent the i-th pixel; 'λ' can be a user-defined parameter, for example, 0.5; d i It can represent the predicted depth value corresponding to the i-th pixel in the training depth map; It can represent the reference depth value corresponding to the i-th pixel in the reference depth map.

[0134] Electronic devices can update the parameters in the depth estimation network by using methods such as stochastic gradient descent and momentum gradient descent, combined with depth estimation loss, without any specific limitations.

[0135] In other words, when the electronic device performs step 720 to train the depth estimation network, it can first block the modules or network layers specific to the human image segmentation network and train the modules and network layers included in the depth estimation network.

[0136] For example, with Figure 5 Taking the image processing model shown as an example, when the electronic device performs step 720, it can use the depth estimation loss to adjust the parameters in the encoder 510, the pyramid pooling layer (ASPP) 520, the first filtering module 530, the third filtering module 570, the second feature pyramid module 580, and the second decoder 590.

[0137] It should be noted that steps 710 and 720 mentioned above do not necessarily have a logical order. The electronic device may execute step 710 first and then step 720; or it may execute step 720 first and then step 710.

[0138] In the foregoing embodiments, the pre-trained image processing model may include the aforementioned pre-trained portrait segmentation network and pre-trained depth estimation network. The image processing model obtained after training in steps 710 and 720 can be used to perform any of the aforementioned image processing methods.

[0139] In some embodiments, the deep neural network included in the image processing module of the image processing method may be obtained after pre-training with the first sample data in step 710; the human image segmentation network included in the image processing module may be obtained after pre-training with the second sample data in step 720.

[0140] In other embodiments, in order to improve the accuracy of the image processing model in portrait segmentation and depth estimation tasks, the electronic device may further perform step 730 to fine-tune the parameters in the image processing model.

[0141] 730. Fine-tune the parameters in the pre-trained depth estimation network and the pre-trained portrait segmentation network using third-sample data.

[0142] The third sample data includes: a third sample image, and a third reference depth map and a third reference segmentation map corresponding to the third sample image; the third reference depth map includes the reference depth value of each pixel in the third sample image, and the third reference segmentation map is used to indicate the human image region in the third sample image.

[0143] In other words, the image processing model used in the image processing method, including the deep neural network, can be obtained by pre-training with the first sample data in step 710 and fine-tuning with the third sample data in step 730; the image processing model including the human image segmentation network can be obtained by pre-training with the second sample data in step 720 and fine-tuning with the third sample data in step 730.

[0144] When fine-tuning the depth estimation network and the portrait segmentation network included in the image processing model, the electronic device can input a third sample image into the image processing model obtained after training through the aforementioned steps 710 and 720, and process the third sample image through the steps and operations performed by the image processing model disclosed in the aforementioned embodiments, so as to obtain the third training segmentation image output by the portrait segmentation network of the image processing model and the third depth estimation image output by the depth estimation network of the image processing model.

[0145] The electronic device can calculate the third segmentation loss corresponding to the human image segmentation network based on the segmentation loss function corresponding to the human image segmentation network, the third training segmentation map, and the third reference segmentation map, and calculate the third depth loss corresponding to the depth estimation network based on the depth loss function corresponding to the depth network, the third training depth map, and the third reference depth map.

[0146] The electronic device can calculate the total loss based on the third segmentation loss and the third depth loss, and adjust the parameters in the human image segmentation network and the depth estimation network based on the total loss.

[0147] For example, the total loss can be calculated using the following formula:

[0148] Loss=α*L seg +β*L depth ; Formula (5);

[0149] Here, Loss represents the total loss, and α and β represent the weights, which can be adjusted according to the actual business needs to meet the accuracy requirements of portrait segmentation and depth estimation. For example, if the image processing model needs to perform portrait segmentation more accurately, α can be adjusted to be greater than β.

[0150] In summary, the training method for the image processing model disclosed in the embodiments of this application can train an image processing model that can simultaneously perform portrait segmentation and depth estimation on the image to be processed, thereby reducing the amount of computation.

[0151] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an image processing device disclosed in one embodiment. Figure 8 The image processing apparatus shown can be applied to any of the electronic devices disclosed in the foregoing embodiments. For example... Figure 8 As shown, the image processing apparatus 800 may include an extraction module 810 and a processing module 820.

[0152] The extraction module 810 is used to extract the common features, the first segmentation features, and the first depth features of the image to be processed through a pre-trained image processing model; the common features are image features shared by portrait segmentation and depth estimation, the first segmentation features are image features used for portrait segmentation, and the first depth features are image features used for depth estimation.

[0153] The processing module 820 is used to fuse common features, first depth features and first segmentation features through an image processing model, and to perform portrait segmentation and depth estimation on the image to be processed, respectively, to obtain a portrait segmentation map and a depth estimation map corresponding to the image to be processed.

[0154] In one embodiment, the processing module 820 can also be configured to fuse the first depth features and common features into a first segmentation feature using an image processing model, and use the fused second segmentation feature to perform portrait segmentation on the image to be processed, thereby obtaining a portrait segmentation image corresponding to the image to be processed; and,

[0155] The image processing model fuses the first segmentation feature and common feature into a first depth feature, and then uses the fused second depth feature to estimate the depth of the image to be processed, thus obtaining a depth estimation map corresponding to the image to be processed.

[0156] In one embodiment, the image processing model includes: a human face segmentation network and a depth estimation network; the human face segmentation network and the depth estimation network include a shared encoding module and a first filtering module; the human face segmentation network further includes: a second filtering module and a first decoding module; the depth estimation network further includes: a third filtering module and a second decoding module.

[0157] The extraction module 810 can also be used to extract the basic image features of the image to be processed through the encoding module; and to filter the basic image features through the first filtering module, the second filtering module and the third filtering module respectively to obtain the common features output by the first filtering module, the first segmentation features output by the second filtering module and the first depth features output by the first filtering module.

[0158] The processing module 820 can also be used to fuse the first depth features and common features into the first segmentation features through the first decoding module, and use the second segmentation features obtained after fusion to perform portrait segmentation on the image to be processed;

[0159] The processing module 820 can also be used to fuse the first segmentation features and common features into the first depth features through the second decoding module, and use the fused second depth features to perform depth estimation on the image to be processed.

[0160] In one embodiment, the encoding module may include an encoder and a pyramid pooling layer.

[0161] The extraction module 810 can also be used to extract the first basic features of the image to be processed through the encoder, process the first basic features through the pyramid pooling layer to obtain multiple second basic features at different scales, and fuse the multiple second basic features at different scales to obtain the basic features of the image.

[0162] In one embodiment, the first decoding module includes a first decoder and a first feature pyramid module.

[0163] The extraction module 810 can also be used to process the first basic features output by the encoder through the first feature pyramid module to obtain multiple third basic features at different scales; and to fuse the first depth features, common features and multiple third basic features at different scales into the first segmentation features through the first decoder, and to perform portrait segmentation on the image to be processed using the second segmentation features obtained after fusion through the first decoder.

[0164] In one embodiment, the second decoding module includes: a second decoder and a second feature pyramid module.

[0165] The extraction module 810 can also be used to process the first basic features output by the encoder through the second feature pyramid module to obtain multiple fourth basic features at different scales; and to fuse the first segmentation features, common features and multiple fourth basic features at different scales into the first depth features through the second decoder, and to perform depth estimation of the image to be processed using the second depth features obtained after fusion through the second decoder.

[0166] In one embodiment, the image processing apparatus 800 may further include a post-processing module.

[0167] The post-processing module can be used to calculate the area of ​​connected components included in the human image region in the human image segmentation map after obtaining the depth estimation map and human image segmentation map output by the image processing model; and to mark connected components with an area less than a threshold as background regions so as to delete connected components with an area less than the threshold from the human image region.

[0168] In one embodiment, the portrait segmentation network disclosed in the foregoing embodiment is obtained by pre-training the portrait segmentation network to be trained using the first sample data; the depth estimation network disclosed in the foregoing embodiment is obtained by pre-training the depth estimation network to be trained using the second sample data.

[0169] The first sample data includes: a first sample image, and a first reference segmentation map corresponding to the first sample image, wherein the first reference segmentation map is used to indicate the human figure region in the first sample image;

[0170] The second sample data includes: a second sample image, and a second reference depth map corresponding to the second sample image, wherein the second reference depth map includes the reference depth value of each pixel in the second sample image.

[0171] In one embodiment, the human segmentation network disclosed in the foregoing embodiment may be obtained by pre-training the human segmentation network to be trained using the first sample data and fine-tuning the pre-trained human segmentation network using the third sample data; the depth estimation network disclosed in the foregoing embodiment is obtained by pre-training the depth estimation network to be trained using the second sample data and fine-tuning the pre-trained depth estimation network using the third sample data.

[0172] The third sample data includes: a third sample image, and a third reference depth map and a third reference segmentation map corresponding to the third sample image; the third reference depth map includes the reference depth value of each pixel in the third sample image, and the third reference segmentation map is used to indicate the human image region in the third sample image.

[0173] In one embodiment, the image processing apparatus 800 may further include a training module.

[0174] The training module can be used to train the human image segmentation network using the first sample data before the encoding module of the image processing model extracts the basic image features of the image to be processed, so as to obtain a pre-trained human image segmentation network; and to train the depth estimation network using the second sample data, so as to obtain a pre-trained depth estimation network.

[0175] The training module can also be used to fine-tune the parameters in the pre-trained depth estimation network and the pre-trained portrait segmentation network using third-party sample data.

[0176] As can be seen, in the foregoing embodiments, the image processing device can perform both portrait segmentation and depth estimation tasks on the image to be processed based on the same image processing model, eliminating the need for separate processing using two different models. This saves computational resources and reduces computational load. Furthermore, the image processing model can supervise the portrait segmentation task and the depth estimation task using common features and the first segmentation feature, which helps improve the generalization ability of the image processing model in both portrait segmentation and depth estimation tasks, effectively improving the accuracy of both portrait segmentation and depth estimation.

[0177] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an electronic device disclosed in one embodiment. For example... Figure 9 As shown, the electronic device 900 may include:

[0178] Memory 910 storing executable program code;

[0179] Processor 920 coupled to memory 910;

[0180] The processor 920 calls the executable program code stored in the memory 910 to execute any of the image processing methods disclosed in the embodiments of this application.

[0181] It should be noted that, Figure 9 The electronic device shown may also include components not shown, such as a power supply, input buttons, camera, speaker, screen, RF circuit, Wi-Fi module, Bluetooth module, and sensors, which will not be described in detail in this embodiment.

[0182] This application discloses a computer-readable storage medium storing a computer program that causes a computer to execute any of the image processing methods disclosed in this application.

[0183] This application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform any of the image processing methods disclosed in this application.

[0184] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0185] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0186] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0188] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0189] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0190] The foregoing has provided a detailed description of an image processing method, apparatus, electronic device, and storage medium disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image processing method, characterized by, The method comprises: extracting common features, first segmentation features and first depth features of the to-be-processed image through an image processing model; the common features are image features commonly used for portrait segmentation and depth estimation, the first segmentation features are image features used for portrait segmentation, and the first depth features are image features used for depth estimation; fusing the common features, the first segmentation features and the first depth features through the image processing model to perform portrait segmentation and depth estimation on the to-be-processed image respectively, and obtaining a portrait segmentation image and a depth estimation image corresponding to the to-be-processed image; the fusing the common features, the first segmentation features and the first depth features through the image processing model to perform portrait segmentation and depth estimation on the to-be-processed image respectively, and obtaining a portrait segmentation image and a depth estimation image corresponding to the to-be-processed image comprises: fusing the first depth features and the common features to the first segmentation features through the image processing model, and performing portrait segmentation on the to-be-processed image by using second segmentation features obtained after the fusing, to obtain a portrait segmentation image corresponding to the to-be-processed image; fusing the first segmentation features and the common features to the first depth features through the image processing model, and performing depth estimation on the to-be-processed image by using second depth features obtained after the fusing, to obtain a depth estimation image corresponding to the to-be-processed image.

2. The method of claim 1, wherein, The image processing model comprises a portrait segmentation network and a depth estimation network; the portrait segmentation network and the depth estimation network comprise a common encoding module and a common first filtering module; the portrait segmentation network further comprises a second filtering module and a first decoding module; the depth estimation network further comprises a third filtering module and a second decoding module; and the extracting common features, first segmentation features and first depth features of the to-be-processed image through a pre-trained image processing model comprises: extracting image basic features of the to-be-processed image through the encoding module; performing filtering processing on the image basic features through the first filtering module, the second filtering module and the third filtering module respectively, to obtain common features output by the first filtering module, first segmentation features output by the second filtering module, and first depth features output by the first filtering module; the fusing the first depth features and the common features to the first segmentation features through the image processing model and performing portrait segmentation on the to-be-processed image by using second segmentation features obtained after the fusing comprises: fusing the first depth features and the common features to the first segmentation features through the first decoding module, and performing portrait segmentation on the to-be-processed image by using second segmentation features obtained after the fusing; and the fusing the first segmentation features and the common features to the first depth features through the image processing model and performing depth estimation on the to-be-processed image by using second depth features obtained after the fusing comprises: The second decoding module fuses the first segmentation feature and the common feature into the first depth feature, and uses a second depth feature obtained after the fusion to perform depth estimation on the image to be processed.

3. The method of claim 2, wherein, The encoding module comprises an encoder and a pyramid pooling layer, and the extraction of the base image feature of the image to be processed by the encoding module comprises: The first base feature of the image to be processed is extracted by the encoder, and the first base feature is processed by the pyramid pooling layer to obtain a plurality of second base features of different scales, and the plurality of second base features of different scales are fused to obtain the image base feature.

4. The method of claim 3, wherein, The first decoding module comprises a first decoder and a first feature pyramid module, and the fusion of the first depth feature and the common feature into the first segmentation feature by the first decoding module and the use of a second segmentation feature obtained after the fusion to perform portrait segmentation on the image to be processed comprise: The first base feature output by the encoder is processed by the first feature pyramid module to obtain a plurality of third base features of different scales, and the first depth feature, the common feature and the plurality of third base features of different scales are fused into the first segmentation feature by the first decoder, and a second segmentation feature obtained after the fusion is used by the first decoder to perform portrait segmentation on the image to be processed; Alternatively, the second decoding module comprises a second decoder and a second feature pyramid module, and the fusion of the first segmentation feature and the common feature into the first depth feature by the second decoding module and the use of a second depth feature obtained after the fusion to perform depth estimation on the image to be processed comprise: The first base feature output by the encoder is processed by the second feature pyramid module to obtain a plurality of fourth base features of different scales, and the first segmentation feature, the common feature and the plurality of fourth base features of different scales are fused into the first depth feature by the second decoder, and a second depth feature obtained after the fusion is used by the second decoder to perform depth estimation on the image to be processed.

5. The method according to any one of claims 1 to 4, characterized in that, After obtaining the depth estimation map and the portrait segmentation map corresponding to the image to be processed respectively, the method further comprises: Statistically obtaining the area of a connected domain included in the portrait region in the portrait segmentation map; Marking the connected domain with an area less than a threshold value as a background region to delete the connected domain with the area less than the threshold value from the portrait region.

6. The method according to any one of claims 2-4, characterized in that, The portrait segmentation network is obtained by pre-training a portrait segmentation network to be trained using first sample data and fine-tuning the pre-trained portrait segmentation network using third sample data; The depth estimation network is obtained by pre-training a depth estimation network to be trained using second sample data and fine-tuning the pre-trained depth estimation network using the third sample data; The first sample data includes a first sample image and a first reference segmentation map corresponding to the first sample image; the second sample data includes a second sample image and a second reference depth map corresponding to the second sample image; and the third sample data includes a third sample image, a third reference depth map corresponding to the third sample image, and a third reference segmentation map corresponding to the third sample image.

7. An image processing apparatus characterized by comprising: The method comprises: The extraction module is configured to extract, by using a pre-trained image processing model, a common feature, a first segmentation feature, and a first depth feature of the to-be-processed image respectively; The common feature is an image feature commonly used for portrait segmentation and depth estimation, the first segmentation feature is an image feature used for portrait segmentation, and the first depth feature is an image feature used for depth estimation; The processing module is configured to fuse the common feature, the first segmentation feature, and the first depth feature by using the image processing model, and perform portrait segmentation and depth estimation on the to-be-processed image respectively to obtain a portrait segmentation map and a depth estimation map corresponding to the to-be-processed image; The processing module is specifically configured to fuse the first depth feature and the common feature into the first segmentation feature by using the image processing model, and perform portrait segmentation on the to-be-processed image by using a second segmentation feature obtained after the fusion to obtain a portrait segmentation image corresponding to the to-be-processed image; The processing module is specifically configured to fuse the first segmentation feature and the common feature into the first depth feature by using the image processing model, and perform depth estimation on the to-be-processed image by using a second depth feature obtained after the fusion to obtain a depth estimation map corresponding to the to-be-processed image.

8. An electronic device, comprising: The computer program is executed by the processor to implement the method according to any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method according to any one of claims 1 to 6.

Images