Image processing method and apparatus, and electronic device

Abstract

Provided in the present disclosure are an image processing method and apparatus, and an electronic device. A particular embodiment of the method comprises: acquiring a first image to be processed; determining a region segmented image corresponding to the first image; performing foreground prediction on the basis of the region segmented image and the first image, so as to obtain a prediction result; and performing matting processing on the basis of the prediction result, so as to obtain a target image. According to the embodiment, a foreground prediction result is more accurate, the matting accuracy is improved, and a target image which is obtained by means of matting processing has a better effect, thereby improving the user experience.

Description

Image processing method, device and electronic equipment

[0001] This application claims priority to Chinese Patent Application No. 202311000320.2 filed on August 9, 2023, the entire text of which is incorporated herein by reference as a part of this application. Technical Field

[0002] The present disclosure relates to an image processing method, device and electronic equipment. Background Art

[0003] During image processing, it is often necessary to replace the image background in order to beautify the image or increase its interest. Generally speaking, this can be accomplished through a cutout process. While cutout processing technology has matured, some issues remain. For example, the accuracy of cutouts is relatively low, and the edges of the target area to be cutout (especially fine edges such as hair) are processed roughly, resulting in a noticeable image processing artifact and poor quality. Therefore, a cutout processing method with high accuracy is needed.

[0004] Summary of the Invention

[0005] The present disclosure provides an image processing method, apparatus, and electronic device.

[0006] According to a first aspect, there is provided an image processing method, the method comprising:

[0007] Acquire a first image to be processed;

[0008] determining a region segmentation image corresponding to the first image;

[0009] Performing foreground prediction based on the region segmentation image and the first image to obtain a prediction result;

[0010] Perform image cutout processing based on the prediction result to obtain a target image.

[0011] According to a second aspect, there is provided an image processing apparatus, the apparatus comprising:

[0012] an acquisition module, configured to acquire a first image to be processed;

[0013] a determination module, configured to determine a region segmentation image corresponding to the first image;

[0014] A prediction module, configured to perform foreground prediction based on the region segmentation image and the first image to obtain a prediction result;

[0015] The cutout module is used to perform cutout processing based on the prediction result to obtain a target image.

[0016] According to a third aspect, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of the above-mentioned first aspects is implemented.

[0017] According to a fourth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods described in the first aspect when executing the program.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] FIG1 is a schematic diagram of an image processing scenario according to an exemplary embodiment;

[0021] FIG2 is a flowchart of an image processing method according to an exemplary embodiment of the present disclosure;

[0022] FIG3 is a flowchart of another image processing method according to an exemplary embodiment of the present disclosure;

[0023] FIG4 is a flowchart of another image processing method according to an exemplary embodiment of the present disclosure;

[0024] FIG5 is a block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure;

[0025] FIG6 is a schematic block diagram of an electronic device provided by some embodiments of the present disclosure;

[0026] FIG7 is a schematic block diagram of another electronic device provided by some embodiments of the present disclosure;

[0027] FIG8 is a schematic diagram of a storage medium provided by some embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments derived by those skilled in the art based on the embodiments in this specification without creative effort shall fall within the scope of protection of this specification.

[0029] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0030] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms "a," "the," and "the" used in this disclosure are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0031] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0032] Referring to FIG1 , it is a schematic diagram showing an image processing scenario according to an exemplary embodiment.

[0033] As shown in Figure 1, taking the matting process based on the original image A as an example, first, the original image A is downsampled to obtain image B with a resolution smaller than that of the original image A, and then image B is input into the network model a. The network model a includes an encoder a1 and a decoder a2. Image B can be first input into encoder a1 for encoding, and then the result of the encoding process is input into decoder a2 for decoding, thereby obtaining the region segmentation image C corresponding to image B. The resolution of the region segmentation image C is lower than that of the original image A. The region segmentation image C can be upsampled to obtain the region segmentation image D with the same resolution as the original image A.

[0034] Next, the region segmentation image D and the original image A are stacked (concat) to merge the channels to obtain a combined image E, which is then input into the network model b. The number of channels in the combined image E is the sum of the number of channels in the region segmentation image D and the original image A. The network model b includes an encoder b1 and a decoder b2. The combined image E can first be input into the encoder b1 for encoding, and the result of the encoding process is then input into the decoder b2 for decoding, to obtain the weight data of the pixels in the foreground of the original image A and the foreground color prediction image F.

[0035] Finally, the original image A and the foreground color prediction image F can be fused to obtain the image to be cut out H. The image to be cut out H, the background image I for providing the background, and the weight data are used to perform cutout processing to obtain the target image J.

[0036] It should be noted that during the model training phase, network model A and network model B can be trained together. Alternatively, network model A can be trained separately first, and then the trained network model A and network model B can be trained together again. If network model A is trained separately, images of different resolutions can be selected as sample images to train network model A separately, and then images with higher resolution can be selected as sample images to train network model A and network model B together.

[0037] In addition, during training, network model b can also output a fine-grained region segmentation image G with the same resolution as the original image A. The high-resolution fine-grained region segmentation labels and the fine-grained region segmentation image G can be used to calculate the prediction loss to adjust the model parameters of network models a and b. However, in the application of image matting, the region segmentation image G has no practical use.

[0038] The present disclosure will be described in detail below with reference to specific embodiments.

[0039] FIG2 is a flowchart of an image processing method according to an exemplary embodiment. The execution subject of the method can be implemented as any device, platform, server or device cluster with computing and processing capabilities. The method includes the following steps:

[0040] As shown in FIG2 , in step 201 , a first image to be processed is acquired, and in step 202 , a region segmentation image corresponding to the first image is determined.

[0041] In this embodiment, the first image is the original image to be cropped, and the area of ​​the original image other than the foreground object can be replaced as the background area to complete the cropping. A region segmentation image corresponding to the first image can be first determined. The region segmentation image can be a single-channel image obtained by roughly segmenting the foreground and background areas of the image. For example, the region segmentation image can be a binary image. It should be noted that the region segmentation image does not represent an image that accurately segments the foreground and background areas, but only represents a rough segmentation of the foreground and background areas.

[0042] In one implementation, a semantic segmentation image corresponding to the first image can be determined by semantic segmentation, and then based on the semantic segmentation image, at least one semantic region corresponding to the foreground and at least one semantic region corresponding to the background are merged to obtain a region segmentation image corresponding to the first image.

[0043] In another implementation, saliency prediction may be performed on the first image to obtain a saliency value corresponding to each pixel in the first image. Then, based on the saliency value corresponding to each pixel in the first image, a region segmentation image may be obtained. For example, the region corresponding to pixels having saliency values ​​greater than or equal to a preset threshold may be used as the estimated foreground region, and the region corresponding to pixels having saliency values ​​less than the preset threshold may be used as the estimated background region, thereby obtaining a region segmentation image that separates the estimated foreground region from the estimated background region.

[0044] In another implementation, a second image corresponding to the first image can be obtained. The second image has a lower resolution than the first image. For example, the first image can be downsampled to obtain the second image. Then, a saliency prediction can be performed on the second image using the pre-trained first network model to obtain a saliency value corresponding to each pixel in the second image. Finally, based on the saliency value corresponding to each pixel in the second image, a segmented image of the region is obtained.

[0045] In step 203, foreground prediction is performed based on the region segmentation image and the first image to obtain a prediction result.

[0046] The prediction result may include a first weight for a pixel of the first image in the foreground, which can represent the probability that the pixel belongs to the foreground. In one implementation, the image can be segmented based on the aforementioned regions, with the background region of the first image set to white or black, while the color value of the foreground region remains unchanged, to obtain a processed image. A pre-trained model is then used to perform foreground prediction on the processed image to obtain a prediction result.

[0047] In another implementation, a combined image can be obtained by stacking the merged channels based on the region segmentation image and the first image. For example, if the region segmentation image includes one channel and the first image includes three channels, stacking the region segmentation image and the first image can produce a combined image including four channels. The combined image can then be subjected to foreground prediction using the second network model to obtain a prediction result.

[0048] In step 204, a cutout process is performed based on the prediction result to obtain a target image.

[0049] In this embodiment, after obtaining the first weight of the pixel points of the first image in the foreground based on the region segmentation image and the first image, a cutout process can be performed based on the first weight of the pixel points in the foreground to obtain the target image. For example, a background image for replacing the background can be obtained, and the first image can be used as the image to be cutout. Then, the pixel values ​​of the pixel points at the same position in the image to be cutout and the background image are weightedly added to obtain the target image. See the following formula: Ms = ωEs + (1-ω)Fs;

[0050] Among them, Ms represents the pixel value of the pixel at position s in the target image, Es represents the pixel value of the pixel at position s in the image to be cut out, Fs represents the pixel value of the pixel at position s in the background image, and ω represents the first weight of the pixel at position s in the image to be cut out in the foreground.

[0051] The present disclosure provides an image processing method that determines a region segmentation image corresponding to a first image to be processed, performs foreground prediction based on the region segmentation image and the first image, obtains a prediction result, and then performs a cutout process based on the prediction result to obtain a target image. Because this embodiment first performs preliminary region segmentation on the image to be processed and then further performs foreground prediction based on the region segmentation result, the foreground prediction result is more accurate, the accuracy of the cutout is improved, and the target image obtained by the cutout process has a better effect, thereby enhancing the user experience.

[0052] FIG3 is a flow chart of another image processing method according to an exemplary embodiment. This embodiment describes a process of determining a region to segment an image and performing foreground prediction, including the following steps:

[0053] As shown in FIG3 , in step 301 , a first image to be processed is obtained, in step 302 , a second image corresponding to the first image is obtained, and in step 303 , a salient region segmentation process is performed on the second image using the first network model to obtain a region segmentation image.

[0054] In this embodiment, the second image has a lower resolution than the first image. The second image can be obtained by downsampling the first image, and the second image can be segmented using the first network model to obtain a region segmented image. The first network model can be a pre-trained network model for saliency prediction of images with lower resolutions, and the first network model can include a segmentation encoder and a segmentation decoder. Both the segmentation encoder and the segmentation decoder can be composed of a multi-layer convolutional neural network. The segmentation encoder can be used to extract high-dimensional features (such as semantic features, spatial features, etc.) in the image, and the segmentation decoder can be used to gradually amplify the high-dimensional features extracted by the encoder.

[0055] Specifically, the lower-resolution second image can be input into the segmentation encoder included in the first network model, and the output of the segmentation encoder can be input into the segmentation decoder to obtain a relatively coarse region segmentation image. This region segmentation image can provide a rough location of salient regions (regions with saliency values ​​greater than or equal to a preset threshold), thereby roughly segmenting the second image into an imprecise estimated foreground region and an estimated background region.

[0056] According to this embodiment, a second image with lower resolution is obtained based on the first image, and saliency region segmentation processing is performed on the second image to obtain a relatively coarse region segmentation image, thereby improving the efficiency of the region segmentation processing.

[0057] In step 304, the region segmentation image and the first image are stacked to obtain a combined image, and in step 305, the combined image is subjected to foreground prediction processing using the second network model to obtain a prediction result.

[0058] In this embodiment, the region segmentation image and the second image have the same resolution. Therefore, the region segmentation image can be first upsampled to obtain a coarse region segmentation image with the same resolution as the first image. The coarse region segmentation image and the first image are then stacked to obtain a combined image.

[0059] In this embodiment, the second network model may also be a pre-trained network model for saliency prediction of higher-resolution images. The second network model may also include an encoder and a decoder, and both the encoder and the decoder may be composed of a multi-layer convolutional neural network.

[0060] Specifically, the high-resolution combined image can be input into the encoder included in the second network model for processing to obtain first feature data. The combined image, the intermediate result of the encoder's processing of the combined image, and the first feature data are then stacked to obtain second feature data. Finally, the second feature data is processed using the decoder to obtain a prediction result. The intermediate result obtained by the encoder's processing of the combined image can be the intermediate result output by each layer of the convolutional neural network included in the encoder.

[0061] Due to this embodiment, the combined image, the intermediate result of the encoder processing the combined image, and the final result (i.e., the first feature data) are stacked to obtain the second feature data, so that some shallow features in the first image are integrated into the second feature data to be decoded (the shallow features come from the combined image and the intermediate result), and then the decoder is used to decode the second feature data, thereby improving the accuracy of the prediction result and helping to improve the effect of the cutout.

[0062] In step 306, a cutout process is performed based on the prediction result to obtain a target image.

[0063] It should be noted that, for the same steps as those in the embodiment of FIG. 2 , no further description will be given in the embodiment of FIG. 3 , and the relevant contents may be referred to in the embodiment of FIG. 2 .

[0064] Since this embodiment performs stacking processing based on the region segmentation image and the first image to obtain a combined image, the combined image to be processed has more channels and can contain richer image information, thereby improving the accuracy of the foreground prediction results and the accuracy of the cutout, so that the target image obtained by the cutout processing has a better effect, further improving the user experience.

[0065] FIG4 is a flow chart of another image processing method according to an exemplary embodiment. This embodiment describes a process of image cutout processing, including the following steps:

[0066] As shown in FIG4 , in step 401 , a first image to be processed is obtained, in step 402 , a region segmentation image corresponding to the first image is determined, and in step 403 , a foreground prediction is performed based on the region segmentation image and the first image to obtain a prediction result.

[0067] In this embodiment, the prediction result may include, in addition to the first weight of the pixel points in the foreground of the first image, a foreground color prediction image corresponding to the first image and a fine region segmentation image having the same resolution as the first image. The foreground color prediction image corresponding to the first image may be an image that predicts the pixel values ​​of the pixel points in the foreground of the first image.

[0068] Specifically, in the foreground color prediction image, the probability that the pixel value of the pixel point closer to the foreground middle area is the same as the pixel value of the pixel point at the same position in the first image is greater, so that the image features of the foreground color prediction image closer to the foreground middle area are closer to the image features at the same position in the first image. The probability that the pixel value of the pixel point closer to the foreground edge area is the same as the pixel value of the pixel point at the same position in the first image is smaller (but the probability is still greater than the specified probability), so that the image features of the foreground color prediction image closer to the foreground edge area are more different from the image features at the same position in the first image. In the foreground color prediction image, the probability that the pixel value of the pixel point far from the foreground area is the same as the pixel value of the pixel point at the same position in the first image can be close to 0, so that the image features of the foreground color prediction image far from the foreground area are more different from the image features at the same position in the first image.

[0069] It should be noted that the fine region segmentation image with the same resolution as the first image is mainly used in the model training stage, while in the cutout application stage, the fine region segmentation image is not needed.

[0070] In step 404, the image to be cutout is determined based on the foreground color prediction image included in the prediction result.

[0071] In this embodiment, the foreground color prediction image and the first image can be fused to obtain the image to be cut out. Specifically, in one implementation, the foreground color prediction image can be segmented into regions, and pixel information of the estimated foreground region can be extracted. Furthermore, the first image can be segmented into regions, and pixel information of the estimated background region can be extracted. The extracted pixel information can then be spliced ​​together to obtain the image to be cut out.

[0072] In another implementation, the second weight of the pixel in the foreground color prediction image can also be determined. The sum of the second weight of any pixel in the foreground color prediction image and the first weight of the pixel at the same position in the first image is 1. Then, based on the first weight and the second weight, the first image and the foreground color prediction image are weightedly fused to obtain the image to be cut out. For example, the product of the pixel value of the pixel in the first image and the first weight and the product of the pixel value of the pixel at the same position in the foreground color prediction image and the second weight can be added to obtain the pixel value of the pixel at the same position in the image to be cut out.

[0073] See the following formula: Ks=ωTs+(1-ω)Ys;

[0074] Among them, Ks represents the pixel value of the pixel at position s in the image to be cutout, Ts represents the pixel value of the pixel at position s in the first image, Ys represents the pixel value of the pixel at position s in the foreground color prediction image, ω represents the first weight corresponding to the pixel value of the pixel at position s in the first image, and 1-ω represents the second weight corresponding to the pixel value of the pixel at position s in the foreground color prediction image.

[0075] Since in this embodiment, the first weight is the weight of the pixel of the first image in the foreground, it can represent the confidence of the pixel of the first image in the foreground, and the sum of the first weight and the second weight is 1. Therefore, based on the first weight and the second weight, the first image and the foreground color prediction image are weightedly fused, so that the image features of the area with higher confidence in the image to be cut out after weighted fusion are closer to the first image, while the image features of the area with lower confidence are closer to the foreground color prediction image. This can avoid the situation where the edge of the foreground area in the image to be cut out is mixed with the background color due to the fact that the predicted foreground area is not accurate enough. Because if the predicted foreground area is not accurate enough, it may be mixed with part of the background area, so that the cutout foreground area contains the background color, and the color change is abrupt. In the foreground color prediction image, the pixel values ​​of the pixels near the edge of the foreground area combine the colors of the foreground and background, weakening the boundary between the foreground and background, and thus avoiding the situation where the color change is abrupt.

[0076] In step 405, a cutout process is performed based on the image to be cutout and the first weight included in the prediction result to obtain a target image.

[0077] In this embodiment, specifically, a background image for replacing the background can be obtained, and based on the first weight, a third weight of the pixels in the image to be cut out and a fourth weight of the pixels in the background image can be determined. The third weight of any pixel in the image to be cut out is the same as the first weight of the pixel at the same position in the first image, and the sum of the fourth weight of any pixel in the background image and the third weight of the pixel at the same position in the image to be cut out is 1. Based on the third weight and the fourth weight, the image to be cut out and the background image are weightedly fused to obtain the target image. For the specific process, see step 204 in the embodiment of FIG. 2 .

[0078] It should be noted that the same steps as those in FIG. 2 and FIG. 3 are not described in detail in the embodiment of FIG. 4 , and the relevant contents can be found in the embodiments of FIG. 2 and FIG. 3 .

[0079] Since this embodiment determines the image to be cutout based on the foreground color prediction image, and performs cutout processing based on the image to be cutout and the first weight to obtain the target image, the cutout effect can be further improved, thereby enhancing the user experience.

[0080] It should be noted that although the operations of the methods of the embodiments of the present disclosure are described in a specific order in the above embodiments, this does not require or imply that these operations must be performed in this specific order, or that all of the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flowcharts can be performed in a different order. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0081] Corresponding to the aforementioned image processing method embodiment, the present disclosure also provides an embodiment of an image processing device.

[0082] As shown in FIG5 , FIG5 is a block diagram of an image processing device according to an exemplary embodiment of the present disclosure. The device may include: an acquisition module 501 , a determination module 502 , a prediction module 503 and a cutout module 504 .

[0083] The acquisition module 501 is used to acquire a first image to be processed.

[0084] The determination module 502 is configured to determine a region segmentation image corresponding to the first image.

[0085] The prediction module 503 is configured to perform foreground prediction based on the region segmentation image and the first image to obtain a prediction result.

[0086] The cutout module 504 is configured to perform cutout processing based on the prediction result to obtain a target image.

[0087] In some implementations, the determination module 502 may include: an acquisition submodule and a segmentation submodule (not shown in the figure).

[0088] The acquisition submodule is used to acquire a second image corresponding to the first image, where the second image has a lower resolution than the first image.

[0089] The segmentation submodule is used to perform salient region segmentation processing on the second image using the first network model to obtain a region segmentation image.

[0090] In other implementations, the prediction module 503 may include: a stacking submodule and a prediction submodule (not shown in the figure).

[0091] The stacking submodule is used to perform stacking processing based on the region segmentation image and the first image to obtain a combined image.

[0092] The prediction submodule is used to perform foreground prediction processing on the combined image using the second network model to obtain a prediction result.

[0093] In some other embodiments, the second network model may include an encoder and a decoder, wherein the prediction submodule is configured to: process the combined image using the encoder to obtain first feature data, stack the combined image, an intermediate result of the encoder processing the combined image, and the first feature data to obtain second feature data, and process the second feature data using the decoder to obtain a prediction result.

[0094] In other embodiments, the prediction result includes the first weight of the pixel of the first image in the foreground and the foreground color prediction image corresponding to the first image. The prediction module 503 may include: a determination submodule and a cutout submodule (not shown in the figure).

[0095] The determination submodule is used to determine the image to be cutout based on the foreground color prediction image.

[0096] The cutout submodule is used to perform cutout processing based on the image to be cutout and the first weight to obtain a target image.

[0097] In other embodiments, the determination submodule is configured to: fuse the foreground color prediction image and the first image to obtain the image to be cutout.

[0098] In other embodiments, the determination submodule may fuse the foreground color prediction image and the first image to obtain the image to be cutout by determining a second weight of pixels in the foreground color prediction image, wherein the sum of the second weight of any pixel in the foreground color prediction image and the first weight of the pixel at the same position in the first image is 1. Based on the first weight and the second weight, the first image and the foreground color prediction image are weightedly fused to obtain the image to be cutout.

[0099] In other embodiments, the cutout submodule is configured to: obtain a background image for background replacement, and determine, based on the first weight, a third weight for pixels in the image to be cutout and a fourth weight for pixels in the background image. The third weight for any pixel in the image to be cutout is the same as the first weight for the pixel at the same position in the first image, and the sum of the fourth weight for any pixel in the background image and the third weight for the pixel at the same position in the image to be cutout is 1. Based on the third and fourth weights, the image to be cutout and the background image are weightedly fused to obtain a target image.

[0100] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiment scheme of the present disclosure. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0101] Figure 6 is a schematic block diagram of an electronic device provided by some embodiments of the present disclosure. As shown in Figure 6, the electronic device 910 includes a processor 911 and a memory 912, which can be used to implement a client or a server. The memory 912 is used to store computer-executable instructions (such as one or more computer program modules) non-transiently. The processor 911 is used to run the computer-executable instructions, and when the computer-executable instructions are run by the processor 911, one or more steps in the image processing method described above can be executed, thereby implementing the image processing method described above. The memory 912 and the processor 911 can be interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0102] For example, the processor 911 may be a central processing unit (CPU), a graphics processing unit (GPU), or other processing units with data processing capabilities and / or program execution capabilities. For example, the central processing unit (CPU) may be an X86 or ARM architecture. The processor 911 may be a general-purpose processor or a dedicated processor, and may control other components in the electronic device 910 to perform desired functions.

[0103] For example, the memory 912 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, a flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 911 may execute one or more computer program modules to implement various functions of the electronic device 910. Various applications and various data, as well as various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0104] It should be noted that, in the embodiment of the present disclosure, the specific functions and technical effects of the electronic device 910 can be referred to the above description of the image processing method, which will not be repeated here.

[0105] Figure 7 is a schematic block diagram of another electronic device provided by some embodiments of the present disclosure. The electronic device 920 is, for example, suitable for implementing the image processing method provided by the embodiments of the present disclosure. The electronic device 920 may be a terminal device, etc., which can be used to implement a client or a server. The electronic device 920 may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. It should be noted that the electronic device 920 shown in Figure 7 is only an example and does not impose any restrictions on the functions and scope of use of the embodiments of the present disclosure.

[0106] As shown in FIG7 , the electronic device 920 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 921, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 922 or a program loaded from a storage device 928 into a random access memory (RAM) 923. Various programs and data required for the operation of the electronic device 920 are also stored in the RAM 923. The processing device 921, the ROM 922, and the RAM 923 are connected to each other via a bus 924. An input / output (I / O) interface 925 is also connected to the bus 924.

[0107] Typically, the following devices may be connected to the I / O interface 925: an input device 926 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 927 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 928 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 929. The communication device 929 may allow the electronic device 920 to communicate with other electronic devices wirelessly or by wire to exchange data. Although FIG. 7 shows the electronic device 920 having various devices, it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 920 may alternatively implement or have more or fewer devices.

[0108] For example, according to an embodiment of the present disclosure, the above-mentioned image processing method can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the above-mentioned image processing method. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 929, or installed from the storage device 928, or installed from the ROM 922. When the computer program is executed by the processing device 921, the functions defined in the image processing method provided by the embodiment of the present disclosure can be implemented.

[0109] Figure 8 is a schematic diagram of a storage medium provided in some embodiments of the present disclosure. For example, as shown in Figure 8, storage medium 930 may be a non-transitory computer-readable storage medium for storing non-transitory computer-executable instructions 931. When non-transitory computer-executable instructions 931 are executed by a processor, the image processing method described in the embodiments of the present disclosure may be implemented. For example, when non-transitory computer-executable instructions 931 are executed by a processor, one or more steps of the image processing method described above may be performed.

[0110] For example, the storage medium 930 may be applied to the above-mentioned electronic device. For example, the storage medium 930 may include a memory in the electronic device.

[0111] For example, the storage medium may include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a flash memory, or any combination of the above storage media, or other applicable storage media.

[0112] For example, the description of the storage medium 930 can refer to the description of the memory in the embodiment of the electronic device, and the repeated parts are not repeated. The specific functions and technical effects of the storage medium 930 can refer to the description of the image processing method above, and are not repeated here.

[0113] It should be noted that in the context of the present disclosure, a computer-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0114] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

[0115] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, the method comprising: Acquire a first image to be processed; Determine a region segmentation image corresponding to the first image; Performing foreground prediction based on the region segmentation image and the first image to obtain a prediction result; A cutout process is performed based on the prediction result to obtain a target image.

2. According to the method of claim 1, determining the region segmentation image corresponding to the first image comprises: Acquire a second image corresponding to the first image; The second image has a lower resolution than the first image; The second image is processed by salient region segmentation using the first network model to obtain the region segmentation image.

3. The method according to claim 1, wherein: The performing foreground prediction based on the region segmentation image and the first image to obtain a prediction result includes: Perform stacking processing based on the region segmentation image and the first image to obtain a combined image; The second network model is used to perform foreground prediction processing on the combined image to obtain the prediction result.

4. The method according to claim 3, wherein: The second network model includes an encoder and a decoder; wherein the foreground prediction processing of the combined image using the second network model to obtain the prediction result includes: Processing the combined image using the encoder to obtain first feature data; Stacking the combined image, an intermediate result of processing the combined image by the encoder, and the first feature data to obtain second feature data; The second feature data is processed by using the decoder to obtain the prediction result.

5. The method according to any one of claims 1 to 4, wherein: The prediction result includes a first weight of a pixel point of the first image in the foreground and a foreground color prediction image corresponding to the first image; The step of performing a cutout process based on the prediction result to obtain a target image includes: Determining an image to be cutout based on the foreground color prediction image; A cutout process is performed based on the image to be cutout and the first weight to obtain a target image.

6. The method according to claim 5, wherein determining the image to be cutout based on the foreground color prediction image comprises: The foreground color prediction image and the first image are fused to obtain the image to be cutout.

7. The method according to claim 6, wherein: The step of fusing the foreground color prediction image and the first image to obtain the image to be cutout comprises: Determine a second weight of a pixel in the foreground color prediction image; wherein the sum of the second weight of any pixel in the foreground color prediction image and the first weight of a pixel at the same position in the first image is 1; Based on the first weight and the second weight, the first image and the foreground color prediction image are weightedly fused to obtain the image to be cutout.

8. The method according to any one of claims 5 to 7, wherein: The performing a matting process based on the image to be matted and the first weight to obtain a target image includes: Get the background image used to replace the background; Based on the first weight, determine the third weight of the pixel points in the image to be cut out and the fourth weight of the pixel points in the background image; wherein the third weight of any pixel point in the image to be cut out is the same as the first weight of the pixel point at the same position in the first image; and the sum of the fourth weight of any pixel point in the background image and the third weight of the pixel point at the same position in the image to be cut out is 1; Based on the third weight and the fourth weight, the image to be cutout and the background image are weightedly fused to obtain a target image.

9. An image processing device, comprising: An acquisition module, used for acquiring a first image to be processed; A determination module, used to determine a region segmentation image corresponding to the first image; A prediction module, used for performing foreground prediction based on the region segmentation image and the first image to obtain a prediction result; The cutout module is used to perform cutout processing based on the prediction result to obtain a target image.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 8.

11. An electronic device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 8 is implemented.

Images