Portrait-matting-based intelligent background replacement method, and NAS device

Abstract

The present invention relates to the technical field of image processing. Disclosed are a portrait-matting-based intelligent background replacement method, and a NAS device. The method comprises: inputting an image to be processed into a pre-trained portrait processing model, and performing feature channel processing on said image by means of the portrait processing model, in order to obtain a feature channel set; on the basis of the feature channel set, performing a feature processing operation on each feature map in the feature channel set, in order to obtain an image processing result corresponding to said image; and on the basis of binary classification information, determining a target image corresponding to said image, and performing background replacement on the target image. Hence, the present invention can realize feature fusion between channels of an image, so as to improve overall information communication, thereby improving the precision of a processed image, improving the processing efficiency and accuracy of performing portrait matting on the image, and improving the overall effect and precision of an image obtained after background replacement, and thus improving the user experience.

Description

A method for intelligent background replacement of portrait cutout and NAS device Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for intelligent background replacement of a portrait cutout and a NAS device. Background Art

[0002] With the rapid development of science and technology, more and more electronic devices are integrated with camera functions, allowing people to quickly and conveniently take photos to record their lives. As technology advances, people can also modify the photos they take (such as replacing the background) to make the photos achieve better effects.

[0003] Currently, image cutouts for portraits to replace backgrounds are primarily implemented using pixel models. This results in low efficiency and poor results, leading to a poor user experience. Therefore, it is crucial to propose a technical solution that can improve both the efficiency and effectiveness of image cutouts for portraits.

[0004] Summary of the Invention

[0005] The present invention provides a portrait cutout intelligent background replacement method and a NAS device, which can help improve the cutout efficiency and cutout effect of portrait cutout of an image.

[0006] In order to solve the above technical problems, the first aspect of the present invention discloses a method for intelligent background replacement of a portrait cutout, the method comprising:

[0007] Acquire an image to be processed, where the image to be processed includes a portrait area and a background area;

[0008] Inputting the image to be processed into a pre-trained portrait processing model, performing feature channel processing on the image to be processed by the portrait processing model to obtain a feature channel set, wherein the feature channel set includes at least one feature channel corresponding to the image to be processed and a feature map corresponding to each feature channel;

[0009] Based on the feature channel set, performing a feature processing operation on each of the feature maps in the feature channel set to obtain an image processing result corresponding to the image to be processed, wherein the image processing result includes binary classification information corresponding to the image to be processed;

[0010] A target image corresponding to the image to be processed is determined according to the binary classification information, and background replacement is performed on the target image, where the target image includes a portrait image and a background image corresponding to the image to be processed.

[0011] A second aspect of the present invention discloses a NAS device, wherein the NAS device is configured with at least a display screen, and the NAS device includes a memory storing executable program code;

[0012] a processor coupled to the memory;

[0013] The processor calls the executable program code stored in the memory to execute the portrait cutout intelligent background replacement method disclosed in the first aspect of the present invention.

[0014] The third aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the portrait cutout intelligent background replacement method disclosed in the first aspect of the present invention.

[0015] A fourth aspect of the present invention discloses a system for intelligent background replacement of a portrait cutout, the system comprising at least an electronic device and the NAS device disclosed in the second aspect of the present invention, wherein the electronic device is communicatively connected to the NAS device;

[0016] The electronic device is configured with an application program that can access the NAS device, and reads, through the application program, the image processed by the NAS device based on the portrait cutout intelligent background replacement method disclosed in the first aspect of the present invention.

[0017] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0018] In an embodiment of the present invention, the image to be processed can be input into a pre-trained portrait processing model, and the portrait processing model can be used to perform feature channel processing on the image to be processed, thereby achieving feature fusion between the channels of the image, improving overall information exchange, and thereby improving the accuracy of the processed image. Based on the feature channel set, feature processing operations are performed on each feature map in the feature channel set to obtain an image processing result corresponding to the image to be processed, thereby improving the processing efficiency and accuracy of portrait cutouts on the image, improving the overall effect and accuracy of the image after background replacement, and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] FIG1 is a flow chart of a method for intelligent background replacement of a portrait cutout disclosed in an embodiment of the present invention;

[0021] FIG2 is a flow chart of another method for intelligent background replacement of a portrait cutout disclosed in an embodiment of the present invention;

[0022] FIG3 is a schematic diagram of a model structure of a portrait processing model disclosed in an embodiment of the present invention;

[0023] FIG4 is a schematic structural diagram of a NAS device disclosed in an embodiment of the present invention;

[0024] FIG5 is a schematic structural diagram of a system for intelligent background replacement of a portrait cutout disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention discloses a method, apparatus, and NAS device for intelligent background replacement in portrait cutouts. These methods can achieve feature fusion across image channels, improving overall information exchange, thereby increasing the accuracy of processed images, improving the efficiency and accuracy of image cutout processing, and enhancing the overall effect and accuracy of background replacement images, thereby enhancing the user experience. These methods are described in detail below.

[0026] Example 1

[0027] Please refer to Figure 1, which is a flow chart of a method for intelligent background replacement of a portrait cutout disclosed in an embodiment of the present invention. The method for intelligent background replacement of a portrait cutout described in Figure 1 can be applied to a NAS (Network Attached Storage) device, wherein the NAS device may include an intelligent background replacement device for portrait cutout, and the intelligent background replacement device for portrait cutout may include an image processing device for performing portrait cutout background replacement processing on an image, and the image processing device may include an intelligent server or an intelligent platform, and the intelligent server includes a NAS server or a cloud server, which is not limited in the embodiment of the present invention. As shown in Figure 1, the method for intelligent background replacement of a portrait cutout may include the following operations:

[0028] 101. Obtain an image to be processed, input the image to be processed into a pre-trained portrait processing model, perform feature channel processing on the image to be processed by the portrait processing model, and obtain a feature channel set.

[0029] In an embodiment of the present invention, optionally, the image to be processed may include a portrait area and a background area, and the pre-trained portrait processing model may be a model based on the NAS end, that is, the user can perform portrait cutout and background replacement processing on the image stored in the NAS device on the private cloud NAS device, and the feature channel processing of the image to be processed may include downsampling the image to be processed, extracting image features of the image to be processed, and shuffling the feature channels of the image to be processed so that feature information can be exchanged between different feature channels to achieve feature fusion between different feature channels. The feature channel set may include at least one feature channel corresponding to the image to be processed and a feature map corresponding to each feature channel, which is not limited by the present invention.

[0030] 102. Based on the feature channel set, perform a feature processing operation on each feature map in the feature channel set to obtain an image processing result corresponding to the image to be processed.

[0031] In an embodiment of the present invention, optionally, performing a feature processing operation on each feature map in the feature channel set may include performing depthwise convolution and pointwise convolution operations on each feature map in the feature channel set based on depthwise separable convolution, and performing an upsampling operation on each feature map in a preset manner during the depthwise convolution and pointwise convolution operations. The preset manner may include bicubic interpolation, and performing a binary classification judgment on the image after the convolution operation and the upsampling operation to obtain an image processing result corresponding to the image to be processed. The image processing result includes binary classification information corresponding to the image to be processed, which is not limited by the present invention.

[0032] 103. Determine a target image corresponding to the image to be processed according to the binary classification information, and perform background replacement on the target image.

[0033] In an embodiment of the present invention, optionally, the boundary information of the portrait and background in the image to be processed can be determined based on the binary classification information, and then the target image can be obtained. The target image can include a portrait image and a background image. The background of the image to be processed can be replaced intelligently, or the background of the image to be processed can be replaced manually by the user. The present invention does not limit this.

[0034] It can be seen that the implementation of the portrait cutout intelligent background replacement method described in Figure 1 can input the image to be processed into a pre-trained portrait processing model, and perform feature channel processing on the image to be processed through the portrait processing model to obtain a feature channel set, which can realize feature fusion between the channels of the image, improve the overall information exchange, and thus improve the accuracy of the processed image. Based on the feature channel set, feature processing operations are performed on each feature map in the feature channel set to obtain the image processing result corresponding to the image to be processed, and the target image corresponding to the image to be processed is determined according to the binary classification information, and the background of the target image is replaced, which can improve the processing efficiency and accuracy of portrait cutout of the image, improve the overall effect and accuracy of the image after background replacement, and enhance the user experience.

[0035] In an optional embodiment, based on the feature channel set, performing a feature processing operation on each feature map in the feature channel set to obtain an image processing result corresponding to the image to be processed may include the following operations:

[0036] Based on the feature channel set, a depth convolution operation is performed on each feature map in the feature channel set to obtain the target features of each feature map in the feature channel set;

[0037] Perform feature fusion on the target features of each feature map to obtain the fused feature map corresponding to the image to be processed;

[0038] The fused feature map is subjected to binary classification processing to obtain binary classification information corresponding to the image to be processed, and the image processing result corresponding to the image to be processed is determined based on the binary classification information.

[0039] In this optional embodiment, optionally, after channel shuffling is performed on the initial feature channel set corresponding to the image to be processed, a deep convolution operation is performed on each feature map in the feature channel set, and the spatial features of each feature map after feature fusion in the feature channel set are extracted to obtain the target features of each feature map in the feature channel set, that is, the target features include spatial features, and feature fusion of the target features of each feature map may include feature fusion of the target features of each feature map based on point-by-point convolution, and may also include feature channel fusion of the feature channels of each feature map based on point-by-point convolution, which is not limited in this embodiment.

[0040] In this optional embodiment, optionally, performing binary classification processing on the fused feature map may include performing binary classification judgment on each pixel point in the fused feature map to distinguish whether the pixel point is a portrait or a background. Optionally, the portrait pixel point may be assigned a binary classification label of 1, and the background pixel point may be assigned a binary classification label of 0. The binary classification information corresponding to the image to be processed includes the binary classification label information of each pixel point in the image. The image processing result may include a binary map corresponding to the image to be processed, and then the portrait image and background image of the image to be processed are determined based on the binary map. This is not limited in this embodiment.

[0041] It can be seen that the implementation of this optional embodiment can perform a deep convolution operation on each feature map in the feature channel set based on the feature channel set, obtain the target features of each feature map in the feature channel set, perform feature fusion on the target features of each feature map, obtain a fused feature map corresponding to the image to be processed, improve the image fineness of the fused feature map, perform binary classification processing on the fused feature map, obtain binary classification information corresponding to the image to be processed, and determine the image processing result corresponding to the image to be processed based on the binary classification information. The portrait image and background image of the image to be processed can be determined through the binary classification information, thereby improving the accuracy of image cutout and background replacement.

[0042] In another optional embodiment, based on the feature channel set, performing a depthwise convolution operation on each feature map in the feature channel set to obtain a target feature of each feature map in the feature channel set may include the following operations:

[0043] Obtaining the initial image features of the image to be processed through the skip connection layer in the portrait processing model, and upsampling each feature map in the feature channel set based on the initial image features and a preset upsampling method to obtain an upsampled feature channel set of the image to be processed. The upsampling method includes bicubic interpolation upsampling;

[0044] Based on the upsampled feature channel set, a depth convolution operation is performed on each feature map in the feature channel set to obtain the target features of each feature map in the feature channel set.

[0045] In this optional embodiment, optionally, the feature image size corresponding to the initial image feature is the same as the image size after the upsampling operation, which is used to provide multi-level image information during the upsampling process. For segmentation tasks, spatial domain information is very important. Mainstream segmentation networks all adopt an encoder-decoder structure. The encoder part of the network reduces the feature map resolution to a very small level through the downsampling layer, which is not conducive to accurate segmentation mask generation. The encoder and decoder are connected across layers through a skip connection layer, which is more conducive to generating a fine mask to improve the fineness of the upsampled image. The upsampling method may include one of transposed convolution, depooling, interpolation, sub-pixel convolution, etc. Since a checkerboard effect is likely to occur when using transposed convolution for upsampling, bicubic interpolation upsampling is preferred in this embodiment, and each feature map can be upsampled during the deep convolution operation on the feature map. This is not limited in this embodiment.

[0046] It can be seen that the implementation of this optional embodiment can obtain the initial image features of the image to be processed through the jump connection layer in the portrait processing model, and upsample each feature map in the feature channel set based on the initial image features and the preset upsampling method to obtain the upsampled feature channel set of the image to be processed. It can be combined with the initial image features of the image for upsampling, and the fineness and overall quality of the image can be improved without increasing the amount of calculation, while solving the checkerboard effect problem. Based on the upsampled feature channel set, a deep convolution operation is performed on each feature map in the feature channel set to obtain the target features of each feature map in the feature channel set, thereby improving the image quality of the image obtained after convolution processing, and thereby improving the accuracy of subsequent binary classification judgments.

[0047] Example 2

[0048] Please refer to Figure 2, which is a flow chart of a method for intelligent background replacement of a portrait cutout disclosed in an embodiment of the present invention. The method for intelligent background replacement of a portrait cutout described in Figure 2 can be applied to a NAS (Network Attached Storage) device, wherein the NAS device may include an intelligent background replacement device for a portrait cutout, and the intelligent background replacement device for a portrait cutout may include an image processing device for performing portrait cutout background replacement processing on an image, and the image processing device may include an intelligent server or an intelligent platform, and the intelligent server includes a NAS server or a cloud server, which is not limited in the embodiment of the present invention. As shown in Figure 2, the method for intelligent background replacement of a portrait cutout may include the following operations:

[0049] 201. Obtain an image to be processed, input the image to be processed into a pre-trained portrait processing model, downsample the image to be processed, and obtain initial image features of the image to be processed.

[0050] In an embodiment of the present invention, optionally, the image to be processed may be downsampled to extract image features of the image to be processed, and the image features may be compressed and dimensionally reduced to obtain initial image features of the image to be processed, which is not limited in the present invention.

[0051] 202. Perform a group convolution operation on the initial image features of the image to be processed to obtain an initial feature channel set corresponding to the image to be processed.

[0052] In an embodiment of the present invention, optionally, a group convolution operation can be performed on the initial image features based on the depthwise convolution in the depthwise separable convolution to obtain an initial feature channel set corresponding to the image to be processed, and the initial feature channel set includes at least one initial feature channel and a feature map corresponding to each initial feature channel, which is not limited by the present invention.

[0053] 203. Shuffle the initial feature channel set corresponding to the image to be processed according to a preset shuffling rule to obtain a feature channel set.

[0054] In an embodiment of the present invention, optionally, the preset shuffling rules may include random shuffling or transposition shuffling, and channel shuffling is performed on the initial feature channel set corresponding to the image to be processed, so that feature information can be exchanged between different feature channels, and feature fusion between different feature channels can be achieved. The feature channel set may include at least one feature channel corresponding to the image to be processed and a feature map corresponding to each feature channel. The depthwise convolution used in the depthwise separable convolution will group all channels, and each channel is divided into a group, which results in no information exchange between groups. Channel shuffling can ensure that the next grouped convolution input comes from different groups by "reorganizing" the feature map after the grouped convolution, so that information can flow between different groups. The present invention does not limit this.

[0055] 204. Based on the feature channel set, perform a feature processing operation on each feature map in the feature channel set to obtain an image processing result corresponding to the image to be processed.

[0056] 205. Determine a target image corresponding to the image to be processed according to the binary classification information, and perform background replacement on the target image.

[0057] In the embodiment of the present invention, for other descriptions of step 204 and step 205, please refer to the detailed description of step 102 and step 103 in the first embodiment, which will not be repeated in the embodiment of the present invention.

[0058] It can be seen that the implementation of the portrait cutout intelligent background replacement method described in Figure 2 can input the image to be processed into a pre-trained portrait processing model, downsample the image to be processed, obtain the initial image features of the image to be processed, perform group convolution operations on the initial image features of the image to be processed, obtain the initial feature channel set corresponding to the image to be processed, and perform channel shuffling on the initial feature channel set corresponding to the image to be processed according to the preset shuffling rules to obtain a feature channel set. The channel shuffling method can realize feature fusion between channels of the image, improve overall information exchange, and thereby improve the accuracy of the processed image. Based on the feature channel set, feature processing operations are performed on each feature map in the feature channel set to obtain the image processing result corresponding to the image to be processed. The target image corresponding to the image to be processed is determined according to the binary classification information, and the background of the target image is replaced. This can improve the processing efficiency and accuracy of portrait cutout of the image, improve the overall effect and accuracy of the image after background replacement, and enhance the user experience.

[0059] In an optional embodiment, the portrait cutout intelligent background replacement method may further include the following operations:

[0060] Determine the model application information corresponding to the portrait processing model, and build an initial portrait processing model based on the model application information and a preset convolutional neural network. The initial portrait processing model includes a feature processing layer, a channel shuffling layer, a convolution layer, and a binary classification output layer.

[0061] Input the preset test data into the initial portrait processing model for analysis to obtain the test output corresponding to the test data, which includes pre-labeled label data;

[0062] Calculate the model loss corresponding to the initial portrait processing model based on the test output and label data, and determine whether the model loss is greater than the preset loss threshold;

[0063] When the model loss is greater than the preset loss threshold, the initial portrait processing model is optimized and trained according to the model loss. When the model loss is less than or equal to the preset loss threshold, the initial portrait processing model is determined as the portrait processing model.

[0064] In this optional embodiment, optionally, the model application information corresponding to the portrait processing model may include a model application scenario of the portrait processing model, the model application scenario may include a portrait cutout scenario, the preset convolutional neural network may include a depthwise separable convolution, the initial portrait processing model and the model structure of the portrait processing model may be consistent, the initial portrait processing model includes a feature processing layer, a channel shuffling layer, a convolution layer and a binary classification output layer, wherein the model structure of the portrait processing model may be as shown in FIG3 , FIG3 A schematic diagram of the model structure of a portrait processing model disclosed in an embodiment of the present invention, wherein the portrait processing model may include a Conv3*3 layer, a MaxPool layer, a Stage1 layer, a Stage2 layer, a DSConv1 layer, an Upsample1 layer, a DSConv2 layer, a DSConv3 layer, an Upsample2 layer, a Softmax layer and a Conv1*1 layer. The feature processing layer may correspond to the Conv3*3 layer and the MaxPool layer. Taking the image to be processed with a length and width of L as an example, the Conv3*3 layer is used to downsample the image to be processed through a 3*3 convolution kernel, extract image features, and obtain a feature map of L / 2. The MaxPool layer is used to downsample the feature map L / 2 again and compress the feature map to obtain a feature map of L / 4. The channel shuffling layer may correspond to the Stage1 layer and the Stage2 layer. The Stage1 layer is used to perform group convolution on the feature map L / 4 through depth convolution to obtain a feature map of L / 8. The Stage2 The layer is used to shuffle the channels of the feature map L / 8 to obtain a feature map of L / 16. The convolution layer can correspond to the DSConv1 layer, the Upsample1 layer, the DSConv2 layer, the DSConv3 layer, and the Upsample2 layer, where the DSConv1, 2, and 3 layers represent the process of depth-separable convolution. The Upsample1 layer is used to upsample the feature map L / 16 to obtain a feature map of L / 2. The Upsample2 layer is used to upsample the feature map L / 2, and finally obtain a fused feature map with a length and width of L. The Conv1*1 layer is used to introduce the feature information corresponding to the size of the Upsample1 layer into the Upsample1 layer to participate in upsampling. The binary classification output layer can correspond to the Softmax layer. The Softmax layer is used to perform binary classification judgment on each pixel in the fused feature map to distinguish whether the pixel is a portrait or a background. Optionally, the portrait pixel can be assigned a binary classification label of 1, and the background pixel can be assigned a binary classification label of 0. This embodiment is not limited to this.

[0065] In this optional embodiment, optionally, the test data may include pre-labeled label data, and the label data may be a portrait binary image corresponding to the test image in the test data. The model loss corresponding to the initial portrait processing model is calculated based on the test output and the label data. When the model loss is greater than a preset loss threshold, the initial portrait processing model is optimized and trained based on the model loss. When the model loss is less than or equal to the preset loss threshold, the initial portrait processing model is determined as the portrait processing model. This embodiment does not limit this.

[0066] It can be seen that the implementation of this optional embodiment can construct an initial portrait processing model based on model application information and convolutional neural network, improve the design rationality of the model structure, calculate the model loss corresponding to the initial portrait processing model according to the test output and label data, and perform model optimization iteration on the initial portrait processing model according to the model loss, thereby improving the overall computational efficiency and accuracy of the model.

[0067] In another optional embodiment, calculating the model loss corresponding to the initial portrait processing model based on the test output and the label data may include the following operations:

[0068] Calculate the loss information corresponding to the initial portrait processing model based on the test output and label data;

[0069] Calculate the model loss corresponding to the initial portrait processing model based on the loss information;

[0070] Among them, the loss information includes the target loss of the initial portrait processing model and the hyperparameters corresponding to the target loss. The target loss includes cross entropy loss and label distribution aware margin loss. The hyperparameters include the first hyperparameter and the second hyperparameter.

[0071] In this optional embodiment, optionally, the loss information corresponding to the initial portrait processing model may include the target loss of the initial portrait processing model and the hyperparameters corresponding to the target loss, the target loss includes the cross entropy loss and the label distribution aware margin loss, and the hyperparameters include the first hyperparameter and the second hyperparameter, which is not limited in this embodiment.

[0072] In this optional embodiment, the proportion of portraits in the entire image is often small, resulting in an imbalance in the proportion of foreground and background categories. The commonly used cross-entropy loss treats positive and negative samples fairly. When the proportion of positive samples is small, they are overwhelmed by the larger number of negative samples. Therefore, the model loss corresponding to the initial portrait processing model is calculated using cross-entropy loss and label distribution-aware margin loss to improve the model's robustness to category imbalance. This is not limited in this embodiment.

[0073] It can be seen that the implementation of this optional embodiment can design the loss function corresponding to the initial portrait processing model based on cross entropy loss and label distribution aware margin loss, effectively improving the model's processing ability when dealing with category imbalance problems and improving the model's robustness.

[0074] In another optional embodiment, the calculation formula of the model loss is: U_loss=α*CE_loss+β*LDAM_loss

[0075] Among them, U_loss represents model loss, CE_loss represents cross entropy loss, LDAM_loss represents label distribution aware margin loss, α represents the first hyperparameter, and β represents the second hyperparameter.

[0076] In this optional embodiment, α represents a first hyperparameter, and β represents a second hyperparameter. While the initial portrait processing model is optimized and trained according to the model loss, the first hyperparameter and the second hyperparameter are adjusted. The value range of the first hyperparameter and the second hyperparameter is [0, 1]. Preferably, the first hyperparameter can be 0.25 and the second hyperparameter can be 0.75, which is not limited in this embodiment.

[0077] In this optional embodiment, optionally, the calculation formula of the cross entropy loss CE_loss is:

[0078] Among them, y i Indicates the label of sample i, the positive class is 1, the negative class is 0, a i Indicates the probability that sample i is predicted to be positive;

[0079] The calculation formula of label distribution aware margin loss LDAM_loss is:

[0080] Wherein, f represents the initial portrait processing model, and y=j represents label data, which is not limited in this embodiment.

[0081] It can be seen that the implementation of this optional embodiment can design the loss function corresponding to the initial portrait processing model based on cross entropy loss and label distribution aware margin loss, effectively improving the model's processing ability when dealing with category imbalance problems and improving the model's robustness.

[0082] Example 3

[0083] Please refer to Figure 4, which is a schematic diagram of the structure of a NAS device disclosed in an embodiment of the present invention. As shown in Figure 4, the NAS device is configured with at least a display screen 501, and the NAS device includes a memory 502 storing executable program code;

[0084] a processor 503 coupled to the memory 502;

[0085] The processor 503 calls the executable program code stored in the memory 502 to execute the steps of the portrait cutout intelligent background replacement method described in the first embodiment of the present invention or the second embodiment of the present invention.

[0086] Example 4

[0087] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps of the portrait cutout intelligent background replacement method described in Embodiment 1 or Embodiment 2 of the present invention.

[0088] Example 5

[0089] Please refer to Figure 5, which is a schematic diagram of the structure of a system for intelligent background replacement of a portrait cutout disclosed in an embodiment of the present invention. As shown in Figure 5, the system includes at least one electronic device and a NAS device as described in Example 3, wherein the electronic device is in communication with the NAS device;

[0090] The electronic device is configured with an application program that can access the NAS device, and reads, through the application program, the image processed by the NAS device based on the portrait cutout intelligent background replacement method described in the first or second embodiment of the present invention.

[0091] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

Claims

1. An intelligent background replacement method for portrait matting, characterized in that, The method includes: Obtaining an image to be processed, where the image to be processed includes a portrait area and a background area; Inputting the image to be processed into a pre-trained portrait processing model, and performing feature channel processing on the image to be processed through the portrait processing model to obtain a set of feature channels, where the set of feature channels includes at least one feature channel corresponding to the image to be processed and a feature map corresponding to each feature channel; Based on the set of feature channels, performing a feature processing operation on each feature map in the set of feature channels to obtain an image processing result corresponding to the image to be processed, where the image processing result includes binary classification information corresponding to the image to be processed; Determining a target image corresponding to the image to be processed according to the binary classification information, and performing background replacement on the target image, where the target image includes a portrait image and a background image corresponding to the image to be processed.

2. The intelligent portrait matting and background replacement method according to claim 1, wherein The step of inputting the image to be processed into a pre-trained portrait processing model, and performing feature channel processing on the image to be processed through the portrait processing model to obtain a set of feature channels includes: Inputting the image to be processed into a pre-trained portrait processing model, and performing downsampling on the image to be processed to obtain initial image features of the image to be processed; Performing grouped convolution operations on the initial image features of the image to be processed to obtain an initial set of feature channels corresponding to the image to be processed; Performing channel shuffling on the initial set of feature channels corresponding to the image to be processed according to a preset shuffling rule to obtain a set of feature channels.

3. The method for intelligent background replacement of portrait matting according to claim 2, wherein The step of, based on the set of feature channels, performing a feature processing operation on each feature map in the set of feature channels to obtain an image processing result corresponding to the image to be processed includes: Based on the set of feature channels, performing depth convolution operations on each feature map in the set of feature channels to obtain target features of each feature map in the set of feature channels; Performing feature fusion on the target features of each feature map to obtain a fused feature map corresponding to the image to be processed; Performing binary classification processing on the fused feature map to obtain binary classification information corresponding to the image to be processed, and determining the image processing result corresponding to the image to be processed according to the binary classification information.

4. The method for intelligent background replacement of portrait matting according to claim 3, characterized in that, The step of, based on the set of feature channels, performing depth convolution operations on each feature map in the set of feature channels to obtain target features of each feature map in the set of feature channels includes: Obtaining the initial image features of the image to be processed through a skip connection layer in the portrait processing model, and performing upsampling on each feature map in the set of feature channels based on the initial image features and a preset upsampling method, where the upsampling method includes bicubic interpolation upsampling, to obtain an upsampled set of feature channels of the image to be processed; Based on the upsampled set of feature channels, performing depth convolution operations on each feature map in the set of feature channels to obtain target features of each feature map in the set of feature channels.

5. The intelligent portrait matting and background replacement method according to claim 1, wherein The method further includes: Determine the model application information corresponding to the portrait processing model, and construct an initial portrait processing model based on the model application information and a preset convolutional neural network. The initial portrait processing model includes a feature processing layer, a channel shuffle layer, a convolutional layer, and a binary classification output layer; Input the preset test data into the initial portrait processing model for analysis to obtain the test output corresponding to the test data. The test data includes pre-labeled tag data; Calculate the model loss corresponding to the initial portrait processing model according to the test output and the tag data, and determine whether the model loss is greater than a preset loss threshold; When the model loss is greater than the preset loss threshold, perform model optimization training on the initial portrait processing model according to the model loss. When the model loss is less than or equal to the preset loss threshold, determine the initial portrait processing model as the portrait processing model.

6. The method for intelligent background replacement of portrait matting according to claim 5, characterized in that, The calculating the model loss corresponding to the initial portrait processing model according to the test output and the tag data includes: Calculate the loss information corresponding to the initial portrait processing model according to the test output and the tag data; Calculate the model loss corresponding to the initial portrait processing model according to the loss information; Wherein, the loss information includes the target loss of the initial portrait processing model and the hyperparameters corresponding to the target loss. The target loss includes cross-entropy loss and label distribution aware margin loss, and the hyperparameters include a first hyperparameter and a second hyperparameter.

7. The method for intelligent background replacement of portrait matting according to claim 6, wherein The calculation formula of the model loss is: U_loss = α * CE_loss + β * LDAM_loss Wherein, U_loss represents the model loss, CE_loss represents the cross-entropy loss, LDAM_loss represents the label distribution aware margin loss, α represents the first hyperparameter, and β represents the second hyperparameter.

8. The intelligent background replacement method for portrait matting according to claim 6, characterized in that The model application scenario includes a portrait matting scenario. The preset convolutional neural network includes depthwise separable convolution; the initial portrait processing model includes a feature processing layer, a channel shuffle layer, a convolutional layer, and a binary classification output layer.

9. A NAS device, characterized in that, The NAS device is at least configured with a display screen, and the NAS device includes a memory storing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the portrait matting intelligent background replacement method according to any one of claims 1-8.

10. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which are used to execute the portrait matting intelligent background replacement method according to any one of claims 1-8 when called.

11. A system for intelligent background replacement in portrait matting, characterized in that, The system at least includes an electronic device and the NAS device according to claim 9, and the electronic device is communicatively connected to the NAS device; The electronic device is configured with an application program that can access the NAS device, and reads the picture processed by the NAS device based on the portrait matting intelligent background replacement method according to any one of claims 1-8 through the application program.

Images