An image segmentation method, device, electronic device and storage medium

Abstract

The present application provides an image segmentation method, apparatus, electronic device and storage medium, comprising: inputting an image to be segmented into the first two first convolutional layers of a feature extraction network and performing convolution processing in sequence to obtain an original feature map output by the second convolutional layer of the feature extraction network; performing noise separation on the original feature map to obtain an initial noise map, and reconstructing the initial noise map to obtain a noise distribution map; calculating the feature value of each optimized pixel point based on the feature value of each pixel point in the noise distribution map and the original feature map; generating an optimized feature image based on the feature value of each optimized pixel point; performing portrait segmentation on the optimized feature image to obtain a portrait area and a background area, with the aim of improving the accuracy of image segmentation through a deep convolutional neural network.

Description

Technical Field

[0001] The present application relates to the field of image segmentation technology, and more specifically, to an image segmentation method, device, electronic device, and storage medium. Background Art

[0002] Existing methods, leveraging the powerful feature representation capabilities of deep convolutional neural networks (DCNNs), have achieved good performance on public datasets for pedestrian segmentation. However, the actual application scenarios of pedestrian segmentation are often complex. In addition to pedestrians, captured images often contain a variety of interfering factors, such as non-relevant objects and backgrounds. These interfering factors appear as obvious noise to DCNNs, resulting in low segmentation accuracy when segmenting pedestrians in portrait images using DCNNs. Summary of the Invention

[0003] In view of this, the purpose of the present application is to provide an image segmentation method, device, electronic device and storage medium, the purpose of which is to improve the accuracy of image segmentation.

[0004] In a first aspect, an embodiment of the present application provides an image segmentation method, comprising: inputting the image to be segmented into the first two first convolutional layers of a feature extraction network and performing convolution processing in sequence to obtain the original feature map output by the second convolutional layer of the feature extraction network; performing noise separation on the original feature map to obtain an initial noise map, and reconstructing the initial noise map to obtain a noise distribution map; calculating the characteristic value of each optimized pixel point based on the characteristic value of each pixel point in the noise distribution map and the original feature map; generating an optimized feature image based on the characteristic value of each optimized pixel point; performing portrait segmentation on the optimized feature image to obtain a portrait area and a background area.

[0005] Preferably, the steps of performing noise separation on the original feature map to obtain an initial noise map, and reconstructing the initial noise map to obtain a noise distribution map include: inputting the original feature map into multiple sequentially connected first convolution modules of the noise separation model for noise separation, and obtaining an initial noise map output by the last first convolution module; wherein the noise separation model includes multiple sequentially connected first convolution modules and a second convolution layer; performing boundary filling on the initial noise map and then inputting the map into the second convolution layer for convolution processing to obtain a noise distribution map output by the second convolution layer; wherein the width, height, and channel dimensions of the noise distribution map are the same as the width, height, and channel dimensions of the original feature map.

[0006] Preferably, each first convolution module is composed of a second convolution layer, a batch normalization layer and an activation layer connected in series. For each first convolution module, noise separation is performed on the original feature map input into the first convolution module in the following manner: the second convolution layer performs convolution processing on the original feature map and outputs the convolution result to the batch normalization layer; the batch normalization layer normalizes the convolution result and outputs the normalized result to the activation layer; the activation layer sets the values less than zero in the normalized result to zero to obtain the initial noise map output by the activation layer.

[0007] Preferably, the step of calculating the characteristic value of each optimized pixel point based on the characteristic value of each pixel point in the noise distribution map and the original feature map includes: determining the characteristic value of each first pixel point on the original feature map, and determining the characteristic value of each second pixel point on the noise distribution map; for every two first pixel points and second pixel points with the same spatial position, taking the difference between the characteristic value of the first pixel point and the characteristic value of the second pixel point as the characteristic difference value at the spatial position; for each spatial position, comparing the characteristic value of the first pixel point at the spatial position with the characteristic difference value at the spatial position; selecting the maximum value between the characteristic value and the characteristic difference value of the first pixel point as the characteristic value of the optimized pixel point at the spatial position in the optimized feature map.

[0008] Preferably, training is performed in the following manner to obtain a noise separation model: a training data set is obtained, the data set includes multiple data pairs, each data pair includes a training feature map and a standard noise distribution map of the training feature map; for each data pair, the training feature map is input into the noise learning model to obtain a training noise distribution map; based on the training noise distribution map and the corresponding standard noise distribution map, the L1 norm loss value is calculated; based on the L1 norm loss value as the loss of the noise learning model, the parameters of the noise learning model are adjusted to complete the training of the noise separation model based on a single data pair.

[0009] Preferably, a training data set is generated in the following manner: irrelevant objects and backgrounds are removed from a plurality of training samples by manual labeling to obtain a plurality of denoised samples; feature extraction is performed on the training samples and the denoised samples respectively to obtain a training feature map and a denoised feature map; a standard noise distribution map is determined based on the difference between the training feature map and the denoised feature map; and the training feature map and the corresponding standard noise distribution map are combined into a data pair as training data.

[0010] Preferably, the target segmentation model is trained in the following manner to perform portrait segmentation on the optimized feature image: obtain target feature images corresponding to multiple training samples; for each target feature image, input the target feature image into the segmentation learning model to obtain a predicted segmentation image; based on the predicted segmentation image, determine the pixel cross entropy loss value of the current segmentation learning model for each pixel point of the target feature image; determine the average value of the pixel cross entropy loss values of all pixels as the image cross entropy loss value; use the image cross entropy loss value as the loss of the segmentation learning model to adjust the parameters of the segmentation learning model to complete the training process of the segmentation model based on a single training sample.

[0011] In a second aspect, an embodiment of the present application further provides an image segmentation device, comprising:

[0012] The feature extraction module is used to input the image to be segmented into the first two convolutional layers of the feature extraction network and perform convolution processing in sequence to obtain the original feature map output by the second convolutional layer of the feature extraction network;

[0013] The noise separation module is used to perform noise separation on the original feature map to obtain an initial noise map, and reconstruct the initial noise map to obtain a noise distribution map;

[0014] A calculation module is used to calculate the characteristic value of each optimized pixel point based on the characteristic value of each pixel point in the noise distribution map and the original characteristic map;

[0015] A generation module, used for generating an optimized feature image based on the feature value of each optimized pixel point;

[0016] The segmentation module is used to perform portrait segmentation on the optimized feature image to obtain the portrait area and the background area.

[0017] In a third aspect, an embodiment of the present application also provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the above image segmentation method.

[0018] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above image segmentation method are executed.

[0019] An image segmentation method, device and storage medium provided in the embodiments of the present application perform feature extraction on an image to be segmented to obtain an original feature map, perform noise separation and reconstruction on the original feature map to obtain a corresponding noise distribution map, obtain an optimized feature image based on the noise distribution map of the original feature map and the original feature map, perform portrait segmentation based on the optimized feature image to output a portrait segmentation image, and recognize the denoised image. These methods can effectively avoid the influence of noise on the accuracy of portrait segmentation and improve the accuracy of image segmentation.

[0020] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of an image segmentation method provided in an embodiment of the present application;

[0023] Figure 2 A flowchart of the training steps of a noise separation model provided in an embodiment of the present application;

[0024] Figure 3 A flowchart of the training steps of an object segmentation model provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of the structure of a noise separation model provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of a portrait segmentation image provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of the structure of an image segmentation device provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0030] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0031] Existing methods, leveraging the powerful feature representation capabilities of deep convolutional neural networks (DCNNs), have achieved good performance on public datasets for pedestrian segmentation. However, the actual application scenarios of pedestrian segmentation are often complex. In addition to pedestrians, captured images often contain a variety of interfering factors, such as non-relevant objects and backgrounds. These interfering factors appear as obvious noise to DCNNs, making it difficult for the network to learn effective features. Consequently, existing DCNN methods for pedestrian segmentation suffer from reduced accuracy.

[0032] In response to the above problems, embodiments of the present application provide an image segmentation method, apparatus, electronic device, and storage medium, which are described below through embodiments.

[0033] To facilitate understanding of the present application, the technical solutions provided in the present application are described in detail below in conjunction with specific embodiments.

[0034] See also Figure 1 , is an image segmentation method provided in an embodiment of the present application, the method comprising:

[0035] S101: Input the image to be segmented into the first two first convolutional layers of the feature extraction network and perform convolution processing in sequence to obtain the original feature map output by the second convolutional layer of the feature extraction network.

[0036] Here, feature extraction can be performed through the feature extraction network. The feature extraction network here uses the commonly used residual network, which is a type of deep convolutional neural network. The feature extraction network here consists of five first convolutional layers connected in sequence, namely Conv1, Conv2, Conv3, Conv4 and Conv5. Among them, Conv1 is the first convolutional layer of the first layer, and Conv5 is the first convolutional layer of the last layer. The original feature map here is the original feature map X output by the second first convolutional layer (Conv2) in the feature extraction network. Among them, X∈R W×H×C , W, H, and C are the width, height, and channel dimensions of the original feature map X, respectively. Furthermore, considering that Conv1 is a very shallow network layer and the semantic information it learns is insufficient, and that after multiple layers of convolution, Conv3, Conv4, and Conv5 extract features that tend toward high-level, abstract semantics, the noise information is weakened and difficult to model, the original feature map output by Conv2 is selected. In this step, the original feature map output by Conv2 produces a more accurate portrait segmentation image compared to the outputs of other layers.

[0037] S102 , performing noise separation on the original feature map to obtain an initial noise map, and reconstructing the initial noise map to obtain a noise distribution map.

[0038] Specifically, the noise separation model includes multiple sequentially connected first convolution modules and a second convolution layer. The original feature map is input into the multiple sequentially connected first convolution modules of the noise separation model for noise separation, obtaining an initial noise map output by the last first convolution module. After the initial noise map is border-padded, it is input into the second convolution layer for convolution processing, obtaining a noise distribution map output by the second convolution layer. The width, height, and channel dimensions of the noise distribution map are the same as those of the original feature map.

[0039] Each first convolution module consists of a second convolution layer, a batch normalization layer, and an activation layer connected in series. For each first convolution module, the original feature map input to the first convolution module is subjected to noise separation in the following manner:

[0040] The second convolutional layer convolves the original feature map and outputs the convolution result to the batch normalization layer. The batch normalization layer normalizes the convolution result and outputs the normalized result to the activation layer. The activation layer sets all values less than zero in the normalized result to zero, obtaining the initial noise map output by the activation layer.

[0041] like Figure 4 As shown, Figure 4Schematic diagram of the structure of a noise separation model provided in an embodiment of the present application. Specifically, the original feature map is input into the noise separation model to obtain a noise distribution map, where the width, height and channel dimensions of the noise distribution map are the same as those of the original feature map X.

[0042] The noise separation model here is formed by multiple first convolutional modules connected in sequence. Each first convolutional module includes a second convolutional layer (Conv12) with a convolution kernel size of 3×3, a batch normalization layer (BN), and an activation layer (ReLU) connected in sequence. The noise separation model gradually separates the noise from the original feature map X through multiple layers of first convolutional modules connected in series, forming an initial noise distribution map. Preferably, when the number of first convolutional modules is five or six, the noise separation effect is better.

[0043] Specifically, the first convolution module in the noise separation model first convolves the original feature map and then normalizes the convolution result to obtain a normalized output. The normalized output is then activated, and values less than zero are set to zero, thereby obtaining the final initial noise map. This initial noise map is then input into the next first convolution module, and the above process is repeated, ultimately obtaining the final initial noise map output by the last first convolution module.

[0044] Then, after filling the boundaries of the initial noise map, the noise distribution map is input into a second convolution layer (Conv11) with a convolution kernel size of 3×3, and the noise distribution map V is obtained after convolution processing.

[0045] S103 , calculating the characteristic value of each optimized pixel point according to the characteristic value of each pixel point in the noise distribution map and the original characteristic map.

[0046] After obtaining the original feature map X and the noise distribution map V, the optimized feature image X′ is calculated using the following formula:

[0047] X ′ =max(X, XV)

[0048] Specifically, the characteristic value of each first pixel point on the original feature map is determined, and the characteristic value of each second pixel point on the noise distribution map is determined. For every two first pixel points and second pixel points with the same spatial position, the difference between the characteristic value of the first pixel point and the characteristic value of the second pixel point is used as the characteristic difference value at the spatial position. For each spatial position, the characteristic value of the first pixel point at the spatial position and the characteristic difference value at the spatial position are compared. The maximum value between the characteristic value and the characteristic difference value of the first pixel point is selected as the characteristic value of the optimized pixel point at the spatial position in the optimized feature map.

[0049] First, for each first pixel point at a spatial position in the original feature map, calculate the characteristic difference between the characteristic value of the first pixel point and the characteristic value of the second pixel point corresponding to the position of the first pixel point in the noise distribution map, that is, XV. Then, for each first pixel point, compare the characteristic value of the first pixel point with the corresponding characteristic difference value, and select the maximum value of the characteristic value of the first pixel point and the characteristic difference value at the same position as the characteristic value of the optimized pixel point. The optimized pixel point here refers to any pixel point on the optimized feature image.

[0050] S104: Generate an optimized feature image based on the feature value of each optimized pixel point.

[0051] According to the relationship between the eigenvalues of the optimized pixels and the corresponding spatial positions, an optimized feature image X′ with the same size as the original feature image is obtained.

[0052] For example, if the original feature map X is 2×2 in size and includes pixels at four positions: (1, 1), (1, 2), (2, 1), and (2, 2), the feature of the first pixel at (1, 1) in the original feature map is 4, and the second pixel at (1, 1) in the noise distribution map is -4, then the feature difference at (1, 1) is 8, and therefore the feature value of the optimized pixel at (1, 1) in the optimized feature image X′ is 8. The feature value of the optimized pixel at each position is obtained sequentially, and finally a complete optimized feature image X′ of size 2×2 is obtained.

[0053] S105 , performing portrait segmentation on the optimized feature image to obtain a portrait area and a background area.

[0054] After obtaining the optimized feature image, the optimized feature image is input into the third first convolution layer (Conv3) of the feature extraction network. After Conv4 and Conv5, the feature map output by the fifth first convolution layer (Conv5) is finally obtained. Then the feature map is input into the pedestrian segmentation network to obtain the portrait segmentation image. The pedestrian segmentation network here can be a fully convolutional network FCN. Figure 5 As shown, Figure 5 A schematic diagram of a portrait segmentation image provided in an embodiment of the present application is shown. The portrait area and background area of each portrait segmentation image are distinguished.

[0055] The image segmentation method provided in the embodiment of the present application performs feature extraction on the image to be segmented to obtain an original feature map, performs noise separation and reconstruction on the original feature map to obtain a corresponding noise distribution map, obtains an optimized feature image based on the noise distribution map of the original feature map and the original feature map, performs portrait segmentation based on the optimized feature image to output a portrait segmentation image, and recognizes the denoised image. This method can effectively avoid the influence of noise on the accuracy of portrait segmentation, thereby improving the accuracy of image segmentation.

[0056] Furthermore, the embodiment of the present application provides a portrait segmentation model, including a feature extraction network, a noise separation module and a target segmentation model. Figure 2 As shown, Figure 2 A flowchart of the training steps of a noise separation model provided in this application. Figure 2 In , the noise separation model is trained in the following way:

[0057] S201: Obtain a training dataset.

[0058] The data set includes multiple data pairs, and each data pair includes a training feature map and a standard noise distribution map training sample of the training feature map.

[0059] Specifically, irrelevant objects and background are removed from multiple training samples through manual labeling to obtain multiple denoised samples. Feature extraction is performed on the training samples and denoised samples, respectively, to obtain training feature maps and denoised feature maps. Based on the difference between the training feature map and the denoised feature map, a standard noise distribution map is determined. The training feature map and the corresponding standard noise distribution map are combined into a data pair as training data.

[0060] Here, training samples are manually labeled and segmented with masks to remove irrelevant objects and background, resulting in denoised samples. Training samples can be images collected online or captured from surveillance footage, including pedestrians. Irrelevant objects and background refer to images other than human figures. For example, images of animals are considered irrelevant objects, while images of the ground, sky, and buildings are considered background images. Denoised samples only include images of human figures; the rest of the images are blank or black.

[0061] The training samples and denoised samples are fed into the feature extraction network to be trained, respectively, to obtain the training feature map Y and denoised feature map Y' output by Conv2. The difference (Y-Y') between the training feature map Y and the denoised feature map Y' can be regarded as the noise in the training feature map, that is, the noise distribution represented by the standard noise distribution map, which is also the target value to be achieved by the noise separation model.

[0062] S202: For each data pair, input the training feature map into the noise learning model to obtain a training noise distribution map.

[0063] The training feature map Y is processed by the initial noise separation model to output a training noise distribution map W. The training noise distribution map W here represents the current estimated value of the noise distribution of the training feature image by the noise learning model.

[0064] S203: Calculate the L1 norm loss value according to the training noise distribution map and the corresponding standard noise distribution map.

[0065] Specifically, by training the L1 norm loss L between the noise distribution map and the standard noise distribution map nl As the noise learning loss to guide the learning of the noise learning model, the L1 norm loss L nl The calculation formula is as follows:

[0066] L nl =||W-(Y-Y')||1;

[0067] Among them, ||*||1 represents the L1 norm loss.

[0068] S204 , adjusting the parameters of the noise learning model according to the L1 norm loss value as the loss of the noise learning model to complete the training of the noise separation model based on a single data pair.

[0069] Repeat the above steps S202 to S204 to train the noise separation model using each data pair in the training data set. The noise separation model is trained using a large number of data pairs to obtain a trained noise separation model.

[0070] like Figure 3 As shown, Figure 3 The flowchart of the training steps of a target segmentation model provided in this application. nl After the noise separation model is trained based on , the target segmentation model can be trained in the following ways:

[0071] S301: Obtain target feature images corresponding to multiple training samples.

[0072] The target feature image here can be obtained by inputting the training sample into the first two convolutional layers of the feature extraction network. The subsequent steps are the same as those for obtaining the optimized feature image. The noise separation model used is the trained noise separation model.

[0073] S302 : For each target feature image, input the target feature image into a segmentation learning model to obtain a predicted segmentation image.

[0074] The segmentation learning model here takes as input the target feature image to be identified and outputs the output result of the target feature image.

[0075] S303: Determine the pixel cross entropy loss value of each pixel point of the target feature image of the current segmentation learning model according to the predicted segmentation image.

[0076] Specifically, first calculate the pixel cross entropy loss value L for each pixel on the predicted segmented image using the following formula: xy :

[0077]

[0078] Among them, L xy Represents the pixel cross entropy loss value of each pixel on the predicted segmentation image, p xy Represents the prediction result of the initial target segmentation model at the pixel point of the x, y coordinate position in the predicted segmentation image, g xy Indicates the category label corresponding to the pixel point at the x, y coordinate position, and j is the number of pixels.

[0079] Then based on the pixel cross entropy loss value L of all pixels of the predicted segmented image xy Calculate the average value as the image cross entropy loss value L ce .

[0080] S304 : Determine the average value of the pixel cross entropy loss values of all pixels as the image cross entropy loss value.

[0081] S305: Using the image cross entropy loss value as the loss of the segmentation learning model to adjust the parameters of the segmentation learning model to complete the training process of the segmentation learning model based on a single training sample.

[0082] Specifically, through the image cross entropy loss value L ce Guide the segmentation learning model to obtain a trained target segmentation model.

[0083] Among them, the total loss value L of the portrait segmentation model is calculated by the following formula:

[0084] L=L nl +L ce ;

[0085] The total loss value is used as the portrait segmentation learning loss to guide the learning of the portrait segmentation model, thereby obtaining a trained portrait segmentation model.

[0086] In the embodiments of the present application, the provided portrait segmentation model can adaptively learn and remove the noise in the feature map extracted by the deep convolutional neural network, thereby enhancing the expressive ability of features during feature extraction, and improving the accuracy of pedestrian segmentation through the deep convolutional neural network. It can be applied to multiple aspects such as image retrieval, target detection and pedestrian re-identification.

[0087] Based on the same inventive concept, an image segmentation device corresponding to the image segmentation method is also provided in the embodiment of the present application. Since the principle of solving the problem by the image segmentation device in the embodiment of the present application is similar to the above-mentioned image segmentation method in the embodiment of the present application, the implementation of the image segmentation device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0088] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an image segmentation device provided in an embodiment of the present application. Figure 6 As shown in , the image segmentation device 600 includes:

[0089] The feature extraction module 610 is used to input the image to be segmented into the first two first convolutional layers of the feature extraction network and perform convolution processing in sequence to obtain the original feature map output by the second convolutional layer of the feature extraction network;

[0090] The noise separation module 620 is used to perform noise separation on the original feature map to obtain an initial noise map, and reconstruct the initial noise map to obtain a noise distribution map;

[0091] A calculation module 630 is configured to calculate the characteristic value of each optimized pixel point based on the characteristic value of each pixel point in the noise distribution map and the original characteristic map;

[0092] A generating module 640 is configured to generate an optimized feature image based on the feature value of each optimized pixel point;

[0093] The segmentation module 650 is used to perform portrait segmentation on the optimized feature image to obtain a portrait area and a background area.

[0094] In a preferred embodiment, the noise separation module 620 is specifically used to: input the original feature map into multiple first convolution modules connected in sequence of the noise separation model for noise separation, and obtain an initial noise map output by the last first convolution module; wherein the noise separation model includes multiple first convolution modules connected in sequence and a second convolution layer; after filling the boundaries of the initial noise map, input the map into the second convolution layer for convolution processing, and obtain a noise distribution map output by the second convolution layer; wherein the width, height, and channel dimensions of the noise distribution map are the same as the width, height, and channel dimensions of the original feature map.

[0095] In a preferred embodiment, each first convolution module is composed of a second convolution layer, a batch normalization layer, and an activation layer connected in series. For each first convolution module, the noise separation module 620 is further configured to:

[0096] The noise separation of the original feature map input to the first convolution module is performed in the following manner: the second convolution layer convolves the original feature map and outputs the convolution result to the batch normalization layer; the batch normalization layer normalizes the convolution result and outputs the normalized result to the activation layer; the activation layer sets the values less than zero in the normalized result to zero to obtain the initial noise map output by the activation layer.

[0097] In a preferred embodiment, the calculation module 630 is specifically used to: determine the characteristic value of each first pixel point on the original feature map, and determine the characteristic value of each second pixel point on the noise distribution map; for every two first pixel points and second pixel points with the same spatial position, use the difference between the characteristic value of the first pixel point and the characteristic value of the second pixel point as the characteristic difference value at the spatial position; for each spatial position, compare the characteristic value of the first pixel point at the spatial position with the characteristic difference value at the spatial position; select the maximum value between the characteristic value and the characteristic difference value of the first pixel point as the characteristic value of the optimized pixel point at the spatial position in the optimized feature map.

[0098] In a preferred embodiment, the image segmentation device 600 also includes a training module (not shown in the figure), which is used to: obtain a training data set, the data set including multiple data pairs, each data pair including a training feature map and a standard noise distribution map of the training feature map; for each data pair, input the training feature map into the noise learning model to obtain a training noise distribution map; calculate the L1 norm loss value based on the training noise distribution map and the corresponding standard noise distribution map; and adjust the parameters of the noise learning model based on the L1 norm loss value as the loss of the noise learning model to complete the training of the noise separation model based on a single data pair.

[0099] In a preferred embodiment, the training module is also used to: remove irrelevant objects and backgrounds from multiple training samples by manual labeling to obtain multiple denoised samples; perform feature extraction on the training samples and denoised samples respectively to obtain a training feature map and a denoised feature map; determine a standard noise distribution map based on the difference between the training feature map and the denoised feature map; and form a data pair of the training feature map and the corresponding standard noise distribution map as training data.

[0100] In a preferred embodiment, the training module is also used to: obtain target feature images corresponding to multiple training samples; for each target feature image, input the target feature image into the segmentation learning model to obtain a predicted segmentation image; based on the predicted segmentation image, determine the pixel cross entropy loss value of the current segmentation learning model for each pixel point of the target feature image; determine the average of the pixel cross entropy loss values of all pixels as the image cross entropy loss value; use the image cross entropy loss value as the loss of the segmentation learning model to adjust the parameters of the segmentation learning model to complete the training process of the segmentation model based on a single training sample.

[0101] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown in FIG, electronic device 700 includes a processor 710, a memory 720, and a bus 730.

[0102] The memory 720 stores machine-readable instructions executable by the processor 710. When the electronic device 700 is running, the processor 710 and the memory 720 communicate through the bus 730. When the machine-readable instructions are executed by the processor 710, the steps of the above-mentioned image segmentation method can be executed. The specific implementation method can be found in the method embodiment and will not be repeated here.

[0103] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The specific implementation of the steps of the image segmentation method in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0104] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0106] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0107] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0108] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable memory that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0109] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An image segmentation method, characterized in that: include: Inputting the image to be segmented into the first two first convolutional layers of the feature extraction network and performing convolution processing in sequence to obtain the original feature map output by the second first convolutional layer of the feature extraction network; Noise separation is performed on the original feature map to obtain an initial noise map, and the initial noise map is reconstructed to obtain a noise distribution map, specifically comprising inputting the original feature map into a plurality of sequentially connected first convolution modules of a noise separation model for noise separation, and obtaining an initial noise map output by the last first convolution module; wherein the noise separation model comprises a plurality of sequentially connected first convolution modules and a second convolution layer; the initial noise map is border-filled and then input into the second convolution layer for convolution processing, and a noise distribution map output by the second convolution layer is obtained; wherein the width, height, and channel dimensions of the noise distribution map are the same as those of the original feature map. The width, height, and channel dimensions of the original feature map are the same. Each of the first convolution modules is composed of a second convolution layer, a batch normalization layer, and an activation layer connected in series. For each of the first convolution modules, noise separation is performed on the original feature map input into the first convolution module in the following manner: the second convolution layer performs convolution processing on the original feature map and outputs the convolution result to the batch normalization layer; the batch normalization layer normalizes the convolution result and outputs the normalized result to the activation layer; the activation layer sets the values less than zero in the normalized result to zero to obtain the initial noise map output by the activation layer; According to the characteristic values of each pixel point in the noise distribution map and the original feature map, the characteristic value of each optimized pixel point is calculated, specifically including determining the characteristic value of each first pixel point on the original feature map, and determining the characteristic value of each second pixel point on the noise distribution map; for every two first pixel points and second pixel points with the same spatial position, taking the difference between the characteristic value of the first pixel point and the characteristic value of the second pixel point as the characteristic difference value at the spatial position; for each spatial position, comparing the characteristic value of the first pixel point at the spatial position with the characteristic difference value at the spatial position; selecting the maximum value between the characteristic value of the first pixel point and the characteristic difference as the characteristic value of the optimized pixel point at the spatial position in the optimized feature map; generating an optimized feature image based on the feature value of each of the optimized pixel points; Perform portrait segmentation on the optimized feature image to obtain a portrait area and a background area.

2. The method according to claim 1, characterized in that The noise separation model is trained as follows: Acquire a training data set, wherein the data set includes a plurality of data pairs, each data pair including a training feature map and a standard noise distribution map of the training feature map; For each data pair, inputting the training feature map into the noise learning model to obtain a training noise distribution map; Calculating an L1 norm loss value according to the training noise distribution map and the corresponding standard noise distribution map; According to the L1 norm loss value as the loss of the noise learning model, the parameters of the noise learning model are adjusted to complete the training of the noise separation model based on a single data pair.

3. The method according to claim 2, characterized in that Generate the training dataset in the following way: Remove irrelevant objects and backgrounds from multiple training samples through manual labeling to obtain multiple denoised samples; Performing feature extraction on the training samples and the denoised samples respectively to obtain a training feature map and a denoised feature map; Determining a standard noise distribution map according to a difference between the training feature map and the denoising feature map; The training feature map and the corresponding standard noise distribution map are combined into a data pair as training data.

4. The method according to claim 3, characterized in that The target segmentation model is trained in the following way to perform portrait segmentation on the optimized feature image: Obtaining target feature images corresponding to the plurality of training samples; For each of the target feature images, input the target feature image into a segmentation learning model to obtain a predicted segmentation image; Determining a pixel cross entropy loss value of a current segmentation learning model for each pixel of the target feature image according to the predicted segmentation image; The average of the pixel cross entropy loss values of all pixels is determined as the image cross entropy loss value; The image cross entropy loss value is used as the loss of the segmentation learning model to adjust the parameters of the segmentation learning model to complete the training process of the target segmentation model based on the single training sample.

5. An image segmentation device, characterized in that: include: A feature extraction module is used to input the image to be segmented into the first two first convolutional layers of the feature extraction network and perform convolution processing in sequence to obtain the original feature map output by the second first convolutional layer of the feature extraction network; The noise separation module is used to perform noise separation on the original feature map to obtain an initial noise map, and reconstruct the initial noise map to obtain a noise distribution map, specifically including inputting the original feature map into a plurality of first convolution modules connected in sequence to perform noise separation, and obtaining an initial noise map output by the last first convolution module; wherein the noise separation model includes a plurality of first convolution modules connected in sequence and a second convolution layer; after border filling the initial noise map, the initial noise map is input into the second convolution layer for convolution processing, and a noise distribution map output by the second convolution layer is obtained; wherein the width, height, and channel dimensions of the noise distribution map are The width, height, and channel dimensions of the first convolution module are the same as those of the original feature map, each of the first convolution modules is composed of a second convolution layer, a batch normalization layer, and an activation layer connected in series. For each of the first convolution modules, noise separation is performed on the original feature map input into the first convolution module in the following manner: the second convolution layer performs convolution processing on the original feature map and outputs the convolution result to the batch normalization layer; the batch normalization layer normalizes the convolution result and outputs the normalized result to the activation layer; the activation layer sets the values less than zero in the normalized result to zero to obtain an initial noise map output by the activation layer; a calculation module, configured to calculate the characteristic value of each optimized pixel point based on the characteristic value of each pixel point in the noise distribution map and the original feature map, specifically including determining the characteristic value of each first pixel point on the original feature map, and determining the characteristic value of each second pixel point on the noise distribution map; for every two first pixel points and second pixel points at the same spatial position, taking the difference between the characteristic value of the first pixel point and the characteristic value of the second pixel point as the characteristic difference value at the spatial position; for each spatial position, comparing the characteristic value of the first pixel point at the spatial position with the characteristic difference value at the spatial position; and selecting the maximum value between the characteristic value of the first pixel point and the characteristic difference as the characteristic value of the optimized pixel point at the spatial position in the optimized feature map; A generating module, configured to generate an optimized feature image based on the feature value of each of the optimized pixel points; The segmentation module is used to perform portrait segmentation on the optimized feature image to obtain a portrait area and a background area.

6. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the image segmentation method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which, when executed by a processor, executes the steps of the image segmentation method according to any one of claims 1 to 4.

Images