Image processing method and device, computer device and storage medium

Abstract

The present disclosure provides an image processing method and device, computer equipment and a storage medium. The method comprises: obtaining a to-be-processed image, wherein the to-be-processed image comprises a foreground object; inputting the to-be-processed image into an image segmentation model to obtain image segmentation features output by the image segmentation model; and performing segmentation processing on the foreground object in the to-be-processed image according to the image segmentation features to obtain a target image. Thus, the degree of automation of image processing can be effectively improved based on the image segmentation model. When the image segmentation features are determined based on shared image features and feature scales of the shared image features, the computational complexity of image segmentation processing can be effectively reduced, and the image processing effect can be effectively improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to an image processing method, apparatus, computer device, and storage medium. Background Technology

[0002] Image processing is a common technology in daily life. Mobile phone photography makes it easier for users to acquire images, leading to its widespread application. For example, to highlight a person in an image, the subject can be extracted and the background blurred. Adding effects to a person in an image requires determining their position. Furthermore, during video calls or conferences, if you don't want the other party to see your location, you need to extract the person and replace the background.

[0003] Among related technologies, image processing has a low degree of automation, relies on the user's image processing experience, involves a large amount of computation, and cannot guarantee image processing quality. Summary of the Invention

[0004] This disclosure aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, the purpose of this disclosure is to propose an image processing method, apparatus, computer device, and storage medium, which can effectively improve the automation level of image processing based on an image segmentation model. When image segmentation features are determined based on shared image features and the feature scale of shared image features, the computational load of image segmentation processing can be effectively reduced, and the image processing effect can be effectively improved.

[0006] The image processing method proposed in the first aspect of this disclosure includes: acquiring an image to be processed, wherein the image to be processed includes a foreground object; inputting the image to be processed into an image segmentation model to obtain image segmentation features output by the image segmentation model; wherein the image segmentation features are reference features for segmenting the foreground object from the image to be processed, the image segmentation features are based on shared image features shared by the semantic dimension and edge dimension of the foreground object, and the feature scale of the shared image features is determined; and segmenting the foreground object in the image to be processed according to the image segmentation features to obtain a target image.

[0007] The image processing method proposed in the first aspect of this disclosure can effectively improve the automation level of image processing based on the image segmentation model. When the image segmentation features are determined based on shared image features and the feature scale of shared image features, the computational load of image segmentation processing can be effectively reduced, thereby effectively improving the image processing effect.

[0008] The method for generating an image segmentation model according to a second aspect of this disclosure includes: acquiring a sample image and labeled segmentation features corresponding to the sample image, wherein the sample image includes a sample foreground object, and the labeled segmentation features are labeled reference features for segmenting the sample foreground object from the sample image; acquiring an initial neural network model; inputting the sample image into the initial neural network model to obtain predicted segmentation features, wherein the predicted segmentation features are predicted reference features for segmenting the sample foreground object from the sample image; calculating a loss value between the predicted segmentation features and the labeled segmentation features; training the initial neural network model based on the loss value until the initial neural network model converges, and using the trained neural network model as an image segmentation model.

[0009] The image segmentation model generation method proposed in the second aspect of this disclosure can evaluate the performance of the initial neural network model based on the loss value during the model training process, so as to accurately determine the convergence time of the model and thus effectively improve the reliability of the obtained image segmentation model.

[0010] The image processing apparatus proposed in the third aspect embodiment of this disclosure includes:

[0011] A first acquisition module is used to acquire an image to be processed, wherein the image to be processed includes a foreground object; a first processing module is used to input the image to be processed into an image segmentation model to obtain image segmentation features output by the image segmentation model; wherein the image segmentation features are reference features for segmenting the foreground object from the image to be processed, and the image segmentation features are determined based on shared image features shared by the semantic dimension and edge dimension of the foreground object, and the feature scale of the shared image features; a second processing module is used to segment the foreground object in the image to be processed according to the image segmentation features to obtain a target image.

[0012] The image processing apparatus proposed in the third aspect of this disclosure can effectively improve the automation level of image processing based on an image segmentation model. When the image segmentation features are determined based on shared image features and the feature scale of shared image features, the computational load of image segmentation processing can be effectively reduced, thereby effectively improving the image processing effect.

[0013] The image segmentation model generation apparatus proposed in the fourth aspect embodiment of this disclosure includes: a second acquisition module for acquiring a sample image and labeled segmentation features corresponding to the sample image, wherein the sample image includes a sample foreground object, and the labeled segmentation features are labeled reference features for segmenting the sample foreground object from the sample image; a third acquisition module for acquiring an initial neural network model; a third processing module for inputting the sample image into the initial neural network model to obtain predicted segmentation features, wherein the predicted segmentation features are predicted reference features for segmenting the sample foreground object from the sample image; a calculation module for calculating a loss value between the predicted segmentation features and the labeled segmentation features; and a training module for training the initial neural network model based on the loss value until the initial neural network model converges, and using the trained neural network model as an image segmentation model.

[0014] The image segmentation model generation apparatus proposed in the fourth aspect of this disclosure can evaluate the performance of the initial neural network model based on the loss value during the model training process, so as to accurately determine the convergence time of the model and thus effectively improve the reliability of the obtained image segmentation model.

[0015] The computer device proposed in the fifth aspect of this disclosure includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the image processing method proposed in the first aspect of this disclosure, or implements the image segmentation model generation method proposed in the second aspect of this disclosure.

[0016] A sixth aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method as proposed in the first aspect of this disclosure, or implements the image segmentation model generation method as proposed in the second aspect of this disclosure.

[0017] A seventh aspect of this disclosure provides a computer program product in which, when instructions in the computer program product are executed by a processor, an image processing method as proposed in a first aspect of this disclosure is performed, or an image segmentation model generation method as proposed in a second aspect of this disclosure is performed.

[0018] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0020] Figure 1 This is a schematic flowchart of an image processing method proposed in an embodiment of the present disclosure;

[0021] Figure 2 This is a schematic diagram of an image segmentation model structure proposed in an embodiment of this disclosure;

[0022] Figure 3 This is a schematic flowchart of an image processing method proposed in another embodiment of this disclosure;

[0023] Figure 4 This is a schematic flowchart of an image processing method proposed in another embodiment of this disclosure;

[0024] Figure 5 This is an image processing flowchart proposed in an embodiment of the present disclosure;

[0025] Figure 6 This is a schematic diagram of an image processing effect proposed in an embodiment of this disclosure;

[0026] Figure 7 This is a schematic flowchart of a method for generating an image segmentation model according to an embodiment of this disclosure;

[0027] Figure 8 This is a schematic diagram illustrating the usage process of an image segmentation model proposed in an embodiment of this disclosure;

[0028] Figure 9 This is a schematic flowchart of a method for generating an image segmentation model according to an embodiment of this disclosure;

[0029] Figure 10 This is a schematic diagram of the training process of an image segmentation model proposed in an embodiment of this disclosure;

[0030] Figure 11 This is a schematic diagram of the structure of an image processing apparatus according to an embodiment of the present disclosure;

[0031] Figure 12 This is a schematic diagram of the structure of an image processing apparatus according to another embodiment of the present disclosure;

[0032] Figure 13 This is a schematic diagram of the structure of an image segmentation model generation device proposed in an embodiment of the present disclosure;

[0033] Figure 14 This is a schematic diagram of the structure of an image segmentation model generation device proposed in another embodiment of this disclosure;

[0034] Figure 15 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0035] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are used only to explain this disclosure, and should not be construed as limiting this disclosure. Rather, embodiments of this disclosure include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0036] Figure 1 This is a schematic flowchart of an image processing method proposed in an embodiment of the present disclosure.

[0037] It should be noted that the execution subject of the image processing method in this embodiment is an image processing device, which can be implemented by software and / or hardware. The device can be configured in a computer device, which may include, but is not limited to, a terminal, a server, etc., such as a mobile phone, a PDA, etc.

[0038] like Figure 1 As shown, the image processing method includes:

[0039] S101: Obtain the image to be processed, wherein the image to be processed includes a foreground object.

[0040] The image to be processed refers to the image to be processed. This image can be a color image or a grayscale image, without any restrictions.

[0041] The foreground object can refer to the main subject in the image to be processed. For example, it could be a person in a portrait image, or a car in a car image.

[0042] In this embodiment of the disclosure, when acquiring the image to be processed, an image acquisition device (such as a camera) may be pre-configured in the execution subject of this embodiment of the disclosure, and the image to be processed may be acquired by the image acquisition device. For example, a camera device may be configured in the terminal device, and the image to be processed may be acquired by the camera device. Alternatively, a data interface may be pre-configured for the image processing device, and an image processing request may be received through the data interface, and then the image to be processed may be parsed from the image processing request. There are no restrictions on this.

[0043] It is understood that the image to be processed may include one or more foreground objects, which can be people, animals or other objects, without restriction.

[0044] S102: Input the image to be processed into the image segmentation model to obtain the image segmentation features output by the image segmentation model; wherein, the image segmentation model is used to extract the image segmentation features of the image to be processed, and the image segmentation features are reference features for segmenting the foreground objects from the image to be processed; the image segmentation features are determined based on the shared image features of the image to be processed that are shared by the semantic dimension and edge dimension of the foreground objects, and the feature scale of the shared image features.

[0045] Among them, the image segmentation model refers to the model used to extract the image segmentation features corresponding to the image to be processed. The image segmentation model can be a neural network model, a machine learning model, or a graph neural network model. Of course, any other possible model capable of performing image processing tasks can also be used without restriction.

[0046] Image segmentation features refer to image features in the image to be processed that can be used to indicate the image segmentation process, such as the contour features and area features of the foreground object.

[0047] Among these, reference features can be used as a reference to indicate the process of segmenting foreground objects in the image to be processed. Semantics refers to the content represented by the image; for example, the semantics of an image of a puppy can be "puppy". Edges refer to the image edges of foreground objects in the image to be processed.

[0048] Shared image features refer to the image features obtained by processing the image to be processed based on a shared feature extraction network. These shared image features can include image features shared by the semantic and edge dimensions of the foreground object. For example, they can include color features, texture features, shape features, and spatial relationship features related to the foreground object in the image to be processed, without any restrictions.

[0049] Understandably, in the field of image processing, feature scales can map shared image features to one or more scales in scale space to describe the size of the feature. For example, there may be multiple feature scales, and shared image features may include image features with different granularities based on each feature scale.

[0050] The feature scale refers to the granularity of the shared image features. For example, if the granularity of the image to be processed is 1, then the feature size of the shared image features can be 1 or 0.5, etc.

[0051] Optionally, in some embodiments, the image segmentation model includes: a shared feature extraction sub-model, a semantic extraction sub-model, an edge detail extraction sub-model, and a fusion sub-model. The image segmentation model is obtained by jointly training the shared feature extraction sub-model, the semantic extraction sub-model, the edge detail extraction sub-model, and the fusion sub-model. This can effectively improve the structural rationality of the image segmentation model. Jointly training multiple sub-models can ensure the coordination between multiple sub-models, thereby effectively improving the reliability of the trained image segmentation model.

[0052] The model comprises: a shared feature extraction sub-model, used to extract shared image features from the image to be processed; a semantic extraction sub-model, used to process the shared image features to obtain the initial semantic segmentation result of the image to be processed, wherein the initial semantic segmentation result describes the semantic dimension image features of the image to be processed; an edge detail extraction sub-model, used to process the shared image features to obtain the edge image information of the foreground object corresponding to the image to be processed; and a fusion sub-model, used to process the feature scale of the shared image features, the initial semantic segmentation result, and the edge image information to obtain the image segmentation features.

[0053] For example, such as Figure 2 As shown, Figure 2 This is a schematic diagram of an image segmentation model structure proposed in this embodiment. Assuming the image to be processed has a scale of H*W*3, after the image is input into the image segmentation model, it can be processed by a shared feature extraction sub-model to obtain corresponding image features. The obtained image features can be semantically segmented by a semantic extraction sub-model, and the corresponding semantic output can be output at a scale of H / 2*W / 2*2. Of course, the structure of the semantic extraction sub-model can be any general semantic segmentation head, and its output scale can be the original scale, or half the original scale, or a smaller scale. The obtained image features can be predicted for edge details by an edge detail extraction sub-model, and the corresponding edge detail information can be output at a scale of H*W*1. Of course, this edge detail extraction sub-model can also be replaced by any general semantic segmentation head. The fusion sub-model can transform the output of the semantic extraction sub-model to obtain attention at different scales, thereby performing feature fusion at different feature scales. By fusing the image features at different feature scales output by the fusion sub-model and the edge detail extraction sub-model, the corresponding segmentation result, i.e., the image segmentation features, is obtained.

[0054] In this embodiment of the disclosure, the image to be processed is input into the image segmentation model to obtain the image segmentation features output by the image segmentation model, which can effectively improve the image processing efficiency based on the image segmentation model.

[0055] S103: Based on the image segmentation features, segment the foreground objects in the image to be processed to obtain the target image.

[0056] The target image refers to the image obtained by processing the image to be processed based on image segmentation features. This target image can be an image of a person, animal, or other object within the image to be processed.

[0057] In this embodiment, an image to be processed is acquired, which includes a foreground object. The image to be processed is input into an image segmentation model to obtain image segmentation features output by the image segmentation model. The image segmentation model is used to extract image segmentation features from the image to be processed. The image segmentation features are reference features for segmenting the foreground object from the image to be processed. The image segmentation features are determined based on the shared image features of the image to be processed, which are shared by the semantic and edge dimensions of the foreground object, and the feature scale of the shared image features. The foreground object in the image to be processed is segmented according to the image segmentation features to obtain the target image. Thus, the automation level of image processing can be effectively improved based on the image segmentation model. When the image segmentation features are determined based on the shared image features and the feature scale of the shared image features, the computational load of image segmentation processing can be effectively reduced, and the image processing effect can be effectively improved.

[0058] Figure 3 This is a schematic flowchart of an image processing method proposed in another embodiment of this disclosure.

[0059] like Figure 3 As shown, the image processing method includes:

[0060] S301: Obtain the image to be processed, wherein the image to be processed includes a foreground object.

[0061] For a detailed description of S301, please refer to the above embodiments, which will not be repeated here.

[0062] S302: Input the image to be processed into the shared feature extraction sub-model to obtain the shared image features of the image to be processed, and determine the feature scale of the shared image features.

[0063] In this embodiment of the disclosure, based on the shared feature extraction sub-model, the corresponding image features of the image to be processed can be extracted quickly, thereby reducing the amount of computation and power consumption costs in the feature extraction process. By determining the feature scale of the shared image features, a reliable execution basis can be provided for subsequent feature fusion of image features at different feature scales.

[0064] S303: Input the shared image features into the semantic extraction sub-model to obtain the initial semantic segmentation result of the image to be processed.

[0065] The semantic segmentation result refers to the pixel-level segmentation of objects in an image. This result can include object classification information for each pixel; parts that do not belong to any of the supported object classes can be labeled as "background." The initial semantic segmentation result, on the other hand, refers to the semantic segmentation result obtained by processing shared image features through a semantic extraction sub-model.

[0066] Optionally, in some embodiments, the image to be processed includes a foreground sub-image and a background sub-image. The initial semantic segmentation result includes a foreground semantic segmentation result of the foreground sub-image and a background semantic segmentation result of the background sub-image. The output scale of the foreground semantic segmentation result is smaller than the scale of the image to be processed, and the output scale of the background semantic segmentation result is smaller than the scale of the image to be processed. This can effectively improve the semantic representation effect of the obtained initial semantic segmentation result on the foreground sub-image and the background sub-image, so as to accurately distinguish the foreground sub-image and the background sub-image based on the foreground semantic segmentation result and the background semantic segmentation result.

[0067] In this context, the foreground sub-image refers to the main subject image in the image to be processed, such as a portion of the image corresponding to the person in a person image. The background sub-image refers to the portion of the image to be processed excluding the foreground sub-image. The foreground semantic segmentation result can be the semantic segmentation result obtained by processing the foreground sub-image using a semantic extraction sub-model. Similarly, the background semantic segmentation result can be the semantic segmentation result obtained by processing the background sub-image using a semantic extraction sub-model.

[0068] It is understandable that the foreground sub-image and the background sub-image are components of the image to be processed. Therefore, the output scale of the foreground semantic segmentation result is smaller than the scale of the image to be processed, and the output scale of the background semantic segmentation result is smaller than the scale of the image to be processed.

[0069] S304: Input the shared image features into the edge detail extraction sub-model to obtain the edge image information of the foreground object in the image to be processed.

[0070] Among them, edge image information refers to the relevant information of the edges of the foreground objects in the image to be processed, such as the coordinate information of the edges of the foreground objects in the corresponding coordinate space of the image to be processed.

[0071] Optionally, in some embodiments, the image to be processed includes: multiple edge pixels of a foreground object, wherein the edge image information includes: the confidence level of each edge pixel belonging to the foreground object. Thus, the reliability of each edge pixel belonging to the foreground object can be quantified based on the confidence level, so as to effectively improve the representation accuracy of the edge image information, thereby predicting the edge of the foreground object based on the confidence level of each edge pixel belonging to the foreground object.

[0072] Edge pixels refer to pixels located at the edges of foreground objects in the image to be processed.

[0073] Confidence level can be used to describe the degree to which an event is reliable as true.

[0074] S305: Input the feature scale, initial semantic segmentation results, and edge image information into the fusion sub-model to obtain image segmentation features.

[0075] Optionally, in some embodiments, the image to be processed includes: multiple image pixels, wherein the image segmentation feature includes: the confidence level of each image pixel belonging to the foreground object. Thus, the probability of multiple image pixels in the image to be processed belonging to the foreground object can be accurately quantified based on the confidence level, thereby effectively improving the indicative effect of the image segmentation feature in the segmentation process of foreground sub-images and background sub-images.

[0076] Here, an image pixel refers to a pixel in the image to be processed. This image pixel may include the edge pixels mentioned above.

[0077] In other words, after acquiring the image to be processed, the embodiments of this disclosure can input the image to be processed into a shared feature extraction sub-model to obtain shared image features of the image to be processed and determine the feature scale of the shared image features. The shared image features are then input into a semantic extraction sub-model to obtain the initial semantic segmentation result of the image to be processed. The shared image features are then input into an edge detail extraction sub-model to obtain the edge image information of the image to be processed. Finally, the feature scale, the initial semantic segmentation result, and the edge image information are input into a fusion sub-model to obtain image segmentation features. Thus, feature extraction can be performed quickly and accurately on the image to be processed based on multiple sub-models, thereby effectively improving the efficiency of image segmentation feature acquisition and the accuracy of representation.

[0078] S306: Segment the foreground objects in the image to be processed according to the image segmentation features to obtain the target image.

[0079] For a detailed description of S306, please refer to the above embodiments, which will not be repeated here.

[0080] In this embodiment, the shared image features of the image to be processed are obtained by inputting the image to be processed into the shared feature extraction sub-model, and the feature scale of the shared image features is determined. The shared image features are then input into the semantic extraction sub-model to obtain the initial semantic segmentation result of the image to be processed. The shared image features are then input into the edge detail extraction sub-model to obtain the edge image information of the foreground object in the image to be processed. Finally, the feature scale, the initial semantic segmentation result, and the edge image information are input into the fusion sub-model to obtain the image segmentation features. Thus, feature extraction can be performed quickly and accurately on the image to be processed based on multiple sub-models, thereby effectively improving the efficiency of image segmentation feature acquisition and the accuracy of representation.

[0081] Figure 4 This is a schematic flowchart of an image processing method proposed in another embodiment of this disclosure.

[0082] like Figure 4 As shown, the image processing method includes:

[0083] S401: Obtain the image to be processed, wherein the image to be processed includes a foreground object.

[0084] S402: Input the image to be processed into the shared feature extraction sub-model to obtain the shared image features of the image to be processed, and determine the feature scale of the shared image features.

[0085] S403: Input the shared image features into the semantic extraction sub-model to obtain the initial semantic segmentation result of the image to be processed.

[0086] S404: Input the shared image features into the edge detail extraction sub-model to obtain the edge image information of the foreground object in the image to be processed.

[0087] The descriptions of S401-S404 can be found in the above embodiments, and will not be repeated here.

[0088] S405: Transform the initial semantic segmentation result based on each feature scale to obtain the target semantic segmentation result.

[0089] The target semantic segmentation result refers to the semantic segmentation result obtained by transforming the corresponding initial semantic segmentation result based on the feature scale.

[0090] In this embodiment of the disclosure, when the initial semantic segmentation result is transformed based on each feature scale to obtain the target semantic segmentation result, the obtained target semantic segmentation result can provide a reliable execution basis for the process of obtaining fused image features.

[0091] S406: The semantic segmentation result of each target is fused with the edge image information of the corresponding feature scale to obtain the fused image features of the corresponding feature scale.

[0092] Among them, fused image features refer to the image features obtained by fusing the target semantic segmentation results with the edge image information of the corresponding feature scale.

[0093] In this embodiment of the disclosure, when the semantic segmentation result of each target is fused with the edge image information of the corresponding feature scale to obtain the fused image features of the corresponding feature scale, reliable reference data can be provided for subsequent fusion processing of fused image features of different feature scales.

[0094] S407: Input the fused image features at multiple feature scales into the fusion sub-model to obtain image segmentation features.

[0095] In other words, in this embodiment of the present disclosure, after inputting shared image features into the edge detail extraction sub-model to obtain edge image information of the image to be processed, the initial semantic segmentation result can be transformed based on each feature scale to obtain the target semantic segmentation result. Each target semantic segmentation result is then fused with the edge image information of the corresponding feature scale to obtain the fused image features of the corresponding feature scale. The fused image features of multiple feature scales are then input into the fusion sub-model to obtain the image segmentation features. Thus, the target semantic segmentation results and edge image information at different feature scales can be effectively fused in the process of determining image segmentation features. This can effectively avoid the mismatch between the target semantic segmentation results and edge image information, which would affect the image segmentation feature extraction effect, thereby effectively improving the robustness of the image segmentation feature acquisition process.

[0096] S408: Process the image to be processed based on the image segmentation features to obtain the target image.

[0097] For a detailed description of S408, please refer to the above embodiments, which will not be repeated here.

[0098] For example, such as Figure 5 As shown, Figure 5This is an image processing flowchart proposed in an embodiment of the present disclosure. After obtaining an image to be processed through a mobile phone, camera, or other input device, or by other means, the executing entity of this embodiment can process the image to be processed through a shared feature extraction sub-model to obtain shared image features of the image to be processed and determine the feature scale of the shared image features. Then, the shared image features are respectively input into the semantic extraction sub-model and the edge detail extraction sub-model to obtain the initial semantic segmentation result and edge image information. Then, the fusion sub-model processes the initial semantic segmentation result and edge image information to generate a foreground mask corresponding to the image to be processed. Based on the foreground mask, the background sub-image is blurred or replaced to obtain the target image.

[0099] For example, assuming the scale of the image to be processed is 768*768, method A is an image processing method that completes end-to-end matting by learning two sub-tasks in a collaborative manner based on a shared encoder and two separate decoders. Method B is an image processing method based on a target decomposition network. Method C is an image processing method proposed in the embodiments of this disclosure.

[0100] When the above three methods are used to process the image, the number of parameters for methods A, B, and C during image processing are 55.29M, 6.49M, and 3.48M, respectively, and the corresponding computational loads are 165.11 giga-point operations per second (GFLOPS), 38.97 GFLOPS, and 24.15 GFLOPS, respectively. Therefore, compared with methods A and B, the proposed scheme C in this embodiment has a relatively lower number of parameters and computational load, which can effectively reduce the power consumption cost in the image processing process.

[0101] For example, such as Figure 6 As shown, Figure 6 This is a schematic diagram of an image processing effect proposed in an embodiment of the present disclosure. The first column is the image to be processed, the second column is the target image obtained by processing the corresponding image to be processed based on the image processing method in the embodiment of the present disclosure, and the third column is the target image obtained by processing the corresponding image to be processed using the above-mentioned method A.

[0102] Understandable Figure 6 The semantic segmentation results and edge image information corresponding to the two images to be processed in the first method may not match. As a result, holes appear in the third column image obtained by method A, leading to poor image processing results. However, the target image obtained by processing the corresponding images based on the image processing method in this embodiment does not exhibit holes at the corresponding positions.

[0103] In this embodiment, the initial semantic segmentation result is transformed based on each feature scale to obtain the target semantic segmentation result. The target semantic segmentation result is then fused with the edge image information of the corresponding feature scale to obtain the fused image features of the corresponding feature scale. The fused image features of multiple feature scales are input into the fusion sub-model to obtain the image segmentation features. Thus, the target semantic segmentation results and edge image information at different feature scales can be effectively fused in the process of determining the image segmentation features. This can effectively avoid the mismatch between the target semantic segmentation results and edge image information, which would affect the image segmentation feature extraction effect, thereby effectively improving the robustness of the image segmentation feature acquisition process.

[0104] Figure 7 This is a schematic flowchart illustrating a method for generating an image segmentation model according to an embodiment of this disclosure.

[0105] It should be noted that the execution subject of the image segmentation model generation method in this embodiment is the image segmentation model generation device. This device can be implemented by software and / or hardware. This device can be configured in a computer device, which may include, but is not limited to, a terminal, a server, etc. For example, the terminal may be a mobile phone, a PDA, etc.

[0106] like Figure 7 As shown, the method for generating this image segmentation model includes:

[0107] S701: Obtain a sample image and corresponding labeled segmentation features, wherein the sample image includes a sample foreground object, and the labeled segmentation features are the labeled reference features for segmenting the sample foreground object from the sample image.

[0108] Here, "sample image" refers to the image used in the training process of the corresponding image segmentation model. "Sample foreground object" refers to the foreground object corresponding to the sample image. "Annotated segmentation features" refers to the image segmentation features obtained by pre-annotating the sample images.

[0109] In this embodiment of the disclosure, when a sample image and the corresponding labeled segmentation features are obtained, the obtained sample image can provide reliable sample data for the model training process, while the labeled segmentation features corresponding to the sample image can be analyzed and compared with the predicted segmentation features to determine the model performance.

[0110] S702: Obtain the initial neural network model.

[0111] The initial neural network model refers to the neural network model to be trained.

[0112] In this embodiment of the disclosure, the initial neural network model can be a feedforward neural network model, a convolutional neural network model, a recurrent neural network model, etc., and there is no limitation thereto.

[0113] S703: Input the sample image into the initial neural network model to obtain the predicted segmentation features, where the predicted segmentation features are the predicted reference features for segmenting the foreground objects of the sample image.

[0114] Among them, the predicted segmentation features refer to the image segmentation features obtained by predicting the sample image through the initial neural network model.

[0115] S704: Calculate the loss value between the predicted segmentation features and the labeled segmentation features.

[0116] The loss value refers to the numerical value obtained by quantifying the error between the predicted segmentation features and the labeled segmentation features.

[0117] In this embodiment of the disclosure, by calculating the loss value between the predicted segmentation features and the labeled segmentation features, the performance of the initial neural network model can be evaluated in real time.

[0118] S705: Train the initial neural network model based on the loss value until the initial neural network model converges, and use the trained neural network model as the image segmentation model.

[0119] Understandably, the trained image segmentation model has an output terminal during use, which can output the image segmentation features corresponding to the image to be processed. For example... Figure 8 As shown, Figure 8 This is a schematic diagram illustrating the usage flow of an image segmentation model proposed in this embodiment. The input image to be processed can be an RGB three-channel color image, and the output can be a single-channel grayscale image (0-255). The grayscale value represents the confidence that a pixel belongs to the foreground. Removing the output terminals of the semantic extraction sub-model and the edge detail extraction sub-model during use effectively reduces computational load and power consumption.

[0120] In this embodiment, an initial neural network model is obtained by acquiring sample images and corresponding labeled segmentation features. The sample images are then input into the initial neural network model to obtain predicted segmentation features. The loss value between the predicted segmentation features and the labeled segmentation features is calculated. The initial neural network model is trained based on the loss value until it converges. The trained neural network model is then used as the image segmentation model. Thus, the performance of the initial neural network model can be evaluated based on the loss value during the model training process to accurately determine the convergence time of the model, thereby effectively improving the reliability of the obtained image segmentation model.

[0121] Figure 9 This is a schematic flowchart of a method for generating an image segmentation model according to another embodiment of this disclosure.

[0122] like Figure 9 As shown, the method for generating this image segmentation model includes:

[0123] S901: Obtain a sample image and corresponding labeled segmentation features, wherein the sample image includes a sample foreground object, and the labeled segmentation features are the labeled reference features for segmenting the sample foreground object from the sample image.

[0124] S902: Obtain the initial neural network model.

[0125] For a detailed description of S901-S902, please refer to the above embodiments, which will not be repeated here.

[0126] S903: Input the sample image into the initial shared feature extraction sub-model to obtain the sample shared image features of the sample image and determine the sample feature scale of the sample shared image features.

[0127] The initial shared feature extraction sub-model refers to the untrained shared feature extraction sub-model in the initial neural network model.

[0128] Here, "sample shared image features" refers to the shared image features obtained by processing sample images based on the initial shared feature extraction sub-model. "Sample feature scale" refers to the feature scale corresponding to the sample shared image features.

[0129] S904: Input the shared image features of the samples into the initial semantic extraction sub-model to obtain the sample semantic segmentation result of the sample image, wherein the sample semantic segmentation result describes the semantic dimension image features of the sample image.

[0130] The initial semantic extraction sub-model refers to the untrained semantic extraction sub-model in the initial neural network model.

[0131] Among them, the sample semantic segmentation result refers to the semantic segmentation result obtained by processing the shared image features of the samples through the initial semantic extraction sub-model.

[0132] S905: Input the shared image features of the samples into the initial edge detail extraction sub-model to obtain the edge image information of the sample foreground object corresponding to the sample image.

[0133] The initial edge detail extraction sub-model refers to the untrained edge detail extraction sub-model in the initial neural network model.

[0134] S906: Input the sample feature scale, sample semantic segmentation results, and edge image information into the initial fusion sub-model to obtain the predicted segmentation features.

[0135] The initial fusion sub-model refers to the untrained fusion sub-model in the initial neural network model.

[0136] In other words, in this embodiment of the disclosure, the initial neural network model includes: an initial shared feature extraction sub-model, an initial semantic extraction sub-model, an initial edge detail extraction sub-model, and an initial fusion sub-model. After obtaining the initial neural network model, the sample image can be input into the initial shared feature extraction sub-model to obtain the sample shared image features of the sample image and determine the sample feature scale of the sample shared image features. The sample shared image features are then input into the initial semantic extraction sub-model to obtain the sample semantic segmentation result of the sample image. The sample semantic segmentation result describes the semantic dimension image features of the sample image. The sample shared image features are then input into the initial edge detail extraction sub-model to obtain the edge image information of the sample foreground object corresponding to the sample image. The sample feature scale, the sample semantic segmentation result, and the edge image information are then input into the initial fusion sub-model to obtain the predicted segmentation features. Thus, the orderly execution of the predicted segmentation feature extraction process can be ensured based on the various sub-models in the initial neural network model, thereby effectively improving the model training effect.

[0137] S907: Calculate the loss value between the predicted segmentation features and the labeled segmentation features.

[0138] S908: Train the initial neural network model based on the loss value until the initial neural network model converges, and use the trained neural network model as the image segmentation model.

[0139] For details on S907-S908, please refer to the above embodiments; they will not be repeated here.

[0140] For example, such as Figure 10 As shown, Figure 10 This diagram illustrates the training process of an image segmentation model proposed in this embodiment. The training process can be performed using a multi-task approach. During model training, the initial neural network model can have three outputs: semantic output, edge detail output, and predicted segmentation feature output. The semantic output and edge detail output can serve as relay supervision to improve the output performance of the initial semantic extraction sub-model and the initial edge detail extraction sub-model, respectively.

[0141] The semantic output can be a two-channel output at a 1 / 2 output scale, representing the foreground and background respectively, constrained by the cross-entropy loss function. The edge detail output is a single-channel output representing the confidence that an edge belongs to the foreground, constrained by minimum absolute bias, gradient loss, and Laplacian loss. The predicted segmentation feature output is a single-channel (0-255) grayscale image, where the grayscale value represents the confidence that the channel belongs to the foreground, constrained by minimum absolute bias, gradient loss, Laplacian loss, and foreground / background fusion function.

[0142] In this embodiment, by inputting the sample image into the initial shared feature extraction sub-model, the sample shared image features of the sample image are obtained, and the sample feature scale of the sample shared image features is determined. The sample shared image features are then input into the initial semantic extraction sub-model to obtain the sample semantic segmentation result of the sample image. The sample semantic segmentation result describes the semantic dimension image features of the sample image. The sample shared image features are then input into the initial edge detail extraction sub-model to obtain the edge image information of the sample foreground object in the sample image. The sample feature scale, the sample semantic segmentation result, and the edge image information are then input into the initial fusion sub-model to obtain the predicted segmentation features. Thus, the orderly execution of the predicted segmentation feature extraction process can be ensured based on the various sub-models in the initial neural network model, thereby effectively improving the model training effect.

[0143] Figure 11 This is a schematic diagram of the structure of an image processing apparatus according to an embodiment of the present disclosure.

[0144] like Figure 11 As shown, the image processing apparatus 110 includes:

[0145] The first acquisition module 1101 is used to acquire the image to be processed, wherein the image to be processed includes a foreground object;

[0146] The first processing module 1102 is used to input the image to be processed into the image segmentation model to obtain the image segmentation features output by the image segmentation model; wherein, the image segmentation model is used to extract the image segmentation features of the image to be processed, the image segmentation features are reference features for segmenting the foreground object from the image to be processed, and the image segmentation features are determined based on the shared image features shared by the semantic dimension and edge dimension of the foreground object and the feature scale of the shared image features;

[0147] The second processing module 1103 is used to segment the foreground object in the image to be processed according to the image segmentation features to obtain the target image.

[0148] In some embodiments of this disclosure, such as Figure 12 As shown, Figure 12 This is a schematic diagram of the structure of an image processing apparatus according to another embodiment of the present disclosure, wherein the image segmentation model includes: a shared feature extraction sub-model, a semantic extraction sub-model, an edge detail extraction sub-model, and a fusion sub-model; wherein the image segmentation model is obtained by jointly training the shared feature extraction sub-model, the semantic extraction sub-model, the edge detail extraction sub-model, and the fusion sub-model; wherein,

[0149] A shared feature extraction sub-model is used to extract shared image features from the image to be processed;

[0150] The semantic extraction sub-model is used to process shared image features to obtain the initial semantic segmentation result of the image to be processed, wherein the initial semantic segmentation result describes the semantic dimension image features of the image to be processed;

[0151] An edge detail extraction sub-model is used to process shared image features to obtain edge image information of the foreground object in the image to be processed.

[0152] The fusion sub-model is used to process the feature scale of shared image features, the initial semantic segmentation results, and edge image information to obtain image segmentation features.

[0153] In some embodiments of this disclosure, the first processing module 1102 includes:

[0154] The first processing submodule 11021 is used to input the image to be processed into the shared feature extraction submodel, obtain the shared image features of the image to be processed, and determine the feature scale of the shared image features;

[0155] The second processing submodule 11022 is used to input the shared image features into the semantic extraction submodel to obtain the initial semantic segmentation result of the image to be processed;

[0156] The third processing submodule 11023 is used to input the shared image features into the edge detail extraction submodel to obtain the edge image information of the foreground object in the image to be processed;

[0157] The fourth processing submodule 11024 is used to input the feature scale, the initial semantic segmentation result, and the edge image information into the fusion submodel to obtain image segmentation features.

[0158] In some embodiments of this disclosure, there are multiple types of feature scales, and the first processing module 1102 further includes:

[0159] The fifth processing submodule 11025 is used to transform the initial semantic segmentation result based on each feature scale to obtain the target semantic segmentation result;

[0160] The sixth processing submodule 11026 is used to fuse the semantic segmentation result of each target with the edge image information of the corresponding feature scale to obtain the fused image features of the corresponding feature scale.

[0161] The fourth processing submodule 11024 is specifically used for:

[0162] The fused image features at multiple feature scales are input into the fusion sub-model to obtain image segmentation features.

[0163] In some embodiments of this disclosure, the image to be processed includes a foreground sub-image and a background sub-image, wherein the initial semantic segmentation result includes a foreground semantic segmentation result of the foreground sub-image and a background semantic segmentation result of the background sub-image, wherein the output scale of the foreground semantic segmentation result is smaller than the scale of the image to be processed, and the output scale of the background semantic segmentation result is smaller than the scale of the image to be processed.

[0164] In some embodiments of this disclosure, the image to be processed includes: multiple edge pixels of a foreground object, wherein the edge image information includes: the confidence level of each edge pixel belonging to the foreground object.

[0165] In some embodiments of this disclosure, the image to be processed includes: a plurality of image pixels, wherein the image segmentation features include: the confidence level of each image pixel belonging to a foreground object.

[0166] It should be noted that the foregoing explanation of the image processing method also applies to the image processing apparatus of this embodiment, and will not be repeated here.

[0167] In this embodiment, an image to be processed is acquired, which includes a foreground object. The image to be processed is input into an image segmentation model to obtain image segmentation features output by the image segmentation model. The image segmentation model is used to extract image segmentation features from the image to be processed. The image segmentation features are reference features for segmenting the foreground object from the image to be processed. The image segmentation features are determined based on the shared image features of the image to be processed, which are shared by the semantic and edge dimensions of the foreground object, and the feature scale of the shared image features. The foreground object in the image to be processed is segmented according to the image segmentation features to obtain the target image. Thus, the automation level of image processing can be effectively improved based on the image segmentation model. When the image segmentation features are determined based on the shared image features and the feature scale of the shared image features, the computational load of image segmentation processing can be effectively reduced, and the image processing effect can be effectively improved.

[0168] Figure 13 This is a schematic diagram of the structure of an image segmentation model generation device proposed in an embodiment of the present disclosure.

[0169] like Figure 13As shown, the image segmentation model generation device 130 includes:

[0170] The second acquisition module 1301 is used to acquire sample images and corresponding labeled segmentation features; the third acquisition module 1302 is used to acquire an initial neural network model; the third processing module 1303 is used to input the sample images into the initial neural network model to obtain predicted segmentation features; the calculation module 1304 is used to calculate the loss value between the predicted segmentation features and the labeled segmentation features; the training module 1305 is used to train the initial neural network model based on the loss value until the initial neural network model converges, and use the trained neural network model as an image segmentation model.

[0171] In some embodiments of this disclosure, such as Figure 14 As shown, Figure 14 This is a schematic diagram of the structure of an image segmentation model generation device proposed in another embodiment of the present disclosure. The initial neural network model includes: an initial shared feature extraction sub-model, an initial semantic extraction sub-model, an initial edge detail extraction sub-model, and an initial fusion sub-model.

[0172] The third processing module 1303 includes: a seventh processing submodule 13031, used to input the sample image into the initial shared feature extraction submodel to obtain the sample shared image features of the sample image and determine the sample feature scale of the sample shared image features; an eighth processing submodule 13032, used to input the sample shared image features into the initial semantic extraction submodel to obtain the sample semantic segmentation result of the sample image, wherein the sample semantic segmentation result describes the semantic dimension image features of the sample image; a ninth processing submodule 13033, used to input the sample shared image features into the initial edge detail extraction submodel to obtain the edge image information corresponding to the sample foreground object in the sample image; and a tenth processing submodule 13034, used to input the sample feature scale, the sample semantic segmentation result, and the edge image information into the initial fusion submodel to obtain the predicted segmentation features.

[0173] It should be noted that the foregoing explanation of the image segmentation model generation method also applies to the image segmentation model generation apparatus of this embodiment, and will not be repeated here.

[0174] In this embodiment, by acquiring sample images and corresponding labeled segmentation features, where the sample images include sample foreground objects and the labeled segmentation features are reference features labeled for segmenting the sample foreground objects from the sample images, an initial neural network model is obtained. The sample images are input into the initial neural network model to obtain predicted segmentation features, where the predicted segmentation features are the predicted reference features for segmenting the sample foreground objects from the sample images. The loss value between the predicted segmentation features and the labeled segmentation features is calculated, and the initial neural network model is trained based on the loss value until the initial neural network model converges. The trained neural network model is then used as the image segmentation model. Thus, the model performance of the initial neural network model can be evaluated based on the loss value during the model training process to accurately determine the model convergence time, thereby effectively improving the reliability of the obtained image segmentation model.

[0175] Figure 15 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Figure 15 The computer device 12 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0176] like Figure 15 As shown, the computer device 12 is represented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0177] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0178] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0179] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 15 Not shown; usually referred to as a "hard drive".

[0180] although Figure 15 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0181] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0182] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable human interaction with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0183] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the image processing method and image segmentation model generation method mentioned in the foregoing embodiments.

[0184] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image processing method and the image segmentation model generation method proposed in the foregoing embodiments of this disclosure.

[0185] To implement the above embodiments, this disclosure also proposes a computer program product that, when executed by an instruction processor, performs the image processing method and the image segmentation model generation method as proposed in the foregoing embodiments of this disclosure.

[0186] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0187] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0188] It should be noted that in the description of this disclosure, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0189] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0190] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0191] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0192] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0193] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0194] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0195] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. An image processing method, characterized in that, include: Obtain the image to be processed, wherein the image to be processed includes a foreground object; The image to be processed is input into the shared feature extraction sub-model of the image segmentation model to obtain the shared image features of the image to be processed, and the feature scale of the shared image features is determined. The type of feature scale is multiple. The shared image features are input into the semantic extraction sub-model of the image segmentation model to obtain the initial semantic segmentation result of the image to be processed; The shared image features are input into the edge detail extraction sub-model of the image segmentation model to obtain the edge image information of the foreground object in the image to be processed; The initial semantic segmentation result is transformed based on each of the aforementioned feature scales to obtain the target semantic segmentation result; The semantic segmentation result of each target is fused with the edge image information of the corresponding feature scale to obtain the fused image features of the corresponding feature scale. The fused image features at multiple feature scales are input into the fusion sub-model to obtain image segmentation features; The foreground object in the image to be processed is segmented according to the image segmentation features to obtain the target image.

2. The method as described in claim 1, characterized in that, The initial semantic segmentation result describes the semantic dimension image features of the image to be processed. The image segmentation model includes: a shared feature extraction sub-model, a semantic extraction sub-model, an edge detail extraction sub-model, and a fusion sub-model. The image segmentation model is obtained by jointly training the shared feature extraction sub-model, the semantic extraction sub-model, the edge detail extraction sub-model, and the fusion sub-model.

3. The method as described in claim 2, characterized in that, The image to be processed includes: a foreground sub-image and a background sub-image; wherein... The initial semantic segmentation result includes: the foreground semantic segmentation result of the foreground sub-image and the background semantic segmentation result of the background sub-image. The output scale of the foreground semantic segmentation result is smaller than the scale of the image to be processed, and the output scale of the background semantic segmentation result is smaller than the scale of the image to be processed.

4. The method as described in claim 2, characterized in that, The image to be processed includes: multiple edge pixels of a foreground object, wherein the edge image information includes: the confidence level of each edge pixel belonging to the foreground object.

5. The method according to any one of claims 1-4, characterized in that, The image to be processed includes: multiple image pixels, wherein the image segmentation feature includes: the confidence level of each image pixel belonging to the foreground object.

6. A method for generating an image segmentation model, characterized in that, include: Obtain a sample image and corresponding labeled segmentation features, wherein the sample image includes a sample foreground object, and the labeled segmentation features are labeled reference features for segmenting the sample foreground object from the sample image; Obtain the initial neural network model; The sample image is input into the initial shared feature extraction sub-model to obtain the sample shared image features of the sample image, and the sample feature scale of the sample shared image features is determined. There are multiple types of feature scales. The shared image features of the samples are input into the initial semantic extraction sub-model to obtain the sample semantic segmentation result of the sample image; The shared image features of the samples are input into the initial edge detail extraction sub-model to obtain the edge image information of the foreground object of the sample corresponding to the sample image; The semantic segmentation results of the samples are transformed based on each of the aforementioned feature scales to obtain the target semantic segmentation results; The semantic segmentation result of each target is fused with the edge image information of the corresponding feature scale to obtain the fused image features of the corresponding feature scale. The fused image features at multiple feature scales are input into the initial fusion sub-model to obtain the predicted segmentation features; Calculate the loss value between the predicted segmentation features and the labeled segmentation features; The initial neural network model is trained based on the loss value until it converges, and the trained neural network model is then used as an image segmentation model.

7. An image processing apparatus, characterized in that, include: The first acquisition module is used to acquire an image to be processed, wherein the image to be processed includes a foreground object; The first processing module is used to input the image to be processed into the image segmentation model to obtain the image segmentation features output by the image segmentation model; The second processing module is used to segment the foreground object in the image to be processed according to the image segmentation features to obtain the target image; The first processing module includes: The first processing submodule is used to input the image to be processed into the shared feature extraction submodel to obtain the shared image features of the image to be processed, and to determine the feature scale of the shared image features, wherein there are multiple types of feature scales; The second processing submodule is used to input the shared image features into the semantic extraction sub-model to obtain the initial semantic segmentation result of the image to be processed; The third processing submodule is used to input the shared image features into the edge detail extraction submodel to obtain the edge image information of the foreground object in the image to be processed; The fourth processing submodule is used to input the feature scale, the initial semantic segmentation result, and the edge image information into the fusion sub-model to obtain the image segmentation features; The fifth processing submodule is used to transform the initial semantic segmentation result based on each of the aforementioned feature scales to obtain the target semantic segmentation result; The sixth processing submodule is used to fuse each of the target semantic segmentation results with the edge image information of the corresponding feature scale to obtain the fused image features of the corresponding feature scale; The fourth processing submodule is specifically used for: The image segmentation features are obtained by inputting the fused image features of multiple feature scales into the fusion sub-model.

8. The apparatus as claimed in claim 7, characterized in that, The initial semantic segmentation result describes the semantic dimension image features of the image to be processed. The image segmentation model includes: a shared feature extraction sub-model, a semantic extraction sub-model, an edge detail extraction sub-model, and a fusion sub-model. The image segmentation model is obtained by jointly training the shared feature extraction sub-model, the semantic extraction sub-model, the edge detail extraction sub-model, and the fusion sub-model.

9. The apparatus as claimed in claim 8, characterized in that, The image to be processed includes a foreground sub-image and a background sub-image. The initial semantic segmentation result includes a foreground semantic segmentation result of the foreground sub-image and a background semantic segmentation result of the background sub-image. The output scale of the foreground semantic segmentation result is smaller than the scale of the image to be processed, and the output scale of the background semantic segmentation result is smaller than the scale of the image to be processed.

10. The apparatus as claimed in claim 8, characterized in that, The image to be processed includes: multiple edge pixels of a foreground object, wherein the edge image information includes: the confidence level of each edge pixel belonging to the foreground object.

11. The apparatus according to any one of claims 7-10, characterized in that, The image to be processed includes: multiple image pixels, wherein the image segmentation feature includes: the confidence level of each image pixel belonging to the foreground object.

12. An apparatus for generating an image segmentation model, characterized in that, include: The second acquisition module is used to acquire a sample image and labeled segmentation features corresponding to the sample image, wherein the sample image includes a sample foreground object, and the labeled segmentation features are labeled reference features for segmenting the sample foreground object from the sample image; The third acquisition module is used to acquire the initial neural network model; The third processing module is used to input the sample image into the initial neural network model to obtain the predicted segmentation features; The calculation module is used to calculate the loss value between the predicted segmentation features and the labeled segmentation features; The training module is used to train the initial neural network model based on the loss value until the initial neural network model converges, and to use the trained neural network model as an image segmentation model. The third processing module includes: The seventh processing submodule is used to input the sample image into the initial shared feature extraction sub-model to obtain the sample shared image features of the sample image and determine the sample feature scale of the sample shared image features; The eighth processing submodule is used to input the shared image features of the samples into the initial semantic extraction submodel to obtain the sample semantic segmentation result of the sample image, wherein the sample semantic segmentation result describes the semantic dimension image features of the sample image; The ninth processing submodule is used to input the sample shared image features into the initial edge detail extraction sub-model to obtain the edge image information of the sample foreground object in the sample image; The tenth processing submodule is used to transform the sample semantic segmentation result based on each of the aforementioned feature scales to obtain the target semantic segmentation result; to fuse each of the aforementioned target semantic segmentation results with the edge image information of the corresponding feature scale to obtain the fused image feature of the corresponding feature scale; and to input the fused image features of multiple feature scales into the initial fusion sub-model to obtain the predicted segmentation feature.

13. A computer device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5, or to perform the method of any one of claims 6.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, The computer instructions are used to cause the computer to perform the method of any one of claims 5, or to perform the method of any one of claims 6.

15. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 5, or implements the steps of the method according to any one of claims 6.

Images