Image processing method, network model training method, device and electronic equipment

Abstract

The present disclosure provides an image processing method, a network model training method, device and electronic equipment, relating to the technical field of image processing, the image processing method comprising: obtaining image features of an image to be processed; inputting the image features into a pre-trained network model to output a mask image corresponding to the image to be processed; and extracting a target image region from the image to be processed based on the mask image; wherein the network model is configured to: extract multi-level features from the image features; perform first fusion processing on level features of at least two levels in the multi-level features to obtain initial fusion features; and perform second fusion processing on the initial fusion features to obtain the mask image. This method can reduce the amount of data processed by the network model, so that the network model can run on a terminal device with limited hardware conditions, provide real-time matting results to users, and improve user experience.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, a network model training method, a device, and an electronic device. Background Art

[0002] Cutout is the process of separating a portion of an image area from the original image. The separated image area can include the main objects in the original image, such as people, animals, still lifes, or backgrounds. These image areas can be used for subsequent image creation. There are many cutout algorithms. One of them is that the user manually calibrates the trimap image and then expands it using the similarity between pixels to obtain the cutout result. However, for more complex images, the cutout effect and speed of this method are poor. Another cutout algorithm is implemented using a network model. The original image and the corresponding trimap image are input into the network model, and the cutout result is output. However, this method requires the network model to be deployed on an enterprise-level GPU server. Due to IO limitations, it is difficult for users to obtain the cutout result in real time, resulting in a low user experience. Summary of the Invention

[0003] In view of this, the purpose of the present disclosure is to provide an image processing method, a network model training method, an apparatus and an electronic device to reduce the amount of data processed by the network model, so that the network model can run on terminal devices with limited hardware conditions, provide users with real-time clipping results, and improve user experience.

[0004] In a first aspect, an embodiment of the present disclosure provides an image processing method, comprising: obtaining image features of an image to be processed; inputting the image features into a pre-trained network model, and outputting a mask image corresponding to the image to be processed; extracting a target image area from the image to be processed based on the mask image; wherein the network model is used to: extract multi-level features from the image features; perform a first fusion processing on the hierarchical features of at least two levels of the multi-level features to obtain an initial fused feature; and perform a second fusion processing on the initial fused feature to obtain a mask image.

[0005] The above-mentioned network model includes an encoder, a fusion network and a decoder; the above-mentioned step of inputting image features into a pre-trained network model and outputting a mask image corresponding to the image to be processed includes: extracting multi-level features from the image features through an encoder; dividing the multi-level features into multiple level groups through a fusion network, and performing a first fusion process on the level features in each level group to obtain an initial fusion feature; wherein the levels in each level group are adjacent; the number of levels of the initial fusion feature matches the number of levels of the calculation layer in the decoder; for the level features of each level in the initial fusion feature, the current level feature is input into the calculation layer in the decoder that matches the current level feature, and the initial fusion feature is subjected to a second fusion process in a preset order through the decoder to output a mask image.

[0006] The above-mentioned network model also includes a compression network; before the above-mentioned step of dividing the multi-level features into multiple hierarchical groups through the fusion network and performing the first fusion processing on the hierarchical features in each hierarchical group, the method also includes: reducing the number of channels of the hierarchical features of a specified level in the multi-level features through the compression network so that the number of channels of the hierarchical features in the same hierarchical group is the same.

[0007] The above-mentioned step of inputting the current level feature into the calculation layer in the decoder that matches the current level feature for each level feature in the initial fusion feature, and performing a second fusion processing on the initial fusion feature through the decoder in a preset order to output the mask image includes: for the highest level calculation layer of the decoder, performing a first convolution processing on the input feature of the highest level calculation layer to obtain the output feature; for the calculation layers of the decoder other than the highest level, performing a fusion processing on the output feature of the previous calculation layer adjacent to the current calculation layer and the input feature of the current calculation layer to obtain a fusion result; performing a convolution processing on the fusion result to obtain the output feature; wherein, the output result of the lowest level calculation layer of the decoder is the mask image.

[0008] The above-mentioned steps of fusing the output features of the previous calculation layer adjacent to the current calculation layer and the features input to the current calculation layer for the calculation layers other than the highest level of the decoder to obtain a fusion result; and performing convolution processing on the fusion result to obtain the output features include: for the calculation layers of the intermediate levels of the decoder, interpolating the output features of the previous calculation layer adjacent to the current calculation layer so that the scale of the output features matches the scale of the features input to the current calculation layer; performing a second convolution processing on the output features after the interpolation processing to obtain a processing result; fusing the processing result with the features input to the current calculation layer to obtain a first sub-fusion result; performing a first convolution processing on the first sub-fusion result to obtain the output features; for the calculation layers of the lowest level of the decoder, interpolating the output features of the previous calculation layer adjacent to the current calculation layer so that the scale of the output features matches the scale of the features input to the current calculation layer; fusing the output features after the interpolation processing with the features input to the current calculation layer to obtain a second sub-fusion result; and performing a third convolution processing on the second sub-fusion result to obtain a mask image.

[0009] Before the above-mentioned step of inputting the image features into the pre-trained network model and outputting the mask image corresponding to the image to be processed, the method also includes: determining whether the scale of the image features meets the preset conditions; wherein the preset conditions are related to the scale of the features output by each computing layer in the network model; if the scale of the image features does not meet the preset conditions, adjusting the scale of the image features until the preset conditions are met.

[0010] In a second aspect, an embodiment of the present disclosure provides a method for training a network model, the method comprising: determining a sample image based on a preset training sample set; the sample image carries a mask label; obtaining image features of the sample image; training the preset network model based on the image features to obtain a trained network model; wherein the network model is used to: extract multi-level features from the image features of the sample image; perform a first fusion processing on the hierarchical features of at least two levels of the multi-level features to obtain an initial fusion feature; perform a second fusion processing on the initial fusion feature to obtain a mask image of the sample image.

[0011] The above network model includes a pre-trained encoder, a pre-trained fusion network, and a decoder to be trained; the decoder includes multiple levels of computing layers; the above steps of training the preset network model based on image features to obtain a trained network model include: extracting multi-level features from image features through the encoder; dividing the multi-level features into multiple level groups through the fusion network, performing a first fusion process on the level features in each level group to obtain an initial fusion feature; wherein the levels in each level group are adjacent; the number of levels of the initial fusion feature matches the number of levels of the computing layer in the decoder; and for each level in the initial fusion feature, performing a first fusion process on the level features in the decoder. Level features, input the current level features into the calculation layer in the decoder that matches the current level features, and perform a second fusion process on the initial fusion features in a preset order through the decoder to obtain a processing result; based on the processing result and the mask label, determine the loss value; for the highest level calculation layer of the decoder, update the parameters of the highest level calculation layer based on the loss value until the network model converges, and obtain the parameters of the highest level calculation layer; for the calculation layers other than the highest level of the decoder, update the parameters of the current calculation layer based on the parameters of all calculation layers higher than the current calculation layer and the loss value until the network model converges, and obtain the parameters of the current calculation layer.

[0012] The above-mentioned step of updating the parameters of the current calculation layer based on the parameters of all calculation layers higher than the current calculation layer and the loss value until the network model converges for the calculation layers other than the highest layer of the decoder, and obtaining the parameters of the current calculation layer includes: while fixing the parameters of all calculation layers higher than the current calculation layer, updating the parameters of the current calculation layer based on the loss value until the network model converges; updating the parameters of the calculation layers higher than the current calculation layer and the parameters of the current calculation layer based on the loss value until the network model converges.

[0013] In a third aspect, an embodiment of the present disclosure provides an image processing device, which includes: an acquisition module for acquiring image features of an image to be processed; an output module for inputting the image features into a pre-trained network model and outputting a mask image corresponding to the image to be processed; an extraction module for extracting a target image area from the image to be processed based on the mask image; wherein the network model is used to: extract multi-level features from the image features; perform a first fusion processing on the hierarchical features of at least two levels of the multi-level features to obtain an initial fused feature; and perform a second fusion processing on the initial fused feature to obtain a mask image.

[0014] In a fourth aspect, an embodiment of the present disclosure provides a training device for a network model, the device comprising: a determination module for determining a sample image based on a preset training sample set; the sample image carries a mask label; a training module for obtaining image features of the sample image; training the preset network model based on the image features to obtain a trained network model; wherein the network model is used to: extract multi-level features from the image features of the sample image; perform a first fusion processing on the hierarchical features of at least two levels of the multi-level features to obtain an initial fusion feature; and perform a second fusion processing on the initial fusion feature to obtain a mask image of the sample image.

[0015] In a fifth aspect, an embodiment of the present disclosure provides an electronic device comprising a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the above-mentioned image processing method or the above-mentioned network model training method.

[0016] In a sixth aspect, an embodiment of the present disclosure provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the above-mentioned image processing method or the above-mentioned network model training method.

[0017] The embodiments of the present disclosure bring the following beneficial effects:

[0018] The above-mentioned image processing method, network model training method, device and electronic device input the image features of the image to be processed into a pre-trained network model, and output a mask image corresponding to the image to be processed; based on the mask image, extract the target image area from the image to be processed; wherein the network model extracts multi-level features from the image features; performs a first fusion process on the hierarchical features of at least two levels of the multi-level features to obtain initial fused features; and performs a second fusion process on the initial fused features to obtain a mask image. In the process of processing image features, the network model performs a second fusion on the features, which reduces the amount of feature data processed by the model while preserving the features, thereby improving the cutout processing speed while ensuring the cutout effect; due to the low amount of data processed by the network model, the network model can run on terminal devices with limited hardware conditions, and can provide users with real-time cutout results, thereby improving the user experience.

[0019] Other features and advantages of the present disclosure will be described in the following description, and in part will become apparent from the description, or understood by practicing the present disclosure. The objectives and other advantages of the present disclosure are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0020] In order to make the above objectives, characteristics and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0022] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0023] Figure 2 A structural schematic diagram of a network model provided by an embodiment of the present disclosure;

[0024] Figure 3 A structural schematic diagram of another network model provided by an embodiment of the present disclosure;

[0025] Figure 4 A flowchart of a training method of a network model provided by an embodiment of the present disclosure;

[0026] Figure 5 A structural schematic diagram of an image processing device provided by an embodiment of the present disclosure;

[0027] Figure 6 A structural schematic diagram of a training device of a network model provided by an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In order to make the above objectives, characteristics and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows.

[0030] The automatic matting algorithm in the related art mainly has the following three kinds:

[0031] The first is the traditional method, which uses sampling algorithms, pixel affinities algorithms, or a combination of the two to achieve cutouts. However, these methods require solving a large linear system and require the user to manually calibrate a trimap. This trimap is used to calibrate the target area for cutouts. The similarity between adjacent pixels is then used to perform extended matching combinations to obtain the final cutout result. The processing speed of this method is proportional to the number of unknown points in the target area.

[0032] The second method is based on the CNN (Convolutional Neural Networks) model. The RGB original image and the manually annotated trimap image are combined as the input of the CNN model. The trimap image is used to calibrate the key areas of the model's attention. The model finally outputs an alpha image; finally, the cutout result is obtained based on the alpha image.

[0033] The third method is also based on the CNN model. This method usually uses a semantic segmentation model to extract the target body, and then indirectly obtains the Trimap image through corrosion and expansion. The RGB original image and the Trimap image are then combined, and the Alpha image is finally obtained using the second method mentioned above.

[0034] Among them, for the first method, which uses the similarity between pixels to expand the cutout, the processing speed is usually proportional to the size of the image. For example, for an image of about 1M, the processing time varies from a few seconds to a few minutes depending on the computer hardware conditions. Moreover, in slightly complex scenes, the cutout effect of this method is not ideal. In terms of processing effect and processing speed, this method is difficult to meet the user's creative needs.

[0035] The second and third methods mentioned above directly or indirectly obtain the trimap image and combine it with the original RGB image to input into the CNN model. This CNN model is typically deployed on enterprise-level GPU (Graphics Processing Unit) servers. Due to the consumption of I / O (Input / Output) ports, it cannot meet real-time requirements. In addition, the manual calibration of the trimap image by the user increases system complexity and reduces the user experience.

[0036] In summary, the present embodiments provide an image processing method, network model training method, device, and electronic device. This technology can be applied to scenarios where a target area is extracted from an original image, such as extracting a main object or foreground image from an original image. To facilitate understanding of this embodiment, the image processing method disclosed in this embodiment is first described in detail.

[0037] like Figure 1 As shown, the method includes the following steps:

[0038] Step S102, obtaining image features of the image to be processed;

[0039] Image features typically include detailed features of the image being processed. Specifically, the image being processed can be input into a feature extraction network, which outputs the image features of the image being processed. This feature extraction network can be based on a convolutional neural network backbone, also known as a CNN backbone. Image features typically have a high number of channels, such as 16 or 32 channels. The scale of image features is typically related to the scale of the image being processed, for example, the height of an image feature is 1 / 2 the height of the image being processed, and the width of an image feature is 1 / 2 the width of the image being processed. These scales can be set based on actual needs.

[0040] Step S104: input the image features into a pre-trained network model and output a mask image corresponding to the image to be processed;

[0041] The network model is used to: extract multi-level features from image features; perform a first fusion process on the hierarchical features of at least two levels of the multi-level features to obtain an initial fused feature; and perform a second fusion process on the initial fused feature to obtain a masked image. For example, the network model may include multiple sequentially connected computing layers, which may be convolutional layers, fully connected layers, pooling layers, etc.; each computing layer may output a hierarchical feature; the width, height, number of channels, etc. of the hierarchical features output by different computing layers may be the same or different, and the specific details may be determined based on the operators in the computing layer.

[0042] After the network model extracts multi-level features from the image features, it can first perform a first fusion process on the hierarchical features of at least two levels to obtain an initial fused feature; since the hierarchical features of at least two levels are fused, the number of levels of features in the initial fused feature is less than the number of levels of the multi-level features extracted from the image features; which levels of hierarchical features are fused can be determined based on the characteristics of the features of each level, for example, if two hierarchical features complement or reinforce each other, they can be fused; for features of levels with richer feature quantities, especially features of the middle level, they can be not fused. Through the above-mentioned first fusion process, the number of levels of features can be reduced, thereby reducing the amount of features that the model needs to process later, thereby improving the processing speed.

[0043] The aforementioned second fusion process of the initial fused features can also be implemented through sequentially connected computational layers, which can also be convolutional layers, fully connected layers, pooling layers, and so on. Specifically, the process can start with the highest-level features of the initial fused features and fuse the highest-level features into the features of the next level, and so on, until the lowest-level features are reached. During the fusion process of the two-level features, the scale, number of channels, and other factors of the hierarchical features can be adjusted to facilitate fusion.

[0044] Step S106, extracting a target image area from the image to be processed based on the mask image;

[0045] The pixel values ​​of the pixels in the mask image usually include two types, 1 and 0; by multiplying the corresponding pixels of the mask image and the image to be processed, the area of ​​the image to be processed corresponding to the area with pixel value 1 in the mask image can be extracted, that is, the above-mentioned target image area.

[0046] In the above-mentioned image processing method, the image features of the image to be processed are input into a pre-trained network model, and a mask image corresponding to the image to be processed is output; based on the mask image, the target image region is extracted from the image to be processed; the network model extracts multi-level features from the image features; the hierarchical features of at least two levels of the multi-level features are subjected to a first fusion process to obtain initial fused features; and the initial fused features are subjected to a second fusion process to obtain a mask image. In the process of processing image features, the network model performs a secondary fusion of the features, which preserves the features while reducing the amount of feature data processed by the model, thereby improving the processing speed of the cutout while ensuring the cutout effect. Due to the low amount of data processed by the network model, the network model can run on terminal devices with limited hardware conditions, providing users with real-time cutout results and improving the user experience.

[0047] The following examples further describe the structure of the network model and the feature processing method. Figure 2As shown, the network model includes an encoder, a fusion network and a decoder; the image features of the image to be processed are first input into the encoder, and the multi-level features are extracted from the image features by the encoder; Figure 2 The six-level features are taken as an example for illustration.

[0048] The above multi-level features are input into the fusion network, and the multi-level features are divided into multiple level groups by the fusion network, the level features in each level group are subjected to first fusion processing to obtain initial fusion features; the levels of the initial fusion features match the levels of the calculation layers in the decoder.

[0049] As shown in Figure 2 In the example, the six-level features are divided into four level groups, wherein the level feature 1 and the level feature 2 in the multi-level features are divided into one level group, and the level feature 1 and the level feature 2 are subjected to first fusion processing to obtain the fusion feature 1 in the initial fusion features; the level feature 3 in the multi-level features is divided into one level group, and the fusion feature 2 is directly obtained based on the level feature 3 without feature fusion processing; similarly, the level feature 4 in the multi-level features is divided into one level group, and the fusion feature 3 is directly obtained based on the level feature 4 without feature fusion processing; the level feature 5 and the level feature 6 in the multi-level features are divided into one level group, and the fusion feature 4 in the initial fusion features is obtained after the first fusion processing of the level feature 5 and the level feature 6.

[0050] Of course, the level groups in the fusion network can also have other division manners, for example, the level feature 3 and the level feature 4 can also be divided into one level group for first fusion processing; the level feature 2 and the level feature 3 can also be divided into one level group for first fusion processing. The division manner of the level group can be set according to the requirement, for example, a plurality of division manners can be set in advance, and experiments are conducted for each division manner, the division manner with the most accurate final output mask image is determined as the final division manner.

[0051] In order to facilitate feature fusion or make the fused features more effective, the levels in the same level group are adjacent regardless of which division manner; the initial fusion features also include multi-level features, as shown in Figure 2 In the example, there are four levels of fusion features; usually, the number of levels of the initial fusion features matches the number of levels of the calculation layers in the decoder, so as to facilitate inputting the fusion features of each level in the initial fusion features into the corresponding calculation layers in the decoder for re-fusion calculation.

[0052] For each level of the initial fusion feature, the current level feature is input into the calculation layer that matches the current level feature in the decoder, and the decoder performs a second fusion process on the initial fusion feature in a preset order to output a mask image. For example, Figure 2 In the figure, fusion feature 4 is processed by the calculation layer that matches the features of this level and then fused with fusion feature 3; the fusion result is processed by the calculation layer that matches the fusion feature 3 and then fused with fusion feature 2 until it reaches fusion feature 1. After the fusion feature 1 is fused, the mask image can be output after certain calculations.

[0053] In addition, in order to facilitate the fusion of hierarchical features in the same hierarchical group, the above network model also includes a compression network; after the encoder outputs multi-level features, the multi-level features are first input into the compression network; through the compression network, the number of channels of the hierarchical features of a specified level in the multi-level features is reduced so that the number of channels of the hierarchical features in the same hierarchical group is the same. Figure 2 , assuming that hierarchical features 5 and hierarchical features 6 are divided into the same hierarchical group, where the number of channels of hierarchical feature 5 is 96 and the number of channels of hierarchical feature 6 is 180; in order to fuse hierarchical features 5 and 6, it is necessary to compress the number of channels of hierarchical feature 6 in the hierarchical group, and compress the number of channels of hierarchical feature 6 from 180 to 96; the compression calculation can be achieved through a 1*1 convolution kernel, that is, convolution operation is performed on hierarchical feature 6 using a 1*1 convolution kernel to obtain hierarchical feature 6 with compressed channels.

[0054] The two hierarchical features to be fused need to have the same number of channels, as well as the same width and height. If the width and height are different, they also need to be adjusted. Fusion can be understood as the merging of features. When performing fusion calculations, the two hierarchical features involved in the fusion calculation are merged into one hierarchical feature. Specifically, the number of channels, width, and height of the two hierarchical features involved in the fusion calculation are the same, so the number of feature points of the two hierarchical features is the same. For the feature point at each position, the values ​​of the feature points at that position in the two hierarchical features are added together to obtain the value of the fused feature point at that position. Therefore, the fused feature after the two hierarchical features are fused has the same number of channels, width, and height as the two hierarchical features involved in the fusion calculation.

[0055] The following combination Figure 3 , specifically describes the specific process of the decoder performing the second fusion processing on the initial fusion features and outputting the mask image. Figure 3In the encoder, 8 layers of computational layers are connected in sequence, and a total of 8 hierarchical features are output. Each hierarchical feature has two parameters. For example, in hierarchical feature 1, "X / 2" represents the width or height of the image feature. "X / 2" means that the width of hierarchical feature 1 is 1 / 2 of the width of the image feature, and the height is 1 / 2 of the height of the image feature. The other parameter 16 represents the number of channels of hierarchical feature 1 is 16. The parameters of other hierarchical features and fusion features have the same meaning.

[0056] Among the 8-layer hierarchical features, the number of channels of hierarchical features 1, 6, and 8 are compressed to facilitate fusion with adjacent hierarchical features. The fused features include 5 layers in total, which are input to the corresponding calculation layers of the decoder. After the fused features are input to the decoder, the first convolution process is performed on the input features of the highest level calculation layer of the decoder to obtain the output features; Figure 3 In the decoder, the calculation layer 5 is the highest-level calculation layer. After the fusion feature 5 is processed by the calculation layer 5 for the first convolution, the output feature is obtained.

[0057] For all computational layers except the highest level of the decoder, the output features of the previous computational layer adjacent to the current computational layer and the input features of the current computational layer are fused to obtain a fusion result; the fusion result is convolved to obtain an output feature; the output result of the lowest level computational layer of the decoder is a mask image. As an example, for Figure 3 The output features of the computing layer 4 in the computing layer 5 are fused with the features input by the computing layer 4 to obtain the fusion result, which is then subjected to the first convolution processing by the computing layer 4 to obtain the output features of the computing layer 4; for example, Figure 3 The output features of the computing layer 3 and the computing layer 4 are fused with the features input by the computing layer 3 to obtain a fusion result, which is then subjected to the first convolution processing by the computing layer 3 to obtain the output features of the computing layer 3.

[0058] In addition, for layers other than the highest layer, the scale of the features needs to be adjusted before feature fusion. Based on this, for the computational layer of the intermediate layer of the decoder, the output features of the previous computational layer adjacent to the current computational layer are interpolated to match the scale of the output features with the scale of the features input to the current computational layer; the output features after interpolation are subjected to a second convolution process to obtain a processing result; the processing result is fused with the features input to the current computational layer to obtain a first sub-fusion result; the first sub-fusion result is subjected to a first convolution process to obtain the output feature; Figure 3The intermediate computational layers in the

[15] include computational layer 4, computational layer 3, and computational layer 2. Taking computational layer 4 as an example, the output features of computational layer 5 are interpolated, and the width and height of the output features are doubled. At this time, the width and height of the output features are the same as the width and height of the fusion feature 4. The output features after interpolation are then convolved for the second time using a 3*3 convolution kernel, and then fused with the fusion feature 4.

[0059] For the lowest level calculation layer of the decoder, the output features of the previous calculation layer adjacent to the current calculation layer are interpolated to match the scale of the output features with the scale of the features input to the current calculation layer; the output features after interpolation are fused with the features input to the current calculation layer to obtain the second sub-fusion result; the second sub-fusion result is subjected to the third convolution process to obtain the mask image. Figure 3 In the figure, calculation layer 1 is the lowest level calculation layer. For calculation layer 1, the output features of calculation layer 2 are interpolated. The width and height of the output features are doubled. At this time, the width and height of the output features are the same as the width and height of fusion feature 1. After the output features are fused with fusion feature 1, a fusion result is obtained. The fusion result is then convolved for the second time using a 3*3 convolution kernel to obtain a mask image. The width and height of the mask image can also be adjusted by interpolation.

[0060] In addition, in order to facilitate the calculation of the model, the image features can be preprocessed before being input into the network model. First, determine whether the scale of the image features meets the preset conditions; the preset conditions are related to the scale of the features output by each computing layer in the network model; if the scale of the image features does not meet the preset conditions, adjust the scale of the image features until it meets the preset conditions. For example, when the encoder is calculating, the scale of the hierarchical features output by each computing layer is reduced. Figure 3 In the example, the scale of the hierarchical feature 8 output by the computation layer 8 is 1 / 32 of the image feature scale. Based on this, the scale of the image feature can be adjusted to a multiple of 32 to facilitate the operation of the encoder.

[0061] It should also be noted that in order to enable the model to run on mobile terminals such as mobile phones, the operators of each computing layer in the model all use basic operators, such as convolution kernels, upsampling, downsampling and other operators, thereby obtaining a lightweight network model and facilitating model migration.

[0062] Based on the above-mentioned embodiment of the image processing method, this embodiment also provides a training method for a network model, such as Figure 4 As shown, the method includes the following steps:

[0063] Step S402: determining a sample image based on a preset training sample set; the sample image carries a mask label;

[0064] Step S404: obtaining image features of the sample image; training a preset network model based on the image features to obtain a trained network model;

[0065] Among them, the network model is used to: extract multi-level features from the image features of the sample image; perform a first fusion process on the hierarchical features of at least two levels in the multi-level features to obtain initial fusion features; and perform a second fusion process on the initial fusion features to obtain a mask image of the sample image.

[0066] In the process of processing image features, the network model performs a secondary fusion of the features, which reduces the amount of feature data processed by the model while preserving the features, thereby improving the cutout processing speed while ensuring the cutout effect; since the amount of data processed by the network model is relatively low, the network model can run on terminal devices with limited hardware conditions, and can provide users with real-time cutout results, thereby improving user experience.

[0067] The above-mentioned network model includes a pre-trained encoder, a pre-trained fusion network, and a decoder to be trained; the decoder includes multiple levels of computing layers; during the training process of the network model, the encoder and the fusion network can be trained first, and after the encoder and the fusion network are trained, the decoder can be trained.

[0068] When training the encoder and fusion network, you can first fix the parameters of the decoder, input the sample image into the encoder, and obtain the output result after processing by the encoder, fusion network and decoder; calculate the loss value based on the output result and the label of the sample image, and adjust the parameters of the encoder and fusion network according to the loss value; then continue to input the sample image into the encoder, and repeat this cycle multiple times until the loss value converges and the training of the encoder and fusion network is completed.

[0069] The following focuses on the training process of the decoder. First, the encoder extracts multi-level features from the image features of the sample image; the multi-level features are divided into multiple level groups through the fusion network, and the level features in each level group are subjected to a first fusion process to obtain initial fusion features; wherein, the levels in each level group are adjacent; the number of levels of the initial fusion features matches the number of levels of the computational layers in the decoder; for each level of the level features in the initial fusion features, the current level features are input into the computational layer in the decoder that matches the current level features, and the decoder performs a second fusion process on the initial fusion features in a preset order to obtain the processing results; based on the processing results and the mask labels, the loss value is determined.

[0070] In order to compress the feature quantity of each layer as much as possible to reduce the parameters of the model while ensuring the effectiveness of each layer of features, the decoder adopts a step-by-step output method for supervised training. When the low-scale feature output is stable, the output training of high-scale features is carried out, and the parameters of the calculation layer that outputs low-scale features serve as the basis for parameter training of the calculation layer that outputs high-scale features.

[0071] Specifically, starting from the highest-level computing layer, training is performed one computing layer at a time; for the highest-level computing layer of the decoder, the parameters of the highest-level computing layer are updated based on the above-mentioned loss value until the network model converges and the parameters of the highest-level computing layer are obtained; when training the parameters of the highest-level computing layer, the parameters of other levels remain unchanged, and the parameters of the highest-level computing layer are adjusted during the training process until the output results of the network model converge and the parameters of the highest-level computing layer are obtained.

[0072] At this point, continue the aforementioned process of determining the sample image based on the preset training sample set to obtain a new loss value, and then train the calculation layer of the next level of the highest level based on the loss value; for the calculation layers of the decoder other than the highest level, update the parameters of the current calculation layer based on the parameters of all calculation layers higher than the current calculation layer and the loss value until the network model converges and the parameters of the current calculation layer are obtained. The decoder starts from the highest level and trains step by step. After obtaining the parameters of the calculation layer of the highest level in the above manner, the parameters of the calculation layer of the highest level can be fixed, and the parameters of the next calculation layer of the highest level can be trained until the network model converges and the parameters of the calculation layer are obtained; and so on, until the training of the calculation layer of the lowest level is completed.

[0073] For computing layers other than the highest level, each computing layer can be trained specifically in the following manner: while fixing the parameters of all computing layers higher than the current computing layer, update the parameters of the current computing layer based on the loss value until the network model converges; update the parameters of the computing layers higher than the current computing layer and the parameters of the current computing layer based on the loss value until the network model converges. That is, the training of each computing layer is divided into two steps. The first step is to fix the parameters of the computing layers higher than the current computing layer and only train the parameters of the current computing layer. After the network model converges, the parameters of the computing layers higher than the current computing layer and the parameters of the current computing layer are trained simultaneously until the network model converges and the training of the current computing layer is completed. Then, the next computing layer of the current computing layer is trained in the same way until the training of the computing layer at the lowest level is completed.

[0074] As an example, continue to refer to the above Figure 3In the network model, taking the computation layer 3 in the decoder as an example, when the output of the trained network model of computation layer 4 converges, all parameters of computation layer 4 and all computation layers higher than computation layer 4 are fixed, and computation layer 3 is trained using a small learning rate. When the output of the network model converges again, the fixed state of the parameters of layer_4 and all computation layers higher than computation layer 4 is released, and computation layer 3 is trained as a whole until the output of the network model converges.

[0075] The above loss function can be a SAD (Sum of Absolute Difference) function, Where Pa and Pr represent the sample's mask label and the model's output mask image, respectively, with values ​​ranging from [0, 1]. A lower SAD indicates a closer match between the model's output and the expected label, and a better model output. Otherwise, the model's output alpha is worse. The above loss function can also be expressed as pixel IOU (Intersection-over-Union), which is the intersection-over-union ratio between the sample's mask label and the model's output mask image. This is used to measure the semantic integrity of the network model; higher IOU values ​​indicate more complete semantics.

[0076] The above network model can significantly improve the speed of image processing while ensuring the processing effect. During the cutout process, there is no need for users to manually mark the trimap, and the cutout speed is at the millisecond level, which improves the user experience.

[0077] Corresponding to the above method embodiment, see Figure 5 The structure diagram of an image processing device shown in FIG. 1 includes:

[0078] An acquisition module 50 is used to acquire image features of an image to be processed;

[0079] The output module 51 is used to input the image features into the pre-trained network model and output the mask image corresponding to the image to be processed;

[0080] An extraction module 52 is used to extract a target image area from the image to be processed based on the mask image;

[0081] The network model is used to: extract multi-level features from image features; perform a first fusion process on the hierarchical features of at least two levels in the multi-level features to obtain initial fusion features; and perform a second fusion process on the initial fusion features to obtain a mask image.

[0082] In the image processing device, an image feature of a to-be-processed image is input into a network model that is pre-trained, and a mask image corresponding to the to-be-processed image is output; a target image region is extracted from the to-be-processed image based on the mask image; the network model extracts multi-level features from the image feature; at least two levels of level features in the multi-level features are subjected to first fusion processing to obtain initial fusion features; and the initial fusion features are subjected to second fusion processing to obtain the mask image. In the process of processing the image feature, the network model performs secondary fusion on the feature, reduces the amount of feature data processed by the model while saving the feature, and thus improves the matting processing speed while ensuring the matting effect. Since the amount of data processed by the network model is low, the network model can run on a terminal device with limited hardware conditions, and can provide real-time matting results to users, thereby improving the user experience.

[0083] The network model includes an encoder, a fusion network and a decoder; and the output module is further configured to: extract multi-level features from the image feature through the encoder; divide the multi-level features into multiple level groups through the fusion network, and perform first fusion processing on the level features in each level group to obtain initial fusion features; the levels in each level group are adjacent; the number of levels of the initial fusion features matches the number of levels of the calculation layers in the decoder; for the level features of each level of the initial fusion features, the current level feature is input into a calculation layer in the decoder that matches the current level feature, and the initial fusion features are subjected to second fusion processing in a preset order through the decoder to output the mask image.

[0084] The network model further includes a compression network; and the output module is further configured to: before the step of performing first fusion processing on the level features in each level group, the method further includes: reducing the number of channels of the level features of a specified level in the multi-level features through the compression network, so that the number of channels of the level features in the same level group is the same.

[0085] The output module is further configured to: for the calculation layer of the highest level of the decoder, perform first convolution processing on the features input into the calculation layer of the highest level to obtain output features; for the calculation layers of the decoder other than the highest level, perform fusion processing on the output features of a previous calculation layer adjacent to a current calculation layer and the features input into the current calculation layer to obtain a fusion result; and perform convolution processing on the fusion result to obtain output features; and the output result of the calculation layer of the lowest level of the decoder is the mask image.

[0086] The output module is further configured to: for a calculation layer at a middle level of the decoder, interpolate output features of a previous calculation layer adjacent to the current calculation layer, so that a scale of the output features matches a scale of features input to the current calculation layer; perform second convolution processing on the interpolated output features to obtain a processing result; perform fusion processing on the processing result and the features input to the current calculation layer to obtain a first sub-fusion result; perform first convolution processing on the first sub-fusion result to obtain the output features; for a calculation layer at a lowest level of the decoder, interpolate output features of a previous calculation layer adjacent to the current calculation layer, so that a scale of the output features matches a scale of features input to the current calculation layer; perform fusion processing on the interpolated output features and the features input to the current calculation layer to obtain a second sub-fusion result; and perform third convolution processing on the second sub-fusion result to obtain the mask image.

[0087] The device further includes a preprocessing module configured to: determine whether a scale of the image features meets a preset condition; the preset condition is related to scales of features output by each calculation layer in the network model; and if the scale of the image features does not meet the preset condition, adjust the scale of the image features until the preset condition is met.

[0088] Referring to Figure 6 The device includes:

[0089] A determination module 60 is configured to determine a sample image based on a preset sample set; the sample image carries a mask label.

[0090] A training module 61 is configured to: obtain image features of the sample image; and train a preset network model based on the image features to obtain a trained network model.

[0091] The network model is configured to: extract multi-level features from the image features of the sample image; perform first fusion processing on at least two level features in the multi-level features to obtain initial fusion features; and perform second fusion processing on the initial fusion features to obtain a mask image of the sample image.

[0092] In the process of processing the image features, the network model performs secondary fusion on the features, saves the features, and reduces the amount of feature data processed by the model, thereby improving the matting processing speed while ensuring the matting effect. Since the amount of data processed by the network model is relatively low, the network model can be run on a terminal device with limited hardware conditions, can provide real-time matting results to users, and improves the user experience.

[0093] The above network model includes a pre-trained encoder, a pre-trained fusion network and a decoder to be trained; the decoder includes a multi-level computing layer; the above training model is also used to: extract multi-level features from image features through the encoder; divide the multi-level features into multiple level groups through the fusion network, perform a first fusion process on the level features in each level group to obtain an initial fusion feature; wherein the levels in each level group are adjacent; the number of levels of the initial fusion feature matches the number of levels of the computing layer in the decoder; for the level features of each level in the initial fusion feature, the current level feature is input to the decoder. The first step is to update the parameters of the highest-level calculation layer of the decoder based on the loss value until the network model converges and obtain the parameters of the highest-level calculation layer; for the calculation layers other than the highest level of the decoder, the parameters of the current calculation layer are updated based on the parameters of all calculation layers higher than the current calculation layer and the loss value until the network model converges and obtain the parameters of the current calculation layer.

[0094] The above training model is also used to: update the parameters of the current calculation layer based on the loss value while fixing the parameters of all calculation layers higher than the current calculation layer until the network model converges; update the parameters of the calculation layers higher than the current calculation layer and the parameters of the current calculation layer based on the loss value until the network model converges.

[0095] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the above-mentioned image processing method or network model training method.

[0096] See also Figure 7 As shown, the electronic device includes a processor 100 and a memory 101, wherein the memory 101 stores machine executable instructions that can be executed by the processor 100, and the processor 100 executes the machine executable instructions to implement the above-mentioned image processing method or network model training method.

[0097] Furthermore, Figure 7 The electronic device shown further includes a bus 102 and a communication interface 103 , and the processor 100 , the communication interface 103 and the memory 101 are connected via the bus 102 .

[0098] Among them, the memory 101 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 102 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0099] The processor 100 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 100 or by software instructions. The above processor 100 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as a random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or register. The storage medium is located in the memory 101. The processor 100 reads the information in the memory 101 and, in conjunction with its hardware, completes the steps of the method of the aforementioned embodiment.

[0100] This embodiment also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the above-mentioned image processing method or network model training method.

[0101] The computer program products of the image processing method, network model training method, device, and electronic device provided in the embodiments of the present disclosure include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. For specific implementation, please refer to the method embodiments and will not be repeated here.

[0102] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0103] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0104] Finally, it should be noted that the above embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Obtaining image features of the image to be processed; Input the image features into a pre-trained network model, and output a mask image corresponding to the image to be processed; extracting a target image region from the image to be processed based on the mask image; The network model is used to: extract multi-level features from the image features; perform a first fusion process on the hierarchical features of at least two levels in the multi-level features to obtain an initial fused feature; and perform a second fusion process on the initial fused feature to obtain the mask image; Wherein, the network model includes an encoder, a fusion network and a decoder; The step of inputting the image features into a pre-trained network model and outputting a mask image corresponding to the image to be processed includes: Extracting the multi-level features from the image features by the encoder; Dividing the multi-level features into a plurality of level groups through the fusion network, performing a first fusion process on the level features in each level group to obtain an initial fused feature; wherein the levels in each level group are adjacent; and the number of levels of the initial fused feature matches the number of levels of the computational layer in the decoder; For each level of the hierarchical features in the initial fusion features, the current level features are input into the calculation layer in the decoder that matches the current level features, and the initial fusion features are subjected to a second fusion process by the decoder in a preset order to output the mask image.

2. The method according to claim 1, characterized in that The network model also includes a compression network; Before the step of dividing the multi-level features into a plurality of level groups by the fusion network and performing a first fusion process on the level features in each level group, the method further includes: The compression network is used to reduce the number of channels of the hierarchical features of a specified level among the multi-level features so that the number of channels of the hierarchical features in the same level group is the same.

3. The method according to claim 1, characterized in that For each level of the initial fused features, the current level features are input into a calculation layer in the decoder that matches the current level features, and the decoder performs a second fusion process on the initial fused features in a preset order to output the mask image, including: For the highest-level computing layer of the decoder, performing a first convolution process on the input features of the highest-level computing layer to obtain output features; For the calculation layers of the decoder other than the highest level, the output features of the previous calculation layer adjacent to the current calculation layer and the input features of the current calculation layer are fused to obtain a fusion result; the fusion result is convolved to obtain output features; wherein the output result of the calculation layer of the lowest level of the decoder is the mask image.

4. The method according to claim 3, characterized in that For calculation layers other than the highest level of the decoder, fusing output features of a previous calculation layer adjacent to the current calculation layer with input features of the current calculation layer to obtain a fusion result; The step of performing convolution processing on the fusion result to obtain output features includes: For the computation layer at the intermediate level of the decoder, interpolation processing is performed on the output features of the previous computation layer adjacent to the current computation layer, so that the scale of the output features matches the scale of the features input to the current computation layer; a second convolution processing is performed on the output features after the interpolation processing to obtain a processing result; the processing result is fused with the features input to the current computation layer to obtain a first sub-fusion result; and a first convolution processing is performed on the first sub-fusion result to obtain an output feature; For the lowest-level computing layer of the decoder, interpolation processing is performed on the output features of the previous computing layer adjacent to the current computing layer to match the scale of the features input to the current computing layer; the output features after interpolation processing are fused with the features input to the current computing layer to obtain a second sub-fusion result; and a third convolution processing is performed on the second sub-fusion result to obtain the mask image.

5. The method according to claim 1, wherein Before the step of inputting the image features into a pre-trained network model and outputting a mask image corresponding to the image to be processed, the method further includes: Determining whether the scale of the image feature meets a preset condition; wherein the preset condition is related to the scale of the features output by each computing layer in the network model; If the scale of the image feature does not meet the preset condition, the scale of the image feature is adjusted until the preset condition is met.

6. A network model training method, characterized in that: The method comprises: Determining a sample image based on a preset training sample set; the sample image carries a mask label; Acquiring image features of the sample image; training a preset network model based on the image features to obtain a trained network model; The network model is used to: extract multi-level features from the image features of the sample image; perform a first fusion process on the hierarchical features of at least two levels of the multi-level features to obtain an initial fused feature; and perform a second fusion process on the initial fused feature to obtain a mask image of the sample image; The network model includes a pre-trained encoder, a pre-trained fusion network, and a decoder to be trained; the decoder includes multiple computing layers; The step of training a preset network model based on the image features to obtain a trained network model includes: Extracting the multi-level features from the image features by the encoder; Dividing the multi-level features into a plurality of level groups through the fusion network, performing a first fusion process on the level features in each level group to obtain an initial fused feature; wherein the levels in each level group are adjacent; and the number of levels of the initial fused feature matches the number of levels of the computational layer in the decoder; For each level of the hierarchical features in the initial fused features, input the current level features into a calculation layer in the decoder that matches the current level features, and perform a second fusion process on the initial fused features in a preset order through the decoder to obtain a processing result; determining a loss value based on the processing result and the mask label; For a highest-level computing layer of the decoder, updating parameters of the highest-level computing layer based on the loss value until the network model converges, thereby obtaining parameters of the highest-level computing layer; For the computing layers of the decoder other than the highest level, the parameters of the current computing layer are updated based on the parameters of all computing layers higher than the current computing layer and the loss value until the network model converges to obtain the parameters of the current computing layer.

7. The method according to claim 6, characterized in that For computing layers other than the highest level of the decoder, the step of updating the parameters of the current computing layer based on the parameters of all computing layers higher than the current computing layer and the loss value until the network model converges, and obtaining the parameters of the current computing layer includes: While fixing parameters of all computing layers higher than the current computing layer, updating the parameters of the current computing layer based on the loss value until the network model converges; Based on the loss value, parameters of a higher-level computing layer of the current computing layer and parameters of the current computing layer are updated until the network model converges.

8. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring image features of an image to be processed; An output module, configured to input the image features into a pre-trained network model and output a mask image corresponding to the image to be processed; an extraction module, configured to extract a target image region from the image to be processed based on the mask image; The network model is used to: extract multi-level features from the image features; perform a first fusion process on the hierarchical features of at least two levels in the multi-level features to obtain an initial fused feature; and perform a second fusion process on the initial fused feature to obtain the mask image; Wherein, the network model includes an encoder, a fusion network and a decoder; Inputting the image features into a pre-trained network model and outputting a mask image corresponding to the image to be processed includes: Extracting the multi-level features from the image features by the encoder; Dividing the multi-level features into a plurality of level groups through the fusion network, performing a first fusion process on the level features in each level group to obtain an initial fused feature; wherein the levels in each level group are adjacent; and the number of levels of the initial fused feature matches the number of levels of the computational layer in the decoder; For each level of the hierarchical features in the initial fusion features, the current level features are input into the calculation layer in the decoder that matches the current level features, and the initial fusion features are subjected to a second fusion process by the decoder in a preset order to output the mask image.

9. A network model training device, characterized in that: The device comprises: A determination module, configured to determine a sample image based on a preset training sample set; the sample image carries a mask label; A training module is used to obtain image features of the sample image; train a preset network model based on the image features to obtain a trained network model; The network model is used to: extract multi-level features from the image features of the sample image; perform a first fusion process on the hierarchical features of at least two levels of the multi-level features to obtain an initial fused feature; and perform a second fusion process on the initial fused feature to obtain a mask image of the sample image; The network model includes a pre-trained encoder, a pre-trained fusion network, and a decoder to be trained; the decoder includes multiple computing layers; The step of training a preset network model based on the image features to obtain a trained network model includes: Extracting the multi-level features from the image features by the encoder; Dividing the multi-level features into a plurality of level groups through the fusion network, performing a first fusion process on the level features in each level group to obtain an initial fused feature; wherein the levels in each level group are adjacent; and the number of levels of the initial fused feature matches the number of levels of the computational layer in the decoder; For each level of the hierarchical features in the initial fused features, input the current level features into a calculation layer in the decoder that matches the current level features, and perform a second fusion process on the initial fused features in a preset order through the decoder to obtain a processing result; determining a loss value based on the processing result and the mask label; For a highest-level computing layer of the decoder, updating parameters of the highest-level computing layer based on the loss value until the network model converges, thereby obtaining parameters of the highest-level computing layer; For the computing layers of the decoder other than the highest level, the parameters of the current computing layer are updated based on the parameters of all computing layers higher than the current computing layer and the loss value until the network model converges to obtain the parameters of the current computing layer.

10. An electronic device, characterized in that: It includes a processor and a memory, the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the image processing method described in any one of claims 1 to 5, or the network model training method described in any one of claims 6 to 7.

11. A machine-readable storage medium, characterized in that The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions prompt the processor to implement the image processing method described in any one of claims 1 to 5, or the network model training method described in any one of claims 6 to 7.

Images