Generalized small sample segmentation model training method based on dynamic optimization adjustment and image segmentation method

Abstract

The application provides a generalized small sample segmentation model training method based on dynamic optimization adjustment and an image segmentation method, and relates to the technical field of image processing. The training method comprises the following steps: obtaining a first sample to-be-segmented image; inputting the first sample to-be-segmented image into an image encoder of a trained image segmentation model; obtaining a first type prototype based on the output of the image encoder, the first type prototype corresponding to a first category of the first sample to-be-segmented image; the image segmentation model is trained based on a plurality of second sample to-be-segmented images, and the second sample to-be-segmented images correspond to a second category; updating the first type prototype based on the similarity between the first type prototype and a second type prototype, the second type prototype corresponding to the second category and a background category; and training a classifier of the image segmentation model based on the updated first type prototype. The application can effectively improve the new category segmentation performance of the segmentation model and obtain more accurate image segmentation results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a generalized few-shot segmentation model training method based on dynamic optimization adjustment and an image segmentation method. BACKGROUND

[0002] Semantic segmentation, as an important task in the field of computer vision, aims to assign each pixel of an image to a corresponding semantic class. Traditional methods usually rely on large-scale labeled data and deep convolutional neural networks to obtain high accuracy through supervised learning. However, in practical application scenarios, the cost of obtaining a large number of pixel-level labels is extremely high, which limits the promotion of traditional methods in new classes or long-tail classes.

[0003] To solve the problem of label scarcity, researchers have proposed few-shot segmentation (FSS) technology, which aims to perform pixel-level segmentation on new class targets under the condition of only having a small number of labeled samples. On this basis, generalized few-shot segmentation (GFSS) further proposes the demand to complete segmentation in a mixed scene containing both base classes and new classes. Generalized few-shot segmentation usually includes two training stages: base class learning and new class updating. In the first stage, the model is trained on a large amount of labeled base class data to learn the classification ability of base class pixels. In the second stage, a small amount of labeled new class samples are used to update the model so that it can recognize and segment new class pixels.

[0004] In the existing model training scheme of generalized few-shot segmentation, a fixed class prototype is constructed to alleviate the distribution difference between base classes and new classes. However, there is a lack of dynamic optimization of new class prototypes before segmentation, which leads to insufficient segmentation performance of new classes while ensuring the performance of base classes. SUMMARY

[0005] The present application provides a generalized few-shot segmentation model training method based on dynamic optimization adjustment and an image segmentation method to solve the defect of insufficient segmentation performance of new classes by the model in the prior art and to improve the segmentation performance of new classes by the model.

[0006] The present application provides a generalized few-shot segmentation model training method based on dynamic optimization adjustment, comprising:

[0007] obtaining a first sample image to be segmented, inputting the first sample image to be segmented into an image encoder of a trained image segmentation model, obtaining a first prototype based on an output of the image encoder, the first prototype corresponding to a first category of the first sample image to be segmented, the image segmentation model being trained based on a plurality of second sample images to be segmented, the second sample images to be segmented corresponding to a second category;

[0008] updating the first prototype based on similarity between the first prototype and a second prototype, the second prototype corresponding to the second category and a background category;

[0009] training a classifier of the image segmentation model based on the updated first prototype.

[0010] According to the training method of the generalized small sample segmentation model based on dynamic optimization adjustment provided by the application, the first prototype is updated based on the similarity between the first prototype and the second prototype, which comprises:

[0011] obtaining a similarity matrix between the first prototype and the second prototype;

[0012] determining a similarity weight between the first prototype and the second prototype based on the similarity matrix;

[0013] weighting the normalized second prototype based on the similarity weight to obtain an incremental category prototype;

[0014] updating the first prototype based on the incremental category prototype.

[0015] According to the training method of the generalized small sample segmentation model based on dynamic optimization adjustment provided by the application, the similarity weight between the first prototype and the second prototype is determined based on the similarity matrix, which comprises:

[0016] normalizing the similarity matrix to obtain a normalized similarity matrix;

[0017] amplifying the normalized similarity matrix within a limited range to obtain a temperature hyperparameter;

[0018] scaling and normalizing the similarity matrix based on the temperature hyperparameter to obtain the similarity weight.

[0019] According to the training method of the generalized small sample segmentation model based on dynamic optimization adjustment provided by the application, the first prototype is updated based on the incremental category prototype, which comprises:

[0020] acquire the similarity weight between the first category and each of the second categories and the background category;

[0021] determine a transition weight based on each of the similarity weights;

[0022] update the first prototype based on the transition weight and the delta prototype.

[0023] According to the application, a generalized small sample segmentation model training method based on dynamic optimization adjustment is provided, and the transition weight is determined based on each of the similarity weights, which includes:

[0024] select the maximum similarity weight as the transition weight.

[0025] The application further provides an image segmentation method, which includes:

[0026] acquiring an image to be segmented, and inputting the image to be segmented into an image segmentation model, wherein the image segmentation model is trained based on the generalized small sample segmentation model training method based on dynamic optimization adjustment.

[0027] acquiring an image segmentation result for the image to be segmented output by the image segmentation model.

[0028] The application further provides a generalized small sample segmentation model training device based on dynamic optimization adjustment, which includes:

[0029] an image encoding module, configured to acquire a first sample image to be segmented, input the first sample image to be segmented into an image encoder of a trained image segmentation model, and acquire a first prototype based on an output of the image encoder, wherein the first prototype corresponds to a first category, and the image segmentation model is trained based on a plurality of second sample images to be segmented, and the second sample images to be segmented correspond to a second category;

[0030] a prototype updating module, configured to update the first prototype based on a similarity between the first prototype and a second prototype, wherein the second prototype corresponds to the second category and a background category;

[0031] a model updating module, configured to train a classifier of the image segmentation model based on the updated first prototype.

[0032] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned any one of the generalized small sample segmentation model training method based on dynamic optimization adjustment and / or the image segmentation method when executing the program.

[0033] The application further provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements any of the above-mentioned generalized small sample segmentation model training method based on dynamic optimization adjustment and / or image segmentation method.

[0034] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements any of the above-mentioned generalized small sample segmentation model training method based on dynamic optimization adjustment and / or image segmentation method.

[0035] The application provides a generalized small sample segmentation model training method based on dynamic optimization adjustment and an image segmentation method. In the first stage of the model training process of the generalized small sample segmentation task, the image segmentation model is trained based on base class data (second sample to-be-segmented images) to obtain second class prototypes of the base class and the background class. In the second stage of training, the first class prototype of a new class (first class) is generated according to the output of the image encoder of the image segmentation model obtained in the first stage. Then, the first class prototype is not fixed for training and updating the image segmentation model, but is updated based on the similarity between the first class prototype and the second class prototype, and the classifier in the image segmentation model is trained based on the updated first class prototype. In this way, by introducing dynamic optimization adjustment of the new class prototype in the new class segmentation stage, the stable prototype information of the base class and the background class is fully utilized to improve the discrimination ability of the new class prototype, thereby effectively improving the new class segmentation performance of the segmentation model and obtaining more accurate image segmentation results. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0037] Figure 1 is a flowchart of the generalized small sample segmentation model training method based on dynamic optimization adjustment provided by the present application.

[0038] Figure 2 is a schematic diagram of the data processing process in the generalized small sample segmentation model training method based on dynamic optimization adjustment provided by the present application.

[0039] Figure 3 is a structural schematic diagram of the generalized small sample segmentation model training device based on dynamic optimization adjustment provided by the present application.

[0040] Figure 4It is a structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0041] To make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0042] The application provides a generalized small sample segmentation model training method based on dynamic optimization adjustment, and the method comprises the steps of: Figures 1-2 The application provides a generalized small sample segmentation model training method based on dynamic optimization adjustment, and the method comprises the steps of: Figure 1 The application provides a generalized small sample segmentation model training method based on dynamic optimization adjustment, and the method comprises the steps of:

[0043] S110, a first sample to-be-segmented image is acquired, and the first sample to-be-segmented image is input into an image encoder of a trained image segmentation model, a first-class prototype is obtained based on the output of the image encoder, the first-class prototype corresponds to a first class, the image segmentation model is trained based on a plurality of second sample to-be-segmented images, and the second sample to-be-segmented images correspond to a second class;

[0044] S120, the first-class prototype is updated based on the similarity between the first-class prototype and a second-class prototype, and the second-class prototype corresponds to the second class and a background class;

[0045] S130, a classifier of the image segmentation model is trained based on the updated first-class prototype.

[0046] The method provided by the application, in the first stage of the model training process of the generalized small sample segmentation task, the image segmentation model is first trained based on the base class data (the second sample to-be-segmented image), the second-class prototype of the base class and the background class is obtained, in the second stage of training, the first-class prototype of a new class (the first class) is generated according to the output of the image encoder of the image segmentation model obtained in the first stage, then the first-class prototype is not fixed to update the training of the image segmentation model, but the first-class prototype is updated based on the similarity between the first-class prototype and the second-class prototype, and the classifier in the image segmentation model is trained based on the updated first-class prototype, in this way, by introducing the dynamic optimization adjustment of the new class prototype in the new class segmentation stage, the stable prototype information of the base class and the background class is fully utilized, the discrimination ability of the new class prototype is improved, the new class segmentation performance of the segmentation model is effectively improved, and a more accurate image segmentation result is obtained.

[0047] In the generalized small sample segmentation (GFSS) task, it is assumed that a data set contains one base class and one new class, and each new class has annotated images. A large number of annotated images are received in the base class stage for training, and a small number of samples are used in the new class stage to complete model adaptation. The goal of GFSS is to distinguish categories (including the background class) at the inference stage. The model outputs a probability distribution of the corresponding category for each pixel.

[0048] The method provided by the application comprises two parts: an image encoder and a classifier. The image encoder is used to extract the features of the input image, and the classifier is used to classify each pixel in the image based on the features and class prototypes, thereby realizing image segmentation. The class prototypes are determined based on the features output by the image encoder during the training process of the image segmentation model, and different categories correspond to different class prototypes. During the training process of the image segmentation model, the parameters of the image encoder and the classifier and the class prototypes need to be trained.

[0049] The method provided by the application comprises two parts: an image encoder and a classifier. The image encoder is used to extract the features of the input image, and the classifier is used to classify each pixel in the image based on the features and class prototypes, thereby realizing image segmentation. The class prototypes are determined based on the features output by the image encoder during the training process of the image segmentation model, and different categories correspond to different class prototypes. During the training process of the image segmentation model, the parameters of the image encoder and the classifier and the class prototypes need to be trained.

[0050] Specifically, the first sample to-be-segmented image is a new class image, that is, the category of the object in the first sample to-be-segmented image is different from the category of the object in the second sample to-be-segmented image. The first sample to-be-segmented image is input into the image encoder, and the image encoder outputs the features of the first sample to-be-segmented image. Based on the features, the first class prototype of the first category can be obtained. Figure 2 As shown in the figure, there can be multiple first categories, and each first category corresponds to a first class prototype. There can also be multiple second categories, and each second category and the background category corresponds to a second class prototype. For convenience of description, all the first class prototypes can be represented as a matrix All the second class prototypes can be represented as a matrix where R represents the set of real numbers, n is the number of first categories, n is the number of second categories, and n is the feature dimension.

[0051] In the prior art, after determining the first-class prototype based on the features output by the image encoder, the first-class prototype is directly used for training. Since the image encoder mainly learns base-class features in the first training stage, the new-class prototype representation is unstable and has a large overlap with the base-class and background features. Moreover, due to the limited number of samples of the new class, it is difficult to form sufficient discrimination, and in a multi-class mixed scene, it is easy to be misjudged as the background or base class. The method provided in the application dynamically optimizes the class prototype of the new class based on the prototype information of the base class and the background class. Specifically, the similarity between the first-class prototype and the second-class prototype is updated based on the similarity between the first-class prototype and the second-class prototype, including:

[0052] obtaining a similarity matrix between the first-class prototype and the second-class prototype;

[0053] determining a similarity weight between the first-class prototype and the second-class prototype based on the similarity matrix;

[0054] performing weighted processing on the normalized first-class prototype based on the similarity weight to obtain an incremental class prototype;

[0055] updating the first-class prototype based on the incremental class prototype.

[0056] The similarity matrix between the first-class prototype and the second-class prototype can be calculated by using the cosine similarity calculation method. In one possible implementation manner, the cosine similarity calculation can be directly performed based on the first-class prototype and the second-class prototype, or the first-class prototype and the second-class prototype can be normalized first and then the cosine similarity calculation is performed. The calculation process of the similarity matrix can be represented by a formula as follows: wherein, is the similarity matrix, represents the normalization of In the feature dimension, the normalization is performed, and T represents the matrix transposition.

[0057] The first-class prototype includes class prototypes of each first-class, and the second-class prototype includes class prototypes of each second-class and a background class. As can be seen, the similarity matrix includes the similarity between the class prototype of each first-class and the class prototype of each second-class and the background class. In one possible implementation manner, the similarity weight between the first-class prototype and the second-class prototype can be determined based on the similarity matrix, that is, the class prototype of the first-class is normalized with the class prototype of each second-class and the background class to obtain the similarity weight of the class prototype of the first-class with the class prototype of each second-class and the background class. The formula can be represented as follows:

[0058] ;

[0059] wherein, represents a similarity weight vector of the first category i, which includes similarity weights of the first category i with the class prototypes of the first category and the background category respectively, represents a similarity vector corresponding to the class prototype of the first category i, which includes similarities between the class prototype of the first category i and each of the second categories and the background category, and is the i-th row in the similarity matrix.

[0060] In another possible implementation, the similarity weight between the first class prototype and the second class prototype is determined based on the similarity matrix, including:

[0061] The similarity matrix is normalized to obtain a normalized similarity matrix;

[0062] The normalized similarity matrix is amplified within a limited range to obtain a temperature hyperparameter;

[0063] The similarity matrix is scaled and normalized based on the temperature hyperparameter to obtain the similarity weight.

[0064] In this implementation, a temperature hyperparameter T is introduced, The similarity matrix is scaled and normalized by using the temperature hyperparameter to control the smoothness of the similarity distribution, which can be specifically expressed as:

[0065] ;

[0066] where W represents the similarity weight matrix, and the element represents the similarity weight between the first category i and the second category or the background category j. By using division, numerical instability in the exponential operation process can be avoided, and multiplication is prone to numerical overflow, which can cause softmax to lose control, causing all weights to tend to 0 or 1.

[0067] A fixed temperature hyperparameter T is prone to cause smooth or excessively sharp weight distribution, affecting the feature fusion effect. In the method provided by the present application, T is dynamically adjusted according to local feature changes, which can be specifically expressed as: wherein represents a normalization operation on represents a range limiting operation, that is, when the result of exceeds the limited range, it is corrected to be within the limited range.

[0068] Finally, the similarity weight of the first category i is obtained based on T and S:

[0069] .

[0070] ​This adaptive scaling can smooth the uncertain prototype distribution, achieve clearer alignment when similarity is strong, and thus prevent overconfidence caused by noise or ambiguous features.

[0071] The normalized second-class prototype is weighted based on the similarity weight to obtain an incremental class prototype, which can be expressed by a formula as follows: , The incremental class prototype is denoted as Pnew.

[0072] The first-class prototype is updated based on the incremental class prototype, including:

[0073] The similarity weight between the first class and each of the first class and the background class is obtained;

[0074] The transition weight is determined based on each similarity weight;

[0075] The first-class prototype is updated based on the transition weight and the incremental class prototype.

[0076] Further, in the method provided by the application, in order to realize fine-grained adjustment for different classes and feature distributions and avoid overfitting or oversmoothing, the new class prototype is updated by interpolation, which can be expressed by a formula as follows:

[0077] ;

[0078] wherein, is a transition weight for controlling the proportion of reserved new class feature representation and fused base class feature representation, The updated new class prototype is denoted as Pnew.

[0079] By using the prototype interpolation based on similarity weighting, the discriminability of the new class is enhanced while avoiding imposing too strong constraints on the feature space, so as to ensure the generalization ability. The model has stronger adaptability and generalization performance in a multi-class mixed scene, and prevents the problem of insufficient expression of new class features caused by too strong constraints.

[0080] Further, the transition weight is determined based on each similarity weight, including:

[0081] The maximum similarity weight is selected as the transition weight.

[0082] For the similarity difference of the class prototype of different new classes and the base class and the background class, an adaptive strategy based on the maximum weight is designed in the method provided by the application. Specifically, it can be expressed by a formula as follows:

[0083] ;

[0084] wherein is the basic range parameter. When a new prototype can be reliably aligned with a specific base class, this formula will increase the impact of the corresponding category, otherwise it will reduce the impact.

[0085] The method provided by the application adjusts the Softmax temperature parameter based on the similarity, and combines the offset weight based on the maximum alignment confidence, so as to realize fine-grained adjustment for different categories and feature distributions and avoid overfitting or oversmoothing. The new class prototype is dynamically optimized by the base class and the background class prototype, and is updated by similarity weight calculation and interpolation, so that the new class prototype enhances the discrimination ability while maintaining semantic independence.

[0086] As can be seen from the foregoing description, the method provided by the application does not need to modify the backbone network result of the model or retrain, and therefore, the method provided by the application can be designed in a plug-and-play form and can be directly integrated into various existing small sample segmentation frameworks and backbone networks without greatly adjusting the original architecture. This feature improves the engineering practicability and adaptability of the method, reduces the deployment cost of the method provided by the application, and has strong industrial value.

[0087] Based on the generalized small sample segmentation model training method based on dynamic optimization adjustment provided by the application, the application further provides an image segmentation method, comprising:

[0088] obtaining an image to be segmented, and inputting the image to be segmented into an image segmentation model;

[0089] obtaining an image segmentation result for the image to be segmented output by the image segmentation model.

[0090] The image segmentation model is trained by the generalized small sample segmentation model training method based on dynamic optimization adjustment provided by the application.

[0091] The generalized small sample segmentation model training device based on dynamic optimization adjustment provided by the application is described below, and the generalized small sample segmentation model training device based on dynamic optimization adjustment described below can be correspondingly referred to the generalized small sample segmentation model training method based on dynamic optimization adjustment described above. As Figure 3 shown, the generalized small sample segmentation model training device based on dynamic optimization adjustment provided by the application comprises:

[0092] ​The image coding module 310 is configured to obtain a first sample image to be segmented, input the first sample image to be segmented into an image encoder of a trained image segmentation model, obtain a first prototype based on an output of the image encoder, the first prototype corresponding to a first category of the first sample image to be segmented, and the image segmentation model being trained based on a plurality of second sample images to be segmented corresponding to a second category.

[0093] The prototype updating module 320 is configured to update the first prototype based on a similarity between the first prototype and a second prototype corresponding to the second category and a background category.

[0094] The model updating module 330 is configured to train a classifier of the image segmentation model based on the updated first prototype.

[0095] Figure 4 An example of an entity structure diagram of an electronic device is shown in FIG. 4. Figure 4 As shown in FIG. 4, the electronic device can include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 can communicate with each other through the communications bus 440. The processor 410 can invoke a logical instruction in the memory 430 to execute a generalized small sample segmentation model training method based on dynamic optimization adjustment and / or an image segmentation method. The generalized small sample segmentation model training method based on dynamic optimization adjustment includes obtaining a first sample image to be segmented, inputting the first sample image to be segmented into an image encoder of a trained image segmentation model, obtaining a first prototype based on an output of the image encoder, the first prototype corresponding to a first category of the first sample image to be segmented, and the image segmentation model being trained based on a plurality of second sample images to be segmented corresponding to a second category. The first prototype is updated based on a similarity between the first prototype and a second prototype corresponding to the second category and a background category. The classifier of the image segmentation model is trained based on the updated first prototype. The image segmentation method includes obtaining an image to be segmented, inputting the image to be segmented into the image segmentation model, and obtaining an image segmentation result for the image to be segmented output by the image segmentation model, wherein the image segmentation model is trained by the generalized small sample segmentation model training method based on dynamic optimization adjustment provided by the present application.

[0096] In addition, the logic instructions in the memory 430 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0097] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the general small sample segmentation model training method based on dynamic optimization adjustment and / or the image segmentation method provided by the above-mentioned methods. The general small sample segmentation model training method based on dynamic optimization adjustment comprises: obtaining a first sample to-be-segmented image, inputting the first sample to-be-segmented image into an image encoder of a trained image segmentation model, obtaining a first prototype based on the output of the image encoder, the first prototype corresponding to a first category of the first sample to-be-segmented image, the image segmentation model being trained based on a plurality of second sample to-be-segmented images, the second sample to-be-segmented image corresponding to a second category; updating the first prototype based on the similarity between the first prototype and a second prototype, the second prototype corresponding to the second category and a background category; and training a classifier of the image segmentation model based on the updated first prototype. The image segmentation method comprises: obtaining a to-be-segmented image, inputting the to-be-segmented image into the image segmentation model; and obtaining an image segmentation result for the to-be-segmented image output by the image segmentation model, wherein the image segmentation model is trained by the general small sample segmentation model training method based on dynamic optimization adjustment provided by the present application.

[0098] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the method for training a generalized small sample segmentation model based on dynamic optimization adjustment and / or the image segmentation method provided by the above method, the method for training a generalized small sample segmentation model based on dynamic optimization adjustment comprising: obtaining a first sample image to be segmented, inputting the first sample image to be segmented into an image encoder of a trained image segmentation model, obtaining a first prototype based on an output of the image encoder, the first prototype corresponding to a first category, the image segmentation model being trained based on a plurality of second sample images to be segmented, the second sample images to be segmented corresponding to a second category; updating the first prototype based on a similarity between the first prototype and a second prototype, the second prototype corresponding to the second category and a background category; and training a classifier of the image segmentation model based on the updated first prototype. The image segmentation method comprises: obtaining an image to be segmented, inputting the image to be segmented into the image segmentation model; and obtaining an image segmentation result for the image to be segmented output by the image segmentation model, wherein the image segmentation model is trained by the method for training a generalized small sample segmentation model based on dynamic optimization adjustment provided by the present application.

[0099] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0100] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in terms of the contribution to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0101] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for training a generalized small sample segmentation model based on dynamic optimization adjustment, characterized in that, The method comprises: obtaining a first sample image to be segmented, inputting the first sample image to be segmented into an image encoder of a trained image segmentation model, obtaining a first prototype based on an output of the image encoder, the first prototype corresponding to a first category of the first sample image to be segmented, the image segmentation model being trained based on a plurality of second sample images to be segmented, the second sample images to be segmented corresponding to a second category; updating the first prototype based on similarity between the first prototype and a second prototype corresponding to the second category and a background category; training a classifier of the image segmentation model based on the updated first prototype; the updating the first prototype based on the similarity between the first prototype and the second prototype comprises: obtaining a similarity matrix between the first prototype and the second prototype; determining a similarity weight between the first prototype and the second prototype based on the similarity matrix; weighting the normalized second prototype based on the similarity weight to obtain an incremental category prototype; updating the first prototype based on the incremental category prototype; the updating the first prototype based on the incremental category prototype comprises: obtaining the similarity weight between the first category and each of the second categories and the background category; determining a transition weight based on each of the similarity weights; updating the first prototype based on the transition weight and the incremental category prototype.

2. The generalized small sample partition model training method based on dynamic optimization adjustment according to claim 1, characterized in that, the determining the similarity weight between the first prototype and the second prototype based on the similarity matrix comprises: normalizing the similarity matrix to obtain a normalized similarity matrix; performing amplification within a limited range on the normalized similarity matrix to obtain a temperature hyperparameter; scaling and normalizing the similarity matrix based on the temperature hyperparameter to obtain the similarity weight.

3. The generalized small sample partition model training method based on dynamic optimization adjustment according to claim 1, characterized in that, the determining the transition weight based on each of the similarity weights comprises: selecting the maximum similarity weight as the transition weight.

4. An image segmentation method characterized by, The method comprises: obtaining an image to be segmented, inputting the image to be segmented into an image segmentation model, the image segmentation model being trained based on the generalized small sample segmentation model training method based on dynamic optimization adjustment of any one of claims 1-3; obtaining an image segmentation result for the image to be segmented output by the image segmentation model.

5. A generalized small sample segmentation model training device based on dynamic optimization adjustment, characterized in that, The device comprises: an image encoding module configured to obtain a first sample image to be segmented, input the first sample image to be segmented into an image encoder of a trained image segmentation model, obtain a first prototype based on an output of the image encoder, the first prototype corresponding to a first category of the first sample image to be segmented, the image segmentation model being trained based on a plurality of second sample images to be segmented, the second sample images to be segmented corresponding to a second category; The class prototype updating module is configured to update the first class prototype based on the similarity between the first class prototype and a second class prototype corresponding to the second class and the background class. The model updating module is configured to train the classifier of the image segmentation model based on the updated first class prototype. The updating of the first class prototype based on the similarity between the first class prototype and a second class prototype includes: obtaining a similarity matrix between the first class prototype and the second class prototype; determining a similarity weight between the first class prototype and the second class prototype based on the similarity matrix; performing weighted processing on the normalized second class prototype based on the similarity weight to obtain an incremental class prototype; updating the first class prototype based on the incremental class prototype; The updating of the first class prototype based on the incremental class prototype includes: obtaining the similarity weight between the class prototype of the first class and each of the second classes and the background class; determining a transition weight based on each of the similarity weights; updating the first class prototype based on the transition weight and the incremental class prototype.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the generalized small sample segmentation model training method based on dynamic optimization adjustment according to any one of claims 1 to 3 and / or the image segmentation method according to claim 4.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the generalized small sample segmentation model training method based on dynamic optimization adjustment according to any one of claims 1 to 3 and / or the image segmentation method according to claim 4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the generalized small sample segmentation model training method based on dynamic optimization adjustment according to any one of claims 1 to 3 and / or the image segmentation method according to claim 4.

Images