Small sample image semantic segmentation system based on local information index and feature bidirectional alignment

Abstract

The invention discloses a small sample image semantic segmentation system based on local information index and feature bidirectional alignment. The system comprises a feature information extraction module used for extracting feature information of a support data set and a query data set; the local information index module is used for confirming local information indexes of middle-level features and local information indexes of high-level features of the supporting picture data according to the labels of the supporting picture data; and the feature bidirectional alignment module is used for segmenting the support picture data and the query picture data to carry out feature alignment, and segmenting the query features again to obtain a final segmentation result. According to the method, the local information index module is arranged to generate the local information index to activate the target category features in the query features by utilizing the relevance between the local information of the features, so that the feature information can be better utilized, the co-occurrence features are effectively measured, the local information index is more accurately calculated, and the query efficiency is improved. The method has a good image semantic segmentation effect.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition processing, and in particular to a small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment. Background Art

[0002] In the field of computer vision, the task of semantic segmentation of small sample images plays a vital role. The key to this task is to accurately divide different objects or regions in the image. However, traditional methods face many challenges in the case of small samples. These challenges mainly come from the scarcity of training samples, which is caused by the high cost of data collection or the difficulty of annotating data. This limitation affects the performance of traditional methods on small sample images and reduces their accuracy and robustness in semantic segmentation tasks.

[0003] In order to solve the problem of small sample size image segmentation, metric-based meta-learning methods have emerged in recent years. These methods attempt to achieve rapid adaptation to new tasks by learning metric relationships between image features, especially when samples are scarce. By learning from a large number of training tasks, these methods are able to quickly learn and adapt to new semantic segmentation tasks with a small number of annotated examples, thereby improving the performance of small sample size image segmentation.

[0004] However, the current method has the problem of low efficiency in utilizing feature information. This is mainly reflected in the utilization of local information and the accurate measurement of features. The current method has not yet solved how to efficiently utilize local information in the image, but simply splices it and uses the classifier to output the segmentation result. This also leads to the inability to effectively discover co-occurrence features during measurement and improve the performance of semantic segmentation. Summary of the invention

[0005] In view of the above problems, the present invention is proposed to provide a small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment, which overcomes the above problems or at least partially solves the above problems.

[0006] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0007] According to a first aspect of an embodiment of the present invention, a small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment is provided, and the small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment includes: a feature information extraction module, used to extract feature information of a support data set and a query data set, the support data set is a set of support image data with existing labels, and the query data set is a set of query image data;

[0008] A local information index module, used to confirm the local information index of the middle-level features and the local information index of the high-level features of the supporting image data according to the label of the supporting image data;

[0009] The feature bidirectional alignment module is used to segment the supporting image data and the query image data respectively for feature alignment, and then segment the query features again according to the local information index of the middle-level features and the local information index of the high-level features to obtain the final segmentation result.

[0010] In some embodiments of the present invention, the feature information includes mid-level features and high-level features; the feature information extraction module is composed of 5 feature extraction layers connected in sequence, and the supporting image data is sequentially extracted through 5 feature extraction layers connected in sequence to obtain the mid-level features of the supporting images and the high-level features of the supporting images of the supporting data set; the query image data is sequentially extracted through 5 feature extraction layers connected in sequence to obtain the mid-level features of the query images and the high-level features of the query images of the query data set.

[0011] In some embodiments of the present invention, the feature information extraction module extracts middle-level features through the feature extraction layers located at the 3rd and 4th layers, and the feature information extraction module extracts high-level features through the feature extraction layer located at the 5th layer.

[0012] In some embodiments of the present invention, the local information index module is used to:

[0013] Calculate and obtain a first correlation matrix of the mid-level features of the support image and the mid-level features of the query image;

[0014] Calculate the len values ​​with the highest similarity between each local feature in the mid-level features of the query image and the local features of all supporting mid-level features of the image, where len is a positive integer;

[0015] Based on the labels of the supporting image data, confirming that the len values ​​belong to target features or background features;

[0016] Performing weighted calculation on the len values ​​to obtain a first probability that each local feature in the mid-level features of the query image is a foreground feature;

[0017] A local information index of the middle-level feature is obtained according to the first probability.

[0018] In some embodiments of the present invention, the local information index module is used to:

[0019] Calculate and obtain a second correlation matrix of the supporting image high-level features and the query image high-level features;

[0020] Calculate the len values ​​with the highest similarity between each local feature in the high-level features of the query image and all local features supporting the high-level features of the image, where len is a positive integer;

[0021] Based on the labels of the supporting image data, confirming that the len values ​​belong to target features or background features;

[0022] Performing weighted calculation on the len values ​​to obtain a second probability that each local feature in the high-level features of the query image is a foreground feature;

[0023] A local information index of the high-level feature is obtained according to the second probability.

[0024] In some embodiments of the present invention, the feature bidirectional alignment module is specifically used to:

[0025] Calculate the supported prototypes of the target category based on the labels of the supporting image data;

[0026] The support prototype is expanded and spliced ​​with the feature information of the query data set, and a feature measurement tool is used to activate the co-occurrence feature to obtain a segmentation result of the query image data, and then the support image data and the query image data are exchanged to obtain a segmentation result of the support image data, and the loss is calculated using the segmentation result of the query image data and the segmentation result of the support image data to perform feature alignment;

[0027] The query features are segmented again to obtain the final result by combining the local information index of the middle-level features and the local information index of the high-level features.

[0028] The technical solution provided in the embodiments of the present invention has at least the following technical effects or advantages:

[0029] An embodiment of the present invention provides a small sample image semantic segmentation system based on local information index and feature bidirectional alignment. By setting a local information index module, the correlation between local information of features is utilized to generate a local information index to activate target category features in query features, so that feature information can be better utilized. After the normal segmentation process is completed, the feature bidirectional alignment module will swap the support data set and the query data set to complete the segmentation task again and calculate the loss. The model parameters can be optimized, and the co-occurrence features can be bidirectionally adjusted and mapped, so as to achieve effective measurement of co-occurrence features and more accurate calculation of local information indexes, and have a better image semantic segmentation effect.

[0030] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 A schematic diagram of the principle structure of a small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment provided by an embodiment of the present invention;

[0033] Figure 2 Schematic diagram for reference of test results of an embodiment of the present invention. DETAILED DESCRIPTION

[0034] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0035] Various structural schematic diagrams according to embodiments of the present disclosure are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may further design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0036] In the context of the present disclosure, when a layer / element is referred to as being "on" another layer / element, the layer / element may be directly on the other layer / element, or there may be an intervening layer / element therebetween. Additionally, if a layer / element is "on" another layer / element in one orientation, the layer / element may be "below" the other layer / element when the orientation is reversed. In the context of the present disclosure, similar or identical components may be denoted by the same or similar reference numerals.

[0037] In order to better understand the above-mentioned technical scheme, the above-mentioned technical scheme will be described in detail below in combination with specific implementation methods. It should be understood that the embodiments of the present disclosure and the specific features in the embodiments are detailed descriptions of the technical scheme of the present invention, rather than limitations on the technical scheme of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0038] Figure 1 is a schematic diagram of the principle structure of a small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment provided by an embodiment of the present invention, such as Figure 1As shown, the small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment includes:

[0039] A feature information extraction module 100 is used to extract feature information of a support data set and a query data set, wherein the support data set is a set of support image data with labels, and the query data set is a set of query image data;

[0040] A local information indexing module 200, configured to determine the local information index of the middle-level features and the local information index of the high-level features of the supporting image data according to the label of the supporting image data;

[0041] The feature bidirectional alignment module 300 is used to segment the supporting image data and the query image data respectively for feature alignment, and then segment the query feature again according to the local information index of the middle-level feature and the local information index of the high-level feature to obtain the final segmentation result.

[0042] The embodiment of the present invention sets a local information index module 200 to utilize the correlation between the local information of the features and generate a local information index to activate the target category features in the query features, so as to better utilize the feature information. After the normal segmentation process is completed, the feature bidirectional alignment module 300 will swap the support data set and the query data set to complete the segmentation task again and calculate the loss. It can optimize the model parameters, bidirectionally adjust and map the co-occurrence features, effectively measure the co-occurrence features and more accurately calculate the local information index, and has a better image semantic segmentation effect.

[0043] Before performing the image semantic segmentation task, the small sample image semantic segmentation system based on local information index and feature bidirectional alignment in the embodiment of the present invention needs to prepare the data set. Specifically, the data set in the embodiment of the present invention includes a training set D train and the test set D test , the segmentation categories contained in the two do not overlap, that is, Train the model on the training set D train After training, it will be directly transferred to the test set D test The system training of the present invention adopts a meta-learning method and is carried out in a batch training mode. Each batch is a small sample segmentation task, including a support data set S and a query data set Q.

[0044] Specifically, according to the N-way-K-shot setting of the small sample training model, each support dataset S includes K samples with annotations M, that is, The query dataset Q should include 1 query image data X q and the label M which is only used to calculate the loss during training q , Q = {Xq , M q}. The training set after batch sampling can also be expressed as The test set is Where N train and N test are the number of sampling batches during training and testing, respectively. During training, the labels of query images are used to calculate the loss, while the labels of query images in the testing phase are not available.

[0045] In the embodiment of the present invention, the feature information includes middle-level features and high-level features.

[0046] The feature information extraction module 100 is based on a convolutional neural network, which continuously expands the receptive field through layer-by-layer convolution to extract local features of the image. That is, when the receptive field reaches a certain size, each pixel in the feature map can be regarded as an abstraction of a local feature in the original image.

[0047] In the embodiment of the present invention, by drawing on the use of feature extractors in the field of semantic segmentation of small sample images, the feature information extraction module 100 is composed of five feature extraction layers connected in sequence. i ), the supporting picture data is successively passed through 5 sequentially connected feature extraction layers to extract the supporting picture middle-level features and the supporting picture high-level features of the supporting data set, and the query picture data is successively passed through 5 sequentially connected feature extraction layers to extract the query picture middle-level features and the query picture high-level features of the query data set.

[0048] When the supporting image data and the query image data pass through each feature extraction layer in turn, each layer will output the extracted features for the image. When the features are defined by f:

[0049] f i =Layer i (f i-1 )

[0050] In the embodiment of the present invention, the number of channels, height, and width of all feature maps are defined as: c, h, w, and the middle layer feature is defined as f. 3 , f 4 ∈R c×h×w , the high-level feature is f 5 ∈R c×h×w .

[0051] Among them, in the deep learning of convolutional neural networks, the middle-level features contain the common features that constitute the objects. The use of middle-level features can make the segmentation task easier to apply to unseen categories, while the high-level features are more biased towards specific categories.

[0052] The embodiment of the present invention uses the generated middle-level features to obtain support features (including support picture middle-level features and support picture high-level features) and query features (including query picture middle-level features and query picture high-level features) for subsequent calculations:

[0053]

[0054] In the formula, represents the convolution operation, Represents the concatenation operation in the channel dimension; and defines the supported image high-level features and the query image high-level features as

[0055]

[0056] Specifically, the feature information extraction module 100 extracts middle-level features through the feature extraction layers located at the 3rd and 4th layers, and the feature information extraction module 100 extracts high-level features through the feature extraction layer located at the 5th layer.

[0057] In an embodiment of the present invention, the local information index module 200 is based on the attention mechanism, and can activate the query features that co-occur with the annotated objects in the support picture according to the existing annotated support picture, and finally output the local information index to provide support for segmentation. Since the target local features in the query picture are often highly correlated with multiple target local features in the support picture, the local information index module 200 will first calculate the association matrix between the support features and the query features, and calculate the len values ​​(len is a positive integer) with the highest similarity between each local feature in the query feature and the local features of all support features, and then determine whether the above len values ​​belong to the target feature or the background feature according to the label of the support picture. Finally, the len values ​​are weighted and calculated to obtain the probability that each local feature in the query feature is a foreground feature.

[0058] In an embodiment of the present invention, the local information index module 200 is used to: calculate the first association matrix of the mid-level features of the supporting image and the mid-level features of the query image; calculate the len values ​​with the highest similarity between each local feature in the mid-level features of the query image and the local features of all the mid-level features of the supporting images, where len is a positive integer; based on the label of the supporting image data, confirm that the len values ​​belong to the target feature or the background feature; perform weighted calculation on the len values ​​to obtain the first probability that each local feature in the mid-level features of the query image is a foreground feature; and obtain the local information index of the mid-level feature according to the first probability.

[0059] In an embodiment of the present invention, the local information index module 200 is also used to: calculate a second association matrix of the supporting image high-level features and the query image high-level features; calculate the len values ​​with the highest similarity between each local feature in the query image high-level features and the local features of all supporting image high-level features, where len is a positive integer; based on the label of the supporting image data, confirm that the len values ​​belong to the target feature or the background feature; perform weighted calculation on the len values ​​to obtain a second probability that each local feature in the query image high-level features is a foreground feature; and obtain the local information index of the high-level feature based on the second probability.

[0060] For example, the embodiment of the present invention first calculates the corresponding local information index for the middle-level features, and first converts the dimension of the features into:

[0061]

[0062] in, Resizes the input to (c, hw).

[0063] Through matrix multiplication, the embodiment of the present invention calculates the correlation between the supporting features and the query features to obtain the corresponding correlation matrix A:

[0064]

[0065] Any similarity vector a in the association matrix A i ∈R 1×hw ,i∈[1,2,…,hw], can be regarded as the associated value of the corresponding local feature in the query image data to support all local features in the image data.

[0066] Then sort all similarity vectors from small to large and get the sorted local information index:

[0067]

[0068] in, Indicates a sorting operation. Then take out the len values ​​with the highest correlation in each similarity vector and their values ​​in the support image M s The local information index in the tag is briefly recorded as:

[0069] The embodiment of the present invention supports the image M s The category values ​​in a are calculated separately i The len categories with the highest similarity values ​​are assigned, and the average values ​​are calculated for each category:

[0070]

[0071] in, Indicates based on M s The index correspondence of gets the corresponding foreground (1) or background (0) association value len fg It indicates the number of foreground elements indexed.

[0072] The embodiment of the present invention calculates a in the query feature i Local information index indi:

[0073]

[0074] In order to make full use of local information, the local information index module 200 is also applied to high-level features. Similarly, referring to the calculation smoothness of the previous middle-level features, the local information index corresponding to the high-level features is obtained as index. h .

[0075] In an embodiment of the present invention, the feature bidirectional alignment module 300 is specifically used to: calculate the support prototype of the target category based on the label of the support image data; after the support prototype is expanded, it is spliced ​​with the feature information of the query data set, and the co-occurrence feature is activated using a feature measurement tool to obtain a segmentation result of the query image data, and then the support image data and the query image data are exchanged to obtain a segmentation result of the support image data, and the loss is calculated using the segmentation result of the query image data and the segmentation result of the support image data to perform feature alignment; combined with the local information index of the middle-level feature and the local information index of the high-level feature, the query feature is segmented again to obtain the final result.

[0076] In standard small sample image semantic segmentation algorithms, prototypes are usually extracted using labeled support images, and then the prototypes are used to measure co-occurrence features in the query image. Since this method is one-way and does not utilize the information of the query image, the embodiment of the present invention swaps the positions of the support image data and the query image data in the segmentation task, uses the query prototype to measure the support features and calculate the loss, so that the model parameters can be optimized, and the co-occurrence features can be adjusted and mapped bidirectionally to keep them similar in a certain metric space, which is not only conducive to the effective measurement of co-occurrence features, but also conducive to more accurate calculation of local information indexes.

[0077] Specifically, the embodiment of the present invention first calculates the support prototype of the target category based on the label of the support image data:

[0078]

[0079] Among them, P represents the supporting prototype, (x, y) represents the spatial coordinates, and the function δ[.] is an indicator function, which takes 1 when the parameter is true and 0 otherwise. Then, the supporting prototype Ps After expansion, it is directly concatenated with the query feature, and the co-occurrence feature is activated using the feature measurement tool:

[0080]

[0081] in, As a measurement tool, Can expand the input to size (c×h×w).

[0082] Reuse classifier Get the classification results:

[0083]

[0084] Swap the positions of the support dataset and the query dataset to get the output based on the above formula s .

[0085] The local information index is used to index the features of the same category in the query image and activate them to provide a basis for segmentation, as shown in the following formula:

[0086]

[0087] Output of the feature bidirectional alignment module 300: output s ,output q ,output respectively calculate the cross entropy loss function to supervise the optimization process of the model:

[0088]

[0089] in is the cross entropy loss function, the labels have been downsampled to adapt to the feature resolution, and α, β, γ are the loss weights.

[0090] In the embodiment of the present invention, the system M is trained and deployed, and the small sample test task {S i ,Q i}; Use the feature information extraction module 100 to extract multi-layer features of the support data set and the query data set; Use the local information index module 200 to obtain the corresponding local information index; Combine the local information index to obtain the final output output; Use the output to perform accuracy testing and visualization testing; Thereby achieving testing of the training completion system and improving the availability of the system.

[0091] Based on the above embodiment, the embodiment of the present invention also includes a specific embodiment. For example, this example uses a public data set PASCAL-5 iThe experiments were conducted in a 1-way-1-shot setting, using pytorch as the AI ​​model framework.

[0092] The feature information extraction module 100 is improved from the deep convolutional neural network VGG16, and its composition is shown in Table 1. First, this example divides the VGG16 network into 5 layers, and concatenates the middle layer features (the third and fourth layer features) and then reduces the dimension to 256 as the final feature output (f s ,f q ), and the high-level features are kept at 1024 dimensions. During training, the pre-trained parameters of the VGG16 network on the ImageNet dataset are loaded to speed up the training process.

[0093] Table 1 Feature extraction module structure in this implementation example

[0094]

[0095] In the embodiment of the present invention All use convolution with a kernel size of 3, a stride of 1, and a padding of 1. Then use a convolution with a convolution kernel of 1, a stride of 1, a padding of 0, and an output channel number of 2.

[0096] During the training and testing phases, the input image resolution is 473×473 pixels, and the size of the feature map output by the feature extraction module is 1 / 8 of the input image size. During training, the training images are randomly cropped and flipped. The SGD optimizer is used to train the entire system for 6000 iterations, with a learning rate of 0.005, a batch size of 8, a momentum of 0.99, and a weight decay rate of 0.0005. In formula 6-14, α is 1, β is 0.5, and γ is 0.5.

[0097] After the training is completed, the system is tested on a randomly selected small sample segmentation task. The test results are as follows: Figure 2 As shown, Figure 2 This is a reference diagram of the test results of the embodiment of the present invention. Different colors have been used to mark the pictures. The experimental results show that the system trained in this example can achieve accurate semantic segmentation of small sample images, and the proposed innovations are effective. (The baseline model is the model trained after removing the innovations of the present invention)

[0098] Compared with the prior art, the small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment described in the embodiment of the present invention has the following advantages:

[0099] 1. The local information index module 200 uses the attention mechanism to calculate the correlation matrix between features, and then uses the correspondence between the local features of the target category to calculate the local information index;

[0100] 2. When calculating the local information index using the correspondence between the local features of the target category, the index weight of any query local feature is associated with the category with the largest len ​​values ​​of the local feature in support of it;

[0101] 3. Concatenate the local information index with the query feature to index the target category feature in the query feature;

[0102] 4. Setting a feature bidirectional alignment module 300 to complete an additional segmentation task by swapping the positions of the query set and the support set in the small sample segmentation during training, so as to perform bidirectional adjustment and mapping of the co-occurrence features.

[0103] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0104] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0105] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims.

Claims

1. A small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment, characterized in that: The small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment includes: A feature information extraction module, used to extract feature information of a support data set and a query data set, wherein the support data set is a set of support image data with labels, and the query data set is a set of query image data; A local information index module, used to confirm the local information index of the middle-level features and the local information index of the high-level features of the supporting image data according to the label of the supporting image data; The feature bidirectional alignment module is used to segment the supporting image data and the query image data respectively for feature alignment, and then segment the query features again according to the local information index of the middle-level features and the local information index of the high-level features to obtain the final segmentation result.

2. According to claim 1, a small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment is characterized by: The feature information includes middle-level features and high-level features; The feature information extraction module is composed of 5 feature extraction layers connected in sequence. The support picture data is sequentially used to extract the support picture middle-level features and the support picture high-level features of the support data set through the 5 feature extraction layers connected in sequence. The query picture data is sequentially used to extract the query picture middle-level features and the query picture high-level features of the query data set through the 5 feature extraction layers connected in sequence.

3. The small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment according to claim 2, characterized in that: The feature information extraction module extracts middle-level features through the feature extraction layers located at the 3rd and 4th layers, and the feature information extraction module extracts high-level features through the feature extraction layer located at the 5th layer.

4. The small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment according to claim 2, characterized in that: The local information index module is used for: Calculate and obtain a first correlation matrix of the mid-level features of the support image and the mid-level features of the query image; Calculate the len values ​​with the highest similarity between each local feature in the mid-level features of the query image and the local features of all supporting mid-level features of the image, where len is a positive integer; Based on the labels of the supporting image data, confirming that the len values ​​belong to target features or background features; Performing weighted calculation on the len values ​​to obtain a first probability that each local feature in the mid-level features of the query image is a foreground feature; A local information index of the middle-level feature is obtained according to the first probability.

5. The small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment according to claim 4, characterized in that: The local information index module is used for: Calculate and obtain a second correlation matrix of the supporting image high-level features and the query image high-level features; Calculate the len values ​​with the highest similarity between each local feature in the high-level features of the query image and all local features supporting the high-level features of the image, where len is a positive integer; Based on the labels of the supporting image data, confirming that the len values ​​belong to target features or background features; Performing weighted calculation on the len values ​​to obtain a second probability that each local feature in the high-level features of the query image is a foreground feature; A local information index of the high-level feature is obtained according to the second probability.

6. The small sample image semantic segmentation system based on local information indexing and feature bidirectional alignment according to claim 5, characterized in that: The feature bidirectional alignment module is specifically used for: Calculate the supported prototypes of the target category based on the labels of the supporting image data; The support prototype is expanded and spliced ​​with the feature information of the query data set, and a feature measurement tool is used to activate the co-occurrence feature to obtain a segmentation result of the query image data, and then the support image data and the query image data are exchanged to obtain a segmentation result of the support image data, and the loss is calculated using the segmentation result of the query image data and the segmentation result of the support image data to perform feature alignment; The query features are segmented again to obtain the final result by combining the local information index of the middle-level features and the local information index of the high-level features.

Images