Text-guided camouflage instance segmentation method and device, equipment and storage medium

Abstract

The invention discloses a text-guided camouflage instance segmentation method and device, equipment and a storage medium, and relates to the technical field of images, and the method comprises the steps: generating a pseudo tag of each sample camouflage instance according to a foreground text prompt of each sample camouflage instance and a camouflage region of each sample camouflage instance, obtaining a plurality of sample training images through the pseudo labels of the sample camouflage instances, wherein the sample camouflage instances are located in the sample camouflage images; evaluating the camouflage degree of each sample training image according to a camouflage measurer; according to the camouflage scheduler, the camouflage degree of each sample training image and each sample training image, obtaining a plurality of training batches and training information of each training batch, and training the instance segmentation model to obtain a target segmentation model; and segmenting the target disguise image through the target segmentation model. Through the above mode, the accuracy, robustness and generalization ability of the model in processing the disguise instance segmentation task are significantly improved, and the dependence on the annotation data is reduced.

Description

Technical Field

[0001] The present application relates to the field of image technology, and in particular to a text-guided disguised instance segmentation method, apparatus, device, and storage medium. Background Art

[0002] Camouflaged instance segmentation is a task focused on detecting and segmenting objects that blend in with their surroundings. This technology has important applications in a variety of fields, including wildlife conservation, medical image segmentation, and industrial defect detection. With the development of deep learning, especially transformer architectures, camouflaged instance segmentation has made significant progress under fully supervised learning methods. However, these fully supervised learning methods have some significant drawbacks. First, they require a large amount of accurately labeled data, which is not only costly but also very difficult to accurately label when the camouflaged objects are highly similar to the background. Second, existing technologies still face challenges when dealing with highly camouflaged objects, especially in terms of accuracy and generalization ability. In addition, existing methods often lack sufficient flexibility and adaptability to cope with segmentation tasks in new scenes or new objects, which limits their application in a wider range of environments. Summary of the Invention

[0003] The main purpose of this application is to provide a text-guided disguised instance segmentation method, device, equipment and storage medium, aiming to solve the technical problem that the existing disguised instance segmentation method requires relying on a large amount of accurately labeled data while having poor accuracy and generalization ability.

[0004] To achieve the above objectives, the present application proposes a text-guided disguised instance segmentation method, which includes:

[0005] generating a pseudo label for each sample disguised instance according to a foreground text prompt of each sample disguised instance and a disguised region of each sample disguised instance, and obtaining a plurality of sample training images using the pseudo label of each sample disguised instance, wherein the sample disguised instance is located in the sample disguised image;

[0006] Evaluate the camouflage degree of each sample training image according to the camouflage metric;

[0007] Dividing each sample training image according to the disguise scheduler and the disguise degree of each sample training image to obtain multiple training batches and training information of each training batch;

[0008] The instance segmentation model is trained according to the training information of each training batch and the training batch to obtain the target segmentation model;

[0009] The target disguised image to be segmented is input into the target segmentation model to obtain a segmentation result of the disguised instances in the target disguised image.

[0010] In one embodiment, before the step of generating a pseudo label for each disguised sample instance based on the foreground text prompt of each disguised sample instance and the disguised region of each disguised sample instance, the step further includes:

[0011] Input each sample disguised image into the target object detection model for region recognition to obtain the disguised region of each sample disguised instance;

[0012] Generate multiple disguise category hints based on the generative pre-trained model;

[0013] The disguised area of ​​each sample disguised instance is matched with each disguised category prompt to obtain the foreground text prompt of each sample disguised instance.

[0014] In one embodiment, the step of matching the disguised area of ​​each sample disguised instance with each disguised category prompt to obtain the foreground text prompt of each sample disguised instance includes:

[0015] Determine the image feature vector of the disguised area of ​​each sample disguised instance and the text feature vector of each disguised category hint through the image-text matching model;

[0016] Calculate the cosine similarity between the image feature vector of the disguised area of ​​each sample disguised instance and each text feature vector;

[0017] The foreground text prompt of each sample disguised instance is determined from multiple disguised category prompts according to the cosine similarity calculation result.

[0018] In one embodiment, the step of generating a pseudo label for each disguised sample instance based on the foreground text prompt of each disguised sample instance and the disguised region of each disguised sample instance includes:

[0019] Generate a semantic mask for each disguised sample instance based on a semantic mask generator, a foreground text hint of each disguised sample instance, and a disguised region of each disguised sample instance, wherein the semantic mask generator is constructed based on a gradient class activation map;

[0020] Generate a spatial mask for each sample camouflage instance based on the spatial mask generator and the camouflage area of ​​each sample camouflage instance;

[0021] The spatial mask and semantic mask of each disguised instance are iteratively refined according to the iterative mask fuser to generate a pseudo label for each disguised instance.

[0022] In one embodiment, the step of generating a spatial mask for each sample camouflage instance based on the spatial mask generator and the camouflage area of ​​each sample camouflage instance includes:

[0023] Generate multiple patches corresponding to the camouflaged area of ​​each sample camouflage instance and key features of each patch according to the target object detection model;

[0024] Calculating patch affinity between patches in each camouflaged area based on key features of each patch in each camouflaged area;

[0025] A spatial mask of each sample camouflage instance is generated based on the patch affinity between patches in each camouflaged region and the normalized cut algorithm.

[0026] In one embodiment, the step of iteratively refining the spatial mask and the semantic mask of each disguised sample instance according to the iterative mask fuser to generate a pseudo label for each disguised sample instance includes:

[0027] Calculate the similarity score ratio of each disguised sample instance based on the image-text matching model and the semantic mask of each disguised sample instance;

[0028] Calculating the angular energy distribution ratio and radial energy distribution ratio of each sample camouflage instance according to the spatial mask of each sample camouflage instance;

[0029] Iterative refinement is performed based on the similarity score ratio, angular energy distribution ratio, radial energy distribution ratio and iterative mask fuser of each sample disguised instance to generate a pseudo label for each sample disguised instance.

[0030] In one embodiment, the dividing of the sample training images according to the disguise scheduler and the disguise degree of each sample training image to obtain multiple training batches and training information of each training batch includes:

[0031] Sort the camouflage degree of each sample training image to obtain a ranking result;

[0032] Dividing each sample training image according to the sorting result and the camouflage scheduler to obtain multiple training batches, a training order for each training batch, and a training gradient for each training batch;

[0033] The training information of each training batch is obtained according to the training order of each training batch and the training gradient of each training batch.

[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes a text-guided disguised instance segmentation device, which includes:

[0035] a generation module, configured to generate a pseudo label for each sample disguised instance based on a foreground text prompt of each sample disguised instance and a disguised region of each sample disguised instance, and obtain a plurality of sample training images using the pseudo label of each sample disguised instance, wherein the sample disguised instance is located in the sample disguised image;

[0036] An evaluation module, configured to evaluate the degree of camouflage of each sample training image based on a camouflage metric;

[0037] A division module is used to divide each sample training image according to the disguise scheduler and the disguise degree of each sample training image to obtain multiple training batches and training information of each training batch;

[0038] A training module is used to train the instance segmentation model according to the training information of each training batch and the training batch to obtain a target segmentation model;

[0039] The segmentation module is used to input the target disguised image to be segmented into the target segmentation model to obtain a segmentation result of the disguised instances in the target disguised image.

[0040] In addition, to achieve the above-mentioned objectives, the present application also proposes a text-guided disguised instance segmentation device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the text-guided disguised instance segmentation method as described above.

[0041] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. When the computer program is executed by a processor, it implements the steps of the text-guided disguised instance segmentation method as described above.

[0042] In addition, to achieve the above-mentioned object, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the text-guided disguised instance segmentation method as described above.

[0043] This application generates pseudo labels for each sample disguised instance based on the foreground text prompt and the disguised area of ​​each sample disguised instance, and obtains multiple sample training images through the pseudo labels of each sample disguised instance, wherein the sample disguised instance is located in the sample disguised image; evaluates the disguise degree of each sample training image based on a disguise metric; divides each sample training image based on a disguise scheduler and the disguise degree of each sample training image to obtain multiple training batches and training information for each training batch; trains an instance segmentation model based on the training information of each training batch and the training batches to obtain a target segmentation model; inputs the target disguised image to be segmented into the target segmentation model to obtain a segmentation result of the disguised instance present in the target disguised image. Through the above method, based on weakly supervised learning, the accuracy and robustness of the model in processing the disguised instance segmentation task are significantly improved, and it can be applied to various complex scenarios, with sufficient flexibility and generalization, while maintaining low requirements for training data and reducing dependence on accurately labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 A flowchart of the first embodiment of the method for segmenting disguised instances guided by the text of this application is provided;

[0047] Figure 2 A flowchart of the second embodiment of the method for segmenting disguised instances guided by the text of this application is provided;

[0048] Figure 3 A schematic diagram of a simplified flow chart of the text-guided disguised instance segmentation method provided in the second embodiment of the present application;

[0049] Figure 4 This is a schematic diagram of the module structure of the text-guided disguised instance segmentation device according to an embodiment of the present application;

[0050] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the text-guided disguised instance segmentation method in the embodiment of the present application.

[0051] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0053] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0054] The main solution of the embodiment of the present application is: based on the foreground text prompt of each sample disguised instance and the disguised area of ​​each sample disguised instance, a pseudo label is generated for each sample disguised instance, and multiple sample training images are obtained through the pseudo label of each sample disguised instance, wherein the sample disguised instance is located in the sample disguised image; the disguise degree of each sample training image is evaluated according to a disguise metric; each sample training image is divided according to a disguise scheduler and the disguise degree of each sample training image to obtain multiple training batches and training information for each training batch; an instance segmentation model is trained according to the training information of each training batch and the training batches to obtain a target segmentation model; the target disguised image to be segmented is input into the target segmentation model to obtain a segmentation result of the disguised instance present in the target disguised image.

[0055] With the development of deep learning, especially transformer architectures, camouflaged instance segmentation has made significant progress under fully supervised learning methods. However, these fully supervised learning methods have some significant drawbacks. First, they require a large amount of accurately labeled data, which is not only costly but also very difficult to accurately label when the camouflaged objects are highly similar to the background. Second, existing technologies still face challenges when dealing with highly camouflaged objects, especially in terms of accuracy and generalization ability. In addition, existing methods often lack sufficient flexibility and adaptability, and have difficulty in coping with segmentation tasks of new scenes or new objects, which limits their application in a wider range of environments. Therefore, there is an urgent need for a new technology that can reduce the dependence on accurately labeled data while improving the segmentation accuracy and robustness of complex camouflaged scenes.

[0056] While these segmentation methods based on memory networks and feature matching achieve good segmentation results, they all update the memory network at fixed intervals. This makes it difficult to fully adjust the memory based on the changing characteristics of the objects in the video, and it is easy to lose key features that appear during the object's transformation, which is crucial for effectively segmenting objects that undergo significant changes. Furthermore, because it cannot accurately judge the degree of object change, the memory network cannot promptly integrate redundant features when the foreground object changes slightly, nor can it specifically enhance key features that appear when the foreground object changes significantly, making it difficult to effectively segment target objects in complex scene videos.

[0057] This application provides a solution based on weakly supervised learning, which significantly improves the accuracy and robustness of the model in processing disguised instance segmentation tasks. It can be applied to various complex scenarios, has sufficient flexibility and generalization, while maintaining low requirements for training data and reducing dependence on accurately labeled data.

[0058] It should be noted that the execution entity of this embodiment can be a computing service device with data processing, network communication, and program execution capabilities, such as a tablet computer, personal computer, or mobile phone, or a text-guided disguised instance segmentation device capable of performing the aforementioned functions. This embodiment and the following embodiments will be described below using a text-guided disguised instance segmentation device as the execution entity.

[0059] Based on this, the embodiment of the present application provides a text-guided disguised instance segmentation method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the disguised instance segmentation method guided by the present application text.

[0060] In this embodiment, the text-guided disguised instance segmentation method includes steps S10 to S50:

[0061] Step S10: generating a pseudo label for each sample disguised instance according to the foreground text prompt and the disguised area of ​​each sample disguised instance, and obtaining a plurality of sample training images through the pseudo label of each sample disguised instance, wherein the sample disguised instance is located in the sample disguised image.

[0062] It should be noted that this embodiment involves a weakly supervised instance segmentation framework based on text cues. It aims to improve the segmentation accuracy of disguised objects by combining visual and semantic information. The framework primarily comprises the following modules: a text-cue image matching module, a semantic space iterative fusion module, and a gradual camouflage learning mechanism. The text-cue matching module takes an image containing a disguised object as input, identifies the camouflaged regions using a target object detection model, and uses the entire image as the region prediction. The text branch generates diverse camouflage category cues using a generative pre-trained model. The image-text matching model encodes and calculates cosine similarity between the matching regions and the text, generating the camouflaged region and the corresponding foreground text cue for a single disguised instance. The semantic space iterative fusion module combines semantic and spatial information through an iterative process to improve segmentation accuracy. The semantic space iterative fusion module uses the semantic mask generated by the text cues and the image-based spatial mask to evaluate and adjust the fusion of the two to generate a more refined pseudo-mask. This approach effectively enhances the recognition of disguised objects. The gradual camouflage learning mechanism evaluates the degree of camouflage in the image, ranks training samples by camouflage level, and dynamically adjusts the training difficulty based on this ranking. The step-by-step camouflage learning mechanism consists of two key components: a camouflage metric and a camouflage scheduler. The metric is responsible for evaluating the level of image camouflage, while the scheduler adjusts the selection of training samples and the gradient of the loss function based on the evaluation results.

[0063] It can be understood that the text prompt-based weakly supervised instance segmentation framework of this embodiment is used to train the instance segmentation model, and the trained target segmentation model can perform disguised instance segmentation on disguised images in various complex scenarios.

[0064] In a specific implementation, a sample disguised instance (i.e., a sample disguised object) is located in a sample disguised image. A sample disguised image may contain one or more sample disguised instances. The framework's text-cue image matching module performs region recognition on each sample disguised image, obtains the disguised region containing the sample disguised instance, and assigns the foreground text cue corresponding to the sample disguised instance.

[0065] It should be noted that the framework's semantic space iterative fusion module, combined with the given foreground text hint and camouflaged region of each sample camouflaged instance, can generate a more refined mask for the sample camouflaged instance. In this embodiment, the pseudo-label of the sample camouflaged instance refers to the more refined mask of the sample camouflaged instance generated by the semantic space iterative fusion module.

[0066] It is understandable that after obtaining the pseudo labels of the sample disguised instances, the sample disguised images containing the pseudo labels of the sample disguised instances are used as sample training images for subsequent training of the instance segmentation model.

[0067] In a feasible implementation manner, before step S10, steps A11 to A13 may also be included:

[0068] In step A11, each sample disguised image is input into a target object detection model for region recognition to obtain a disguised region of each sample disguised instance.

[0069] Step A12: Generate multiple disguise category prompts based on the generative pre-trained model.

[0070] In step A13, the disguised area of ​​each sample disguised instance is matched with each disguised category prompt to obtain a foreground text prompt of each sample disguised instance.

[0071] It should be noted that the target object detection model refers to a model that can identify camouflaged areas in an image. In this embodiment, the target object detection model is an object detection model based on DINO (DETR with Improved Object Queries). DINO is an object detection model based on DETR (DEtection TRansformer). It improves the detection performance by improving the object query (Object Queries) mechanism in DETR. Other object detection models can also be used, and this embodiment does not limit this.

[0072] It can be understood that a sample disguised image containing a disguised object is input into the text prompt image matching module, the target object detection model in the text prompt image matching module is used to identify the disguised area, and the entire image is used as the area prediction, thereby obtaining the disguised area corresponding to the sample disguised instance existing in the sample disguised image.

[0073] In a specific implementation, the text branch in the text prompt image matching module generates a variety of disguise category prompts through a generative pre-trained model. In this embodiment, the generative pre-trained model can be GPT (Generative Pre-trained Transformer) or other models, which is not limited in this embodiment.

[0074] It should be noted that for each sample disguised instance, the text prompt image matching module calculates the similarity between the disguised area and multiple disguised category prompts, obtains the similarity between the disguised area and each disguised category prompt, and selects the disguised category prompt that is similar to the disguised area as the foreground text prompt of the sample disguised instance.

[0075] In a feasible implementation, step A13 may further include steps B11 to B13:

[0076] Step B11 , determining the image feature vector of the disguised area of ​​each sample disguised instance and the text feature vector of each disguised category hint through an image-text matching model.

[0077] Step B12: Calculate the cosine similarity between the image feature vector of the disguised region of each sample disguised instance and each text feature vector.

[0078] Step B13: determining a foreground text prompt of each sample disguised instance from the plurality of disguised category prompts based on the cosine similarity calculation result.

[0079] It should be noted that for each sample disguised instance, an image-text matching model is used to extract features from the disguised region and each disguised category prompt, obtaining an image feature vector corresponding to the disguised region and a text feature vector corresponding to each disguised category prompt. The cosine similarity between the image feature vector and each text feature vector is then calculated, and the resulting cosine similarity calculation result includes the cosine similarity between the image feature vector and each text feature vector. Based on the cosine similarity calculation result, the disguised category prompt similar to the disguised region is selected as the foreground text prompt for the sample disguised instance. In this embodiment, other similarity calculation methods may also be used and are not limited to this embodiment.

[0080] Step S20: Evaluate the camouflage degree of each sample training image according to the camouflage metric.

[0081] It should be noted that because different sample training images exhibit varying degrees of camouflage, even for the same semantic instance, the degree of camouflage can vary significantly depending on the complexity of the scene. This variability poses a challenge to the first stage of pseudo-mask generation, as the quality of the mask is directly affected by the degree of camouflage. As the degree of camouflage increases, achieving accurate segmentation becomes more difficult, and the quality of the pseudo-mask decreases accordingly. If these less refined pseudo-masks generated in the first stage are directly used in a typical self-supervised training setting, they will fail to fully utilize the data potential, potentially leading to suboptimal model performance. Therefore, this embodiment provides a gradient camouflage learning mechanism for camouflage degree, used to train an instance segmentation model that can learn to segment and learn highly camouflaged samples after learning from simple samples. After the model becomes proficient in handling simple samples, more challenging camouflaged samples are gradually introduced for training. This ensures that more difficult samples are introduced only after the model has acquired sufficient foundational knowledge. This gradient learning mechanism can help the model better adapt to different degrees of camouflage and improve its performance in different camouflage situations.

[0082] It is understood that the gradient camouflage learning mechanism, also known as the stepwise camouflage learning mechanism, includes a camouflage metric and a camouflage scheduler. The camouflage metric uses a graph-text matching model to measure the degree of camouflage of a sample training image. In this embodiment, a generative pre-trained model is used to empirically design a variety of graph-text matching model prompts in different formats, each representing a different quantization level. For each sample training image in the camouflage dataset, the graph-text matching model is used to embed it, and a similarity score is calculated between it and each prompt. Subsequently, the degree of camouflage of the sample training image is determined by identifying the prompt associated with the highest similarity score.

[0083] Step S30 : dividing each sample training image according to the disguise scheduler and the disguise degree of each sample training image to obtain a plurality of training batches and training information of each training batch.

[0084] It should be noted that the training information includes the training order and training gradients for each training batch. After determining the degree of camouflage for each sample training image, the camouflage scheduler determines the training batch for each sample training image and determines the training information for each training batch. This ensures that the instance segmentation model starts learning from easier camouflage instances and gradually adapts to increasingly complex camouflage instances. By systematically increasing training with more challenging examples, the complexity of the learning task gradually increases, allowing the model to effectively adapt to a wider range of camouflage scenarios.

[0085] In a feasible implementation, step S30 may include steps C11 to C13:

[0086] Step C11: sorting the disguise degree of each sample training image to obtain a sorting result.

[0087] Step C12: Divide each sample training image according to the sorting result and the camouflage scheduler to obtain multiple training batches, a training order for each training batch, and a training gradient for each training batch.

[0088] Step C13: Obtain training information of each training batch according to the training order of each training batch and the training gradient of each training batch.

[0089] It should be noted that the multiple sample training images are arranged in descending order of the degree of camouflage of each sample training image, and the camouflage scheduler divides the multiple sample training images into training batches for each sample training image, and determines the training information for each training batch. In this embodiment, the training information includes but is not limited to the training gradient and the training order.

[0090] In this embodiment, in a training batch, the lower the degree of disguise of the sample training image, the earlier the training order, so that the model can learn the training batch earlier. At the same time, the lower the degree of disguise, the higher the training gradient will be assigned to the training batch, emphasizing the learning of its features.

[0091] Step S40: training the instance segmentation model according to the training information of each training batch and the training batch to obtain a target segmentation model.

[0092] It should be noted that the instance segmentation model is trained according to the training information of each training batch combined with the training batches to obtain a target segmentation model that can be used for disguised instance segmentation.

[0093] In this embodiment, during the specific training process: in the initial stage of training, priority is given to samples with a lower degree of disguise, specifically "easy disguise subset and pseudo mask" and "slightly disguise subset and pseudo mask", allowing the model to start learning from easier disguise instances. For each training batch, the training gradient is adjusted according to the degree of disguise of the samples in the batch. Instances with a lower degree of disguise obtain higher gradients by increasing the learning rate, emphasizing the learning of their features. In contrast, instances with a higher degree of disguise are assigned lower gradients to reduce the negative impact that may be caused by mask inaccuracy. As training progresses, the camouflage scheduler gradually adjusts the degree of disguise of samples in the training batch. More complex and highly disguised samples, such as "highly disguised subset and pseudo mask", are gradually introduced, allowing the model to adapt to increasingly complex disguise instances.

[0094] Step S50: Input the target disguised image to be segmented into the target segmentation model to obtain a segmentation result of the disguised instances in the target disguised image.

[0095] This embodiment generates pseudo-labels for each sample disguised instance based on its foreground text prompt and the disguised region. Multiple sample training images are obtained using the pseudo-labels of each sample disguised instance, where the sample disguised instance is located within the sample disguised image. The degree of disguise of each sample training image is evaluated using a disguise metric. Each sample training image is divided into multiple training batches and training information for each training batch based on a disguise scheduler and the degree of disguise of each training image. An instance segmentation model is trained based on the training information and the training batches to obtain a target segmentation model. The target disguised image to be segmented is input into the target segmentation model to obtain a segmentation result for the disguised instance present in the target disguised image. Through the above-described method, based on weakly supervised learning, the accuracy and robustness of the model in handling disguised instance segmentation tasks are significantly improved. The model is applicable to various complex scenarios, exhibits sufficient flexibility and generalization, and maintains low training data requirements, reducing reliance on precisely labeled data.

[0096] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 2 In the text-guided disguised instance segmentation method, step S10 further includes steps S11 to S13:

[0097] Step S11 , generating a semantic mask of each sample disguised instance according to a semantic mask generator, a foreground text hint of each sample disguised instance, and a disguised region of each sample disguised instance, wherein the semantic mask generator is constructed based on gradient class activation mapping.

[0098] It should be noted that previous unsupervised or weakly supervised learning studies often rely on methods such as spectral decomposition or class activation mapping for image segmentation. However, these methods have limitations. For example, spectral decomposition mainly uses spatial information and may not fully utilize the semantic clues of segmentation. Similarly, although class activation mapping generates heat map results to identify relevant image areas, it may not effectively capture fine spatial details. Although a simple approach is to simply combine the two masks using a weighted method, this approach may affect the quality of the generated masks. Therefore, in this embodiment, a semantic spatial iterative fusion module is proposed to properly integrate the semantic and spatial masks generated using class activation mapping and spectral decomposition respectively. It consists of three main components: a semantic mask generator, a spatial mask generator, and an iterative mask fuser. It first uses the semantic mask generator and the spatial mask generator to create masks from the image. Then, the iterative mask fusion component iteratively refines these masks to produce accurate pseudo labels.

[0099] It can be understood that for each sample disguised instance present in each sample disguised image: in order to utilize the foreground text clue corresponding to the sample disguised instance, the semantic mask generator is constructed on the basis of the class activation map. It uses the feature map of the final convolutional layer to generate a heat map. The gradient class activation map improves the class activation map by replacing the weights of the fully connected layer after global average pooling with the gradient to weight the activation map, thereby producing a more refined class activation map. Therefore, the semantic mask generator in this embodiment is constructed by gradient class activation mapping (Grad-CAM).

[0100] In the specific implementation, the gradient class activation map in the semantic mask generator can be effectively used in the image-text matching model to generate a class activation map for the disguised image hint, allowing the precise localization of the object category in the visual transformer. For each sample disguised instance in each sample disguised image: based on the disguised area and foreground text hint of the sample disguised instance, the gradient class activation map is used to distinguish the foreground class from the background class. Its formula is expressed as: in, Represents the weight of the foreground and the activation value of the kth feature map respectively, RELU is the activation function, RELU=max(0,x), M se Disguise the semantic mask of the instance for the generated sample.

[0101] Step S12: Generate a spatial mask for each sample camouflage instance based on the spatial mask generator and the camouflage area of ​​each sample camouflage instance.

[0102] It should be noted that the spatial mask generator uses an object detection model based on DINO (DETR with Improved Object Queries) and a Normalized Cut (NCut) algorithm, combined with the camouflaged area corresponding to the sample camouflaged instance in each sample camouflaged image, to promote the generation of a spatial mask for each sample camouflaged instance.

[0103] In a feasible implementation, step S12 may include steps D11 to D13:

[0104] Step D11 : generating a plurality of patches corresponding to the camouflaged area of ​​each sample camouflaged instance and key features of each patch according to the target object detection model.

[0105] Step D12 : calculating the patch affinity between the patches in each camouflaged area according to the key features of the patches in each camouflaged area.

[0106] Step D13 : generating a spatial mask of each sample camouflage instance based on the patch affinity between each patch of each camouflage area and the normalized cut algorithm.

[0107] It should be noted that for each sample camouflage instance: In this embodiment, the target object detection model is used to generate patches for the camouflage area, and the key features of multiple patches in the camouflage area are extracted. The key features of each patch are used to calculate the patch affinity between all patches in the DINO feature space. The patch affinity matrix between patches is used as input and segmented using NCut. The goal of NCut is to minimize the intra-group difference while maximizing the inter-group difference. By setting an appropriate threshold, it is determined which areas should be included in the final spatial mask, thereby obtaining the spatial mask of the sample camouflage instance.

[0108] In step S13, the spatial mask and the semantic mask of each sample disguised instance are iteratively refined according to the iterative mask fuser to generate a pseudo label of each sample disguised instance, and a plurality of sample training images are obtained through the pseudo label of each sample disguised instance.

[0109] It should be noted that for each sample disguised instance, the semantic mask and spatial mask are iteratively fused by the iterative mask fuser, and the generated pseudo mask is refined according to the evaluation of the fused mask, and finally the pseudo label of each sample disguised instance is obtained.

[0110] In a feasible implementation, step S13 may include steps E11 to E13:

[0111] Step E11 : Calculate the similarity score ratio of each sample disguised instance based on the image-text matching model and the semantic mask of each sample disguised instance.

[0112] Step E12 : calculating the angular energy distribution ratio and radial energy distribution ratio of each sample camouflage instance according to the spatial mask of each sample camouflage instance.

[0113] Step E13 , performing iterative refinement based on the similarity score ratio, angular energy distribution ratio, radial energy distribution ratio, and iterative mask fuser of each sample disguised instance to generate a pseudo label for each sample disguised instance.

[0114] It should be noted that in terms of semantics, the image-text matching model is used in conjunction with the semantic mask of each sample disguised instance to evaluate the similarity between the foreground and background images of the semantic mask and the textual hint of the disguised category. The ratio between these similarity scores is then calculated. The entire process can be expressed as: in The features of the foreground and background masks passed through the visual encoder, x T is the feature of the disguised category passed by the text encoder, and SIM is the cosine similarity function. In this embodiment, the similarity score ratio of the sample disguised instance is the ratio E between the similarity scores. se .

[0115] It can be understood that in terms of space, the spatial mask of each sample camouflage instance is combined and the frequency characteristics are used to evaluate the spatial quality of the fused mask. Although the camouflaged object and the background have strong visual similarities, there are differences in energy distribution in the frequency domain. The use of frequency characteristics may be able to identify subtle differences in spatial quality. In this embodiment, the directional energy distribution metric and the concentric circle energy metric are introduced. These metrics calculate the energy distribution ratio E of the original image o and the foreground f along the angular and radial directions in the polar coordinate frequency domain respectively. CCE and E DE , as shown below: where M and N are the amount of energy in the frequency domain along the angular and radial directions. and are the energy along the angular and radial directions in the frequency domain, They represent the average energy value of the original image along the angular and radial directions in the frequency domain, and the average energy value along the angular and radial directions in the frequency domain, respectively. In this embodiment, the angular energy distribution ratio of the sample disguised instance refers to the energy distribution ratio E along the angle in the polar coordinate frequency domain corresponding to the sample disguised instance. CCE The radial energy distribution ratio of the sample camouflage instance refers to the energy distribution ratio E along the radial direction in the polar coordinate frequency domain corresponding to the sample camouflage instance. DE .

[0116] In the specific implementation, for each sample disguised instance: guided by the semantic and spatial mask evaluation results, the iterative mask fuser iteratively fuses the two masks to generate the final pseudo mask. Higher indicators indicate better quality of the corresponding mask, and thus increase the weight. Specifically, the spatial mask and semantic mask generated by spectral decomposition and class activation mapping, respectively, are first merged with fixed weights. In each iteration, momentum-weighted iterative fusion is used to update the pseudo mask. The entire update process can be expressed as: in and They represent the weights of semantic mask and spatial mask in the next iteration respectively; α represents the momentum parameter, which controls the influence of the previous iteration result on the current iteration. The value of the momentum parameter is pre-set; M t+1 represents the pseudo mask generated in the iteration, and this iterative process is performed three times to obtain a significantly refined mask; E sp =E CCE +E DE In this embodiment, the final mask generated after iterative refinement is the pseudo label of each sample disguised instance.

[0117] This embodiment generates a semantic mask for each disguised instance based on a semantic mask generator, its foreground text hint, and its disguised region. The semantic mask generator is constructed based on gradient class activation mapping. A spatial mask is generated based on a spatial mask generator and its disguised region. An iterative mask fuser iteratively refines the spatial mask and semantic mask of each disguised instance to generate a pseudo-label for each disguised instance. This approach enables the generation of high-quality pseudo-labels.

[0118] For example, to help understand the implementation process of the text-guided disguised instance segmentation method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 3 , Figure 3 This paper provides a brief flowchart of the text-guided disguise instance segmentation method, specifically:

[0119] The method of this embodiment primarily includes the following steps: 1. Introducing textual cues as weak supervisory signals, leveraging the text associated with the image to provide additional semantic information to help distinguish visually similar but semantically distinct objects. Specifically, based on the input image and textual cues, an object detection model is used to identify camouflaged regions in the image and generate region proposals. The textual cues are generated using a generative pre-trained model to generate a large number of candidate text categories, which are then selected by encoding and calculating cosine similarity to select matching region-text pairs. 2. The semantic-spatial iterative fusion module combines semantic and spatial information through an iterative process to improve segmentation accuracy. The semantic mask generated by the textual cues and the spatial mask based on the image are evaluated and adjusted to generate a more refined pseudo mask. 3. Training is performed using previously obtained pseudo labels. A stepwise camouflage learning mechanism evaluates the degree of camouflage in the image, ranks training samples by camouflage level, and dynamically adjusts the training difficulty based on this ranking to optimize the learning process during the self-training phase. The method of this embodiment significantly improves the model's accuracy in the camouflaged instance segmentation task through weakly supervised learning, while maintaining low training data requirements, a feature that has not been fully explored in previous research.

[0120] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the disguised instance segmentation method guided by the text of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0121] This application also provides a text-guided disguised instance segmentation device, please refer to Figure 4 , the text-guided disguised instance segmentation device includes:

[0122] The generating module 10 is used to generate a pseudo label for each sample disguised instance according to the foreground text prompt of each sample disguised instance and the disguised area of ​​each sample disguised instance, and obtain multiple sample training images through the pseudo label of each sample disguised instance, wherein the sample disguised instance is located in the sample disguised image.

[0123] The evaluation module 20 is configured to evaluate the camouflage degree of each sample training image according to the camouflage metric.

[0124] The division module 30 is configured to divide each sample training image according to the disguise scheduler and the disguise degree of each sample training image to obtain multiple training batches and training information of each training batch.

[0125] The training module 40 is used to train the instance segmentation model according to the training information of each training batch and the training batch to obtain a target segmentation model.

[0126] The segmentation module 50 is configured to input the target disguised image to be segmented into the target segmentation model to obtain a segmentation result of the disguised instances in the target disguised image.

[0127] Optionally, the generating module 10 is further configured to:

[0128] Each sample disguised image is input into the target object detection model for region recognition to obtain the disguised region of each sample disguised instance; multiple disguised category prompts are generated based on the generative pre-trained model; the disguised region of each sample disguised instance is matched with each disguised category prompt to obtain the foreground text prompt of each sample disguised instance.

[0129] Optionally, the generating module 10 is further configured to:

[0130] The image feature vector of the disguised area of ​​each sample disguised instance and the text feature vector of each disguised category prompt are determined through an image-text matching model; the cosine similarity between the image feature vector of the disguised area of ​​each sample disguised instance and the text feature vector of each disguised category prompt is calculated; and the foreground text prompt of each sample disguised instance is determined from multiple disguised category prompts based on the cosine similarity calculation result.

[0131] Optionally, the generating module 10 is further configured to:

[0132] A semantic mask for each disguised instance is generated based on a semantic mask generator, the foreground text hint of each disguised instance, and the disguised region of each disguised instance. The semantic mask generator is constructed based on gradient class activation mapping. A spatial mask for each disguised instance is generated based on a spatial mask generator and the disguised region of each disguised instance. The spatial mask and semantic mask of each disguised instance are iteratively refined using an iterative mask fuser to generate a pseudo label for each disguised instance.

[0133] Optionally, the generating module 10 is further configured to:

[0134] Based on the target object detection model, multiple patches corresponding to the camouflaged area of ​​each sample camouflaged instance and the key features of each patch are generated; the patch affinity between each patch in each camouflaged area is calculated based on the key features of each patch in each camouflaged area; and the spatial mask of each sample camouflaged instance is generated based on the patch affinity between each patch in each camouflaged area and the normalized cutting algorithm.

[0135] Optionally, the generating module 10 is further configured to:

[0136] Based on the image-text matching model and the semantic mask of each sample disguised instance, the similarity score ratio of each sample disguised instance is calculated; based on the spatial mask of each sample disguised instance, the angular energy distribution ratio and radial energy distribution ratio of each sample disguised instance are calculated; based on the similarity score ratio, angular energy distribution ratio, radial energy distribution ratio of each sample disguised instance and the iterative mask fuser, iterative refinement is performed to generate a pseudo label for each sample disguised instance.

[0137] Optionally, the updating module 40 is further configured to:

[0138] The camouflage degree of each sample training image is sorted to obtain a sorting result; each sample training image is divided according to the sorting result and a camouflage scheduler to obtain multiple training batches, a training order of each training batch, and a training gradient of each training batch; and training information of each training batch is obtained according to the training order of each training batch and the training gradient of each training batch.

[0139] The text-guided disguised instance segmentation device provided in this application utilizes the text-guided disguised instance segmentation method described in the aforementioned embodiments, resolving the technical issues with prior art disguised instance segmentation methods, which rely on large amounts of precisely annotated data and suffer from poor accuracy and generalization. Compared to prior art, the text-guided disguised instance segmentation device provided in this application achieves the same beneficial effects as the text-guided disguised instance segmentation method described in the aforementioned embodiments. Other technical features of the text-guided disguised instance segmentation device are the same as those disclosed in the aforementioned embodiments and are not further elaborated upon here.

[0140] The present application provides a text-guided disguised instance segmentation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the text-guided disguised instance segmentation method in the above-mentioned embodiment 1.

[0141] Reference below Figure 5, which shows a schematic structural diagram of a text-guided disguised instance segmentation device suitable for implementing an embodiment of the present application. The text-guided disguised instance segmentation device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The text-guided disguised instance segmentation device shown is only an example and should not limit the functions and scope of use of the embodiments of the present application.

[0142] like Figure 5 As shown, the text-guided disguised instance segmentation device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the operation of the text-guided disguised instance segmentation device are also stored in RAM 1004. Processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the text-guided disguised instance segmentation device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a text-guided disguised instance segmentation device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have instead.

[0143] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0144] The text-guided disguised instance segmentation device provided by this application utilizes the text-guided disguised instance segmentation method of the aforementioned embodiment, resolving the technical issues of prior art disguised instance segmentation methods, which rely on large amounts of precisely annotated data and suffer from poor accuracy and generalization. Compared to the prior art, the beneficial effects of the text-guided disguised instance segmentation device provided by this application are the same as those of the text-guided disguised instance segmentation method provided by the aforementioned embodiment. Other technical features of this text-guided disguised instance segmentation device are the same as those disclosed in the aforementioned embodiment and are not further elaborated upon here.

[0145] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0146] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0147] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the text-guided disguised instance segmentation method in the above-mentioned embodiment.

[0148] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0149] The computer-readable storage medium may be included in the text-guided disguised instance segmentation device; or may exist independently without being incorporated into the text-guided disguised instance segmentation device.

[0150] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a text-guided disguised instance segmentation device, the text-guided disguised instance segmentation device: generates a pseudo-label for each sample disguised instance based on a foreground text prompt and a disguised area of ​​each sample disguised instance, and obtains multiple sample training images based on the pseudo-label of each sample disguised instance, wherein the sample disguised instance is located in the sample disguised image; evaluates the disguise degree of each sample training image based on a disguise metric; divides each sample training image based on a disguise scheduler and the disguise degree of each sample training image to obtain multiple training batches and training information for each training batch; trains an instance segmentation model based on the training information of each training batch and the training batches to obtain a target segmentation model; inputs the target disguised image to be segmented into the target segmentation model to obtain a segmentation result of the disguised instances present in the target disguised image.

[0151] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0152] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0153] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0154] The computer-readable storage medium provided in this application is a computer-readable storage medium storing computer-readable program instructions (i.e., a computer program) for executing the above-described text-guided disguised instance segmentation method. This computer-readable storage medium can address the technical issues in the prior art where disguised instance segmentation methods rely on a large amount of precisely labeled data, resulting in poor accuracy and generalization. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the text-guided disguised instance segmentation method provided in the above-described embodiments, and are not further elaborated here.

[0155] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned text-guided disguised instance segmentation method when executed by a processor.

[0156] The computer program product provided in this application addresses the technical issues of existing methods for masked instance segmentation, which rely on large amounts of precisely labeled data and suffer from poor accuracy and generalization. Compared to existing techniques, the computer program product provided in this application offers the same beneficial effects as the text-guided masked instance segmentation method provided in the aforementioned embodiments, and will not be further elaborated upon here.

[0157] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A text-guided forged instance segmentation method, It is characterized in that The method comprises: Generate a pseudo label of each sample disguised instance according to a foreground text prompt of each sample disguised instance and a disguised region of each sample disguised instance, and obtain a plurality of sample training images through the pseudo label of each sample disguised instance, wherein the sample disguised instance is located in the sample disguised image; Evaluate the camouflage degree of each sample training image according to the camouflage metric; Dividing each sample training image according to the disguise scheduler and the disguise degree of each sample training image to obtain multiple training batches and training information of each training batch; The instance segmentation model is trained according to the training information of each training batch and the training batch to obtain a target segmentation model; The target disguised image to be segmented is input into the target segmentation model to obtain a segmentation result of the disguised instances existing in the target disguised image.

2. The method according to claim 1, It is characterized in that Before the step of generating a pseudo label of each sample disguised instance according to the foreground text prompt of each sample disguised instance and the disguised area of ​​each sample disguised instance, the method further includes: Input each sample disguised image into the target object detection model for region recognition, and obtain the disguised region of each sample disguised instance; Generate multiple camouflage category hints based on the generative pre-trained model; The disguised area of ​​each sample disguised instance is matched with each disguised category prompt to obtain the foreground text prompt of each sample disguised instance.

3. The method according to claim 2, It is characterized in that The step of matching the disguised area of ​​each sample disguised instance with each disguised category prompt to obtain the foreground text prompt of each sample disguised instance comprises: Determine the image feature vector of the disguised area of ​​each sample disguised instance and the text feature vector of each disguised category prompt through the image-text matching model; Calculate the cosine similarity between the image feature vector of the disguised area of ​​each sample disguised instance and each text feature vector; The foreground text prompt of each sample disguised instance is determined from multiple disguised category prompts according to the cosine similarity calculation result.

4. The method according to claim 1, It is characterized in that The step of generating a pseudo label of each sample disguised instance according to the foreground text prompt of each sample disguised instance and the disguised area of ​​each sample disguised instance comprises: Generate a semantic mask of each sample disguised instance according to a semantic mask generator, a foreground text hint of each sample disguised instance, and a disguised region of each sample disguised instance, wherein the semantic mask generator is constructed based on a gradient class activation map; Generate a spatial mask of each sample camouflage instance according to the spatial mask generator and the camouflage area of ​​each sample camouflage instance; The spatial mask of each sample disguised instance and the semantic mask of each sample disguised instance are iteratively refined according to the iterative mask fuser to generate a pseudo label of each sample disguised instance.

5. The method according to claim 4, It is characterized in that The step of generating a spatial mask of each sample camouflage instance according to the spatial mask generator and the camouflage area of ​​each sample camouflage instance comprises: Generate multiple patches corresponding to the camouflaged area of ​​each sample camouflage instance and key features of each patch according to the target object detection model; Calculating patch affinity between patches in each camouflaged area according to key features of patches in each camouflaged area; A spatial mask of each sample camouflage instance is generated according to the patch affinity between each patch of each camouflage region and the normalized cut algorithm.

6. The method according to claim 4, It is characterized in that The step of iteratively refining the spatial mask of each sample disguised instance and the semantic mask of each sample disguised instance according to the iterative mask fuser to generate a pseudo label of each sample disguised instance includes: Calculate the similarity score ratio of each sample disguised instance based on the image-text matching model and the semantic mask of each sample disguised instance; Calculate the angular energy distribution ratio and radial energy distribution ratio of each sample camouflage instance according to the spatial mask of each sample camouflage instance; Iterative refinement is performed according to the similarity score ratio, angular energy distribution ratio, radial energy distribution ratio and iterative mask fuser of each sample camouflage instance to generate a pseudo label for each sample camouflage instance.

7. The method according to any one of claims 1 to 6, It is characterized in that The method of dividing each sample training image according to the disguise scheduler and the disguise degree of each sample training image to obtain multiple training batches and training information of each training batch includes: Sort the camouflage degree of each sample training image to obtain a sorting result; Dividing each sample training image according to the sorting result and the camouflage scheduler to obtain multiple training batches, a training order of each training batch, and a training gradient of each training batch; The training information of each training batch is obtained according to the training order of each training batch and the training gradient of each training batch.

8. A text-guided disguised instance segmentation device, It is characterized in that The text-guided disguised instance segmentation device comprises: A generating module, used for generating a pseudo label of each sample disguised instance according to a foreground text prompt of each sample disguised instance and a disguised region of each sample disguised instance, and obtaining a plurality of sample training images through the pseudo label of each sample disguised instance, wherein the sample disguised instance is located in the sample disguised image; An evaluation module, used for evaluating the camouflage degree of each sample training image according to the camouflage metric; A division module, used for dividing each sample training image according to the disguise scheduler and the disguise degree of each sample training image, to obtain multiple training batches and training information of each training batch; A training module, used for training the instance segmentation model according to the training information of each training batch and the training batch to obtain a target segmentation model; The segmentation module is used to input the target disguised image to be segmented into the target segmentation model to obtain the segmentation result of the disguised instance existing in the target disguised image.

9. A text-guided disguised instance segmentation device, It is characterized in that The text-guided disguised instance segmentation device includes: a memory, a processor, and a text-guided disguised instance segmentation program stored in the memory and executable on the processor, wherein the text-guided disguised instance segmentation program is configured to implement the text-guided disguised instance segmentation method according to any one of claims 1 to 7.

10. A storage medium, It is characterized in that The storage medium stores a text-guided disguised instance segmentation program, and when the text-guided disguised instance segmentation program is executed by a processor, the text-guided disguised instance segmentation method according to any one of claims 1 to 7 is implemented.

Images