A semi-supervised feature mining method for power transmission channels based on positive and negative samples

Abstract

This technology mainly involves the fields of power transmission safety supervision and industrial image feature mining, and mainly provides a semi-supervised feature mining method for power transmission channels based on positive and negative samples. It solves the problem of traditional target detection algorithms in identifying small targets, occluded targets and blurred targets in complex scenes, and improves the accuracy of risk control and safety supervision. The main scheme includes independent SwinTransformer feature encoders and feature diffusion forward module processing of positive and negative samples to obtain a multi-head feature generator for the power transmission scene; processing negative and positive samples, generating a power transmission scene feature map through a cross-attention module and a multi-head feature generator, and processing it to obtain instance segmentation prediction, scene object representation encoding, and instance category or instance description prediction.

Description

Technical Field

[0001] This technology mainly involves the fields of power transmission safety supervision and industrial image feature mining. It mainly provides a semi-supervised feature mining method for power transmission channels based on positive and negative samples to meet business needs related to safety supervision. Background Art

[0002] In the field of power transmission safety supervision, traditional safety supervision relies on target detection algorithms, such as the identification of pedestrians, vehicles, tower cranes and other objects. There are defects such as misalignment between the identified objects and the safety supervision targets and poor recognition performance.

[0003] Common algorithms include the YOLO series. These models are pre-trained on open-source datasets, meaning the base models already have the ability to recognize pedestrians and vehicles. This facilitates transfer learning, but they struggle to recall physical objects within the scope of power transmission safety regulation. This is because traditional deep learning models significantly degrade their recall for small, occluded, and blurred objects. Furthermore, images of power transmission channels present complex imaging environments, often with small and blurry objects. Consequently, object detection algorithms are prone to missing key physical objects, and identified objects cannot be directly used for risk control.

[0004] In the field of industrial image feature mining, the current main focus is on image vision technologies such as CNNs and Transformers. The training of these deep learning feature mining frameworks has also evolved through two phases: supervised and unsupervised. Traditional supervised algorithms require extensive data annotation and often yield low returns. Recently, the trendy unsupervised (self-supervised) algorithms rely on the data samples themselves, designing learning tasks for feature mining, and have demonstrated impressive results.

[0005] Mainstream industrial feature mining technologies are focused on the development of large models. Large models offer powerful representational capabilities for image restoration and image and video generation. However, in power transmission channel imaging, background objects are more numerous and larger than foreground objects. During model learning, a significant amount of attention is devoted to background objects, resulting in a lack of feature mining for the foreground objects that require attention.

[0006] In summary, in response to the existing difficulties in the field of power transmission safety supervision: traditional target recognition technology cannot be directly applied to risk identification; the feature differences of blurred objects in images are small; the target objects are small in size and lack details; the samples are highly homogeneous, making it difficult to collect rich data; conventional models tend to learn background knowledge and ignore the foreground knowledge of the subject. This patent proposes a semi-supervised feature mining method for power transmission channels based on positive and negative samples. Summary of the Invention

[0007] The purpose of this invention is to process positive and negative samples using a SwinTransformer feature encoder and a feature diffusion forward module to generate a multi-head feature generator capable of characterizing the complex characteristics of power transmission scenarios. This method combines text and image information and enhances the model's instance recognition capabilities through a cross-attention module, improving the accuracy of scene analysis and object description. This method primarily addresses the challenges of traditional target detection algorithms in recognizing small, occluded, and blurred targets in complex scenarios, thereby enhancing the accuracy of risk control and safety supervision.

[0008] In order to achieve the above-mentioned purpose, the present invention adopts the following technical means:

[0009] The present invention provides a semi-supervised feature mining method for power transmission channels based on positive and negative samples, comprising the following steps:

[0010] Step 1: Process the positive and negative samples using independent SwinTransformer feature encoders and feature diffusion forward modules to obtain a multi-head feature generator for the power transmission scenario.

[0011] Step 2: Process negative and positive samples, generate a power transmission scene feature map through a cross-attention module and a multi-head feature generator, and process it to obtain instance segmentation prediction, scene object representation encoding, and instance category or instance description prediction. Combining the processing method of text and image information, it not only enhances the model's ability to recognize instances, but also improves its accuracy in scene analysis and object description.

[0012] In the above technical solution, step 1 specifically includes:

[0013] Step 1.1: For positive and negative samples, independent SwinTransformer feature encoders and feature diffusion forward modules are processed to obtain different feature combinations under the same sample distribution. This improves the model's ability to distinguish foreground and background object features in power transmission scenarios, which helps with subsequent instance segmentation and object representation encoding.

[0014] Step 1.2: Perform the feature combination-guided learning in step 1.1 on the randomly generated Gaussian noise feature map to obtain a feature generator for power transmission scenarios. This feature generator can generate meaningful power transmission scenario features from noise, demonstrating powerful feature mining and scenario reconstruction capabilities.

[0015] Step 1.3: Create a negative sample image reconstruction task head for the feature generator. This allows the feature generator to learn to generate high-frequency semantic representations of background objects in the unsupervised reconstruction task. This effectively trains the model to distinguish and reconstruct the semantic information of background objects, facilitating subsequent scene analysis and risk control.

[0016] Step 1.4: Create a positive and negative sample difference feature map fitting task head for the feature generator. This allows the feature generator to learn to generate low-frequency semantic representations of foreground objects during the difference image fitting process. The difference learning strategy effectively guides the model to focus on objects with significant changes, thereby improving the detection and recognition capabilities of key targets.

[0017] Step 1.5: Input the image reconstruction task head in step 1.3 and the difference feature map fitting task head in step 1.4 into the feature decoder to obtain the negative sample prediction and the positive and negative sample difference map prediction;

[0018] In step 1.6, the losses for the negative sample predictions and the positive-negative sample difference map predictions are calculated separately from the negative sample and positive-negative sample difference feature maps. Backpropagation is used to optimize the model, resulting in a multi-headed power transmission scenario feature generator. This process not only optimizes the model's performance but also enables in-depth exploration and characterization of complex features in power transmission scenarios through the multi-headed feature generator. This optimization strategy significantly improves the model's capability and efficiency in generating power transmission scenario features.

[0019] In the above technical solution, step 2 specifically includes: step 2.1, for the negative sample, input it into the feature encoder and feature diffusion forward module in step 1.1 to obtain a diffused feature map;

[0020] Step 2.2: Input the diffused feature map into the backbone of Swintransformer to obtain the feature map of the negative sample.

[0021] Step 2.3: Input the positive sample into the backbone of VIT training to obtain the feature map of the positive sample;

[0022] Step 2.4: Input the feature maps of positive and negative samples into the transformer module for training to obtain the cross-attention feature maps of positive and negative samples;

[0023] Step 2.5: Input the text description information of the negative sample annotation into the TextEncode module to obtain the text prompt code;

[0024] Step 2.6: Encode the cross-attention feature maps and text prompts of the positive and negative samples, input them into the cross-attention module, and obtain the text-image cross-attention prompt encoding;

[0025] Step 2.7: Encode the diffused feature map and the cross-attention cue of the text and image in step 2.1, input them into the multi-head feature generator in step 1.6, and iterate to generate the power transmission scene feature map;

[0026] Step 2.8: Use the ROIGenerate module to process the transmission scene feature map to obtain the mask encoding of the instance region of interest to the model, that is, to obtain the foreground instance segmentation prediction;

[0027] Step 2.9: Input the power transmission scene feature map into the feature decoder in step 1.5 for processing to obtain the scene object representation code;

[0028] Step 2.10: Segment the instance region, segment the corresponding region on the scene object representation code, and obtain the feature representation of the instance;

[0029] Step 2.11: Encode the instance feature representation and text prompt, perform CLIP text-image alignment training, and obtain instance category or instance description prediction.

[0030] Because the present invention adopts the above technical solution, it has the following beneficial effects:

[0031] 1. By combining the SwinTransformer feature encoder and the feature diffusion forward module, the system effectively processes positive and negative samples. Through feature encoding and diffusion, it generates multi-head features that can characterize the complex characteristics of power transmission scenarios. This technical approach solves the problem of traditional target detection algorithms in recognizing small, occluded, and blurred targets in complex scenarios (such as power transmission channels), thereby improving the accuracy of risk control and safety supervision.

[0032] 2. Multimodal feature fusion: By combining the SwinTransformer feature encoder and the feature diffusion forward module, we effectively process positive and negative samples and generate multi-head features that can characterize the complex characteristics of power transmission scenes. This multimodal feature fusion improves the model's ability to distinguish foreground and background objects in power transmission scenes, facilitating subsequent instance segmentation and object representation encoding.

[0033] 3. Unsupervised Learning and Data Augmentation: The feature generator learns to generate high-frequency semantic representations of background objects in unsupervised reconstruction tasks and implements feature-level data augmentation through mask operations. This approach not only reduces the reliance on large amounts of labeled data but also improves the model's ability to understand complex scenes.

[0034] 4. Differential learning strategy: By creating a feature map of positive and negative sample differences to fit the task head, the model is guided to focus on objects with significant changes, thereby improving the detection and recognition capabilities of key targets.

[0035] 5. Combination of text and image information: By combining text description information and image information, through the cross-attention module, the model can more accurately identify and understand instances in the scene, improving the accuracy of instance segmentation and object description.

[0036] 6. Optimize model performance: By calculating losses and backpropagation, the model can continuously optimize its performance and generate a transmission scene feature generator with the ability to semantically represent foreground and background objects.

[0037] Overall, this method effectively improves the ability to identify key targets and control risks in complex transmission channel environments through innovative feature extraction and fusion techniques, combined with unsupervised learning and data enhancement strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 : Schematic diagram of the SDMMS structure;

[0039] Figure 2 :Schematic diagram of CDIBMS structure. DETAILED DESCRIPTION

[0040] The following is a detailed description of the embodiments of the present invention. Although the present invention will be described and illustrated in conjunction with certain specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, modifications or equivalent substitutions of the present invention are intended to fall within the scope of the claims of the present invention.

[0041] In addition, in order to better illustrate the present invention, numerous specific details are given in the following detailed description. It will be understood by those skilled in the art that the present invention can also be implemented without these specific details.

[0042] The present invention provides a semi-supervised feature mining method for power transmission channels based on positive and negative samples, comprising the following steps:

[0043] Step 1: Process the positive and negative samples using independent SwinTransformer feature encoders and feature diffusion forward modules to obtain a multi-head feature generator for the power transmission scenario.

[0044] Step 2: Process the negative and positive samples, generate the power transmission scene feature map through the cross attention module and the multi-head feature generator, and process it to obtain instance segmentation prediction, scene object representation encoding and instance category or instance description prediction.

[0045] Step 1.1: For positive and negative samples, independent SwinTransformer feature encoders and feature diffusion forward modules are processed to obtain different feature combinations under the same sample distribution. This step involves building a diffusion model architecture, which requires extensive collection of power transmission scene images and establishing the main framework of the diffusion model. A power transmission scene-aware diffusion model is trained to enable the model to form a semantic representation of the power transmission scene. Diffusion model learning is based on reconstruction tasks and a large amount of learning data. These data depend on the distribution of the learning data, so the distribution of the original data is crucial. Learning data collection follows two principles: first, image data from different preset positions is collected from a large number of cameras; second, different images are collected for the same preset position to ensure that multiple images are available for each preset position for diffusion model learning. During the training phase, images of the same preset position must be fed into batch learning in pairs. For each pair, a PersetImgl-2Diff image is generated for subsequent feature learning and model optimization.

[0046] In step 1.2, the randomly generated Gaussian noise feature map is subjected to the feature combination-guided learning described in step 1.1 to obtain a feature generator for the power transmission scenario. This feature generator is based on the Symmetric Diffusion of Positive and Negative Samples (SDMMS) framework, which includes five modules: an image encoder, a feature diffusion forward module, a feature diffusion backward module, a feature decoder, and an image alignment module. The image encoder uses a backbone feature extractor trained on open source data. The feature diffusion forward module cyclically adds normally distributed noise to the feature map until the feature map becomes a noise feature map that follows a normal distribution. A mask operation is also introduced to achieve feature-level data enhancement.

[0047] In step 1.3, a negative image reconstruction task head is created for the feature generator, enabling it to learn to generate high-frequency semantic representations of background objects in an unsupervised reconstruction task. The feature decoder operates on the diffused feature map, deeply reconstructing the image feature distribution to produce the predicted image. The image alignment module calculates the difference between the predicted PersetImg1Pred and PersetImg1, reconciles the loss between the two images, and uses backpropagation to optimize the deep learning model in the feature backward module.

[0048] In step 1.4, create a positive-negative sample difference feature map fitting task head for the feature generator. This allows the feature generator to learn to generate low-frequency semantic representations of foreground objects during the differential image fitting process. The model tends to learn semantic representations of foreground objects in the PersetImg1-2Diff reconstruction task.

[0049] In step 1.5, the image reconstruction task head from step 1.3 and the difference feature map fitting task head from step 1.4 are fed into the feature decoder to obtain the negative sample prediction and the positive-negative sample difference map prediction. This step involves the application of the feature decoder, which deeply reconstructs the image feature distribution to obtain the predicted image.

[0050] In step 1.6, we calculate the loss for the negative sample predictions and the positive-negative sample difference map predictions, respectively, and then optimize the model through backpropagation to obtain a multi-headed power transmission scene feature generator. This step is crucial for model training. By calculating the loss and performing backpropagation, the model continuously optimizes its performance, ultimately generating a power transmission scene feature generator capable of semantically representing foreground and background objects.

[0051] Step 2: Process the negative and positive samples, generate a power transmission scene feature map through a cross-attention module and a multi-head feature generator, and process it to obtain instance segmentation prediction, scene object representation encoding, and instance category or instance description prediction.

[0052] Step 2.1: For negative samples, input them into the feature encoder and feature diffusion forward module from step 1.1 to obtain a diffused feature map. This step involves constructing a positive-negative sample-based text-image cross-diffusion model (CDIBMS), which uses a pair of pre-set point images and a text description of the object as input.

[0053] Step 2.2: Input the diffused feature map into the Swintransformer backbone to obtain the feature map of the negative sample. In this step, the diffusion architecture reuses the denoising module (feature generation model) trained by the diffused backward module in SDMMS. The denoising model branch receives an original image and uses the image encoder to generate a semantic feature map.

[0054] In step 2.3, the positive sample is fed into the VIT-trained backbone to generate a feature map of the positive sample. In this step, the trained image encoder is used to generate a semantic feature map for the other image. This feature map is then modulated by a trainable Transformer module to form a semantic encoding of the image.

[0055] Step 2.4: The feature maps of the positive and negative samples are fed into the transformer module for training to obtain the cross-attention feature maps of the positive and negative samples. In this step, the cross-attention fusion text encoding is used as another input to the diffusion module.

[0056] Step 2.5: Input the text description information of the negative sample annotation into the TextEncode module to obtain the text prompt encoding. In this step, TextEncoder is a pre-trained text encoder in NLP for information embedding of text information.

[0057] Step 2.6: The cross-attention feature maps and text hint encodings of the positive and negative samples are fed into the cross-attention module to obtain the text-image cross-attention hint encoding. In this step, the global semantic guidance factor is obtained by cross-embedding the text encoding and image encoding.

[0058] Step 2.7: Encode the diffused feature map and the cross-attention cue from step 2.1 and feed it into the multi-head feature generator from step 1.6. This iterative loop generates a feature map for the power transmission scene. In this step, the diffused feature map is passed through a trainable ROIGenerate module to generate a mask for the instance region of interest.

[0059] Step 2.8: Use the ROIGenerate module to process the transmission scene feature map to obtain a mask encoding of the instance region of interest, thus obtaining the foreground instance segmentation prediction. In this step, the mask is used for supervised instance segmentation training.

[0060] Step 2.9: Input the power transmission scene feature map into the feature decoder in step 1.5 to obtain the scene object representation code. In this step, the feature decoder in SDMMS is used to generate the instance feature code.

[0061] Step 2.10: Segment the instance region. Segment the corresponding region on the scene object representation encoding to obtain the instance's feature representation. In this step, the feature encoding is used for contrastive learning on the text-image pair to align the text and image semantic spaces.

[0062] Step 2.11: Encode the instance feature representation and textual hints, and perform CLIP text-image alignment training to predict the instance category or description. This step addresses the confusion caused by the diffusion model in representing the instance object category, while also introducing object descriptions to guide the model in distinguishing instance state attributes.

[0063] Example 1

[0064] This paper proposes a symmetric diffusion model based on multiple samples (SDMMS) framework for learning image feature distributions. The schematic diagram is shown below. The framework consists of five modules: an image encoder, a feature diffusion forward module, a feature diffusion backward module, a feature decoder, and an image alignment module. The image encoder uses a backbone feature extractor trained on open-source data as the basic image encoder. The model first generates a mixed feature map of basic and mid-level semantics, such as pre-trained backbone structure codes from VIT and SwinTransformer. The feature encoder also participates in model training. The feature diffusion forward module, similar to conventional diffusion models, repeatedly adds normally distributed noise to the feature map until it becomes a normally distributed noise feature map (the process marked with the ellipsis on the left in the figure below). The difference is that the feature-based diffusion learning adds a mask operation, which randomly crops part of the feature domain in the model's feature distribution, achieving feature-level data enhancement. The feature diffusion forward module of this invention is similar to the forward module of MAGE, but in the backward module, cross-learning is introduced to assist feature generation. As shown in the figure below, the model has two main branches: PersetImg1 and PersetImg2. These two branches share modules in the encoder, feature forward module, and feature backward model, but execute independently. The only intersection is the noise feature map and the learning objective of the feature backward module. Traditionally, the backward module only learns the distribution of the noise added in the forward pass to the corresponding image. This invention introduces a positive and negative sample mechanism, expanding the learning objective in the backward module to the fused noise distribution of the PersetImg1 and PersetImg2 branches, providing the model with more sequential learning guidance. There are also two branches on the right side of the image. The upper branch reconstructs PersetImg1 to obtain the PersetImg1Pred image; the lower branch reconstructs the imaging difference map at the same preset position. The feature decoder acts on the diffusion-generated feature map to deeply reconstruct the image feature distribution, resulting in the predicted image. The image alignment module is primarily responsible for calculating the difference between the predicted PersetImglPred and PersetImg1, and the difference between PersetImg1-2DiffPred and PersetImg1-2Diff, reconciling the losses between the two images, and optimizing the deep learning model in the feature backward module using backpropagation. The model tends to learn semantic representations of background objects when reconstructing PersetImg1, and foreground objects when reconstructing PersetImg1-2Diff. After unsupervised training, the model possesses basic foreground and background feature generation capabilities for power transmission scenarios.

[0065] Constructing a text-image cross-diffusion model based on positive and negative samples

[0066] The model after SDMMS has a strong ability to generate image features and has a good representation ability for the foreground and background of the image. However, there may also be large differences in background targets. For example, the spatial positions of moving vehicles and pedestrians in the same preset position in different pictures are very different. SDMMS will guide the model to learn these differences in the Diff branch. The present invention draws on the contrastive learning structure of the Clip algorithm, introduces the same preset position pictures in each batch, and uses the contrastive learning structure to strengthen the semantic differences of the pictures. Because the confused moving targets in the DIFF branch are semantically convergent, the contrastive learning module will allocate more attention to the changing things.

[0067] The Cross Diffusion Model Based on Multiple Samples (CDIBMS) uses a pair of images at preset locations and a textual description of the object as input (corresponding to the multi-source visual information input in step 2.1 and the textual description input in step 2.2). For example, two images of the same preset location in a camera image and the text "Damaged Greenhouse Film" are combined. A diffusion backbone network is designed to fuse multiple samples. The diffusion structure reuses the denoising module (feature generation model) trained by the diffusion backward module in SDMMS. The denoising model branch receives an original image and generates a semantic feature map using an image encoder. Normally distributed noise is introduced multiple times to form a noise feature map, which serves as one of the diffusion inputs of the CDIBMS model. Semantic feature maps are obtained from another image using the trained image encoder. A trainable Transformer module modulates the semantic feature maps of the positive and negative samples to form an image semantic encoding. The text encoding is fused using cross-attention and serves as the other input to the diffusion module. The diffusion model embeds the pixel encoding and image encoding in each feature generation layer using cross-attention. The feature map generated by diffusion is passed through a trainable ROIGenerate module to generate a mask of the instance region of interest, and the feature decoder in SDMMS is used to generate the instance feature encoding. The mask is used for supervised instance segmentation training, and the feature encoding is used for comparative learning of text-image pairs to align the semantic spaces of text and images. In the image branch, we introduce multi-image cross-attention modeling to improve the model's representation capabilities. In the text branch, this paper aligns the text semantic space with the image semantic space, resolving the confusion caused by the diffusion model in the representation of instance object categories. At the same time, the introduction of object descriptions guides the model to distinguish the state attributes of instances, such as "shed" and "broken shed."

[0068] Figure 2In the code, "loop n" indicates the addition of normal distribution noise n times, which corresponds to the feature diffusion forward module in the SDMMS model; "Encode" represents the visual backbone network pre-trained on the open dataset, which is used to extract the semantic feature map of the image; "TransformerBlock" is a learnable deep learning module for modulating the semantics of the image; "TextEncoder" is a pre-trained text encoder in NLP, which is used for information embedding of text information; the present invention uses cross-embedded text encoding and image encoding to obtain a global semantic guidance factor, and this guidance code is then cross-attentive calculated with the image generation features after "n loops". This process occurs in all layers of the SDMMS model.

[0069] In summary, the present invention has the following characteristics:

[0070] The SDMMS model introduces cross-learning of positive and negative samples to guide the model to distinguish foreground and background targets, making the model's attention distribution more reasonable and facilitating the discrimination of target objects and confusing targets.

[0071] SDMMS introduces differential learning to guide the model to pay more attention to changing objects, providing a basis for feature representation for object modeling;

[0072] The CDIBMS model introduces text encoding and image encoding as hint encoding for the diffusion model, guiding the features generated by diffusion to be more consistent with object representation.

[0073] Positive and negative samples are introduced into the CDIBMS model, and negative sample encoding is used to guide the diffusion model to generate features in different distribution domains to improve the discrimination ability.

Claims

1. A semi-supervised feature mining method for power transmission channels based on positive and negative samples, characterized in that: The following steps are involved: Step 1: Process the positive and negative samples through independent SwinTransformer feature encoders and feature diffusion forward modules to obtain a multi-head feature generator for the power transmission scenario; Step 1 specifically includes: Step 1.1: For positive and negative samples, independent SwinTransformer feature encoders and feature diffusion forward modules are used to obtain different feature combinations under the same sample distribution. Step 1.2: Perform the feature combination guided learning in step 1.1 on the randomly generated Gaussian noise feature map to obtain a feature generator for the power transmission scenario; Step 1.3: For the feature generator, create a negative sample image reconstruction task head so that the feature generator can learn to generate high-frequency semantic representations of background objects in the unsupervised reconstruction task; Step 1.4: For the feature generator, create a positive and negative sample difference feature map fitting task head, so that the feature generator learns to generate low-frequency foreground object semantic representation during the differential image fitting process; Step 1.5: Input the image reconstruction task head in step 1.3 and the difference feature map fitting task head in step 1.4 into the feature decoder respectively to obtain the negative sample prediction and the positive and negative sample difference map prediction; Step 1.6: For the prediction of negative samples and the prediction of the difference map of positive and negative samples, the loss is calculated with the negative sample and the difference feature map of positive and negative samples respectively, and a multi-head power transmission scene feature generator is obtained through the back propagation optimization model; Step 2: Process the negative and positive samples, generate a power transmission scene feature map through a cross-attention module and a multi-head feature generator, and process it to obtain instance segmentation prediction, scene object representation encoding, and instance category or instance description prediction; Step 2 specifically includes: Step 2.1: For negative samples, input them into the feature encoder and feature diffusion forward module in step 1.1 to obtain the diffused feature map; Step 2.2: Input the diffused feature map into the backbone of Swintransformer to obtain the feature map of the negative sample. Step 2.3: Input the positive sample into the backbone of VIT training to obtain the feature map of the positive sample; Step 2.4: Input the feature maps of positive samples and negative samples into the transformer module for training to obtain the cross-attention feature maps of positive samples and negative samples; Step 2.5: Input the text description information of the negative sample annotation into the TextEncode module to obtain the text prompt code; Step 2.6, encode the cross-attention feature map and text prompt of the positive and negative samples, input them into the cross-attention module, and obtain the text-image cross-attention prompt encoding; Step 2.7, encode the diffused feature map and the cross-attention prompt of the text and image in step 2.1, input them into the multi-head feature generator in step 1.6, and iterate to generate the transmission scene feature map; Step 2.8: Use the ROIGenerate module to process the transmission scene feature map to obtain the mask encoding of the instance region of interest to the model, that is, to obtain the foreground instance segmentation prediction; Step 2.9: Input the power transmission scene feature map into the feature decoder in step 1.5 for processing to obtain the scene object representation code; Step 2.10, segment the instance region, segment the corresponding region on the scene object representation code, and obtain the feature representation of the instance; Step 2.11: Encode the instance feature representation and text prompt, perform CLIP text-image alignment training, and obtain instance category or instance description prediction.

Images