Target detection method based on vision-text interaction of infrared visible light aerial image

Abstract

The invention relates to the technical field of remote sensing scene multi-mode aviation target detection, and provides a visual-text interaction target detection method based on an infrared visible light aviation image, and the method comprises the steps: building a feature extraction network; sequentially constructing an infrared and visible light feature fusion module, a fusion feature enhancement module, a vision-text interaction module and a vision-text detection module, and connecting the vision-text detection module with the feature extraction network, the feature fusion module and the fusion feature enhancement module to construct an aerial target detection model; and training the aviation target detection model, and packaging and deploying the trained aviation target detection model. According to the scheme, the target in the aviation image can be accurately distinguished according to the text prompt, and the accuracy of aviation image target detection is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of multimodal aerial target detection technology in remote sensing scenes, specifically to a visual-text interactive target detection method based on infrared and visible light aerial images. Background Technology

[0002] Aerial target detection, a core supporting technology for critical tasks such as urban planning, environmental monitoring, and disaster response, aims to accurately locate and classify various targets in aerial imagery (such as images taken by drones). Although deep learning-based methods have made significant progress in this field, current mainstream detection models still have obvious limitations: firstly, these models typically rely on single-source, single-modal data for training; secondly, their design follows a closed detection paradigm of "known category." These two factors together restrict the model's adaptability to different scenarios and limit its potential for generalization to open environments.

[0003] In recent years, open-vocabulary object detection has made positive progress in natural scene image processing. These methods, by introducing a visual-language model, align image region features with semantic text features, thereby enabling the identification of targets outside the training set with the help of textual cues, demonstrating strong cross-class generalization capabilities. However, natural scene images and aerial imagery differ significantly in terms of shooting perspective, target scale, and background complexity, making it difficult for open-vocabulary detection models trained on natural scene data to be directly generalized to aerial scene object detection tasks.

[0004] To address these issues, existing research has attempted to train visual-text interaction object detection models on aerial datasets. For example, CastDet pioneered open-vocabulary detection in remote sensing scenarios, proposing a teacher-student model detection framework and validating its effectiveness on multiple remote sensing datasets. DescReg, on the other hand, designed a zero-shot detection method based on visual regularization to address the weak semantic-visual correlation in remote sensing images. However, these methods still rely solely on visible light single-modal images for training, exhibiting limited generalization ability for low-quality images acquired under conditions such as low light or severe weather, and have not fully leveraged the detection advantages of multimodal data in complex aerial environments. Summary of the Invention

[0005] To address the problems existing in the background technology, this invention proposes a visual-text interactive target detection method based on infrared and visible light aerial images. This method can accurately distinguish targets in aerial images based on text prompts, effectively improving the accuracy of target detection in aerial images.

[0006] To achieve the above objectives, the present invention adopts the following solution: A visual-text interactive target detection method based on infrared and visible light aerial images, the method comprising: Step S1: Construct a feature extraction network; Step S2: Based on the feature extraction network, construct an infrared and visible light feature fusion module; Step S3: Based on the feature fusion module, construct a fusion feature enhancement module; Step S4: Based on the fusion feature enhancement module, construct a visual-text interaction module; Step S5: Construct a visual-text detection module and connect the visual-text detection module with the feature extraction network, feature fusion module, fusion feature enhancement module, and visual-text interaction module to construct an aviation target detection model; Step S6: Train the aviation target detection model and encapsulate and deploy the trained aviation target detection model.

[0007] Optionally, in step S1, the feature extraction network includes a parallel dual-branch convolutional network model and a text encoder; the dual-branch convolutional network model includes a first encoder and a second encoder with identical structures, the first encoder is used to extract infrared image features, and the second encoder is used to extract visible light image features, each encoder sequentially including a first convolutional module, a second convolutional module, a third convolutional module, a fourth convolutional module, and a fifth convolutional module; the text encoder is used to extract text features, and it includes a text segmenter, a position encoder, 12 identical attention modules, and an output projection layer.

[0008] Optionally, in step S2, the feature fusion module includes a channel spatial attention module, which includes a channel attention submodule and a spatial attention submodule.

[0009] Optionally, in step S2.1, the first encoder... i The infrared image feature map output by the first convolutional module and the first feature map in the second encoder i The visible light image feature maps output by each convolutional module are summed element-wise to obtain the result of summing two features element-wise. The result is obtained using the following formula: , ), In the formula, 3,4,5} indicates that the corresponding features belong to the third, fourth, and fifth convolutional modules; Indicates infrared image features; Represents visible light image features; Indicates the first encoder; Indicates the second encoder; and These represent infrared and visible light images, respectively; all feature maps have a dimension of . ; c Indicates the number of channels; h and w Describe the width and height of the feature map respectively; Step S2.2: The result The input is fed into the channel attention submodule to generate channel weights. and will and Multiplying by channel, the formula is as follows: , , In the formula, Represents a multilayer perceptron; express sigmoid function; For the first i In the nth convolutional module The weights corresponding to the feature maps of each channel; This indicates the characteristics after channel fusion; Step S2.3: Input the multiplied features into the spatial attention submodule to generate spatial pixel weights. and the spatial pixel weights The features input to the spatial attention submodule are multiplied element-wise to obtain the final fusion result of the infrared image features and the visible light image features, which is achieved by the following formula: , , In the formula, Indicates the first i Feature maps in each convolutional module The weight corresponding to the position; (•) indicates a convolution operation; (•) indicates that features are spliced ​​along the channel dimension; This indicates the fusion feature.

[0010] Optionally, in step S3, the feature enhancement module includes a top-down feature enhancement module and a bottom-up feature enhancement module, and the top-down feature enhancement module and the bottom-up feature enhancement module perform feature enhancement sequentially in the order of first top-down and then bottom-up. The top-down feature enhancement module includes two upsampling modules, a first enhanced convolution module, a second enhanced convolution module, and two feature concatenation modules; the bottom-up feature enhancement module includes two downsampling modules, a third enhanced convolution module, a fourth enhanced convolution module, and two feature concatenation modules.

[0011] Optionally, the execution steps of the feature enhancement module include: Step S3.1: The top-down feature enhancement module performs feature enhancement: ... The fused features are upsampled using bilinear interpolation to make the feature map size consistent with... After feature size alignment and concatenation along the channel dimension, the feature number is reduced by the first enhancement convolution module to obtain the feature. The features Continue upsampling using bilinear interpolation to make the feature map size consistent with... After feature size alignment and concatenation along the channel dimension, the feature number is reduced by a second enhanced convolutional module to obtain the feature. ; Step S3.2: The bottom-up feature enhancement module performs feature enhancement: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] Downsampling is performed using a downsampling convolution module to make the feature map size equal to the stated size. After feature size alignment and concatenation along the channel dimension, the number of feature channels is reduced by a third enhanced convolutional module to obtain the feature. The features Continue downsampling using the downsampling convolution module to make the feature map size similar to the stated... After feature size alignment and concatenation along the channel dimension, the feature number is reduced by a fourth enhanced convolutional module to obtain the feature. ; Step S3.3: Obtain the enhanced view features: ， ， .

[0012] Optionally, in step S4, the visual-text interaction module includes a text feature adaptation module and a visual-text feature alignment module, wherein the text feature adaptation module includes a multilayer perceptron module; and the visual-language feature alignment module is trained based on a contrastive loss function.

[0013] Optionally, the execution steps of the text feature adaptation module include: extracting text features through a text encoder, and adjusting the dimension of the text features to be consistent with the dimension of the visual features using the multilayer perceptron module; The execution steps of the visual-text feature alignment module include: obtaining the output of the fusion feature enhancement module. ， ， Three visual feature maps of different sizes are used to uniformly represent the visual feature maps as follows: , After projecting visual features onto the same dimension as text features using a convolutional module, the similarity between text features and visual features is calculated. The similarity calculation formula is as follows: , In the formula, It is the first k Text features, Indicates the first j A visual feature, and It is a learnable scaling and offset factor. This represents the similarity value between text and visual features. D Representing feature dimension, d Indicates which dimension of the feature; Training is completed using a similarity-based visual-text contrastive loss function, the formula for which the contrastive loss function is calculated is as follows: , In the formula, N The total logarithm of the visual features of the text. 1,2,...N}, Indicates positive sample similarity. Indicates negative sample similarity. For temperature coefficient, exp (•) indicates taking natural numbers. e The logarithm of .

[0014] Optionally, in step S5, the visual-text detection module includes a contrast classification head and a bounding box prediction head; the contrast classification head is used to predict the category of the target, and the bounding box prediction head is used to predict the position of the target in the image.

[0015] Optionally, the execution steps of the visual-text detection module include: For the classification head, calculate the visual region features. V Text features T The similarity is used to determine the target category, wherein the target category is the category corresponding to the text feature that has the greatest similarity to the region feature; For the bounding box prediction head, for any location in the feature map ( i , j ), and the regression method was used to predict NThe location information of each object is used to remove duplicate bounding boxes using a non-maximum suppression algorithm to determine the final location of the target object. The model is then optimized using a size-aware fitting loss function and a distributed focal loss function.

[0016] The beneficial effects of this invention are as follows: This scheme overcomes the limitation of traditional single-modality detection models due to the single data source by extracting visual features from infrared images and visible light images, as well as semantic features from text descriptions in parallel; to effectively fuse the thermal radiation feature information of targets in infrared images and the detailed texture feature information of targets in visible light images, an infrared and visible light feature fusion module is constructed to improve the accuracy and generalization ability of target detection in complex aviation scenarios such as low light and severe weather; to remove redundant information from the fused features and extract multi-scale target information, an infrared and visible light feature enhancement module is constructed to avoid the repeated expression of the same background area information by the two types of features and to achieve the detection of targets of different sizes; to achieve accurate matching between visual features and text semantic features, a visual-text interaction module is constructed to establish a mapping relationship between visual features and text features, giving full play to the guiding role of text semantics in target detection; to obtain the results of the target detection task, a visual-text detection module is constructed to achieve accurate output of target category and location.

[0017] Therefore, this method can accurately identify and distinguish target objects in aerial images based on text prompts, effectively enhancing the model's adaptability and generalization performance in complex aviation scenarios, and achieving a significant improvement in the model's scenario adaptability and generalization ability. Attached Figure Description

[0018] Figure 1 A flowchart of the target detection method provided in the embodiments of the present invention; Figure 2 This is an overall structural diagram of the target detection model provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the present invention clearer and more understandable, the present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the given embodiments are only one implementation method and do not represent all embodiments.

[0020] Combination Figure 1 This invention provides a visual-text interactive target detection method based on infrared and visible light aerial images, the method comprising the following steps: Step S1: Build a feature extraction network.

[0021] In this embodiment, the feature extraction network includes a dual-branch convolutional network model and a text encoder, which extract features in parallel. The dual-branch convolutional network model includes a first encoder and a second encoder with identical structures. The first encoder is used to extract features from infrared images, and the second encoder is used to extract features from visible light images. Each encoder sequentially includes a first convolutional module, a second convolutional module, a third convolutional module, a fourth convolutional module, and a fifth convolutional module. The text encoder is used to extract text features and includes a text segmenter, a position encoder, 12 identical attention modules, and an output projection layer.

[0022] Step S2: Based on the feature extraction network, construct an infrared and visible light feature fusion module.

[0023] In this embodiment, the feature fusion module includes a channel spatial attention module, which includes a channel attention submodule and a spatial attention submodule.

[0024] Specifically, the execution steps of the channel spatial attention module include: Step S2.1: The first encoder... i The infrared image feature map output by the first convolutional module and the first feature map in the second encoder i The visible light image feature maps output by each convolutional module are summed element-wise to obtain the result of summing two features element-wise. The result is obtained using the following formula: , ), In the formula, 3,4,5} indicates that the corresponding features belong to the third, fourth, and fifth convolutional modules; Indicates infrared image features; Represents visible light image features; Indicates the first encoder; Indicates the second encoder; and These represent infrared and visible light images, respectively; all feature maps have a dimension of . ; c Indicates the number of channels; h and w These represent the width and height of the feature map, respectively. Element-wise addition refers to adding each channel... c Each spatial location ( h , w The eigenvalues ​​of ) are superimposed.

[0025] Step S2.2: The result The input is fed into the channel attention submodule to generate channel weights. and will and Multiplying by channel, the formula is as follows: , , In the formula, Represents a multilayer perceptron; express sigmoid function; For the first i In the nth convolutional module The weights corresponding to the feature maps of each channel; This indicates the characteristics after channel fusion; Step S2.3: Input the multiplied features into the spatial attention submodule to generate spatial pixel weights. and the spatial pixel weights The features input to the spatial attention submodule are multiplied element-wise to obtain the final fusion result of the infrared image features and the visible light image features, i.e. , , It is achieved through the following formula: , , In the formula, Indicates the first i Feature maps in each convolutional module The weight corresponding to the position; (•) indicates a convolution operation; (•) indicates that features are spliced ​​along the channel dimension; This indicates the fusion feature.

[0026] Step S3: Based on the feature fusion module, construct a fusion feature enhancement module.

[0027] In this embodiment, the feature enhancement module includes a top-down feature enhancement module and a bottom-up feature enhancement module. The top-down feature enhancement module and the bottom-up feature enhancement module perform feature enhancement sequentially in the order of first top-down and then bottom-up.

[0028] The top-down feature enhancement module includes two upsampling modules, a first enhanced convolution module, a second enhanced convolution module, and two feature concatenation modules; the bottom-up feature enhancement module includes two downsampling modules, a third enhanced convolution module, a fourth enhanced convolution module, and two feature concatenation modules.

[0029] Specifically, the execution steps of the feature enhancement module include: Step S3.1: The top-down feature enhancement module performs feature enhancement: ... The fused features are upsampled using bilinear interpolation to make the feature map size consistent with... After feature size alignment and concatenation along the channel dimension, the feature number is reduced by the first enhanced convolution module to obtain the feature. The features Continue upsampling using bilinear interpolation to make the feature map size consistent with... After feature size alignment and concatenation along the channel dimension, the feature number is reduced by a second enhanced convolutional module to obtain the feature. .

[0030] Step S3.2: The bottom-up feature enhancement module performs feature enhancement: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] Downsampling is performed using a downsampling convolution module to make the feature map size equal to the stated size. After feature size alignment and concatenation along the channel dimension, the number of feature channels is reduced by a third enhanced convolutional module to obtain the feature. The features Continue downsampling using the downsampling convolution module to make the feature map size similar to the stated... After feature size alignment and concatenation along the channel dimension, the feature number is reduced by a fourth enhanced convolutional module to obtain the feature. .

[0031] Step S3.3: Obtain the enhanced visual features: , , .

[0032] Step S4: Based on the fusion feature enhancement module, construct a visual-text interaction module.

[0033] In this embodiment, the visual-text interaction module includes a text feature adaptation module and a visual-text feature alignment module. The text feature adaptation module includes a multilayer perceptron module. The visual-language feature alignment module is trained based on a contrastive loss function.

[0034] Specifically, the execution steps of the text feature adaptation module include: Step S4.1: Extract text features using a text encoder. The expression for this feature is: , In the formula, Represents text, Indicates a text encoder. Representing text features; Step S4.2: Use the multilayer perceptron module to adjust the dimension of the text features to be consistent with the dimension of the visual features. The expression is: , In the formula, (•) is the activation function. 1,2,3} represents the number of layers in a multilayer perceptron. and These represent the weights and biases of each perceptron layer, respectively. This represents the text features output by each layer of the perceptron.

[0035] Specifically, the execution steps of the visual-text feature alignment module include: Step S4.3: Obtain the output of the fusion feature enhancement module. , , Three visual feature maps of different sizes are used to uniformly represent the visual feature maps as follows: The visual feature maps are used to detect targets of different sizes.

[0036] Step S4.4: After projecting the visual features onto the same dimension as the text features using a convolution module, calculate the similarity between the text features and the visual features. The similarity calculation formula is as follows: , In the formula, It is the first k Text features, Indicates the first j A visual feature, and It is a learnable scaling and offset factor. This represents the similarity value between text and visual features. D Representing feature dimension, d Indicates which dimension of the feature; Step S4.5: With the training objective of maximizing the similarity between matched text features and visual features, and minimizing the similarity between mismatched text features and visual features, a similarity-based visual-text contrast loss function is introduced to complete the training. The calculation formula of the contrast loss function is as follows: , In the formula, N The total logarithm of the visual features of the text. 1,2,...N}, Indicates positive sample similarity. Indicates negative sample similarity. For temperature coefficient, exp (•) indicates taking natural numbers. eThe logarithm of .

[0037] Step S5: Construct a visual-text detection module and connect the visual-text detection module with the feature extraction network, feature fusion module, fusion feature enhancement module, and visual-text interaction module to construct an aviation target detection model.

[0038] In this embodiment, the visual-text detection module includes a contrast classification head and a bounding box prediction head; the contrast classification head is used to predict the category of the target, and the bounding box prediction head is used to predict the position of the target in the image.

[0039] Specifically, the execution steps of the visual-text detection module include: For the classification head, calculate the visual region features. V Text features T The similarity is used to determine the target category, which is the category corresponding to the text feature with the highest similarity to the region feature.

[0040] For the bounding box prediction head, for any location in the feature map ( i , j ), and the regression method was used to predict N The location information of each object is used to remove duplicate bounding boxes using a non-maximum suppression algorithm to determine the final location of the target object. To improve the accuracy of the predicted object location, a size-aware fitting loss function and a distributed focal loss function are used to optimize the model.

[0041] The safit loss formula is as follows: , In the formula, A It is the actual bounding box area. C It is a constant. IOU It is the intersection-union ratio of the predicted results and the actual results. NWD It is normalized Wasserstein distance.

[0042] Step S6: Train the aviation target detection model and encapsulate and deploy the trained aviation target detection model.

[0043] Specifically, the training process adopts a training paradigm of "dual-modal data collaborative input + text prompt guidance". First, the infrared and visible light images are pixel aligned and registered, and their sizes are normalized. At the same time, the text descriptions are standardized to generate structured prompt text, thus constructing a ternary training sample of infrared-visible light-text. After data processing, a suitable optimizer is selected, the number of training rounds is set to 50, the initial learning rate is set to 0.00002, and the model parameters are optimized using a combined loss function and backpropagation algorithm. The training is completed on a graphics card.

[0044] Furthermore, the deployment and packaging process first performs lightweight processing on the trained optimal model, thereby reducing the number of model parameters and improving the model's computing speed while maintaining low accuracy. In order to adapt to different models of graphics cards, the model needs to be converted into different inference formats. Then, a running environment is built for the model and corresponding data interfaces are developed to support the input of infrared and visible light images and text prompts, as well as the output of corresponding detection results. Finally, the real-time performance, accuracy, and stability of the model are verified in a real aviation harsh scenario test set.

[0045] This invention addresses the limitations of single-modal perception in adverse weather conditions by proposing a visual-text interaction-based target detection method using infrared and visible light aerial images. This method overcomes the limitation of traditional single-modal detection models due to their reliance on a single data source by extracting visual features from infrared and visible light images and semantic features from text descriptions in parallel. To effectively fuse the thermal radiation features of targets in infrared images and the detailed texture features of targets in visible light images, an infrared and visible light feature fusion module is constructed to improve the accuracy and generalization ability of target detection in complex aviation scenarios such as low light and adverse weather. To remove redundant information from the fused features and extract multi-scale target information, an infrared and visible light feature enhancement module is constructed to avoid redundant representation of the same background area by the two types of features and to enable the detection of targets of different sizes. To achieve accurate matching between visual features and text semantic features, a visual-text interaction module is constructed to establish a mapping relationship between visual and text features, fully leveraging the guiding role of text semantics in target detection. Finally, to obtain the results of the target detection task, a visual-text detection module is constructed to achieve accurate output of target category and location.

[0046] In summary, as Figure 2 As shown, this invention achieves the ability to accurately distinguish targets in aerial images based on text prompts through a feature extraction network, a feature fusion module, a fusion feature enhancement module, a visual-text interaction module, and a visual-text detection module, thereby significantly improving the model's adaptability and generalization ability in complex aviation scenarios.

[0047] The specific embodiments of the present invention have been described in detail above with reference to the figures, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A target detection method based on infrared and visible light aerial images for visual-text interaction, characterized in that, The method includes: Step S1: Construct a feature extraction network; Step S2: Based on the feature extraction network, construct an infrared and visible light feature fusion module; Step S3: Based on the feature fusion module, construct a fusion feature enhancement module; Step S4: Based on the fusion feature enhancement module, construct a visual-text interaction module; Step S5: Construct a visual-text detection module and connect the visual-text detection module with the feature extraction network, feature fusion module, fusion feature enhancement module, and visual-text interaction module to construct an aviation target detection model; Step S6: Train the aviation target detection model and encapsulate and deploy the trained aviation target detection model.

2. The method according to claim 1, characterized in that: In step S1, the feature extraction network includes a parallel dual-branch convolutional network model and a text encoder; The dual-branch convolutional network model includes a first encoder and a second encoder with identical structures. The first encoder is used to extract infrared image features, and the second encoder is used to extract visible light image features. Each encoder includes a first convolutional module, a second convolutional module, a third convolutional module, a fourth convolutional module, and a fifth convolutional module in sequence. The text encoder is used to extract text features and includes a text segmenter, a position encoder, 12 identical attention modules, and an output projection layer.

3. The method according to claim 1, characterized in that: In step S2, the feature fusion module includes a channel spatial attention module, which includes a channel attention submodule and a spatial attention submodule.

4. The method according to claim 3, characterized in that: The execution steps of the channel space attention module include: Step S2.1: Transfer the first encoder to the first encoder. i The infrared image feature map output by the first convolutional module and the first feature map in the second encoder i The visible light image feature maps output by each convolutional module are summed element-wise to obtain the result of summing two features element-wise. The result is obtained using the following formula: ， ）， In the formula, 3,4,5} indicates that the corresponding features belong to the third, fourth, and fifth convolutional modules; Indicates infrared image features; Represents visible light image features; Indicates the first encoder; Indicates the second encoder; and These represent infrared and visible light images, respectively; all feature maps have a dimension of . ; c Indicates the number of channels; h and w Describe the width and height of the feature map respectively; Step S2.2: The result The input is fed into the channel attention submodule to generate channel weights. and will and Multiplying by channel, the formula is as follows: ， ， In the formula, This represents a multilayer perceptron; express sigmoid function; For the first i In the nth convolutional module The weights corresponding to the feature maps of each channel; This indicates the characteristics after channel fusion; Step S2.3: Input the multiplied features into the spatial attention submodule to generate spatial pixel weights. and the spatial pixel weights The features input to the spatial attention submodule are multiplied element-wise to obtain the final fusion result of the infrared image features and the visible light image features, which is achieved by the following formula: ， ， In the formula, Indicates the first i Feature maps in each convolutional module The weight corresponding to the position; (•) indicates a convolution operation; (•) indicates that features are spliced ​​along the channel dimension; This indicates the fusion feature.

5. The method according to claim 1, characterized in that: In step S3, the feature enhancement module includes a top-down feature enhancement module and a bottom-up feature enhancement module. The top-down feature enhancement module and the bottom-up feature enhancement module perform feature enhancement sequentially in the order of first top-down and then bottom-up. The top-down feature enhancement module includes two upsampling modules, a first enhanced convolution module, a second enhanced convolution module, and two feature concatenation modules; the bottom-up feature enhancement module includes two downsampling modules, a third enhanced convolution module, a fourth enhanced convolution module, and two feature concatenation modules.

6. The method according to claim 5, characterized in that: The execution steps of the feature enhancement module include: Step S3.1: The top-down feature enhancement module performs feature enhancement: ... The fused features are upsampled using bilinear interpolation to make the feature map size consistent with... After feature size alignment and concatenation along the channel dimension, the feature number is reduced by the first enhancement convolution module to obtain the feature. The features Continue upsampling using bilinear interpolation to make the feature map size consistent with... After feature size alignment and concatenation along the channel dimension, the feature number is reduced by a second enhanced convolutional module to obtain the feature. ; Step S3.2: The bottom-up feature enhancement module performs feature enhancement: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] Downsampling is performed using a downsampling convolution module to make the feature map size equal to the stated size. After feature size alignment and concatenation along the channel dimension, the number of feature channels is reduced by a third enhanced convolutional module to obtain the feature. The features Continue downsampling using the downsampling convolution module to make the feature map size similar to the stated... After feature size alignment and concatenation along the channel dimension, the feature number is reduced by a fourth enhanced convolutional module to obtain the feature. ; Step S3.3: Obtain the enhanced view features: ， ， .

7. The method according to claim 6, characterized in that: In step S4, the visual-text interaction module includes a text feature adaptation module and a visual-text feature alignment module. The text feature adaptation module includes a multilayer perceptron module. The visual-language feature alignment module is trained based on a contrastive loss function.

8. The method according to claim 7, characterized in that: The execution steps of the text feature adaptation module include: extracting text features through a text encoder, and adjusting the dimension of the text features to be consistent with the dimension of the visual features using the multilayer perceptron module; The execution steps of the visual-text feature alignment module include: obtaining the output of the fusion feature enhancement module. ， ， Three visual feature maps of different sizes are used to uniformly represent the visual feature maps as follows: , After projecting visual features onto the same dimension as text features using a convolutional module, the similarity between text features and visual features is calculated. The similarity calculation formula is as follows: ， In the formula, It is the first k Text features, Indicates the first j A visual feature, and It is a learnable scaling and offset factor. This represents the similarity value between text and visual features. D Representing feature dimension, d Indicates which dimension of the feature; Training is completed using a similarity-based visual-text contrastive loss function, the formula for which the contrastive loss function is calculated is as follows: ， In the formula, N The total logarithm of the visual features of the text. 1,2,...N}, Indicates positive sample similarity. Indicates negative sample similarity. For temperature coefficient, exp (•) indicates taking natural numbers. e The logarithm of .

9. The method according to claim 1, characterized in that: In step S5, the visual-text detection module includes a contrast classification head and a bounding box prediction head; the contrast classification head is used to predict the category of the target, and the bounding box prediction head is used to predict the position of the target in the image.

10. The method according to claim 9, characterized in that: The execution steps of the visual-text detection module include: For the classification head, calculate the visual region features. V Text features T The similarity is used to determine the target category, wherein the target category is the category corresponding to the text feature that has the greatest similarity to the region feature; For the bounding box prediction head, for any location in the feature map ( i , j The regression method was used to predict the result. N The location information of each object is used to remove duplicate bounding boxes using a non-maximum suppression algorithm to determine the final location of the target object. The model is then optimized using a size-aware fitting loss function and a distributed focal loss function.

Images