Single-domain generalization target detection method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of target detection, in particular to a single-domain generalization target detection method and device, electronic equipment and a storage medium, when it is detected that no target domain potential target is found, a target image is acquired, a text prompt template is constructed to obtain preliminary text description, an image classification model is pre-trained to extract image features of a target image source field, and the target image source field image features are extracted; and pre-training the text classification model to extract a target domain text prompt, obtaining a target domain style parameter, and determining a target domain visual feature in combination with the source domain image feature. And finally, inputting the visual features of the target field and the text prompt into a preset language-image positioning model, so that the preset language-image positioning model outputs a prediction detection frame corresponding to the potential target and a target text prompt. According to the method, the detection capability of the model in the unseen target domain is improved, and more accurate single-domain generalization target detection is realized.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a single-domain generalized target detection method, apparatus, electronic device and storage medium. Background Technology

[0002] In recent years, with the rapid development of deep learning and computer vision technologies, object detection capabilities have significantly improved. Especially with the support of large-scale datasets and advanced models, object detection models perform well based on the assumption that the training and test datasets follow the same distribution. However, in open environments, due to factors such as environment, equipment, and human intervention, the distribution of the test dataset can vary, requiring the labeling of large amounts of data to adapt to any possible data distribution, resulting in significant manpower and computational resources. To address this, domain adaptation techniques aim to enhance the generalization ability of models trained on the training set (source domain) on the test set (target domain), where the two domains are somewhat correlated but follow different distributions. These techniques are widely used in image classification, semantic segmentation, and object detection, achieving excellent performance.

[0003] However, domain adaptation typically requires data from both the source and target domains to participate in the training process, severely limiting the model's generalization ability to unseen target domains. Therefore, it is necessary to improve the generalization ability of models trained on one or more source domains to unseen target domains. Furthermore, existing domain generalization techniques often rely on multiple labeled source domains to obtain highly generalized models, leading to significant annotation costs. Since object detection is much more complex than image classification, one current approach to single-domain generalization in object detection is through cyclic decoupling and self-distillation, decoupling domain-invariant features from domain-specific features without domain-related annotation supervision. Another approach utilizes a contrastive language-image pre-trained model, designing a semantic enhancement method to convert visual features of the source domain into the style of the target domain, while constructing a text-based classification loss, trained under the supervision of the text embeddings of the contrastive language-image pre-trained model. However, contrastive language-image pre-trained models typically provide image-level visual representations, failing to capture object-level visual representations in object detection tasks, limiting the accuracy of style transfer methods. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, the present invention provides a single-domain generalized target detection method, apparatus, electronic device and storage medium to achieve more accurate single-domain generalized target detection.

[0005] A first aspect of this application provides a single-domain generalized target detection method, the method comprising: When a potential target not found in the target domain is detected, the target image of the potential target is acquired; Construct a text prompt template to obtain a preliminary text description of the target image; A pre-trained image classification model is used to extract source domain image features from the target image source region. A pre-trained text classification model is used to extract target domain text prompts for the initial text description target domain through the text classification model; Obtain target domain style parameters, and determine target domain visual features based on the target domain style parameters and source domain image features; The visual features of the target domain and the text prompts of the target domain are input into a preset language-image localization model, so that the language-image localization model outputs the predicted detection box corresponding to the potential target and the corresponding target text prompts.

[0006] In an optional implementation, the method further includes: Construct object detection and localization loss functions and region-phrase loss functions; The total loss function of the language-image localization model is constructed based on the target detection and localization loss function and the region-phrase loss function.

[0007] In an optional implementation, the construction of the target detection and localization loss function includes: The target detection and localization loss function is determined using the following formula: ; in, The target detection and localization loss function is... Indicates the detector's prediction of the first i The j-th bounding box parameter of the image. Indicates the first i The first picture j The true target parameters of a bounding box n This represents the total number of images in the training set. m Indicates the first i The total number of bounding boxes in the images; The target detection and localization loss function is implemented using the Smooth L1 loss, which is defined by the following formula: ; in, x This represents the difference between the predicted and actual values.

[0008] In an optional implementation, the construction of the region-phrase loss function includes: The region-phrase loss function is determined using the following formula: ; in, For the region-phrase loss function, , , For the pre-trained image classification model, For the pre-trained text classification model, Represents an object region, the number of which is , This indicates phrase embeddings, the number of which is... , Represents the source domain image. This indicates the target field text prompt.

[0009] In an optional implementation, the step of constructing the total loss function of the language-image localization model based on the target detection and localization loss function and the region-phrase loss function includes: The total loss function is determined using the following formula: ; in, These are the hyperparameters for the corresponding terms.

[0010] In an optional implementation, determining the visual features of the target domain based on the target domain style parameters and the source domain image features includes: The visual features of the target domain are determined using the following formula: ; in, The visual features of the target domain and The target domain style parameters, The source domain image features, and These represent the channel mean and standard deviation of the target domain features, respectively.

[0011] In one alternative implementation, the image classification model uses the open-source SwinTransformer model pre-trained on the ImageNet dataset, and the text classification model uses the mainstream BERT architecture.

[0012] A second aspect of this application provides a single-domain generalized target detection device, the device comprising: The acquisition module is used to acquire the target image of the potential target when a potential target in an unseen target domain is detected. A construction module is used to construct a text prompt template to obtain a preliminary text description of the target image; The first extraction module is used to pre-train an image classification model to extract source domain image features of the target image source region through the image classification model; The second extraction module is used to pre-train a text classification model to extract target domain text prompts of the preliminary text description target domain through the text classification model; The style transfer module is used to obtain target domain style parameters and determine target domain visual features based on the target domain style parameters and the source domain image features; The detection module is used to input the visual features of the target domain and the text prompts of the target domain into a preset language-image localization model, so that the language-image localization model outputs the predicted detection box corresponding to the potential target and the corresponding target text prompts.

[0013] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the single-domain generalized target detection method.

[0014] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described single-domain generalized target detection method.

[0015] In summary, the single-domain generalized object detection method, apparatus, electronic device, and storage medium provided in this application acquire target images of potential targets and construct text prompt templates to generate text descriptions that are initially related to the target images, helping to establish the association between language and images. A pre-trained image classification model is responsible for extracting useful features from source domain images to retain relevant information about potential targets and adapt to new target domains. The text classification model extracts feature prompts from the target domain through the initial text descriptions, further enhancing semantic understanding. The acquisition of target domain style parameters and their combination with source domain image features aim to generate visual features that conform to the style of the target domain. Finally, the language-image localization model receives visual and text features, performs comprehensive analysis, and outputs predicted detection boxes and target text prompts, achieving accurate object detection and semantic description. By acquiring features from the source domain and adjusting them to new features suitable for the target domain, combined with language and visual information, it effectively solves the problem of generalization in unseen target domains, improves the model's adaptability and detection accuracy in new environments, and achieves more accurate single-domain generalized object detection. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a single-domain generalized target detection method according to an embodiment of this application; Figure 2This is a schematic diagram illustrating one usage scenario in an embodiment of this application; Figure 3 This is a schematic diagram illustrating a single-domain generalized target detection method based on phrase localization and style transfer, as shown in an embodiment of this application. Figure 4 This is a functional block diagram of a single-domain generalized target detection device shown in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0018] The following will clearly and completely describe the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. Furthermore, all connections / linkages involved in the patent do not simply refer to direct contact between components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this invention can be combined interactively without contradicting each other.

[0019] Reference Figure 1 The diagram shown is a flowchart illustrating a single-domain generalized target detection method according to an embodiment of this application. The single-domain generalized target detection method includes the following steps.

[0020] S11, when a potential target in an unseen target domain is detected, acquire the target image of the potential target.

[0021] In street scene target (vehicle, pedestrian, etc.) detection tasks under severe weather conditions, refer to Figure 2 As shown, in order to detect the actual specific target, when a potential target in the unseen target domain is obtained, an image containing the potential target is acquired, which is called the target image of the potential target.

[0022] S12, construct a text prompt template to obtain a preliminary text description of the target image.

[0023] In some embodiments, the electronic device can predefine text cue templates, including domain-specific cue text templates and general cue text templates. For domain-specific cue text templates, a series of phrases are used to construct text cuees. For example, when processing a driving dataset captured under foggy conditions, prefixes or suffixes are added around the category labels to generate phrases describing categories in a specific domain, and these phrases are connected by commas to form a text cue for the target domain. For general cue text templates, all potential style information from unseen target domains is included, along with prefixes and suffixes. For example, phrases like "a car driving in a sunny, foggy, or rainy scene" might be used. Domain-specific cueing facilitates style transfer from a source domain to a given target domain where images are inaccessible, thereby improving the model's domain-specific generalization ability.

[0024] When a potential object in an unseen target region is detected and its image is acquired, a text description of the potential object in the image is constructed using a pre-built text cue template; this is called a preliminary text description. That is, based on the text cue template, a set of phrases is used to describe each potential object in an unseen target region, such as a person or car in a road scene under different weather conditions.

[0025] S13, pre-train an image classification model to extract source domain image features of the target image source region through the image classification model.

[0026] In some embodiments, the electronic device may pre-train a backbone network for image classification. In this application embodiment, the mainstream open-source SwinTransformer model, pre-trained on the ImageNet dataset, can be used as the image classification model, denoted as . This is used to extract image features from the source neighborhood, and the extracted source neighborhood image features are denoted as... , where h, w, and c are the height, width, and number of channels, respectively.

[0027] S14, Pre-train a text classification model to extract target domain text prompts for the preliminary text description target domain through the text classification model.

[0028] In some embodiments, the electronic device may pre-train a backbone network for text classification. In this application embodiment, the mainstream BERT architecture can be used as the text image classification model, denoted as... It is used to extract target domain phrase embeddings, also known as target domain text hints.

[0029] S15, obtain the target domain style parameters, and determine the target domain visual features based on the target domain style parameters and the source domain image features.

[0030] Among them, the style parameters of the target domain (also known as the target domain style parameters) are the channel mean values ​​of the image features in the target domain. and standard deviation This method transforms image features from the source domain to the target domain using target domain style parameters. However, since data from the target domain is unavailable, the corresponding target domain style parameters are also unavailable. Therefore, we first calculate the channel mean... and standard deviation Two parameters are randomly initialized and then input into the style transfer module to obtain the transferred target domain features. Next, the electronic device can calculate the loss function and, by continuously updating the loss function, achieve backpropagation to update the target domain style parameters. and After initializing the target domain style parameters, the initialized target domain style parameters and the source domain image features extracted by the image classification model are input into the style transfer module to achieve style transfer of image features and obtain the target domain image features. Specifically, electronic devices can use adaptive instance normalization technology to construct the style transfer module, using the mean and standard deviation to characterize the style of the domain. The style of the source domain can be abstracted as... and , Let the source neighborhood image features be represented by two functions. and Returns the channel mean and standard deviation of the feature, respectively. Therefore, any target domain style parameters can be used. Source domain image features Stylization is achieved using the following formula: ; in, and These represent the channel mean and standard deviation of the target domain style parameters, or the channel mean and standard deviation of the target domain features, respectively.

[0031] Compared to existing object detection models, which exhibit poor generalization ability in complex environments (such as varying weather and lighting conditions), especially under adverse conditions like nighttime, rainy, and foggy weather, where model detection performance significantly degrades, this application utilizes a style transfer method. This method trains the model using data from only a single source domain and generates features with styles similar to the target domain through style transfer, thereby improving the model's generalization ability in complex environments.

[0032] It should be noted that after step S13 is completed, step S15 can be executed first, followed by step S14; or step S14 can be executed first, followed by steps S13 and S15; or steps S13 and S14 can be executed simultaneously, followed by step S15.

[0033] S16, input the visual features of the target domain and the text prompts of the target domain into a preset language-image localization model, so that the language-image localization model outputs the predicted detection box corresponding to the potential target and the corresponding target text prompts.

[0034] In some embodiments, the electronic device can construct a language-image localization model based on the SwinTransformer network, the Bert network, and the DynamicHead module. When the visual features and text prompts of the target domain are obtained, the visual features and text prompts of the target domain are constructed into target domain image-text data, and the target domain image-text data is input into the language-image localization model. The language-image localization model outputs the predicted detection box and the corresponding target text prompt.

[0035] Refer to together Figure 3 In some embodiments, to train a language-image localization model, the electronic device needs to acquire source domain images containing targets (e.g., cars, pedestrians, etc.) as a model training dataset. These source domain images are acquired under specific conditions (e.g., sunny during the day, foggy during the day, rainy during the day, rainy at night, etc.). For each target in each training image in the model training dataset, the ground truth label and category information of its detection box position are annotated, typically represented as bounding box coordinates. Simultaneously, language cues for the target domain are defined, such as "Daytime, Car, Foggy, …" and "Daytime, Person, Foggy, …", to describe the characteristics of the target domain and possible target objects, providing basic category information for subsequent phrase embedding. Then, an image encoder is used… Feature extraction is performed on the source domain image, converting it into a high-dimensional visual feature representation, and a text encoder is used. The target domain prompts are encoded, converting textual information into phrase feature representations compatible with the visual feature space. These extracted target domain phrase features are then embedded into the visual feature space, allowing for the identification of regions related to the target domain phrases within the visual features, providing guidance for subsequent style transfer. Furthermore, style transfer is performed based on phrase localization, establishing a correspondence between visual feature regions and phrase features. By calculating the similarity between visual and phrase features, it's determined which visual regions are related to which phrase features. For example, a visual region related to the phrase "car" might be the part of the image containing the car. Based on the region-phrase alignment results, style transfer is performed on the source domain visual features. This means that electronic devices can adjust the feature distribution of regions in the source domain visual features related to the target domain phrases, making them closer to the feature distribution of the target domain. For instance, if the source domain is a street scene in sunny weather and the target domain is a street scene in heavy rain, style transfer transforms the style of the source domain image into the style of heavy rain weather, resulting in transferred target domain features. Additionally, a detection head is added to the feature extraction network to predict the position and category of detection boxes based on the extracted features—that is, the predicted detection box corresponding to the potential target and the corresponding target text prompt. The detection head typically consists of fully connected layers or convolutional layers, outputting the coordinates and class probabilities of the detection boxes. The transferred target domain features are input into the model's detection head, and after processing by the feature extraction network and the detection head, the model's predicted detection box positions are obtained. Before training the language-image localization model, the electronic device needs to construct a loss function for the language-image localization model. First, the true labels of the detection boxes based on the source domain's localization labels and the detection box positions predicted by the model's detection head based on the transferred target domain features are obtained. Based on this, the target detection localization loss function is constructed. Specifically, the target detection localization loss function is determined using the following formula. : ; in, Indicates the detector's prediction of the first i The j-th bounding box parameter of the image. Indicates the first i The first picture j The true target parameters of a bounding box n This represents the total number of images in the training set. m Indicates the first i The total number of bounding boxes in the images; The target detection and localization loss function is implemented using the Smooth L1 loss, which is defined by the following formula: ; in, x This represents the difference between the predicted and actual values.

[0036] By constructing a localization loss and refining the coordinates of the bounding box, the predicted bounding box can more accurately match the real target, which helps to improve the accuracy of target detection.

[0037] Simultaneously, to align the object regions in the source domain with the phrases in the target domain prompts, the electronic device also needs to construct a region-phrase loss function. Specifically, following the same implementation method as steps S12 to S15 above, the electronic device can process training images in the model training dataset and construct text descriptions for the training images. Using an image classification model, source domain image features can be extracted, and based on style transfer, these source domain image features can be transferred to target domain image features. Simultaneously, based on a text classification model, target domain text features can be extracted. A region-phrase loss function is constructed based on the target domain image features and the target domain text features. Specifically, the region-phrase loss function is determined using the following formula. : ; Where R represents the image features of the target domain. W represents the text features of the target domain. , For the pre-trained image classification model, For the pre-trained text classification model, Represents an object region, the number of which is , This indicates phrase embeddings, the number of which is... , This represents the source domain image, also known as the training image. This refers to the target domain text prompt word corresponding to the source domain image.

[0038] Compared to existing style transfer methods that rely on image data of the target domain and can only perform global style transfer at the image level, failing to achieve object-level style transfer in object detection tasks and resulting in insufficient accuracy, this application uses a text-image pre-trained localization model to perform style transfer at the object level by defining text cues for the target domain. Through region-phrase alignment loss, it ensures that the features of the source domain are aligned with the style of the target domain at the object level, thereby achieving more accurate object-level style learning and transfer.

[0039] After constructing the target detection and localization loss function and region-phrase loss function The target detection localization loss function can be used. and region-phrase loss function Construct a total loss function and use it as the loss function for the language-image localization model. Specifically, the total loss function L is defined by the following formula: ; in, These are the hyperparameters for the corresponding terms.

[0040] Furthermore, electronic devices can use the total loss function to backpropagate the language-image localization model, thereby updating the parameters and making the model's predictions gradually approach the true labels.

[0041] In an optional implementation, by using backpropagation of the total loss function to update the style parameters of the target domain, style parameters of multiple target domains can be obtained. This allows for the construction of a set of style transfer parameters. ,from A set of style transfer parameters is randomly sampled and applied to the low-level visual features of the source domain to construct the visual features of the target domain and generate the corresponding object region.

[0042] In some embodiments, after the language-image localization model has been trained, the electronic device can use different datasets to verify the accuracy of the language-image localization model. Table 1 shows the detection results of the language-image localization model of this application on the Foggy Cityscapes dataset; Table 2 shows the detection results of the language-image localization model of this application on the BDD-100k dataset.

[0043] Table 1:

[0044] Table 2:

[0045] Reference Figure 4 The diagram shown is a functional block diagram of a single-domain generalized target detection device according to an embodiment of this application.

[0046] In some embodiments, the single-domain generalized target detection device 40 may include multiple functional modules composed of computer program segments. The computer programs for each program segment of the single-domain generalized target detection device 40 may be stored in the memory of an electronic device and executed by at least one processor to perform (see details). Figure 1(Description) This describes the function of single-domain generalized object detection. Based on its function, it can be divided into multiple functional modules. These modules may include: an acquisition module 401, a construction module 402, a first extraction module 403, a second extraction module 404, a style transfer module 405, and a detection module 406. The term "module" in this application refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0047] The acquisition module 401 is used to acquire the target image of the potential target when a potential target in an unseen target domain is detected.

[0048] The construction module 402 is used to construct a text prompt template to obtain a preliminary text description of the target image.

[0049] The first extraction module 403 is used to pre-train an image classification model to extract source domain image features of the target image source region through the image classification model.

[0050] The second extraction module 404 is used to pre-train a text classification model to extract target domain text prompts of the preliminary text description target domain through the text classification model.

[0051] The style transfer module 405 is used to obtain target domain style parameters and determine target domain visual features based on the target domain style parameters and the source domain image features.

[0052] The detection module 406 is used to input the visual features of the target domain and the text prompts of the target domain into a preset language-image localization model, so that the language-image localization model outputs the predicted detection box corresponding to the potential target and the corresponding target text prompts.

[0053] The detection module 406 is further configured to: construct an object detection and localization loss function and a region-phrase loss function; and construct the total loss function of the language-image localization model based on the object detection and localization loss function and the region-phrase loss function.

[0054] The detection module 406 is also specifically used for: The target detection and localization loss function is determined using the following formula: ; in, The target detection and localization loss function is... Indicates the detector's prediction of the first i The j-th bounding box parameter of the image. Indicates the firsti The first picture j The true target parameters of a bounding box n This represents the total number of images in the training set. m Indicates the first i The total number of bounding boxes in the images; The target detection and localization loss function is implemented using the Smooth L1 loss, which is defined by the following formula: ; in, x This represents the difference between the predicted and actual values.

[0055] The detection module 406 is also specifically used for: The region-phrase loss function is determined using the following formula: ; in, For the region-phrase loss function, , , For the pre-trained image classification model, For the pre-trained text classification model, Represents an object region, the number of which is , This indicates phrase embeddings, the number of which is... , Represents the source domain image. This indicates the target field text prompt.

[0056] The detection module 406 is also specifically used for: The total loss function is determined using the following formula: ; in, These are the hyperparameters for the corresponding terms.

[0057] The style transfer module 405 is also specifically used for: The visual features of the target domain are determined using the following formula: ; in, The visual features of the target domain and The target domain style parameters, The source domain image features, and These represent the channel mean and standard deviation of the target domain features, respectively.

[0058] It should be understood that the various variations and specific embodiments of the single-domain generalized target detection method provided in the above embodiments are also applicable to the single-domain generalized target detection device of this embodiment. Through the foregoing detailed description of the single-domain generalized target detection method, those skilled in the art can clearly understand the implementation method of the single-domain generalized target detection device in this embodiment. For the sake of brevity, it will not be described in detail here.

[0059] See Figure 5 The diagram shown is a schematic representation of the structure of an electronic device according to an embodiment of this application. In a preferred embodiment of this application, the electronic device 5 includes a memory 51, at least one processor 52, and at least one communication bus 53.

[0060] Those skilled in the art should understand that Figure 5 The structure of the electronic device shown does not constitute a limitation of the embodiments of this application. It can be a bus structure or a star structure. The electronic device 5 may also include more or fewer other hardware or software than shown, or different component arrangements.

[0061] In some embodiments, the electronic device 5 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital processors, and embedded devices. The electronic device 5 may also include user equipment, which includes, but is not limited to, any electronic product capable of human-computer interaction with a user via a keyboard, mouse, remote control, touchpad, or voice control device, such as a personal computer, tablet computer, smartphone, or digital camera.

[0062] In the embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, computer-readable storage media, and electronic devices can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple components or modules may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices, components, or modules may be electrical, mechanical, or other forms.

[0063] The components described as separate parts may or may not be physically separate. The components shown as components may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the components can be selected to achieve the purpose of this embodiment according to actual needs.

[0064] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each component can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0065] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0066] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0067] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0068] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A single-domain generalized target detection method, characterized in that, The method includes: When a potential target not found in the target domain is detected, the target image of the potential target is acquired; Construct a text prompt template to obtain a preliminary text description of the target image; A pre-trained image classification model is used to extract source domain image features from the target image source region. A pre-trained text classification model is used to extract target domain text prompts for the initial text description target domain through the text classification model; Obtain target domain style parameters, and determine target domain visual features based on the target domain style parameters and source domain image features; The visual features of the target domain and the text prompts of the target domain are input into a preset language-image localization model, so that the language-image localization model outputs the predicted detection box corresponding to the potential target and the corresponding target text prompts.

2. As described in claim 1, characterized in that, The method further includes: Construct object detection and localization loss functions and region-phrase loss functions; The total loss function of the language-image localization model is constructed based on the target detection and localization loss function and the region-phrase loss function.

3. The single-domain generalized target detection method according to claim 2, characterized in that, The constructed target detection and localization loss function includes: The target detection and localization loss function is determined using the following formula: ； in, The target detection and localization loss function is... Indicates the detector's prediction of the first i The j-th bounding box parameter of the image. Indicates the first i The first picture j The true target parameters of a bounding box n This represents the total number of images in the training set. m Indicates the first i The total number of bounding boxes in the images; The target detection and localization loss function is implemented using the Smooth L1 loss, which is defined by the following formula: ； in, x This represents the difference between the predicted and actual values.

4. The single-domain generalized target detection method according to claim 2, characterized in that, The constructed region-phrase loss function includes: The region-phrase loss function is determined using the following formula: ； in, For the region-phrase loss function, , , For the pre-trained image classification model, For the pre-trained text classification model, Represents an object region, the number of which is , This indicates phrase embeddings, the number of which is... , Represents the source domain image. This indicates the target field text prompt.

5. The single-domain generalized target detection method according to claim 2, characterized in that, The total loss function for constructing the language-image localization model based on the target detection and localization loss function and the region-phrase loss function includes: The total loss function is determined using the following formula: ； in, These are the hyperparameters for the corresponding terms.

6. The single-domain generalized target detection method according to claim 1, characterized in that, The step of determining the visual features of the target domain based on the target domain style parameters and the source domain image features includes: The visual features of the target domain are determined using the following formula: ； in, The visual features of the target domain and The target domain style parameters, The source domain image features, and These represent the channel mean and standard deviation of the target domain features, respectively.

7. The single-domain generalized target detection method according to claim 1, characterized in that, The image classification model uses the open-source SwinTransformer model pre-trained on the ImageNet dataset, while the text classification model uses the mainstream BERT architecture.

8. A single-domain generalized target detection device, characterized in that, The device includes: The acquisition module is used to acquire the target image of the potential target when a potential target in an unseen target domain is detected. A construction module is used to construct a text prompt template to obtain a preliminary text description of the target image; The first extraction module is used to pre-train an image classification model to extract source domain image features of the target image source region through the image classification model; The second extraction module is used to pre-train a text classification model to extract target domain text prompts of the preliminary text description target domain through the text classification model; The style transfer module is used to obtain target domain style parameters and determine target domain visual features based on the target domain style parameters and the source domain image features; The detection module is used to input the visual features of the target domain and the text prompts of the target domain into a preset language-image localization model, so that the language-image localization model outputs the predicted detection box corresponding to the potential target and the corresponding target text prompts.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the single-domain generalized target detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the single-domain generalized target detection method according to any one of claims 1 to 7.

Images