Infrared target intelligent detection method based on non-pre-training model field generalization
Through the methods of style conversion and domain feature extraction, the problem of increasing the calculation amount and parameter amount of pre-trained models in the prior art is solved, and the accuracy and generalization ability of infrared object detection are improved.
Patent Information
- Application Number
- CN202411342512.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2025-02-11
AI Technical Summary
When using pre-trained models to convert single domains to other target domains, the calculation amount and parameter amount are increased, and the domain generalization learning ability is insufficient, resulting in poor detection effect.
The visible light image is converted into HSV color gamut and grayscale images through the style conversion module, and the Yolov5 feature extraction network is input to obtain domain features, and the loss is calculated through the domain classification module and the domain comparison module to constrain the model to learn the domain constant features.
The number of parameters and calculations of the model is reduced, the generalization ability of the model is improved, and the accuracy of infrared object detection is improved.
Smart Images
Figure CN119418089A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an infrared target intelligent detection method based on generalization of a field without a pre-training model, and belongs to the field of infrared guided imaging target detection. Background Art
[0002] Domain generalization without pre-training models focuses on how to enable object detection algorithms to maintain high performance in different image domains. The domain here usually refers to factors such as the source, style, and lighting conditions of the image. The purpose of domain generalization without pre-training models is to create a model that can learn common features across different domains during training, so that it can also perform effective object detection in new and unseen domains. The main application areas of domain generalization object detection without pre-training models are autonomous driving, video surveillance, medical image analysis, agricultural automation, robotic vision, retail, drone applications, smartphone photography, augmented reality, etc. With the continuous development of deep learning, research on domain generalization object detection without pre-training models will continue to deepen to achieve object detection capabilities in a wider range of scenarios.
[0003] Visible light relies on light reflected from objects for imaging, while infrared images are based on thermal radiation emitted by objects. This essential difference results in completely different representations of the same object in the two images. Visible light usually contains rich color and texture information, while infrared images mainly reflect the temperature distribution of objects, and the details and texture information are not as rich as visible light. Visible light data is easy to collect and low-cost, while infrared datasets are expensive to build. A target detection model trained on visible light may not be directly applicable to infrared images and needs to be generalized to adapt to different imaging conditions.
[0004] So far, many literatures have conducted research on domain generalization target detection without pre-training models. Literature 1: Lee, Wooju, et al. "Object-Aware Domain Generalization for Object Detection." Proceedings of the AAAI Conference on ArtificialIntelligence. Vol. 38. No. 4. 2024. (Object-aware domain generalization for target detection) proposes an object-oriented domain generalization method without pre-training models and its system and method for single-domain generalization target detection. The method includes a data augmentation strategy and a training strategy, respectively called OA-Mix and OA-Loss. OA-Mix generates multi-domain data through multi-level transformation and object-aware hybrid strategy. OA-Loss enables the model to learn domain-invariant representations of objects and backgrounds from original images and images generated by OA-Mix. Reference 2: Wu,Aming,and Cheng Deng. "Single-domain generalized object detection in urban scene via cyclic-disentangled self-distillation." Proceedings of the IEEE / CVF Conference on computer vision and pattern recognition.2022. (Achieving single-domain generalized object detection in urban scenes through cyclic decoupled self-distillation) A method for enhancing the generalization ability of object detectors is proposed, which is suitable for the scenario of single-domain generalized object detection. It aims to train an object detector that can perform well in multiple unseen target domains by training with only one source domain. Specifically, a cyclic decoupled self-distillation method is proposed to decouple the domain invariant representation (DIR) from the domain-specific representation without relying on domain-related annotations (such as domain labels). In this method, a cyclic decoupling module is designed to cyclically extract the domain invariant representation (DIR) from the input visual features, and the decoupling ability is enhanced through cyclic operations without relying on domain-related annotations. At the same time, the extracted domain invariant representation (DIR) is used as a teacher, and a self-distillation module is designed to further enhance the generalization ability of the model.Reference 3: Rao, Zhijie, et al. "Srcd: Semantic reasoning with compound domains for single-domain generalized object detection." arXiv preprint arXiv:2307.01750(2023). (Domain-Conforming Semantic Reasoning for Single-Domain Generalized Object Detection) proposes a new framework for single-domain generalized object detection (Single-DGOD), called semantic reasoning with compound domains (SRCD). Different from traditional multi-source domain generalized object detection (DGOD), the challenge faced by Single-DGOD is to achieve good generalization to multiple target domains using only a single source domain data. To address this problem, this paper innovatively introduces two main components: a texture-based self-augmentation (TBSA) module and a local-global semantic reasoning (LGSR) module. The TBSA module eliminates the influence of irrelevant label-related attributes (such as lighting, shadow, color, etc.) at the image level through a lightweight and efficient self-augmentation method, thereby reducing the pseudo-attribute label association problem caused by the scarcity of single-domain data. The LGSR module models the semantic relationship of instance features, aiming to discover and maintain the intrinsic semantic structure, solve the affinity problem of instance-level semantic relationships in samples that is usually ignored in traditional methods, and thus enhance the generalization ability of the model. Reference 4: Vidit, Vidit, Martin Engilberge, and MathieuSalzmann. "Clip the gap: A single domain generalization approach for object detection." Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2023. A single domain generalization (SDG) target detection method is proposed to solve the problem of training a model on a single source domain so that it can generalize to any unseen target domain. Although this problem has been deeply studied in image classification, there is almost no research in the field of target detection. In order to simultaneously address the challenges of robust target localization and representation learning, it is proposed to use a pre-trained visual-language model to introduce the concept of semantic domain through text cues. Specifically, a semantic enhancement strategy is adopted to act on the features extracted by the detector backbone and combined with a text-based classification loss.
[0005] The defects of the above-mentioned prior art are: using a pre-trained model to transform a single domain into other target domains increases the amount of calculation and parameters for the model. At the same time, the domain generalization learning ability is insufficient, and it cannot have a good generalization effect when applied, resulting in poor detection results. Summary of the invention
[0006] The technical problem solved by the present invention is: to overcome the deficiencies of the prior art, to provide an infrared target intelligent detection method based on generalization of the field without pre-training model, and to improve the infrared target detection accuracy.
[0007] The technical solution of the present invention is: in the first aspect, a method for intelligent infrared target detection based on generalization of the field without pre-training model is provided, comprising:
[0008] The style conversion module converts the input visible light image into an HSV color gamut enhanced image, a grayscale enhanced image, and the original visible light image, and outputs the above three domain images and their corresponding domain labels;
[0009] The HSV color gamut enhanced image, grayscale enhanced image and original visible light image are input into Yolov5 feature extraction network to obtain the domain feature output corresponding to HSV color gamut, grayscale and visible light;
[0010] Various domain features output by the Yolov5 feature extraction network are input into the domain classification module and the domain comparison module, and the loss is calculated by the domain classification module and the domain comparison module, and the calculated loss is back-propagated to the Yolov5 feature extraction network through the gradient, so as to guide the Yolov5 feature extraction network to learn domain-invariant features and obtain the trained generalized Yolov5 feature extraction network;
[0011] When performing infrared target intelligent detection on infrared images, the input image is passed through the trained generalized yolov5 feature extraction network to obtain the corresponding infrared target intelligent detection results.
[0012] Preferably, the processing flow of the field classification module is specifically as follows:
[0013] The visible light image domain features, HSV image domain features and grayscale image domain features output by the Yolov5 feature extraction network are classified through the domain classifier, the domain classification results are compared with the true value labels of each domain, the classification loss is calculated and back-propagated to the Yolov5 feature extraction network.
[0014] Preferably, the workflow of the domain classifier is as follows:
[0015] Each domain feature is sequentially subjected to two layers of convolution and ReLU activation function, and then further expanded into a one-dimensional vector after adaptive average downsampling, which is input into the linear layer, and then the domain classification result is obtained through softmax.
[0016] Preferably, the field comparison module is specifically:
[0017] Contrastive learning is adopted to calculate the contrast loss from the spatial, channel and variance dimensions of the input features in each field, and then back-propagate the gradient to the Yolov5 feature extraction network.
[0018] Preferably, the spatial contrast loss calculation method is: after calculating the spatial dimension variance of the features in each field, the spatial contrast loss value is obtained by global maximum pooling calculation.
[0019] Preferably, the channel contrast loss calculation method is: after calculating the channel dimension variance of each field feature, the channel contrast loss value is obtained by global maximum pooling calculation.
[0020] Preferably, the variance contrast loss calculation method is: first perform L2 regularization on the features of each field, then perform matrix multiplication of the L2 regularization result and its own transposition, and finally perform global maximum pooling calculation to obtain the variance contrast loss value.
[0021] In a second aspect, a terminal device is provided, including:
[0022] a memory for storing instructions executed by at least one processor;
[0023] The processor is used to execute instructions stored in the memory to implement the infrared target intelligent detection method based on field generalization without pre-training model as described above.
[0024] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes the infrared target intelligent detection method based on generalization of the field without a pre-trained model as described above.
[0025] Compared with the prior art, the present invention has the following advantages:
[0026] (1) The present invention does not need to use a pre-trained model to convert the source domain to other domains, which greatly reduces the number of model parameters and the amount of calculation.
[0027] (2) The present invention combines domain classification with domain comparison, enabling the model to learn and extract domain-invariant features more quickly, thereby improving the generalization ability of the model and the detection effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1It is an overall flow chart of an infrared target intelligent detection method based on field generalization without pre-training model according to an embodiment of the present invention;
[0029] Figure 2 A structural diagram of a style conversion module of an infrared target intelligent detection method based on generalization of a domain without a pre-trained model according to an embodiment of the present invention;
[0030] Figure 3 A structural diagram of a domain classification module of an infrared target intelligent detection method based on domain generalization without a pre-trained model according to an embodiment of the present invention;
[0031] Figure 4 A domain classification one-hot encoding conversion diagram of an infrared target intelligent detection method based on domain generalization without a pre-training model according to an embodiment of the present invention;
[0032] Figure 5 This is a structural diagram of a domain comparison module of an infrared target intelligent detection method based on domain generalization without a pre-trained model according to an embodiment of the present invention.
[0033] Figure 6 This is a flowchart of domain contrast loss calculation for an infrared target intelligent detection method based on domain generalization without a pre-trained model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0034] In view of the insufficient performance of cross-modal infrared target detection in the prior art, the huge cost of building infrared data sets, and the problems of missed detection and false detection in intelligent visible light detection methods, the present invention provides an infrared target intelligent detection method based on generalization of the field without pre-training models, including:
[0035] The input visible light image is converted into an HSV color domain enhanced image, a grayscale enhanced image and the original visible light image through the style conversion module, and the above three domain images and their corresponding domain labels are output;
[0036] The HSV color gamut enhanced image, grayscale enhanced image and original visible light image are input into Yolov5 feature extraction network to obtain the domain feature output corresponding to HSV color gamut, grayscale and visible light;
[0037] Various domain features are input into the domain classification module and the domain comparison module together, and the loss function is calculated through the domain classification module and the domain comparison module. The gradient is back-propagated to the Yolov5 feature extraction network to constrain the model to learn domain-invariant features and obtain the trained generalized Yolov5 feature extraction network.
[0038] When performing intelligent infrared target detection on infrared images, the corresponding feature decoding output can be obtained by simply passing the input image through the trained generalized Yolov5 feature extraction network.
[0039] The style transfer module includes: converting the input image from the visible light domain to the HSV domain, and from the visible light domain to the grayscale domain without additional pre-training weights, and finally outputting a visible light image, an HSV image, a grayscale image and corresponding domain labels.
[0040] The Yolov5 feature extraction network includes: using the Yolov5 feature extraction network to extract the corresponding features of the visible light image, HSV image, and grayscale image output by the style conversion module and outputting them to the subsequent module. The subsequent module guides the Yolov5 feature extraction network to extract domain invariant features through back propagation optimization.
[0041] The domain classification module includes: two layers of convolution and ReLU activation function; the Yolov5 feature extraction network outputs the domain features of the visible light image, HSV image, and grayscale image, which are sequentially subjected to two layers of convolution and ReLU activation function, and then further expanded into a one-dimensional vector after adaptive average downsampling, which is input into the linear layer, and then the domain classification result is obtained through softmax, the domain classification result is compared with the true value label of each domain, and the classification loss output is calculated.
[0042] The domain contrast module includes: using contrast learning, by calculating contrast loss from the spatial, channel and variance dimensions of the input visible light, HSV and grayscale domain features, driving the Yolov5 feature extraction network to learn domain invariant features. The spatial contrast loss calculation includes: calculating the spatial dimension variance and then obtaining the spatial contrast loss value through global maximum pooling. The channel contrast loss calculation includes: calculating the channel dimension variance and then obtaining the channel contrast loss value through global maximum pooling. The variance contrast loss calculation includes: first performing L2 regularization on the features of each domain, then performing matrix multiplication on the features and their own transposition, and finally performing global maximum pooling calculation to obtain the variance contrast loss value output.
[0043] The Yolov5 feature extraction network extracts the features corresponding to the visible light image, HSV image, and grayscale image output by the style transfer module and outputs them to the domain classification module and the domain comparison module to calculate the loss. The loss is optimized by back-propagation gradient direction to guide the Yolov5 feature extraction network to extract domain-invariant features.
[0044] When an infrared image is input for infrared target intelligent detection: only the parameters of the backbone Yolov5 feature extraction network are extracted from the trained model, the style transfer module, the domain classification module, and the domain comparison module are not used, the infrared image is input into the trained generalized Yolov5 network, and the infrared target intelligent detection result is output.
[0045] The present invention proposes an infrared target intelligent detection method based on domain generalization without pre-training model. The input visible light three-channel image is converted into other domain image output through the style conversion module, and these domain images are input into the Yolov5 feature extraction network to obtain the corresponding domain features, and the domain features are input into the domain classification module and the domain comparison module. The domain classification module and the domain comparison module constrain the model to learn the domain invariant features and output the corresponding loss. The loss optimizes the weight of the backbone network through back propagation. When testing the infrared image, the corresponding output can be obtained only through the Yolov5 network. The target detection accuracy of the infrared target intelligent detection method based on domain generalization without pre-training model is generally better than that of traditional and other intelligent processing methods.
[0046] A second aspect of the present invention provides a terminal device, comprising:
[0047] a memory for storing instructions executed by at least one processor;
[0048] The processor is used to execute instructions stored in the memory to implement the infrared target intelligent detection method based on field generalization without pre-training model as described above.
[0049] The present invention also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the infrared target intelligent detection method based on generalization of the field without pre-training model as described above.
[0050] Example:
[0051] This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0052] The overall structure of the infrared target intelligent detection method based on field generalization without pre-training model is as follows Figure 1 As shown in the figure, the image is converted into other domain image outputs through the style conversion module, and these domain images are input into the Yolov5 feature extraction network to obtain the corresponding domain feature outputs, and the domain features are input into the domain classification module and the domain comparison module. The domain classification module and the domain comparison module constrain the model to learn domain invariant features and output losses, and the losses are optimized through back propagation to optimize the weights of the backbone extraction network. When testing infrared images, the corresponding output can be obtained by simply passing through the Yolov5 network.
[0053] Style transfer modules without pre-trained weights such as Figure 2As shown in the figure, the input image is converted from the visible light RGB field to the HSV field, and from the visible light RGB field to the grayscale field, and finally the visible light image and visible light field label, HSV image and HSV field label, grayscale image and grayscale field label are output.
[0054] Domain classification modules such as Figure 3 As shown in the figure, the features of visible light, HSV, and grayscale fields output by the Yolov5 feature extraction network are converted into one-dimensional vectors in the form of one-hot encoding by the field classifier, and then compared with the category labels of each field output by the image style conversion module to calculate the classification loss.
[0055] The domain classifier is expanded into a one-dimensional vector such as Figure 4 As shown in the figure, the network model outputs the features of each field through two layers of convolution and ReLU activation function, and is expanded into a one-dimensional vector after adaptive average downsampling, input into the linear layer, and then the field classification result is obtained through softmax.
[0056] Field comparison modules such as Figure 5 , Figure 6 As shown in the figure, contrastive learning is adopted to drive the model to learn domain-invariant features by performing contrastive loss calculations on the input visible light, HSV, and grayscale features from the spatial, channel, and variance dimensions. The spatial contrast loss calculation includes: calculating the spatial dimension variance and then performing global maximum pooling to obtain the spatial contrast loss value. The channel contrast loss calculation includes: calculating the channel dimension variance and then performing global maximum pooling to obtain the channel contrast loss value. The variance contrast loss calculation includes: first performing L2 regularization on the features of each domain, then performing matrix multiplication on the features by transposing them, and finally performing global maximum pooling to obtain the variance contrast loss value.
[0057] The detection performance of the existing technology for infrared targets needs to be improved, and infrared target detection data sets are difficult to obtain. The existing visible light direct training algorithm is insufficient in performance when applied to infrared target detection, and missed detection and false detection are serious, and it cannot be used in practical applications. In order to solve the above technical problems, the present invention provides an infrared target intelligent detection method based on domain generalization without pre-training model, including: converting visible light images into other domain images through a style conversion module, inputting these domain images into the Yolov5 feature extraction network to obtain corresponding domain features, and inputting the domain features into the domain classification module and the domain comparison module. The domain classification module and the domain comparison module constrain the model to learn domain invariant features. When testing infrared images, the corresponding output can be obtained by simply passing through the Yolov5 network.
[0058] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.
[0059] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in this field.
Claims
1. An infrared target intelligent detection method based on domain generalization without pre-training model, characterized in that include: The style conversion module converts the input visible light image into an HSV color gamut enhanced image, a grayscale enhanced image, and the original visible light image, and outputs the above three domain images and their corresponding domain labels; The HSV color gamut enhanced image, grayscale enhanced image and original visible light image are input into Yolov5 feature extraction network to obtain the domain feature output corresponding to HSV color gamut, grayscale and visible light; Various domain features output by the Yolov5 feature extraction network are input into the domain classification module and the domain comparison module, and the loss is calculated by the domain classification module and the domain comparison module, and the calculated loss is back-propagated to the Yolov5 feature extraction network through the gradient, so as to guide the Yolov5 feature extraction network to learn domain-invariant features and obtain the trained generalized Yolov5 feature extraction network; When performing infrared target intelligent detection on infrared images, the input image is passed through the trained generalized yolov5 feature extraction network to obtain the corresponding infrared target intelligent detection results.
2. The infrared target intelligent detection method based on domain generalization without pre-training model according to claim 1 is characterized by: The processing flow of the domain classification module is as follows: The visible light image domain features, HSV image domain features and grayscale image domain features output by the Yolov5 feature extraction network are classified through the domain classifier, the domain classification results are compared with the true value labels of each domain, the classification loss is calculated and back-propagated to the Yolov5 feature extraction network.
3. The infrared target intelligent detection method based on domain generalization without pre-training model according to claim 2 is characterized by: The workflow of the domain classifier is as follows: Each domain feature is sequentially subjected to two layers of convolution and ReLU activation function, and then further expanded into a one-dimensional vector after adaptive average downsampling, which is input into the linear layer, and then the domain classification result is obtained through softmax.
4. The infrared target intelligent detection method based on domain generalization without pre-training model according to claim 1 is characterized by: The specific field comparison modules are: Contrastive learning is adopted to calculate the contrast loss from the spatial, channel and variance dimensions of the input features in each field, and then back-propagate the gradient to the Yolov5 feature extraction network.
5. The infrared target intelligent detection method based on domain generalization without pre-training model according to claim 4 is characterized in that: The spatial contrast loss calculation method is as follows: after calculating the spatial dimension variance of the features in each field, the spatial contrast loss value is obtained through global maximum pooling calculation.
6. The infrared target intelligent detection method based on domain generalization without pre-training model according to claim 4 is characterized in that: The channel contrast loss calculation method is as follows: after calculating the channel dimension variance of the features in each field, the channel contrast loss value is obtained through global maximum pooling.
7. The infrared target intelligent detection method based on domain generalization without pre-training model according to claim 4 is characterized by: The variance contrast loss calculation method is as follows: first perform L2 regularization on the features of each field, then perform matrix multiplication of the L2 regularization result and its own transpose, and finally perform global maximum pooling calculation to obtain the variance contrast loss value.
8. A terminal device, characterized in that: include: a memory for storing instructions executed by at least one processor; A processor, configured to execute instructions stored in a memory to implement a method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.