Cross-domain infrared target intelligent detection method based on lightweight

Abstract

The invention provides a lightweight-based cross-domain infrared target intelligent detection method, which comprises the following steps of: firstly, performing multi-channel data enhancement on an input image, extracting local feature information of different levels of the enhanced image by using a lightweight feature extraction network, and obtaining features of different scales through multi-scale maximum pooling; and a global self-attention module is added to extract global features, a multi-level feature fusion module is used to fuse feature information of different levels output by the lightweight feature extraction module and the global self-attention module, and finally the feature information is decoded to obtain a target prediction result. According to the method, the problems that the existing cross-domain infrared target detection performance is insufficient, and infrared target missing detection and false detection exist in an intelligent detection method can be solved.

Description

Technical Field

[0001] The invention relates to a lightweight cross-domain infrared target intelligent detection method, belonging to the field of target detection. Background Art

[0002] The purpose of cross-domain infrared target detection is to identify the location and category of targets in infrared images or videos. The main application areas include security monitoring, autonomous driving, environmental monitoring, forest fire prevention, etc., and remarkable achievements have been made. In these fields, infrared target detection is an inevitable requirement for practical applications. Although target detection methods have developed rapidly with the rise of deep learning, infrared images cannot be directly used for training due to the difficulty and high cost of collecting them. Visible light images are easy to collect and easy to build large data sets, but models trained with visible light are difficult to achieve high detection rates for cross-domain infrared image target detection.

[0003] Infrared cross-domain target detection uses visible light images for training and infrared images for testing. Due to the different imaging mechanisms of visible light and infrared images, there are modal differences between the two, which brings difficulties to detection. Most methods can only focus on learning domain-invariant features such as the shape and scale of the target, and selectively ignore or forget domain-variable features such as target texture features. The imaging mechanisms of infrared and visible light are different. The wavelength range of visible light is approximately 400 nanometers to 700 nanometers. Visible light images usually consist of three bands: red, green, and blue, and each channel records the light intensity of the color band. The three channels are combined to form a full-color image, also known as an RGB image, which has the characteristics of high clarity, rich color and texture information, and image quality is greatly affected by lighting conditions. The wavelength range of infrared light is longer than that of visible light, usually from 700 nanometers to 1 millimeter. Infrared light can be further divided into near-infrared, short-wave infrared, medium-wave infrared, and long-wave infrared according to different wavelengths. Infrared images are generally single-channel images that only record the light intensity of one band and generate grayscale images. Infrared images reflect the thermal radiation intensity or reflection characteristics of objects. Temperature or reflectivity is usually expressed in terms of brightness, which mainly reflects the thermal radiation characteristics of objects. In infrared images, the color information of objects is missing, mainly brightness information, which also increases the difficulty of cross-domain infrared target detection. So far, many papers have conducted research on cross-domain infrared target detection. The purpose is to reduce the inherent distribution mismatch between the source domain and the target domain, thereby enhancing the applicability and accuracy of the model in the new environment. Reference 2: CHEN Y, LI W, et al. Domain adaptive faster r-CNN for object detection in the wild [C] / / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 3339-3348. (Domain adaptive Faster-RCNN algorithm for object detection in natural environments) Firstly, unsupervised domain adaptation technology is introduced in the object detection task. The proposed DA Faster R-CNN model effectively narrows the gap between the source domain and the target domain through adversarial learning strategies at the image level and the object level.Reference 3: SAITO K, USHIKU Y, et al. Strong-weak distribution alignment for adaptive object detection [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019: 6956-6965. (Strong-weak distribution alignment for adaptive object detection) In the process of performing domain adaptation, differentiated processing strategies are adopted for different levels of features: local features are accurately matched by strengthening alignment, while global features learn the commonalities between domains by weakening alignment, thereby optimizing the extraction of domain-invariant features. Reference 4: LIU X, LI W, et al. Towards robust adaptive object detection under noisy annotations [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. New Orleans, LA, USA: IEEE, 2022: 14207-14216. (Towards robust adaptive object detection under noisy annotations) Aware that datasets containing noisy labels may cause differences in source domain distribution, the NLTE model was developed to address and alleviate the problems caused by noisy labels.

[0004] The defects / shortcomings of the above-mentioned prior art are: a large amount of source domain and target domain data is required, and the target domain data needs to be additionally processed for conversion, which increases the computational complexity of the model. The additional modules also greatly increase the number of model parameters. At the same time, there are also problems such as insufficient performance of existing cross-domain infrared target detection and missed and false detection of infrared targets in intelligent detection methods. Summary of the invention

[0005] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art, to provide a lightweight cross-domain infrared target intelligent detection method, to fit the infrared image by converting visible light into a single-channel image through data enhancement, to improve the existing cross-domain infrared target detection performance is insufficient, intelligent detection methods have the problem of infrared target missed detection and false detection.

[0006] The technical solution of the present invention is: in the first aspect, a lightweight cross-domain infrared target intelligent detection method is provided, comprising:

[0007] Perform multi-channel data enhancement on the input image, and output any one channel image of the enhanced three-channel image, or the original visible light image;

[0008] Extract local feature information at different levels of the input enhanced image and output the extracted local features;

[0009] Perform multi-scale pooling and downsampling processing on local features to obtain features of different scales and splice them, and output multi-scale splicing features after multi-scale splicing;

[0010] Extract global features from multi-scale splicing features and output global features;

[0011] Fuse local features and global features and output fused multi-scale features;

[0012] The fused multi-scale features are decoded to obtain the target prediction result.

[0013] Preferably, the local feature information of different levels of the input enhanced image is extracted through a lightweight feature extraction network, and the extracted local features are output; the lightweight feature extraction network includes: ordinary convolution, phantom convolution and phantom module, specifically:

[0014] After the input image undergoes a normal convolution, it is repeatedly stacked with a phantom convolution and a phantom module five times. The phantom module in the second stack outputs feature F3, the phantom module in the third stack outputs feature F4, the phantom module in the fourth stack outputs feature F5, and the output features of the phantom module in the fifth stack are used as the output of the lightweight feature extraction network.

[0015] Preferably, each phantom convolution includes: a normal convolution and three linear transformations:

[0016] After the input undergoes ordinary convolution, the ordinary convolution result is retained. At the same time, the convolution result is passed through three linear transformation branches respectively. Finally, the ordinary convolution result and the results of each linear transformation are spliced ​​and summarized into the phantom convolution output.

[0017] Preferably, each phantom module includes ordinary convolution and phantom bottleneck: first, the output of the phantom convolution is accepted as input for an ordinary convolution, and two branches are output: one branch passes through three layers of phantom bottleneck, and the other branch passes through one layer of ordinary convolution; the outputs of the two branches are spliced, and the spliced ​​result is output after passing through ordinary convolution.

[0018] Preferably, the phantom bottleneck in the phantom module is a stacked two-layer phantom convolution, where:

[0019] The number of feature channels of the first phantom convolution output is twice the number of channels of the input feature map. The increase in the number of feature channels is used for expansion. The number of feature channels of the second phantom convolution output is 1 / 2 of the number of channels of the input feature map. The output of the second phantom convolution and the input residual before the first layer of phantom convolution are used as the output of the phantom bottleneck.

[0020] Preferably, a multi-scale pooling down-sampling process is performed on the local features through a multi-scale maximum pooling module to obtain features of different scales and splice them, and a multi-scale splicing feature after multi-scale splicing is output; the multi-scale maximum pooling module includes: ordinary convolution, batch normalization, SiLU activation function, and maximum pooling layer; specifically:

[0021] The feature map output by the lightweight feature extraction network is first subjected to 1*1 ordinary convolution and then batch normalization. The normalized result passes through the SiLU activation function and then passes through three stacked maximum pooling layers. The outputs of each maximum pooling layer and the output of the aforementioned SiLU activation function are spliced. The spliced ​​result passes through 1*1 convolution, and then once again passes through batch normalization and SiLU activation function to output a multi-scale spliced ​​feature map.

[0022] Preferably, a global feature is extracted from the multi-scale splicing feature through a global self-attention module, and the global feature is output; the processing flow of the global self-attention module is specifically as follows:

[0023] A linear mapping calculation is performed on the input multi-scale spliced ​​feature map to obtain the representation of the query Q, key K and value V, where the query Q represents the Query vector, the key K represents the key-value vector, and the value V represents the value vector; then, these representations are input into multiple attention heads to independently calculate the global attention and then spliced, and the splicing result is output through linear transformation, which is recorded as feature F6.

[0024] Preferably, the local features and the global features are fused through a hierarchical feature fusion module and the fused multi-scale features are output; the processing flow of the multi-level feature fusion module is specifically as follows:

[0025] Features F3, F4, F5 and F6 are used. Feature F6 is concatenated with feature F5 after upsampling, and feature M5 is obtained through the phantom module. The M5 feature is concatenated with the F4 feature after one upsampling, and the M4 feature is obtained through the phantom module. The M4 feature is concatenated with the F3 feature after one upsampling, and feature P3 is obtained through the phantom module. The P3 feature is concatenated with the M4 feature after phantom convolution, and feature P4 is obtained through the phantom module. Feature P4 is concatenated with the M5 feature after phantom convolution, and feature P5 is obtained through the phantom module. Feature P5 is concatenated with feature F6 after phantom convolution, and feature P6 is obtained through the phantom module. Finally, the fused multi-scale features P3, P4, P5 and P6 are output.

[0026] Preferably, a feature decoding module is used to decode the fused multi-scale features to obtain a target prediction result; the module includes: a decoupling head of a two-layer ordinary convolution combination, and the decoupling head of the two-layer ordinary convolution combination performs decoupling calculations on the output fused multi-scale features of the multi-level feature information fusion module to obtain a target prediction result.

[0027] In the second aspect, a lightweight cross-domain infrared target intelligent detection system is provided, comprising:

[0028] Data enhancement module: performs multi-channel data enhancement on the input visible light image, and outputs any one channel image of the enhanced three-channel image, or the original visible light image;

[0029] Lightweight feature extraction network module: extracts local feature information of different levels from the input enhanced image through a lightweight feature extraction network, and outputs the extracted local features;

[0030] Multi-scale maximum pooling module: Use multi-scale maximum pooling technology to perform multi-scale pooling downsampling processing on the local feature map, obtain features of different scales and splice them, and output multi-scale splicing features after multi-scale splicing;

[0031] Global attention module: extracts global features from multi-scale spliced ​​feature maps and outputs global features;

[0032] Multi-level feature information fusion module: The multi-level feature information fusion module is used to fuse local features and global features and output fused multi-scale features;

[0033] Feature decoding module: decodes the fused multi-scale features to obtain the target prediction result.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] (1) The present invention reduces the need for a large amount of target domain data and reduces the training time;

[0036] (2) The present invention combines phantom convolution and phantom module, which greatly improves the detection speed and reduces the number of parameters;

[0037] (3) The present invention fits the infrared image by converting visible light into a single-channel image through data enhancement, thereby overcoming the defects of target omission and false detection when training infrared target intelligent detection with visible light, so that the proposed multi-channel data enhancement module can bring a significant improvement in the infrared target detection task without adding training cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the method of the present invention;

[0039] Figure 2 This is a structural diagram of a multi-channel data enhancement module based on a lightweight cross-domain infrared target intelligent detection method according to an embodiment of the present invention;

[0040] Figure 3 A structural diagram of a lightweight feature extraction network module based on a lightweight cross-domain infrared target intelligent detection method according to an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of a phantom convolution structure of a lightweight cross-domain infrared target intelligent detection method according to an embodiment of the present invention;

[0042] Figure 5 This is a structure diagram of a phantom module based on a lightweight cross-domain infrared target intelligent detection method according to an embodiment of the present invention;

[0043] Figure 6 This is a structural diagram of a multi-scale maximum pooling module based on a lightweight cross-domain infrared target intelligent detection method according to an embodiment of the present invention.

[0044] Figure 7 This is a structural diagram of a multi-head self-attention module based on a lightweight cross-domain infrared target intelligent detection method according to an embodiment of the present invention;

[0045] Figure 8 This is a structural diagram of a multi-level feature fusion module based on a lightweight cross-domain infrared target intelligent detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The prior art has insufficient cross-domain infrared target detection performance, and intelligent detection methods have problems of missed detection and false detection of infrared targets. The present invention provides a lightweight cross-domain infrared target intelligent detection method, such as Figure 1 As shown, the input data is enhanced by a multi-channel data enhancement module, multi-level local information is extracted in a lightweight feature extraction module, features of different scales are obtained using multi-scale maximum pooling, global features are extracted by a global self-attention module, and feature information of different levels is fused and output using a multi-level feature information fusion module. Finally, the output of the neural network is decoded to obtain the target prediction result; compared with the traditional and other intelligent methods, the present invention improves the target detection accuracy of the cross-domain infrared target intelligent detection method. Specifically including:

[0047] Input a visible light image, perform multi-channel data enhancement on the input image through a data enhancement module, and output any one channel image of the enhanced three-channel image or the original visible light image;

[0048] Extract local feature information of different levels from the input enhanced image through a lightweight feature extraction network, and output the extracted local feature map;

[0049] The feature map output by the lightweight feature extraction network is subjected to multi-scale pooling downsampling through the multi-scale maximum pooling technology to obtain features of different scales and splice them, and then a multi-scale spliced ​​feature map is output after multi-scale splicing;

[0050] The global attention module extracts global features from the multi-scale spliced ​​feature map output by the multi-scale maximum pooling, and outputs the global feature map;

[0051] By using a multi-level feature information fusion module, the local feature map output by the lightweight feature extraction network and the global feature map output by the global attention module are fused and the fused multi-scale features are output;

[0052] The fused multi-scale features output by the multi-level feature information fusion module are decoded to obtain the target prediction result.

[0053] The multi-channel data enhancement includes: inputting a three-channel visible light image, using the information of the three channels (R, G, B) in the visible light image H*W*3, respectively extracting and copying the three channels (R, G, B) of the visible light image and converting them into a red grayscale image H*W*(R, R, R), a blue grayscale image H*W*(B, B, B) and a green grayscale image H*W*(G, G, G). Outputting any one channel image of the enhanced three-channel image or the original visible light image to the network. H represents the image height, and W represents the image width in pixels.

[0054] The lightweight feature extraction network uses stacked phantom convolution and phantom module to extract multi-level local feature information of the input image and output the feature map of the corresponding channel respectively. It includes: ordinary convolution, phantom convolution and phantom module: specifically, after the input image undergoes an ordinary convolution, it is stacked with a phantom convolution and a phantom module for 5 times in sequence. The phantom module in the second stacking outputs feature F3, the phantom module in the third stacking outputs feature F4, the phantom module in the fourth stacking outputs feature F5, and the phantom module in the fifth stacking outputs feature input to the multi-scale maximum pooling module; when the image size is 640*640, the F3, F4, and F5 features are 80*80*256, 40*40*512, and 20*20*768 scale features respectively.

[0055] Each phantom convolution includes: a normal convolution and three linear transformations; the input retains the normal convolution result after the normal convolution, and the convolution result also passes through three linear transformation branches respectively. Finally, the normal convolution result and the results of each linear transformation are concatenated and summarized into the phantom convolution output. The linear transformation operation uses a 3*3 or 5*5 linear convolution kernel.

[0056] Each phantom module includes ordinary convolution and phantom bottleneck: first, the output of phantom convolution is accepted as input for an ordinary convolution, and two branches are output, one branch passes through three layers of phantom bottleneck, and the other branch passes through one layer of ordinary convolution. The outputs of the two branches are concatenated, and the concatenated result is output after ordinary convolution.

[0057] The phantom bottleneck includes: stacking two layers of phantom convolution, the number of feature channels of the first phantom convolution output is twice the number of input feature map channels, the increase in the number of feature channels is used for expansion, the number of feature map channels of the second phantom convolution output is 1 / 2 of the number of input feature map channels, and the output of the second phantom convolution and the input residual before the first layer of phantom convolution are used as the output of the phantom bottleneck.

[0058] The multi-scale maximum pooling module includes: ordinary convolution, batch normalization, SiLU activation function, and maximum pooling layer; the feature map output by the lightweight feature extraction network is first subjected to 1*1 ordinary convolution and then batch normalization. After the normalization result passes through the SiLU activation function, it passes through three stacked maximum pooling layers in sequence, and the pooling kernel size is 5*5. The outputs of each layer of the maximum pooling layer and the output of the aforementioned SiLU activation function are spliced. The splicing result passes through 1*1 convolution, and once again passes through batch normalization and SiLU activation function to output a multi-scale spliced ​​feature map.

[0059] The global self-attention module includes: performing linear mapping calculation on the input multi-scale spliced ​​feature map to obtain the representation of query Q, key K and value V. Among them, query Q represents the query vector, key K represents the key-value vector, and value V represents the value vector. Then, these representations are input into multiple attention heads to independently calculate the global attention and then spliced, and the spliced ​​results are output through linear transformation, that is, the image features are input into a linear layer with query, key, and value distribution to obtain the corresponding output, and the scaled dot product attention calculation is performed to obtain the attention features, and the attention features are spliced ​​and passed through the linear layer to output the global features.

[0060] The multi-level feature fusion module includes: features F3, F4, F5 extracted by a lightweight feature extraction network and feature F6 output by global self-attention. Feature F6 is spliced ​​with feature F5 after upsampling, and feature M5 is obtained through a phantom module. When the image size is 640*640, feature M5 is 20*20*768. M5 feature is spliced ​​with F4 feature after one upsampling, and M4 feature is obtained through a phantom module. M4 feature is 40*40*512. M4 feature is spliced ​​with F4 feature after one upsampling. After sampling, it is concatenated with feature F3, and feature P3 is obtained through the phantom module. Feature P3 is 80*80*256. After the phantom convolution, the P3 feature is concatenated with the M4 feature, and feature P4 is obtained through the phantom module. Feature P4 is 40*40*512. Feature P4 is concatenated with the M5 feature after phantom convolution, and feature P5 is obtained through the phantom module. Feature P5 is 20*20*768. Feature P5 is concatenated with feature F6 after phantom convolution, and feature P6 is obtained through the phantom module. The feature P6 is 10*10*1024. The feature scale size of P3 is 80*80*256, the feature scale size of P4 is 40*40*512, the feature scale size of P5 is 20*20*768, and the feature scale size of P6 is 10*10*1024. The final output is the fusion of multi-scale features P3, P4, P5, and P6.

[0061] The feature decoding module decodes the output of the multi-level feature information fusion module to obtain the target prediction result. The module includes: a decoupling head composed of two layers of ordinary convolutions. The decoupling head composed of two layers of ordinary convolutions decouples the output of the multi-level feature information fusion module and fuses the multi-scale features to obtain the target prediction result.

[0062] The present invention proposes a lightweight cross-domain infrared target intelligent detection method, which enhances input data through a multi-channel data enhancement module, extracts multi-level local information in a lightweight feature extraction module, uses multi-scale maximum pooling to obtain features of different scales, extracts global features through a global self-attention module, uses a multi-level feature information fusion module to fuse feature information of different levels for output, and finally decodes the output of the neural network to obtain the target prediction result. The target detection accuracy of the lightweight cross-domain infrared target intelligent detection method is generally better than that of traditional and other intelligent methods.

[0063] In a second aspect, the present invention provides a lightweight cross-domain infrared target intelligent detection system, comprising:

[0064] Data enhancement module: performs multi-channel data enhancement on the input visible light image, and outputs any one channel image of the enhanced three-channel image, or the original visible light image;

[0065] Lightweight feature extraction network module: extracts local feature information of different levels from the input enhanced image through a lightweight feature extraction network, and outputs the extracted local features;

[0066] Multi-scale maximum pooling module: Use multi-scale maximum pooling technology to perform multi-scale pooling downsampling processing on the local feature map, obtain features of different scales and splice them, and output multi-scale splicing features after multi-scale splicing;

[0067] Global attention module: extracts global features from multi-scale spliced ​​feature maps and outputs global features;

[0068] Multi-level feature information fusion module: The multi-level feature information fusion module is used to fuse local features and global features and output fused multi-scale features;

[0069] Feature decoding module: decodes the fused multi-scale features to obtain the target prediction result.

[0070] This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0071] First, it includes: using a multi-channel data enhancement module to enhance the data of the visible light image of H*W*3. The multi-channel data enhancement is as follows: Figure 2 As shown, it includes converting the R channel into an R channel image, the G channel into a G channel image, and the B channel into a B channel image. After the multi-channel data enhancement module, the original image will be saved as the original image, the R channel image, the G channel image, and the B channel image with equal probability. The dimensions of each image are the same.

[0072] For the image enhanced by multi-channel data, a lightweight feature extraction network is used to extract features from the H*W*3 image, where the lightweight feature extraction network is as follows: Figure 2 As shown, it includes phantom convolution and phantom module. Phantom convolution is represented by GhostConv, phantom module is represented by C3Ghost, and welcome module includes phantom bottleneck, which is composed of two layers of phantom convolution, represented by GhostBottleNeck. First, a normal convolution layer is used, denoted as Conv, to extract the input image features as feature maps. Then, 5 layers of phantom convolution and phantom module are passed through continuously.

[0073] Phantom convolution Figure 4 As shown in the figure, firstly, conventional convolution is used to obtain the intrinsic feature map, then the feature map of each channel is used to generate the phantom feature map by linear transformation operation, and finally the intrinsic feature map and the phantom feature map are concatenated to obtain the final result.

[0074] Phantom modules such as Figure 5As shown in the figure, the feature map output by the previous module is used as its own input. First, the feature map is divided into two parts. The left half only passes through one conv module and waits for the right half to be processed and fused. The right half first passes through the same conv module, then passes through n Ghostbottlenecks, and finally merges with the left half. After fusion, another conv module is performed to output the feature map.

[0075] Multi-scale maximum pooling module such as Figure 6 As shown in the figure, it first undergoes 1x1 convolution, normalization, and SiLU activation function activation, then passes through the maximum pooling layer with a pooling kernel size of 5 in series + parallel, and then concatenates the outputs, and finally undergoes 1x1 convolution, normalization, and SiLU activation function activation to obtain the output.

[0076] The global self-attention module is as follows Figure 7 As shown in the figure, the input multi-scale spliced ​​feature map is linearly mapped to obtain the representation of query Q, key K and value V. Among them, query Q represents the query vector, key K represents the key-value vector, and value V represents the value vector. Then, these representations are input into multiple attention heads to independently calculate the global attention and then spliced ​​and output through linear transformation. That is, the image features are input into a linear layer with query, key, and value distribution to obtain the corresponding output, and the scaled dot product attention calculation is performed to obtain the attention features. After the attention features are spliced, they are passed through the linear layer to output the global features.

[0077] The multi-level feature fusion module is as follows: Figure 8 As shown in the figure, the F3, F4, and F5 adjacent features extracted by the lightweight feature extraction network and the F6 feature output by the global self-attention are used. The F6 feature is upsampled and concatenated with the F5 feature, and the M5 feature is obtained through the phantom module. The M5 feature is upsampled once and concatenated with the F4 feature, and the M4 feature is obtained through the phantom module. The M4 feature is upsampled once and concatenated with the F3 feature, and the P3 feature is obtained through the phantom module. The P3 feature is concatenated with the M4 feature after phantom convolution, and the P4 feature is obtained through the phantom module. The P4 feature is concatenated with the M5 feature after phantom convolution, and the P5 feature is obtained through the phantom module. The P5 feature is concatenated with the F6 feature after phantom convolution, and the P6 feature map is obtained through the phantom module. The scale size of the P3 feature map is 80*80, the scale size of the P4 feature map is 40*40, the scale size of the P5 feature map is 20*20, and the scale size of the P6 feature map is 10*10.

[0078] The performance of the existing technology for cross-domain infrared target detection needs to be improved, and the intelligent detection method has the problems of high infrared catalog missed detection rate and high false detection rate. In order to solve the above technical problems, the present invention proposes a lightweight cross-domain infrared target intelligent detection method. First, the input data is converted by a multi-channel data enhancement method, and then the feature information of the input image is extracted by the designed lightweight backbone network. The global attention mechanism is added to the network to extract interactive global features. Finally, a multi-level feature information fusion module is used to fuse feature information at different levels as output, and finally the output of the neural network is decoded to obtain the target prediction result.

[0079] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

[0080] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in this field.

Claims

1. A lightweight cross-domain infrared target intelligent detection method, characterized in that include: Perform multi-channel data enhancement on the input image, and output any one channel image of the enhanced three-channel image, or the original visible light image; Extract local feature information at different levels of the input enhanced image and output the extracted local features; Perform multi-scale pooling and downsampling processing on local features to obtain features of different scales and splice them, and output multi-scale splicing features after multi-scale splicing; Extract global features from multi-scale splicing features and output global features; Fuse local features and global features and output fused multi-scale features; The fused multi-scale features are decoded to obtain the target prediction result.

2. According to claim 1, a lightweight cross-domain infrared target intelligent detection method is characterized by: The lightweight feature extraction network extracts local feature information of different levels from the input enhanced image and outputs the extracted local features; the lightweight feature extraction network includes: ordinary convolution, phantom convolution and phantom module, specifically: After the input image undergoes a normal convolution, it is repeatedly stacked with a phantom convolution and a phantom module five times. The phantom module in the second stack outputs feature F3, the phantom module in the third stack outputs feature F4, the phantom module in the fourth stack outputs feature F5, and the output features of the phantom module in the fifth stack are used as the output of the lightweight feature extraction network.

3. The lightweight cross-domain infrared target intelligent detection method according to claim 2 is characterized in that: Each phantom convolution consists of: a normal convolution and three linear transformations: After the input undergoes ordinary convolution, the ordinary convolution result is retained. At the same time, the convolution result is passed through three linear transformation branches respectively. Finally, the ordinary convolution result and the results of each linear transformation are spliced ​​and summarized into the phantom convolution output.

4. The lightweight cross-domain infrared target intelligent detection method according to claim 2 is characterized in that: Each phantom module includes ordinary convolution and phantom bottleneck: first, the output of phantom convolution is accepted as input for an ordinary convolution, and two branches are output: one branch passes through three layers of phantom bottleneck, and the other branch passes through one layer of ordinary convolution; the outputs of the two branches are spliced, and the spliced ​​result is output after ordinary convolution.

5. The lightweight cross-domain infrared target intelligent detection method according to claim 4 is characterized in that: The phantom bottleneck in the phantom module is a stacked two-layer phantom convolution, where: The number of feature channels of the first phantom convolution output is twice the number of channels of the input feature map. The increase in the number of feature channels is used for expansion. The number of feature channels of the second phantom convolution output is 1 / 2 of the number of channels of the input feature map. The output of the second phantom convolution and the input residual before the first layer of phantom convolution are used as the output of the phantom bottleneck.

6. The lightweight cross-domain infrared target intelligent detection method according to claim 5 is characterized in that: The multi-scale maximum pooling module is used to perform multi-scale pooling downsampling processing on the local features, and features of different scales are obtained and spliced, and the multi-scale splicing features after multi-scale splicing are output; the multi-scale maximum pooling module includes: ordinary convolution, batch normalization, SiLU activation function, and maximum pooling layer; specifically: The feature map output by the lightweight feature extraction network is first subjected to 1*1 ordinary convolution and then batch normalization. The normalized result passes through the SiLU activation function and then passes through three stacked maximum pooling layers. The outputs of each maximum pooling layer and the output of the aforementioned SiLU activation function are spliced. The spliced ​​result passes through 1*1 convolution, and then once again passes through batch normalization and SiLU activation function to output a multi-scale spliced ​​feature map.

7. The lightweight cross-domain infrared target intelligent detection method according to claim 6 is characterized in that: The global self-attention module is used to extract global features from multi-scale splicing features and output global features. The processing flow of the global self-attention module is as follows: A linear mapping calculation is performed on the input multi-scale spliced ​​feature map to obtain the representation of the query Q, key K and value V, where the query Q represents the Query vector, the key K represents the key-value vector, and the value V represents the value vector; then, these representations are input into multiple attention heads to independently calculate the global attention and then spliced, and the splicing result is output through linear transformation, which is recorded as feature F6.

8. The lightweight cross-domain infrared target intelligent detection method according to claim 7 is characterized in that: The local features and global features are fused through the hierarchical feature fusion module and the fused multi-scale features are output; the processing flow of the multi-level feature fusion module is as follows: Features F3, F4, F5 and F6 are used. Feature F6 is concatenated with feature F5 after upsampling, and feature M5 is obtained through the phantom module. The M5 feature is concatenated with the F4 feature after one upsampling, and the M4 feature is obtained through the phantom module. The M4 feature is concatenated with the F3 feature after one upsampling, and feature P3 is obtained through the phantom module. The P3 feature is concatenated with the M4 feature after phantom convolution, and feature P4 is obtained through the phantom module. Feature P4 is concatenated with the M5 feature after phantom convolution, and feature P5 is obtained through the phantom module. Feature P5 is concatenated with feature F6 after phantom convolution, and feature P6 is obtained through the phantom module. Finally, the fused multi-scale features P3, P4, P5 and P6 are output.

9. The lightweight cross-domain infrared target intelligent detection method according to claim 1 is characterized in that: A feature decoding module is used to decode the fused multi-scale features to obtain a target prediction result; the module includes: a decoupling head of a two-layer ordinary convolution combination, and the decoupling head of the two-layer ordinary convolution combination performs decoupling calculation on the output fused multi-scale features of the multi-level feature information fusion module to obtain a target prediction result.

10. A lightweight cross-domain infrared target intelligent detection system, characterized by include: Data enhancement module: performs multi-channel data enhancement on the input visible light image, and outputs any one channel image of the enhanced three-channel image, or the original visible light image; Lightweight feature extraction network module: extracts local feature information of different levels from the input enhanced image through a lightweight feature extraction network, and outputs the extracted local features; Multi-scale maximum pooling module: Use multi-scale maximum pooling technology to perform multi-scale pooling downsampling processing on the local feature map, obtain features of different scales and splice them, and output multi-scale splicing features after multi-scale splicing; Global attention module: extracts global features from multi-scale spliced ​​feature maps and outputs global features; Multi-level feature information fusion module: The multi-level feature information fusion module is used to fuse local features and global features and output fused multi-scale features; Feature decoding module: decodes the fused multi-scale features to obtain the target prediction result.

Images