A small target detection method based on an adaptive intelligent network model

Abstract

The application discloses a small target detection method based on an adaptive intelligent network model, relates to the technical field of target detection, and comprises the following steps: inputting original feature maps into a fusion main branch and a fusion residual connection branch respectively to obtain a fusion main branch output feature map and a fusion residual connection branch output feature map; fusing the fusion main branch output feature map and the fusion residual connection branch output feature map to obtain a Rep-EFEM output feature; inputting an original image into an edge feature extraction and noise suppression module to obtain target edge features; defining bright and dark target query vectors; inputting the bright and dark target query vectors, the target edge features and the Rep-EFEM output feature into a decoder module to obtain target bright and dark target query vectors; and generating a target segmentation image based on the target bright and dark target query vectors, wherein the target segmentation image comprises a category label of each pixel. In this way, the target detection accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a small target detection method based on an adaptive intelligent network model. Background Technology

[0002] In related technologies, infrared target detection is performed based on the brightness attributes of the preset target or the prior distribution of the background. However, the above detection methods usually assume that the gray level of the target is higher than that of the background and ignore the gray level characteristics of dark targets. This makes it difficult for the target detection method to be adapted to the detection of dark targets, resulting in a low target detection accuracy. Summary of the Invention

[0003] In view of this, this application provides a small target detection method based on an adaptive intelligent network model to solve the problem of low detection accuracy in the prior art.

[0004] The objective of this application can be achieved through the following technical solutions:

[0005] The first aspect of this application is to provide a small target detection method based on an adaptive intelligent network model. The adaptive intelligent network model includes at least one Rep-EFEM, an edge feature extraction and noise suppression module, and at least one decoder module. The Rep-EFEM and decoder modules correspond one-to-one. The Rep-EFEM includes a fusion main branch and a fusion residual connection branch, including:

[0006] The original feature maps are input into the fusion main branch and the fusion residual connection branch respectively to obtain the output feature maps of the fusion main branch and the fusion residual connection branch.

[0007] The output feature maps of the fused main branch and the fused residual connection branch are fused to obtain the Rep-EFEM output features;

[0008] The original image is input into the edge feature extraction and noise suppression module to obtain the target edge features;

[0009] Define the query vector for light and dark targets;

[0010] The target query vector (bright and dark), target edge features, and Rep-EFEM output features are input into the decoder module to obtain the target query vector (bright and dark).

[0011] A target segmentation image is generated based on the target brightness / darkness query vector. The target segmentation image includes a category label for each pixel.

[0012] In one optional embodiment, the fusion main branch includes a first EFEM fusion unit, a first depthwise convolutional-batch normalization layer, and a second EFEM fusion unit. The original feature map is input into the fusion main branch to obtain the output feature map of the fusion main branch, including:

[0013] The original feature map is input into the first EFEM fusion unit, and feature enhancement operation is performed on the original feature map to obtain the first feature map;

[0014] The first feature map is input into the first depthwise convolutional-batch normalization layer, and spatial filtering and normalization operations are performed on the first feature map to obtain the second feature map.

[0015] The second feature map is input into the second EFEM fusion unit, and feature enhancement operation is performed on the second feature map to obtain the fusion main branch output feature map.

[0016] In one optional embodiment, the fusion residual connection branch includes a second depthwise convolutional-batch normalization layer and a third depthwise convolutional-batch normalization layer. The original feature map is input into the fusion residual connection branch to obtain the output feature map of the fusion residual connection branch, including:

[0017] The original feature map is input into the second deep convolutional-batch normalization layer, and spatial filtering and normalization operations are performed on the original feature map to obtain the third feature map.

[0018] The third feature map is input into the third depth convolution-batch normalization layer, and spatial filtering and normalization operations are performed on the third feature map to obtain the output feature map of the fused residual connection branch.

[0019] In one optional embodiment, the original image is input into the edge feature extraction and noise suppression module to obtain target edge features, including:

[0020] The original image is input into the edge feature extraction and noise suppression module to calculate the edge feature information corresponding to the original image.

[0021] Based on the edge features obtained from the previous iteration, the preset iteration step size, the first weight parameter, the second weight parameter, the edge feature information, and the denoising and enhanced edge feature information corresponding to the edge features obtained from the previous iteration, the edge features of the current iteration are obtained. The above operation is repeated until the target edge features are obtained. The initial value of the edge features is the edge feature information, and the target edge features are the edge features corresponding to the target iteration number. The target iteration number is the quotient of the total integration time and the iteration step size.

[0022] In one optional embodiment, the original image is input into the edge feature extraction and noise suppression module, and the edge feature information corresponding to the original image is calculated using the following formula:

[0023]

[0024] in, This refers to the edge feature information corresponding to the original image. For the original image, For learnable Sobel operator kernels, For learnable Sobel operator kernels, The width of the original image, For the height of the original image, For the XOR operator, For the Sigmoid function, It is a lightweight convolutional network.

[0025] In one optional embodiment, the edge feature extraction and noise suppression module includes a noise modeling network, and further includes:

[0026] Using a noise modeling network, the denoising and enhanced edge feature information corresponding to the edge features obtained in the previous iteration is obtained based on the following formula:

[0027]

[0028] in, For noise modeling networks, The edge features obtained in the previous iteration, which was the k-th iteration, For the original image, For connections along the channel dimension, MLP() is a multilayer perceptron.

[0029] In one optional embodiment, the edge features of the current iteration are obtained based on the edge features obtained in the previous iteration, a preset iteration step size, a first weight parameter, a second weight parameter, edge feature information, and the denoising and enhanced edge feature information corresponding to the edge features obtained in the previous iteration, including:

[0030] Based on the edge features obtained from the previous iteration, the preset iteration step size, the first weight parameter, the second weight parameter, the edge feature information, and the denoising and enhanced edge feature information corresponding to the edge features obtained from the previous iteration, the edge features of the current iteration are obtained using the following formula:

[0031]

[0032] in, The edge features obtained in the (k+1)th iteration are... The edge features obtained in the k-th iteration are... To preset the iteration step size, As the first weight parameter, This is the second weighting parameter. This refers to the edge feature information corresponding to the original image. For the original image, Model the noise network.

[0033] In one optional embodiment, the bright / dark target query vector, target edge features, and Rep-EFEM output features are input into the decoder module to obtain the target bright / dark target query vector, including:

[0034] The bright and dark target query vectors are processed by a self-attention mechanism to obtain the first enhanced bright and dark target query vectors.

[0035] Layer normalization is performed on the bright and dark target query vector and the first enhanced bright and dark target query vector to obtain the second enhanced bright and dark target query vector;

[0036] A cross-attention mechanism is applied to the second enhanced bright and dark target query vector, the Rep-EFEM output features, and the positional encoding corresponding to the Rep-EFEM output features to obtain the fourth feature map.

[0037] Perform layer normalization on the second enhanced bright and dark target query vector and the fourth feature map to obtain the third enhanced bright and dark target query vector;

[0038] Perform layer normalization on the third enhanced bright and dark target query vector to obtain the fourth enhanced bright and dark target query vector;

[0039] The fifth feature map is obtained by applying a cross-attention mechanism to the Rep-EFEM output features, the position encoding corresponding to the Rep-EFEM output features, and the fourth enhanced bright and dark target query vector.

[0040] The fifth feature map and the Rep-EFEM output features are fused to obtain the fused features;

[0041] The fused features are subjected to layer normalization to obtain the sixth feature map;

[0042] The sixth feature map is upsampled to obtain the seventh feature map;

[0043] An attention mechanism operation is performed on the bright and dark target query vector, the fourth enhanced bright and dark target query vector, the seventh feature map, and the position encoding corresponding to the seventh feature map to obtain the eighth feature map;

[0044] Layer normalization is performed on the eighth feature map and the fourth enhanced bright / dark target query vector to obtain the target bright / dark target query vector.

[0045] In an optional embodiment, during the training phase, Rep-EFEM includes a main branch, a residual connection branch, and a dynamically gated branch, and further includes:

[0046] During the training phase, the original feature maps of the samples are input into the main branch, the residual connection branch, and the dynamic gating branch, respectively, to obtain the output feature maps of the main branch, the residual connection branch, and the dynamic gating branch.

[0047] An adaptive intelligent network model is obtained by training the model based on the output feature maps of the main branch, the residual connection branch, and the dynamic gating branch.

[0048] In one optional embodiment, generating a target segmentation image based on a target brightness / darkness query vector includes:

[0049] A target segmentation image is generated based on the target brightness / darkness query vector and the seventh feature map.

[0050] The second aspect of this application is to provide a small target detection device based on an adaptive intelligent network model. The adaptive intelligent network model includes at least one Rep-EFEM, an edge feature extraction and noise suppression module, and at least one decoder module. The Rep-EFEM and decoder modules correspond one-to-one. The Rep-EFEM includes a fusion main branch and a fusion residual connection branch, comprising:

[0051] The first input module is used to input the original feature map into the fusion main branch and the fusion residual connection branch respectively, so as to obtain the output feature map of the fusion main branch and the output feature map of the fusion residual connection branch.

[0052] The fusion module is used to fuse the output feature map of the fusion main branch and the output feature map of the fusion residual connection branch to obtain the Rep-EFEM output features;

[0053] The second input module is used to input the original image into the edge feature extraction and noise suppression module to obtain the target edge features;

[0054] Define the module to define the query vector for light and dark targets;

[0055] The third input module is used to input the bright / dark target query vector, target edge features and Rep-EFEM output features into the decoder module to obtain the target bright / dark target query vector.

[0056] The generation module is used to generate a target segmentation image based on the target brightness / darkness target query vector. The target segmentation image includes a category label for each pixel.

[0057] A third aspect of this application is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the method as described in the first aspect.

[0058] A fourth aspect of this application is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the method as described in the first aspect.

[0059] Compared with existing technologies, the small target detection method based on an adaptive intelligent network model provided in this application inputs the original feature map into the fusion main branch and the fusion residual connection branch respectively to obtain the output feature map of the fusion main branch and the output feature map of the fusion residual connection branch; the output feature map of the fusion main branch and the output feature map of the fusion residual connection branch are fused to obtain the Rep-EFEM output feature; the original image is input into the edge feature extraction and noise suppression module to obtain the target edge feature; a bright / dark target query vector is defined; the bright / dark target query vector, the target edge feature, and the Rep-EFEM output feature are input into the decoder module to obtain the target bright / dark target query vector; a target segmentation image is generated based on the target bright / dark target query vector, and the target segmentation image includes the category label of each pixel. This improves the target detection accuracy. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 A flowchart illustrating a small target detection method based on an adaptive intelligent network model provided in this application embodiment;

[0062] Figure 2 A schematic diagram of an EFEM provided in an embodiment of this application;

[0063] Figure 3 A schematic diagram of the main branch provided in an embodiment of this application;

[0064] Figure 4 A schematic diagram of a residual connection branch provided in an embodiment of this application;

[0065] Figure 5 A schematic diagram of a dynamic gating branch provided in an embodiment of this application;

[0066] Figure 6 This is a schematic diagram of a structure of an edge embedding unit provided in an embodiment of this application;

[0067] Figure 7Another flowchart illustrating the small target detection method based on an adaptive intelligent network model provided in this application embodiment;

[0068] Figure 8 A structural block diagram of a small target detection device based on an adaptive intelligent network model provided in an embodiment of this application;

[0069] Figure 9 This is a structural block diagram of an electronic device for implementing a small target detection method based on an adaptive intelligent network model, as provided in an embodiment of this application. Detailed Implementation

[0070] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0071] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0072] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship. "Contains A, B and / or C" means containing any one, two, or three of A, B, and C.

[0073] It should be understood that in the embodiments of this application, "B corresponding to A", "B corresponding to A", "A corresponds to B" or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0074] To address the technical problems existing in related technologies, this application provides a small target detection method based on an adaptive intelligent network model.

[0075] The small target detection method based on an adaptive intelligent network model provided in this application can be executed by an electronic device, such as a terminal or a server. The terminal can be a smartphone, tablet, laptop, or other similar device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. It is understood that this application does not limit the specific entity executing the small target detection method based on the adaptive intelligent network model.

[0076] The technical solution of this application will be described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments described below are used to explain the technical solution of this application and are not intended to limit actual use.

[0077] To address the technical problems existing in related technologies, embodiments of this application provide a small target detection method based on an adaptive intelligent network model, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating a small target detection method based on an adaptive intelligent network model provided in an embodiment of this application. It should be noted that the steps shown may be executed in a different logical order than those shown in the flowchart. The method may include the following steps S101 to S106.

[0078] Step S101: Input the original feature maps into the fusion main branch and the fusion residual connection branch respectively to obtain the output feature maps of the fusion main branch and the fusion residual connection branch.

[0079] In an optional embodiment, the adaptive intelligent network model in the small target detection method based on the adaptive intelligent network model provided in this application includes at least one Rep-EFEM (Reparameterization Efficient Feature Enhancement Module), which includes a fusion main branch and a fusion residual connection branch.

[0080] In one optional embodiment, the original feature map can be an infrared image containing small-sized targets, where small-sized targets are those smaller than a preset threshold. The infrared image includes targets with brightness greater than the background brightness and targets with brightness less than the background brightness.

[0081] In one optional embodiment, the fusion main branch includes a first EFEM (Efficient Feature Enhancement Module) fusion unit, a first depthwise convolutional-batch normalization layer, and a second EFEM fusion unit. The original feature map is input into the fusion main branch to obtain the output feature map of the fusion main branch. Specifically, the steps include: inputting the original feature map into the first EFEM fusion unit and performing feature enhancement operations on the original feature map to obtain a first feature map; inputting the first feature map into the first depthwise convolutional-batch normalization layer and performing spatial filtering and normalization operations on the first feature map to obtain a second feature map; and inputting the second feature map into the second EFEM fusion unit and performing feature enhancement operations on the second feature map to obtain the output feature map of the fusion main branch.

[0082] In an optional embodiment, the original feature map is input into the fusion main branch, and the output feature map of the fusion main branch is obtained using the following formula:

[0083] (1)

[0084] in, This indicates the fusion of the main branch's output feature map. This indicates the operation performed through EFEM. This indicates a batch normalization operation. This indicates a depthwise separable convolution operation. This represents the original feature map.

[0085] In one specific embodiment, such as Figure 2As shown, EFEM includes a PW Conv (Point-Wise Convolution) layer, two ReLU (Rectified Linear Unit) activation layers, two Batch Normalization (BN) layers, a DW Conv (Depthwise Separable Convolution) layer, and a Concat (connection) layer. The PW Conv layer, ReLU layer, BN layer, DW Conv layer, ReLU layer, BN layer, and Concat layer are connected sequentially.

[0086] In one optional embodiment, the fusion residual connection branch includes a second depthwise convolutional-batch normalization layer and a third depthwise convolutional-batch normalization layer. The original feature map is input into the fusion residual connection branch to obtain the output feature map. Specifically, this includes the following steps: inputting the original feature map into the second depthwise convolutional-batch normalization layer, performing spatial filtering and normalization operations on the original feature map to obtain a third feature map; inputting the third feature map into the third depthwise convolutional-batch normalization layer, performing spatial filtering and normalization operations on the third feature map to obtain the output feature map of the fusion residual connection branch.

[0087] In a more specific embodiment, the original feature map is input into the fusion residual connection branch, and the output feature map of the fusion residual connection branch is obtained using the following formula:

[0088] (2)

[0089] in, This represents the feature map output from the fused residual connection branch. This indicates a batch normalization operation. This represents the standard convolution operation. This represents the original feature map.

[0090] Step S102: Fuse the output feature map of the fused main branch and the output feature map of the fused residual connection branch to obtain the Rep-EFEM output feature.

[0091] In one optional embodiment, the fused main branch output feature map and the fused residual connection branch output feature map are added together to obtain the Rep-EFEM output feature.

[0092] In one specific embodiment, the fused main branch output feature map and the fused residual connection branch output feature map are added together using the following formula:

[0093] (3)

[0094] in, This represents the output characteristics of Rep-EFEM. This indicates the fusion of the main branch's output feature map. This represents the feature map output by the fused residual connection branch.

[0095] In another alternative embodiment, such as Figure 3 As shown, the main branch includes a first FEFM unit, a second FEFM unit, a first depthwise separable convolutional layer, and a first batch of normalized layers. The first FEFM unit, the first depthwise separable convolutional layer, the first batch of normalized layers, and the second FEFM unit are connected sequentially. Figure 4 As shown, the residual connection branch includes a first convolutional layer, a second convolutional layer, a second batch normalization layer, and a third batch normalization layer, which are connected sequentially; as... Figure 5 As shown, the dynamic gating branch includes a third convolutional layer and a fourth batch normalized layer, which are connected sequentially.

[0096] During the training phase, Rep-EFEM includes a main branch, a residual connection branch, and a dynamic gating branch, and also includes the following steps: During the training phase, the original feature maps of the samples are input into the main branch, the residual connection branch, and the dynamic gating branch, respectively, to obtain the output feature maps of the main branch, the residual connection branch, and the dynamic gating branch; the model is trained based on the output feature maps of the main branch, the residual connection branch, and the dynamic gating branch to obtain an adaptive intelligent network model.

[0097] In one specific embodiment, the original feature map of the sample is input into the dynamic gating branch, and the output feature map of the dynamic gating branch is obtained using the following formula:

[0098] (4)

[0099] in, This represents the feature map of the output of the dynamic gated branch. Scaling factor This indicates batch normalization, and Conv indicates the standard convolution operation. This represents the original feature map of the sample.

[0100] In one alternative embodiment, The initial value can be set based on experience, for example, The value may be 0.3, 0.5, or 0.6; this application does not specify which value to use.

[0101] In this embodiment, considering the model's execution efficiency, parameter fusion and reparameterization are performed during testing. While preserving model performance, the multi-branch structure is simplified into a two-branch structure to improve the model's running efficiency.

[0102] Furthermore, by introducing residual connection branches in Rep-EFEM, effective information transfer is ensured, and the vanishing gradient problem in deep networks is mitigated. The dynamic gating branch introduces a learnable scaling factor. To dynamically adjust features.

[0103] Step S103: Input the original image into the edge feature extraction and noise suppression module to obtain the target edge features.

[0104] In one optional embodiment, the adaptive intelligent network model includes an edge feature extraction and noise suppression module. The original image is input into the edge feature extraction and noise suppression module to obtain target edge features, specifically including the following steps:

[0105] The original image is input into the edge feature extraction and noise suppression module to calculate the edge feature information corresponding to the original image.

[0106] Based on the edge features obtained from the previous iteration, the preset iteration step size, the first weight parameter, the second weight parameter, the edge feature information, and the denoising and enhanced edge feature information corresponding to the edge features obtained from the previous iteration, the edge features of the current iteration are obtained. The above operation is repeated until the target edge features are obtained. The initial value of the edge features is the edge feature information, and the target edge features are the edge features corresponding to the target iteration number. The target iteration number is the quotient of the total integration time and the iteration step size.

[0107] In one optional embodiment, the original image is input into the edge feature extraction and noise suppression module, and the edge feature information corresponding to the original image is calculated using the following formula:

[0108] (5)

[0109] in, This refers to the edge feature information corresponding to the original image. For the original image, For learnable Sobel operator kernels, For learnable Sobel operator kernels, The width of the original image, For the height of the original image, For the XOR operator, For activation function, It is a lightweight convolutional network.

[0110] In an optional embodiment, the edge feature extraction and noise suppression module includes a noise modeling network. The small target detection method based on an adaptive intelligent network model provided in this application embodiment further includes:

[0111] Using a noise modeling network, the denoising and enhanced edge feature information corresponding to the edge features obtained in the previous iteration is obtained based on the following formula:

[0112] (6)

[0113] in, For noise modeling networks, The edge features obtained in the previous iteration, which was the k-th iteration, For the original image, MLP() is a multilayer perceptron for splicing along the channel dimension.

[0114] In one optional embodiment, based on the edge features obtained in the previous iteration, the preset iteration step size, the first weight parameter, the second weight parameter, the edge feature information, and the denoised and enhanced edge feature information corresponding to the edge features obtained in the previous iteration, the edge features of the current iteration are obtained using the following formula:

[0115] (7)

[0116] in, The edge features obtained in the (k+1)th iteration are... The edge features obtained in the k-th iteration are... To preset the iteration step size, As the first weight parameter, This is the second weighting parameter. This refers to the edge feature information corresponding to the original image. For the original image, Model the noise network.

[0117] In this step, to address the noise introduction problem caused by gradient operators in complex scenes, an edge feature extraction and noise suppression module is proposed. Edge feature extraction and noise suppression are modeled as a physical process of finding steady-state solutions to partial differential equations, thus achieving coupling between edge extraction and noise suppression. An implicit noise learning network is used to approximate noise distributions of arbitrary complexity, thereby achieving better edge noise suppression capabilities.

[0118] In another alternative embodiment, in order to control memory usage during the solution process, the computation graph is truncated by a detach operation every 5 iterations during the iterative solution process (e.g., after the calculations in steps 5 and 10, if the total number of iterations is 10) to control memory overhead.

[0119] In another optional embodiment, the edge feature extraction and noise suppression module further includes an edge embedding unit, such as... Figure 6 As shown, the edge embedding unit comprises three Conv (convolutional) layers, two Norm (normalization) layers, and two GELU (Gaussian Error Linear Unit) activation function layers, wherein the Conv, Norm, GELU, Conv, Norm, GELU, and Conv layers are connected sequentially. The edge embedding module maps the target edge features to a spatial scale that matches the decoder module mentioned later, generating a feature map aligned with the decoder module. This allows the subsequent decoder to focus more on the enhanced and denoised edge features, improving the accuracy of capturing small target features.

[0120] Step S104: Define the query vector for bright and dark targets.

[0121] In an alternative embodiment, the bright / dark target query vector can be represented as: ,in, Let d represent the real number field, where d is the spatial dimension. The two components of the first dimension correspond to the bright target vector and the dark target vector, respectively.

[0122] In another alternative embodiment, cosine distance is used during model training to calculate the distance between the bright target vector and the dark target vector, thereby driving one vector to focus on capturing the bright target features of the original image and another vector to focus on capturing the dark target features of the original image.

[0123] The expression for calculating the cosine distance is as follows:

[0124] (8)

[0125] in, This represents the distance between the bright target vector and the dark target vector. Represents the bright target vector. This represents the dark target vector.

[0126] Step S105: Input the bright / dark target query vector, target edge features, and Rep-EFEM output features into the decoder module to obtain the target bright / dark target query vector.

[0127] In an optional embodiment, the adaptive intelligent network model further includes a decoder module. The bright / dark target query vector, target edge features, and Rep-EFEM output features are input into the decoder module to obtain the target bright / dark target query vector. Specifically, this includes the following steps: performing self-attention processing on the bright / dark target query vector to obtain a first enhanced bright / dark target query vector; performing layer normalization processing on the bright / dark target query vector and the first enhanced bright / dark target query vector to obtain a second enhanced bright / dark target query vector; performing cross-attention processing on the second enhanced bright / dark target query vector, the Rep-EFEM output features, and the positional encoding corresponding to the Rep-EFEM output features to obtain a fourth feature map; and performing layer normalization on the second enhanced bright / dark target query vector and the fourth feature map to obtain a third enhanced bright / dark target query vector. The process involves: 1) Performing layer normalization on the third enhanced bright / dark target query vector to obtain the fourth enhanced bright / dark target query vector; 2) Applying a cross-attention mechanism to the Rep-EFEM output features, the positional encoding corresponding to the Rep-EFEM output features, and the fourth enhanced bright / dark target query vector to obtain the fifth feature map; 3) Fusing the fifth feature map and the Rep-EFEM output features to obtain the fused feature; 4) Performing layer normalization on the fused feature to obtain the sixth feature map; 5) Upsampling the sixth feature map to obtain the seventh feature map; 6) Applying an attention mechanism to the bright / dark target query vector, the fourth enhanced bright / dark target query vector, the seventh feature map, and the positional encoding corresponding to the seventh feature map to obtain the eighth feature map; 7) Performing layer normalization on the eighth feature map and the fourth enhanced bright / dark target query vector to obtain the target bright / dark target query vector.

[0128] In one specific embodiment, the bright / dark target query vector is processed using a self-attention mechanism using the following formula to obtain a first enhanced bright / dark target query vector:

[0129] (9)

[0130] in, This represents the first enhanced bright / dark target query vector. This represents the query vector for bright and dark targets. For activation function, The projection matrix is ​​learnable. This indicates the dimension of the query vector representing the light and dark targets.

[0131] In one specific embodiment, layer normalization is performed on the bright / dark target query vector and the first enhanced bright / dark target query vector using the following formula to obtain the second enhanced bright / dark target query vector:

[0132] (10)

[0133] in, This represents the second enhanced bright / dark target query vector. Representation layer normalization, This represents the first enhanced bright / dark target query vector. This represents the query vector for both light and dark targets.

[0134] In one specific embodiment, the second enhanced bright / dark target query vector, the Rep-EFEM output features, and the positional encoding corresponding to the Rep-EFEM output features are processed using a cross-attention mechanism through the following formula to obtain the fourth feature map:

[0135] (11)

[0136] in, This represents the fourth feature map. This represents the second enhanced bright / dark target query vector. This represents the output characteristics of Rep-EFEM. This indicates the positional encoding corresponding to the Rep-EFEM output features. The projection matrix is ​​learnable. This represents the dimension of the second enhanced bright / dark target query vector.

[0137] In one specific embodiment, a layer normalization operation is performed on the second enhanced bright / dark target query vector and the fourth feature map using the following formula to obtain the third enhanced bright / dark target query vector:

[0138] (12)

[0139] in, This represents the third enhanced bright / dark target query vector. Representation layer normalization, This represents the fourth feature map. This represents the second enhanced bright / dark target query vector.

[0140] In one specific embodiment, a layer normalization operation is performed on the third enhanced bright / dark target query vector using the following formula to obtain the fourth enhanced bright / dark target query vector:

[0141] (13)

[0142] in, This represents the fourth enhanced bright / dark target query vector. Representation layer normalization, It is a multilayer perceptron. This represents the third enhanced bright / dark target query vector.

[0143] It should be noted that MLP is a feedforward network that includes two layers of linear transformation and nonlinear activation.

[0144] In one specific embodiment, the Rep-EFEM output features, the positional encoding corresponding to the Rep-EFEM output features, and the fourth enhanced bright / dark target query vector are processed using a cross-attention mechanism using the following formula to obtain the fifth feature map:

[0145] (14)

[0146] in, This represents the fifth feature map. This represents the output characteristics of Rep-EFEM. Indicates position code, The projection matrix is ​​learnable. This represents the dimension of the fourth enhanced bright / dark target query vector. This represents the fourth enhanced bright / dark target query vector.

[0147] In one specific embodiment, the fused features are subjected to layer normalization using the following formula to obtain the sixth feature map:

[0148] (15)

[0149] in, Represents the sixth feature map. This represents the output characteristics of Rep-EFEM. This represents the fifth feature map. Representation layer normalization.

[0150] In one specific embodiment, the following formula is used to perform an attention mechanism operation on the bright / dark target query vector, the fourth enhanced bright / dark target query vector, the seventh feature map, and the positional encoding corresponding to the seventh feature map to obtain the eighth feature map:

[0151] (16)

[0152] in, This represents the eighth feature map. This represents the fourth enhanced bright / dark target query vector. This represents the query vector for bright and dark targets. The projection matrix is ​​learnable. This represents the dimension of the fourth enhanced bright / dark target query vector. Represents the seventh feature map. This represents the positional encoding corresponding to the seventh feature map. .

[0153] In one specific embodiment, the target brightness / darkness query vector is obtained by performing a layer normalization operation on the eighth feature map and the fourth enhanced brightness / darkness target query vector using the following formula:

[0154] (17)

[0155] in, The query vector represents the target's brightness or darkness. This represents the eighth feature map. This represents the fourth enhanced bright / dark target query vector.

[0156] By introducing learnable query vectors to specifically characterize the light and dark features of the target, and using an attention mechanism to adaptively learn the image feature representations most relevant to these two prior features, the module achieves adaptive recognition of targets of different polarities. It dynamically decouples and fuses light and dark features through a multi-level cross-attention mechanism.

[0157] Step S106: Generate a target segmentation image based on the target brightness / darkness query vector.

[0158] It should be noted that the target segmentation image includes a category label for each pixel.

[0159] In one optional embodiment, a target segmentation image is generated based on the target brightness / darkness query vector, specifically including the following steps:

[0160] A target segmentation image is generated based on the target brightness / darkness query vector and the seventh feature map.

[0161] In one specific embodiment, based on the target brightness / darkness query vector and the seventh feature map, a target segmentation image is generated using the following formula:

[0162] (18)

[0163] in, Represents the target segmented image. The query vector represents the target's brightness or darkness. This represents the seventh feature map.

[0164] In a more specific embodiment, such as Figure 5 As shown, the small target detection method based on an adaptive intelligent network model provided in this application, when the adaptive intelligent network model includes three Rep-EFEM modules, an edge feature extraction and noise suppression module, and three decoder modules, specifically includes the following steps:

[0165] Step 1: Input the original feature map into the first Rep-EFEM module to obtain feature map A;

[0166] Step 2: Input feature map A into the second Rep-EFEM module to obtain feature map B;

[0167] Step 3: Input feature map B into the second Rep-EFEM module to obtain feature map C;

[0168] Step 4: Input the original image into the edge feature extraction and noise suppression module to obtain the target edge features;

[0169] Step 5: Fuse the target edge features and feature map C, and input the fused features and the first bright / dark target query vector into the first decoder module to obtain the first target bright / dark target query vector;

[0170] Step 6: Fuse the feature map B and the first target brightness / darkness query vector, and input the fused feature map and the introduced second brightness / darkness query vector into the second decoder module to obtain the second target brightness / darkness query vector;

[0171] Step 7: Fuse feature map A and the second target bright / dark target query vector, and input the fused feature and the introduced third bright / dark target query vector into the third decoder module to obtain the third target bright / dark target query vector;

[0172] Step 8: Generate a target segmentation image based on the third target brightness / darkness query vector and the feature map output from the third decoder module.

[0173] The small target detection method based on an adaptive intelligent network model provided in this application demonstrates significant advantages in handling infrared small target detection tasks involving complex brightness variations. It achieves a pixel accuracy of 91.14%, an average intersection-over-union ratio of 85.15%, a detection probability of 94.69%, and a false alarm rate of 4.282 × 10⁻⁶. -6 .

[0174] Corresponding to the small target detection method based on an adaptive intelligent network model provided in the embodiments of this application, the embodiments of this application also provide a small target detection device based on an adaptive intelligent network model, such as... Figure 8 As shown, the adaptive intelligent network model includes at least one Rep-EFEM, an edge feature extraction and noise suppression module, and at least one decoder module. The Rep-EFEM and decoder modules correspond one-to-one. The Rep-EFEM includes a fusion main branch and a fusion residual connection branch, including:

[0175] The first input module 801 is used to input the original feature map into the fusion main branch and the fusion residual connection branch respectively to obtain the output feature map of the fusion main branch and the output feature map of the fusion residual connection branch;

[0176] The fusion module 802 is used to fuse the output feature map of the fusion main branch and the output feature map of the fusion residual connection branch to obtain the Rep-EFEM output features;

[0177] The second input module 803 is used to input the original image into the edge feature extraction and noise suppression module to obtain the target edge features;

[0178] Define module 804 to define the query vector for bright and dark targets;

[0179] The third input module 805 is used to input the bright and dark target query vector, target edge features and Rep-EFEM output features into the decoder module to obtain the target bright and dark target query vector;

[0180] The generation module 806 is used to generate a target segmentation image based on the target brightness / darkness target query vector. The target segmentation image includes a category label for each pixel.

[0181] Corresponding to the small target detection method based on an adaptive intelligent network model provided in the embodiments of this application, the embodiments of this application also provide an electronic device for executing the small target detection method based on an adaptive intelligent network model, such as... Figure 9 As shown, the electronic device includes: a processor 901; and a memory 902 for storing a program for a small target detection method based on an adaptive intelligent network model. After the device is powered on and the program for the small target detection method based on the adaptive intelligent network model is run by the processor, the following steps are performed:

[0182] The original feature maps are input into the fusion main branch and the fusion residual connection branch respectively to obtain the output feature maps of the fusion main branch and the fusion residual connection branch.

[0183] The output feature maps of the fused main branch and the fused residual connection branch are fused to obtain the Rep-EFEM output features;

[0184] The original image is input into the edge feature extraction and noise suppression module to obtain the target edge features;

[0185] Define the query vector for light and dark targets;

[0186] The target query vector (bright and dark), target edge features, and Rep-EFEM output features are input into the decoder module to obtain the target query vector (bright and dark).

[0187] A target segmentation image is generated based on the target brightness / darkness query vector. The target segmentation image includes a category label for each pixel.

[0188] Corresponding to the small target detection method based on an adaptive intelligent network model provided in the embodiments of this application, the embodiments of this application also provide a computer-readable storage medium storing a program for a small target detection method based on an adaptive intelligent network model. This program is executed by a processor to perform the following steps:

[0189] The original feature maps are input into the fusion main branch and the fusion residual connection branch respectively to obtain the output feature maps of the fusion main branch and the fusion residual connection branch.

[0190] The output feature maps of the fused main branch and the fused residual connection branch are fused to obtain the Rep-EFEM output features;

[0191] The original image is input into the edge feature extraction and noise suppression module to obtain the target edge features;

[0192] Define the query vector for light and dark targets;

[0193] The target query vector (bright and dark), target edge features, and Rep-EFEM output features are input into the decoder module to obtain the target query vector (bright and dark).

[0194] A target segmentation image is generated based on the target brightness / darkness query vector. The target segmentation image includes a category label for each pixel.

[0195] Corresponding to the small target detection method based on an adaptive intelligent network model provided in the embodiments of this application, the embodiments of this application also provide a computer program containing instructions, which, when executed by a computer, cause the computer to perform the following steps:

[0196] The original feature maps are input into the fusion main branch and the fusion residual connection branch respectively to obtain the output feature maps of the fusion main branch and the fusion residual connection branch.

[0197] The output feature maps of the fused main branch and the fused residual connection branch are fused to obtain the Rep-EFEM output features;

[0198] The original image is input into the edge feature extraction and noise suppression module to obtain the target edge features;

[0199] Define the query vector for light and dark targets;

[0200] The target query vector (bright and dark), target edge features, and Rep-EFEM output features are input into the decoder module to obtain the target query vector (bright and dark).

[0201] A target segmentation image is generated based on the target brightness / darkness query vector. The target segmentation image includes a category label for each pixel.

[0202] It should be noted that for a detailed description of the small target detection device, electronic device, computer-readable storage medium and computer program based on the adaptive intelligent network model provided in the embodiments of this application, please refer to the relevant description of the small target detection method based on the adaptive intelligent network model provided in the embodiments of this application, which will not be repeated here.

[0203] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0204] In a typical configuration, an electronic device includes one or more processors (Central Processing Units), input / output interfaces, network interfaces, and memory.

[0205] Memory may include non-persistent storage in computer-readable media, such as random access memory and / or non-volatile memory, like read-only memory or flash memory. Memory is an example of computer-readable media.

[0206] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable operations, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PCM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DMCD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0207] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, compact disc read-only memory, optical storage, etc.) containing computer-usable program code.

[0208] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. A method for small target detection based on an adaptive intelligent network model, characterized in that, The adaptive intelligent network model comprises at least one Rep-EFEM, an edge feature extraction and noise suppression module and at least one decoder module, the Rep-EFEM and the decoder module correspond to each other, the Rep-EFEM comprises a fusion main branch and a fusion residual connection branch, the fusion main branch comprises a first EFEM fusion unit, a first deep convolution-batch normalization layer and a second EFEM fusion unit, the fusion residual connection branch comprises a second deep convolution-batch normalization layer and a third deep convolution-batch normalization layer, comprising: inputting an original feature map into the first EFEM fusion unit, performing a feature enhancement operation on the original feature map to obtain a first feature map; inputting the first feature map into the first deep convolution-batch normalization layer, performing a spatial filtering operation and a normalization operation on the first feature map to obtain a second feature map; inputting the second feature map into the second EFEM fusion unit, performing a feature enhancement operation on the second feature map to obtain a fusion main branch output feature map; inputting the original feature map into the second deep convolution-batch normalization layer, performing a spatial filtering operation and a normalization operation on the original feature map to obtain a third feature map; inputting the third feature map into the third deep convolution-batch normalization layer, performing a spatial filtering operation and a normalization operation on the third feature map to obtain a fusion residual connection branch output feature map; fusing the fusion main branch output feature map and the fusion residual connection branch output feature map to obtain a Rep-EFEM output feature; inputting an original image into the edge feature extraction and noise suppression module to obtain a target edge feature; defining a light and dark target query vector; inputting the light and dark target query vector, the target edge feature and the Rep-EFEM output feature into the decoder module to obtain a target light and dark target query vector; generating a target segmentation image based on the target light and dark target query vector, the target segmentation image comprising a class label of each pixel.

2. The method for small object detection based on adaptive intelligent network model according to claim 1, characterized in that, The inputting an original image into the edge feature extraction and noise suppression module to obtain a target edge feature comprises: inputting an original image into the edge feature extraction and noise suppression module to calculate edge feature information corresponding to the original image; obtaining an edge feature of a current iteration based on an edge feature obtained in a last iteration, a preset iteration step, a first weight parameter, a second weight parameter, the edge feature information and denoising enhanced edge feature information corresponding to the edge feature obtained in the last iteration, and repeatedly performing the above operation until a target edge feature is obtained, an initial value of the edge feature being the edge feature information, the target edge feature being an edge feature corresponding to a target iteration number, and the target iteration number being a quotient of an integral total time length and an iteration step.

3. The method for small object detection based on adaptive intelligent network model according to claim 2, characterized in that, inputting an original image into the edge feature extraction and noise suppression module, and calculating edge feature information corresponding to the original image by using the following formula: Wherein, the is the edge feature information corresponding to the original image, the is the original image, the is a learnable Sobel operator kernel, the is a learnable Sobel operator kernel, the is the width of the original image, the is the height of the original image, the is an exclusive or operator, the is a Sigmoid function, the is a lightweight convolutional network.

4. The method for small object detection based on adaptive intelligent network model according to claim 2, characterized in that, The edge feature extraction and noise suppression module comprises a noise modeling network, and further comprises: The noise modeling network obtains denoised enhanced edge feature information corresponding to the edge feature obtained in the last iteration based on the following formula: Wherein, the is a noise modeling network, the is an edge feature obtained in the last iteration, the is the original image, the is connected along the channel dimension, and the MLP() is a multi-layer perceptron.

5. The method for small object detection based on adaptive intelligent network model according to claim 2, characterized in that, The edge feature obtained in the current iteration is obtained based on the edge feature obtained in the last iteration, the preset iteration step, the first weight parameter, the second weight parameter, the edge feature information, and the denoised enhanced edge feature information corresponding to the edge feature obtained in the last iteration, and the following formula is used: The target light and dark target query vector is obtained by inputting the light and dark target query vector, the target edge feature, and the Rep-EFEM output feature into the decoder module. Wherein, the edge feature information corresponding to the original image is obtained by the noise modeling network. is the edge feature obtained in the k+1th iteration, and the edge feature obtained in the kth iteration is is the edge feature obtained in the k+1th iteration, and the edge feature obtained in the kth iteration is is a preset iteration step, and the edge feature obtained in the k+1th iteration is is a first weight parameter, and the edge feature obtained in the k+1th iteration is is a second weight parameter, and the edge feature obtained in the k+1th iteration is is the edge feature information corresponding to the original image, and the edge feature information corresponding to the original image is obtained by the noise modeling network. is the original image, and the original image is obtained by the image preprocessing network. is a noise modeling network.

6. The method for small object detection based on adaptive intelligent network model according to claim 1, characterized in that, The light and dark target query vector is processed by a self-attention mechanism to obtain a first enhanced light and dark target query vector. The light and dark target query vector and the first enhanced light and dark target query vector are subjected to layer normalization processing to obtain a second enhanced light and dark target query vector. The second enhanced light and dark target query vector, the Rep-EFEM output feature, and the position encoding corresponding to the Rep-EFEM output feature are subjected to cross-attention mechanism processing to obtain a fourth feature map. The second enhanced light and dark target query vector and the fourth feature map are subjected to layer normalization operation to obtain a third enhanced light and dark target query vector. The third enhanced light and dark target query vector is subjected to layer normalization operation to obtain a fourth enhanced light and dark target query vector. The Rep-EFEM output feature, the position encoding corresponding to the Rep-EFEM output feature, and the fourth enhanced light and dark target query vector are subjected to cross-attention mechanism processing to obtain a fifth feature map. The fifth feature map and the Rep-EFEM output feature are fused to obtain a fusion feature. The fusion feature is subjected to layer normalization operation to obtain a sixth feature map. The sixth feature map is subjected to upsampling operation to obtain a seventh feature map. The light and dark target query vector, the fourth enhanced light and dark target query vector, the seventh feature map, and the position encoding corresponding to the seventh feature map are subjected to attention mechanism operation to obtain an eighth feature map. The eighth feature map and the fourth enhanced light and dark target query vector are subjected to layer normalization operation to obtain a target light and dark target query vector. In the training phase, the Rep-EFEM includes a main branch, a residual connection branch, and a dynamic gating branch, and further includes:

7. The method for small object detection based on adaptive intelligent network model according to claim 1, characterized in that, In the training phase, the sample original feature map is input into the main branch, the residual connection branch, and the dynamic gating branch to obtain a main branch output feature map, a residual connection branch output feature map, and a dynamic gating branch output feature map. The main branch output feature map, the residual connection branch output feature map, and the dynamic gating branch output feature map are used for model training to obtain an adaptive intelligent network model. ​ 8. The method for small object detection based on adaptive intelligent network model according to claim 7, characterized in that, The generating a target segmentation image based on the target light-dark target query vector comprises: Generating a target segmentation image based on the target light-dark target query vector and the seventh feature map.

Images