A symmetric progressively decreasing network quantization method for single-frame infrared dim target detection

Through the symmetric gradual decreasing network quantization method, the gradually decreasing bit width is allocated to the encoder and decoder convolution blocks of deep learning networks, solving the deployment problem of infrared weak target detection on resource-limited devices, and achieving efficient computing resource compression and accuracy is maintained.

CN117634567BActive Publication Date: 2024-11-22NAT UNIV OF DEFENSE TECH
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202311688198.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-11
Publication Date
2024-11-22
Estimated Expiration
2043-12-11

AI Technical Summary

Technical Problem

Infrared weak object detection methods based on deep learning networks require large computing resources and hardware requirements, and are difficult to deploy on devices with limited resources.

Method used

The symmetric progressive decreasing network quantization method is adopted to allocate progressive decreasing weights and activation bit widths to the encoder and decoder convolution blocks of deep learning networks, reducing the computing resource requirements, so that the network can run on resource-limited hardware.

Benefits of technology

While maintaining the detection accuracy of weak infrared targets, it significantly reduces memory footprint and computing costs, allowing the network to be deployed and operated on devices with limited resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117634567B_ABST
    Figure CN117634567B_ABST
Patent Text Reader

Abstract

The present application discloses a symmetrical progressively decreasing network quantization method for detecting single-frame infrared dim targets, which relates to the field of infrared image processing technology, including: allocating floating point weight bit width and activation bit width of 32 bits to the first-layer encoder convolution block in the initial deep learning network model, to obtain the first deep learning network model; allocating weight bit width to all encoder convolution blocks and decoder convolution blocks of the first deep learning network model in a symmetrical progressively decreasing manner of weight bit width, to obtain the second deep learning network model; allocating activation bit width to all encoder convolution blocks and decoder convolution blocks in the second deep learning network model in a symmetrical progressively decreasing manner of activation bit width, to obtain the target deep learning network model for detecting single-frame infrared dim targets. The present application can reduce memory usage and computing costs while maintaining detection accuracy, reduce computing resources, and enable the network to be deployed and run on hardware devices with limited resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of infrared image processing technology, and in particular to a symmetrical progressively decreasing network quantization method for single-frame infrared dim small target detection. Background Art

[0002] Currently, infrared small target detection technology has been widely used in tasks such as marine target monitoring and body environment monitoring. It is used to segment the target of interest from the infrared image collected by the infrared payload camera and obtain its spatial position and motion state. The traditional infrared small target detection method is to achieve the segmentation of infrared small targets by measuring the discontinuity between the target and the background, including filtering-based methods, local contrast measurement-based methods, and low-rank-based methods. However, such methods are difficult to handle these more complex changes using manually extracted features and fixed hyperparameters due to the more changes in the size, shape, and background clutter of the target in the real scene.

[0003] In order to address the shortcomings of traditional methods, a deep learning network-based infrared dim target detection method has been proposed. This method learns trainable parameters in a data-driven manner to achieve better performance than traditional methods. However, the deep learning network-based method relies on a highly over-parameterized 32-bit model, which requires expensive computational costs and hardware requirements, making it difficult to deploy this method on resource-limited target detection hardware devices.

[0004] Therefore, how to solve the practical problem that the deep learning network for single-frame infrared dim target detection requires large computing resources and hardware requirements, and is difficult to deploy on devices with limited resources is a problem that needs to be further solved in this field. Summary of the invention

[0005] In view of this, the purpose of this application is to provide a symmetric progressively decreasing network quantization method for single-frame infrared dim target detection, which can compress the computing resources of the deep learning network while maintaining the accuracy of infrared dim target detection, reduce memory usage and computing costs, and enable the network to be deployed and run on hardware devices with limited resources. The specific scheme is as follows:

[0006] The present application discloses a symmetric progressively decreasing network quantization method for single-frame infrared dim target detection, comprising:

[0007] Obtain a deep learning network model for single-frame infrared dim target detection and obtain an initial deep learning network model;

[0008] Allocating a weight bit width and an activation bit width of 32 bits as floating point numbers to a first layer encoder convolution block in the initial deep learning network model to obtain a first deep learning network model;

[0009] Allocating weight bit widths to all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution block in the first deep learning network model in a symmetrical and progressively decreasing manner, to obtain a second deep learning network model;

[0010] Activation bit widths are assigned to all encoder convolution blocks and all decoder convolution blocks in the second deep learning network model except the first-layer encoder convolution blocks in a symmetrical and progressively decreasing manner to obtain a target deep learning network model, and the single-frame infrared dim target is detected using the target deep learning network model.

[0011] Optionally, allocating activation bit widths to all encoder convolution blocks and all decoder convolution blocks in the second deep learning network model except the first-layer encoder convolution block in a symmetrical and progressively decreasing manner to obtain a target deep learning network model includes:

[0012] Allocating activation bit widths to all encoder convolution blocks and all decoder convolution blocks in the second deep learning network model except the first-layer encoder convolution block in a symmetrical and progressively decreasing manner to obtain a third deep learning network model;

[0013] A higher bit width than that of a decoder convolution block in the same layer is respectively allocated to the encoder convolution block in the third deep learning network model to obtain a target deep learning network model.

[0014] Optionally, the activation bit width is allocated to all encoder convolution blocks and all decoder convolution blocks in the second deep learning network model except the first layer encoder convolution block in a symmetrical and progressively decreasing manner to obtain a third deep learning network model, including:

[0015] According to the preset activation bit width allocation interval and based on the symmetrical and gradual decrease of the activation bit width, activation bit width is allocated to all encoder convolution blocks and all decoder convolution blocks in the second deep learning network model except the first layer encoder convolution block, to obtain a third deep learning network model; wherein the activation bit width allocation interval is an interval between 3 bits and 4 bits.

[0016] Optionally, the weight bit width is allocated to all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution block in the first deep learning network model in a symmetrical and progressively decreasing manner to obtain a second deep learning network model, including:

[0017] According to the preset weight bit width allocation interval and based on the symmetrical and gradual decrease of the weight bit width, weight bit widths are allocated to all encoder convolution blocks and all decoder convolution blocks in the first deep learning network model except the first layer encoder convolution block, to obtain a second deep learning network model; wherein the weight bit width allocation interval is an interval between 2 bits and 8 bits.

[0018] Optionally, the acquiring of a deep learning network model for single-frame infrared dim small target detection to obtain an initial deep learning network model includes:

[0019] Obtain the ResUNet network model for single-frame infrared dim target detection and obtain the initial deep learning network model.

[0020] Optionally, the symmetric progressively decreasing network quantization method for single-frame infrared dim small target detection further includes:

[0021] The target deep learning network model is tested using the NUAA-SIRST dataset.

[0022] Optionally, the symmetric progressively decreasing network quantization method for single-frame infrared dim small target detection further includes:

[0023] The network performance of the target deep learning network model is tested using the disk memory, graphics memory, FLOP, and MAC occupied when running the target deep learning network model.

[0024] It can be seen that the present application first obtains a deep learning network model for single-frame infrared dim target detection to obtain an initial deep learning network model, and then allocates a weight bit width and an activation bit width of 32 bits as a floating point number to the first-layer encoder convolution block in the initial deep learning network model to obtain a first deep learning network model, and then allocates a weight bit width to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution block in the first deep learning network model in a symmetrical and progressively decreasing manner to obtain a second deep learning network model, and then allocates an activation bit width to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution block in the second deep learning network model in a symmetrical and progressively decreasing manner to obtain a target deep learning network model, and uses the target deep learning network model to detect the single-frame infrared dim target. The present application adopts a symmetrical and progressively decreasing method to quantize and adjust the encoder convolution blocks and decoder convolution blocks of each layer in the deep learning network model for single-frame infrared dim target detection. By allocating lower weight bit widths and activation bit widths to quantization-insensitive layers, namely the deep layer and decoder sub-network, the memory usage and computational cost can be reduced, and the highly coupled feature maps in the shallow layer and encoder sub-network can be well preserved. In this way, the weak response of deep small targets is retained and the weight and activation bit width of the network are reduced, which greatly reduces the computing resources required for the single-frame infrared dim target detection network, so that it can be deployed on hardware devices with limited resources for operation, while maintaining the accuracy of infrared dim target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0026] Figure 1 A flow chart of a symmetrical progressively decreasing network quantization method for single-frame infrared dim target detection disclosed in the present application;

[0027] Figure 2 A schematic diagram of a specific deep learning network model structure disclosed in this application;

[0028] Figure 3 A flow chart of a specific symmetrical progressively decreasing network quantization method for single-frame infrared dim target detection disclosed in this application;

[0029] Figure 4 This is a schematic diagram of a specific deep learning network model structure disclosed in this application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] The present application embodiment discloses a symmetric progressively decreasing network quantization method for single-frame infrared dim target detection, see Figure 1 As shown, the method includes:

[0032] Step S11: Obtain a deep learning network model for single-frame infrared dim target detection to obtain an initial deep learning network model.

[0033] The deep learning network in the infrared small target detection scheme proposed in this application is specifically a network model for single-frame infrared small target (SIRST, Single-frame InfraRed Small Target) detection. When infrared small target detection is required, the deep learning network model to be quantized is first determined to obtain the initial deep learning network model. Among them, the initial deep learning network model includes but is not limited to ResUNet network, ACM network (ie ACM-Net, ActionContext Modeling Network), LIRDNet network, DNANet network and other models. It should be pointed out that for the quantization of the deep learning network, when a and w represent the activation and weight of the output hidden layer respectively, the goal of the network quantization in this scheme is to use a quantized low resolution (ie a low and w low ) replaces high resolution activation a high and weight w high and minimize the impact on model accuracy.

[0034] Step S12: Allocate a weight bit width and an activation bit width of 32 bits as floating point numbers to the first layer encoder convolution block in the initial deep learning network model to obtain a first deep learning network model.

[0035] In this embodiment, considering that the infrared image first passes through the first layer encoder convolution block after being input into the deep learning network model, that is, the input distance of the infrared image is closest to the first layer encoder convolution block, if the first layer encoder convolution block is over-quantized at this time, it will bring huge information loss, thereby bringing a greater negative impact to the subsequent layers. On the contrary, if the weight bit width and activation bit width of the first layer encoder convolution block are allocated with a floating point number of 32 bits, sufficient quantization space can be left for the subsequent layers, and it is ensured that the final output layer has full-precision information input to maintain the segmentation performance. Therefore, the weight space and activation space of the first layer encoder convolution block can be allocated with a floating point number of 32 bits to maintain the segmentation performance. Specifically, after obtaining the initial deep learning network model for single-frame infrared dim target detection, the first layer encoder convolution block in the above initial deep learning network model is allocated with a floating point number of 32 bits (bit) weight (weight) bit width and activation bit width (i.e., activation function bit width), such as Relu activation function (The Rectified Linear Unit, corrected linear unit) bit width, to obtain the first deep learning network model.

[0036] It should be noted that the deep learning network model used for single-frame infrared dim target detection adopts a symmetrical network structure, and the deepest layer (i.e. the layer with the largest number of layers) has only one encoder convolution block, and all layers except the innermost layer include one encoder convolution block and one decoder convolution block. For details, see Figure 2 As shown, Figure 2 A deep learning network model with a total of 5 layers is shown in the figure. Any layer in the model is represented by i and L i,0 and L 1,1 The encoder convolution block and the decoder convolution block of the i-th layer are represented, and the weight bit width of the encoder convolution block, the weight bit width of the decoder convolution block, the activation bit width of the encoder convolution block, and the activation bit width of the decoder convolution block of the i-th layer are represented as In addition, the data flow in the deep learning network model is as follows: first, the original infrared image collected by the infrared device is input into the encoder convolution block L of the first layer. 1,0 , and then the encoder convolution block L of the first layer 1,0 The first feature map output is input to the encoder convolution block L of the second layer 2,0 , and are input into the encoder convolution block of the next layer in sequence, that is, the encoder convolution block L of the second layer 2,0 The output second feature map is input to the encoder convolution block L of the third layer 3,0 , the encoder convolution block L of the third layer 3,0 The output third feature map is input to the encoder convolution block L of the 4th layer 4,0, and the encoder convolution block L of the 4th layer 4,0 The output fourth feature map is input to the encoder convolution block L of the 5th layer 5,0 , then the encoder convolution block L of the 5th layer 5,0 The output fifth feature map is sequentially input to the decoder convolution block of the next symmetrical layer, that is, the fifth feature map is input to the decoder convolution block L of the 4th layer. 4,1 , and then the decoder convolution block L of the 4th layer 4,1 The output sixth feature map is input to the decoder convolution block L of the third layer 3,1 , and then the decoder convolution block L of the third layer 3,1 The seventh feature map output is input to the decoder convolution block L of the second layer 2,1 , and finally the decoder convolution block L of the second layer 2,1 The output eighth feature map is input to the decoder convolution block L of the first layer 1,1 , thus obtaining the decoder convolution block L of the first layer 1,1 The output single frame infrared weak target segmentation image. In this embodiment, after obtaining Figure 2 After the deep learning network model in , the first layer of encoder convolution blocks in the model are assigned a weight width and activation width of 32 bits, that is,

[0037] Step S13: Allocate weight bit widths to all encoder convolution blocks and all decoder convolution blocks in the first deep learning network model except the first-layer encoder convolution block in a symmetrical and progressively decreasing manner to obtain a second deep learning network model.

[0038] In this embodiment, in order to make the shallow layer show higher quantization sensitivity than the deep layer under low bit width, the weight bit width and activation bit width of 32 bits are allocated to the first layer encoder convolution block in the initial deep learning network model. After obtaining the first deep learning network model, the weight bit width can be further allocated to all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution block in the above-mentioned first deep learning network model in a symmetrical and progressively decreasing manner to obtain a second deep learning network model. It can be understood that after step S12, the weight bit width and activation bit width of 32 bits of floating point numbers have been allocated to the first layer encoder convolution block in the initial deep learning network model, that is, the weight bit width and activation bit width in the first layer encoder convolution block have been determined, and then the bit width is allocated for the layer to which the weight bit width is not allocated in the current deep learning network model, that is, the weight bit width is allocated to all encoder convolution blocks and all decoder convolution blocks except the above-mentioned first layer encoder convolution block. It should be pointed out that in this embodiment, the weight bit width in the first deep learning network model is allocated in a manner that the weight bit width is symmetrically and gradually decreased, wherein the weight bit width is symmetrically and gradually decreased in a manner that specifically means that the shallower the layer, the larger the weight bit width allocated to it. Therefore, the weight bit width corresponding to the shallowest encoder convolution block and the decoder convolution block is the largest, and the weight bit width of the deeper encoder convolution block and the decoder convolution block gradually decreases. In addition, it should be pointed out that the above-mentioned weight bit width is symmetrically and gradually decreased in a manner that specifically means that the encoder convolution block and the corresponding decoder convolution block in the deep learning network model decrease simultaneously from the shallow layer to the deep layer, and does not describe a linear relationship of quantity. For example, according to the method of symmetrically and gradually decreasing weight bit width: Figure 2 In the deep learning network model, all encoder convolution blocks except the first layer encoder convolution block and all decoder convolution blocks are assigned weight bit widths of 32, 8, 6, 3, 2, 2, 3, 6, and 8, corresponding to

[0039] Specifically, the method of allocating weight bit widths to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution block in the first deep learning network model in a manner of symmetrically and gradually decreasing weight bit widths to obtain a second deep learning network model may include: allocating weight bit widths to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution block in the first deep learning network model in a manner of symmetrically and gradually decreasing weight bit widths according to a preset weight bit width allocation interval and based on a manner of symmetrically and gradually decreasing weight bit widths to obtain a second deep learning network model; wherein the weight bit width allocation interval is an interval between 2 bits and 8 bits. In this embodiment, weight bit widths can be allocated to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution block in the first deep learning network model in a manner of symmetrically and gradually decreasing weight bit widths according to a preset weight bit width allocation interval; wherein, the weight bit width allocation interval has been verified through experiments to be specifically an interval between 2 bits and 8 bits. That is,

[0040]

[0041] It should be noted that the last layer, i.e. the deepest layer I, contains only one encoder convolutional block, so the above equation does not include

[0042] Step S14: Allocate activation bit widths to all encoder convolution blocks and all decoder convolution blocks in the second deep learning network model except the first-layer encoder convolution block in a symmetrical and progressively decreasing manner to obtain a target deep learning network model, and use the target deep learning network model to detect the single-frame infrared dim target.

[0043] In this embodiment, after allocating weight bit widths to all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution blocks in the first deep learning network model to obtain the second deep learning network model, further, in the same symmetrical and progressive decreasing manner as the above-mentioned symmetrical and progressive decreasing manner of the weight bit width, all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution blocks in the above-mentioned second deep learning network model are allocated activation bit widths, thereby obtaining the target deep learning network model, and using the quantized target deep learning network model to detect the above-mentioned single-frame infrared dim small target. For example, in the manner of symmetrical and progressive decreasing of the activation bit width, Figure 2 In the deep learning network model, all encoder convolution blocks except the first layer encoder convolution block and all decoder convolution blocks are assigned activation bit widths of 32, 4, 4, 3, 3, 3, 4, 4, 4, corresponding to

[0044] In a specific implementation, the activation bit width is allocated to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution blocks in the second deep learning network model in a manner that the activation bit width decreases symmetrically and gradually to obtain a target deep learning network model. Specifically, it may include: allocating activation bit width to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution blocks in the second deep learning network model in a manner that the activation bit width decreases symmetrically and gradually to obtain a third deep learning network model; allocating a higher bit width to the encoder convolution blocks in the third deep learning network model than the decoder convolution blocks of the same layer to obtain the target deep learning network model. In this embodiment, after allocating activation bit width to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution blocks in the second deep learning network model in a manner that the activation bit width decreases symmetrically and gradually to obtain a higher quantization sensitivity than the decoder convolution blocks, all encoder convolution blocks may be further allocated a higher bit width than the decoder convolution blocks of the same layer, which may be specifically expressed as:

[0045]

[0046] Specifically, the method of allocating activation bit widths to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution blocks in the second deep learning network model in a manner that the activation bit width decreases symmetrically and gradually to obtain a third deep learning network model may include: allocating activation bit widths to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution blocks in the second deep learning network model in a manner that the activation bit width decreases symmetrically and gradually to obtain a third deep learning network model according to a preset activation bit width allocation interval and based on a manner that the activation bit width decreases symmetrically and gradually to obtain a third deep learning network model; wherein the activation bit width allocation interval is an interval between 3 bits and 4 bits. In this embodiment, the activation bit width can be allocated to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution blocks in the second deep learning network model in a manner that the activation bit width allocation interval is preset and based on a manner that the activation bit width decreases symmetrically and gradually; wherein, it has been verified through experiments that the activation bit width allocation interval is specifically an interval between 3 bits and 4 bits. That is,

[0047]

[0048] It should be noted that the last layer, i.e. the deepest layer I, contains only one encoder convolutional block, so the above equation does not include

[0049] Furthermore, after obtaining the target deep learning network model, it can also include: testing the network performance of the target deep learning network model using the disk memory, graphics memory, FLOP and MAC occupied when running the target deep learning network model. In the first specific implementation, the symmetric progressively decreasing network quantization method for single-frame infrared dim target detection proposed in this application is used to adjust the bit width of the ACM network, and the quantized network model is compared with the unadjusted original ACM network model. It is concluded that the running calculation of this scheme only requires 1 / 13 of the disk memory, 1 / 3.8 of the graphics memory, 1 / 14 of the FLOP (Floating point operations per second) and 1 / 14 of the MAC (Memory Access Cost, video memory / memory access).

[0050] In a second specific implementation, the LIRDNet network is bit-width quantized and adjusted using the symmetric progressively decreasing network quantization method for single-frame infrared dim target detection proposed in this application, and the quantized adjusted network model is compared with the original LIRDNet network model that has not been adjusted. It is found that the running calculation of this scheme only requires 1 / 11 of the disk memory, 1 / 3.5 of the graphics memory, 1 / 39 of the FLOPs, and 1 / 39 of the MACs.

[0051] In a second specific implementation, the DNANet network is bit-width quantized and adjusted using the symmetric progressively decreasing network quantization method for single-frame infrared dim target detection proposed in this application, and the quantized adjusted network model is compared with the original DNANet network model that has not been adjusted. It is found that the running calculation of this scheme only requires 1 / 12 of the disk memory, 1 / 5.1 of the graphics memory, 1 / 30 of the FLOP and 1 / 31 of the MAC.

[0052] It can be seen that the embodiment of the present application first obtains a deep learning network model for single-frame infrared dim target detection to obtain an initial deep learning network model, and then allocates a weight bit width and an activation bit width of 32 bits as a floating point number to the first-layer encoder convolution block in the initial deep learning network model to obtain a first deep learning network model, and then allocates a weight bit width to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution block in the first deep learning network model in a symmetrical and progressively decreasing manner to obtain a second deep learning network model, and then allocates an activation bit width to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution block in the second deep learning network model in a symmetrical and progressively decreasing manner to obtain a target deep learning network model, and uses the target deep learning network model to detect the single-frame infrared dim target. The embodiment of the present application adopts a symmetrical and progressively decreasing method to quantize and adjust the encoder convolution blocks and decoder convolution blocks of each layer in the deep learning network model for single-frame infrared dim small target detection. By allocating lower weight bit widths and activation bit widths to quantization-insensitive layers, namely the deep layer and decoder sub-network, it is possible to reduce memory usage and computational costs, and to preserve the highly coupled feature maps in the shallow layer and encoder sub-network. In this way, the weak response of deep small targets is preserved and the weight and activation bit width of the network are reduced, which greatly reduces the computing resources required for the single-frame infrared dim small target detection network, so that it can be deployed on hardware devices with limited resources for operation, while maintaining the accuracy of infrared dim small target detection.

[0053] The present application embodiment discloses a specific symmetric progressively decreasing network quantization method for single-frame infrared dim target detection, see Figure 3 As shown, the method includes:

[0054] Step S21: Obtain a ResUNet network model for single-frame infrared dim target detection to obtain an initial deep learning network model.

[0055] In this embodiment, the ResUNet network model for single-frame infrared dim target detection is first determined to obtain an initial deep learning network model. Figure 4 As shown in the figure, the ResUNet network model consists of 5 layers in total, and the ResUNet network model includes encoder (Conv+BN+ReLU), decoder (Conv+BN+ReLU), maximum pooling layer, upsampling layer and other modules; among them, Conv represents convolution kernel, BN (Batch Normalization) represents normalization, and ReLU represents activation function.

[0056] Step S22: Allocate a weight bit width and an activation bit width of 32 bits as floating point numbers to the first layer encoder convolution block in the initial deep learning network model to obtain a first deep learning network model.

[0057] Step S23: Allocate weight bit widths to all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution blocks in the first deep learning network model according to the preset weight bit width allocation interval and based on a symmetrical and gradual decrease in weight bit width, to obtain a second deep learning network model; wherein the weight bit width allocation interval is an interval between 2 bits and 8 bits.

[0058] For example, Figure 4 The weight bit width (Wbit) of the convolution blocks Conv1-0, Conv2-0, Conv3-0, Conv4-0, Conv5-0, Conv4-1, Conv3-1, Conv2-1, and Conv1-1 of the ResUNet network model are quantized to 32, 8, 6, 3, 2, 2, 3, 6, and 8, respectively.

[0059] Step S24: Allocate activation bit widths to all encoder convolution blocks and all decoder convolution blocks in the second deep learning network model except the first-layer encoder convolution blocks according to the preset activation bit width allocation interval and based on a symmetrical and progressive decrease in activation bit width to obtain a third deep learning network model; wherein the activation bit width allocation interval is an interval between 3 bits and 4 bits.

[0060] For example, Figure 4 The activation bit width (Abit) of the convolution blocks Conv1-0, Conv2-0, Conv3-0, Conv4-0, Conv5-0, Conv4-1, Conv3-1, Conv2-1, and Conv1-1 of the ResUNet network model are quantized to 32, 4, 4, 3, 3, 3, 4, 4, respectively.

[0061] Step S25: Assign a higher bit width to the encoder convolution block in the third deep learning network model than to the decoder convolution block in the same layer to obtain a target deep learning network model, and use the target deep learning network model to detect the single-frame infrared dim target.

[0062] In this embodiment, a higher bit width is allocated to the encoder convolution block in the third deep learning network model than to the decoder convolution block in the same layer. After obtaining the target ResUNet network model, the quantized target ResUNet network model is compared with the unquantized original ResUNet network model. It is found that the quantized solution only requires 1 / 13 of the disk memory, 1 / 4.6 of the graphics memory, 1 / 29 of the FLOP and 1 / 29 of the MAC to run the calculation.

[0063] Furthermore, the above-mentioned target deep learning network model is tested using an infrared image dataset, such as the NUAA-SIRST dataset. In this embodiment, the obtained target ResUNet network model is tested using the NUAA-SIRST dataset, and it is found that its network performance is well maintained. Compared with the original ResUNet network model, there is only a 3% fluctuation in IoU (the degree of overlap between the predicted box and the real box), that is, this solution maintains good network accuracy and performance.

[0064] For a more specific processing procedure of the above step S22, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0065] It can be seen that the embodiment of the present application first obtains the ResUNet network model for single-frame infrared dim target detection to obtain an initial deep learning network model, and then allocates a weight bit width and an activation bit width of 32 bits as a floating point number to the first layer encoder convolution block in the initial deep learning network model to obtain a first deep learning network model, and then allocates a weight bit width to all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution block in the first deep learning network model according to a preset weight bit width allocation interval and based on a symmetrical and progressive decrease in weight bit width to obtain a second deep learning network model, and then allocates an activation bit width to all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution block in the second deep learning network model according to a preset activation bit width allocation interval and based on a symmetrical and progressive decrease in activation bit width to obtain a third deep learning network model, and finally allocates a higher bit width to the encoder convolution blocks in the third deep learning network model than to the decoder convolution blocks of the same layer to obtain a target deep learning network model. The embodiment of the present application allocates weight bit width and activation bit width to the encoder convolution block and the decoder convolution block in the deep learning network model in a symmetrical and progressively decreasing manner. This can compress the computing resources of the deep learning network while maintaining the accuracy of infrared weak and small target detection, reduce memory usage and computing cost, and enable the network to be deployed and run on hardware devices with limited resources.

[0066] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0067] The above is a detailed introduction to the symmetric progressively decreasing network quantization method for single-frame infrared weak target detection provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A symmetric progressively decreasing network quantization method for single-frame infrared dim target detection, characterized in that: include: Obtain a deep learning network model for single-frame infrared dim target detection and obtain an initial deep learning network model; Allocating a weight bit width and an activation bit width of 32 bits as floating point numbers to a first layer encoder convolution block in the initial deep learning network model to obtain a first deep learning network model; Allocating weight bit widths to all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution block in the first deep learning network model in a symmetrical and progressively decreasing manner, to obtain a second deep learning network model; Allocating activation bit widths to all encoder convolution blocks and all decoder convolution blocks except the first-layer encoder convolution block in the second deep learning network model in a symmetrical and progressively decreasing manner to obtain a target deep learning network model, and using the target deep learning network model to detect the single-frame infrared dim small target; Among them, the method of allocating activation bit widths to all encoder convolution blocks and all decoder convolution blocks except the encoder convolution blocks of the first layer in the second deep learning network model in a manner that the activation bit widths decrease symmetrically and gradually to obtain a target deep learning network model includes: allocating activation bit widths to all encoder convolution blocks and all decoder convolution blocks except the encoder convolution blocks of the first layer in the second deep learning network model in a manner that the activation bit widths decrease symmetrically and gradually to obtain a third deep learning network model; allocating a higher bit width to the encoder convolution blocks in the third deep learning network model than that to the decoder convolution blocks of the same layer to obtain a target deep learning network model.

2. The symmetrical progressively decreasing network quantization method for single-frame infrared dim target detection according to claim 1 is characterized in that: The method allocates activation bit widths to all encoder convolution blocks and all decoder convolution blocks in the second deep learning network model except the first layer encoder convolution block in a symmetrical and progressively decreasing manner to obtain a third deep learning network model, including: According to the preset activation bit width allocation interval and based on the symmetrical and gradual decrease of the activation bit width, activation bit width is allocated to all encoder convolution blocks and all decoder convolution blocks in the second deep learning network model except the first layer encoder convolution block, to obtain a third deep learning network model; wherein the activation bit width allocation interval is an interval between 3 bits and 4 bits.

3. The symmetrical progressively decreasing network quantization method for single-frame infrared dim target detection according to claim 1 is characterized in that: The method allocates weight bit widths to all encoder convolution blocks and all decoder convolution blocks except the first layer encoder convolution block in the first deep learning network model in a symmetrical and progressively decreasing manner to obtain a second deep learning network model, including: According to the preset weight bit width allocation interval and based on the symmetrical and gradual decrease of the weight bit width, weight bit widths are allocated to all encoder convolution blocks and all decoder convolution blocks in the first deep learning network model except the first layer encoder convolution block, to obtain a second deep learning network model; wherein the weight bit width allocation interval is an interval between 2 bits and 8 bits.

4. The symmetrical progressively decreasing network quantization method for single-frame infrared dim target detection according to claim 1 is characterized in that: The step of obtaining a deep learning network model for single-frame infrared dim target detection to obtain an initial deep learning network model includes: Obtain the ResUNet network model for single-frame infrared dim target detection and obtain the initial deep learning network model.

5. The symmetrical progressively decreasing network quantization method for single-frame infrared dim target detection according to claim 1 is characterized in that: Also includes: The target deep learning network model is tested using the NUAA-SIRST dataset.

6. The symmetrical progressively decreasing network quantization method for single-frame infrared dim target detection according to any one of claims 1 to 5, characterized in that: Also includes: The network performance of the target deep learning network model is tested using the disk memory, graphics memory, FLOP, and MAC occupied when running the target deep learning network model.

Citation Information

Patent Citations

  • Target detection processing method, apparatus and device

    CN113222108A

  • Single-frame infrared small target detection method and device

    CN115690536A

  • Quantization method and device of convolutional neural network, electronic equipment and storage medium

    CN115983349A

  • Deep learning network model optimization method based on parameter quantization

    CN116524173A

  • Lightweight method and device suitable for unmanned equipment edge end deep learning model

    CN116663616A