Target detection method, device and system for night infrared image

Abstract

The invention relates to a target detection method, device and system for a night infrared image, and the method comprises the steps: inputting the night infrared image into a feature extraction network, carrying out the feature extraction, obtaining a multi-scale feature, and employing a group shuffling convolution block and a residual error classification feature enhancement block in the network, according to the method, deep feature extraction is carried out on a night infrared image, multi-scale features are input into a feature fusion network for feature fusion to obtain multi-scale fusion features, the network adopts a bidirectional feature pyramid network structure, and the multi-scale fusion features are input into a detection head network; target detection is carried out on the fusion features of different scales through the multiple detection head branches, detection results of multiple targets of different sizes are obtained, and detection of the small-size targets is achieved by adding the P2 small-target detection head branch. By adopting the method, high-precision detection of multi-size targets can be realized for the night infrared image based on the neural network with a lightweight structure.

Description

Technical Field

[0001] The present application relates to the technical field of nighttime infrared image detection, and in particular to a target detection method, device, and system for nighttime infrared images. Background Art

[0002] With the rapid development of inland and offshore shipping, how to quickly and effectively identify floating objects on the water surface and provide early warning and real-time monitoring information for areas such as water surface safety avoidance, pollutant cleaning, and water surface traffic safety has become one of the important topics in the fields of intelligent identification, informatization, and sensors.

[0003] Nighttime ship detection technology holds significant application value in areas such as shipping regulation and military defense. However, existing nighttime ship detection methods struggle to meet the accuracy and real-time requirements due to harsh conditions such as low illumination and complex sea clutter. Traditional infrared ship detection methods, while lightweight, suffer from poor robustness, thus failing to meet accuracy requirements. Traditional deep neural network-based models, while robust, suffer from large parameter sizes, making them difficult to deploy effectively on resource-constrained embedded platforms. Therefore, designing a lightweight and robust infrared ship detection method that can be deployed on embedded platforms remains a pressing research topic. This is of great significance for maritime security and early warning missions. Summary of the Invention

[0004] Based on this, it is necessary to provide a target detection method, device and system for nighttime infrared images that can achieve lightweight and high detection accuracy in order to address the above technical problems.

[0005] A target detection method for nighttime infrared images, the method comprising: Acquire nighttime infrared images for target detection; Inputting the nighttime infrared image into a feature extraction network to perform feature extraction to obtain multi-scale features, wherein the feature extraction network utilizes a group shuffled convolution block and a residual classification feature enhancement block to achieve deep feature extraction of the nighttime infrared image; Inputting the multi-scale features into a feature fusion network to perform feature fusion to obtain multi-scale fusion features, wherein the feature fusion network adopts a bidirectional feature pyramid network structure to perform bidirectional transmission feature fusion on the multi-scale features; The multi-scale fusion features are input into the detection head network, and multiple detection head branches are used to perform target detection on the fusion features of different scales to obtain detection results of multiple targets of different sizes. Among them, the P2 small target detection head branch is added to the detection head network to realize the detection of small-size targets.

[0006] In one embodiment, the group of shuffled convolution blocks includes a convolution layer, a DSC branching layer, a channel splicing layer, and a channel shuffling layer; The input data is subjected to channel compression processing by the convolution layer to obtain channel-compressed features; The features after channel compression are passed through the DSC branch layer and lightweight processed by depth convolution and point convolution to obtain focused features; The focused features and the channel-compressed features are channel-joined by the channel joining layer, and then pass through the channel shuffling layer to shuffle the channel order, thereby obtaining the output data of the group of shuffled convolution blocks.

[0007] In one embodiment, the residual classification feature enhancement block includes a channel compression layer, a cascaded convolutional splicing structure, a channel splicing fusion layer, a residual connection, and a SiLU activation layer; After the input data passes through the channel compression layer, it is input into the cascaded convolutional splicing structure for feature extraction. In the cascaded convolutional splicing structure, the output features of each convolution layer serve as the input of the next convolution layer; The output features of each convolution layer and the features after the channel compression layer are fused through the channel splicing and fusion layer, and then the channels are adjusted through the convolution layer. If the adjusted features are consistent with the dimension of the input data, the residual connection is triggered, and the adjusted features are spliced ​​with the input data to obtain spliced ​​features; The splicing features pass through the SiLU activation layer to obtain the output data of the residual classification feature enhancement block.

[0008] In one embodiment, the feature extraction network includes four feature extraction units connected in sequence, wherein the first and second feature extraction units are both composed of ordinary convolution blocks and C2f blocks, and are used to extract shallow features, and the third and fourth feature extraction units are both composed of the group of shuffled convolution blocks and residual classification feature enhancement blocks, and are used to extract deep features.

[0009] In one embodiment, in the feature fusion network, the feature concatenation layer is used as the feature fusion core, and the upsampling layer, the residual classification feature enhancement block, and the group shuffle convolution block are combined to extract and optimize features, thereby realizing the fusion of deep features and shallow features, and gradually obtaining the multi-scale fusion features; Among them, the shallow fusion features are input into the P2 small target detection head branch to realize the detection of small-sized targets.

[0010] In one embodiment, a nighttime infrared image target detection network is constructed based on the feature extraction network, the feature fusion network, and the detection head network, and a nighttime infrared image to be subjected to target detection is input into the nighttime infrared image target detection network to achieve multi-target detection; When training the nighttime infrared image target detection network, a mixed precision training method is adopted.

[0011] The present application also provides a target detection device for nighttime infrared images, the device comprising: The target image acquisition module to be detected is used to acquire the night infrared image to be detected; a multi-scale feature extraction module, configured to input the nighttime infrared image into a feature extraction network to perform feature extraction to obtain multi-scale features, wherein the feature extraction network utilizes a group shuffled convolution block and a residual classification feature enhancement block to achieve deep feature extraction of the nighttime infrared image; A multi-scale feature fusion module is used to input the multi-scale features into a feature fusion network to perform feature fusion to obtain multi-scale fusion features. The feature fusion network adopts a bidirectional feature pyramid network structure to perform bidirectional feature fusion on the multi-scale features. The multi-target detection module is used to input the multi-scale fusion features into the detection head network, and use multiple detection head branches to perform target detection on the fusion features of different scales to obtain detection results of multiple targets of different sizes. Among them, the P2 small target detection head branch is added to the detection head network to realize the detection of small-sized targets.

[0012] The present application also provides a target detection system for nighttime infrared images, wherein the system includes an RK3588 chip as a target detection chip, in which multi-target detection is performed on the received nighttime infrared images according to the above-mentioned target detection method for nighttime infrared images.

[0013] In one embodiment, a nighttime infrared image target detection network is constructed based on a feature extraction network, a feature fusion network, and a detection head network; Convert the trained nighttime infrared image target detection network model pt file format to ONNX format; A symmetric quantization method is used to convert the model from FP32 precision to INT8 precision to obtain a nighttime infrared image target detection network in RKNN format, and the nighttime infrared image target detection network in RKNN format is embedded into the target detection chip.

[0014] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned target detection method for nighttime infrared images when executing the computer program.

[0015] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the target detection method for nighttime infrared images.

[0016] The above-mentioned target detection method, device, and system for nighttime infrared imagery inputs nighttime infrared images into a feature extraction network for feature extraction to obtain multi-scale features. In this network, group shuffled convolution blocks and residual classification feature enhancement blocks are used to achieve deep feature extraction of nighttime infrared images. The multi-scale features are then input into a feature fusion network for feature fusion to obtain multi-scale fused features. This network adopts a bidirectional feature pyramid network structure, inputs the multi-scale fused features into a detection head network, and utilizes multiple detection head branches to perform target detection based on the fused features of different scales, obtaining detection results for multiple targets of different sizes. The addition of a P2 small target detection head branch enables the detection of small-sized targets. This method can achieve high-precision detection of multi-sized targets in nighttime infrared images based on a lightweight neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 1 is a flow chart of a method for detecting targets in nighttime infrared images according to an embodiment; Figure 2 Schematic diagram of the structure of a group shuffle convolution block in one embodiment; Figure 3 Schematic diagram of the structure of the residual splitting feature enhancement block in one embodiment; Figure 4 Schematic diagram of the structure of the BiFPN fusion P2 small target detection head branch in one embodiment; Figure 5 A schematic diagram of the structure of the nighttime infrared image target detection network GRBP-YOLO in one embodiment; Figure 6 A schematic diagram of a thread pool multi-threaded reasoning process in one embodiment; Figure 7 is a structural block diagram of a target detection device for nighttime infrared images in one embodiment; Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0019] In view of the problems existing in nighttime infrared image detection, such as small target detection difficulty, complex background, and limited computing resources, in this application, Figure 1 As shown in FIG, a target detection method for nighttime infrared images is provided, which specifically includes the following steps: Step S100: Acquire a nighttime infrared image for target detection.

[0020] In step S110, the nighttime infrared image is input into a feature extraction network to perform feature extraction to obtain multi-scale features. In the feature extraction network, a group shuffled convolution block and a residual classification feature enhancement block are used to achieve deep feature extraction of the nighttime infrared image.

[0021] In step S120 , the multi-scale features are input into a feature fusion network to perform feature fusion to obtain multi-scale fusion features. The feature fusion network adopts a bidirectional feature pyramid network structure to perform bidirectional feature fusion on the multi-scale features.

[0022] In step S130, the multi-scale fusion features are input into the detection head network, and multiple detection head branches are used to perform target detection on the fusion features of different scales to obtain detection results of multiple targets of different sizes. Among them, the P2 small target detection head branch is added to the detection head network to realize the detection of small-sized targets.

[0023] In this embodiment, multi-target detection in nighttime infrared images is achieved through the use of a neural network. In this neural network, the traditional convolution block is replaced by a group shuffled convolution (GSConv) block. Channel grouping and feature rearrangement mechanisms are used to balance computational efficiency and representational capability, significantly reducing model complexity while maintaining detection accuracy. Secondly, a residual split feature enhancement block (RS_C2f) is proposed. Through the collaborative design of residual connections and split stacking structures, while inheriting the lightweight advantages of the C2f block, it strengthens cross-level feature interaction capabilities and further reduces parameter redundancy. Finally, a bidirectional feature pyramid network (BiFPN) is constructed and integrated with the P2 layer small target detection head. This enhances the performance of small target detection through a cross-scale feature recalibration mechanism, effectively addressing the problem of missed detection of small distant ship targets due to low contrast and insufficient resolution.

[0024] In step S100, the method is for target detection in nighttime infrared images. At the same time, the method can also be used for target detection in a more specific scene. The scene in the nighttime target image is a water surface such as the sea surface or an inland river. Ships, floating objects or waterside buildings on the sea surface or river surface are detected as targets. Since these scenes are relatively open and the sizes of the targets themselves are quite different, target detection is difficult. The method is specifically designed for such scenes and has achieved good detection results.

[0025] In step S110, the group shuffle convolution block (GSConv) structure in the feature extraction unit is as follows: Figure 2 As shown, it includes a convolution layer, a DSC branch layer, a channel splicing layer and a channel shuffling layer. The input data is subjected to channel compression processing by the convolution layer to obtain channel-compressed features. The channel-compressed features are subjected to the DSC branch layer and lightweight processing by depth convolution and point convolution to obtain focused features. The focused features and the channel-compressed features are subjected to channel splicing by the channel splicing layer, and then passed through the channel shuffling layer to shuffle the channel order to obtain the output data of the group shuffled convolution block.

[0026] In this embodiment, the proposed group shuffle convolution block uses significantly fewer floating-point operations than standard convolution and depthwise separable convolution with equivalent output channels, significantly reducing model inference latency and resource usage. Compared to standard convolution with equivalent output channels, the floating-point operations of GSConv convolution can be reduced to less than half. This design enables GSConv to achieve efficient inference while maintaining diverse feature representation, meeting the needs of real-time maritime detection.

[0027] In this embodiment, the structure of the residual classification feature enhancement block (RS_C2f) in the feature extraction unit is as follows: Figure 3 As shown in the figure, it includes a channel compression layer, a cascaded convolutional splicing structure, a channel splicing fusion layer, a residual connection, and a SiLU activation layer. After the input data passes through the channel compression layer, it is input into the cascaded convolutional splicing structure for feature extraction. In the cascaded convolutional splicing structure, the output features of each convolution layer serve as the input of the next convolution layer. The output features of each convolution layer are fused with the features after the channel compression layer through the channel splicing fusion layer, and then the channels are adjusted through the convolution layer. If the adjusted features are consistent with the dimension of the input data, the residual connection is triggered, and the adjusted features are spliced ​​with the input data to obtain the spliced ​​features. The spliced ​​features pass through the SiLU activation layer to obtain the output data of the residual classification feature enhancement block.

[0028] In this embodiment, to balance model lightweighting and feature expression capabilities, an improved feature fusion structure, Residual Split Cross-Stage Partial Fusion (RS_C2f), is proposed. RS_C2f builds on traditional C2f by innovatively introducing cascaded convolutional concatenation and residual connections, aiming to enhance multi-scale feature expression and contextual semantic modeling.

[0029] Specifically, in the cascaded convolutional splicing structure, 1×1 convolution is first used for channel compression, and then multiple 3×3 convolutional units are cascaded. The output of each convolution layer is spliced ​​with the result of the previous layer in the channel dimension to form a multi-level receptive field aggregation structure, realizing the fusion of local details and high-level semantics, significantly improving spatial perception and semantic extraction capabilities.

[0030] Specifically, the addition of the residual connection mechanism enables the introduction of shortcut connections under the condition that the input and output data dimensions are consistent, realizes direct feature superposition, effectively alleviates gradient vanishing and network degradation, stabilizes deep information transmission, and accelerates training convergence.

[0031] Furthermore, the residual split feature enhancement block, RS_C2f, enhances feature expression while controlling structural complexity and parameter count through channel compression and convolution reuse. It uses convolution and SiLU activation functions from the standard PyTorch API, ensuring good compatibility with Automatic Mixed Precision Training (AMP), facilitating efficient deployment in resource-constrained environments such as embedded platforms.

[0032] In this embodiment, the feature extraction network includes four feature extraction units connected in sequence, wherein the first and second feature extraction units are both composed of ordinary convolution blocks and C2f blocks, which are used to extract shallow features, and the third and fourth feature extraction units are both composed of group shuffled convolution blocks and residual classification feature enhancement blocks, which are used to extract deep features.

[0033] Specifically, in the feature extraction network, a convolutional layer is used to process the input data, namely the nighttime infrared image. The image then passes through the first feature extraction unit, where the C2f block outputs the first shallow feature. The first shallow feature passes through the second feature extraction unit, where the C2f block outputs the second shallow feature. The second shallow feature then passes through the third feature extraction unit, where the residual classification feature enhancement block outputs the first deep feature. The first deep feature passes through the fourth feature extraction unit and then passes through an SPPF (Spatial Pyramid Pooling Fast) block to obtain the second deep feature.

[0034] In step S120, in the feature fusion network, the feature concatenation layer is used as the feature fusion core, and the upsampling layer, the residual classification feature enhancement block, and the group shuffle convolution block are combined to extract and optimize features, realize the fusion of deep features and shallow features, and gradually obtain multi-scale fusion features.

[0035] Specifically, in the feature fusion network, the second deep feature passes through the convolution layer (Conv) to obtain the first intermediate feature. After upsampling (Upsample), the first intermediate feature is spliced ​​and fused with the first deep feature through the feature splicing layer (BiFPN_Concat). It is then enhanced and optimized by the residual classification feature enhancement block (RS_C2f) and the convolution layer (Conv) to obtain the second intermediate feature. After upsampling (Upsample), the second intermediate feature is spliced ​​and fused with the second shallow feature through the feature splicing layer (BiFPN_Concat). It is then enhanced and optimized by the residual classification feature enhancement block (RS_C2f) and the convolution layer to obtain the third intermediate feature. After upsampling (Upsample), the third intermediate feature is spliced ​​and fused with the first shallow feature through the feature splicing layer (BiFPN_Concat) to obtain the first shallow fused feature, which is then input into the detection head network.

[0036] Furthermore, after passing through the convolution layer (Conv), the first shallow fusion feature is spliced ​​and fused with the third intermediate feature through the feature splicing layer (BiFPN_Concat), and then passed through the residual classification feature enhancement block to obtain the second shallow fusion feature, and the shallow fusion feature is input into the detection head network.

[0037] Furthermore, the second shallow fusion feature passes through the group shuffle convolution block (GSConv), is spliced ​​and fused with the second intermediate feature through the feature splicing layer (BiFPN_Concat), and then passes through the residual classification feature enhancement block to obtain the first deep fusion feature, and the deep fusion feature is input into the detection head network.

[0038] Furthermore, the first deep fusion feature passes through the group shuffle convolution block (GSConv), is spliced ​​and fused with the first intermediate feature through the feature splicing layer (BiFPN_Concat), and then passes through the residual classification feature enhancement block to obtain the second deep fusion feature, and the deep fusion feature is input into the detection head network.

[0039] In step S130, the detection head network includes multiple detection head branches corresponding to the multi-scale fusion features, wherein the first shallow fusion feature is input to the P2 small object detection head branch to detect small-sized objects.

[0040] In this embodiment, the P2 small target detection head branches to a high-resolution layer (160×160) to identify small targets with a size of 4×4 pixels or more. Figure 4 Figure 1 shows the structure of the BiFPN-fused P2 small object detection layer. To achieve efficient multi-scale fusion of the P2 small object detection head with other detection heads, a bidirectional feature pyramid network (BiFPN) is introduced into the feature fusion network. BiFPN utilizes a weighted fusion mechanism and learnable feature selection to achieve bidirectional cross-layer information transfer from P2 to P5. On the one hand, the shallow P2 features are expanded in receptive field and enriched with contextual information by BiFPN, effectively improving the perception of extremely small objects. On the other hand, deep, highly semantic features are integrated with shallow details to further optimize the localization of large object boundaries. This bidirectional fusion mechanism achieves the complementarity of semantic and detail information, significantly improving the performance of object detection at all scales.

[0041] Furthermore, in addition to the P2 small target detection head branch, the detection head network also includes P3-P5 target detection branches to detect targets of different sizes using the second shallow fusion features, the first deep fusion features, and the second deep fusion features, respectively.

[0042] In this embodiment, a nighttime infrared image target detection network is constructed based on a feature extraction network, a feature fusion network, and a detection head network. Nighttime infrared images to be detected are input into the nighttime infrared image target detection network to achieve multi-target detection. When training the nighttime infrared image target detection network, a mixed precision training (AMP) method is adopted to accelerate convergence, reduce video memory usage, and improve model training efficiency.

[0043] Specifically, when training the nighttime infrared image target detection network, the training sample image size was processed to 640x640, the batch size was set to 16, and the nighttime infrared image target detection network was trained for 200 epochs to avoid the problem of poor generalization ability caused by underfitting or overfitting.

[0044] like Figure 5 The figure shows the structural diagram of the night infrared image target detection network.

[0045] In this paper, a simulation comparison experiment is conducted between this method and other mainstream models to prove the effectiveness of this method. The results are shown in Table 1.

[0046] Table 1 Experimental data table comparing the improved algorithm with other mainstream algorithms

[0047] In the aforementioned target detection method for nighttime infrared images, multi-target detection is achieved through the use of a neural network. In this neural network, the traditional convolutional block is replaced by a group shuffled convolution (GSConv) block. Channel grouping and feature reordering are used to balance computational efficiency and representational power, significantly reducing model complexity while maintaining detection accuracy. Secondly, a residual split feature enhancement block (RS_C2f) is proposed. Through the collaborative design of residual connections and split stacking structures, while inheriting the lightweight advantages of the C2f block, it strengthens cross-level feature interaction and further reduces parameter redundancy. Finally, a bidirectional feature pyramid network (BiFPN) is constructed and integrated with the P2-layer small target detection head. This enhances the performance of small target detection through a cross-scale feature recalibration mechanism, effectively addressing the problem of missed detection of small distant ship targets due to low contrast and insufficient resolution.

[0048] This method achieves significant performance improvements in infrared maritime ship detection. Experimental results show that the proposed nighttime infrared image target detection network improves the mean average detection precision (mAP@0.5) and recall by 1.5% and 1.9%, respectively, on the infrared Seaships dataset, reaching 92.5% and 88%, respectively. The improvement in small target detection is particularly significant. Furthermore, by introducing BiFPN_Concat and an additional 160×160 detection head, the recall of small targets is significantly improved, effectively reducing missed detections. In terms of computational efficiency, the model parameter count is reduced by 43.19% to 1.71MB, resulting in a file size of only 3.9MB. This ensures real-time detection and inference for deployment on resource-constrained embedded devices, such as the RK3588 platform, while meeting high-precision detection requirements and enabling efficient deployment. Overall, this method provides a high-precision and high-performance infrared maritime ship detection solution that is adaptable to complex marine environments and provides strong technical support for maritime safety monitoring.

[0049] In this application, a target detection system for nighttime infrared images is also proposed. In this system, the RK3588 chip is used as the target detection chip. In the target detection chip, multi-target detection is performed on the received nighttime infrared images according to the above-mentioned target detection method for nighttime infrared images.

[0050] In this embodiment, a nighttime infrared image target detection network is constructed according to the feature extraction network, feature fusion network, and detection head network in the above method, and the trained nighttime infrared image target detection network is converted into ONNX format and post-quantized to obtain a quantized model. A symmetric quantization method is used to convert the model from FP32 precision to INT8 precision to obtain a nighttime infrared image target detection network in RKNN format, and the nighttime infrared image target detection network in RKNN format is embedded into the target detection chip.

[0051] It should be noted here that, according to the chip model used in the target detection system of night infrared images, the format of the night infrared image target detection network can be adapted so that it can be embedded in the corresponding image post-processing chip.

[0052] Specifically, the trained night infrared image target detection network file in pt format is converted to onnx format, and then the RKNN toolkit RKNN Toolkit is installed in the virtual machine. The API named rknn.export is used to export the model file in rknn format. The float16 quantization model is used during export for subsequent target prediction.

[0053] Furthermore, the quantized model is converted into RKNN format and deployed on an embedded platform (the RK3588 development board is used for this model deployment).

[0054] Write pre-processing and post-processing code to optimize image normalization, feature decoding, and NMS (non-maximum suppression) to improve inference efficiency and real-time performance.

[0055] By calling model reasoning through the API interface, the real-time detection function of the infrared ship detection system is realized.

[0056] Specific steps: Step 1: First install the Debian system for the RK3588 development board, and then install the rknn_lite2 package, which is used to call various rknn interfaces in the future. Step 2: Create an RKNN object to initialize the RKNN SDK environment Step 3: Call the load_rknn interface to import the RKNN model Step 4: Call the init_runtime interface to initialize the runtime environment Step 5: Call the inference interface to infer the input and obtain the inference results Step 6: Call the release interface to release the RKNN object.

[0057] In the target detection system for nighttime infrared images, through model quantization, RKNN conversion, pre-processing, NPU reasoning, and post-processing optimization, thread pool multi-threading processing is adopted on the embedded platform NPU chip, such as Figure 6 As shown in the figure, efficient real-time reasoning is achieved, output results are continuously read from the input queue, detection boxes with higher confidence are selected to mark the frames on the original image, and the reasoning results are displayed in real time.

[0058] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0059] In one embodiment, Figure 7 As shown, a target detection device for nighttime infrared images is provided, comprising: a target image acquisition module 200 to be detected, a multi-scale feature extraction module 210, a multi-scale feature fusion module 220 and a multi-target detection module 230, wherein: The target image acquisition module 200 is used to acquire a nighttime infrared image for target detection. A multi-scale feature extraction module 210 is configured to input the nighttime infrared image into a feature extraction network to extract multi-scale features, wherein the feature extraction network utilizes a group shuffled convolution block and a residual classification feature enhancement block to achieve deep feature extraction of the nighttime infrared image; A multi-scale feature fusion module 220 is used to input the multi-scale features into a feature fusion network to perform feature fusion to obtain multi-scale fusion features. The feature fusion network adopts a bidirectional feature pyramid network structure to perform bidirectional feature fusion on the multi-scale features. The multi-target detection module 230 is used to input the multi-scale fusion features into the detection head network, and use multiple detection head branches to perform target detection on the fusion features of different scales to obtain detection results of multiple targets of different sizes. Among them, the P2 small target detection head branch is added to the detection head network to realize the detection of small-sized targets.

[0060] The specific limitations of the target detection device for nighttime infrared images can be found in the limitations of the target detection method for nighttime infrared images described above and will not be further elaborated here. Each module in the aforementioned target detection device for nighttime infrared images can be implemented in whole or in part via software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0061] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a target detection method for nighttime infrared images is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0062] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0063] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented: Acquire nighttime infrared images for target detection; Inputting the nighttime infrared image into a feature extraction network to perform feature extraction to obtain multi-scale features, wherein the feature extraction network utilizes a group shuffled convolution block and a residual classification feature enhancement block to achieve deep feature extraction of the nighttime infrared image; Inputting the multi-scale features into a feature fusion network to perform feature fusion to obtain multi-scale fusion features, wherein the feature fusion network adopts a bidirectional feature pyramid network structure to perform bidirectional transmission feature fusion on the multi-scale features; The multi-scale fusion features are input into the detection head network, and multiple detection head branches are used to perform target detection on the fusion features of different scales to obtain detection results of multiple targets of different sizes. Among them, the P2 small target detection head branch is added to the detection head network to realize the detection of small-size targets.

[0064] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: Acquire nighttime infrared images for target detection; Inputting the nighttime infrared image into a feature extraction network to perform feature extraction to obtain multi-scale features, wherein the feature extraction network utilizes a group shuffled convolution block and a residual classification feature enhancement block to achieve deep feature extraction of the nighttime infrared image; Inputting the multi-scale features into a feature fusion network to perform feature fusion to obtain multi-scale fusion features, wherein the feature fusion network adopts a bidirectional feature pyramid network structure to perform bidirectional transmission feature fusion on the multi-scale features; The multi-scale fusion features are input into the detection head network, and multiple detection head branches are used to perform target detection on the fusion features of different scales to obtain detection results of multiple targets of different sizes. Among them, the P2 small target detection head branch is added to the detection head network to realize the detection of small-size targets.

[0065] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0066] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A target detection method for nighttime infrared images, characterized in that: The method comprises: Acquire nighttime infrared images for target detection; Inputting the nighttime infrared image into a feature extraction network to perform feature extraction to obtain multi-scale features, wherein the feature extraction network utilizes a group shuffled convolution block and a residual classification feature enhancement block to achieve deep feature extraction of the nighttime infrared image; Inputting the multi-scale features into a feature fusion network to perform feature fusion to obtain multi-scale fusion features, wherein the feature fusion network adopts a bidirectional feature pyramid network structure to perform bidirectional transmission feature fusion on the multi-scale features; The multi-scale fusion features are input into the detection head network, and multiple detection head branches are used to perform target detection on the fusion features of different scales to obtain detection results of multiple targets of different sizes. Among them, the P2 small target detection head branch is added to the detection head network to realize the detection of small-size targets.

2. The target detection method for nighttime infrared images according to claim 1, characterized in that: The group shuffled convolution block includes a convolution layer, a DSC branch layer, a channel splicing layer and a channel shuffling layer; The input data is subjected to channel compression processing by the convolution layer to obtain channel-compressed features; The features after channel compression are passed through the DSC branch layer and lightweight processed by depth convolution and point convolution to obtain focused features; The focused features and the channel-compressed features are channel-joined by the channel joining layer, and then pass through the channel shuffling layer to shuffle the channel order, thereby obtaining the output data of the group of shuffled convolution blocks.

3. The target detection method for nighttime infrared images according to claim 2, characterized in that: The residual classification feature enhancement block includes a channel compression layer, a cascaded convolutional splicing structure, a channel splicing fusion layer, a residual connection and a SiLU activation layer; After the input data passes through the channel compression layer, it is input into the cascaded convolutional splicing structure for feature extraction. In the cascaded convolutional splicing structure, the output features of each convolution layer serve as the input of the next convolution layer; The output features of each convolution layer and the features after the channel compression layer are fused through the channel splicing and fusion layer, and then the channels are adjusted through the convolution layer. If the adjusted features are consistent with the dimension of the input data, the residual connection is triggered, and the adjusted features are spliced ​​with the input data to obtain spliced ​​features; The splicing features pass through the SiLU activation layer to obtain the output data of the residual classification feature enhancement block.

4. The target detection method for nighttime infrared images according to any one of claims 1 to 3, characterized in that: The feature extraction network includes four feature extraction units connected in sequence, wherein the first and second feature extraction units are both composed of ordinary convolution blocks and C2f blocks, and are used to extract shallow features; the third and fourth feature extraction units are both composed of the group of shuffled convolution blocks and residual classification feature enhancement blocks, and are used to extract deep features.

5. The target detection method for nighttime infrared images according to claim 4, characterized in that: In the feature fusion network, the feature splicing layer is used as the feature fusion core, and the upsampling layer, the residual classification feature enhancement block and the group shuffle convolution block are combined to extract and optimize features, thereby realizing the fusion of deep features and shallow features, and gradually obtaining the multi-scale fusion features; Among them, the shallow fusion features are input into the P2 small target detection head branch to realize the detection of small-sized targets.

6. The target detection method for nighttime infrared images according to claim 5, characterized in that: A nighttime infrared image target detection network is constructed based on the feature extraction network, the feature fusion network, and the detection head network, and a nighttime infrared image to be subjected to target detection is input into the nighttime infrared image target detection network to achieve multi-target detection; When training the nighttime infrared image target detection network, a mixed precision training method is adopted.

7. A target detection device for nighttime infrared images, characterized in that: The device comprises: The target image acquisition module to be detected is used to acquire the night infrared image to be detected; a multi-scale feature extraction module, configured to input the nighttime infrared image into a feature extraction network to perform feature extraction to obtain multi-scale features, wherein the feature extraction network utilizes a group shuffled convolution block and a residual classification feature enhancement block to achieve deep feature extraction of the nighttime infrared image; A multi-scale feature fusion module is used to input the multi-scale features into a feature fusion network to perform feature fusion to obtain multi-scale fusion features. The feature fusion network adopts a bidirectional feature pyramid network structure to perform bidirectional feature fusion on the multi-scale features. The multi-target detection module is used to input the multi-scale fusion features into the detection head network, and use multiple detection head branches to perform target detection on the fusion features of different scales to obtain detection results of multiple targets of different sizes. Among them, the P2 small target detection head branch is added to the detection head network to realize the detection of small-sized targets.

8. A target detection system for nighttime infrared images, characterized in that: The system includes an RK3588 chip as a target detection chip, in which the target detection method for nighttime infrared images according to any one of claims 1 to 6 is used to perform multi-target detection on the received nighttime infrared images.

9. The target detection system for nighttime infrared images according to claim 8, characterized in that: Constructing a nighttime infrared image target detection network based on the feature extraction network, feature fusion network and detection head network; Convert the trained nighttime infrared image target detection network model pt file format to ONNX format; A symmetric quantization method is used to convert the model from FP32 precision to INT8 precision to obtain a nighttime infrared image target detection network in RKNN format, and the nighttime infrared image target detection network in RKNN format is embedded into the target detection chip.

Images