Target recognition method, device and product based on infrared image colorization

Abstract

The application discloses a target recognition method and device based on infrared image colorization, and relates to the technical field of infrared image target recognition. The method comprises the following steps: acquiring a single-channel infrared image; inputting the preprocessed infrared image into a trained infrared colorization detection model to output an infrared target recognition result, wherein the infrared colorization detection model is obtained by jointly optimizing and training an infrared colorization network and a target detection network. Thus, the single-channel infrared image is adaptively colorized and reconstructed, the image feature dimension is effectively expanded, and the target feature expression capability is strengthened; through multi-scale feature fusion and infrared gradient characteristic constraint, the complete structure and the detail features of the target are retained, and the feature distinguishability of different targets is improved; through double-network joint optimization constraint, the colorization effect and the target recognition accuracy are optimized in linkage, the misrecognition and the missed detection problems in a complex scene are improved, and the accuracy and the robustness of the infrared target recognition are improved.

Description

Technical Field

[0001] This application relates to the field of infrared image target recognition technology, and in particular to a target recognition method, device and product based on infrared image colorization. Background Technology

[0002] With the increasing demand for all-weather visual recognition in fields such as security monitoring, industrial inspection, and autonomous driving, target recognition technology based on infrared images has been widely applied.

[0003] To achieve accurate target recognition in complex environments, the industry currently employs several infrared target detection and recognition methods. However, the mainstream solutions in the existing technologies mostly use direct detection of single-channel infrared images, fixed-rule pseudo-color mapping, simple image enhancement, or single-model feature extraction. These methods have limited feature dimensions and cannot fully explore the deep details and differentiated features of infrared images. Consequently, the false detection and false negative rates are extremely high in complex scenarios, resulting in insufficient recognition accuracy and robustness. Summary of the Invention

[0004] The main purpose of this application is to provide a target recognition method, device and product based on infrared image colorization, which aims to solve the problems of low image information dimension, weak feature expression ability, high false detection and false detection rate and low overall recognition accuracy in existing infrared target recognition schemes in complex scenarios.

[0005] To achieve the above objectives, this application proposes a target recognition method based on infrared image colorization, the method comprising: Acquire single-channel infrared images; The single-channel infrared image is input into the trained infrared colorization detection model, and the infrared target recognition result is output. The infrared colorization detection model is obtained by joint optimization training of the infrared colorization network and the target detection network.

[0006] In one embodiment, after acquiring the single-channel infrared image, the process includes: The single-channel infrared image is normalized, denoised, and enhanced to obtain training sample images; An infrared colorization network is constructed, and feature extraction and color reconstruction are performed on training sample images through the infrared colorization network to generate color infrared images; The colorized infrared image is input into the target detection network, and the target detection result is output. Construct a total loss function that includes a detection loss function, a colorization reconstruction loss function, and a detection-guided consistency loss function; Substitute the target detection result into the total loss function to calculate the total loss value; Based on the total loss value, backpropagation error is performed, and the parameters of the infrared colorization network and the target detection network are updated synchronously and iteratively to complete the joint constraint optimization training of the two networks. When the dual-network joint constraint optimization training iteration reaches the convergence condition, parameter updates are stopped, and the trained infrared colorization detection model is obtained.

[0007] In one embodiment, the construction of the infrared colorization network, and the extraction of features and color reconstruction of training sample images through the infrared colorization network to generate a colorized infrared image, includes: An infrared colorization network is constructed, wherein the infrared colorization network is an encoder-decoder structure; The overall structural features of the image are extracted by the encoder to obtain the image structural feature map; The image structure feature map is upsampled and feature restored by the decoder to recover image details and spatial information, and a color infrared image is reconstructed.

[0008] In one embodiment, the encoder integrates a multi-scale feature fusion module and an infrared gradient characteristic guidance module. After extracting the overall structural features of the image through the encoder to obtain the image structural feature map, the process includes: The multi-scale feature fusion module sets up multiple convolutional branches with different receptive fields to extract features from the image's basic structural feature map. The features extracted by the convolutional branches are then weighted and fused using self-learning weights to optimize the image's basic structural feature map. The infrared gradient characteristic guidance module extracts the edge gradient information of the optimized image basic structure feature map, and constrains the target contour in the optimized basic structure feature image.

[0009] In one embodiment, inputting the colorized infrared image into the target detection network and outputting the target detection result includes: Choose any one of the following detection networks as the object detection network: YOLO series, RT-DETR, or Faster R-CNN. The colorized infrared image is fed into the target detection network for forward inference calculation, and the category information, confidence score, target bounding box coordinates and semantic features corresponding to each target to be identified in the colorized infrared image are output, and the target detection result is output.

[0010] In one embodiment, substituting the target detection result into the total loss function to calculate the total loss value includes: Based on the category information, confidence score, and the recognition results of the target bounding box coordinates, and the real labeled data, the detection loss function in the total loss function is used to calculate the target classification loss, bounding box regression loss, and target confidence loss, thus obtaining the detection loss value. The pixel and gradient information of the original infrared image and the colorized infrared image are substituted into the colorization reconstruction loss function in the total loss function to calculate the overall pixel difference and edge gradient difference between the generated color image and the reference image, and the colorization reconstruction loss value is obtained. The semantic features corresponding to the original infrared image and the colorized infrared image are substituted into the detection guidance consistency loss function in the total loss function to calculate the difference in semantic features extracted by the target detection network under the two input methods, and the detection guidance consistency loss value is obtained. Based on the preset weight coefficients corresponding to various types of losses, the detection loss value, color reconstruction loss value, and detection guidance consistency loss value are weighted and accumulated to obtain the total loss value corresponding to the current iteration.

[0011] In one embodiment, the step of synchronously iteratively updating the parameters of the infrared colorization network and the target detection network based on the backpropagation error of the total loss value to complete the joint constraint optimization training of the two networks includes: Based on the total loss value, back gradient propagation is performed to sequentially solve the parameter gradients of the infrared colorization network and the target detection network; The weights and bias parameters of the two networks are updated synchronously according to the parameter gradient until the total loss value tends to stabilize. Through multiple rounds of forward inference, loss calculation, and reverse update iterative loops, the joint constraint optimization training of the infrared colorization network and the target detection network is achieved.

[0012] In one embodiment, stopping parameter updates and obtaining the trained infrared colorization detection model when the dual-network joint constraint optimization training iteration reaches the convergence condition includes: When the dual-network joint constraint optimization training satisfies any convergence condition, the training is determined to be converged and the network parameter update is stopped. The convergence condition includes: after multiple iterations, the total loss function value tends to stabilize and no longer decreases, or the number of training iterations reaches the preset maximum number of training iterations. When the convergence condition is met, the training converges and all internal parameters of the infrared colorization network and the target detection network are locked, finally obtaining the trained infrared colorization detection model.

[0013] Furthermore, to achieve the above objectives, this application also proposes a target recognition device based on infrared image colorization, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the target recognition method based on infrared image colorization as described above.

[0014] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the target recognition method based on infrared image colorization as described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: acquiring a single-channel infrared image; inputting the preprocessed infrared image into a trained infrared colorization detection model, and outputting an infrared target recognition result, wherein the infrared colorization detection model is obtained by joint optimization training of an infrared colorization network and a target detection network. Thus, by adaptively reconstructing the single-channel infrared image with colorization, the image feature dimension is effectively expanded, and the target feature expression capability is strengthened; through multi-scale feature fusion and infrared gradient characteristic constraints, the complete structure and detailed features of the target are preserved, improving the feature discrimination of different targets; through joint optimization constraints of the two networks, the colorization effect and target recognition accuracy are optimized in tandem, effectively improving the false detection and false negative detection problems in complex scenes, and significantly improving the accuracy and robustness of infrared target recognition. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the target recognition method based on infrared image colorization in this application; Figure 2 This is a flowchart illustrating the second embodiment of the target recognition method based on infrared image colorization in this application; Figure 3 This is a schematic diagram of the overall process of an embodiment of the target recognition method based on infrared image colorization in this application; Figure 4 This is a schematic diagram of the real-time reasoning process for target identification in this application; Figure 5 This is a schematic diagram of the adaptive color conversion process for infrared images in this application; Figure 6 This is a schematic diagram of the module structure of the target recognition device based on infrared image colorization according to an embodiment of this application; Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the target recognition method based on infrared image colorization in the embodiments of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application embodiment is: acquiring a single-channel infrared image; inputting the preprocessed infrared image into a trained infrared colorization detection model, and outputting the infrared target recognition result, wherein the infrared colorization detection model is obtained by joint optimization training of an infrared colorization network and a target detection network. Thus, by adaptively reconstructing the single-channel infrared image with colorization, the image feature dimension is effectively expanded, and the target feature expression capability is strengthened; through multi-scale feature fusion and infrared gradient characteristic constraints, the complete structure and detailed features of the target are preserved, improving the feature discrimination of different targets; through joint optimization constraints of the two networks, the colorization effect and target recognition accuracy are optimized in tandem, effectively improving the false detection and missed detection problems in complex scenes, and significantly improving the accuracy and robustness of infrared target recognition.

[0023] This application's embodiments take into account that existing mainstream infrared target recognition schemes mostly use single-channel infrared grayscale image detection directly, or employ fixed lookup table pseudo-color preprocessing. These schemes have a single dimension of image features, cannot amplify the feature differences between different infrared targets, and suffer from poor recognition performance in complex scenes. Specifically, this is manifested as follows: Low information dimensionality: Infrared images only have single-channel grayscale intensity information and lack color dimension, so the number of features available for model learning is limited.

[0024] Low target discrimination: In complex scenes, different objects have similar infrared grayscale distributions, making it difficult for the model to distinguish between various targets and easily leading to confusion in recognition.

[0025] Poor adaptability: Traditional pseudo-color relies on fixed mapping rules and cannot dynamically adjust color conversion according to the heat distribution of the image and the target scale, resulting in very little improvement in recognition performance.

[0026] Failure in complex environments: Feature recognition accuracy further declines in low-contrast and target-occluded scenes, resulting in high frequencies of missed detections and false detections.

[0027] Independent network training: In traditional solutions, color processing and object detection networks are trained separately. Color images cannot meet the requirements of detection tasks, and feature learning is fragmented.

[0028] This application provides a solution that addresses the problems of low image information dimensionality, weak feature expression capability, high false detection and false negative rates, and low overall recognition accuracy in existing infrared target recognition schemes under complex scenarios.

[0029] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device, target recognition device or system based on infrared image colorization, etc., capable of realizing the above functions. The following description uses a target recognition system based on infrared image colorization as an example to illustrate this embodiment and the subsequent embodiments.

[0030] Based on this, embodiments of this application provide a target recognition method based on infrared image colorization, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the target recognition method based on infrared image colorization in this application.

[0031] In this embodiment, the target recognition method based on infrared image colorization includes steps S10 to S20. The following provides a detailed explanation of each step.

[0032] like Figure 1 As shown, the first embodiment of this application proposes a target recognition method based on infrared image colorization, the method comprising: Step S10: Acquire a single-channel infrared image; Specifically, the original infrared image of the scene can be acquired by an infrared imaging device. The acquired infrared image is a single-channel grayscale image, which only contains the infrared radiation intensity information of the scene.

[0033] By acquiring single-channel infrared images in real time or offline, this method provides the original data for subsequent feature extraction, color reconstruction, and target recognition. It also provides the original visual input for feature learning and target inference of the subsequent network model, ensuring the effective implementation of the overall recognition process and serving as the data foundation for achieving accurate infrared target recognition.

[0034] Step S20: Input the single-channel infrared image into the trained infrared colorization detection model and output the infrared target recognition result. The infrared colorization detection model is obtained by joint optimization training of the infrared colorization network and the target detection network.

[0035] Specifically, this step utilizes a pre-trained, jointly optimized infrared colorization detection model to perform target recognition on the original single-channel infrared image. The infrared colorization network, with its built-in encoder-decoder structure, performs deep feature extraction and adaptive colorization reconstruction on the single-channel infrared image. Through a multi-scale feature fusion module and an infrared gradient characteristic guidance module within the encoder, it optimizes the multi-scale structural features of the image and constrains target contour details, transforming the low-dimensional single-channel grayscale image into a three-channel color infrared image with richer feature information. Furthermore, the reconstructed high-dimensional color infrared image is input into the jointly trained target detection network. Using the rich feature information enhanced by colorization, it completes target category determination, bounding box localization, and confidence assessment within the scene, ultimately outputting an infrared target recognition result containing target category, target location, and recognition confidence.

[0036] Unlike traditional infrared recognition schemes that separate colorization processing from target detection, this invention employs a dual-network joint constraint training method. By simultaneously optimizing the parameters of the two networks through a total loss function consisting of detection loss, colorization reconstruction loss, and detection-guided consistency loss, the infrared colorization process is adapted to the requirements of target detection tasks. This effectively improves the accuracy and robustness of target recognition in complex scenes, compensates for the shortcomings of the original infrared image in terms of single feature dimension and insufficient discriminative power, and is compatible with multiple mainstream target detection frameworks, demonstrating excellent scene adaptability.

[0037] This embodiment provides a complete infrared target recognition method flow, from infrared image data acquisition, feature optimization and extraction, image color reconstruction to intelligent target recognition. The method acquires single-channel infrared images as raw data, utilizes the encoder's built-in multi-scale feature fusion module and infrared gradient characteristic guidance module to complete image feature optimization processing, reconstructing the single-channel infrared image into a three-channel color infrared image, and then completes target detection and recognition through a jointly trained dual-network structure. This provides a systematic and automatically executable algorithm solution for infrared scene target recognition, overcoming the shortcomings of traditional infrared recognition schemes such as single image feature dimensions, low target discrimination, and poor stability in complex scenes.

[0038] In this embodiment, based on the first embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. Based on this, refer to... Figure 2 , Figure 2This is a flowchart illustrating the second embodiment of the target recognition method based on infrared image colorization of this application. After step S10, the method further includes steps S01-S07: Step S01: Normalize, denoise, and enhance the single-channel infrared image to obtain the training sample image.

[0039] Specifically, the single-channel infrared image undergoes normalization, denoising, and enhancement preprocessing to obtain training sample images, including: Step S011: Normalize the grayscale of all pixels according to the minimum and maximum values ​​of the overall grayscale of the single-channel infrared image to unify the pixel value range of the image and obtain a normalized image. Specifically, the grayscale values ​​of each pixel in a single-channel infrared image are read, the minimum and maximum grayscale values ​​of the entire image are calculated, and the grayscale values ​​of all pixels in the image are linearly mapped one by one using the pixel grayscale normalization formula. The image pixel grayscale values ​​are uniformly mapped to the standard value range, and the image normalization calculation is completed to obtain the normalized image.

[0040] More specifically, a single-channel infrared image I(x) is acquired by an infrared imaging device. The image resolution can be 640×512, 1280×720, 1920×1080 or higher. The pixel grayscale values ​​in the image correspond to infrared radiation intensity information. The grayscale values ​​of each pixel in the single-channel infrared image are read, and the minimum grayscale value I_min and the maximum grayscale value I_max of the entire image are calculated. The following pixel grayscale normalization formula is used to linearly map and calculate all pixels of the image one by one: I_norm(x)=(I(x)-I_min) / (I_max-I_min). Where: I_min and I_max are the minimum and maximum values ​​of the grayscale dynamic range of a single-channel infrared image, respectively. The image pixel grayscale is uniformly mapped to the standard value range through calculation, and the image normalization calculation is completed to obtain the normalized image I_norm(x).

[0041] This achieves uniformity in the grayscale dynamic range of single-channel infrared images, eliminating data distribution differences caused by different imaging resolutions and imaging gains of different devices.

[0042] Step S012: Apply Gaussian filtering or median filtering to the normalized image to eliminate image noise and obtain a denoised image; Specifically, a denoising operation is performed on the normalized image. Depending on the distribution of noise in the image, either Gaussian filtering or median filtering algorithm is selected. All pixels in the image are processed through neighborhood pixel operations to complete the smoothing and correction of image noise, resulting in a denoised image.

[0043] The denoised image expression is: I_den = Denoise(I_norm). Denoise(I_norm) represents the image denoising operation.

[0044] Therefore, invalid interference information such as thermal noise and impulse noise generated during infrared image acquisition is removed, and the basic distribution of image pixels is optimized.

[0045] Step S013: Perform detail enhancement processing on the denoised image and output a standardized preprocessed image as a training sample image.

[0046] Specifically, detail enhancement operations are performed on the denoised image to adjust and optimize the overall grayscale difference and target edge features of the image, and a standardized preprocessed infrared image is output, which is then used as a training sample image for subsequent network training.

[0047] The enhancement expression is: I_pre = Enhance(Denoise(I_norm)).

[0048] Where: Enhance(Denoise(I_norm)) represents the image detail enhancement operation, and the final output is the normalized preprocessed image I_pre.

[0049] This enhances the details and edge features of targets in infrared images, forming standardized training sample data, and providing standardized input for subsequent feature extraction and model training of infrared colorization networks.

[0050] Step S02: Construct an infrared colorization network, and use the infrared colorization network to extract features and reconstruct colors from training sample images to generate colorized infrared images; Specifically, an infrared colorization network is constructed, and feature extraction and color reconstruction are performed on training sample images through the infrared colorization network to generate colorized infrared images, including: Step S021: Construct an infrared colorization network, wherein the infrared colorization network is an encoder-decoder structure; Specifically, a deep neural network model for infrared image color reconstruction is constructed. The network adopts an encoder-decoder architecture, and the colorization network is denoted as F(·;θ). The network takes a preprocessed single-channel infrared image I_pre∈R^(H×W×1) as input, where R represents the real number field, H and W represent the image height and width in pixels, respectively, and the channel dimension is 1. The reconstructed three-channel color infrared image I_rgb∈R^(H×W×3) is taken as output, where the channel dimension is 3, and I_rgb is the three-channel color infrared image output by the network. The encoder is used to extract infrared structural features: Z=Encoder(I_pre). The decoder is used to generate the color space representation: I_rgb=Decoder(Z), thereby constructing an end-to-end mapping network structure from single-channel infrared image to three-channel color image.

[0051] Therefore, an end-to-end infrared image colorization mapping model is constructed to establish the conversion relationship between single-channel infrared features and three-channel color images.

[0052] Step S022: Extract the overall structural features of the image using an encoder to obtain the image structural feature map; Specifically, the encoder integrates a multi-scale feature fusion module and an infrared gradient characteristic guidance module. After extracting the overall structural features of the image through the encoder to obtain the image structural feature map, the process includes: Step S0221: The multi-scale feature fusion module sets up multiple convolutional branches with different receptive fields to extract features from the image basic structure feature map, and optimizes the image basic structure feature map by weighting and fusing the features extracted by the convolutional branches through self-learning weights. Specifically, during the encoder's feature encoding process, a multi-scale feature fusion module is invoked to introduce multi-scale feature branches and construct multiple convolutional branches with different receptive fields to extract basic image structure features. The network obtains the weight coefficients corresponding to each branch through autonomous learning, and performs weighted summation and fusion of the features from multiple branches to complete the optimization and update of the image's basic structure feature map.

[0053] The expression for multi-scale feature extraction is: Z = Σw_i·f_i(I_pre). Where: Z is the weighted fused image feature map, I_pre(x) is the preprocessed single-channel infrared image, f_i represents the convolutional features of different receptive fields, and w_i is the learnable weight.

[0054] Therefore, by using a multi-branch learnable weight fusion mode, the network can adapt to the feature extraction requirements of infrared targets at different scales, thereby enhancing its ability to characterize the outline and thermal distribution structure of infrared targets.

[0055] Step S0222: Extract edge gradient information of the optimized image basic structure feature map through the infrared gradient characteristic guidance module, and constrain the target contour in the optimized basic structure feature image.

[0056] Specifically, the feature map optimized by multi-scale fusion is input into the infrared gradient characteristic guidance module to extract the edge gradient information of the preprocessed infrared image, accurately locate the high gradient edge region of the image, and constrain and correct the target contour region of the optimized feature map based on the infrared gradient characteristics.

[0057] The formula for extracting edge gradient information by the infrared gradient characteristic guidance module is: G(x) = | I_pre(x)|. Where: I_pre(x) is the preprocessed single-channel infrared image. G(x) is the image gradient operator, and G(x) is the edge gradient information. The higher the value of G(x), the more significant the edge contour at the pixel.

[0058] Therefore, the inherent gradient characteristics of infrared images are used to guide network training, guiding the network to maintain structural consistency in high gradient regions and avoiding the problems of target contour distortion and loss of details during color reconstruction.

[0059] Step S023: The image structure feature map is upsampled and feature restored by the decoder to restore image details and spatial information, and a color infrared image is reconstructed.

[0060] Specifically, the optimized structural feature map output by the encoder is input into the decoder. The decoder performs upsampling operations and spatial and channel feature restoration operations to restore the deep abstract features into three-dimensional color image data. This application abandons the traditional fixed LUT lookup table pseudo-color mapping scheme and adopts a dynamic color generation strategy bound to the local structure of the image. The color generation calculation formula for any pixel position x in the image is as follows: C(x)=g(I_pre(x),Z(x)). Where: g(·) represents the network adaptive mapping function, I_pre(x) is the pixel grayscale value corresponding to coordinate x in the preprocessed single-channel infrared image, and Z(x) is the local structural feature corresponding to the same pixel coordinate x in the global feature map Z. The color generation result is jointly determined by the image grayscale information and local structural features. The color distribution fits the target's own contour and thermal distribution structure, rather than using a fixed grayscale-color mapping rule, effectively avoiding the defects of traditional fixed LUT color distortion and detachment from scene content.

[0061] Based on the pixel-level dynamic color mapping logic described above, the complete colorization output expression is: I_rgb=F(I_pre;θ). Where: I_pre is the preprocessed single-channel infrared image, θ is the learnable parameter of the colorization network, F(·;θ) refers to the colorization network of the complete encoder-decoder deep network model, which fully includes the entire operation process of encoder feature extraction, decoder pixel-by-pixel calculation of C(x), and stitching to generate a color image; I_rgb is the final output three-channel color infrared image.

[0062] Thus, abstract deep features are restored into visualized color image information, breaking away from the limitations of traditional pseudo-color solidified mapping. Relying on a pixel-level adaptive color mapping mechanism, adaptive reconstruction from a single-channel infrared image to a three-channel color infrared image is completed.

[0063] Step S03: The colorized infrared image is input into the target detection network, and the target detection result is output.

[0064] Specifically, the colorized infrared image is input into the target detection network, and the target detection result is output, including: Step S031: Select any one of the following detection networks as the target detection network: YOLO series, RT-DETR, or Faster R-CNN; Specifically, based on the actual deployment scenario and detection accuracy requirements, one of the YOLO series networks, RT-DETR network, and Faster R-CNN network is selected as the target detection network for this method to construct a detection model structure adapted to feature inference of colorized infrared images.

[0065] Therefore, it is compatible with mainstream general-purpose object detection frameworks, ensuring the portability and engineering adaptability of the algorithm.

[0066] Step S032: The colorized infrared image is fed into the target detection network for forward inference calculation, and the category information, confidence score, target bounding box coordinates and semantic features corresponding to each target to be identified in the colorized infrared image are output, and the target detection result is output.

[0067] Specifically, the three-channel color infrared image obtained by network reconstruction is input into the selected target detection network D(·), and forward inference is performed to analyze the features and infer the target of all targets to be detected in the image. The corresponding target category information, confidence score, target bounding box coordinates and image semantic features are generated one by one, and all target information is integrated to obtain the complete target detection result.

[0068] The target detection result output formula is: O=D(I_rgb). Where: I_rgb is the three-channel color infrared image output by the decoder, and O is the final output set of target detection results, specifically expressed as O=(cls_i, score_i, bbox_i). cls_i represents the category information of the i-th detected target, score_i is the target detection confidence score corresponding to the i-th detected target, and bbox_i is the target bounding box coordinate information of the i-th detected target.

[0069] This completes the target feature analysis and target information output of the colorized infrared image, providing complete pre-detection data for subsequent loss function calculation and network parameter optimization.

[0070] Step S04: Construct a total loss function that includes a detection loss function, a colorization reconstruction loss function, and a detection-guided consistency loss function.

[0071] Specifically, a total loss function is constructed, comprising a detection loss function, a colorization reconstruction loss function, and a detection-guided consistency loss function, including: Step S041: Set independent weight coefficients for the detection loss function, color reconstruction loss function, and detection-guided consistency loss function, and sum the three types of loss functions by weight to obtain the total loss function; Specifically, independent weight coefficients are configured for detection loss, color reconstruction loss, and detection-guided consistency loss. The three types of loss terms are weighted and integrated according to the focus of the training task, and the total loss function used for joint iterative optimization of the two networks is obtained by superimposing them, so as to realize the collaborative constraint training of the infrared colorization network and the target detection network.

[0072] During the training phase, a joint optimization strategy is introduced to constrain the contribution of colorization results to the detection task. The total loss function is defined as: L_total = λ1L_det + λ2L_color + λ3L_cons. Where: L_det represents the detection loss, used to constrain the accuracy of target detection; L_color represents the colorization reconstruction loss, used to ensure the structural consistency of the colorized image; L_cons represents the detection-guided consistency loss, used to establish the association constraint between colorization and detection, which ensures that colorization does not destroy the semantic information of the target; λ1, λ2, and λ3 are the weight coefficients of the corresponding loss terms.

[0073] Therefore, by updating the network parameters through synchronous backpropagation of the total loss, joint constraint collaborative training of the infrared colorization network and the target detection network can be achieved; by balancing the training ratio of the three constraints through adjustable weights, the bidirectional optimization requirements of colorization reconstruction accuracy and target detection accuracy can be met.

[0074] Step S042, the detection loss function includes calculating the target classification loss, bounding box regression loss, and target confidence loss; Specifically, the detection loss function consists of three parts: classification loss, bounding box regression loss, and target confidence loss. The classification loss uses cross-entropy calculation to constrain the target category prediction accuracy, the bounding box regression loss uses the intersection-over-union (IoU) algorithm to constrain the target localization accuracy, and the target confidence loss is used to correct the network's confidence judgment results for foreground and background targets.

[0075] The detection loss function is expressed as: L_det = L_cls + L_box + L_obj. Where: L_cls is the classification loss, which uses cross-entropy calculation to constrain the accuracy of target category prediction; L_box is the bounding box regression loss, which can use IoU, GIoU, or CIoU cross-union classification algorithms to constrain the accuracy of target localization; L_obj is the target confidence loss, which is used to correct the network's confidence judgment results for foreground and background targets.

[0076] Therefore, the inference accuracy of the detection network is simultaneously constrained from three dimensions: category recognition, location regression, and target confidence, to ensure the accuracy of the target detection results.

[0077] Step S043, the color reconstruction loss function is used to calculate the overall pixel difference and edge gradient difference between the generated color image and the reference image; Specifically, the color reconstruction loss function constrains both the pixel dimension and the gradient dimension of the image, calculating the overall pixel error between the network-generated color infrared image and the reference image, and simultaneously calculating the edge gradient error between the two images. The pixel difference and gradient difference together constitute the complete color reconstruction loss. The reference image is either a weakly supervised pseudo-color reference image or an image output by the teacher network.

[0078] The formula for calculating the color reconstruction loss function is: L_color = ||F(I_pre) - I_ref||_1 + || F(I_pre)- I_ref||_1. Where: I_ref is a weakly supervised pseudo-color reference image or the output image of the teacher network; the first term of the formula calculates the overall pixel error between the color infrared image generated by the network and the reference image using the L1 norm; the second term calculates the edge gradient L1 error between the two images using the L1 norm, and the pixel difference and gradient difference together constitute the complete color reconstruction loss.

[0079] This ensures that the overall pixel distribution of the colorized image conforms to the real reference features, while also constraining the gradient structure of the image edges, avoiding contour distortion and detail blurring during the colorization process, and guaranteeing the consistency of the image structure.

[0080] Step S044, the detection-guided consistency loss function is used to calculate the difference in semantic features extracted by the target detection network under two input methods, wherein the two input methods are directly inputting the preprocessed infrared image and inputting the colorized infrared image, respectively. Specifically, the detection-guided consistency loss function obtains the deep semantic features of the detection network corresponding to two different sets of inputs. One set is the semantic features extracted by the detection network directly inputting the preprocessed single-channel infrared image, and the other set is the semantic features extracted by the detection network inputting the colorized infrared image. The feature difference between the two sets of semantic features is calculated to construct the detection-guided consistency loss.

[0081] The formula for calculating the consistency loss function is: L_cons=||φ(D(F(I_pre)))-φ(D(I_pre))||_2. φ(·) represents the output operator of the feature layer of the detection network; one set of features is the semantic features φ(D(I_pre)) extracted by the detection network directly inputting the preprocessed single-channel infrared image I_pre; the other set of features is the semantic features φ(D(F(I_pre))) extracted by the detection network inputting the colorized image F(I_pre). The feature difference between the two sets of semantic features is calculated by the L2 norm, thereby constructing the detection-guided consistency loss.

[0082] Therefore, a correlation constraint is established between the colorization network and the detection network to ensure that the image colorization process does not destroy the inherent semantic information of the target, and to achieve a two-way matching between colorization feature enhancement and semantic consistency of the detection task.

[0083] Step S05: Substitute the target detection result into the total loss function to calculate the total loss value.

[0084] Specifically, the target detection result is substituted into the total loss function to calculate the total loss value, including: Step S051: Based on the category information, confidence score, and the recognition results of the target bounding box coordinates, and the real labeled data, substitute them into the detection loss function in the total loss function to calculate the target classification loss, bounding box regression loss, and target confidence loss, and obtain the detection loss value; Specifically, the target category information, confidence score, and target bounding box coordinate recognition data output by the target detection network are collected and matched with the corresponding real-world labeled data of the image. The two sets of data are then substituted into the detection loss function to calculate the target classification loss, bounding box regression loss, and target confidence loss, respectively. The results are then summarized to obtain the detection loss value corresponding to a single inference.

[0085] Therefore, the degree of deviation between target category prediction, bounding box localization, and target confidence determination is quantified, and the error data of the detection task is output.

[0086] Step S052: Substitute the pixel and gradient information of the original infrared image and the colorized infrared image into the colorization reconstruction loss function in the total loss function to calculate the overall pixel difference and edge gradient difference between the generated color image and the reference image, and obtain the colorization reconstruction loss value. Specifically, pixel information and edge gradient information of the preprocessed original infrared image, as well as pixel information and edge gradient information of the colorized infrared image generated by the network, are extracted. The two types of image data are substituted into the colorization reconstruction loss function to calculate the overall pixel deviation and edge gradient deviation between the color image and the reference image, and the colorization reconstruction loss value is obtained statistically.

[0087] Therefore, by quantifying the deviation data of the colorization network in the dimensions of pixel fitting and edge structure restoration, the error quantization of image reconstruction accuracy is completed.

[0088] Step S053: Substitute the semantic features corresponding to the original infrared image and the colorized infrared image into the detection guidance consistency loss function in the total loss function to calculate the difference in semantic features extracted by the target detection network under the two input methods and obtain the detection guidance consistency loss value. Specifically, the semantic features of the input detection network from the single-channel raw preprocessed infrared image and the input detection network from the colorized infrared image are obtained respectively. The two sets of semantic feature data from different sources are substituted into the detection guidance consistency loss function, and the difference between the two sets of semantic features is calculated to obtain the detection guidance consistency loss value.

[0089] Therefore, the degree of deviation of the target semantic features before and after colorization is quantified, and the degree of influence of the colorization process on the target semantic information is quantified.

[0090] Step S054: Based on the preset weight coefficients corresponding to various types of losses, the detection loss value, color reconstruction loss value, and detection guidance consistency loss value are weighted and accumulated to obtain the total loss value corresponding to the current iteration.

[0091] Specifically, the independent weight coefficients corresponding to the preset detection loss, color reconstruction loss, and detection-guided consistency loss are retrieved. Each individual loss value is multiplied by its corresponding weight coefficient and then summed to complete the weighted fusion of multi-dimensional loss data, and the total loss value corresponding to the current iteration of training is calculated.

[0092] Therefore, by integrating the three types of constraint errors—detection accuracy, image reconstruction accuracy, and semantic consistency—a unified iterative optimization supervision index is output, providing a basis for the iterative update of dual network parameters.

[0093] Step S06: Based on the total loss value, backpropagate the error and synchronously iterate and update the parameters of the infrared colorization network and the target detection network to complete the joint constraint optimization training of the two networks. Specifically, based on the backpropagation error of the total loss value, the parameters of the infrared colorization network and the target detection network are synchronously and iteratively updated to complete the joint constraint optimization training of the two networks, including: Step S061: Based on the total loss value, perform back gradient propagation to solve the parameter gradients of the infrared colorization network and the target detection network in sequence; Specifically, the total loss value obtained from a single iteration is used as the supervision signal. The error backpropagation gradient operation is performed, and the solution is obtained layer by layer along the forward inference path of the infrared colorization network and the target detection network to obtain the gradient information corresponding to the parameters of each layer of the two networks.

[0094] Thus, the backpropagation of network errors and gradient calculation are completed, providing a gradient basis for the iterative update of network parameters.

[0095] Step S062: Update the weights and bias parameters of the two networks synchronously according to the parameter gradient until the total loss value tends to stabilize. Specifically, the preset optimizer is invoked, and the gradients of the network parameters obtained by solving are used to synchronously update all weight parameters and bias parameters of the infrared colorization network and the target detection network, respectively. The training process is continuously looped, and the change in the total loss value is monitored in real time.

[0096] Therefore, by using gradient information to correct the network parameter values, the model prediction error is gradually reduced, and the network training effect continues to converge.

[0097] Step S063: Through multiple rounds of forward inference, loss calculation and reverse update iterative loop, the joint constraint optimization training of the infrared colorization network and the target detection network is realized.

[0098] Specifically, the complete process of image forward inference, multi-dimensional loss calculation, total loss solution, gradient backpropagation, and network parameter update is repeatedly executed to perform multiple rounds of iterative training. Under the common constraints of the three types of loss functions, the overall performance of the infrared colorization network and the target detection network is optimized simultaneously to complete the joint constraint optimization training of the two networks.

[0099] Thus, the collaborative optimization of the color enhancement task and the target detection task is achieved, an end-to-end dual-network model adapted to infrared target recognition scenarios is constructed, and the overall training and convergence of the model is completed.

[0100] Step S07: When the dual-network joint constraint optimization training iteration reaches the convergence condition, stop parameter updating and obtain the trained infrared colorization detection model.

[0101] Specifically, when the dual-network joint constraint optimization training iteration reaches the convergence condition, parameter updates are stopped, and the trained infrared colorization detection model is obtained, including: Step S071: When the dual-network joint constraint optimization training satisfies any convergence condition, the training is determined to be converged and the network parameter update is stopped. The convergence condition includes: after multiple iterations, the total loss function value tends to stabilize and no longer decreases, or the number of training iterations reaches the preset maximum number of training iterations. Specifically, during continuous iterative training, the total loss value and the number of training iterations for each iteration are recorded in real time to continuously monitor the model training status. When the total loss value remains stable and no longer decreases significantly after multiple iterations, or when the current iteration number reaches the preset maximum number of training iterations, the joint training of the two networks is deemed to have met the convergence condition, and the subsequent network parameter update process is terminated.

[0102] Therefore, setting dual-determinable training termination conditions standardizes the timing of model training termination and avoids model overfitting and ineffective iterative training.

[0103] Step S072: When the convergence condition is met, after training convergence, all internal parameters of the infrared colorization network and the target detection network are locked, and finally the trained infrared colorization detection model is obtained.

[0104] Specifically, after determining that the model training has converged, all weight parameters and bias parameters of the infrared colorization network and the target detection network are fixed, and gradient backpropagation and parameter iteration update operations are no longer performed. The overall structure and parameters of the two networks after joint optimization training are saved, and the final infrared colorization detection model is output.

[0105] Thus, the training and solidification of the entire network are completed, resulting in a complete model that can be directly used for infrared image color reconstruction and target recognition reasoning.

[0106] This embodiment realizes the integrated processing of single-channel infrared image preprocessing, adaptive color reconstruction and infrared target detection. Through the dual-network joint constraint optimization training mechanism, the color reconstruction task and the target detection task are deeply coupled, which solves the technical problems of traditional infrared recognition methods, such as the colorization process and detection task being independent, the infrared image features being single, the target contour being easily distorted, and the semantic information being easily lost. It realizes accurate, stable and automated recognition and detection of infrared targets in complex scenes.

[0107] The complete process of this embodiment can be referred to Figure 3 As shown. Figure 3 This is a schematic diagram of the overall process of the target recognition method based on infrared image colorization involved in this application.

[0108] like Figure 3 As shown in the diagram, the image acquisition module, image preprocessing module, infrared colorization module, target detection module, and joint optimization training module are arranged from top to bottom. Each module is connected sequentially by a unidirectional arrow, clearly demonstrating the complete execution flow logic of the entire method. The image acquisition module acquires the raw single-channel infrared image; the image preprocessing module performs normalization, noise reduction, and detail enhancement standardization operations on the grayscale infrared image; the infrared colorization module maps the single-channel grayscale image to a multi-channel RGB color image adapted to the recognition task using a learnable neural network; the target detection module infers and outputs target category, location, and confidence information based on the color image; the joint optimization training module constructs multiple loss constraints during the training phase and simultaneously completes the collaborative iterative optimization of the colorization network and the detection network. This diagram fully covers the entire process of image acquisition, preprocessing, adaptive colorization, target detection, and dual-network joint training, intuitively demonstrating the functional division and data transfer relationships of each software module.

[0109] With the support of the above-described system module architecture, the real-time inference process of the online recognition stage after deployment of this invention can be referred to... Figure 4 As shown, Figure 4 This is a schematic diagram of the real-time inference process for target identification in this application. For example... Figure 4 As shown, this inference process is geared towards real-time detection scenarios after the device goes online, and fully demonstrates the image recognition execution chain without training: First, infrared image acquisition is performed to obtain the original grayscale infrared image. Then, image preprocessing is performed on the original image. The preprocessed image is sent to the neural network to complete the infrared image adaptive colorization processing, generating and outputting an RGB color image. The color image is then input into the target detection network to complete target inference, and finally, the category and location information corresponding to the target in the image are output.

[0110] The core logic for converting infrared grayscale images to color images can be found in [reference needed]. Figure 5 As shown, Figure 5 This is a schematic diagram of the adaptive color conversion process for infrared images in this application. Figure 5 As shown, this color conversion process fully presents the processing steps for mapping a single-channel grayscale infrared image to a multi-channel RGB image: First, a single-channel grayscale infrared image is input, and deep feature learning of the image is carried out through a neural network. The extracted features are then subjected to multi-channel mapping and reconstruction operations, and finally, a three-channel RGB color image that can be used by the target detection network is output. This process abandons the traditional fixed lookup table pseudo-color mapping method and relies on a learnable network to generate color expressions that fit the image content and are suitable for the recognition task.

[0111] Furthermore, without altering the technical principles of this invention, various alternative implementation methods can be adopted. For example, different neural network models with different structures can be used in the infrared colorization stage to adapt to different accuracy or computing power requirements. In the target detection stage, different types of detection network structures can be used to adapt to real-time or accuracy requirements. At the same time, colorization processing and target detection can also be jointly optimized or end-to-end trained to further improve performance. This invention is not only applicable to static image processing, but can also be extended to infrared video stream processing scenarios to achieve continuous target recognition through frame-by-frame processing or temporal modeling.

[0112] This embodiment addresses the problems of low image information dimensionality, weak feature representation ability, high false positive and false negative rates, and low overall recognition accuracy in existing infrared target recognition solutions, which are often found in complex scenarios. It establishes a hierarchical, modular software architecture consisting of image acquisition, image preprocessing, infrared colorization processing, target detection, and joint optimization training. A learnable neural network enables task-oriented adaptive infrared colorization, expanding image feature information. Joint loss constraints facilitate collaborative optimization of colorization and target recognition tasks, enhancing the model's ability to perceive infrared target details and contours. The entire solution is compatible with existing mainstream detection frameworks, can run stably on conventional terminals without requiring high-performance GPUs, and balances recognition accuracy with lightweight deployment requirements.

[0113] Compared with the prior art, the technical solution of this embodiment has the following advantages: Image feature enhancement: By using neural network adaptive infrared colorization, single-channel grayscale infrared images are converted into multi-channel RGB images, enriching the dimensions of image information and significantly improving the ability to express image features.

[0114] Adaptive colorization generation: Breaking away from the limitations of traditional fixed pseudo-color mapping rules, the network autonomously generates colors based on the scene's own structure and heat distribution, resulting in higher image content fit.

[0115] Target recognition performance optimization: After the colorized image is fed into the detection network, it can provide richer visual features, enhance the model's perception of target contours and details, and reduce the occurrence of missed detections and false detections in complex scenes.

[0116] Deployment is versatile and lightweight: It is compatible with various mainstream detection frameworks such as YOLO series, RT-DETR, and Faster R-CNN, does not require high-performance GPUs, can run stably in real time on ordinary terminals, and has high engineering application value.

[0117] Furthermore, to achieve the above objectives, this application also proposes a target recognition device based on infrared image colorization, such as... Figure 6 As shown, the device includes: Image acquisition module 10 is used to acquire single-channel infrared images; The target recognition module 20 is used to input the single-channel infrared image into the trained infrared colorization detection model and output the infrared target recognition result. The infrared colorization detection model is obtained by joint optimization training of the infrared colorization network and the target detection network.

[0118] The target recognition device based on infrared image colorization provided in this application, employing the target recognition method based on infrared image colorization in the above embodiments, can solve the problems of low image information dimensionality, weak feature expression ability, high false detection and false negative rates, and low overall recognition accuracy in existing infrared target recognition schemes under complex scenes. Compared with the prior art, the beneficial effects of the target recognition device based on infrared image colorization provided in this application are the same as those of the target recognition method based on infrared image colorization provided in the above embodiments, and other technical features in the target recognition device based on infrared image colorization are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0119] Furthermore, to achieve the above objectives, this application also proposes a target recognition device based on infrared image colorization, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the target recognition method based on infrared image colorization as described above.

[0120] The following is for reference. Figure 7 This document illustrates a structural schematic diagram of a target recognition device based on infrared image colorization suitable for implementing embodiments of this application. The target recognition device based on infrared image colorization in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The target recognition device based on infrared image colorization shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0121] like Figure 7As shown, the infrared image colorization-based target recognition device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the infrared image colorization-based target recognition device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the infrared image colorization-based target recognition device to communicate wirelessly or wiredly with other devices to exchange data. While various systems based on infrared image colorization are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0122] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0123] The target recognition device based on infrared image colorization provided in this application, employing the target recognition method based on infrared image colorization in the above embodiments, can solve the problems of low image information dimensionality, weak feature expression ability, high false detection and false negative rates, and low overall recognition accuracy in existing infrared target recognition schemes under complex scenes. Compared with the prior art, the beneficial effects of the target recognition device based on infrared image colorization provided in this application are the same as those of the target recognition method based on infrared image colorization provided in the above embodiments, and other technical features in this target recognition device based on infrared image colorization are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0124] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0125] Furthermore, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the target recognition method based on infrared image colorization as described above.

[0126] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0127] The aforementioned computer-readable storage medium may be included in a target recognition device based on infrared image colorization; or it may exist independently and not assembled into a target recognition device based on infrared image colorization.

[0128] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a target recognition device based on infrared image colorization, the device performs the following actions: acquires a single-channel infrared image; inputs the preprocessed infrared image into a trained infrared colorization detection model; and outputs an infrared target recognition result. The infrared colorization detection model is jointly optimized and trained by an infrared colorization network and a target detection network. Thus, by adaptively reconstructing the single-channel infrared image with colorization, the image feature dimension is effectively expanded, enhancing the target feature expression capability. Through multi-scale feature fusion and infrared gradient characteristic constraints, the complete structure and detailed features of the target are preserved, improving the feature discrimination of different targets. Through joint optimization constraints of the two networks, the colorization effect and target recognition accuracy are optimized in tandem, effectively improving the false detection and missed detection problems in complex scenarios, and significantly improving the accuracy and robustness of infrared target recognition.

[0129] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Python, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0131] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0132] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described target recognition method based on infrared image colorization. This addresses the problems of existing infrared target recognition schemes, such as low image information dimensionality, weak feature representation capabilities, high false positive and false negative rates in complex scenarios, and low overall recognition accuracy. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the target recognition method based on infrared image colorization provided in the above embodiments, and will not be repeated here.

[0133] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the target recognition method based on infrared image colorization as described above.

[0134] The computer program product provided in this application can solve the problems of low image information dimensionality, weak feature expression ability, high false detection and false negative rates, and low overall recognition accuracy in existing infrared target recognition schemes under complex scenes. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the target recognition method based on infrared image colorization provided in the above embodiments, and will not be repeated here.

[0135] One or more technical solutions proposed in this application have at least the following technical effects: acquiring a single-channel infrared image; inputting the preprocessed infrared image into a trained infrared colorization detection model, and outputting an infrared target recognition result, wherein the infrared colorization detection model is obtained by joint optimization training of an infrared colorization network and a target detection network. Thus, by adaptively reconstructing the single-channel infrared image with colorization, the image feature dimension is effectively expanded, and the target feature expression capability is strengthened; through multi-scale feature fusion and infrared gradient characteristic constraints, the complete structure and detailed features of the target are preserved, improving the feature discrimination of different targets; through joint optimization constraints of the two networks, the colorization effect and target recognition accuracy are optimized in tandem, effectively improving the false detection and false negative detection problems in complex scenes, and significantly improving the accuracy and robustness of infrared target recognition.

[0136] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A target recognition method based on infrared image colorization, characterized in that, The method includes: Acquire single-channel infrared images; The single-channel infrared image is input into the trained infrared colorization detection model, and the infrared target recognition result is output. The infrared colorization detection model is obtained by joint optimization training of the infrared colorization network and the target detection network. After acquiring the single-channel infrared image, the process further includes: The single-channel infrared image is normalized, denoised, and enhanced to obtain training sample images; An infrared colorization network is constructed, and feature extraction and color reconstruction are performed on training sample images through the infrared colorization network to generate color infrared images; The colorized infrared image is input into the target detection network, and the target detection result is output. Construct a total loss function that includes a detection loss function, a colorization reconstruction loss function, and a detection-guided consistency loss function; Substitute the target detection result into the total loss function to calculate the total loss value; Based on the total loss value, backpropagation error is performed, and the parameters of the infrared colorization network and the target detection network are updated synchronously and iteratively to complete the joint constraint optimization training of the two networks. When the dual-network joint constraint optimization training iteration reaches the convergence condition, parameter updates are stopped, and the trained infrared colorization detection model is obtained.

2. The method as described in claim 1, characterized in that, The construction of the infrared colorization network, which extracts features and reconstructs colors from training sample images to generate colorized infrared images, includes: An infrared colorization network is constructed, wherein the infrared colorization network is an encoder-decoder structure; The overall structural features of the image are extracted by the encoder to obtain the image structural feature map; The decoder upsamples and restores the feature map of the image structure to recover image details and spatial information, and reconstructs a color infrared image.

3. The method as described in claim 2, characterized in that, The encoder integrates a multi-scale feature fusion module and an infrared gradient characteristic guidance module. After extracting the overall structural features of the image through the encoder to obtain the image structural feature map, the method further includes: The multi-scale feature fusion module sets up multiple convolutional branches with different receptive fields to extract features from the image's basic structural feature map. The features extracted by the convolutional branches are then weighted and fused using self-learning weights to optimize the image's basic structural feature map. The infrared gradient characteristic guidance module extracts the edge gradient information of the optimized image basic structure feature map, and constrains the target contour in the optimized basic structure feature image.

4. The method as described in claim 1, characterized in that, The step of inputting the colorized infrared image into the target detection network and outputting the target detection result includes: Choose any one of the following detection networks as the object detection network: YOLO series, RT-DETR, or Faster R-CNN. The colorized infrared image is fed into the target detection network for forward inference calculation, and the category information, confidence score, target bounding box coordinates and semantic features corresponding to each target to be identified in the colorized infrared image are output, and the target detection result is output.

5. The method as described in claim 4, characterized in that, The step of substituting the target detection result into the total loss function to calculate the total loss value includes: Based on the category information, confidence score, and the recognition results of the target bounding box coordinates, and the real labeled data, the detection loss function in the total loss function is used to calculate the target classification loss, bounding box regression loss, and target confidence loss, thus obtaining the detection loss value. The pixel and gradient information of the original infrared image and the colorized infrared image are substituted into the colorization reconstruction loss function in the total loss function to calculate the overall pixel difference and edge gradient difference between the generated color image and the reference image, and the colorization reconstruction loss value is obtained. The semantic features corresponding to the original infrared image and the colorized infrared image are substituted into the detection guidance consistency loss function in the total loss function to calculate the difference in semantic features extracted by the target detection network under the two input methods, and the detection guidance consistency loss value is obtained. Based on the preset weight coefficients corresponding to various types of losses, the detection loss value, color reconstruction loss value, and detection guidance consistency loss value are weighted and accumulated to obtain the total loss value corresponding to the current iteration.

6. The method as described in claim 1, characterized in that, The backpropagation error based on the total loss value is used to synchronously and iteratively update the parameters of the infrared colorization network and the target detection network, completing the joint constraint optimization training of the two networks, including: Based on the total loss value, back gradient propagation is performed to sequentially solve the parameter gradients of the infrared colorization network and the target detection network; The weights and bias parameters of the two networks are updated synchronously according to the parameter gradient until the total loss value tends to stabilize. Through multiple rounds of forward inference, loss calculation, and reverse update iterative loops, the joint constraint optimization training of the infrared colorization network and the target detection network is achieved.

7. The method as described in claim 1, characterized in that, When the dual-network joint constraint optimization training iteration reaches the convergence condition, parameter updates are stopped, and the trained infrared colorization detection model is obtained, including: When the dual-network joint constraint optimization training satisfies any convergence condition, the training is determined to be converged and the network parameter update is stopped. The convergence condition includes: after multiple iterations, the total loss function value tends to stabilize and no longer decreases, or the number of training iterations reaches the preset maximum number of training iterations. When the convergence condition is met, the training converges and all internal parameters of the infrared colorization network and the target detection network are locked, finally obtaining the trained infrared colorization detection model.

8. A target recognition device based on infrared image colorization, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the target recognition method based on infrared image colorization as described in any one of claims 1 to 7.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the target recognition method based on infrared image colorization as described in any one of claims 1 to 7.

Images