Infrared speckle fusion image separation method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image processing, in particular to an infrared speckle fusion image separation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an infrared speckle fusion image of a target object, inputting the infrared speckle fusion image to a target image separation model, and obtaining an original visual image and an original speckle image. According to the method, the visual image and the speckle image can be separated from the infrared speckle fusion image, so that high-quality separation of the visual image and the infrared speckle image is realized on the premise of not increasing system complexity and cost, and the requirement of the industry for high-precision and low-cost three-dimensional reconstruction is met.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an infrared speckle fusion image separation method, device, electronic device and storage medium. Background Art

[0002] In the field of computer vision, the reconstruction technology of monocular two-dimensional (2D) images to three-dimensional (3D) shapes has become a key tool in diversified applications such as virtual reality, augmented reality, autonomous driving, robot navigation and medical imaging. Currently, 3D reconstruction methods cover a variety of approaches such as stereo vision, laser scanning and structured light technology. Among them, structured light technology has attracted much attention due to its high precision and fast scanning capabilities. However, although the binocular laser speckle projection system can provide high-precision depth information, its complex hardware architecture and high cost pose significant challenges to the deployment and maintenance of the system.

[0003] In contrast, the monocular laser speckle projection system achieves cost reduction and system simplification by projecting speckle patterns and capturing speckle images with a monochrome camera. However, the monocular laser speckle projection system cannot simultaneously obtain visible images, resulting in the lack of necessary visual context when users interpret infrared speckle image information, making it difficult to accurately judge the shape and position of objects. Although the number of cameras is increased, such as equipping an infrared camera to capture infrared speckle information and an RGB camera to obtain visible light images, this not only increases the complexity of system calibration, but also brings data alignment problems. If the infrared speckle information and RGB information are in different camera coordinate systems, the difference in alignment effects directly affects the quality of the generated RGB-D information. In other words, the existing binocular laser speckle projection system and monocular laser speckle projection system cannot simultaneously obtain visible image information and infrared speckle image information without increasing system complexity and cost. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides an infrared speckle fusion image separation method, device, electronic device and storage medium, which can separate visible images and speckle images from infrared speckle fusion images, thereby achieving high-quality separation of visible images and infrared speckle images without increasing system complexity and cost, and meeting the industry's demand for high-precision and low-cost three-dimensional reconstruction.

[0005] A first aspect of the present application provides an infrared speckle fusion image separation method, the method comprising: Acquire an infrared speckle fusion image of the target object; The infrared speckle fusion image is input into a target image separation model to obtain an original visible image and an original speckle image.

[0006] In an optional implementation, the step of inputting the infrared speckle fusion image into a target image separation model to obtain an original visible image and an original speckle image comprises: Extracting features from the infrared speckle fusion image to obtain a first feature map; separating the first feature map from a channel dimension based on a DGR mechanism to obtain a first sub-feature map and a second sub-feature map; Processing the first sub-feature graph based on the DGR mechanism to obtain a first output feature, and processing the second sub-feature graph to obtain a second output feature; Processing the first output feature based on DGR Block to obtain a third output feature, and processing the second output feature to obtain a fourth output feature; The third output feature is subjected to spatial image conversion to obtain the original visible image, and the fourth output feature is subjected to spatial image conversion to obtain the original speckle image.

[0007] In an optional implementation, the processing of the first sub-feature graph based on the DGR mechanism to obtain a first output feature, and the processing of the second sub-feature graph to obtain a second output feature include: Separating the first sub-feature map from the channel dimension based on the DGR mechanism to obtain a first sub-feature map and a second sub-feature map; Separating the second sub-feature map from the channel dimension based on the DGR mechanism to obtain a third sub-feature map and a fourth sub-feature map; Combining the first sub-feature map with the fourth sub-feature map to obtain the first output feature; The second sub-feature map and the third sub-feature map are combined to obtain the second output feature.

[0008] In an optional embodiment, the method further comprises: Inputting the original visible image and the original speckle image into an encoder-decoder of a U-Net structure, so that the encoder performs feature extraction on the original visible image to obtain a third sub-feature map, and performs feature extraction on the original speckle image to obtain a fourth sub-feature map; Performing enhancement processing on the third sub-feature map and the fourth sub-feature map based on a CAB module, wherein the encoder and the decoder are connected through the CAB module; The decoder processes the enhanced third sub-feature map to obtain a first target visible image, and processes the fourth sub-feature map to obtain the first target speckle image.

[0009] In an optional embodiment, the method further comprises: Extracting features of the first target visible image based on the CRB module in the target image separation model to obtain a first residual feature, and extracting features of the first target speckle image to obtain a second residual feature; Determine a first fusion weight from the first residual feature to the second residual feature, and a second fusion weight from the second residual feature to the first residual feature; Performing weighted processing on the first residual feature based on the first fusion weight and the second fusion weight to obtain a visible image feature; Performing weighted processing on the second residual feature based on the first fusion weight and the second fusion weight to obtain a speckle image feature; A spatial image conversion is performed on the visible image feature to obtain a second target visible image, and a spatial image conversion is performed on the speckle image feature to obtain a second target speckle image.

[0010] In an optional implementation, determining a first fusion weight of the first residual feature to the second residual feature includes: The first fusion weight is determined by the following formula: ; in, is the first fusion weight, is a predefined initial fusion weight from the first residual feature to the second residual feature, is the first residual feature, For the second residual feature, C represents point-by-point convolution. Represents the Sigmoid activation function.

[0011] In an optional implementation, before inputting the infrared speckle fusion image into the target image separation model, the method further includes: Performing feature extraction on the infrared speckle fusion image based on a pre-trained VGG model to obtain a second feature map; The second feature map is input into the target image separation model.

[0012] A second aspect of the present application provides an infrared speckle fusion image separation device, the device comprising: An acquisition module, used for acquiring an infrared speckle fusion image of a target object; The separation module is used to input the infrared speckle fusion image into the target image separation model to obtain the original visible image and the original speckle image.

[0013] A third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the infrared speckle fusion image separation method when executing the computer program.

[0014] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned infrared speckle fusion image separation method are implemented.

[0015] In summary, the infrared speckle fusion image separation method, device, electronic device and storage medium provided by the present application, when the infrared speckle fusion image of the target object is acquired, the infrared speckle fusion image is input into the target image separation model, so as to separate the visible image and the speckle image from the infrared speckle fusion image through the target image separation model, thereby achieving high-quality separation of visible images and infrared speckle images without increasing the complexity and cost of the system, and meeting the industry's demand for high-precision and low-cost three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of an infrared speckle fusion image separation method shown in an embodiment of the present application; Figure 2 is another flow chart of an infrared speckle fusion image separation method shown in an embodiment of the present application; Figure 3 is a schematic diagram of an image in a separation process of an infrared speckle fusion image shown in an embodiment of the present application, wherein (A) is an infrared speckle fusion image, (B) is a visible image, and (C) is a speckle image; Figure 4 is a network architecture diagram of a target image separation model shown in an embodiment of the present application; Figure 5 is a schematic diagram of a gating mechanism of a dual-flow gated rectifier unit shown in an embodiment of the present application; Figure 6 is a schematic diagram of a plug-and-play block designed based on a DGR mechanism shown in an embodiment of the present application; Figure 7 is a structural diagram of a CRB module shown in an embodiment of the present application; Figure 8 It is a functional module diagram of an infrared speckle fusion image separation device shown in an embodiment of the present application; Fig. 9 It is a structural schematic diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0018] The following will clearly and completely describe the concept, specific structure and technical effects of the present invention in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present invention. In addition, all the connection / connection relationships involved in the patent do not refer to the direct connection of components, but refer to the formation of a better connection structure by adding or reducing connection accessories according to the specific implementation situation. The various technical features in the invention can be combined interchangeably without conflicting with each other.

[0019] Reference Figure 1 , which is a flow chart of an infrared speckle fusion image separation method shown in an embodiment of the present application, the infrared speckle fusion image separation method includes the following steps.

[0020] S11, acquiring an infrared speckle fusion image of the target object.

[0021] The target object refers to any object that needs to collect visible light images and infrared speckle images at the same time, such as toys, ornaments, etc.

[0022] In the embodiment of the present application, in order to solve the complexity and high cost of the existing binocular laser speckle projection system, and the limitation that the monocular laser speckle projection system cannot obtain visible images, the present application uses an infrared speckle projector to emit infrared speckles, combined with an RGB camera without an infrared cutoff filter (IR-CUT), so as to collect an infrared speckle fusion image VIRS that includes both a visible light image and an infrared speckle image (hereinafter referred to as a speckle image) in the same image, with reference to Figure 3 (A) shown.

[0023] When the infrared speckle fusion image of the target object is collected by the RGB camera without IR-CUT, the RGB camera can transmit the infrared speckle fusion image to the electronic device through wired or wireless communication. The electronic device stores the infrared speckle fusion image of the target object and performs image separation on the infrared speckle fusion image to obtain the original speckle image and the original visible image.

[0024] Compared with the monocular laser speckle projection system in the prior art, which usually relies only on a monochrome camera to capture the speckle image, resulting in the limitation of missing RGB information, the present application simultaneously obtains visible light images and infrared speckle images through a monocular laser speckle projection system based on an ordinary camera without IR-CUT, thereby significantly improving the intuitiveness and practicality of the system's perception of information.

[0025] S12, inputting the infrared speckle fusion image into a target image separation model to obtain an original visible image and an original speckle image.

[0026] In some embodiments, after acquiring the infrared speckle fusion image, the electronic device can input the infrared speckle fusion image into a target image separation model pre-trained based on a dual-stream convolutional fusion network (DCFNet), and process the infrared speckle fusion image through the target image separation model to separate the original speckle image and the original visible image. The target image separation model includes a gating mechanism of a dual-stream gated rectifier unit (DGR) (hereinafter referred to as the DRG mechanism) and a DGR block constructed based on the DGR mechanism, that is, a plug-and-play block is designed based on the DGR mechanism.

[0027] In an optional implementation, the step of inputting the infrared speckle fusion image into a target image separation model to obtain an original visible image and an original speckle image comprises: Performing feature extraction on the infrared speckle fusion image to obtain a first feature map; Separating the first feature map from the channel dimension based on the DGR mechanism to obtain a first sub-feature map and a second sub-feature map; Processing the first sub-feature graph based on the DGR mechanism to obtain a first output feature, and processing the second sub-feature graph to obtain a second output feature; Processing the first output feature based on DGR Block to obtain a third output feature, and processing the second output feature to obtain a fourth output feature; The third output feature is subjected to spatial image conversion to obtain the original visible image, and the fourth output feature is subjected to spatial image conversion to obtain the original speckle image.

[0028] In some embodiments, after the target image separation model receives the infrared speckle fusion image, the infrared speckle fusion image first passes through a series of convolution layers to extract local features of the input infrared speckle fusion image through a sliding convolution kernel to obtain a corresponding feature map (referred to as a first feature map).

[0029] Next, refer to Figure 5, electronic devices can use the DGR mechanism to enhance the expressiveness of the model by dynamically controlling the flow of information, thereby better integrating feature information from different input streams. Among them, the first feature map is divided into two information streams in the channel dimension, namely the visible image information stream (represented in the form of a feature map) and the infrared image information stream (represented in the form of a feature map). In order to facilitate understanding and distinction, the feature map corresponding to the visible image information stream is called the first sub-feature map, and the feature map corresponding to the infrared image information stream is called the second sub-feature map. Similarly, the first sub-feature map is divided into two in the channel dimension, namely the first sub-feature map The first feature map And the second feature map , and divide the second sub-feature map into two in the channel dimension, that is, the second sub-feature map The third feature map And the fourth feature map The electronic device uses the DGR mechanism to perform element-level multiplication. and , and Combined to generate new feature maps, respectively, the first output feature and the second output feature .

[0030] The first output feature is determined by the following formula: : ; The second output feature is determined by the following formula: : ; in, is the first intermediate feature of the first sub-feature graph, , represents the intermediate features used to predict the original visible image RGB, is the second intermediate feature of the second sub-feature graph, , represents the intermediate features used to predict the original speckle image IR, and and Both are used to indicate the output after the DGR mechanism. is the first output feature, , is the second output feature, , and Indicates the image height, and Indicates the image width, and Indicates the number of channels.

[0031] Through the DGR mechanism, , , and The feature map and , Same size, but only has 1 channel and half of , and In addition, the interdependence of two information streams in the feature space can also be captured through the DGR mechanism. If either information stream contains a signal required by the other information stream, the two information streams tend to be split into the second half and transferred to the sibling stream through the gate.

[0032] It should be noted that and always and The first half of and always and The second half of .

[0033] Since the convolution algorithm is composed of a large number of multiplications and additions, and the convolution algorithm is linear, the convolutional neural network composed of a large number of convolutional algorithms will degenerate into a simple linear model under the premise of the introduction of nonlinear factors, that is, it is meaningless to use multi-layer convolution for processing. Therefore, the target image separation model of the embodiment of the present application can also introduce a nonlinear activation function, so that the convolutional neural network can arbitrarily approximate any nonlinear function. Therefore, after being processed by the DGR mechanism, the output feature information will be nonlinearly transformed through the ReLU activation function, and refer to Figure 5 As shown in the figure, the ReLU shown is an integral part of the DGR mechanism. The ReLU activation function truncates negative values ​​to zero and retains only positive values, thereby enhancing the sparsity of the target image separation model and effectively reducing the gradient vanishing problem. The feature information after ReLU activation will provide a richer representation for subsequent processing and feature fusion.

[0034] Further, please refer to Figure 6As shown in the figure, DGR Block consists of two parts, namely the preliminary feature extraction part and the enhanced feature extraction part, which are used for preliminary feature extraction and further enhancement respectively. Among them, both the preliminary feature extraction part and the enhanced feature extraction part adopt a dual-stream structure, which can simultaneously process feature information from two input information streams. In the preliminary feature extraction part, DGR Block improves the stability of input features through layer normalization and extracts features using a series of convolution operations. At the same time, a dynamic gating mechanism is introduced to control the flow of feature information, and a channel attention mechanism is used to enhance the expression of important features. After the preliminary feature extraction process, the enhanced feature extraction part further extracts and enhances features to ensure that the output information is richer and more comprehensive. In addition, DGR Block also uses residual connection technology to combine input features (that is, the visible image information stream and speckle image information stream processed in the previous order) with processed features (that is, the input features are regularized (Layer Normalization, Layernorm), a layer of convolution kernel size 1×1, a step size of 1, a layer of convolution kernel 3×3, a step size of 1, and the features processed by DGR mechanism and CAB) to promote information transmission.

[0035] In an embodiment of the present application, the electronic device can use the preliminary feature extraction part of the DGR Block to perform preliminary feature extraction on the first output feature and the second output feature of the input, including layer normalization and convolution operations. Specifically, firstly, the first output feature and the second output feature are layer normalized respectively, and then a plurality of convolution kernels are used to perform convolution operations on the feature map after layer normalization to obtain a new feature map. Then, the new feature map output by the preliminary feature extraction part is input to the enhanced feature extraction part for processing, including a dynamic gating mechanism and a channel attention mechanism. Specifically, the dynamic gating mechanism processing is implemented through another convolution layer or a fully connected layer, that is, a gating weight map is generated, and then the gating weight map is applied to the feature map after the convolution operation, and an element-by-element multiplication operation is performed to obtain a weighted feature map, in which important features are enhanced and unimportant features are suppressed. Finally, the channel attention mechanism is used to reweight the features of different channels to emphasize the channels that are more useful for the task, that is, to generate channel descriptors through global average pooling (or other pooling methods), and then generate channel attention weights through a fully connected layer (or convolutional layer) and a nonlinear activation function (such as ReLU), and apply the channel attention weights to the feature map processed by the dynamic gating mechanism, perform channel-by-channel multiplication, and further obtain a weighted enhanced feature map, in which the features of each channel are reweighted according to their importance. The output of DGRBlock is a weighted enhanced feature map including the weighted enhanced first output feature map (called the third output feature map) and the weighted enhanced second output feature map (called the fourth output feature map).

[0036] After obtaining the third output feature map corresponding to the visible image information stream and the fourth output feature map corresponding to the speckle image information stream, the electronic device can convert the feature map back to the image space through a network structure such as a decoder or a deconvolution network in the target separation model, that is, output the original visible image and the original speckle image respectively.

[0037] In an embodiment of the present application, in the U-Net structure, feature representation is enhanced by a channel attention mechanism and a spatial attention mechanism, and efficient image reconstruction is achieved by a multi-level feature fusion technology based on deep learning. In other embodiments, the electronic device can also use different attention mechanisms, such as a non-local attention mechanism or a bilinear attention mechanism, to replace the existing attention mechanism to improve the adaptability and performance of the model to various scenarios. In addition, different feature extraction methods, such as DenseNet or ResNet, can be introduced into U-Net to further enhance the feature capture capability, thereby enhancing the overall segmentation effect. And using other fusion methods, such as attention-guided feature fusion or image pyramid fusion, to replace the current fusion strategy, can provide better feature retention and reconstruction quality when processing complex images. In addition, the electronic device can also use traditional image fusion techniques, such as weighted averaging or wavelet transform, to further optimize the reconstruction results and improve the clarity and accuracy of the final image.

[0038] Through the above optional implementation, multi-level feature extraction is achieved through DGR Block, which can effectively capture all-round features from basic edges, textures to complex shapes and structural information from the input data. Through the DGR mechanism and DGRBlock, specific information can be adaptively and selectively retained or suppressed, thereby optimizing feature representation. At the same time, combined with the channel attention mechanism, the emphasis effect of key features is further enhanced, and the recognition and classification performance of the model is significantly improved. Secondly, DGR Block can also effectively alleviate the gradient vanishing problem in deep networks and promote the rapid training of models. And by introducing residual connections, the retention of important input information is ensured, thereby improving the learning ability of the model in complex data scenarios. In addition, the flexible parameter configuration of DGR Block allows shared or independent parameter settings in different situations, so that the model can be flexibly adjusted to improve training efficiency and performance. Finally, the output features processed by dynamic gating and attention mechanism show stronger expressive power, can better adapt to diverse input data, and further improve the generalization ability of the model.

[0039] In an optional implementation, the electronic device may pre-train a target image separation model based on DCFNet. Specifically, the electronic device may first construct an initial image separation model based on DCFNet, and construct a loss function of the initial image separation model by combining multiple loss strategies (e.g., pixel loss and gradient loss, etc.).

[0040] Among them, the pixel loss adopts a multi-loss strategy, combining multiple loss functions, including mean square error loss (MSELoss) and Charbonnier loss, to comprehensively evaluate the model performance and enhance the training effect.

[0041] The mean square error loss evaluates the performance of the model by calculating the square difference between the predicted value and the true value, emphasizing the penalty for large prediction errors, so that the model pays more attention to the overall accuracy. In an embodiment of the present application, in the target image separation model, the pixel loss of the visible image is evaluated using the mean square error loss, which calculates the square sum of the differences between pixels, and can better reflect the overall similarity between the generated image and the real image, especially in terms of image brightness and contrast. In addition, the sensitivity of the mean square error loss to the overall pixel difference makes it play an important role in the reconstruction process of the visible image.

[0042] The mean square error loss function is expressed by the following formula: : ; in, is a real visual image, RGB is a visual image generated based on DCFNet separation, is the weight, usually set to 2.

[0043] Charbonnier loss is a smooth L1 loss, commonly used in image reconstruction and denoising tasks. The Charbonnier loss function takes into account the stability of numerical calculations, avoids instability caused by small errors, and Charbonnier loss has a smaller penalty for small errors and a larger penalty for large errors, providing better robustness and performance, and is particularly suitable for processing noisy images. In an embodiment of the present application, in the target image separation model, the pixel loss of the speckle image is usually evaluated using the Charbonnier loss. The Charbonnier loss function can effectively process the noise and details in the image, so that the generated speckle image can reduce the impact of noise on the result while retaining key features. In addition, Charbonnier loss can also provide a good guide for the optimization process by calculating the differences between pixels, combined with smoothness and robustness.

[0044] The Charbonnier loss function is expressed by the following formula: : ; in, is the real speckle image, IR is the speckle image generated based on DCFNet separation, is a very small constant. In order to stabilize the value, it can be set to 1e-6.

[0045] In addition to pixel loss, gradient loss is also incorporated into the loss function to further enhance the edge and detail information of the image. Gradient loss evaluates the detail retention ability of the model output by calculating the difference in image gradients, helping the model to retain the edges and details of the image and improve the quality of the generated image. Similarly, gradient loss also adopts a multi-loss strategy, including Structural Similarity Index Measure (SSIM) loss, Recons Loss, Exclusion Loss, and Perceptual Loss, to ensure that the generated image has better performance in details such as edges and textures, thereby improving the visual quality of the generated image. As an indicator for evaluating image quality, SSIM is designed based on the characteristics of the human visual system. SSIM can fully reflect the similarity between images by considering factors such as image brightness, contrast, and structure. The higher the SSIM value in the loss function, the higher the structural similarity between the output image and the target image. Minimizing the SSIM loss helps improve the quality and realism of the generated image.

[0046] The structural similarity index is expressed by the following formula: : ; Among them, when it is necessary to evaluate the quality of the generated visual image RGB, x For RGB, y for , is the average value of RGB, yes The average value of is the variance of RGB, yes The variance of is RGB and When it is necessary to evaluate the generation quality of the speckle image IR, x For IR, y for , It is IR yes The average value of is the variance of IR, yes The variance of is IR and In addition, , , and are constants used to maintain stability, L is the dynamic range of pixel values, , .

[0047] The SSIM loss function is expressed by the following formula: : .

[0048] The reconstruction loss is mainly used to measure the reconstruction quality between the model output and the input. Since the advantage of L1 loss is that it gives a relatively small penalty for larger errors, the model can be smoother when generating output while retaining more detail information, thereby improving the reconstruction effect. Therefore, in the embodiment of the present application, the reconstruction loss uses L1 loss to calculate the absolute difference between the reconstruction result and the original input image.

[0049] The reconstruction loss function is expressed by the following formula: : ; Where I is the fused image of the input RGB and IR.

[0050] The exclusion loss is used to enhance the model's focus on specific features while also suppressing some unnecessary features or enhancing required features. By setting different "levels", the exclusion loss function can control the model's focus on features, thereby achieving fine-tuning of the model output and improving the usability of the final result.

[0051] The exclusion loss function is expressed by the following formula: : ; ; in, N is the total number of downsampling layers For conduct times downsampling, Express conduct times of downsampling. It should be noted that the downsampling is at most times. Tanh is the hyperbolic tangent function, and are normalization factors, is the gradient, Represents element-wise multiplication.

[0052] Perceptual loss is an emerging loss function for evaluating image quality, which aims to measure the similarity between images through features extracted by pre-trained deep networks. Unlike traditional pixel-level losses such as mean squared error or L1 loss, perceptual loss focuses on the high-level features and semantic information of the image. Specifically, perceptual loss is usually calculated by comparing the representation of the generated image and the target image in the feature space, which is usually extracted from a convolutional neural network such as a VGG network. Using perceptual loss can better capture the structure and content of the image, thereby providing better results in visual effects. By comparing images in the feature space, the model can more effectively reduce artifacts and distortion in the generated images and ensure that the output images are more visually similar to the real images.

[0053] The perceptual loss is expressed by the following formula: : ; in, are the weights of different layers, is the feature output at the i-th layer.

[0054] Finally, the electronic device obtains the loss function of the initial image separation model by synthesizing the above loss functions.

[0055] The loss function of the initial image separation model is expressed by the following formula: : .

[0056] After the loss function of the initial image separation model is constructed, the electronic device also needs to prepare a training sample set for adjusting the loss function of the initial image separation model to train the initial image separation model until the preset iteration termination condition is met.

[0057] Through the above optional implementation, by combining multiple loss strategies to construct the loss function of the image separation model, the optimized training of the image separation model is achieved, thereby improving the image separation effect.

[0058] In an optional embodiment, the method further comprises: Inputting the original visible image and the original speckle image into an encoder-decoder of a U-Net structure, so that the encoder performs feature extraction on the original visible image to obtain a third sub-feature map, and performs feature extraction on the original speckle image to obtain a fourth sub-feature map; Performing enhancement processing on the third sub-feature map and the fourth sub-feature map based on a CAB module, wherein the encoder and the decoder are connected through the CAB module; The decoder processes the enhanced third sub-feature map to obtain a first target visible image, and processes the fourth sub-feature map to obtain the first target speckle image.

[0059] Refer to Figure 2 In some embodiments, after the infrared speckle fusion image is separated by the target image separation model to obtain the original speckle image and the original visible image, the electronic device can further optimize the separated original speckle image and the original visible image using the encoder-decoder of the U-shaped convolutional network (U-Net) structure, effectively restore the image layer by layer and capture the features, so as to obtain more accurate and clear target speckle image and target visible image. Among them, U-Net is a powerful network architecture with symmetrical encoding path and decoding path, which can effectively capture features while restoring the image layer by layer. The encoder part and the decoder part of the U-Net are connected through the channel attention (Convolutional Block Attention, CAB) module to enhance the expression ability of the features. The CAB module is composed of convolution operation, channel attention mechanism and feature fusion, which is used to enhance the expression ability of important channels in the feature map, thereby improving the performance of the model in image processing tasks.

[0060] Specifically, when the original visible image and the original speckle image obtained by preliminary separation are input to the encoder-decoder of the U-Net network structure, in the encoder part of the U-Net, the input original visible image and the original speckle image (hereinafter referred to as the input image for the sake of description) first pass through a series of convolution layers, which extract local features of the input image through a sliding convolution kernel. In each layer of the encoder, the convolution operation will generate a new feature map. The feature map of the original visible image is called the third sub-feature map, and the feature map of the original speckle image is called the fourth sub-feature map. Among them, the third sub-feature map includes the feature information of the original visible image at different scales and different positions, and similarly, the fourth sub-feature map includes the feature information of the original speckle image at different scales and different positions. Next, the CAB module is used to perform preliminary processing on the third and fourth sub-feature maps through a series of convolutional layers to extract local features. Then, the channel attention mechanism (CALayer) is used to calculate the weights of the third and fourth sub-feature maps, adaptively adjust the importance of the first and second sub-feature maps, and perform feature fusion on the extracted local features to output the enhanced feature map. Finally, the decoder is based on layer-by-layer upsampling and combined with jump connections to restore the spatial resolution of the enhanced third and fourth sub-feature maps. After each layer of sampling, the resolution of the third and fourth sub-feature maps will increase, gradually approaching the resolution of the final desired image. In addition, at each layer of the decoder, a channel attention module is used to process the input feature map to focus on significant features and suppress unimportant features. At the last layer of the decoder, a convolution layer is used to reduce the number of channels of the third sub-feature map and the fourth sub-feature map to the same number of channels as the original image (i.e., the original visible image and the original speckle image) (for example, for an RGB image, the number of channels is 3), and an activation function (such as Sigmoid or Softmax) is used to convert the third sub-feature map and the fourth sub-feature map into the corresponding first target visible image and first target speckle image.

[0061] It should be noted that the embodiment of the present application provides two CAB modules, which are respectively used to process the visible image information stream and the speckle image information stream, that is, one CAB module is used to enhance the third sub-feature map, and the other CAB module is used to enhance the fourth sub-feature map.

[0062] In some embodiments, the electronic device can add the third sub-feature map and the fourth sub-feature map processed by the decoder to the third sub-feature map and the third sub-feature map of the original input to form a residual connection. On the feature map after the residual connection, the model continues to perform subsequent operations, that is, the decoder performs spatial image conversion based on the feature map after the residual connection, and outputs the corresponding first target visible image and first target speckle image. The residual connection promotes information flow, alleviates the gradient vanishing problem, and helps the model to train and converge more effectively.

[0063] Through the above optional implementation, by introducing the encoder-decoder of the U-Net structure and combining the CAB module, the original speckle image and the original visible image obtained by preliminary separation are further optimized, which can restore the image layer by layer and effectively capture the features, significantly improving the clarity and accuracy of the image. Through the channel attention mechanism of the CAB module, the expression ability of important channels is enhanced, and unimportant features are suppressed, thereby improving the performance of the model in image processing tasks. In addition, two CAB modules are used to process the visible image information flow and the speckle image information flow respectively, realizing the targeted enhancement of the two types of image information, further improving the effect of image processing.

[0064] In an optional embodiment, the method further comprises: Extracting features of the first target visible image based on the CRB module in the target image separation model to obtain a first residual feature, and extracting features of the first target speckle image to obtain a second residual feature; Determine a first fusion weight from the first residual feature to the second residual feature, and a second fusion weight from the second residual feature to the first residual feature; Performing weighted processing on the first residual feature based on the first fusion weight and the second fusion weight to obtain a visible image feature; Performing weighted processing on the second residual feature based on the first fusion weight and the second fusion weight to obtain a speckle image feature; A spatial image conversion is performed on the visible image feature to obtain a second target visible image, and a spatial image conversion is performed on the speckle image feature to obtain a second target speckle image.

[0065] Refer to Figure 2In some embodiments, after the original visible image and the original speckle image are preliminarily refined to obtain the first target visible image and the first target speckle image, the electronic device may further optimize the first target visible image and the first target speckle image using the coupling representation block (CRB) module in the target separation model to obtain a more accurate and clear second target visible image and a second target speckle image. Figure 7 As shown, the CRB module refines the rough representation of the first target visible image and the first target speckle image by establishing a mutual correspondence, thereby improving the quality of the first target visible image and the first target speckle image, and ensuring the clarity and detail expression of the first target visible image and the first target speckle image.

[0066] Specifically, the electronic device first uses one or more convolutional layers in the CRB module to process the first target visible image and the first target speckle image to extract their corresponding residual features, and the residual features of the first target visible image are called first residual features, and the residual features of the first target speckle image are called second residual features. Then, two learnable parameters are defined: and ,in It is used to adjust the fusion weight from the visible image stream to the speckle image stream, that is, the first initial fusion weight from the first residual feature to the second residual feature, The fusion weight from the speckle image stream to the visible image stream, that is, the second initial fusion weight from the second residual feature to the first residual feature, and and The initialization value of is set to 0.2.

[0067] Among them, the CRB module contains two convolution operations, which are used to process the first residual feature and the second residual feature respectively. First, point-by-point convolution is used C To achieve cross-feature channel interaction, the convolution operation does not use the bias term and uses the Sigmoid activation function The output value is compressed to between 0 and 1, thereby generating a weight that reflects the importance of the input feature (that is, the first residual feature and the second residual feature). In the forward propagation process, the CRB module receives the first residual feature and the second residual feature, first obtains the fusion weight through the convolution operation, and converts the first residual feature To the second residual feature The fusion weight of is called the first fusion weight, and the second residual feature To the first residual feature The fusion weight of is called the second fusion weight.

[0068] The first fusion weight is determined by the following formula: : ; Among them, C represents point-by-point convolution. Represents the Sigmoid activation function.

[0069] The second fusion weight is determined by the following formula: : .

[0070] It should be noted that and It is learnable, that is, it allows the target image separation model to adaptively adjust the first initial fusion weight and the second initial fusion weight to optimize the feature fusion effect. Through weight adjustment during training, the target image separation model can learn how to optimally balance the contributions between different feature streams, which helps reduce the risk of overfitting.

[0071] Next, the CRB module uses the calculated first fusion weight and the second fusion weight The weighted processing of the first residual feature and the second residual feature means that the residual feature in the first target visible image will affect the first target speckle image, and vice versa. Specifically, the weighted result from the first target speckle image to the first target visible image will be subtracted from the first target visible image to adjust and correct the first residual feature, that is, from the first residual feature Subtract the weighted influence of the second residual feature on it , and obtain the adjusted visual image features.

[0072] The visible image features are determined by the following formula: : ; The weighted result from the first target visible image to the first target speckle image is subtracted from the first target speckle image to adjust and correct the second residual feature, that is, to obtain the second residual feature. Subtract the weighted influence of the first residual feature on it , and get the adjusted speckle image features.

[0073] The speckle image characteristics are determined by the following formula: : .

[0074] Among them, after being processed by the CRB module, the visible image features The speckle image feature is a visual image feature that integrates the speckle image flow information. It is the speckle image feature fused with the visible image flow information.

[0075] Furthermore, after receiving the visible image features and the speckle image features, two convolutional layers are defined inside the CRB module, namely Conv_RGB for processing the visible image features and Conv_IR for processing the speckle image features. Among them, the first convolutional layer Conv_RGB is used to reconstruct and generate the corresponding visible image from the visible image features, which is called the second target visible image; the second convolutional layer Conv_IR is used to reconstruct and generate the corresponding speckle image from the speckle image features, which is called the second target speckle image. Specifically, the visible image features are input into Conv_RGB, and a convolution operation is performed, that is, the convolution kernel is used to slide on the visible image features, the dot product between the visible image features and the convolution kernel is calculated, and a nonlinear transformation is performed through the activation function ReLU, and a visible image feature map corresponding to the visible image features is output, wherein the visible image feature map contains the image information of the second target visible image reconstructed from the visible image features, and then the visible image feature map is further normalized and processed to generate the second target visible image VIR, refer to Figure 3 Similarly, the speckle image features are input into Conv_IR, and according to the same implementation as above, a second target speckle image S is generated, referring to Figure 3 (C) shown.

[0076] It should be noted that in the embodiment of the present application, the convolution kernel size of the first convolution layer and the second convolution layer are both 3×3, the step size is 1, and the same parameter settings are adopted, including the number of feature channels, convolution kernel size, and bias item configuration.

[0077] In other examples, the electronic device may also use other types of refinement modules, such as a self-attention module or a depthwise separable convolution module, to replace the current CRB module to refine the effect of infrared speckle image separation. By providing different feature extraction methods and computational efficiency, the quality of image separation is maintained or improved. In addition, various residual learning methods may be used to achieve similar refinement effects, thereby flexibly adjusting the network structure to adapt to different application scenarios.

[0078] Through the above optional implementation, the first target visible image and the first target speckle image are further optimized by introducing the CRB module, which can effectively reduce the number of encoder-decoder layers of the U-Net structure, not only greatly improve the processing speed of the network, but also significantly enhance the efficiency of practical applications, and have the potential to achieve real-time modeling. Specifically, by calculating the interaction weights across feature channels, namely the first fusion weight and the second fusion weight, the two residual features are weighted, and the mutual fusion and adjustment of the feature streams are achieved, which not only improves the clarity and detail of the image, but also reduces the risk of overfitting. At the same time, the fused features are reconstructed using the convolutional layer to generate a higher quality second target visible image and a second target speckle image, thereby improving the accuracy and robustness of image processing.

[0079] In an optional implementation, before inputting the infrared speckle fusion image into the target image separation model, the method further includes: Performing feature extraction on the infrared speckle fusion image based on a pre-trained VGG model to obtain a second feature map; The second feature map is input into the target image separation model.

[0080] Combined Figure 2In some embodiments, before the electronic device inputs the infrared speckle fusion image into the target image separation model, the electronic device may use the Visual Geometry Group (VGG) network architecture to extract features from the infrared speckle fusion image, and input the extracted features into the target image separation model for further processing to output the corresponding visible image and speckle image. Specifically, after the infrared speckle fusion image is input into the pre-trained VGG model, a specific convolution layer is selected in the VGG model, and the infrared speckle fusion image is transferred to the selected convolution layer layer by layer for processing by forward propagation. The convolution operation of each layer processes the infrared speckle fusion image and extracts specific features of each layer. In each selected convolution layer, the VGG model first uses the convolution kernel to perform sliding operations and weighted sum operations on the input infrared speckle fusion image, that is, the convolution kernel is slid in the local area of ​​the infrared speckle fusion image, the weighted sum is calculated and a new pixel value is generated, thereby generating a new feature map, so as to capture local features while maintaining spatial structure information. Then, the feature map generated after the convolution operation uses the rectified linear unit (ReLU) as the activation function to increase the nonlinear expression ability of the VGG model, help capture more complex features, and improve the accuracy of feature extraction. Then, a pooling operation (such as maximum pooling or average pooling) is performed to reduce the dimension of the feature map, reduce the amount of calculation, while retaining important information, improving the computational efficiency of the VGG model and reducing overfitting. Finally, the extracted features will be output in the form of a feature map (called the second feature map) as the input of the subsequent target image separation model. Among them, the second feature map contains visual information and speckle information, providing a solid foundation for subsequent image separation. In an embodiment of the present application, the electronic device can select the convolution layer corresponding to the [2nd, 7th, 12th, 21st, 30th] layer from the VGG model as the key layer for feature extraction, so that each layer can capture different levels of features in the infrared speckle fusion image. Among them, the second layer is used to extract low-level features, such as edge information and texture information, which can help identify the basic structure of the infrared speckle fusion image; the seventh layer is used to capture intermediate features, including shapes and features of partial objects, to provide richer contextual information for subsequent separation; the 12th layer is used to extract more complex shapes and patterns, which helps to identify the details of different objects in the infrared speckle fusion image; the 21st layer is used to focus on global contextual information and enhance the understanding of large-scale features; the 30th layer is used to extract high-level features, which can identify high-level objects and scene information, laying the foundation for subsequent separation tasks.

[0081] In some embodiments, the electronic device can pre-build and train a VGG model based on the VGG network, wherein the VGG model includes multiple continuous convolutional layers and pooling layers. By stacking layers, smaller convolution kernels (for example, a kernel size of 3×3) and a small step size (for example, a step size of 1) are used to gradually deepen the network level, thereby being able to more finely and comprehensively extract the diverse feature information in the infrared speckle fusion image. In other embodiments, the electronic device can also use an existing trained VGG model, such as VGG11, VGG13, VGG16 or VGG19, etc., to extract features from the infrared speckle fusion image.

[0082] Through the above optional implementation, by introducing the pre-trained VGG model and selecting a specific convolutional layer to extract features from the infrared speckle fusion image, it is possible to capture features of different levels in the infrared speckle fusion image, from low-level to high-level, which significantly enhances the effect of subsequent image separation while also improving the accuracy and efficiency of image processing.

[0083] Reference Figure 8 , which is a functional module diagram of the infrared speckle fusion image separation device shown in an embodiment of the present application.

[0084] In some embodiments, the infrared speckle fusion image separation device 80 may include a plurality of functional modules composed of computer program segments. The computer program of each program segment of the infrared speckle fusion image separation device 80 may be stored in a memory of an electronic device and executed by at least one processor to perform (see Figure 1 The function of separating infrared speckle fusion images (described in detail) can be divided into multiple functional modules according to the functions performed. The functional modules may include: an acquisition module 801, a separation module 802 and a feature extraction module 803. The module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0085] The acquisition module 801 is used to acquire an infrared speckle fusion image of a target object.

[0086] The separation module 802 is used to input the infrared speckle fusion image into a target image separation model to obtain an original visible image and an original speckle image.

[0087] The separation module 802 is further specifically used to: perform feature extraction on the infrared speckle fusion image to obtain a first feature map; separate the first feature map from the channel dimension based on the DGR mechanism to obtain a first sub-feature map and a second sub-feature map; process the first sub-feature map based on the DGR mechanism to obtain a first output feature, and process the second sub-feature map to obtain a second output feature; process the first output feature based on the DGR Block to obtain a third output feature, and process the second output feature to obtain a fourth output feature; perform spatial image conversion on the third output feature to obtain the original visible image, and perform spatial image conversion on the fourth output feature to obtain the original speckle image.

[0088] The separation module 802 is further specifically used to: separate the first sub-feature map from the channel dimension based on the DGR mechanism to obtain a first sub-feature map and a second sub-feature map; separate the second sub-feature map from the channel dimension based on the DGR mechanism to obtain a third sub-feature map and a fourth sub-feature map; combine the first sub-feature map with the fourth sub-feature map to obtain the first output feature; combine the second sub-feature map with the third sub-feature map to obtain the second output feature.

[0089] The separation module 802 is further configured to: input the original visible image and the original speckle image into an encoder-decoder of a U-Net structure, so that the encoder performs feature extraction on the original visible image to obtain a third sub-feature map, and performs feature extraction on the original speckle image to obtain a fourth sub-feature map; perform enhancement processing on the third sub-feature map and the fourth sub-feature map based on a CAB module, wherein the encoder and the decoder are connected via the CAB module; process the enhanced third sub-feature map based on the decoder to obtain a first target visible image, and process the fourth sub-feature map to obtain the first target speckle image.

[0090] The separation module 802 is further used to: perform feature extraction on the first target visible image based on the CRB module in the target image separation model to obtain a first residual feature, and perform feature extraction on the first target speckle image to obtain a second residual feature; determine a first fusion weight from the first residual feature to the second residual feature, and a second fusion weight from the second residual feature to the first residual feature; perform weighted processing on the first residual feature based on the first fusion weight and the second fusion weight to obtain a visible image feature; perform weighted processing on the second residual feature based on the first fusion weight and the second fusion weight to obtain a speckle image feature; perform spatial image conversion on the visible image feature to obtain a second target visible image, and perform spatial image conversion on the speckle image feature to obtain a second target speckle image.

[0091] The separation module 802 is further specifically configured to determine the first fusion weight by the following formula: ; in, is the first fusion weight, is a predefined initial fusion weight from the first residual feature to the second residual feature, is the first residual feature, For the second residual feature, C represents point-by-point convolution. Represents the Sigmoid activation function.

[0092] The feature extraction module 803 is used to: extract features from the infrared speckle fusion image based on a pre-trained VGG model to obtain a second feature map; and input the second feature map into the target image separation model.

[0093] It should be understood that the various variations and specific embodiments of the infrared speckle fusion image separation method provided in the above embodiments are also applicable to the infrared speckle fusion image separation device of the present embodiment. Through the above detailed description of the infrared speckle fusion image separation method, those skilled in the art can clearly know the implementation method of the infrared speckle fusion image separation device in the present embodiment. For the sake of brevity of the specification, it will not be described in detail here.

[0094] See also Fig. 9 FIG. 1 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. In a preferred embodiment of the present application, the electronic device 9 includes a memory 91 , at least one processor 92 and at least one communication bus 93 .

[0095] Those skilled in the art should understand that Fig. 9The structure of the electronic device 9 shown does not constitute a limitation of the embodiments of the present application, and may be either a bus structure or a star structure. The electronic device 9 may also include more or less other hardware or software than shown in the figure, or a different component arrangement.

[0096] In some embodiments, the electronic device 9 is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, and embedded devices. The electronic device 9 may also include user equipment, which includes but is not limited to any electronic product that can interact with a user through a keyboard, mouse, remote control, touchpad, or voice control device, such as a personal computer, tablet computer, smart phone, digital camera, etc.

[0097] In the above embodiments provided in the present application, it should be understood that the disclosed methods, devices, computer-readable storage media, and electronic devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple components or modules can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or components or modules, which can be electrical, mechanical or other forms.

[0098] The components described as separate components may or may not be physically separated, and the components shown as components may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the components may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0099] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each component may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0100] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0101] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0102] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0103] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for separating infrared speckle fusion images, characterized in that: The method comprises: Acquire an infrared speckle fusion image of the target object; The infrared speckle fusion image is input into a target image separation model to obtain an original visible image and an original speckle image.

2. The infrared speckle fusion image separation method according to claim 1, characterized in that: Inputting the infrared speckle fusion image into the target image separation model to obtain the original visible image and the original speckle image comprises: Performing feature extraction on the infrared speckle fusion image to obtain a first feature map; Separating the first feature map from the channel dimension based on the DGR mechanism to obtain a first sub-feature map and a second sub-feature map; Processing the first sub-feature graph based on the DGR mechanism to obtain a first output feature; and processing the second sub-feature graph to obtain a second output feature; Processing the first output feature based on DGR Block to obtain a third output feature, and processing the second output feature to obtain a fourth output feature; The third output feature is subjected to spatial image conversion to obtain the original visible image, and the fourth output feature is subjected to spatial image conversion to obtain the original speckle image.

3. The infrared speckle fusion image separation method according to claim 2, characterized in that: The processing of the first sub-feature graph based on the DGR mechanism to obtain a first output feature, and the processing of the second sub-feature graph to obtain a second output feature include: Separating the first sub-feature map from the channel dimension based on the DGR mechanism to obtain a first sub-feature map and a second sub-feature map; Separating the second sub-feature map from the channel dimension based on the DGR mechanism to obtain a third sub-feature map and a fourth sub-feature map; Combining the first sub-feature map with the fourth sub-feature map to obtain the first output feature; The second sub-feature map and the third sub-feature map are combined to obtain the second output feature.

4. The infrared speckle fusion image separation method according to any one of claims 1 to 3, characterized in that: The method further comprises: Inputting the original visible image and the original speckle image into an encoder-decoder of a U-Net structure, so that the encoder performs feature extraction on the original visible image to obtain a third sub-feature map, and performs feature extraction on the original speckle image to obtain a fourth sub-feature map; Performing enhancement processing on the third sub-feature map and the fourth sub-feature map based on a CAB module, wherein the encoder and the decoder are connected through the CAB module; The decoder processes the enhanced third sub-feature map to obtain a first target visible image, and processes the fourth sub-feature map to obtain the first target speckle image.

5. The infrared speckle fusion image separation method according to claim 4, characterized in that: The method further comprises: Extracting features of the first target visible image based on the CRB module in the target image separation model to obtain a first residual feature, and extracting features of the first target speckle image to obtain a second residual feature; Determine a first fusion weight from the first residual feature to the second residual feature, and a second fusion weight from the second residual feature to the first residual feature; Performing weighted processing on the first residual feature based on the first fusion weight and the second fusion weight to obtain a visible image feature; Performing weighted processing on the second residual feature based on the first fusion weight and the second fusion weight to obtain a speckle image feature; A spatial image conversion is performed on the visible image feature to obtain a second target visible image, and a spatial image conversion is performed on the speckle image feature to obtain a second target speckle image.

6. The infrared speckle fusion image separation method according to claim 5, characterized in that: The determining of a first fusion weight from the first residual feature to the second residual feature comprises: The first fusion weight is determined by the following formula: ； in, is the first fusion weight, is a predefined initial fusion weight from the first residual feature to the second residual feature, is the first residual feature, For the second residual feature, C represents point-by-point convolution. Represents the Sigmoid activation function.

7. The infrared speckle fusion image separation method according to claim 1, characterized in that: Before inputting the infrared speckle fusion image into the target image separation model, the method further includes: Performing feature extraction on the infrared speckle fusion image based on a pre-trained VGG model to obtain a second feature map; The second feature map is input into the target image separation model.

8. An infrared speckle fusion image separation device, characterized in that: The device comprises: An acquisition module, used for acquiring an infrared speckle fusion image of a target object; The separation module is used to input the infrared speckle fusion image into the target image separation model to obtain the original visible image and the original speckle image.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the infrared speckle fusion image separation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the infrared speckle fusion image separation method described in any one of claims 1 to 7 are implemented.

Images