Image fusion method and device, computer device and readable storage medium

Abstract

The application relates to an image fusion method and device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring a first data set of multi-source images of a power equipment, and training a preset self-encoder model according to the first data set to optimize the self-encoder model, wherein the multi-source images refer to images composed of frame rate synchronized historical polarization degree images and historical RGB images; acquiring a second data set of real-time polarization degree images and a third data set of real-time RGB images of the power equipment, and acquiring high-dimensional input feature vectors of the two images according to the two data sets and the optimized self-encoder model; acquiring weight vectors and fusion feature vectors of each input feature vector in the spatial dimension and the channel dimension according to the input feature vectors corresponding to the two images and a preset attention mechanism; and acquiring a fusion image of the power equipment according to the fusion feature vectors and the optimized self-encoder model. The method can improve the quality of the acquired images.

Description

Technical Field

[0001] This application relates to the field of power system technology, and in particular to an image fusion method and apparatus, computer equipment and readable storage medium. Background Technology

[0002] Today's power system is undergoing a new stage of reform and development, with new energy technologies and intelligent technologies gradually driving the formation of "smart grids." The formation of smart grids is an important pathway to promoting sustainable development and achieving the national goal of a low-carbon economy.

[0003] In recent years, with the continuous improvement of my country's comprehensive national strength and people's living standards, the demand for various electrical equipment and overall electricity consumption has shown an exponential upward trend, which undoubtedly poses a huge challenge to the national power system. Frequent power outages can cause irreparable losses to individuals and even regional economic development. Therefore, effective monitoring of power equipment to accurately locate faults is of paramount importance for the safe and stable operation of the existing power system. Current technologies typically use RGB imaging to monitor power equipment, but the image quality obtained by these technologies is relatively low. Summary of the Invention

[0004] Therefore, it is necessary to provide an image fusion method and apparatus, computer equipment and readable storage medium that can improve the quality of the acquired images in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides an image fusion method. The image fusion method includes:

[0006] The first dataset of multi-source images of power equipment is obtained, and the pre-set autoencoder model is trained based on the first dataset to optimize the autoencoder model. Multi-source images refer to images composed of historical polarization images and historical RGB images with frame rate synchronization.

[0007] A second dataset of real-time polarization images of power equipment and a third dataset of real-time RGB images were obtained, and high-dimensional input feature vectors of the two images were obtained based on the second dataset, the third dataset, and the optimized autoencoder model.

[0008] Based on the input feature vectors corresponding to the two images and the preset attention mechanism, the weight vectors of each input feature vector in the spatial dimension and the channel dimension, as well as the fused feature vectors associated with the weight vectors, are obtained respectively.

[0009] The fused image of the power equipment is obtained based on the fused feature vector and the optimized autoencoder model.

[0010] In one embodiment, acquiring a first dataset of multi-source images of power equipment includes:

[0011] Acquire historical polarization images and historical RGB images of power equipment;

[0012] Multi-source images are obtained from historical polarization images and historical RGB images to obtain the first dataset of multi-source images.

[0013] In one embodiment, acquiring a historical polarization image of the power equipment includes:

[0014] The polarization degree images of power equipment at multiple different polarization angles are acquired, and the horizontal component and diagonal component of polarized light are obtained based on the polarization degree images and the preset Stokes vector.

[0015] Historical polarization images are obtained based on the horizontal and diagonal components of polarized light.

[0016] In one embodiment, training a preset autoencoder model based on a first dataset to optimize the autoencoder model includes:

[0017] The output image is obtained based on the first dataset and the preset autoencoder model;

[0018] Loss parameters are obtained by calculating the loss based on the output image, multiple source images, and a preset loss function.

[0019] The model parameters of the autoencoder model are iteratively updated based on the image loss parameters to optimize the autoencoder model.

[0020] In one embodiment, obtaining the output image based on a first dataset and a preset autoencoder model includes:

[0021] High-dimensional multi-source feature vectors of multi-source images are obtained based on the first dataset and the preset autoencoder model;

[0022] The output image is obtained based on multi-source feature vectors and an autoencoder model.

[0023] In one embodiment, the weight vectors of each input feature vector in the spatial dimension and the channel dimension, and the fused feature vector associated with the weight vectors, are obtained based on the input feature vectors corresponding to the two images and a preset attention mechanism, including:

[0024] Based on the input feature vectors corresponding to the two images and the preset attention mechanism, the weight vectors of each input feature vector in the spatial dimension and the channel dimension are obtained respectively.

[0025] The fused feature vector is obtained based on each input feature vector, the weight vectors corresponding to each input feature vector in the spatial and channel dimensions, and the attention mechanism.

[0026] In one embodiment, the fused feature vector can be obtained using the following formula:

[0027]

[0028] In the formula, and Let α be the input feature vector of the real-time polarization degree image and the input feature vector of the real-time RGB image, respectively. p β p α RGB and β RGB These are the weight vectors corresponding to the real-time polarization image and the real-time RGB image in the spatial and channel dimensions, respectively. and These are the feature vectors output by the attention mechanism based on the spatial dimension and the channel dimension, respectively. To fuse feature vectors.

[0029] Secondly, this application also provides an image fusion apparatus, which includes:

[0030] The model optimization module is used to acquire the first dataset of multi-source images of power equipment and train the preset autoencoder model based on the first dataset to optimize the autoencoder model. Multi-source images refer to images composed of historical polarization images and historical RGB images with frame rate synchronization.

[0031] The input vector acquisition module is used to acquire a second dataset of real-time polarization degree images of power equipment and a third dataset of real-time RGB images, and to acquire high-dimensional input feature vectors of the two images based on the second dataset, the third dataset and the optimized autoencoder model respectively.

[0032] The fusion vector acquisition module is used to obtain the weight vectors of each input feature vector in the spatial dimension and the channel dimension, as well as the fusion feature vector associated with the weight vectors, based on the input feature vectors corresponding to the two images and the preset attention mechanism.

[0033] The fused image acquisition module is used to acquire fused images of power equipment based on the fused feature vector and the optimized autoencoder model.

[0034] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0035] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0036] The aforementioned image fusion method and apparatus, computer equipment, and readable storage medium first acquire a first dataset of multi-source images composed of historical polarization images and historical RGB images of the power equipment, synchronized with the frame rate. A pre-defined autoencoder model is then trained and optimized based on this first dataset. Next, a second dataset of real-time polarization images and a third dataset of real-time RGB images of the power equipment are acquired. High-dimensional input feature vectors for the two images are obtained based on the second and third datasets and the optimized autoencoder model. Further, weight vectors for each input feature vector in the spatial and channel dimensions, along with associated fusion feature vectors, are obtained based on the corresponding input feature vectors and a pre-defined attention mechanism. Spatial and channel attention mechanisms are introduced into the autoencoder fusion strategy to achieve adaptive updating of the weight vectors and realize effective fusion of adaptive multi-scale features. Finally, a fused image of the power equipment is obtained based on the fused feature vectors and the optimized autoencoder model. By employing an autoencoder network structure and introducing spatial and channel attention mechanisms to construct a fusion strategy, polarization degree images and RGB images are fused to obtain fused images of power equipment. This achieves the complementary advantages of polarization degree images and RGB images, provides multi-dimensional feature information, improves the quality of the acquired images, and is conducive to improving the monitoring effect of power equipment. It also facilitates all-weather real-time monitoring of power equipment and accurate fault location. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating an image fusion method in one embodiment;

[0038] Figure 2 This is a flowchart illustrating the process of acquiring a first dataset of multi-source images of power equipment in one embodiment.

[0039] Figure 3 This is a flowchart illustrating the process of obtaining historical polarization images of power equipment in one embodiment;

[0040] Figure 4 This is a schematic diagram illustrating the process of training a preset autoencoder model based on a first dataset to optimize the autoencoder model in one embodiment.

[0041] Figure 5 This is a schematic diagram of the process of obtaining an output image based on a first dataset and a preset autoencoder model in one embodiment;

[0042] Figure 6This is a flowchart illustrating the process of obtaining the weight vectors of each input feature vector in the spatial dimension and the channel dimension, and the fused feature vectors associated with the weight vectors, based on the input feature vectors corresponding to two images and a preset attention mechanism, in one embodiment.

[0043] Figure 7 This is a structural block diagram of an image fusion device in one embodiment;

[0044] Figure 8 This is an internal structural diagram of a computer device in one embodiment.

[0045] Explanation of icon numbers:

[0046] Image fusion device: 10; Model optimization module: 11; Input vector acquisition module: 12; Fusion vector acquisition module: 13; Fusion image acquisition module: 14. Detailed Implementation

[0047] To facilitate understanding of the embodiments of this application, a more comprehensive description of the embodiments of this application will be provided below with reference to the accompanying drawings. The drawings illustrate preferred embodiments of the embodiments of this application. However, the embodiments of this application can be implemented in many different forms and are not limited to the embodiments described herein. Rather, these embodiments are provided to make the disclosure of the embodiments of this application more thorough and complete.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which embodiments of this application belong. The terminology used herein in the description of embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments of this application.

[0049] It is understood that the term "comprising / including" specifies the presence of the stated feature, whole, step, operation, component, part, or combination thereof, but does not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. The terms "first," "second," etc., may be used herein to describe various parameters, but these parameters are not limited by these terms. These terms are used only to distinguish one parameter from another. For example, without departing from the scope of this application, the first dataset may be referred to as the second dataset, and the second dataset may be referred to as the first dataset.

[0050] Example 1

[0051] Current technologies typically employ RGB imaging for monitoring power equipment. However, ordinary RGB imaging is susceptible to light scattering and complex weather conditions, leading to blurred or even lost texture features of the target scene. Secondly, in foggy or other complex weather conditions, atmospheric scattering particles make it difficult for ordinary RGB imaging to detect the detailed texture features of minute components, resulting in low-quality images. Furthermore, traditional image fusion algorithms often require manual feature extraction, relying on human perception to define feature extraction criteria, which lacks generalization ability and is inefficient. Polarization imaging effectively avoids the scattering of light by atmospheric particles, enabling image acquisition even in complex weather conditions. Power equipment exhibits varying surface conductivity due to differences in surface materials. When light strikes the equipment surface, the polarization state of the incident light changes. Compared to ordinary RGB imaging, polarization imaging can capture deeper details on the equipment surface, enabling detailed detection and identification of minute components, eliminating interference from complex weather conditions, and improving the overall quality of the acquired images.

[0052] The fusion technology of polarization degree images and RGB images has gradually become a research hotspot. By acquiring the complementarity of detailed features between polarization degree images and RGB images, and then fusing them, the detailed texture of small parts can be enhanced and the image quality under complex weather conditions can be improved, thereby improving the monitoring effect of power equipment. With the continuous development of image fusion technology, image fusion technology can be divided into three categories: pixel-level fusion, feature-level fusion, and decision-level fusion. Pixel-level fusion directly processes image pixels, ensuring the diversity and losslessness of information; feature-level fusion first needs to extract features from the source image, select useful information, and fuse them along the feature dimension; decision-level fusion needs to identify the source image according to certain rules, and is generally used for weak target detection.

[0053] This application provides an image fusion method for power equipment based on polarization imaging, such as... Figure 1 As shown, the image fusion method includes steps 110 to 140.

[0054] Step 110: Obtain the first dataset of multi-source images of the power equipment, and train the preset autoencoder model based on the first dataset to optimize the autoencoder model. Multi-source images refer to images composed of historical polarization images and historical RGB images with synchronized frame rates. In this embodiment, to meet the requirements of algorithm training, it is first necessary to establish the first dataset corresponding to the multi-source images of the power equipment.

[0055] Preferably, such as Figure 2 As shown, the first dataset of multi-source images of power equipment obtained in step 110 above includes steps 210 to 220.

[0056] Step 210: Acquire historical polarization images and historical RGB images of the power equipment. In this embodiment, a polarization and RGB dual-channel camera deployed on the monitoring robot acquires video, and the two channels synchronously output video signals at the same frame rate. Based on this, historical polarization images and historical RGB images can be acquired separately by acquiring the synchronously output video signals from the dual-channel cameras. It should be understood that the historical RGB image can be directly acquired from the video signal corresponding to the RGB channel output by the polarization and RGB dual-channel camera, while only the polarization image can be directly acquired from the video signal corresponding to the polarization channel output by the polarization and RGB dual-channel camera.

[0057] Preferably, such as Figure 3 As shown, obtaining the historical polarization degree image of the power equipment in step 210 above includes steps 310 to 320.

[0058] Step 310: Obtain polarization degree images of multiple different polarization angles of the power equipment, and obtain the horizontal component and diagonal component of polarized light based on the polarization degree images and the preset Stokes vector.

[0059] Polarization is another important characteristic of light waves, in addition to their three fundamental properties: intensity, phase, and spectrum. During reflection, scattering, and transmission at the surface of an object, light waves acquire different polarization information due to the inherent properties of the surface. Typically, the Stokes vector is used to describe the polarization state of light waves. The Stokes vector can be represented as: stokes = (I, Q, U, V) T In the formula, I represents the illumination intensity, Q represents the linearly polarized light in the horizontal direction (i.e., the horizontal component of the polarized light), and U represents the linearly polarized light in the diagonal direction (i.e., the diagonal component of the polarized light). Since the circularly polarized light component in atmospheric scattering is very small, V is generally not considered.

[0060] When the polarizer makes an angle θ with the X-axis, the illumination intensity of the incident light after passing through the lens is I. θ Stokes' theory of total light intensity is:

[0061]

[0062] When θ takes four angles of 0°, 45°, 90°, and 135°, four components of light intensity can be obtained. The components of polarized light can then be expressed as:

[0063]

[0064] In this embodiment, the video signal directly output by the polarization channel is a polarization degree image of four polarization angles. Therefore, based on the above theory, the horizontal component Q and the diagonal component U of polarized light can be obtained from the polarization degree images of four polarization angles and the Stokes vector.

[0065] Step 320: Obtain historical polarization degree images based on the horizontal component and diagonal component of polarized light.

[0066] After the reflected light passes through the polarizer and forms an image, the linear polarization degree image (DOLP) and polarization angle image (AOP) of the target can be obtained according to the theory of polarized light propagation.

[0067]

[0068] In the formula, 0≤DOLP≤1 represents the proportion of linearly polarized light to the total light intensity, and 0°≤AOP≤180° represents the angle θ between the vibration direction of the polarized light component and the reference direction.

[0069] In this embodiment, based on the above theory, a linear polarization degree image can be obtained from the horizontal component Q and the diagonal component U of the polarized light, that is, a historical polarization degree image can be obtained.

[0070] Step 220: Obtain multi-source images based on historical polarization images and historical RGB images to obtain a first dataset of multi-source images. In this embodiment, the historical polarization images and historical RGB images synchronized with the frame rate are combined to form multi-source images, thereby establishing a first dataset corresponding to the multi-source images of the power equipment.

[0071] After obtaining the first dataset, a pre-defined autoencoder model capable of extracting multi-scale, multi-source image features is trained using data from the first dataset. The autoencoder includes an encoder and a decoder; the pre-defined autoencoder model can be the U-net model.

[0072] The U-Net model is mainly divided into two parts: upsampling and downsampling. Downsampling primarily utilizes consecutive convolutional pooling layers to extract feature information from the image and progressively maps this feature information to a high dimension. The highest dimension of the entire network represents the rich feature information within the entire image. Then, deconvolution is used to map the high-dimensional features back to a lower dimension. During this mapping process, to enhance segmentation accuracy, images with the same dimension in the shrunk network are fused. Since the dimension becomes twice the original dimension during fusion, another convolution is required to ensure that the processed dimension is the same as the dimension before the fusion operation. This allows for a second deconvolution and subsequent concatenation with images of the same dimension, until the final output image achieves the same dimension as the original image.

[0073] Preferably, such as Figure 4As shown, step 110 above, which involves training a preset autoencoder model based on the first dataset to optimize the autoencoder model, includes steps 410 to 430.

[0074] Step 410: Obtain the output image based on the first dataset and the preset autoencoder model.

[0075] Preferably, such as Figure 5 As shown, step 410 above, which obtains the output image based on the first dataset and the preset autoencoder model, includes steps 510 to 520.

[0076] Step 510: Obtain high-dimensional multi-source feature vectors of multi-source images based on the first dataset and the preset autoencoder model.

[0077] Step 520: Obtain the output image based on the multi-source feature vector and the autoencoder model.

[0078] In this embodiment, taking the U-net model as an example of an autoencoder model, the encoder in the autoencoder continuously downsamples to extract image features from the multi-source images corresponding to the first dataset, and maps the image features to a high dimension to reduce the resolution of the image, so as to obtain a high-dimensional multi-source feature vector of the multi-source image; then, through the upsampling structure of the decoder in the autoencoder, the high-dimensional image features are mapped to a low dimension again, and the original features of the multi-source image are gradually restored based on the multi-source feature vector and the image is output.

[0079] Step 420: Calculate the image loss parameters based on the output image, multi-source images, and a preset loss function. In this embodiment, the output image is the predicted image of the power equipment output by the autoencoder model, and the multi-source images are the actual images of the power equipment. The degree of difference between the images can be obtained based on the output image, multi-source images, and the preset loss function, i.e., the image loss parameters are obtained.

[0080] In this embodiment, the preset loss function can be a combination of pixel loss and structural similarity loss to measure the feature extraction and recovery capabilities of the autoencoder.

[0081] Pixel loss can be expressed as:

[0082]

[0083] Structural similarity can be expressed as:

[0084] L ssim =1-SSIM(O,I)

[0085] The loss function can be expressed as:

[0086] L total =Lpixel +L ssim

[0087] In the formula, O represents the output image, and I represents the multi-source image.

[0088] Step 430: Iteratively update the model parameters of the autoencoder model based on the image loss parameters to optimize the autoencoder model.

[0089] Step 120: Obtain a second dataset of real-time polarization images of the power equipment and a third dataset of real-time RGB images. Based on the second dataset, the third dataset, and the optimized autoencoder model, obtain high-dimensional input feature vectors for both images. In this embodiment, a second dataset corresponding to the real-time polarization images of the power equipment and a third dataset corresponding to the real-time RGB images are established. The encoder corresponding to the optimized autoencoder model continuously performs downsampling to extract image features from the real-time polarization images corresponding to the second dataset and image features from the real-time RGB images corresponding to the third dataset, thereby obtaining high-dimensional input feature vectors for the real-time polarization images and real-time RGB images, respectively.

[0090] Step 130: Based on the input feature vectors corresponding to the two images and a preset attention mechanism, obtain the weight vectors of each input feature vector in the spatial and channel dimensions, and the fused feature vector associated with the weight vectors. In this embodiment, the preset attention mechanism refers to the spatial and channel attention mechanism. In the prior art, the fusion strategy is usually implemented based on direct superposition. In this case, it is difficult to flexibly extract the deep features of different source images. This embodiment introduces the spatial and channel attention mechanism to construct a fusion strategy, which can flexibly extract the deep features corresponding to the real-time polarization image and the real-time RGB image respectively.

[0091] Preferably, such as Figure 6 As shown, step 130 above obtains the weight vectors of each input feature vector in the spatial dimension and the channel dimension and the fusion feature vector associated with the weight vectors based on the input feature vectors corresponding to the two images and the preset attention mechanism, including steps 610 to 620.

[0092] Step 610: Obtain the weight vectors of each input feature vector in the spatial dimension and channel dimension based on the input feature vectors corresponding to the two images and the preset attention mechanism. In this embodiment, the introduction of spatial and channel attention mechanisms can obtain the weight vectors of each input feature vector and achieve adaptive updating of the weight vectors. Specifically, if the spatial dimension is compressed to 1 and attention is extracted through the channel dimension, making the input feature vectors corresponding to the two images both 1*64 and the channel dimension size 64, then after processing with the normalization function, the weight vectors corresponding to the two input feature vectors in the channel dimension can be obtained respectively. The weight vectors change with the input feature vectors, and the weight vectors are adaptively updated when the input feature vectors are updated.

[0093] Step 620: Obtain the fused feature vector based on each input feature vector, the weight vectors corresponding to each input feature vector in the spatial and channel dimensions, and the attention mechanism. In this embodiment, the weight vectors corresponding to each input feature vector in the spatial and channel dimensions are assigned to the corresponding input feature vectors to obtain the fused feature vector. When the weight vectors are adaptively updated, effective fusion of adaptive multi-scale features is achieved.

[0094] In this embodiment, the fusion strategy does not require parameter training. It simply performs attention calculations in the spatial and channel dimensions based on the input feature vectors and assigns the generated attention map as a weight vector to the corresponding input feature vectors. This can effectively enhance the feature advantages of multi-source data, thereby improving the image fusion effect.

[0095] Preferably, the fused feature vector can be obtained using the following formula:

[0096]

[0097] In the formula, and Let α be the input feature vector of the real-time polarization degree image and the input feature vector of the real-time RGB image, respectively. p β p α RGB and β RGB These are the weight vectors corresponding to the real-time polarization image and the real-time RGB image in the spatial and channel dimensions, respectively. and These are the feature vectors output by the attention mechanism based on the spatial dimension and the channel dimension, respectively. To fuse feature vectors.

[0098] Step 140: Obtain the fused image of the power equipment based on the fused feature vector and the optimized autoencoder model. In this embodiment, the high-dimensional image features are mapped to low dimensions through the upsampling structure of the decoder corresponding to the optimized autoencoder model. The original features of the image are gradually recovered based on the fused feature vector, and the fused image of the power equipment is output.

[0099] In this embodiment, firstly, a first dataset of multi-source images composed of historical polarization images and historical RGB images of the power equipment is acquired, and a preset autoencoder model is trained and optimized based on the first dataset. Next, a second dataset of real-time polarization images and a third dataset of real-time RGB images of the power equipment are acquired, and high-dimensional input feature vectors for the two images are obtained based on the second dataset, the third dataset, and the optimized autoencoder model. Further, based on the input feature vectors corresponding to the two images and a preset attention mechanism, weight vectors for each input feature vector in the spatial and channel dimensions, as well as fusion feature vectors associated with the weight vectors, are obtained. Spatial and channel attention mechanisms are introduced into the autoencoder fusion strategy to achieve adaptive updating of the weight vectors and realize effective fusion of adaptive multi-scale features. Finally, a fused image of the power equipment is obtained based on the fused feature vectors and the optimized autoencoder model. By employing an autoencoder network structure and introducing spatial and channel attention mechanisms to construct a fusion strategy, polarization degree images and RGB images are fused to obtain fused images of power equipment. This achieves the complementary advantages of polarization degree images and RGB images, provides multi-dimensional feature information, improves the quality of the acquired images, and is conducive to improving the monitoring effect of power equipment. It also facilitates all-weather real-time monitoring of power equipment and accurate fault location.

[0100] It should be understood that, although the flowcharts involved in the embodiments described above are... Figures 1-6 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, the flowcharts involved in the embodiments described above... Figures 1-6 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0101] Example 2

[0102] Based on the same inventive concept, this application also provides an image fusion apparatus for implementing the image fusion method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image fusion apparatus embodiments provided below can be found in the limitations of the image fusion method described above, and will not be repeated here.

[0103] This application also provides an image fusion apparatus, such as... Figure 7 As shown, the image fusion device 10 includes a model optimization module 11, an input vector acquisition module 12, a fusion vector acquisition module 13, and a fused image acquisition module 14. The model optimization module 11 acquires a first dataset of multi-source images of the power equipment and trains a preset autoencoder model based on the first dataset to optimize the autoencoder model. The multi-source images refer to images composed of historical polarization images and historical RGB images with synchronized frame rates. The input vector acquisition module 12 acquires a second dataset of real-time polarization images and a third dataset of real-time RGB images of the power equipment, and acquires high-dimensional input feature vectors for the two images based on the second dataset, the third dataset, and the optimized autoencoder model. The fusion vector acquisition module 13 acquires weight vectors for each input feature vector in the spatial and channel dimensions, and fusion feature vectors associated with the weight vectors, based on the corresponding input feature vectors of the two images and a preset attention mechanism. The fused image acquisition module 14 acquires a fused image of the power equipment based on the fused feature vectors and the optimized autoencoder model.

[0104] Preferably, the model optimization module 11 includes an image acquisition unit and a dataset acquisition unit. The image acquisition unit is used to acquire historical polarization images and historical RGB images of the power equipment. The dataset acquisition unit is used to acquire multi-source images based on the historical polarization images and historical RGB images to obtain a first dataset of multi-source images.

[0105] Preferably, the image acquisition unit is further configured to acquire polarization degree images of multiple different polarization angles of the power equipment, and acquire the horizontal component and diagonal component of polarized light based on the polarization degree images and a preset Stokes vector; and acquire historical polarization degree images based on the horizontal component and diagonal component of polarized light.

[0106] Preferably, the model optimization module 11 further includes an output image acquisition unit, a loss parameter acquisition unit, and a model optimization unit. The output image acquisition unit is used to acquire an output image based on the first dataset and a preset autoencoder model. The loss parameter acquisition unit is used to perform loss calculation based on the output image, multi-source images, and a preset loss function to obtain image loss parameters. The model optimization unit is used to iteratively update the model parameters of the autoencoder model based on the image loss parameters to optimize the autoencoder model.

[0107] Preferably, the output image acquisition unit is further configured to acquire high-dimensional multi-source feature vectors of the multi-source image based on the first dataset and the preset autoencoder model; and acquire the output image based on the multi-source feature vectors and the autoencoder model.

[0108] Preferably, the aforementioned fusion vector acquisition module 13 includes a weight vector acquisition unit and a fusion vector acquisition unit. The weight vector acquisition unit is used to acquire the weight vectors of each input feature vector in the spatial dimension and the channel dimension, respectively, based on the input feature vectors corresponding to the two images and a preset attention mechanism. The fusion vector acquisition unit is used to acquire the fusion feature vector based on each input feature vector, the weight vectors corresponding to each input feature vector in the spatial dimension and the channel dimension, and the attention mechanism.

[0109] Each module in the image fusion device 10 described above can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0110] Example 3

[0111] This application also provides a computer device, such as... Figure 8 As shown, it includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described image fusion method.

[0112] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0113] Example 4

[0114] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described image fusion method.

[0115] Example 5

[0116] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described image fusion method.

[0117] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0118] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image fusion method, characterized by, The method includes: A first dataset of multi-source images of power equipment is acquired, and a preset autoencoder model is trained based on the first dataset to optimize the autoencoder model. The multi-source images refer to images composed of historical polarization images and historical RGB images with frame rate synchronization. A second dataset of real-time polarization images of power equipment and a third dataset of real-time RGB images are obtained, and high-dimensional input feature vectors of the two images are obtained based on the second dataset, the third dataset, and the optimized autoencoder model, respectively. The weight vectors of each input feature vector in the spatial dimension and the channel dimension are obtained based on the input feature vectors corresponding to the two images and the preset attention mechanism, respectively. The fused feature vector is obtained based on each input feature vector, the weight vector corresponding to each input feature vector in the spatial dimension and the channel dimension, and the attention mechanism. The fused feature vector is obtained using the following formula: In the formula, and These are the input feature vectors of the real-time polarization image and the real-time RGB image, respectively. , , and These are the weight vectors corresponding to the real-time polarization image and the real-time RGB image in the spatial and channel dimensions, respectively. and These are the feature vectors output by the attention mechanism based on the spatial dimension and the channel dimension, respectively. The fused feature vector; The fused image of the power equipment is obtained based on the fused feature vector and the optimized autoencoder model.

2. The method of claim 1, wherein, The first dataset for acquiring multi-source images of power equipment includes: Acquire the historical polarization image and the historical RGB image of the power equipment; The multi-source image is obtained based on the historical polarization image and the historical RGB image to obtain the first dataset of the multi-source image.

3. The method of claim 2, wherein, The acquisition of the historical polarization image of the power equipment includes: The polarization degree images of the power equipment at multiple different polarization angles are acquired, and the horizontal component and diagonal component of the polarized light are obtained based on the polarization degree images and the preset Stokes vector. The historical polarization image is obtained based on the horizontal component and the diagonal component of the polarized light.

4. The method of claim 1, wherein, The step of training a preset autoencoder model based on the first dataset to optimize the autoencoder model includes: The output image is obtained based on the first dataset and the preset autoencoder model; Image loss parameters are obtained by performing loss calculation based on the output image, the multi-source image, and a preset loss function; The model parameters of the autoencoder model are iteratively updated based on the image loss parameters to optimize the autoencoder model.

5. The method of claim 4, wherein, The step of obtaining the output image based on the first dataset and the preset autoencoder model includes: The high-dimensional multi-source feature vector of the multi-source image is obtained based on the first dataset and the preset autoencoder model; The output image is obtained based on the multi-source feature vector and the autoencoder model.

6. An image fusion apparatus characterized by comprising: The device includes: The model optimization module is used to acquire a first dataset of multi-source images of power equipment and train a preset autoencoder model based on the first dataset to optimize the autoencoder model. The multi-source images refer to images composed of historical polarization images and historical RGB images with frame rate synchronization. The input vector acquisition module is used to acquire a second dataset of real-time polarization degree images of power equipment and a third dataset of real-time RGB images, and to acquire high-dimensional input feature vectors of the two images based on the second dataset, the third dataset, and the optimized autoencoder model, respectively. The fusion vector acquisition module is used to acquire the weight vectors of each input feature vector in the spatial dimension and the channel dimension according to the input feature vectors corresponding to the two images and a preset attention mechanism; and to acquire the fusion feature vector according to each input feature vector, the weight vectors corresponding to each input feature vector in the spatial dimension and the channel dimension and the attention mechanism. The fused feature vector is obtained using the following formula: wherein, and are the input feature vectors of the real-time polarization degree image and the real-time RGB image, respectively, , , and are the weight vectors corresponding to the real-time polarization degree image and the real-time RGB image in the spatial dimension and the channel dimension, respectively, and are the feature vectors output by the attention mechanism based on the spatial dimension and the channel dimension, respectively, is the fusion feature vector The fused image acquisition module is used to acquire a fused image of the power equipment based on the fused feature vector and the optimized autoencoder model. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images