Image fusion method, device and equipment, readable storage medium and program product
By employing image fusion and feature vector dimensionality reduction compression techniques, the problem of large storage space for multimodal images of power equipment was solved, enabling efficient monitoring and prediction of power equipment status.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-07
AI Technical Summary
In existing power equipment inspection and fault diagnosis, the storage space requirements for multimodal image data are large, making it difficult to effectively conduct long-term monitoring.
By converting images captured in different modalities into grayscale images and mapping them to the channels of a preset color model, a fused image is generated. The feature vector is then extracted using a convolutional neural network and compressed for storage, reducing storage requirements.
It enables comprehensive condition monitoring and prediction of power equipment while reducing storage space, providing long-term monitoring and early warning functions.
Smart Images
Figure CN121810501A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of power equipment condition monitoring technology, and in particular to an image fusion method, apparatus, device, readable storage medium, and program product. Background Technology
[0002] Inspecting and diagnosing faults in various power equipment in the power grid is the cornerstone of ensuring the safe and stable operation of the power grid. Through regular or real-time monitoring, potential defects such as overheating, mechanical damage, and insulation aging in power equipment can be proactively detected, thereby enabling timely warnings and handling before faults occur, effectively reducing the operation and maintenance costs of power equipment.
[0003] However, current power equipment inspection and fault diagnosis usually require the collection of multiple image data of different modes of power equipment. However, when long-term monitoring of power equipment is required, the storage space required to store multiple image data is large. Summary of the Invention
[0004] Therefore, it is necessary to provide an image fusion method, apparatus, device, readable storage medium, and program product to address the aforementioned technical problems.
[0005] In a first aspect, this application provides an image fusion method, the method comprising:
[0006] Acquire images of power equipment in different modes; wherein, the images captured in different modes are used to reflect different characteristic information of the power equipment;
[0007] Each modal image is converted into a corresponding grayscale image.
[0008] The grayscale image is mapped to each channel of a preset color model to obtain a fused image of the power equipment.
[0009] In one embodiment, the preset color model is an RGB color model. The grayscale image is mapped to each channel of the preset color model to obtain a fused image of the power equipment, including:
[0010] The grayscale image is mapped to each channel of the RGB color model to obtain the fused image.
[0011] In one embodiment, the captured images of different modalities include infrared images, acoustic images, and visible light images; the grayscale images are respectively mapped to the channels of the RGB color model to obtain a fused image, including:
[0012] Map the grayscale image corresponding to the infrared image to the R channel;
[0013] Map the grayscale image corresponding to the visible light image to the G channel;
[0014] Map the grayscale image corresponding to the acoustic image to the B channel.
[0015] In one embodiment, the method further includes:
[0016] The feature vector of the fused image is extracted using a convolutional neural network;
[0017] The feature vectors are dimensionality-reduced and compressed, and then stored in a preset feature database to store the operating status of the power equipment.
[0018] In one embodiment, the method further includes:
[0019] Obtain feature vectors corresponding to power equipment at different times from a preset feature database;
[0020] By using feature vectors at different times, the operating status information of power equipment is predicted, and the state information prediction result is obtained.
[0021] In one embodiment, the method further includes:
[0022] Acquire images of different modalities from a multimodal imaging device; the multimodal imaging device is an imaging device with infrared imaging, visible light imaging, and acoustic imaging capabilities.
[0023] Secondly, this application also provides an image fusion apparatus, which includes:
[0024] The acquisition module is used to acquire images of the power equipment in different modes; the images in different modes are used to reflect different feature information of the power equipment.
[0025] The conversion module is used to convert the captured images of each mode into the corresponding grayscale images;
[0026] The mapping module is used to map the grayscale image to each channel of the preset color model to obtain the fused image of the power equipment.
[0027] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any one of the first aspects above.
[0028] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects above.
[0029] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects above.
[0030] The aforementioned image fusion method, apparatus, device, readable storage medium, and program product first acquire images of power equipment captured in different modalities, where the images in different modalities reflect different feature information of the power equipment. Then, each modal image is converted into a corresponding grayscale image. Finally, the grayscale images are mapped to the channels of a preset color model to obtain a fused image of the power equipment. In this way, images captured in different modalities of power equipment can be fused to obtain a fused image. This eliminates the need to store multiple images of different modalities; only the fused image needs to be stored, reducing storage space requirements in fields requiring long-term monitoring of power equipment. Attached Figure Description
[0031] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0032] Figure 1 This is a flowchart illustrating an image fusion method in one embodiment;
[0033] Figure 2 This is a flowchart illustrating the color model channel mapping steps in one embodiment;
[0034] Figure 3 This is a flowchart illustrating the image feature extraction step in one embodiment;
[0035] Figure 4 This is a flowchart illustrating the power equipment status information prediction step in one embodiment;
[0036] Figure 5 This is a flowchart illustrating the image fusion method in another embodiment;
[0037] Figure 6 This is a structural block diagram of an image fusion device in one embodiment;
[0038] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0040] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0041] In one embodiment, such as Figure 1 As shown, an image fusion method is provided. This embodiment illustrates the method applied to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:
[0042] Step 101: Acquire images of the power equipment in different modes.
[0043] Different modalities of image capture are used to reflect different characteristic information of power equipment. To gain a more comprehensive understanding of the operating status of power equipment, it is necessary to acquire images of the power equipment in multiple imaging modalities. These images from different modalities can reflect the key characteristic information and operating status information of the power equipment from multiple dimensions.
[0044] For example, visible light images of power equipment can intuitively display the appearance features and surface condition of the equipment, thus enabling the detection of surface defects. Infrared thermal imaging images of power equipment can reflect the internal temperature distribution, thereby enabling the detection of potential overheating hazards. Ultraviolet imaging images of power equipment can detect abnormal electric field distributions such as corona discharge and insulation problems. Acoustic imaging images of power equipment can reflect the spatial distribution of sound source intensity, thereby enabling the detection of potential hazards such as insulation degradation or mechanical structural abnormalities. Multispectral imaging of power equipment can detect early chemical degradation problems such as aging of oil-paper insulation. Ultrasonic imaging of power equipment can detect potential hazards such as abnormal deformation of the mechanical structure. It is understood that different modalities of image capture can be selected according to the actual monitoring needs of the power equipment, and this application embodiment does not impose any limitations on this.
[0045] Step 102: Convert the captured images of each modality into corresponding grayscale images.
[0046] To simplify data dimensions and ensure that images from different modalities maintain the same format in subsequent processing, each modal image can be converted into a corresponding grayscale image. In a grayscale image, each pixel contains only brightness information, using different shades of gray to represent variations in brightness. Each pixel is typically represented by an 8-bit binary number (an integer between 0 and 255), where 0 represents pure black, 255 represents pure white, and values in between correspond to different levels of gray. In this embodiment, each modal image can be converted into a grayscale image with a resolution of 640*480.
[0047] For example, in an infrared thermal imaging image, each pixel corresponds to a temperature value. By setting a preset temperature range, the temperature range can be linearly mapped to a grayscale range of 0-255. For instance, the lowest temperature in the temperature range is displayed as pure black (0), and the highest temperature in the temperature range is displayed as pure white (255). Then, the temperature corresponding to each pixel is displayed as different shades of gray according to the proportion, generating a grayscale image corresponding to the infrared thermal imaging image, which can reflect the temperature distribution of the power equipment.
[0048] For example, for visible light images, the brightness value can be calculated by assigning different weights to the red, green, and blue channel values of each pixel in the visible light image, and then linearly mapped to a range of 0-255 to generate a grayscale image corresponding to the visible light image. Alternatively, for acoustic imaging images, the sound pressure level (SPL) value corresponding to each pixel in the image can be determined, linearly mapped to a grayscale range of 0-255, with the lowest SPL displayed as black and the highest as white. Then, the temperature corresponding to each pixel can be displayed as different shades of gray according to a proportional ratio, which can reflect the location of the sound source in electrical equipment.
[0049] Step 103: Map the grayscale image to each channel of the preset color model to obtain the fused image of the power equipment.
[0050] The preset color models can be RGB (Red Green Blue), HSV (Hue Saturation Value / Lightness), Lab (L*Lightness, a*(green-red axis), b*(blue-yellow axis), or CMYK (Cyan Magenta Yellow Key (Black)). RGB, HSV, and Lab color models each have 3 channels, while CMYK has 4 channels.
[0051] The images captured using different modalities are mapped to different channels of a preset color model. For example, if the images include ultraviolet (UV) images, ultrasonic images, and visible light images, and the preset color model is the HSV color model, the grayscale image corresponding to the UV image can be mapped to the H channel of the HSV color model, the grayscale image corresponding to the ultrasonic image can be mapped to the S channel, and the grayscale image corresponding to the visible light image can be mapped to the V channel. This results in a fused image of the power equipment, which includes image features from UV, ultrasonic, and visible light imaging within a single image, enabling more comprehensive condition monitoring of the power equipment. It is understood that the images captured using different modalities can include images from other modalities, and the preset color model can also be other types of color models, which can be selected according to the actual situation.
[0052] In the above embodiments, firstly, images of the power equipment in different modalities are acquired, where the images in different modalities reflect different feature information of the power equipment. Then, each modal image is converted into a corresponding grayscale image. Finally, the grayscale images are mapped to the channels of a preset color model to obtain a fused image of the power equipment. In this way, images of the power equipment in different modalities can be fused to obtain a fused image. This eliminates the need to store multiple images of different modalities; only the fused image needs to be stored, reducing storage space requirements in fields requiring long-term monitoring of power equipment.
[0053] In one embodiment of this application, the preset color model is an RGB color model. The grayscale image is mapped to each channel of the preset color model to obtain a fused image of the power equipment. This includes mapping the grayscale image to each channel of the RGB color model to obtain a fused image.
[0054] Optionally, currently, images captured in different modalities typically originate from different devices. This can lead to significant difficulties in later fusion and analysis because the field of view and capture time are often inconsistent across different devices. Therefore, to ensure that multiple images of different modalities can be simultaneously acquired from the same field of view during a single acquisition of power equipment, this embodiment acquires images of different modalities from a multimodal imaging device. This multimodal imaging device is equipped with infrared imaging, visible light imaging, and acoustic imaging capabilities. In this way, the images of the three modalities are captured by the same device at the same time, ensuring a consistent field of view and facilitating image fusion and analysis.
[0055] Therefore, images acquired by multimodal imaging devices in different modalities include infrared images, acoustic images, and visible light images; grayscale images are then mapped to the respective channels of the RGB color model to obtain a fused image, such as... Figure 2 As shown, it may include:
[0056] Step 201: Map the grayscale image corresponding to the infrared image to the R channel.
[0057] In this process, the grayscale image corresponding to the infrared image is mapped to the R channel, which means that each pixel value in the grayscale image corresponding to the infrared image is used as the corresponding pixel value in the R channel of the RGB color model.
[0058] Step 202: Map the grayscale image corresponding to the visible light image to the G channel.
[0059] In this process, the grayscale image corresponding to the visible light image is mapped to the G channel, which means that each pixel value in the grayscale image corresponding to the visible light image is used as the corresponding pixel value in the G channel of the RGB color model.
[0060] Step 203: Map the grayscale image corresponding to the acoustic imaging image to the B channel.
[0061] In this process, the grayscale image corresponding to the acoustic imaging image is mapped to the B channel, which means that each pixel value in the grayscale image corresponding to the acoustic imaging image is used as the corresponding pixel value in the B channel of the RGB color model.
[0062] The above method enables the fusion of three grayscale images with different modalities into a single RGB color image with multi-source sensing capabilities. The fused image simultaneously contains thermal field, acoustic field, and structural information, significantly enhancing its feature representation capabilities.
[0063] In the embodiments of this application, such as Figure 3 As shown, the method also includes:
[0064] Step 301: Extract the feature vector of the fused image using a convolutional neural network.
[0065] Optionally, the convolutional neural network can be trained using a VGG16 convolutional neural network and may include convolutional layers, pooling layers, and fully connected layers. The fused image is input into the VGG16 convolutional neural network, where convolutional layers extract feature information. Pooling layers are interspersed within the convolutional layers to achieve data compression, gradually reducing the feature map size. The fully connected layers synthesize all extracted feature information to obtain a fixed-length high-dimensional feature vector. In this embodiment, the feature vector can be 4096-dimensional.
[0066] Step 302: Perform dimensionality reduction and compression on the feature vectors, and store the dimensionality reduction and compression feature vectors in a preset feature database to store the operating status of the power equipment.
[0067] The feature vectors extracted from the above fused images are subjected to dimensionality reduction using PCA (Principal Component Analysis). This reduces the number of data features while preserving key information as much as possible. Furthermore, the dimensionality-reduced feature vectors are further compressed and stored in a preset feature database, significantly reducing the amount of data. This data can be used for long-term archiving, trend analysis, and intelligent comparison of the operating status of power equipment.
[0068] In the above embodiments, feature vectors of the fused images are extracted and then compressed and stored in a preset feature database without saving the original image data of different modalities. Since the amount of compressed feature vector data is much smaller than the original image data of different modalities, it is more suitable for scenarios that require long-term monitoring of power equipment and reduces the amount of data storage occupied.
[0069] In one embodiment, such as Figure 4 As shown, the method also includes:
[0070] Step 401: Obtain the feature vectors corresponding to power equipment at different times from the preset feature database.
[0071] To achieve automatic inspection of power equipment, feature vectors corresponding to power equipment at different times and under different conditions can be periodically acquired from a preset feature database. Based on the changes in the feature vectors, it can be determined whether there are any abnormalities in the power equipment, thus realizing regular monitoring of the status of the power equipment.
[0072] Step 402: Using feature vectors at different times, predict the operating status information of the power equipment to obtain the status information prediction result.
[0073] Based on the feature vectors at different times, the changes in the feature vectors can be determined. For example, by calculating the cosine similarity of the feature vectors at different times and setting corresponding change thresholds, the changes in the feature vectors can be judged based on the comparison results of the cosine similarity and the change thresholds. This reflects the state changes of the power equipment and enables the analysis of the change trends of the power equipment.
[0074] Optionally, a time series can be constructed by building feature vectors at different times, and then modeled using sequence models such as LSTM (Long Short-Term Memory) to obtain a state prediction model. Based on the state prediction model, the future feature vectors of the power equipment can be predicted from the historical feature vectors of the power equipment, thereby achieving accurate prediction and early warning of the operating status of the power equipment, providing core decision support for predictive maintenance, and realizing intelligent operation and maintenance of power equipment.
[0075] In the above embodiments, by analyzing the feature vectors of power equipment at different times, the operating status of the power equipment can be predicted, and the state information prediction results can be obtained. The state information prediction results can be displayed to maintenance personnel through the terminal, which can help maintenance personnel determine the changing trend of the power equipment and identify potential safety hazards in advance.
[0076] In embodiments of this application, an image fusion method is provided, such as... Figure 5 As shown, the steps include:
[0077] Step 501: Acquire images of different modalities from the multimodal imaging device.
[0078] Step 502: Convert the captured images of each modality into corresponding grayscale images.
[0079] Step 503: Map the grayscale image corresponding to the infrared image to the R channel.
[0080] Step 504: Map the grayscale image corresponding to the visible light image to the G channel.
[0081] Step 505: Map the grayscale image corresponding to the acoustic imaging image to the B channel to obtain the fused image of the power equipment.
[0082] Step 506: Extract the feature vector of the fused image using a convolutional neural network.
[0083] Step 507: Perform dimensionality reduction and compression on the feature vectors, and store the dimensionality reduction and compression feature vectors in a preset feature database to store the operating status of the power equipment.
[0084] Step 508: Obtain the feature vectors corresponding to power equipment at different times from the preset feature database.
[0085] Step 509: Using feature vectors at different times, predict the operating status information of the power equipment to obtain the status information prediction result.
[0086] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0087] Based on the same inventive concept, this application also provides an image fusion apparatus for implementing the image fusion method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image fusion apparatus embodiments provided below can be found in the limitations of the image fusion method described above, and will not be repeated here.
[0088] In one exemplary embodiment, such as Figure 6 As shown, an image fusion device 600 is provided, including: an acquisition module 601, a conversion module 602, and a mapping module 603, wherein:
[0089] The acquisition module 601 is used to acquire images of the power equipment in different modes; wherein the images in different modes are used to reflect different feature information of the power equipment.
[0090] The conversion module 602 is used to convert the captured images of each mode into corresponding grayscale images respectively;
[0091] The mapping module 603 is used to map the grayscale image to each channel of the preset color model to obtain the fused image of the power equipment.
[0092] In one embodiment, the preset color model is the RGB color model. The mapping module 603 is specifically used to map the grayscale image to each channel of the RGB color model to obtain the fused image.
[0093] In one embodiment, the captured images of different modalities include infrared images, acoustic images, and visible light images; the mapping module 603 is specifically used to map the grayscale image corresponding to the infrared image to the R channel; to map the grayscale image corresponding to the visible light image to the G channel; and to map the grayscale image corresponding to the acoustic image to the B channel.
[0094] In one embodiment, the device further includes an extraction module for extracting feature vectors from the fused image using a convolutional neural network; performing dimensionality reduction and compression processing on the feature vectors; and storing the dimensionality reduction and compression processing feature vectors in a preset feature database to store the operating status of the power equipment.
[0095] In one embodiment, the device further includes a prediction module, which is used to obtain feature vectors corresponding to power equipment at different times from a preset feature database; and to use the feature vectors at different times to predict the operating status information of the power equipment to obtain the status information prediction result.
[0096] In one embodiment, the acquisition module 601 is specifically used to acquire images of different modes from a multimodal imaging device; the multimodal imaging device is an imaging device with infrared imaging function, visible light imaging function and acoustic imaging function.
[0097] Each module in the aforementioned image fusion device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0098] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an image fusion method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.
[0099] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0100] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring images of a power device in different modes; wherein the images of different modes are used to reflect different feature information of the power device; converting the images of each mode into corresponding grayscale images; and mapping the grayscale images to each channel of a preset color model to obtain a fused image of the power device.
[0101] In one embodiment, the preset color model is the RGB color model, and when the processor executes the computer program, it further performs the following steps: mapping the grayscale image to each channel of the RGB color model to obtain a fused image.
[0102] In one embodiment, the captured images of different modalities include infrared images, acoustic images, and visible light images; when the processor executes the computer program, it also performs the following steps: mapping the grayscale image corresponding to the infrared image to the R channel; mapping the grayscale image corresponding to the visible light image to the G channel; and mapping the grayscale image corresponding to the acoustic image to the B channel.
[0103] In one embodiment, when the processor executes the computer program, it further performs the following steps: extracting feature vectors of the fused image through a convolutional neural network; performing dimensionality reduction and compression processing on the feature vectors; and storing the dimensionality reduction and compression processing feature vectors in a preset feature database to store the operating status of the power equipment.
[0104] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining feature vectors corresponding to power equipment at different times from a preset feature database; using the feature vectors at different times to predict the operating status information of the power equipment, and obtaining the status information prediction result.
[0105] In one embodiment, the processor, when executing the computer program, further performs the following steps: acquiring images of different modalities from a multimodal imaging device; the multimodal imaging device is an imaging device with infrared imaging capabilities, visible light imaging capabilities, and acoustic imaging capabilities.
[0106] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: acquiring images of a power device in different modes; wherein the images of different modes are used to reflect different feature information of the power device; converting the images of each mode into corresponding grayscale images; and mapping the grayscale images to each channel of a preset color model to obtain a fused image of the power device.
[0107] In one embodiment, the preset color model is the RGB color model, and when the computer program is executed by the processor, it further performs the following steps: mapping the grayscale image to each channel of the RGB color model to obtain a fused image.
[0108] In one embodiment, the captured images of different modalities include infrared images, acoustic images, and visible light images; when the computer program is executed by the processor, it also performs the following steps: mapping the grayscale image corresponding to the infrared image to the R channel; mapping the grayscale image corresponding to the visible light image to the G channel; and mapping the grayscale image corresponding to the acoustic image to the B channel.
[0109] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: extracting feature vectors of the fused image through a convolutional neural network; performing dimensionality reduction and compression processing on the feature vectors; and storing the dimensionality reduction and compression processing feature vectors in a preset feature database to store the operating status of the power equipment.
[0110] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining feature vectors corresponding to power equipment at different times from a preset feature database; using the feature vectors at different times to predict the operating state information of the power equipment, and obtaining the state information prediction result.
[0111] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring images of different modalities from a multimodal imaging device; the multimodal imaging device is an imaging device with infrared imaging function, visible light imaging function and acoustic imaging function.
[0112] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: acquiring images of a power device in different modes; wherein the images in different modes are used to reflect different feature information of the power device; converting each mode of image into a corresponding grayscale image; and mapping the grayscale images to each channel of a preset color model to obtain a fused image of the power device.
[0113] In one embodiment, the preset color model is the RGB color model, and when the computer program is executed by the processor, it further performs the following steps: mapping the grayscale image to each channel of the RGB color model to obtain a fused image.
[0114] In one embodiment, the captured images of different modalities include infrared images, acoustic images, and visible light images; when the computer program is executed by the processor, it also performs the following steps: mapping the grayscale image corresponding to the infrared image to the R channel; mapping the grayscale image corresponding to the visible light image to the G channel; and mapping the grayscale image corresponding to the acoustic image to the B channel.
[0115] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: extracting feature vectors of the fused image through a convolutional neural network; performing dimensionality reduction and compression processing on the feature vectors; and storing the dimensionality reduction and compression processing feature vectors in a preset feature database to store the operating status of the power equipment.
[0116] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining feature vectors corresponding to power equipment at different times from a preset feature database; using the feature vectors at different times to predict the operating state information of the power equipment, and obtaining the state information prediction result.
[0117] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring images of different modalities from a multimodal imaging device; the multimodal imaging device is an imaging device with infrared imaging function, visible light imaging function and acoustic imaging function.
[0118] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0119] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0120] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0121] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. An image fusion method, characterized in that, The method includes: Images of power equipment captured in different modes are acquired; wherein the images captured in different modes are used to reflect different feature information of the power equipment. Each modal image is converted into a corresponding grayscale image. The grayscale image is mapped to each channel of a preset color model to obtain a fused image of the power equipment.
2. The method according to claim 1, characterized in that, The preset color model is an RGB color model. The step of mapping the grayscale image to each channel of the preset color model to obtain the fused image of the power equipment includes: The grayscale image is mapped to each channel of the RGB color model to obtain the fused image.
3. The method according to claim 2, characterized in that, The captured images of different modalities include infrared images, acoustic images, and visible light images; the process of mapping the grayscale images to the respective channels of the RGB color model to obtain the fused image includes: Map the grayscale image corresponding to the infrared image to the R channel; Map the grayscale image corresponding to the visible light image to the G channel; The grayscale image corresponding to the acoustic imaging image is mapped to the B channel.
4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The feature vector of the fused image is extracted using a convolutional neural network; The feature vector is subjected to dimensionality reduction and compression, and the dimensionality reduction and compression feature vector is stored in a preset feature database to store the operating status of the power equipment.
5. The method according to claim 4, characterized in that, The method further includes: From the preset feature database, obtain the feature vectors corresponding to the power equipment at different times; By using the feature vectors at different times, the operating status information of the power equipment is predicted, and the status information prediction result is obtained.
6. The method according to claim 3, characterized in that, The method further includes: Images of different modes are acquired from a multimodal imaging device; the multimodal imaging device is an imaging device with infrared imaging function, visible light imaging function and acoustic imaging function.
7. An image fusion apparatus, characterized in that, The device includes: The acquisition module is used to acquire images of the power equipment in different modes; wherein the images in different modes are used to reflect different feature information of the power equipment. The conversion module is used to convert the captured images of each mode into the corresponding grayscale images; The mapping module is used to map the grayscale image to each channel of a preset color model to obtain a fused image of the power equipment.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.