Power equipment multi-source image fusion diagnosis method, device, equipment and medium

Abstract

The application relates to a power equipment multi-source image fusion diagnosis method, device, equipment and medium. The method comprises the following steps: acquiring image data of power equipment collected by at least one image collector; performing image fusion processing on different image data of the power equipment to obtain fused image data; predicting fault probabilities of different faults of the power equipment based on abnormal feature data in different dimensions in the fused image data; and selecting a fault with the highest probability fault as a device fault existing in the power equipment. The method can improve the device fault detection efficiency.

Description

Technical Field

[0001] This application relates to the field of computer equipment technology, and in particular to a method, apparatus, device and medium for multi-source image fusion diagnosis of power equipment. Background Technology

[0002] With the development of image recognition and intelligent detection technologies, multimodal machine vision detection technology has emerged. This technology can simultaneously acquire image data from multiple modalities such as visible light, infrared thermal imaging, and ultraviolet imaging, reflecting the operating status of power equipment from multiple dimensions such as physical morphology, thermal state, and electrical discharge state, thus promoting the upgrading of power equipment detection towards visualization and intelligence.

[0003] In traditional technologies, power equipment inspection mainly relies on manual inspection or single sensor monitoring. Manual inspection depends on the experience of maintenance personnel, which is inefficient, subjective, and difficult to detect early hidden faults. Single sensor monitoring (such as infrared thermometry or ultraviolet detection) can achieve local state quantification, but it can only obtain information about a certain aspect of the equipment and cannot fully reflect the equipment status.

[0004] However, existing multimodal image fusion methods have not solved the problems of low spatial registration accuracy and high noise interference, resulting in poor fused image quality. This leads to insufficient accuracy in fault diagnosis and positioning, failing to meet the needs of refined and intelligent fault detection for power equipment. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, device, and medium for multi-source image fusion diagnosis of power equipment that can improve fault detection efficiency in response to the above-mentioned technical problems.

[0006] In a first aspect, this application provides a multi-source image fusion diagnostic method for power equipment, including:

[0007] Acquire image data of power equipment collected by at least one image acquisition device;

[0008] Image fusion processing is performed on different image data of power equipment to obtain fused image data;

[0009] Based on the abnormal feature data of different dimensions in the fused image data, the failure probability of power equipment to produce different faults is predicted.

[0010] The fault with the highest probability is selected as the equipment fault in the power equipment.

[0011] In one embodiment, image fusion processing is performed on different image data of the power equipment to obtain fused image data, including:

[0012] For each image data, low-frequency subband coefficients and high-frequency subband coefficients in at least one direction of the power equipment are extracted from the image data; the low-frequency subband coefficients are used to characterize the edge features of the power equipment; the high-frequency subband coefficients are used to characterize the visual features of the power equipment in the edge orientation.

[0013] The low-frequency subband coefficients in different image data are fused to obtain fused low-frequency subband coefficients; and, for each direction, the high-frequency subband coefficient with the largest regional contrast is selected from different image data as the target high-frequency subband coefficient in the corresponding direction; the regional contrast is used to characterize the prominence of the high-frequency subband coefficient of the power equipment at the corresponding image location relative to the image background.

[0014] The fused low-frequency subband coefficients and the target high-frequency subband coefficients in different directions are inversely transformed to obtain fused image data.

[0015] In one embodiment, the low-frequency subband coefficients in different image data are fused to obtain fused low-frequency subband coefficients, including:

[0016] For each image data set, obtain the edge smoothness and edge sharpness of the power equipment in the image data;

[0017] Image weights are determined based on the product of edge smoothness and edge sharpness of the image data.

[0018] The low-frequency subband coefficients in different image data and the corresponding image weights are weighted and fused to obtain the fused low-frequency subband coefficients.

[0019] In one embodiment, the process of determining the regional contrast of the high-frequency subband coefficients includes:

[0020] For each edge orientation of the power equipment in the image data, determine the standard deviation and mean of the high-frequency sub-band coefficients at each edge orientation;

[0021] The regional contrast of the high-frequency subband coefficients with upward-facing edges is determined based on the ratio between the standard deviation and the mean.

[0022] In one embodiment, based on anomalous feature data from different dimensions in the fused image data, the probability of different faults occurring in power equipment is predicted, including:

[0023] Acquire historical anomaly characteristic data of power equipment under different dimensions within a historical period;

[0024] For each dimension, the degree of abnormal deviation of the power equipment in the dimension is determined based on the difference between the abnormal feature data in the dimension and the historical abnormal feature data in the corresponding dimension.

[0025] Based on the degree of abnormal deviation in different dimensions, the failure probability of power equipment for different faults is predicted.

[0026] In one embodiment, the image acquisition device includes a visible light image acquisition device, an infrared thermal image acquisition device, and an ultraviolet image acquisition device.

[0027] Secondly, this application also provides a multi-source image fusion diagnostic device for power equipment, comprising:

[0028] An acquisition module is used to acquire image data of power equipment collected by at least one image acquisition device;

[0029] The fusion module is used to perform image fusion processing on different image data of power equipment to obtain fused image data;

[0030] The prediction module is used to predict the probability of different faults in power equipment based on abnormal feature data in different dimensions of fused image data.

[0031] The selection module is used to select the fault with the highest probability as the equipment fault existing in the power equipment.

[0032] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0033] Acquire image data of power equipment collected by at least one image acquisition device;

[0034] Image fusion processing is performed on different image data of power equipment to obtain fused image data;

[0035] Based on the abnormal feature data of different dimensions in the fused image data, the failure probability of power equipment to produce different faults is predicted.

[0036] The fault with the highest probability is selected as the equipment fault in the power equipment.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0038] Acquire image data of power equipment collected by at least one image acquisition device;

[0039] Image fusion processing is performed on different image data of power equipment to obtain fused image data;

[0040] Based on the abnormal feature data of different dimensions in the fused image data, the failure probability of power equipment to produce different faults is predicted.

[0041] The fault with the highest probability is selected as the equipment fault in the power equipment.

[0042] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0043] Acquire image data of power equipment collected by at least one image acquisition device;

[0044] Image fusion processing is performed on different image data of power equipment to obtain fused image data;

[0045] Based on the abnormal feature data of different dimensions in the fused image data, the failure probability of power equipment to produce different faults is predicted.

[0046] The fault with the highest probability is selected as the equipment fault in the power equipment.

[0047] The aforementioned multi-source image fusion diagnostic method, device, equipment, and medium for power equipment acquire image data of power equipment collected by at least one image acquisition device, perform image fusion processing on different image data to obtain fused image data, and then predict the probability of different faults in the power equipment based on abnormal feature data in different dimensions of the fused image data. Finally, the fault with the highest probability is selected as the equipment fault. This can effectively overcome the information limitations of single-modal imaging, realize the complementary advantages of multi-modal data such as visible light, infrared thermal imaging, and ultraviolet imaging, and capture the physical form, thermal state, and electrical discharge state information of the equipment. By using non-subsampled shear wave transform combined with low-frequency weighted fusion and high-frequency region contrast-selective fusion strategy, the effective features of each modality are maximized, solving the problems of feature blurring and detail loss in traditional fusion methods and significantly improving the quality of fused images. Furthermore, based on the degree of deviation between multi-dimensional abnormal features and historical data, the fault probability is predicted, and the fault type is automatically identified according to the principle of maximum membership, replacing manual analysis, thereby greatly improving the comprehensiveness, accuracy, and intelligence level of power equipment fault detection. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is an application environment diagram of a multi-source image fusion diagnostic method for power equipment provided in this embodiment;

[0050] Figure 2 This is a flowchart illustrating the first method for multi-source image fusion diagnosis of power equipment provided in this embodiment;

[0051] Figure 3 This is a flowchart illustrating a step for obtaining fused image data in this embodiment;

[0052] Figure 4 This is a structural block diagram of a multi-source image fusion diagnostic device for power equipment provided in this embodiment;

[0053] Figure 5 This is an internal structural diagram of a computer device provided in this embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] The multi-source image fusion diagnostic method for power equipment provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. The computer device acquires image data of power equipment collected by at least one image acquisition device; performs image fusion processing on different image data of the power equipment to obtain fused image data; predicts the probability of different faults in the power equipment based on abnormal feature data in different dimensions of the fused image data; and selects the fault with the highest probability as the equipment fault present in the power equipment. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0056] In one exemplary embodiment, such as Figure 2 As shown, a multi-source image fusion diagnostic method for power equipment is provided, which can be applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps S201 to S204. Wherein:

[0057] S201 Acquire image data of power equipment acquired by at least one image acquisition device.

[0058] The image acquisition unit includes a visible light image acquisition unit, an infrared thermal image acquisition unit, and an ultraviolet image acquisition unit. The image data includes visible light image data, infrared thermal image data, and ultraviolet image data.

[0059] In some embodiments, visible light image data is acquired by a visible light image acquisition device, infrared thermal image data is acquired by an infrared thermal image acquisition device, and ultraviolet image data is acquired by an ultraviolet image acquisition device.

[0060] S202 performs image fusion processing on different image data of power equipment to obtain fused image data.

[0061] In some embodiments, for each image data, preprocessing operations such as registration, denoising, and enhancement are performed on the image data to obtain fused image data.

[0062] S203 predicts the probability of different faults in power equipment based on abnormal feature data in different dimensions of fused image data.

[0063] In some embodiments, a fault prediction model is pre-trained to predict the probability of different faults occurring in power equipment based on abnormal feature data in different dimensions of fused image data.

[0064] S204 selects the fault with the highest probability as the equipment fault existing in the power equipment.

[0065] In some embodiments, the fault with the highest probability is directly selected as the equipment fault existing in the power equipment.

[0066] In some embodiments, historical anomaly feature data of power equipment in different dimensions over a historical period are obtained; for each dimension, the degree of anomaly deviation of the power equipment in the dimension is determined based on the difference between the anomaly feature data in the dimension and the historical anomaly feature data in the corresponding dimension; based on the degree of anomaly deviation in different dimensions, the failure probability of the power equipment to produce different faults is predicted.

[0067] For example, from fused image data Extracting multidimensional feature vectors , ;in This includes temperature gradient features, discharge region geometric features, and texture features; a fault type discrimination model based on Gaussian membership functions is constructed to calculate whether the fused image features belong to a preset fault type. membership degree :

[0068]

[0069] in, Fault type Next The standard center value of each feature, Fault type Next The standard width of each feature and Based on historical fault data statistics, the fault type corresponding to the maximum membership degree was selected as the diagnostic result: It outputs the fault type and its corresponding fault location.

[0070] Specifically, based on infrared thermal information in the fused image, gradient operators (such as Sobel and Prewitt) are used to calculate the rate of temperature change of each pixel in the fused image, obtaining features such as the magnitude and direction of the temperature gradient, reflecting the temperature change areas of power equipment, and quantifying the temperature characteristics of faults such as local overheating; based on ultraviolet discharge information in the fused image, image segmentation (such as threshold segmentation and semantic segmentation) is used to extract the discharge region, and then the geometric features of the discharge region are calculated, including area, perimeter, circularity, centroid coordinates, etc., to quantify the discharge region features of faults such as corona discharge and partial discharge; based on visible light texture and fault region texture in the fused image, gray-level co-occurrence matrix (GLCM) is used to extract texture features, including contrast, energy, entropy, correlation, etc., to quantify the texture features of faults such as appearance defects and insulation aging of power equipment; these are used to calculate the feature vector of the fused image. Belongs to the preset fault type The membership degree. Its essence is a multidimensional Gaussian kernel similarity function: Fault type The standard center value of the i-th feature (obtained from historical fault data statistics). This is the standard width for the corresponding feature. The closer The closer the contribution of this feature term is to 1, the greater the deviation, and the more exponentially the contribution decreases. Ultimately... The closer it is to 1, the higher the degree of matching between the current fused image features and the fault type.

[0071] When extracting multidimensional features, the pixel coordinate information of each fault feature in the fused image is recorded simultaneously. After the fault type is determined by the maximum membership degree, the pixel coordinates of the core features corresponding to the fault type (such as the temperature gradient features corresponding to local overheating faults and the geometric features of the discharge area corresponding to corona discharge faults) in the fused image are extracted. Then, through the mapping relationship between the image pixel coordinates and the actual spatial coordinates of the power equipment (this mapping relationship is pre-established by the imaging equipment parameters and calibration information during multi-source image acquisition), the pixel coordinates are converted into the actual spatial location information of the power equipment. Finally, the fault type and its corresponding actual fault location are output, realizing dual diagnosis of fault type and location.

[0072] The aforementioned multi-source image fusion diagnostic method, device, equipment, and medium for power equipment acquire image data of power equipment collected by at least one image acquisition device, perform image fusion processing on different image data to obtain fused image data, and then predict the probability of different faults in the power equipment based on abnormal feature data in different dimensions of the fused image data. Finally, the fault with the highest probability is selected as the equipment fault. This can effectively overcome the information limitations of single-modal imaging, realize the complementary advantages of multi-modal data such as visible light, infrared thermal imaging, and ultraviolet imaging, and capture the physical form, thermal state, and electrical discharge state information of the equipment. By using non-subsampled shear wave transform combined with low-frequency weighted fusion and high-frequency region contrast-selective fusion strategy, the effective features of each modality are maximized, solving the problems of feature blurring and detail loss in traditional fusion methods and significantly improving the quality of fused images. Furthermore, based on the degree of deviation between multi-dimensional abnormal features and historical data, the fault probability is predicted, and the fault type is automatically identified according to the principle of maximum membership, replacing manual analysis, thereby greatly improving the comprehensiveness, accuracy, and intelligence level of power equipment fault detection.

[0073] Figure 3 This is a schematic diagram illustrating the process of obtaining fused image data in one embodiment. This embodiment refines the steps of image fusion processing of different image data of power equipment to obtain fused image data as described in the above embodiments. This embodiment provides an optional method for obtaining fused image data, including the following steps:

[0074] S301 For each image data, extract the low-frequency subband coefficient and the high-frequency subband coefficient in at least one direction of the power equipment from the image data.

[0075] Among them, the low-frequency subband coefficient is used to characterize the edge features of power equipment; the high-frequency subband coefficient is used to characterize the visual features of power equipment in terms of edge orientation.

[0076] In some embodiments, the preprocessed visible light images are respectively... Infrared thermal images and ultraviolet images The non-subsampled shear wave transform is performed using the following formula:

[0077]

[0078] in, For low-frequency subband coefficients, For the first The high-frequency subband coefficients in each direction are obtained through non-subsampled shear wave transform, which is achieved through multi-scale decomposition and directional localization, with the number of decomposition levels being [number missing]. Number of directional sub-bands per layer , This is the index for the decomposition layer.

[0079] S302 performs fusion processing on the low-frequency subband coefficients in different image data to obtain fused low-frequency subband coefficients; and, for each direction, selects the high-frequency subband coefficient with the largest regional contrast from different image data as the target high-frequency subband coefficient in the corresponding direction.

[0080] Among them, regional contrast is used to characterize the prominence of the high-frequency subband coefficients of power equipment at the corresponding image location relative to the image background.

[0081] In some embodiments, for each image data, the edge smoothness and edge sharpness of the power equipment in the image data are obtained; the image weight of the image data is determined based on the product between the edge smoothness and edge sharpness of the image data; and the low-frequency subband coefficients in different image data and the corresponding image weights are weighted and fused to obtain the fused low-frequency subband coefficients.

[0082] For example, Unsampled Shear Wave Transform (NSST) is a multi-scale, multi-directional image decomposition technique that combines the multi-scale decomposition characteristics of unsampled pyramids with the directional localization characteristics of shear waves. The decomposition process involves no image pixel loss or shift, making it suitable for decomposing multi-source images of power equipment. The number of decomposition layers J is a pre-defined positive integer, the decomposition layer index l ranges from 1 to J, and the number of directional sub-bands per layer k=2. l The decomposition increases exponentially with the number of decomposition layers, achieving refined directional decomposition of image details. (Formula) This indicates that NSST is performed on the three preprocessed modal images respectively, and each modal image is decomposed into low-frequency subband coefficients. and high-frequency subband coefficients in multiple directions ;

[0083] Among them, the low-frequency subband coefficient Low-frequency information, such as the overall outline and background of an image, is used to reflect the core basic information of the image; high-frequency subband coefficients High-frequency information such as edges, textures, and details of an image is reflected by the high-frequency sub-band coefficients in different directions, which correspond to the detailed features of the image in different directions.

[0084] Low-frequency subband coefficients carry the overall contour information of the image and are the basis for image fusion. Therefore, a weighted fusion strategy is adopted, which determines the weights by multiplying the local energy with the improved Laplacian sum, so as to realize the adaptive weighted fusion of low-frequency coefficients of each mode. The core purpose of designing this series of formulas is to allow the fused low-frequency coefficients to retain the regional features with richer contour information and higher signal strength in each modal image, and to adapt to the complementarity of contour information of different modes in power equipment images (such as the equipment contour in visible light images and the temperature contour in infrared images).

[0085] formula:

[0086] This is the core weighting formula for low-frequency fusion. Given the coordinates of any pixel in the image, adaptive weights are assigned to the low-frequency coefficients of the three modes for each pixel. The low-frequency subband coefficients of the pixel are obtained by weighted summation, thus achieving pixel-level low-frequency coefficient fusion and ensuring the continuity and integrity of the low-frequency information after fusion.

[0087] Weighting coefficient formula For adaptive weight calculation, normalization is used to ensure that the sum of the weights of all modes at that pixel is 1. The weights are determined by local energy. and improved Laplace and The product determines the weight. The larger the product, the more important the low-frequency information of that mode is at that pixel (high signal strength and clear details). The higher the weight is assigned, the more adaptive the weight allocation is achieved, avoiding the subjectivity of manually setting the weight.

[0088] Local energy formula In this model, r is a pre-defined local window radius (e.g., 3, 5). By traversing all pixels in the neighborhood centered at (x,y) with a radius of r, the sum of squares of the pixel coefficients is calculated. The larger the sum of squares, the higher the signal strength of the neighborhood and the more stable the low-frequency information. This model can effectively measure the activity level of the image pixel neighborhood and reflect the signal characteristics of the outline area of ​​the power equipment.

[0089] Improved Laplace formula It is an improvement on the traditional Laplacian operator. It calculates the sum of the absolute values ​​of the second-order differences of pixels in the horizontal and vertical directions, and sums them after traversing the neighborhood. The larger the value, the clearer the details of the edges, textures and other features of the region. It can effectively reflect the detailed features of the outline of power equipment and make up for the insufficiency of local energy in measuring signal strength only.

[0090] In some embodiments, for each edge orientation of the power equipment in the image data, the standard deviation and mean of each high-frequency subband coefficient on the edge orientation are determined; and the regional contrast of the high-frequency subband coefficient on the edge orientation is determined based on the ratio between the standard deviation and the mean.

[0091] For example, the high-frequency fusion selection rule adopts a "select the maximum" strategy, comparing the regional contrast of the three modes for each pixel position (x, y) and each directional subband k. The high-frequency coefficients of the mode with the highest contrast are selected as the fusion result. This strategy avoids edge blurring caused by weighted averaging and makes independent decisions for each direction, so that the fused image can retain the detail advantages of different modes in different directions at the same time. For example, the horizontal edge of visible light and the vertical edge of infrared light can be preserved at the same time.

[0092]

[0093] Regional contrast is a quantitative indicator of the significance of high-frequency subbands. (Molecular) The local standard deviation reflects the intensity of fluctuations in high-frequency coefficients within the neighborhood; that is, the larger the standard deviation, the more obvious the edge or texture. This is a local mean, with a tiny positive number added to prevent division by zero. The intention behind this ratio is to prioritize features that are more prominent relative to the background (i.e., areas with a smaller mean) when detail intensity is the same. For example, a faint but dramatically changing discharge spot will have higher contrast than an area of ​​equal intensity but located in a large textured area, and thus be more easily selected.

[0094] Local standard deviation and mean It is the basic statistic for regional contrast. Let (x, y) be a neighborhood window of size (2r+1)×(2r+1). This reflects the average level of high-frequency coefficients in the region, since the high-frequency sub-band coefficients are mostly distributed around zero. A value close to zero indicates that the region is dominated by noise or minor fluctuations. A deviation from zero indicates a directional bias. The contrast reflects the intensity of fluctuations and is a direct measure of edge strength. The combination of these two factors creates a contrast that considers both detail intensity and relative salience.

[0095] S303 performs an inverse transform on the fused low-frequency subband coefficients and the target high-frequency subband coefficients in different directions to obtain fused image data.

[0096] In some embodiments, the fused low-frequency subband coefficients and high-frequency subband coefficients in all directions Perform inverse non-subsampled shear wave transform to reconstruct the final fused image. .

[0097] For example, the inverse non-subsampled shear wave transform is the inverse operation of NSST. Its principle is to combine the fused low-frequency subband coefficients according to the decomposition scale and direction of NSST. And the fusion of high-frequency subband coefficients in all directions The process involves layer-by-layer and direction-by-direction reconstruction, followed by inverse transformation to restore the decomposed coefficients to the image pixel matrix, ultimately yielding the fused image. The inverse transform perfectly matches the decomposition rules of NSST, restoring the coefficients after dimensional fusion into a visualized fused image, providing a foundation for subsequent fault diagnosis.

[0098] It's important to note that the core purpose of first transforming and then inversely transforming the low-frequency and high-frequency subband coefficients is to achieve reconstruction after divide-and-conquer, that is, to decompose the image into different components, process them separately, and then reconstruct them to obtain better fusion quality. If fusion is performed directly in the spatial domain, only a uniform weighted average or selection strategy can be used, which cannot take into account both contour energy and directional details. Visible light edges, infrared hot spots, and ultraviolet discharge spots have different positions; simple averaging will lead to blurring of all features, while a single selection will lead to discontinuous energy. Through non-subsampled shear wave transform, the image is decomposed into low-frequency subbands (main energy, contour) and high-frequency subbands in multiple directions (edges, texture, discharge boundaries). Weighted averaging is used for low frequencies to maintain structural stability, while the "maximum selection" strategy is used for high frequencies to retain the clearest details. The inverse transform then reconstructs the optimized low-frequency and high-frequency coefficients into a complete fused image for human observation and subsequent diagnosis.

[0099] In the above embodiments, each image data is decomposed into low-frequency subband coefficients representing the edge features of power equipment and high-frequency subband coefficients representing the visual features of edge orientation through non-subsampled shear wave transform. An adaptive weighted fusion strategy is adopted for the low-frequency subband coefficients, combining local energy with improved Laplacian and weight allocation to retain the regional features with richer contour information and higher signal strength in each mode, ensuring the continuity and integrity of the low-frequency information after fusion. For the high-frequency subband coefficients, a "maximum selection" strategy based on regional contrast is adopted, selecting the mode coefficient with the highest contrast in each direction as the target high-frequency subband coefficient, avoiding the edge blurring problem caused by weighted averaging, so that the fused image can simultaneously retain the detail advantages of different modes in different directions (such as the horizontal edge of visible light and the vertical edge of infrared can be retained at the same time). Then, a high-quality fused image with clear contours and rich details is obtained through inverse transformation reconstruction, effectively solving the feature blurring and detail loss problems caused by traditional spatial domain weighted average fusion method.

[0100] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0101] Based on the same inventive concept, this application also provides a power equipment multi-source image fusion diagnostic device for implementing the aforementioned power equipment multi-source image fusion diagnostic method. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of one or more power equipment multi-source image fusion diagnostic device embodiments provided below can be found in the limitations of the power equipment multi-source image fusion diagnostic method described above, and will not be repeated here.

[0102] In one exemplary embodiment, such as Figure 4 As shown, a multi-source image fusion diagnostic device for power equipment is provided, comprising: an acquisition module 401, a fusion module 402, a prediction module 403, and a selection module 404, wherein:

[0103] The acquisition module 401 is used to acquire image data of power equipment collected by at least one image acquisition device;

[0104] The fusion module 402 is used to perform image fusion processing on different image data of power equipment to obtain fused image data;

[0105] The prediction module 403 is used to predict the probability of different faults in power equipment based on the abnormal feature data of different dimensions in the fused image data.

[0106] The selection module 404 is used to select the fault with the highest probability as the equipment fault existing in the power equipment.

[0107] In some embodiments, the fusion module 402 is further configured to, for each image data, extract low-frequency subband coefficients and high-frequency subband coefficients in at least one direction of the power equipment from the image data; the low-frequency subband coefficients are used to characterize the edge features of the power equipment; the high-frequency subband coefficients are used to characterize the visual features of the power equipment in the edge orientation; perform fusion processing on the low-frequency subband coefficients in different image data to obtain fused low-frequency subband coefficients; and, for each direction, select the high-frequency subband coefficient with the largest regional contrast from different image data as the target high-frequency subband coefficient in the corresponding direction; the regional contrast is used to characterize the prominence of the high-frequency subband coefficient of the power equipment at the corresponding image position relative to the image background; and perform inverse transformation on the fused low-frequency subband coefficients and the target high-frequency subband coefficients in different directions to obtain fused image data.

[0108] In some embodiments, the fusion module 402 is further configured to, for each image data, obtain the edge smoothness and edge sharpness of the power equipment in the image data; determine the image weight of the image data based on the product between the edge smoothness and edge sharpness of the image data; and perform weighted fusion of the low-frequency subband coefficients in different image data and the corresponding image weights of the image data to obtain the fused low-frequency subband coefficients.

[0109] In some embodiments, the fusion module 402 is further configured to determine the standard deviation and mean of each high-frequency subband coefficient on each edge orientation of the power equipment in the image data; and to determine the regional contrast of the high-frequency subband coefficient on each edge orientation based on the ratio between the standard deviation and the mean.

[0110] In some embodiments, the prediction module 403 is further configured to acquire historical abnormal feature data of the power equipment in different dimensions within a historical period; for each dimension, based on the difference between the abnormal feature data in the dimension and the historical abnormal feature data in the corresponding dimension, determine the degree of abnormal deviation of the power equipment in the dimension; and based on the degree of abnormal deviation in different dimensions, predict the probability of different faults occurring in the power equipment.

[0111] In some embodiments, the image acquisition device includes a visible light image acquisition device, an infrared thermal image acquisition device, and an ultraviolet image acquisition device.

[0112] Each module in the aforementioned multi-source image fusion diagnostic device for power equipment can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0113] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a multi-source image fusion diagnostic method for power equipment.

[0114] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0115] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0116] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0117] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0118] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0119] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0120] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for multi-source image fusion diagnosis of power equipment, characterized in that, The method includes: Acquire image data of power equipment collected by at least one image acquisition device; Image fusion processing is performed on different image data of the power equipment to obtain fused image data; Based on the abnormal feature data of different dimensions in the fused image data, the failure probability of the power equipment to produce different faults is predicted. The fault with the highest probability is selected as the equipment fault existing in the power equipment.

2. The method according to claim 1, characterized in that, The image fusion processing of different image data of the power equipment to obtain fused image data includes: For each image data, low-frequency subband coefficients and high-frequency subband coefficients in at least one direction of the power equipment are extracted from the image data; the low-frequency subband coefficients are used to characterize the edge features of the power equipment; the high-frequency subband coefficients are used to characterize the visual features of the power equipment in the edge orientation. The low-frequency subband coefficients in different image data are fused to obtain fused low-frequency subband coefficients; and, for each direction, the high-frequency subband coefficient with the largest regional contrast is selected from different image data as the target high-frequency subband coefficient in the corresponding direction; the regional contrast is used to characterize the prominence of the high-frequency subband coefficient of the power equipment relative to the image background at the corresponding image position. The fused low-frequency subband coefficients and the target high-frequency subband coefficients in different directions are inversely transformed to obtain fused image data.

3. The method according to claim 2, characterized in that, The process of fusing low-frequency subband coefficients from different image data to obtain fused low-frequency subband coefficients includes: For each image data, the edge smoothness and edge sharpness of the power equipment in the image data are obtained; The image weights of the image data are determined based on the product between the edge smoothness and the edge sharpness of the image data. The low-frequency subband coefficients in different image data and the corresponding image weights are weighted and fused to obtain the fused low-frequency subband coefficients.

4. The method according to claim 2, characterized in that, The process of determining the regional contrast of the high-frequency sub-band coefficients includes: For each edge orientation of the power equipment in the image data, determine the standard deviation and mean of the high-frequency sub-band coefficients at each edge orientation; The regional contrast of the high-frequency subband coefficients facing upwards is determined based on the ratio between the standard deviation and the mean.

5. The method according to claim 1, characterized in that, The method of predicting the probability of different faults in the power equipment based on abnormal feature data in different dimensions of the fused image data includes: Acquire historical anomaly characteristic data of the power equipment under different dimensions within a historical time period; For each dimension, based on the difference between the abnormal feature data under the dimension and the historical abnormal feature data under the corresponding dimension, the degree of abnormal deviation of the power equipment under the dimension is determined; Based on the degree of abnormal deviation in different dimensions, the failure probability of the power equipment to produce different faults is predicted.

6. The method according to claim 1, characterized in that, The image acquisition device includes a visible light image acquisition device, an infrared thermal image acquisition device, and an ultraviolet image acquisition device.

7. A multi-source image fusion diagnostic device for power equipment, characterized in that, The device includes: An acquisition module is used to acquire image data of power equipment collected by at least one image acquisition device; The fusion module is used to perform image fusion processing on different image data of the power equipment to obtain fused image data; The prediction module is used to predict the probability of different faults occurring in the power equipment based on abnormal feature data in different dimensions of the fused image data. The selection module is used to select the fault with the highest probability as the equipment fault existing in the power equipment.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images