Multi-modal feature fusion method and system for power distribution network device state perception

Abstract

The application relates to the technical field of smart power grids and device state sensing, and discloses a multi-modal feature fusion method and system for distribution network device state sensing, which comprises the following steps: acquiring multi-modal sensing signals, performing signal-to-noise ratio quantization and uniform space alignment, obtaining a confidence degree vector group and an aligned feature set, jointly performing main and auxiliary feature positioning, obtaining orthogonal complementary features through cross-orthogonal decoupling, performing amplitude attenuation and element-by-element fusion to obtain a fusion feature vector, combining a state feature library to obtain a reference feature set, calculating Mahalanobis distances and performing cyclic comparison, and outputting a device state recognition result. The application avoids the pollution of deteriorated signals to a shared space, suppresses negative transfer effects, improves sensing accuracy under extreme boundary working conditions, eliminates edge computing power expenses, and realizes parameter-free fine tuning and plug-and-play of a sensing model.

Description

Technical Field

[0001] This invention relates to the field of smart grid and equipment status sensing technology, specifically to a multimodal feature fusion method and system for distribution network equipment status sensing. Background Technology

[0002] With the development of smart grid technology, distribution network equipment status sensing systems typically employ a multi-sensor collaborative approach to monitor equipment operating status. During this process, edge-side equipment needs to collect electrical signals from transformers or switchgear, as well as visual signals from the field. Then, a dual-stream or multi-stream feature extraction module using a deep neural network extracts the local features of each signal. Finally, feature-level fusion is performed through a stitching operation or a fully connected network, and the results are input into a classifier to output the equipment status identification result.

[0003] However, under the boundary conditions of complex electromagnetic environments and extreme climates, the signal-to-noise ratio of visual or electrical signals undergoes abrupt degradation. During the interaction process of feature-level static stitching, the high-dimensional noise features of the relatively degraded signal are directly mapped into the shared hidden feature space. This forcibly distorts the data distribution characteristics of the healthy signal in the manifold space, resulting in a negative intermodal migration effect. Consequently, the discriminative power of the final fused features is lower than that of the independent representation of the main features extracted using only a single healthy signal. Therefore, how to avoid the contamination of the shared feature space by degraded signals and suppress the negative migration effect during multimodal feature fusion by edge devices is a problem that urgently needs to be solved in this field. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention aims to provide a multimodal feature fusion method for distribution network equipment status perception. This method involves performing signal-to-noise ratio quantization on the multimodal sensing signals to construct a confidence vector set containing electrical and visual confidence levels. It also employs a main-auxiliary feature localization mechanism combined with a cross-orthogonal decoupling mechanism to extract positive interactive complementary features. This achieves the goals of avoiding polluting the shared feature space with degraded signals, suppressing negative migration effects, and significantly reducing the false alarm rate of equipment status. Simultaneously, by utilizing the fused feature vector and a reference feature set embedded in the status feature library, parameterless distance metric matching is performed through Mahalanobis distance calculation. This enables high-precision status classification completely independent of trainable neural network classifiers, eliminating the backpropagation and computational overhead of joint training of edge-side devices. This achieves the effect of parameter-free fine-tuning and plug-and-play deployment of the multimodal sensing model.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] In a first aspect, the present invention provides a multimodal feature fusion method for state perception of distribution network equipment, including:

[0007] Acquire multimodal sensing signals, perform signal-to-noise ratio quantization on the multimodal sensing signals to obtain a confidence vector set, and perform deterministic feature extraction and unified spatial alignment on the multimodal sensing signals to obtain an aligned feature set;

[0008] The joint alignment feature set and confidence vector group are used to perform primary and secondary feature localization to obtain primary and secondary feature pairs. Cross-orthogonal decoupling is performed on the primary and secondary feature pairs to obtain positive interactive complementary features. Amplitude attenuation and element-wise fusion are performed based on the positive interactive complementary features to obtain the fused feature vector.

[0009] The system acquires a state feature library and binds it with the fused feature vector to obtain a reference feature set. Based on the reference feature set, it calculates the Mahalanobis distance for each operating state. The system then performs a loop comparison with the fused feature vector to obtain the device state identification result.

[0010] Furthermore, the method for obtaining the confidence vector set includes:

[0011] Acquire multimodal sensing signals, which include electrical signals and visual signals;

[0012] A sliding time window is set on the electrical signal, and the variance of the high-frequency components of the electrical signal within the sliding time window is calculated.

[0013] The visual signal is divided into multiple local image blocks, and the local information entropy of the visual signal in all local image blocks is calculated.

[0014] Based on the preset signal-to-noise ratio quantization function, the variance of high-frequency components is mapped to electrical confidence, and the local information entropy is mapped to visual confidence.

[0015] The electrical confidence score and the visual confidence score are combined to obtain a confidence vector set.

[0016] Furthermore, the method for calculating the variance of the high-frequency components includes:

[0017] The discrete wavelet transform algorithm is called to perform single-level decomposition on the electrical signal within the sliding time window, resulting in a high-frequency detail coefficient sequence composed of high-frequency detail coefficients;

[0018] Calculate the arithmetic mean of all high-frequency detail coefficients in the high-frequency detail coefficient sequence;

[0019] The variance of the high-frequency components is obtained by subtracting the arithmetic mean from each high-frequency detail coefficient in the high-frequency detail coefficient sequence, squaring the squared result, summing all the squared results, and dividing by the total number of high-frequency detail coefficients.

[0020] Furthermore, the method for calculating the local information entropy includes:

[0021] The probability of occurrence of each gray level is obtained by counting the number of pixels corresponding to each gray level within each local image block and dividing the corresponding number of pixels by the total number of pixels in that local image block.

[0022] The probability of each gray level is multiplied by its base-2 logarithm and the result is negative. When the probability of occurrence is zero, the contribution of that gray level is set to zero and the accumulation is skipped. The results of the calculations for all gray levels are summed to obtain the information entropy of the local image patch.

[0023] Furthermore, the method for obtaining the alignment feature set includes:

[0024] Extracting timing features from electrical signals;

[0025] The pre-deployed image feature extraction network model is invoked to extract spatial features of visual signals;

[0026] By using a pre-set linear mapping layer with a unified target dimension, the spatial features are subjected to dimensionality reduction projection to obtain the dimensionality-reduced spatial component vector, and the time series features are subjected to zero-mean unit variance standardization to obtain the standardized electrical component vector.

[0027] The reduced spatial component vector and the standardized electrical component vector are collectively defined as an alignment feature set, which includes visual alignment feature vectors and electrical alignment feature vectors.

[0028] Furthermore, the acquisition of the primary and secondary feature pairs includes:

[0029] Compare the numerical values ​​of electrical confidence and visual confidence;

[0030] If the electrical confidence level is greater than the visual confidence level, the electrical alignment feature vector is defined as the main feature vector, the visual alignment feature vector is defined as the auxiliary feature vector, and the visual confidence level is defined as the auxiliary modal confidence scalar.

[0031] If the visual confidence is greater than the electrical confidence, the visual alignment feature vector is defined as the main feature vector, the electrical alignment feature vector is defined as the auxiliary feature vector, and the electrical confidence is defined as the auxiliary modal confidence scalar.

[0032] If the visual confidence and electrical confidence are equal, the electrical alignment feature vector is defined as the primary feature vector, the visual alignment feature vector is defined as the secondary feature vector, and the visual confidence is defined as the secondary modal confidence scalar, based on the preset default priority.

[0033] The principal feature vector, the auxiliary feature vector, and the auxiliary modality confidence scalar are collectively defined as a principal-auxiliary feature pair.

[0034] Furthermore, the method for obtaining the positive interactive complementary features includes:

[0035] Calculate the dot product between the auxiliary feature vector and the main feature vector to obtain the cross dot product value;

[0036] Calculate the dot product between the principal eigenvector and itself to obtain the dot product value;

[0037] Divide the cross inner product value by the self inner product value to obtain the projection coefficient;

[0038] Multiplying the projection coefficients by the principal eigenvector yields the projection vector of the secondary eigenvector onto the direction of the principal eigenvector.

[0039] Subtracting the corresponding element of the projection vector from the auxiliary feature vector element by element yields the positive interactive complementary feature that is orthogonal to the main feature vector.

[0040] Furthermore, the method for obtaining the fused feature vector includes:

[0041] Multiply each element of the positive interactive complementary feature with the auxiliary modality confidence scalar to obtain the weighted complementary feature after amplitude attenuation;

[0042] The weighted complementary features are added element-wise to the main feature vector to obtain the fused feature vector.

[0043] Furthermore, the method for obtaining the reference feature set includes:

[0044] Multiple operating states are obtained from the state feature library. For each operating state, multiple calibration samples are collected and the fusion feature vector corresponding to each calibration sample is obtained. At the same time, each operating state is assigned a state label.

[0045] For all fused feature vectors under the same operating state, perform an arithmetic average operation on each element according to the order of their arrangement to obtain the standard reference feature vector for that operating state.

[0046] Calculate and invert the covariance matrix of all fused feature vectors under this operating state to obtain the inverse covariance matrix of this operating state;

[0047] The reference feature set is obtained by binding the status label, standard reference feature vector, and inverse covariance matrix of each operating state.

[0048] Furthermore, the method for calculating the Mahalanobis distance includes:

[0049] The difference vector is obtained by subtracting the standard reference feature vector of the current running state from the fused feature vector element by element.

[0050] Perform matrix multiplication operations on the transpose of the difference vector, the inverse covariance matrix of the current running state, and the difference vector in sequence to obtain a scalar result;

[0051] Perform the arithmetic square root operation on the scalar result to obtain the Mahalanobis distance between the fused feature vector and the current running state.

[0052] Furthermore, the method for obtaining the device status identification result includes:

[0053] Allocate the minimum distance record and the optimal tag cache size in memory, and set the initial value of the minimum distance record to positive infinity and the initial value of the optimal tag cache size to null.

[0054] Iterate through the Mahalanobis distance corresponding to each running state and compare it with the current value in the minimum distance record;

[0055] If the current Mahalanobis distance is less than the value in the minimum distance record, then update the minimum distance record to the current Mahalanobis distance, and update the optimal label cache size to the status label of this running state;

[0056] After all operating states have been traversed and compared, the final saved state label will be used as the device state identification result.

[0057] Secondly, the present invention provides a multimodal feature fusion system for distribution network equipment status perception, which is used to implement the above-mentioned multimodal feature fusion method for distribution network equipment status perception, the system comprising:

[0058] Feature alignment module: used to acquire multimodal sensing signals, perform signal-to-noise ratio quantization on the multimodal sensing signals to obtain a confidence vector group, and perform deterministic feature extraction and unified spatial alignment on the multimodal sensing signals to obtain an aligned feature set;

[0059] Orthogonal decoupling module: used to jointly align feature set and confidence vector group to perform main and auxiliary feature localization, obtain main and auxiliary feature pairs, perform cross-orthogonal decoupling on main and auxiliary feature pairs to obtain positive interactive complementary features, and perform amplitude attenuation and element-wise fusion based on positive interactive complementary features to obtain fused feature vector;

[0060] State matching module: used to acquire state feature library, bind it with fused feature vector to obtain reference feature set, calculate Mahalanobis distance for each operating state based on reference feature set, and perform loop traversal comparison with Mahalanobis distance and fused feature vector to obtain device state recognition result.

[0061] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0062] This invention obtains a confidence vector group by performing signal-to-noise ratio quantization on multimodal sensing signals. Guided by this, it performs main and auxiliary feature localization and cross-orthogonal decoupling, stripping collinear redundant components from the mathematical level and extracting positive interactive complementary features. This solves the problem that in the traditional feature-level static splicing architecture, high-dimensional noise of degraded signals is forcibly mapped to the shared hidden layer feature space, thus distorting data distribution and causing negative migration effects. Based on confidence-guided amplitude attenuation and element-by-element fusion to generate fused feature vectors, it transforms blind data interaction into dynamic absorption, causing severely degraded signal features to attenuate to near zero and smoothly degenerate into main feature vectors, significantly improving the robustness of state perception under extreme boundary conditions. The parameterless distance metric matching mechanism, combined with the reference feature set of the state feature library, completes pure algebraic state discrimination by calculating Mahalanobis distance and comparing iterative traversal, eliminating the dependence on classifiers such as multilayer perceptrons and completely eliminating the computational overhead of gradient updates. This enables parameterless fine-tuning and plug-and-play functionality at edge terminals with extremely low computing power. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 A flowchart of the multimodal feature fusion method for power distribution equipment status perception provided in this embodiment of the invention;

[0065] Figure 2 This is a graph showing the signal-to-noise ratio quantization function mapping relationship of a multimodal sensing signal provided in an embodiment of the present invention.

[0066] Figure 3 A schematic diagram illustrating the extraction of temporal features and frequency band distribution provided in an embodiment of the present invention;

[0067] Figure 4 This embodiment provides a schematic diagram of a confidence-guided cross-orthogonal decoupling and feature reconstruction spatial vector.

[0068] Figure 5 This is a schematic diagram of the state discrimination space for parameterless distance metric matching provided in an embodiment of the present invention;

[0069] Figure 6 A functional block diagram of a multimodal feature fusion system for power distribution equipment status perception provided in an embodiment of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Example 1

[0072] Please see Figure 1 As shown, this embodiment provides a multimodal feature fusion method for distribution network equipment status awareness, including:

[0073] Step S10: Acquire multimodal sensing signals, perform signal-to-noise ratio quantization on the multimodal sensing signals to obtain a confidence vector group, and perform deterministic feature extraction and unified spatial alignment on the multimodal sensing signals to obtain an aligned feature set.

[0074] Further, step S10 includes:

[0075] Step S11: Acquire the multimodal sensing signal, perform signal-to-noise ratio quantization on the multimodal sensing signal, and obtain a confidence vector group.

[0076] During the continuous operation of distribution network equipment status sensing systems, especially in scenarios where transformers and switchgear require multi-sensor collaborative sensing of equipment operating status, edge-side devices determine the health status of equipment by collecting electrical and visual signals. When the operating environment of distribution network equipment encounters boundary conditions such as heavy fog, strong electromagnetic interference, or extreme weather, the signal-to-noise ratio (SNR) of electrical or visual signals will undergo a sudden degradation. However, existing multimodal feature fusion schemes typically employ a feature-level static splicing architecture, which indiscriminately splices the feature vectors of each modality end-to-end and sends them into a shared hidden layer feature space. This lacks the physical sensing capability of the transient data quality of each modality, resulting in the high-dimensional noise features of degraded modalities being forcibly mapped into the shared hidden layer feature space, forcibly distorting the data distribution characteristics of healthy modalities in the manifold space and inducing negative intermodal migration effects. Therefore, a modal confidence assessment mechanism based on SNR quantization is established to achieve a scalarized measurement of the transient data quality of each modality. The aim is to transform the indistinguishable isolated modal raw signals into structured sensing inputs carrying physical credibility credentials.

[0077] Specifically, electrical and visual signals of the distribution network equipment during its current operating cycle are synchronously acquired by edge-side devices deployed at the distribution network site, and these electrical and visual signals are collectively defined as multimodal sensing signals. The edge-side devices refer to embedded hardware computing units deployed inside pole-mounted switch intelligent terminals or distribution cabinet edge gateways, possessing multi-channel signal acquisition interfaces and local floating-point computing capabilities, used to achieve synchronous acquisition and localized inference of raw multimodal signals from the field. The electrical signals refer to a one-dimensional discrete numerical sequence, composed of instantaneous voltage and current amplitudes arranged in the order of sampling time, acquired at a fixed sampling frequency by voltage and current transformers installed on the incoming and outgoing lines of the distribution network equipment within a continuous operating cycle. Each discrete value in this sequence is a sampling point, representing the steady-state electrical operating waveform load of the distribution network equipment. The visual signals refer to a two-dimensional pixel matrix, composed of grayscale intensity values ​​of spatial pixels arranged in row and column coordinates, acquired by visible light cameras or infrared thermal imagers installed on the distribution network equipment body, representing the visible light texture or infrared temperature field distribution on the surface of the distribution network equipment body. The operating cycle refers to the time interval corresponding to the completion of a single complete state perception by the edge-side device. Its setting is based on the steady-state observation window of the power frequency electrical quantities of the distribution network equipment; for example, the operating cycle is set to 200 milliseconds. After acquiring the multimodal sensing signal, the edge-side device performs signal-to-noise ratio quantization on both the electrical and visual signals. For the electrical signal, the variance of its high-frequency components within the sliding time window is calculated. The variance of the high-frequency components is a statistical measure of the magnitude of disordered high-frequency jitter energy induced by electromagnetic interference in the electrical signal. Specifically, a fixed-length sliding time window is set on the electrical signal. The length of the sliding time window is determined based on the number of sampling points contained in a single cycle of the power frequency signal to ensure a strict physical correspondence between the number of sampling points obtained based on a fixed sampling frequency and sampling time and the power frequency cycle. For example, the length of the sliding time window is set to cover 20 consecutive sampling points. The discrete wavelet transform algorithm is called to perform single-level decomposition on the electrical signal within the sliding time window. The source input object is the electrical signal within the sliding time window, and the target output variable is the high-frequency detail coefficient sequence characterizing the high-frequency mutation component of the signal. The high-frequency detail coefficient sequence refers to a one-dimensional numerical sequence composed of several high-frequency detail coefficients arranged in discrete time order. Among them, the high-frequency detail coefficients refer to the discrete coefficient values ​​obtained after the electrical signal passes through a high-pass filter during the discrete wavelet transform process, which are used to characterize the magnitude of the high-frequency mutation energy induced by electromagnetic interference within the time window. For the high-frequency detail coefficient sequence, the arithmetic mean of all its elements is calculated. Then, the arithmetic mean is subtracted from each high-frequency detail coefficient, and the square is taken. All squared results are summed and divided by the total number of high-frequency detail coefficients to obtain the variance of the high-frequency components. The variance of the high-frequency components is a dynamic quantized value that changes in real time with the system's operating state.For visual signals, calculate their local information entropy in the spatial domain. The local information entropy is a statistical measure of the degree of disorder in the gray-level distribution and the content of effective texture information in a local spatial region of a visual signal. When an image is contaminated by heavy fog or occlusion, its effective texture collapses and its local information entropy decreases. Specifically, the visual signal is divided into several non-overlapping local image blocks according to a preset window size. The window size is set based on a pixel neighborhood range that balances texture detail preservation and statistical stability; for example, the window size is set to 16 pixels by 16 pixels. For each local image block, the number of pixels appearing at each gray level is counted. The number of pixels at that gray level is divided by the total number of pixels in the local image block to obtain the probability of occurrence of that gray level. The probability of occurrence of each gray level is multiplied by its base-2 logarithm and the negative value is taken. When the probability of occurrence is zero, the contribution value of that gray level is defined as zero and the accumulation is skipped. The results of the calculations for all gray levels are summed to obtain the information entropy of the local image block. The arithmetic mean of the information entropies of all local image blocks is performed to obtain the local information entropy. The local information entropy is a dynamically quantified value that changes in real time with the system's operating state.

[0078] After obtaining the high-frequency component variance and local information entropy, the high-frequency component variance is mapped to electrical confidence based on a preset signal-to-noise ratio quantization function. Electrical confidence is a normalized scalar value strictly constrained between 0 and 1, calculated from the high-frequency component variance through a monotonically decreasing exponential decay mapping. It characterizes the physical reliability of the electrical signal after being affected by electromagnetic interference in the current operating cycle. The local information entropy is mapped to visual confidence, also a normalized scalar value constrained between 0 and 1. It characterizes the texture validity and physical reliability of the visual signal after being affected by pollution such as fog or obstruction in the current operating cycle. The electrical confidence and visual confidence are jointly defined as a confidence vector group. The signal-to-noise ratio (SNR) quantization function is used to monotonically map the variance and local information entropy of high-frequency components, which have different dimensions and value ranges, into a normalized confidence scalar whose value range is strictly constrained to between 0 and 1. This measures the physical reliability of the corresponding mode data within the current operating cycle. For electrical confidence, the analytical expression of the SNR quantization function is: ,in, , These represent the electrical confidence level and the variance of the high-frequency components, respectively. The electrical attenuation coefficient is set based on the background distribution scale of the variance of the high-frequency components of the electrical signal in a calibration environment without electromagnetic interference. Its purpose is to control the rate attenuation of confidence as the energy of high-frequency noise increases. For example, it is set to 5. It is expressed as an exponential function with the natural constant e as the base; for visual confidence, its signal-to-noise ratio quantization function is expressed as: ,in, , These represent visual confidence and local information entropy, respectively. The visual steepness coefficient is set based on the separable interval of local information entropy of the visual signal under both clear and contaminated conditions. Its purpose is to control the steepness of the judgment of confidence near the entropy benchmark threshold. For example, it is set to 2. The information entropy threshold is set based on the statistical boundary between the local information entropy of clear images and fog-occluded images in the calibration dataset. The purpose is to use this threshold as the decision midpoint between visual modality reliability and failure; for example, it is set to 4. The mathematical construction of the signal-to-noise ratio quantization function is based on the physical principle that "the greater the noise energy and the less effective information, the lower the modality reliability." For the electrical modality, since the variance of the high-frequency components directly characterizes the electromagnetic interference noise energy, a monotonically decreasing exponential decay mapping is used, so that the confidence approaches zero when the noise energy approaches infinity. For the visual modality, since the local information entropy positively characterizes the amount of effective texture information, a monotonically increasing logistic function mapping is used, so that the confidence approaches zero when the effective information collapses. The final output confidence vector set is a dynamically quantized value that changes in real time with the system's operating state. See [link to relevant documentation]. Figure 2 This is a graph showing the signal-to-noise ratio quantization function mapping relationship of a multimodal sensing signal provided in an embodiment of the present invention. Figure 2 The diagram illustrates the mathematical mapping logic for normalizing signal quality statistics with different physical dimensions into confidence scalars between 0 and 1. The left coordinate system represents the electrical modal quantization curve, with the horizontal axis representing the high-frequency component variance and the vertical axis representing electrical confidence. This curve exhibits a monotonically decreasing exponential decay trend, intuitively revealing the physical law that the deterministic certainty of electrical modal physical confidence approaches zero as the high-frequency noise energy induced by electromagnetic interference, i.e., the high-frequency component variance, increases. The right coordinate system represents the visual modal quantization curve, with the horizontal axis representing local information entropy and the vertical axis representing visual confidence. This curve exhibits a monotonically increasing logistic mapping trend, and a discriminative jump occurs at the information entropy threshold marked on the horizontal axis. This intuitively reveals the mapping mechanism where, when an image is contaminated by fog or occlusion, causing the effective texture information, i.e., the local information entropy, to collapse below this threshold, the visual modal physical confidence rapidly decreases and approaches zero.

[0079] Step S12: Perform deterministic feature extraction and unified spatial alignment on the multimodal sensing signal to obtain an aligned feature set.

[0080] After obtaining the electrical and visual signals, in order to align the one-dimensional electrical signals and the two-dimensional visual signals in a common geometric space and avoid matrix operation failures caused by the incompatibility of dimensions and scales of heterogeneous features, the two modal signals with different physical properties are transformed into aligned feature vectors with strictly identical dimensions.

[0081] Specifically, for electrical signals, the original electrical signals acquired during the same operating cycle are used as input, and wavelet packet decomposition is performed to extract time-series features. These time-series features are one-dimensional numerical vectors composed of the energy values ​​of the electrical signal in each subdivided frequency band, arranged in descending order of frequency, representing the energy distribution of the electrical operating waveform in the frequency domain. The wavelet packet decomposition algorithm is a well-known deterministic transform algorithm for signal time-frequency analysis. Its multi-level decomposition process for electrical signals involves dividing the original frequency band of the electrical signal into several continuous subdivided frequency bands; where each subdivided frequency band refers to the independent frequency interval corresponding to the leaf node of the wavelet packet decomposition tree in the wavelet packet decomposition algorithm. The number of decomposition levels is determined based on the required frequency band division accuracy of the electrical signal. For example, the number of decomposition levels is set to 4, thereby uniformly dividing the original frequency band into 16 subdivided frequency bands. The target output variable of the wavelet packet decomposition is the set of wavelet packet node coefficients corresponding to each sub-band. The set of wavelet packet node coefficients refers to a collective term for a set of discrete wavelet coefficients obtained after projecting an electrical signal onto the corresponding sub-band, containing several node coefficients. For each sub-band, the node coefficient set is squared and summed to obtain the node energy value of that sub-band. The node energy values ​​of all sub-bands are arranged sequentially according to their center frequencies from low to high, forming a temporal sequence feature. For visual signals, an image feature extraction network model is invoked to extract spatial features. Specifically, the image feature extraction network model uses the ResNet-50 residual convolutional neural network model, which is well-known in the field and pre-trained on the ImageNet open-source dataset. This model has the ability to perform hierarchical abstraction and extraction of general spatial texture and contour features of two-dimensional images. The spatial feature extraction process of this model is as follows: the visual signal is scaled to the standard input size specified by the ResNet-50 model and fed into the model as the input object, for example, a standard input size of 224 pixels by 224 pixels; under the condition that all network weight parameters of the model are strictly frozen, unidirectional forward propagation inference is performed; the one-dimensional feature vector output by the last global average pooling layer of the model is extracted as the output variable and defined as the spatial feature. The spatial feature refers to the one-dimensional high-dimensional feature value that characterizes the spatial texture and temperature field contour of the image. Since the model weights are kept frozen throughout the feature extraction process, the gradient update and backpropagation processes are completely separated, realizing a deterministic computational mapping from the original image to high-dimensional features.

[0082] After obtaining the temporal and spatial features, a linear mapping layer based on the covariance matrix maps them to a unified feature space of the same dimension. This linear mapping layer refers to a linear projection module without learnable parameters, whose transformation matrix is ​​determined deterministically from the covariance structure of the input data. The following deterministic transformations are performed on the temporal and spatial features respectively; specifically, a well-known principal component analysis algorithm is invoked to perform dimensionality reduction projection on the spatial features. To obtain the projection matrix required for the projection, during the calibration phase before system deployment, historical visual signal sets of distribution network equipment under different typical operating conditions are collected in advance. All of these are input into the aforementioned ResNet-50 model, extracting a large number of historical spatial features as a calibration sample set. The covariance matrix of this calibration sample set is calculated, and eigenvalue decomposition is performed on the covariance matrix to extract the feature vectors corresponding to the top N largest eigenvalues. Here, N is the unified target dimension, and the value of N is consistent with the number of subdivided frequency bands determined by the wavelet packet decomposition layer. For example, when the decomposition layer is set to 4 layers, N is set to 16. The N feature vectors are arranged column-wise to construct an orthogonal projection matrix. In the real-time inference of the current running cycle, matrix multiplication is performed between the currently extracted spatial features and the orthogonal projection matrix to deterministically reduce the dimensionality to the unified target dimension, obtaining the dimensionality-reduced spatial component vectors.

[0083] For time-series features, to eliminate the dimensional differences in absolute energy values ​​between different sub-bands, zero-mean, unit-variance standardization is performed. Specifically, during the calibration phase before the system goes online, a set of historical electrical signals is collected and subjected to the same wavelet packet decomposition to extract a large number of historical time-series features as an electrical calibration sample set. For this electrical calibration sample set, the statistical mean and statistical standard deviation are calculated for each sub-band. The statistical mean is arranged sequentially to form a one-dimensional sample mean vector, and the statistical standard deviation is arranged sequentially to form a one-dimensional sample standard deviation vector. The sample mean vector and sample standard deviation vector are then used as a fixed vector. Parameters are pre-stored in edge computing nodes. During real-time inference in the current runtime cycle, the sample mean vector is subtracted element-by-element from the extracted time-series features to obtain the mean-reduced features. Then, the mean-reduced features are divided element-by-element by the sample standard deviation vector. If the sample standard deviation of a subdivided frequency band is zero, the standardization result at that position is directly set to zero. Finally, zero-mean unit variance standardization is completed to obtain the standardized electrical component vector. The dimensionality-reduced spatial component vector and the standardized electrical component vector are respectively denoted as the visual alignment feature vector and the electrical alignment feature vector, and their dimensions are strictly aligned, collectively defined as the alignment feature set. See also Figure 3 This is a schematic diagram illustrating the extraction of temporal features and frequency band distribution provided by an embodiment of the present invention. Figure 3The diagram exemplifies a deterministic signal processing pipeline for converting electrical signals from time-domain waveforms to discrete energy characteristics in the frequency domain. The upper section displays the original waveform of the electrical signal, which fluctuates over time. The middle section, using a four-layer wavelet packet decomposition as an example, demonstrates the multi-layered frequency band division structure constructed by the wavelet packet decomposition algorithm, representing the original frequency band being uniformly divided into 16 continuous sub-bands. The lower section, in the form of a bar chart, shows the node energy values ​​obtained by calculating the sum of squares of the wavelet packet node coefficient sets within each sub-band. These 16 node energy values ​​are arranged sequentially from low to high center frequency, collectively forming a one-dimensional numerical vector with a clear physical distribution, i.e., the temporal sequence characteristic.

[0084] Step S10 addresses the technical challenges of existing feature-level static stitching architectures lacking physical perception of the transient data quality of each modality, and the inability of one-dimensional electrical signals and two-dimensional visual signals to directly participate in unified geometric operations due to their heterogeneity. It achieves scalar measurement of modal physical credibility, deterministic extraction of cross-modal heterogeneous features, and isomorphic alignment in a unified space through confidence vector sets, temporal features, spatial features, and alignment. Specifically, the confidence vector set, through normalized mapping of high-frequency component variance and local information entropy, resolves the blind state of indistinguishable input-side modal quality, constructing a scalar base carrying physical credibility credentials. Temporal and spatial features avoid semantic biases introduced by cross-modal joint training, cutting off the impact of training-stage data pollution on feature extraction results from the source. The alignment feature set, through an orthogonal projection matrix constructed based on the historical sample covariance matrix, and sample mean and standard deviation vectors pre-set based on the statistical characteristics of historical samples, locks heterogeneous features into a homogeneous space through pure algebraic operations, serving as the core data-driven credentials for subsequent processing.

[0085] Step S20: Perform primary and secondary feature localization on the joint alignment feature set and confidence vector group to obtain primary and secondary feature pairs. Perform cross-orthogonal decoupling on the primary and secondary feature pairs to obtain positive interactive complementary features. Perform amplitude attenuation and element-wise fusion based on the positive interactive complementary features to obtain fused feature vectors.

[0086] Further, step S20 includes:

[0087] Step S21: Perform primary and secondary feature localization by jointly aligning the feature set and the confidence vector group to obtain primary and secondary feature pairs.

[0088] After obtaining the alignment feature set and confidence vector set, in order to ensure that the orthogonal projection operation has a clear geometric reference direction and to avoid the direct symmetric fusion without judging the quality of electrical and visual signals, which would lead to high-dimensional noise features dominating the fusion direction and thus causing a negative migration effect, a confidence-oriented master-slave feature localization mechanism is established. Its purpose is to establish the alignment feature vector corresponding to the higher quality signal as the uncontaminated master reference direction based on physical confidence.

[0089] The electrical confidence and visual confidence scores are extracted from the confidence vector set. Their values ​​are compared, and primary and secondary feature localization is performed according to an exhaustive, mutually exclusive branching logic. Specifically, if the electrical confidence score is greater than the visual confidence score, the electrical signal is determined to have higher physical confidence. The electrical alignment feature vector in the alignment feature set is defined as the primary feature vector, the visual alignment feature vector as the secondary feature vector, and the visual confidence score is defined as the secondary modal confidence scalar. If the visual confidence score is greater than the electrical confidence score, the visual signal is determined to have higher physical confidence. The visual alignment feature vector in the alignment feature set is defined as the primary feature vector, the electrical alignment feature vector as the secondary feature vector, and the electrical confidence score as the secondary modal confidence scalar. If the electrical confidence score and visual confidence score are equal, the electrical alignment feature vector is defined as the primary feature vector, the visual alignment feature vector as the secondary feature vector, and the visual confidence score as the secondary modal confidence scalar, according to a preset default priority. The default priority setting is based on the fact that electrical signals have a stronger physical causal directness of faults than visual signals in the steady-state operation monitoring of distribution network equipment. The purpose is to ensure that when the physical credibility of electrical signals and visual signals cannot be distinguished, the fusion direction degenerates into a deterministic output based on the electrical alignment feature vector as the main reference.

[0090] The aforementioned principal feature vector, auxiliary feature vector, and auxiliary modal confidence scalar are collectively defined as a principal-auxiliary feature pair. The principal feature vector refers to the aligned feature vector corresponding to a signal with higher physical confidence, whose direction serves as an uncontaminated orthogonal reference direction. The auxiliary feature vector refers to the aligned feature vector corresponding to a signal with relatively lower physical confidence, whose components will be forcibly stripped to the orthogonal complement space of the principal feature vector. The principal-auxiliary feature pair refers to a structured vector group, composed of the principal feature vector, auxiliary feature vector, and auxiliary modal confidence scalar, used to drive orthogonal projection operations.

[0091] Step S22: Perform cross-orthogonal decoupling on the primary and secondary feature pairs to obtain positive interactive complementary features.

[0092] After obtaining the primary and secondary feature pairs, in order to remove the redundant components that are collinear with the primary feature vector from the relatively degraded secondary feature vector, and to avoid the high-dimensional noise components in the secondary feature vector along the direction of the primary feature vector from directly superimposing and polluting the primary feature vector, a cross-orthogonal decoupling mechanism based on orthogonal complementary space projection is established. Its purpose is to cut off the pollution path of the relatively degraded signal data to the core primary feature from the mathematical level.

[0093] Specifically, the projection vector of the auxiliary feature vector onto the direction of the principal feature vector is calculated, and the auxiliary feature vector is subtracted from the projection vector to obtain the positive interactive complementary feature. Specifically, the dot product of the auxiliary feature vector and the principal feature vector is calculated, defined as the cross dot product value; the dot product of the principal feature vector with itself is calculated, defined as the self dot product value. If the magnitude of the principal feature vector is not zero, the cross dot product value is divided by the self dot product value to obtain the projection coefficient; the projection coefficient is multiplied by the principal feature vector to obtain the projection vector. Then, the corresponding elements of the projection vector are subtracted from the auxiliary feature vector element by element to calculate the positive interactive complementary feature. The process of solving the positive interactive complementary feature is based on the Gram-Schmidt orthogonalization principle in linear algebra. Using the orthogonal projection theorem of vectors on a subspace, the auxiliary feature vector is decomposed into a one-dimensional subspace component spanned by the principal feature vector and its orthogonal complementary space component. Since the calculated dot product of the positive interactive complementary feature and the principal feature vector is theoretically strictly equal to zero, it mathematically ensures that the redundant components collinear with the principal feature vector in the auxiliary feature vector are completely eliminated, retaining only the complementary components that are strictly orthogonal to the principal feature vector. This achieves a geometric severing of the pollution path in the shared feature space. The positive interactive complementary feature refers to the aligned feature vector that, after being stripped of the collinear redundant components with the principal feature vector through orthogonal complementary space projection, retains only the complementary information that is strictly orthogonal to the principal feature vector, and its dot product with the principal feature vector is always equal to zero.

[0094] Step S23: Perform amplitude attenuation and element-by-element fusion based on positive interactive complementary features to obtain the fused feature vector.

[0095] After obtaining the positive interactive complementary features, the auxiliary modal confidence scalar, and the principal feature vector, in order to ensure that the injection intensity of the positive interactive complementary features strictly matches the auxiliary modal confidence scalar, and to avoid diluting the discriminative power of the principal feature vector by injecting the positive interactive complementary features at the same amplitude when the signal with relatively low physical confidence is severely degraded, a confidence-guided amplitude attenuation and element-by-element fusion mechanism is established. The purpose is to enable the fused feature vector to smoothly degenerate into a single principal feature vector when the signal with relatively low physical confidence completely fails.

[0096] Specifically, each element of the positive interactive complementary feature is multiplied by the auxiliary modality confidence scalar to obtain the weighted complementary feature after amplitude attenuation. Then, the weighted complementary feature is added element-wise to the main feature vector to calculate the fused feature vector. The calculation process of amplitude attenuation and element-wise fusion is based on the signal-to-noise ratio weighted multi-source information complementary enhancement principle. The main feature vector, as the unattenuated baseline backbone, ensures the lower bound of the feature discriminative power. The positive interactive complementary feature, as a beneficial increment gated by the auxiliary modality confidence scalar, injects information in a direction strictly orthogonal to the main feature vector, geometrically ensuring that the injection of incremental information does not distort the original directional distribution of the main feature vector. The fused feature vector refers to the final multimodal joint representation vector, constructed by element-wise addition of the main feature vector and the positive interactive complementary feature attenuated by the auxiliary modality confidence scalar, which retains the complete discriminative structure of the main feature vector and adaptively incorporates the positive interactive complementary feature. See also Figure 4 This is a schematic diagram of a confidence-guided cross-orthogonal decoupling and feature reconstruction spatial vector provided in this embodiment. Figure 4 The diagram illustrates a two-dimensional feature projection plane constructed with the principal eigenvector as the geometric reference axis. The longest thick solid arrow pointing horizontally to the right from the origin O represents the principal eigenvector with higher physical confidence; the thin solid arrow pointing diagonally upwards to the right from the origin represents the auxiliary eigenvector with relatively lower physical confidence; the thick dashed arrow on the axis containing the principal eigenvector, pointing from the origin to the foot of the projection perpendicular, represents the projection vector of the auxiliary eigenvector in the direction of the principal eigenvector. The thin solid arrow pointing vertically upwards from the origin represents the positive interactive complementary feature, strictly orthogonal to the principal eigenvector after cross-orthogonal decoupling to remove redundant components; the thick solid arrow pointing vertically upwards from the origin, but with a shortened length, represents the weighted complementary feature obtained by multiplying the positive interactive complementary feature with the auxiliary modality confidence scalar. Using the parallelogram rule, as shown in the dashed box, the weighted complementary feature is vector-added with the principal eigenvector; the thick solid arrow at the diagonal position represents the fused feature vector. This figure visually reveals that relatively degraded signal information is scaled and injected into a direction strictly orthogonal to the principal feature vector, based on physical reliability as a dynamic gain. This achieves the feature reconstruction principle of cutting off the noise pollution path from the bottom layer, under the premise of zero perturbation in the distribution of the principal feature vector direction.

[0097] Step S20 addresses the technical challenges of existing feature-level static splicing mechanisms, such as the direct mapping of high-dimensional noise features of relatively degraded signal data to the shared hidden layer feature space, the forced distortion of high-quality signal data distribution, and the induction of negative intermodal migration effects. It achieves physical credibility-guided geometric localization of the main path, orthogonal stripping of redundant components, and adaptive complementary fusion with confidence gating. Specifically, the main-auxiliary feature pair identifies signals with higher physical credibility as the uncontaminated main reference direction, resolving the disorder problem caused by the lack of quality assessment in the symmetric fusion mechanism, where high-dimensional noise dominates the fusion direction. The positive interactive complementary feature removes collinear redundant components along the main feature vector direction from the auxiliary feature vector, mathematically cutting off the contamination propagation path from relatively degraded data to the core main feature. The fused feature vector ensures that when signals with relatively low physical credibility completely fail, the multimodal joint representation can smoothly degenerate into a single main feature vector, guaranteeing the complete discriminative structure of the main feature vector while adaptively incorporating the positive interactive complementary feature.

[0098] Step S30: Obtain the state feature library and bind it with the fused feature vector to obtain the reference feature set. Calculate the Mahalanobis distance for each operating state based on the reference feature set. Perform a loop traversal comparison between the Mahalanobis distance and the fused feature vector to obtain the device state identification result.

[0099] Further, step S30 includes:

[0100] Step S31: Obtain the state feature library and bind it with the fused feature vector to obtain the reference feature set.

[0101] After obtaining the fused feature vector, in order to enable the fused feature vector to directly complete the state discrimination on edge devices with limited computing power, and to avoid introducing complex parameter iteration and high computing power overhead, a state comparison benchmark in the same unified feature space as the fused feature vector is constructed.

[0102] During the calibration phase before system deployment, a status feature library was constructed for various operating states of distribution network equipment. These operating states include normal operation, overheat warning, insulation degradation, and mechanical failure, each assigned a status label. The normal operation state refers to a healthy operating state where the electrical parameters and surface temperature field of the distribution network equipment are within the design limits. The overheat warning state refers to an early degradation state where increased contact resistance at key connection points causes abnormal local temperature rises, but has not yet led to significant distortion of the electrical waveform. The insulation degradation state refers to an abnormal state where the internal insulation medium of the distribution network equipment is damp or aged, accompanied by partial discharge pulses causing high-frequency abrupt changes in electrical signals, and the equipment itself exhibits abnormal hotspot distribution. The mechanical failure state refers to a failure state where severe physical deformation caused by external mechanical stress or internal structural damage results in transient drops in electrical waveforms and significant distortion of image texture features. Specifically, for each of the above operating states, multiple historical electrical and visual signals from the distribution network equipment are collected as calibration samples. The number of calibration samples is determined based on the statistical completeness requirements for covering different environmental interference types under that operating state. For example, the number of calibration samples for each operating state is set to 500. The fused feature vector of each calibration sample is obtained. Since each fused feature vector is a one-dimensional numerical vector, for all fused feature vectors under the same operating state, an arithmetic mean is calculated element by element according to the element order of the one-dimensional numerical vector. Specifically, the numerical elements of all fused feature vectors in the same order are summed, and the sum is divided by the total number of calibration samples to calculate the arithmetic mean corresponding to that order. The arithmetic means calculated for all orders are combined and arranged according to their original order to obtain the standard reference feature vector for that operating state. Simultaneously, the covariance matrix of all fused feature vectors under that operating state is calculated, and the inverse of the covariance matrix is ​​obtained to obtain the inverse covariance matrix for that operating state.

[0103] Each of the above-mentioned operating states is bound to its respective state label, standard reference feature vector, and inverse covariance matrix to obtain a reference feature set. The standard reference feature vector is a representative vector that characterizes the distribution center of a certain operating state in a unified feature space, and whose dimension is strictly aligned with the fused feature vector. The state feature library is an offline data structure that permanently stores the reference feature set and is pre-deployed in edge devices.

[0104] Step S32: Calculate the Mahalanobis distance for each operating state based on the reference feature set, and perform a loop comparison between the Mahalanobis distance and the fused feature vector to obtain the device state identification result.

[0105] After obtaining the fused feature vector and the reference feature set, in order to accurately identify the fused feature vector to the most matching operating state and ensure the lightweight deployment characteristics of edge devices that are free from parameter fine-tuning and plug-and-play, a parameterless state matching mechanism based on algebraic distance metric is established. The goal is to complete high-precision state classification solely through deterministic algebraic operations.

[0106] For each operating state in the reference feature set, the Mahalanobis distance between its corresponding standard reference feature vector and the fused feature vector is calculated. Specifically, the standard reference feature vector of the current operating state is subtracted element-wise from the fused feature vector to obtain the difference vector. The transpose of the difference vector, the inverse covariance matrix of the current operating state, and the difference vector are then multiplied sequentially to obtain a scalar result. The square root of the scalar result is then calculated. If the result is negative due to numerical error, the absolute value is taken first, and then the square root is calculated to obtain the Mahalanobis distance between the fused feature vector and the current operating state. This calculation process is based on the statistical distance principle that considers the covariance structure of the feature distribution. It is used to measure the degree to which the fused feature vector deviates from the distribution center of each operating state in a unified feature space. The smaller the Mahalanobis distance, the more the current fused feature vector belongs to that operating state.

[0107] After obtaining all Mahalanobis distances between the fused feature vector and all operating states, a loop is performed to compare and determine the optimal match. Specifically, two storage spaces are allocated in the memory of the edge device, defined as the minimum distance record and the optimal label cache size, respectively. The initial value of the minimum distance record is set to positive infinity, and the initial value of the optimal label cache size is set to null. The Mahalanobis distance corresponding to each operating state is traversed sequentially and compared with the current value in the minimum distance record. If the current Mahalanobis distance is less than the value in the minimum distance record, the minimum distance record is updated to the current Mahalanobis distance, and the optimal label cache size is updated to the state label of that operating state. If the current Mahalanobis distance is greater than or equal to the value in the minimum distance record, the minimum distance record and the optimal label cache size remain unchanged. The loop terminates after all operating states have been traversed and compared. The standard reference feature vector corresponding to the final state label saved in the optimal label cache size after the loop terminates is defined as the optimal matching feature, and this state label is used as the device state identification result. The optimal matching feature refers to the standard reference feature vector that has the smallest Mahalanobis distance to the fused feature vector and is determined to be the operating state to which the current distribution network device belongs; the device state identification result refers to the state label characterizing the operating state of the distribution network device in the current operating cycle, which is deterministically output by the parameterless distance metric matching. See also Figure 5 This is a schematic diagram of the state discrimination space for parameterless distance metric matching provided in an embodiment of the present invention. Figure 5The paper presents a unified feature metric space composed of fused feature vectors and multiple operational state distribution centers. Figure 5 The horizontal and vertical axes in the diagram represent "Unified Feature Space Dimension 1" and "Unified Feature Space Dimension 2" respectively. These two dimensions represent the isomorphic geometric space constructed after the aforementioned deterministic feature extraction and linear mapping based on the covariance matrix. They are the abstract principal component directions in the unified feature space, which aim to map the originally high-dimensional feature data to a two-dimensional visualization plane for measurement and display. Figure 5 Four circular markers located at different positions in space represent the standard reference feature vectors for the normal operation state, overheat warning state, insulation degradation state, and mechanical fault state, respectively. The dashed elliptical lines surrounding each circular marker represent the state distribution contour lines determined by the inverse covariance matrix of the corresponding operating state. The triangle marker, slightly to the right of center, represents the currently extracted fused feature vector. The line segments pointing from the triangle markers to the four circular markers represent the Mahalanobis distances between the fused feature vectors and each operating state. Among them, the line segment pointing to the circular marker for the overheat warning state has the shortest distance and is displayed in bold, representing the minimum Mahalanobis distance determined in the iterative comparison. The circular marker corresponding to this overheat warning state is also highlighted with a bold border to indicate that it is the optimal matching feature, and its corresponding state label serves as the equipment state identification result.

[0108] Step S30, through the reference feature set, optimal matching features, and device status recognition results, solves the technical problem of existing technologies relying on large trainable neural network classifiers for edge fine-tuning after multimodal fusion, resulting in excessive computing power overhead for edge computing nodes and an inability to be plug-and-play. It achieves parameter-free fine-tuning and extremely lightweight high-precision status perception based on pure mathematical algebraic distance measurement. Specifically, the reference feature set transforms the various operating states of distribution network equipment into deterministic distribution centers and inverse covariance matrices within a unified feature space, constructing a fixed state comparison benchmark that requires no edge gradient updates or weight fine-tuning. The optimal matching features and device status recognition results calculate the Mahalanobis distance between the fused feature vector and each benchmark and perform iterative comparisons. Utilizing the covariance structure of the feature distribution, it accurately measures the degree of state deviation in isomorphic geometric space, thereby deterministically outputting the current operating state of the distribution network equipment with extremely low time complexity without any backpropagation or gradient updates.

[0109] Example 2

[0110] This embodiment, based on Embodiment 1, provides a multimodal feature fusion system for distribution network equipment status awareness, such as... Figure 6 As shown, it includes:

[0111] Feature alignment module: used to acquire multimodal sensing signals, perform signal-to-noise ratio quantization on the multimodal sensing signals to obtain a confidence vector group, and perform deterministic feature extraction and unified spatial alignment on the multimodal sensing signals to obtain an aligned feature set;

[0112] Orthogonal decoupling module: used to jointly align feature set and confidence vector group to perform main and auxiliary feature localization, obtain main and auxiliary feature pairs, perform cross-orthogonal decoupling on main and auxiliary feature pairs to obtain positive interactive complementary features, and perform amplitude attenuation and element-wise fusion based on positive interactive complementary features to obtain fused feature vector;

[0113] State matching module: used to acquire state feature library, bind it with fused feature vector to obtain reference feature set, calculate Mahalanobis distance for each operating state based on reference feature set, and perform loop traversal comparison with Mahalanobis distance and fused feature vector to obtain device state recognition result.

[0114] In the feature alignment module, the process involves acquiring the multimodal sensing signal, performing signal-to-noise ratio quantization on the multimodal sensing signal to obtain a confidence vector set, and performing deterministic feature extraction and unified spatial alignment on the multimodal sensing signal to obtain an aligned feature set, including:

[0115] Step S11: Acquire the multimodal sensing signal, perform signal-to-noise ratio quantization on the multimodal sensing signal, and obtain a confidence vector group;

[0116] Step S12: Perform deterministic feature extraction and unified spatial alignment on the multimodal sensing signal to obtain an aligned feature set.

[0117] In the orthogonal decoupling module, the joint aligned feature set and confidence vector group perform primary and secondary feature localization to obtain primary and secondary feature pairs. Cross-orthogonal decoupling is then performed on the primary and secondary feature pairs to obtain positive interactive complementary features. Based on the positive interactive complementary features, amplitude attenuation and element-wise fusion are performed to obtain a fused feature vector, including:

[0118] Step S21: Perform primary and secondary feature localization by jointly aligning the feature set and the confidence vector group to obtain primary and secondary feature pairs;

[0119] Step S22: Perform cross-orthogonal decoupling on the primary and secondary feature pairs to obtain positive interactive complementary features;

[0120] Step S23: Perform amplitude attenuation and element-by-element fusion based on positive interactive complementary features to obtain the fused feature vector.

[0121] In the state matching module, the state feature library is acquired and bound to the fused feature vector to obtain a reference feature set. Based on the reference feature set, the Mahalanobis distance for each operating state is calculated. The Mahalanobis distance and the fused feature vector are then combined and iterated repeatedly to obtain the device state identification result, including:

[0122] Step S31: Obtain the state feature library and bind it with the fused feature vector to obtain the reference feature set;

[0123] Step S32: Calculate the Mahalanobis distance for each operating state based on the reference feature set, and perform a loop comparison between the Mahalanobis distance and the fused feature vector to obtain the device state identification result.

[0124] In addition, the parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of the corresponding technical solutions in the prior art have not been described in detail, so as to avoid excessive elaboration.

[0125] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely preferred embodiments of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A multimodal feature fusion method for distribution network equipment status perception, characterized in that, The method includes: Acquire multimodal sensing signals, perform signal-to-noise ratio quantization on the multimodal sensing signals to obtain a confidence vector set, and perform deterministic feature extraction and unified spatial alignment on the multimodal sensing signals to obtain an aligned feature set; The joint alignment feature set and confidence vector group are used to perform primary and secondary feature localization to obtain primary and secondary feature pairs. Cross-orthogonal decoupling is performed on the primary and secondary feature pairs to obtain positive interactive complementary features. Amplitude attenuation and element-wise fusion are performed based on the positive interactive complementary features to obtain the fused feature vector. The system acquires a state feature library and binds it with the fused feature vector to obtain a reference feature set. Based on the reference feature set, it calculates the Mahalanobis distance for each operating state. The system then performs a loop traversal comparison with the fused feature vector to obtain the device state identification result.

2. The multimodal feature fusion method for distribution network equipment status perception according to claim 1, characterized in that, The method for obtaining the confidence vector group includes: Acquire multimodal sensing signals, which include electrical signals and visual signals; A sliding time window is set on the electrical signal, and the variance of the high-frequency components of the electrical signal within the sliding time window is calculated. The visual signal is divided into multiple local image blocks, and the local information entropy of the visual signal in all local image blocks is calculated. Based on the preset signal-to-noise ratio quantization function, the variance of high-frequency components is mapped to electrical confidence, and the local information entropy is mapped to visual confidence. The electrical confidence score and the visual confidence score are combined to obtain a confidence vector set.

3. The multimodal feature fusion method for distribution network equipment status perception according to claim 2, characterized in that, The method for calculating the variance of the high-frequency components includes: The discrete wavelet transform algorithm is called to perform single-level decomposition on the electrical signal within the sliding time window, resulting in a high-frequency detail coefficient sequence composed of high-frequency detail coefficients; Calculate the arithmetic mean of all high-frequency detail coefficients in the high-frequency detail coefficient sequence; The high-frequency component variance is obtained by subtracting the arithmetic mean from each high-frequency detail coefficient in the high-frequency detail coefficient sequence, squaring the result, summing all the squared results, and dividing by the total number of high-frequency detail coefficients.

4. The multimodal feature fusion method for distribution network equipment status perception according to claim 2, characterized in that, The method for calculating the local information entropy includes: The probability of occurrence of each gray level is obtained by counting the number of pixels corresponding to each gray level within each local image block and dividing the corresponding number of pixels by the total number of pixels in that local image block. The probability of each gray level is multiplied by its base-2 logarithm and the result is negative. When the probability of occurrence is zero, the contribution of that gray level is set to zero and the accumulation is skipped. The results of the calculations for all gray levels are summed to obtain the information entropy of the local image patch.

5. The multimodal feature fusion method for distribution network equipment status perception according to claim 2, characterized in that, The method for obtaining the alignment feature set includes: Extracting timing features from electrical signals; The pre-deployed image feature extraction network model is invoked to extract spatial features of visual signals; By using a pre-set linear mapping layer with a unified target dimension, the spatial features are subjected to dimensionality reduction projection to obtain the dimensionality-reduced spatial component vector, and the time series features are subjected to zero-mean unit variance standardization to obtain the standardized electrical component vector. The reduced spatial component vector and the standardized electrical component vector are collectively defined as an alignment feature set, which includes visual alignment feature vectors and electrical alignment feature vectors.

6. The multimodal feature fusion method for distribution network equipment status perception according to claim 5, characterized in that, The acquisition of the primary and secondary feature pairs includes: Compare the numerical values ​​of electrical confidence and visual confidence; If the electrical confidence level is greater than the visual confidence level, the electrical alignment feature vector is defined as the main feature vector, the visual alignment feature vector is defined as the auxiliary feature vector, and the visual confidence level is defined as the auxiliary modal confidence scalar. If the visual confidence is greater than the electrical confidence, the visual alignment feature vector is defined as the main feature vector, the electrical alignment feature vector is defined as the auxiliary feature vector, and the electrical confidence is defined as the auxiliary modal confidence scalar. If the visual confidence and electrical confidence are equal, the electrical alignment feature vector is defined as the primary feature vector, the visual alignment feature vector is defined as the secondary feature vector, and the visual confidence is defined as the secondary modal confidence scalar, based on the preset default priority. The principal feature vector, the auxiliary feature vector, and the auxiliary modality confidence scalar are collectively defined as a principal-auxiliary feature pair.

7. The multimodal feature fusion method for distribution network equipment status perception according to claim 6, characterized in that, The method for obtaining the positive interactive complementary features includes: Calculate the dot product between the auxiliary feature vector and the main feature vector to obtain the cross dot product value; Calculate the dot product between the principal eigenvector and itself to obtain the dot product value; Divide the cross inner product value by the self inner product value to obtain the projection coefficient; Multiplying the projection coefficients by the principal eigenvector yields the projection vector of the secondary eigenvector onto the direction of the principal eigenvector. Subtracting the corresponding element of the projection vector from the auxiliary feature vector element by element yields the positive interactive complementary feature that is orthogonal to the main feature vector.

8. The multimodal feature fusion method for distribution network equipment status perception according to claim 7, characterized in that, The method for obtaining the fused feature vector includes: Multiply each element of the positive interactive complementary feature with the auxiliary modality confidence scalar to obtain the weighted complementary feature after amplitude attenuation; The weighted complementary features are added element-wise to the main feature vector to obtain the fused feature vector.

9. The multimodal feature fusion method for distribution network equipment status perception according to claim 1, characterized in that, The method for obtaining the reference feature set includes: Multiple operating states are obtained from the state feature library. For each operating state, multiple calibration samples are collected and the fusion feature vector corresponding to each calibration sample is obtained. At the same time, each operating state is assigned a state label. For all fused feature vectors under the same operating state, perform an arithmetic average operation on each element according to the order of their arrangement to obtain the standard reference feature vector for that operating state. Calculate and invert the covariance matrix of all fused feature vectors under this operating state to obtain the inverse covariance matrix of this operating state; The reference feature set is obtained by binding the status label, standard reference feature vector, and inverse covariance matrix of each operating state.

10. The multimodal feature fusion method for distribution network equipment status perception according to claim 9, characterized in that, The method for calculating the Mahalanobis distance includes: The difference vector is obtained by subtracting the standard reference feature vector of the current running state from the fused feature vector element by element. Perform matrix multiplication operations on the transpose of the difference vector, the inverse covariance matrix of the current running state, and the difference vector in sequence to obtain a scalar result; Perform the arithmetic square root operation on the scalar result to obtain the Mahalanobis distance between the fused feature vector and the current running state.

11. The multimodal feature fusion method for distribution network equipment status perception according to claim 10, characterized in that, The method for obtaining the device status identification result includes: Allocate the minimum distance record and the optimal tag cache size in memory, and set the initial value of the minimum distance record to positive infinity and the initial value of the optimal tag cache size to null. Iterate through the Mahalanobis distance corresponding to each running state and compare it with the current value in the minimum distance record; If the current Mahalanobis distance is less than the value in the minimum distance record, then update the minimum distance record to the current Mahalanobis distance, and update the optimal label cache size to the status label of this running state; After all operating states have been traversed and compared, the final saved state label will be used as the device state identification result.

12. A multimodal feature fusion system for distribution network equipment status perception, used to implement the multimodal feature fusion method for distribution network equipment status perception as described in any one of claims 1-11, characterized in that, The system includes: Feature alignment module: used to acquire multimodal sensing signals, perform signal-to-noise ratio quantization on the multimodal sensing signals to obtain a confidence vector group, and perform deterministic feature extraction and unified spatial alignment on the multimodal sensing signals to obtain an aligned feature set; Orthogonal decoupling module: used to jointly align feature set and confidence vector group to perform main and auxiliary feature localization, obtain main and auxiliary feature pairs, perform cross-orthogonal decoupling on main and auxiliary feature pairs to obtain positive interactive complementary features, and perform amplitude attenuation and element-wise fusion based on positive interactive complementary features to obtain fused feature vector; State matching module: used to acquire state feature library, bind it with fused feature vector to obtain reference feature set, calculate Mahalanobis distance for each operating state based on reference feature set, and perform loop traversal comparison with Mahalanobis distance and fused feature vector to obtain device state recognition result.

Images