Multi-modal data deep fusion method, system, equipment and medium

Abstract

The invention discloses a multi-modal data deep fusion method, system and device and a medium, and the method comprises the steps: obtaining the multi-source fault associated data of a power distribution network, and verifying and unifying space-time labels; constructing a space-time grid coding system, and forming an index structure containing time, space and data source identifiers; key features such as temperature change rate and vibration frequency are extracted, and initialization classification is carried out; feature matching of fault features and grid attributes is realized by dynamically adjusting grids and weights; and through normalizing multi-modal data, reconstructing spatial-temporal distribution, evaluating a fusion effect, and outputting a deep fusion result. According to the method, the deep data fusion effect is remarkably improved, powerful support is provided for fault prediction and health management of the power distribution network, and the reliability and operation efficiency of the power grid are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of data fusion technology, and in particular to a method, system, device and medium for deep fusion of multimodal data. Background Technology

[0002] In power distribution network fault management, data from multiple sources are often used, such as data collected by SCADA systems, fault waveform records, meteorological information, and equipment condition monitoring data. Each of these data has its own characteristics and varies greatly in time and space, thus exhibiting significant spatiotemporal heterogeneity.

[0003] Existing data fusion methods also face numerous challenges when processing large batches of heterogeneous data. For example, these methods often fail to adequately verify temporal and spatial consistency and possess weak structured representation capabilities, making it difficult to effectively establish spatiotemporal correlations between different modalities. Feature extraction thus becomes incomplete, and the accuracy of modality matching decreases, ultimately affecting the accuracy of fault prediction, fault location, and health assessment. Particularly in scenarios with rapidly changing distribution network operating states and high data concurrency, traditional methods often lack a unified spatiotemporal grid coding mechanism, failing to efficiently support the alignment, fusion, and analysis of multidimensional data. These issues compromise the real-time performance and reliability of intelligent decision-making. Therefore, a grid-based deep fusion method for multimodal data is indeed needed. By constructing a spatiotemporal grid system covering the entire distribution network, standardizing the access of data from various sources, and achieving precise alignment and efficient fusion, the distribution network can operate more safely, stably, and intelligently. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method, system, device, and medium for deep fusion of multimodal data. It addresses the problem that existing technologies, when processing multi-source heterogeneous data from distribution networks, lack effective spatiotemporal grid coding and deep fusion mechanisms, making it difficult to achieve temporal and spatial consistency alignment, accurate feature extraction, and modal matching. This results in inaccurate fault prediction, poor health management, and untimely response, ultimately affecting the safety and efficiency of power grid operation.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for deep fusion of multimodal data, comprising: Obtain fault-related data of the power distribution network, perform consistency verification and format unification on the timestamps and geographic coordinates of each data, and obtain a spatiotemporal label dataset; Based on the aforementioned spatiotemporal label dataset, a spatiotemporal grid coding system covering the power distribution area is constructed. Each piece of data is mapped to the corresponding spatiotemporal grid cell according to its corresponding spatiotemporal label, forming a coding index structure that includes time coding, spatial coding, and data source identification. Based on the aforementioned coding index structure, key features related to distribution network faults are extracted, and the data within the grid is initialized and classified according to the key features to obtain initial classification data containing fault type, impact range, and estimated cause. Based on the initial classification data, the fault characteristics of the distribution network are matched with the predefined attributes of the corresponding spatiotemporal grid. By dynamically adjusting the grid boundary, grid size and data point weight, the alignment relationship between data points and grid is optimized, and feature matching results are generated. Based on the feature matching results, normalization processing is performed on the multimodal data to reconstruct the temporal and spatial distribution of the data. The deep fusion effect is evaluated based on the adjusted location prediction values, and the deep fusion result is output.

[0007] As a preferred embodiment of the multimodal data deep fusion method described in this invention, the encoded index structure includes: Based on the time label of each data point in the spatiotemporal label dataset, the data is divided into preset time intervals and corresponding time codes are generated; Based on the geographic coordinates of each data point in the spatiotemporal label dataset, the data is mapped to a two-dimensional spatial grid covering the power distribution area and a corresponding spatial code is generated. The longitude and latitude differences between the data and the center of the grid are calculated, and the offset is obtained by normalizing the grid width and height. Based on the data source type of each data entry in the spatiotemporal label dataset, a unique data source identifier code is assigned to the data to distinguish between meteorological systems, sensor networks, and power distribution network fault records.

[0008] The beneficial effects of this preferred technical solution are that by unifying the three dimensions of time, space and data source and introducing normalized offset, it achieves accurate alignment and efficient indexing of multi-source heterogeneous data in the spatiotemporal grid, providing a consistent, comparable and locatable data foundation for subsequent feature matching and deep fusion.

[0009] As a preferred embodiment of the multimodal data deep fusion method described in this invention, the key features related to power distribution network faults include the temperature change rate in meteorological data and the vibration frequency in sensor data. The rate of temperature change is obtained by differential calculation of the meteorological temperature sequence within a continuous time window; The vibration frequency is obtained by performing frequency domain analysis on the received sensor sampling signal.

[0010] As a preferred embodiment of the multimodal data deep fusion method described in this invention, the dynamic adjustment of grid boundaries, grid size, and data point weights includes: Based on the data point density distribution within the power distribution area, refine the grid size in high-density areas and coarse the grid size in low-density areas; After adjusting the grid size, calculate the spatial distance between each data point and the center of its respective grid, and set the position weight of the corresponding data point; Based on the aforementioned position weights, recursive segmentation and merging operations are performed on the grid boundaries.

[0011] The beneficial effects of this preferred technical solution are that it adaptively optimizes the grid granularity and boundary according to the data distribution, and performs recursive fine-tuning in combination with position weights, which effectively improves the spatial alignment accuracy and fusion efficiency of the grid for multi-source heterogeneous data.

[0012] As a preferred embodiment of the multimodal data deep fusion method described in this invention, the feature matching result includes: Calculate the matching degree index between the fault characteristics of the distribution network and the corresponding spatiotemporal grid historical attributes; Based on the matched fault data, the number of faults occurring in each spatiotemporal grid per unit time is counted. Based on the fault records and processing times corresponding to the number of fault occurrences, the processing efficiency parameters of each spatiotemporal grid are determined.

[0013] As a preferred embodiment of the multimodal data deep fusion method described in this invention, the adjusted position prediction value includes: Obtain multiple known observation locations and their corresponding location weight coefficients; Based on the known observation locations and location weight coefficients, the weighted deviation is calculated in conjunction with the overall data mean. Based on the weighted deviation and the overall data mean, the final predicted location coordinates are generated.

[0014] As a preferred embodiment of the multimodal data deep fusion method described in this invention, the spatiotemporal label dataset includes: temperature, humidity, and wind direction labels from meteorological data; power, frequency, and operating status labels from sensor data; and fault type, fault duration, and fault handling status labels from power distribution network fault data.

[0015] Secondly, the present invention provides a multimodal data deep fusion system, comprising: The spatiotemporal data integration module is used to acquire fault-related data of the power distribution network, perform consistency verification and format unification of the timestamps and geographic coordinates of each data, and obtain a spatiotemporal tag dataset. The spatiotemporal grid coding module is used to construct a spatiotemporal grid coding system covering the power distribution area based on the spatiotemporal label dataset, and to map each piece of data to the corresponding spatiotemporal grid unit according to the corresponding spatiotemporal label, forming a coding index structure that includes time coding, spatial coding and data source identification. The multimodal feature analysis and classification module is used to extract key features related to distribution network faults based on the coding index structure, and to perform initial classification of data within the grid according to the key features to obtain initial classification data containing fault category, impact range and estimated cause; The grid-fault feature matching optimization module is used to perform feature matching between the fault characteristics of the distribution network and the predefined attributes of the corresponding spatiotemporal grid based on the initial classification data. By dynamically adjusting the grid boundary, grid size and data point weight, the alignment relationship between data points and grid is optimized to generate feature matching results. The spatiotemporal fusion and normalization output module is used to perform normalization processing on multimodal data based on the feature matching results, reconstruct the temporal and spatial distribution of the data, evaluate the deep fusion effect based on the adjusted position prediction values, and output the data deep fusion results.

[0016] Thirdly, the present invention provides an electronic device, comprising: Memory, used to store programs; A processor for executing the computer-executable instructions, which, when executed by the processor, implement the steps of the multimodal data deep fusion method.

[0017] Fourthly, the present invention provides a computer-readable storage medium, comprising: when the program is executed by a processor, the step of implementing the multimodal data deep fusion method.

[0018] The beneficial effects of this invention are as follows: By constructing a spatiotemporal grid coding system covering the power distribution area and standardizing and unifying spatiotemporal labels and performing triple coding (time, space, and data source) indexing on multi-source heterogeneous data, this invention achieves accurate alignment and efficient organization of multimodal data under a unified reference system; by extracting the temperature change rate from meteorological data and the vibration frequency from sensor data as key features, and combining them with the initial classification of data within the grid, this invention enables rapid preliminary identification of the type, scope of impact, and estimated causes of power distribution network faults. By dynamically adjusting the grid boundaries, grid size, and data point weights, and introducing a recursive segmentation and merging mechanism based on data density, the grid structure is adaptively matched to the actual data distribution, improving the accuracy of spatial modeling. By performing multi-dimensional feature matching between the fault characteristics of the distribution network and the historical attributes of the spatiotemporal grid, a comprehensive result including matching degree, fault frequency, and processing efficiency is generated, enabling a collaborative evaluation of fault modes and operation and maintenance response capabilities. By performing normalization processing based on the feature matching results, reconstructing the spatiotemporal distribution, and introducing a weighted location prediction model, a quantitative evaluation and high-quality output of the deep fusion effect of multimodal data are achieved. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a schematic diagram of the basic process of a multimodal data deep fusion method provided in one embodiment of the present invention. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0021] Example 1, referring to Figure 1 As an embodiment of the present invention, a method for deep fusion of multimodal data is provided, comprising: S100: Obtain distribution network fault correlation data, perform consistency verification and format unification on the timestamps and geographic coordinates of each data, and obtain a spatiotemporal label dataset; S200: Based on the spatiotemporal label dataset, construct a spatiotemporal grid coding system covering the power distribution area, map each data to the corresponding spatiotemporal grid unit according to the corresponding spatiotemporal label, and form a coding index structure containing time coding, spatial coding and data source identification; S300: Based on the coded index structure, extract key features related to distribution network faults, perform initial classification on the data in the grid according to the key features, and obtain initial classification data containing fault type, impact range and estimated cause; S400: Based on the initial classification data, the fault characteristics of the distribution network are matched with the predefined attributes of the corresponding spatiotemporal grid. By dynamically adjusting the grid boundary, grid size and data point weight, the alignment relationship between data points and grid is optimized, and feature matching results are generated. S500: Based on the feature matching results, normalization processing is performed on the multimodal data to reconstruct the temporal and spatial distribution of the data, and the deep fusion effect is evaluated based on the adjusted position prediction values, and the data deep fusion result is output.

[0022] It should be noted that in existing technologies, the multi-source heterogeneous data collected during the operation of distribution networks (such as meteorological information, sensor monitoring signals, and fault records) typically have different time granularities, coordinate systems, and data formats, lacking a unified spatiotemporal reference benchmark, making it difficult to align and correlate the data. Traditional fusion methods often use static grids or simple time window divisions, which cannot adapt to the dynamic characteristics of uneven data distribution in distribution areas (such as dense data in urban areas and sparse data in suburban areas), resulting in inaccurate spatial modeling. At the same time, the feature extraction process often processes a single data source in isolation, failing to fully explore the coupling relationship of multimodal data in the same spatiotemporal context, thus limiting the accuracy of fault classification and prediction. In addition, existing solutions generally lack a quantification mechanism and adaptive optimization capability for the matching quality of data points and grids, making it difficult to achieve stable and efficient deep fusion and real-time decision support when facing high-noise, low-coverage, or sudden fault scenarios.

[0023] Therefore, addressing the aforementioned issues in existing technologies for processing multi-source heterogeneous data from distribution networks, which suffer from a lack of effective spatiotemporal grid coding and deep fusion mechanisms, making it difficult to achieve consistent temporal and spatial alignment, accurate feature extraction, and modal matching, leading to inaccurate fault prediction, poor health management, and untimely response, thus affecting the safety and efficiency of power grid operation, this paper addresses these problems by constructing an adaptive multimodal data fusion framework based on spatiotemporal grid coding through steps S100-S500. This framework achieves unified spatiotemporal alignment, dynamic grid optimization, feature collaborative matching, and deep fusion of multi-source heterogeneous data from meteorological, sensor, and distribution network fault sources, significantly improving the accuracy of distribution network fault identification, the precision of spatial positioning, and the robustness and interpretability of multi-source data fusion.

[0024] Example 2, this is an embodiment of the present invention, which provides a multimodal data deep fusion method based on the previous embodiment, including: In this embodiment of the application, step S100 collects distribution network fault association data from different sources, including distribution network fault data, sensor data and meteorological data, filters records with initial timestamps and geographic location identifiers, identifies the voltage, current, temperature and humidity of the data, and obtains the raw data. The execution flow is as follows: In the process of collecting data related to power distribution network faults, data from various sources, including fault data, sensor data, and meteorological data, is integrated. This requires precise screening of records with preliminary timestamps and geographical locations, involving batch data processing, including data cleaning and verification. For example, fault records are extracted from automated power distribution systems, containing detailed information about the time and location of the fault. Data from different types of sensors, such as voltage, current, temperature, and humidity sensors, also need to be synchronized and labeled to ensure consistency in time and geographical information. Meteorological data, such as temperature and humidity, are also collected and correlated with power distribution network fault data to analyze the impact of meteorological conditions on fault occurrence. After preliminary screening, records lacking key timestamps or geographical location markers are excluded to ensure the integrity of the dataset and the effectiveness of subsequent analysis. This provides a solid data foundation for power distribution network fault analysis and prevention, enhances the accuracy of data-driven decision-making, and obtains raw data.

[0025] In this embodiment, the spatiotemporal label dataset in step S100 includes: temperature, humidity, and wind direction labels from meteorological data; power, frequency, and operating status labels from sensor data; and fault type, fault duration, and fault handling status labels from distribution network fault data. The encoding index structure includes time encoding, spatial encoding, and data source identifier encoding. Initialized classification data includes the fault category of the initial group, the predicted fault impact range, and the estimated fault cause. Feature matching results include the correspondence between fault features and grid characteristics, the frequency of fault occurrence within the grid, and the fault handling efficiency. The deep data fusion results include the normalized dataset, adjusted time labels, and spatial coordinates.

[0026] In this embodiment of the application, the spatiotemporal consistency verification method in step S100 includes using a geographic information system to spatially correct the geographic coordinates of the original data and simultaneously calibrating its timestamp; then calculating the comprehensive spatiotemporal offset of each data before and after verification, which comprehensively considers changes in geographic location and time differences; finally comparing the offset with a preset threshold, if it does not exceed the threshold, it is determined to be valid data and included in the spatiotemporal label dataset, otherwise it is discarded or re-corrected.

[0027] In an optional implementation, the spatiotemporal consistency verification method in step S100 can also first independently judge the geographic coordinate offset and timestamp deviation of each data. If the spatial offset does not exceed the preset distance threshold and the time deviation does not exceed the preset time window, the data is retained as valid spatiotemporal label data; otherwise, it is discarded or marked as unreliable.

[0028] In an optional implementation, the spatiotemporal consistency verification method in step S100 can also generate a smooth spatiotemporal trajectory by interpolation or filtering using adjacent valid observation points for data with missing timestamps or abrupt changes in geographic coordinates, and use the reconstructed trajectory points as calibrated data to achieve spatiotemporal consistency alignment.

[0029] In this embodiment of the application, the specific method for verifying the consistency between the timestamps and geographic coordinates of each data in step S100 includes: Based on the original data, a Geographic Information System (GIS) was used to spatially correct the latitude and longitude of the original data, and the timestamps were calibrated simultaneously. Furthermore, the combined offset of each data point before and after verification was calculated using the following spatiotemporal joint deviation formula: in, , For the longitude and latitude before verification, , For the verified longitude and latitude, , These are the corresponding timestamps. The preset spatiotemporal conversion coefficient is used to convert the time difference into an equivalent spatial distance unit. If the calculated Δs is less than the preset threshold, the data point is determined to meet the spatiotemporal consistency requirements and is included in the spatiotemporal label dataset; otherwise, it is removed or recalibrated.

[0030] For example, suppose a data point was located at [location] before verification. , , After verification, it is , , Take the spatiotemporal transformation coefficient ,but: If the preset threshold is 1.0, then the data point passes the consistency check.

[0031] In this embodiment of the application, the encoding index structure in step S200 includes: Based on the time label of each data point in the spatiotemporal label dataset, the data is divided into preset time intervals and corresponding time codes are generated; Based on the geographic coordinates of each data point in the spatiotemporal label dataset, the data is mapped to a two-dimensional spatial grid covering the power distribution area and a corresponding spatial code is generated. The longitude and latitude differences between the data and the center of the grid are calculated, and the offset is obtained by normalizing the grid width and height. Based on the data source type of each data entry in the spatiotemporal label dataset, a unique data source identifier code is assigned to the data to distinguish between meteorological systems, sensor networks, and power distribution network fault records.

[0032] In this embodiment of the application, the calculation of the data point offset within the grid in step S200 involves comparing the latitude and longitude of the data point with the latitude and longitude of the center of its spatiotemporal grid, and combining the grid's dimensions in the longitude and latitude directions to calculate a normalized comprehensive offset, thereby quantifying the degree of deviation of the data point's relative position within the grid.

[0033] In an optional implementation, the calculation of the data point offset within the grid in step S200 can also calculate the Euclidean distance from the data point to the center of the grid, and divide it by the length of the grid diagonal to obtain a dimensionless normalized offset value, which is used to characterize the relative deviation of the data point within the grid.

[0034] In an optional implementation, the calculation of the data point offset within the grid in step S200 can also be based on the historical data distribution characteristics of the power distribution area to construct a covariance matrix, and use this matrix to calculate the Mahalanobis distance between the data point and its grid center to reflect the degree of directional sensitivity deviation considering spatial anisotropy.

[0035] In this embodiment of the application, the offset G in step S200 is calculated using the following formula: in, and These are the longitude and latitude of the data points, respectively. and These are the longitude and latitude of the center of the spatiotemporal grid, respectively. The width of the grid along the longitude direction. The latitude of the grid is represented by the height of the grid. The offset G characterizes the degree of deviation of the data point from its relative position within its grid. The smaller the value, the closer the data point is to the grid center, and the higher the matching confidence, providing a quantitative basis for subsequent feature matching and weight allocation. For example, let the longitude of the data point be... ,latitude The grid center to which it belongs is , The grid width and height are both ,but: A smaller value indicates that the data point is close to the center of the grid, and the matching reliability is high.

[0036] In this embodiment of the application, the key features related to the power distribution network fault in step S300 include the temperature change rate in meteorological data and the vibration frequency in sensor data; The rate of temperature change is obtained by differential calculation of the meteorological temperature series over a continuous time window; The vibration frequency is obtained by performing frequency domain analysis on the received sensor sampling signal.

[0037] In this embodiment of the application, the machine learning model used for initial classification in step S300 includes a supervised learning model such as support vector machine or decision tree based on key features such as temperature change rate and vibration frequency to perform initial classification of data, and then iterative optimization of classification results through random forest or neural network to improve the accuracy of fault category, impact range and cause prediction.

[0038] In an optional implementation, the machine learning model used for initializing classification in step S300 can first perform unsupervised clustering on features such as temperature change rate and vibration frequency to discover the inherent grouping of data, and then domain experts assign fault semantic labels according to the statistical characteristics of each cluster, and combine preset rules to map the new data to the corresponding fault category.

[0039] In an alternative implementation, the machine learning model used to initialize classification in step S300 can also construct a graph structure by taking distribution network equipment as nodes and electrical connections as edges, inputting multimodal features into the graph neural network, and using its message passing mechanism to aggregate neighborhood information, thereby jointly predicting the fault category, impact range and cause of each node based on the power grid topology context.

[0040] In this embodiment of the application, step S300 calculates the statistical characteristics of the temperature change rate sequence or vibration frequency sequence to support initial classification. The statistical characteristics include the mean μ and the standard deviation σ, which are calculated using the following formulas: in, Indicates the first The characteristic values ​​of a data point (such as the rate of temperature change or vibration frequency at a certain moment). This represents the total number of data points within the time window. This characterizes the average level of the feature within the window. It reflects the degree of fluctuation; the mean and standard deviation are used to construct classification criteria. For example, when μ exceeds the preset threshold and σ is small, it is judged as a continuous high temperature risk; when σ increases significantly, it indicates that the equipment may have abnormal vibration, thereby helping to generate initial classification data containing fault category, impact range and estimated cause.

[0041] When processing meteorological and sensor data, identifying the rate of temperature change and vibration frequency are key steps. Referring to a set of meteorological temperature data [22, 23, 22, 21, 22], the statistical characteristics of the data are calculated using the formulas for mean and standard deviation, and the mean is calculated. for: The standard deviation σ is calculated as follows: Obtaining the mean and standard deviation of the data is part of the feature analysis results, providing a quantitative basis for subsequent data classification and fault diagnosis.

[0042] In this embodiment of the application, step S400, which dynamically adjusts the grid boundary, grid size, and data point weights, includes: Based on the data point density distribution within the power distribution area, refine the grid size in high-density areas and coarse the grid size in low-density areas; After adjusting the grid size, calculate the spatial distance between each data point and the center of its respective grid, and set the position weight of the corresponding data point; Based on the aforementioned position weights, recursive segmentation and merging operations are performed on the grid boundaries.

[0043] In this embodiment of the application, the weight allocation mechanism in feature matching in step S400 includes comprehensively determining the weight of each data point in feature matching based on the offset between the data point and the center of its spatiotemporal grid and the reliability of the data source. The smaller the offset and the more reliable the source, the higher the weight, which is used to calculate the matching degree between fault characteristics and grid attributes in the subsequent calculation.

[0044] In an optional implementation, the weight allocation mechanism in feature matching in step S400 can also dynamically calculate the matching weight according to the spatial distance from the data point to the center of its spatiotemporal grid, based on an inverse power function relationship. The closer the distance, the higher the weight, thus emphasizing the influence of neighboring data points in feature matching.

[0045] In an optional implementation, the weight allocation mechanism in feature matching in step S400 can also construct a lightweight attention module, which takes the spatial offset of data points, data source type and historical matching error as input, automatically learns and dynamically generates its personalized weight in feature matching, and realizes context-aware allocation of the contribution of multi-source heterogeneous data.

[0046] In this embodiment of the application, the feature matching result in step S400 includes: Calculate the matching degree index between the fault characteristics of the distribution network and the corresponding spatiotemporal grid historical attributes; Based on the matched fault data, the number of faults occurring in each spatiotemporal grid per unit time is counted. Based on the fault records and processing times corresponding to the number of fault occurrences, the processing efficiency parameters of each spatiotemporal grid are determined.

[0047] In this embodiment of the application, the matching degree index R in step S400 is calculated using the following weighted fit formula: in, Indicates the first The degree of difference (e.g., Euclidean distance or normalization bias) between the fault characteristics of a data point and the predefined properties of its spatiotemporal grid. The location weight coefficient corresponding to the data point (determined by its offset from the grid center and the reliability of the data source) is summed to cover all matching fault data points in the current grid. The matching degree index R reflects the overall goodness of fit of the grid to the fault characteristics. The smaller the R value, the more consistent the fault characteristics are with the grid's historical attributes, and the higher the matching quality, providing a quantitative basis for subsequent grid optimization and deep fusion.

[0048] To evaluate the correspondence between distribution network fault data and spatiotemporal grid characteristics, five data points are set with differences between the data points and grid characteristics of [2, 1, 3, 2, 4], and corresponding weights of [1, 2, 1, 3, 1]. The goodness of fit is then... The calculation is as follows: The results reflect the average degree of matching between data characteristics and grid attributes, and are a key indicator for measuring the effectiveness of the optimized matching dataset.

[0049] In this embodiment of the application, the adjusted position prediction value in step S500 includes: Obtain multiple known observation locations and their corresponding location weight coefficients; Based on the known observation locations and location weight coefficients, the weighted deviation is calculated in conjunction with the overall data mean. Based on the weighted deviation and the overall data mean, the final predicted location coordinates are generated.

[0050] In this embodiment of the application, the location prediction value in step S500 Calculated using the following formula: in, Indicates the position to be predicted Predicted values ​​of target attributes (such as failure probability, temperature anomaly, or equipment condition score). The overall mean of all known observations. For the first Known observation locations The actual observed value at that location, The corresponding location weight coefficient (determined by spatial distance, data source reliability, and offset). This represents the total number of known data points involved in the prediction; for example, suppose there are three known observation points. , , Overall mean Weight , , Then the predicted location value The calculation is as follows: The results show that the predicted value for the data point at location a is 24. This result is determined by the combined observations of the three known points and their respective weights, demonstrating the practicality and accuracy of spatiotemporal data prediction.

[0051] In this embodiment of the application, the deep fusion effect within the same spatiotemporal grid is evaluated based on the adjusted time label and spatial coordinates, specifically including: Calculate the data coverage index to quantify the integrity of multi-source data within each grid after deep fusion; Compute data integration metrics to evaluate the numerical consistency and semantic coherence of different data sources within the same spatiotemporal unit; By comparing the differences between the original data and the adjusted data, the coverage and quality of deep integration are analyzed, providing quantifiable decision-making basis for the optimization of distribution network operation.

[0052] Example 3 is an embodiment of the present invention. This embodiment differs from the first embodiment in that it provides a multimodal data deep fusion system.

[0053] It should be noted that the technical solution of this multimodal data deep fusion system and the technical solution of the above-mentioned multimodal data deep fusion method belong to the same concept. For details not described in detail in the technical solution of the multimodal data deep fusion system in this embodiment, please refer to the description of the technical solution of the above-mentioned multimodal data deep fusion method.

[0054] This embodiment of a multimodal data deep fusion system includes: The spatiotemporal data integration module is used to acquire fault-related data of the power distribution network, perform consistency verification and format unification of the timestamps and geographic coordinates of each data, and obtain a spatiotemporal tag dataset. The spatiotemporal grid coding module is used to construct a spatiotemporal grid coding system covering the power distribution area based on the spatiotemporal label dataset, and to map each piece of data to the corresponding spatiotemporal grid unit according to the corresponding spatiotemporal label, forming a coding index structure that includes time coding, spatial coding and data source identification. The multimodal feature analysis and classification module is used to extract key features related to distribution network faults based on the coding index structure, and to perform initial classification of data within the grid according to the key features to obtain initial classification data containing fault category, impact range and estimated cause; The grid-fault feature matching optimization module is used to perform feature matching between the fault characteristics of the distribution network and the predefined attributes of the corresponding spatiotemporal grid based on the initial classification data. By dynamically adjusting the grid boundary, grid size and data point weight, the alignment relationship between data points and grid is optimized to generate feature matching results. The spatiotemporal fusion and normalization output module is used to perform normalization processing on multimodal data based on the feature matching results, reconstruct the temporal and spatial distribution of the data, evaluate the deep fusion effect based on the adjusted position prediction values, and output the data deep fusion results.

[0055] This embodiment also provides an electronic device applicable to a multimodal data deep fusion method, including: The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement a multimodal data deep fusion method as described in the above embodiments.

[0056] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements a multimodal data deep fusion method as proposed in the above embodiments.

[0057] The storage medium proposed in this embodiment and the method for implementing multimodal data deep fusion proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0058] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0059] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for deep fusion of multimodal data, characterized in that, include: Obtain fault-related data of the power distribution network, perform consistency verification and format unification on the timestamps and geographic coordinates of each data, and obtain a spatiotemporal label dataset; Based on the aforementioned spatiotemporal label dataset, a spatiotemporal grid coding system covering the power distribution area is constructed. Each piece of data is mapped to the corresponding spatiotemporal grid cell according to its corresponding spatiotemporal label, forming a coding index structure that includes time coding, spatial coding, and data source identification. Based on the aforementioned coding index structure, key features related to distribution network faults are extracted, and the data within the grid is initialized and classified according to the key features to obtain initial classification data containing fault type, impact range, and estimated cause. Based on the initial classification data, the fault characteristics of the distribution network are matched with the predefined attributes of the corresponding spatiotemporal grid. By dynamically adjusting the grid boundary, grid size and data point weight, the alignment relationship between data points and grid is optimized, and feature matching results are generated. Based on the feature matching results, normalization processing is performed on the multimodal data to reconstruct the temporal and spatial distribution of the data. The deep fusion effect is evaluated based on the adjusted location prediction values, and the deep fusion result is output.

2. The multimodal data deep fusion method as described in claim 1, characterized in that: The encoded index structure includes: Based on the time label of each data point in the spatiotemporal label dataset, the data is divided into preset time intervals and corresponding time codes are generated; Based on the geographic coordinates of each data point in the spatiotemporal label dataset, the data is mapped to a two-dimensional spatial grid covering the power distribution area and a corresponding spatial code is generated. The longitude and latitude differences between the data and the center of the grid are calculated, and the offset is obtained by normalizing the grid width and height. Based on the data source type of each data entry in the spatiotemporal label dataset, a unique data source identifier code is assigned to the data to distinguish between meteorological systems, sensor networks, and power distribution network fault records.

3. The multimodal data deep fusion method as described in claim 1 or 2, characterized in that: The key features associated with power distribution network faults include the rate of temperature change in meteorological data and the vibration frequency in sensor data. The rate of temperature change is obtained by differential calculation of the meteorological temperature sequence within a continuous time window; The vibration frequency is obtained by performing frequency domain analysis on the received sensor sampling signal.

4. The multimodal data deep fusion method as described in claim 3, characterized in that: The dynamic adjustment of grid boundaries, grid size, and data point weights includes: Based on the data point density distribution within the power distribution area, refine the grid size in high-density areas and coarse the grid size in low-density areas; After adjusting the grid size, calculate the spatial distance between each data point and the center of its respective grid, and set the position weight of the corresponding data point; Based on the aforementioned position weights, recursive segmentation and merging operations are performed on the grid boundaries.

5. The multimodal data deep fusion method as described in claim 4, characterized in that: The feature matching results include: Calculate the matching degree index between the fault characteristics of the distribution network and the corresponding spatiotemporal grid historical attributes; Based on the matched fault data, the number of faults occurring in each spatiotemporal grid per unit time is counted. Based on the fault records and processing times corresponding to the number of fault occurrences, the processing efficiency parameters of each spatiotemporal grid are determined.

6. The multimodal data deep fusion method as described in claim 5, characterized in that: The adjusted position prediction values ​​include: Obtain multiple known observation locations and their corresponding location weight coefficients; Based on the known observation locations and location weight coefficients, the weighted deviation is calculated in conjunction with the overall data mean. Based on the weighted deviation and the overall data mean, the final predicted location coordinates are generated.

7. The multimodal data deep fusion method as described in claim 6, characterized in that: The spatiotemporal label dataset includes: temperature, humidity, and wind direction labels from meteorological data; power, frequency, and operating status labels from sensor data; and fault type, fault duration, and fault handling status labels from power distribution network fault data.

8. A multimodal data deep fusion system, employing the method described in any one of claims 1-7, characterized in that, include: The spatiotemporal data integration module is used to acquire fault-related data of the power distribution network, perform consistency verification and format unification of the timestamps and geographic coordinates of each data, and obtain a spatiotemporal tag dataset. The spatiotemporal grid coding module is used to construct a spatiotemporal grid coding system covering the power distribution area based on the spatiotemporal label dataset, and to map each piece of data to the corresponding spatiotemporal grid unit according to the corresponding spatiotemporal label, forming a coding index structure that includes time coding, spatial coding and data source identification. The multimodal feature analysis and classification module is used to extract key features related to distribution network faults based on the coding index structure, and to perform initial classification of data within the grid according to the key features to obtain initial classification data containing fault category, impact range and estimated cause; The grid-fault feature matching optimization module is used to perform feature matching between the fault characteristics of the distribution network and the predefined attributes of the corresponding spatiotemporal grid based on the initial classification data. By dynamically adjusting the grid boundary, grid size and data point weight, the alignment relationship between data points and grid is optimized to generate feature matching results. The spatiotemporal fusion and normalization output module is used to perform normalization processing on multimodal data based on the feature matching results, reconstruct the temporal and spatial distribution of the data, evaluate the deep fusion effect based on the adjusted position prediction values, and output the data deep fusion results.

9. An electronic device, characterized in that, include: Memory, used to store programs; A processor for loading the program to perform the steps of the method as claimed in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, When the program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

Images