Large model-based power industry data analysis method, apparatus and device, and storage medium

Abstract

The invention discloses an electric power industry data analysis method, device and equipment based on a large model and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining historical multi-modal data of the electric power industry, and determining a target feature corresponding to the historical multi-modal data based on a preset multi-modal fusion strategy; according to the historical multi-modal data and the target features corresponding to the historical multi-modal data, performing fine tuning processing on a pre-trained initial large model to obtain a target large model; and constructing a corresponding knowledge graph based on a preset graph database technology and the historical multi-modal data, and carrying out data analysis on target multi-modal data, collected in real time, of the power industry by using the target large model and the knowledge graph. In this way, data of different modes can be integrated and analyzed through a multi-mode data fusion strategy. In addition, a multi-dimensional data analysis result can be determined through a large model technology and a knowledge graph technology.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large-model-based power industry data analysis method, device, equipment and storage medium. Background Art

[0002] In recent years, with the rapid development of information technologies such as big data, cloud computing, and the Internet of Things, the power industry is rapidly developing in the direction of intelligence and automation. The power industry is a typical high-complexity, high-risk industry, and its operating status is directly related to the national economy and social life. However, traditional power operation and maintenance methods often rely on manual experience, which is not only inefficient, but also difficult to cope with the growing amount of data and complex and changing business scenarios. In addition, data in the power industry usually presents the characteristics of multi-source heterogeneity.

[0003] At present, many AI-based solutions have been proposed and applied in the power industry, but most of them are limited to single-modal data processing or use relatively small-scale AI models, which makes it difficult to fully utilize the advantages of massive multimodal data. Especially in key links such as power grid dispatching, transmission inspection, and customer service, the limitations of traditional AI models are becoming more and more obvious.

[0004] In summary, how to effectively integrate and analyze multimodal data in the power industry is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method, device, equipment and storage medium for power industry data analysis based on a large model, which can effectively integrate and analyze multimodal data in the power industry. The specific scheme is as follows:

[0006] In a first aspect, the present application provides a method for analyzing data in the power industry based on a large model, comprising:

[0007] Acquire historical multimodal data of the power industry, and determine target features corresponding to the historical multimodal data based on a preset multimodal fusion strategy;

[0008] Fine-tuning the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain a target large model;

[0009] A corresponding knowledge graph is constructed based on the preset graph database technology and the historical multimodal data, and the target big model and the knowledge graph are used to perform data analysis on the target multimodal data of the power industry collected in real time.

[0010] Optionally, the determining the target features corresponding to the historical multimodal data based on a preset multimodal fusion strategy includes:

[0011] Determine a feature extraction method corresponding to each modal data, and extract features of each modal data corresponding to the historical multimodal data based on each feature extraction method;

[0012] Based on the preset multimodal fusion strategy, the features of each modal data corresponding to the historical multimodal data are encoded to obtain the target features corresponding to the historical multimodal data, so that the pre-trained initial large model can learn the target features corresponding to the historical multimodal data based on the preset multi-head attention mechanism.

[0013] Optionally, fine-tuning the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain a target large model includes:

[0014] The pre-trained initial large model is fine-tuned according to the historical multimodal data and the learning results of the target features corresponding to the historical multimodal data based on the preset multi-head attention mechanism to obtain the target large model.

[0015] Optionally, the using the target big model and the knowledge graph to perform data analysis on the target multimodal data of the power industry collected in real time includes:

[0016] Acquire the target multimodal data collected in real time through a preset data access interface;

[0017] Inputting the target multimodal data into the target large model, and obtaining a decision result output by the target large model;

[0018] Using the knowledge graph to perform data analysis on the target multimodal data to obtain corresponding multimodal data analysis results;

[0019] A target data analysis result of the target multimodal data is determined based on the decision result and the multimodal data analysis result.

[0020] Optionally, after performing data analysis on the target multimodal data of the power industry collected in real time using the target big model and the knowledge graph, the method further includes:

[0021] Updating the multimodal data in the knowledge graph based on the target multimodal data;

[0022] If it is determined that there is data to be deleted in the knowledge graph based on the timestamps corresponding to each of the multimodal data in the knowledge graph and preset error detection rules, the corresponding multimodal data in the knowledge graph is deleted.

[0023] Optionally, the large model-based power industry data analysis method further includes:

[0024] The target large model is updated and optimized by a preset online learning method and a preset incremental learning method to obtain an optimized target large model;

[0025] Or, acquiring multimodal data in a current time period every preset update period, and updating and optimizing the target macro model based on the multimodal data in the current time period to obtain an optimized target macro model;

[0026] Accordingly, the target large model and the knowledge graph are used to perform data analysis on the target multimodal data of the power industry collected in real time, including:

[0027] The optimized target big model and the knowledge graph are used to perform data analysis on the target multimodal data of the power industry collected in real time.

[0028] Optionally, the large model-based power industry data analysis method further includes:

[0029] Compressing the target large model based on a preset model compression technology to obtain a corresponding lightweight model;

[0030] The lightweight model is deployed to a target edge device so that the target edge device can directly perform data analysis on the target multimodal data collected in real time.

[0031] In a second aspect, the present application provides a large-scale model-based power industry data analysis device, comprising:

[0032] A target feature determination module is used to obtain historical multimodal data of the power industry and determine the target features corresponding to the historical multimodal data based on a preset multimodal fusion strategy;

[0033] A large model fine-tuning module, used for fine-tuning the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data, so as to obtain a target large model;

[0034] The data analysis module is used to construct a corresponding knowledge graph based on the preset graph database technology and the historical multimodal data, and use the target big model and the knowledge graph to perform data analysis on the target multimodal data of the power industry collected in real time.

[0035] In a third aspect, the present application provides an electronic device, including:

[0036] Memory, used to store computer programs;

[0037] The processor is used to execute the computer program to implement the aforementioned method.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, the computer program implements the aforementioned method when executed by a processor.

[0039] In this application, the historical multimodal data of the electric power industry is first obtained, and the target features corresponding to the historical multimodal data are determined based on the preset multimodal fusion strategy; then, the pre-trained initial large model is fine-tuned according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain the target large model; finally, a corresponding knowledge graph is constructed based on the preset graph database technology and the historical multimodal data, and the target large model and the knowledge graph are used to analyze the target multimodal data of the electric power industry collected in real time. As can be seen from the above, in this application, the target features of the historical multimodal data are first determined based on the preset multimodal fusion strategy, and the initial large model is optimized using the historical multimodal data and the target features of the historical multimodal data to obtain the target large model, and then the knowledge graph is constructed using the historical multimodal data, and finally the target multimodal data is analyzed in real time through the target large model and the knowledge graph. In this way, the multimodal data fusion strategy can effectively integrate data of different modes and perform data analysis. In addition, the present application can perform data analysis on multimodal data through large model technology, and can combine knowledge graph technology to determine multi-dimensional data analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0041] Figure 1 A flow chart of a data analysis method for the power industry based on a large model provided for this application;

[0042] Figure 2 A specific flow chart of a large-model-based power industry data analysis method provided for this application;

[0043] Figure 3 A schematic diagram of a system framework applicable to a large-model-based power industry data analysis solution provided in this application;

[0044] Figure 4 A schematic diagram of the structure of a large-scale model-based power industry data analysis device provided in this application;

[0045] Figure 5 A structural diagram of an electronic device provided for this application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Traditional power operation and maintenance methods often rely on manual experience, which is not only inefficient, but also difficult to cope with the growing amount of data and complex and changing business scenarios. In addition, data in the power industry usually presents the characteristics of multi-source heterogeneity. At present, many artificial intelligence-based solutions have been proposed and applied to the power industry, but most of them are limited to single-modal data processing, or use relatively small-scale AI models, which makes it difficult to fully utilize the advantages of massive multimodal data. Especially in key links such as power grid dispatching, transmission inspection, and customer service, the limitations of traditional artificial intelligence models are becoming more and more obvious. To this end, the present application provides a data analysis solution for the power industry based on a large model, which can effectively integrate and analyze multimodal data in the power industry.

[0048] See also Figure 1 As shown, the embodiment of the present invention discloses a method for analyzing data in the power industry based on a large model, which may include:

[0049] Step S11: Acquire historical multimodal data of the electric power industry, and determine target features corresponding to the historical multimodal data based on a preset multimodal fusion strategy.

[0050] In this embodiment, the data of the power industry usually includes multiple types such as images, videos, sounds, and texts. The acquired historical multimodal data of the power industry needs to go through a series of preprocessing steps, such as data cleaning by removing noise and outliers, converting multimodal data of different formats into a unified standard format, and ensuring that the multimodal data have the same timestamp to ensure the time series alignment of the multimodal data. The historical multimodal data of the power industry is preprocessed to ensure the quality and consistency of the data.

[0051] It can be understood that the above-mentioned determination of the target features corresponding to the historical multimodal data based on the preset multimodal fusion strategy may include: determining the feature extraction method corresponding to each modal data, and extracting the features of each modal data corresponding to the historical multimodal data based on each feature extraction method; encoding the features of each modal data corresponding to the historical multimodal data based on the preset multimodal fusion strategy to obtain the target features corresponding to the historical multimodal data, so that the pre-trained initial large model learns the target features corresponding to the historical multimodal data based on the preset multi-head attention mechanism. Specifically, first, for each modality of power industry data, corresponding feature extraction methods can be designed respectively. For example, for image data, a convolutional neural network (CNN) can be used to extract features; for text data, a Transformer model can be used to extract features; for video data, a temporal convolutional network (TCN) can be used to extract features; for sound data, a recurrent neural network (RNN) can be used to extract features. Then, based on the feature extraction method corresponding to each modal data, the feature vector of each modal data is extracted from the pre-collected historical multimodal data of the power industry. Then, the feature vectors of the power industry data of different modalities are converted into a unified vector representation through a fully connected layer or a linear transformation. Then, a cross-modal attention mechanism is introduced in the Transformer encoder, and the feature vectors of the power industry data of different modalities are processed using the Transformer encoder structure. In the output stage of the Transformer encoder, the feature vectors of the power industry data of different modalities can be fused through different fusion strategies. Common fusion strategies include early fusion, that is, splicing or weighted summing the feature vectors of the power industry data of different modalities in the feature extraction stage; mid-term fusion, that is, interacting and integrating the feature vectors of the power industry data of different modalities in the middle layer of the Transformer encoder; late fusion, that is, processing the feature vectors of the power industry data of different modalities independently and fusing them in the final decision layer. It can be understood that by adopting the Transformer-based multimodal fusion strategy, the features of each modal data corresponding to the pre-collected historical multimodal data of the power industry can be encoded into a unified feature representation, that is, the target features of the historical multimodal data. Finally, the pre-trained initial large model can learn the target features of historical multimodal data through a multi-head attention mechanism, so that it can learn the correlation between different modalities of power industry data, thereby improving the accuracy and robustness of large model data processing.

[0052] Step S12: fine-tune the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain a target large model.

[0053] In this embodiment, first, a large-scale public data set such as ImageNet, COCO, Wikipedia, etc. can be used to pre-train the basic large model so that the large model can obtain strong generalization capabilities. In order to train a large model suitable for the characteristics of the power industry and covering functions such as natural language processing (i.e., NLP, Natural Language Processing), computer vision (i.e., CV, Computer Vision), etc., the above-mentioned fine-tuning of the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain the target large model can include: fine-tuning the pre-trained initial large model according to the historical multimodal data and the learning results of the target features corresponding to the historical multimodal data based on the preset multi-head attention mechanism to obtain the target large model. Specifically, on the basis of pre-training the basic large model, the parameters of the large model are adjusted using the correlation between the pre-collected historical multimodal data of the power industry and the power industry data of different modalities learned through the multi-head attention mechanism, and the fine-tuning of the large model has been completed. The historical multimodal data of the power industry collected in advance include power grid operation data, fault cases, equipment status data, etc., to ensure the performance of the big model in the specific field of the power industry. After the target big model is determined, the target big model can be trained, and the performance of the target big model, such as accuracy, loss function, etc., can be monitored during the training process, and the performance of the target big model can be evaluated using the validation data set.

[0054] Step S13: construct a corresponding knowledge graph based on the preset graph database technology and the historical multimodal data, and use the target big model and the knowledge graph to perform data analysis on the target multimodal data of the power industry collected in real time.

[0055] In this embodiment, the pre-collected historical multimodal data of the power industry can first be preprocessed, such as cleaning the text data, removing irrelevant information, typos, redundant words, etc.; preprocessing the image and video data, such as cropping, scaling, denoising, etc.; performing noise reduction, speech recognition, etc. on the audio data, and converting it into an analyzable text form. Then, according to the characteristics of the historical multimodal data, the entities in the knowledge graph, such as equipment, fault types, etc., and the relationships in the knowledge graph, such as the connection relationship between equipment, the causal relationship between faults and causes, etc., are identified. The graph neural network (GNN, Graph Neural Network) in the GraphRAG (Graph-based Retrieval Augmented Generation, enhanced generation model based on graph retrieval) technology is used to encode and represent the entities and relationships of the knowledge graph. Then, according to the identified entities and relationships, the corresponding nodes and edges of the knowledge graph are constructed, and the graph retrieval and enhanced generation capabilities in the GraphRAG technology are used to supplement and improve the knowledge graph. The knowledge graph contains equipment information, fault cases, dispatch procedures, etc. related to the power industry, which can be used to enhance the AI-driven question-answering system.

[0056] It can be understood that the above-mentioned use of the target big model and the knowledge graph to analyze the target multimodal data of the power industry collected in real time can include: obtaining the target multimodal data collected in real time through the preset data access interface; inputting the target multimodal data into the target big model, and obtaining the decision result output by the target big model; using the knowledge graph to perform data analysis on the target multimodal data to obtain the corresponding multimodal data analysis result; determining the target data analysis result of the target multimodal data based on the decision result and the multimodal data analysis result. Specifically, the target multimodal data can be obtained in real time from different types of sensors, monitoring equipment, SCADA systems (SCADA, i.e. Supervisory Control And Data Acquisition), user feedback platforms, etc. The target multimodal data collected in real time includes but is not limited to the status information of power equipment, environmental monitoring data, user electricity consumption behavior data, social media feedback, etc. The target multimodal data is input into the target big model, and the target big model can analyze the target multimodal data collected in real time to provide decision support for activities such as scheduling, inspection, and customer service. For example, in the dispatching scenario, automatic fault diagnosis and disposal suggestion generation can be achieved; in customer service, a more humanized interactive experience can be provided. In addition, the multi-modal data analysis results obtained by analyzing the target multi-modal data using the knowledge graph can be combined to determine the multi-dimensional target data analysis results.

[0057] It should be noted that, in order to ensure the accuracy and timeliness of the knowledge graph, after the target multimodal data of the electric power industry collected in real time is analyzed using the target large model and the knowledge graph, it can also include: updating the multimodal data in the knowledge graph based on the target multimodal data; if it is determined that there is data to be deleted in the knowledge graph based on the timestamps corresponding to each of the multimodal data in the knowledge graph and the preset error detection rules, the corresponding multimodal data in the knowledge graph is deleted. Specifically, after the data analysis of the target multimodal data collected in real time, the automatic update capability of GraphRAG technology can be used to continuously learn and update the knowledge graph according to the target multimodal data collected in real time. And set the time threshold and preset error detection rules of the multimodal data in the knowledge graph. The time threshold defines the period when the multimodal data is judged to be outdated. The preset error detection rules can include data format check, data range verification, data consistency check, etc. Traverse the multimodal data in the knowledge graph, compare the timestamp corresponding to each multimodal data with the set time threshold, and apply the preset error detection rules to each multimodal data in the knowledge graph. Then, combine the comparison result of the timestamp and the time threshold with the detection result of the preset error detection rule to determine whether there is data to be deleted in the knowledge graph, and delete the corresponding multimodal data from the knowledge graph. At the same time, the time, reason and result of the data deletion operation can be recorded, and it is necessary to ensure that the deletion operation does not destroy the structure of the knowledge graph. In addition, a monitoring mechanism can be set to continuously track the status and performance of the knowledge graph, and the time threshold and preset error detection rules can be adjusted in a timely manner based on the monitoring results and user feedback.

[0058] In this embodiment, in order to adapt to the new multimodal data and business needs of the power industry, the above-mentioned data analysis of the target multimodal data of the power industry collected in real time can also include: updating and optimizing the target large model through a preset online learning method and a preset incremental learning method to obtain an optimized target large model; or, obtaining multimodal data within the current time period at every preset update period, and updating and optimizing the target large model based on the multimodal data within the current time period to obtain an optimized target large model; correspondingly, the target multimodal data collected in real time from the power industry using the target large model and the knowledge graph are analyzed, including: using the optimized target large model and the knowledge graph to analyze the target multimodal data collected in real time from the power industry. Specifically, the target large model can be continuously updated and optimized through online learning and incremental learning techniques, so that the target large model can better adapt to the new multimodal data of the power industry while maintaining the original performance. In the process of updating and optimizing the target large model, the performance change of the target large model can be monitored by evaluating the accuracy of the target large model on the validation set. If the performance of the target large model decreases or becomes unstable, it is necessary to stop fine-tuning and make adjustments immediately. In addition, a reasonable update cycle can be set according to the business needs and data update frequency of the power system. At the beginning of each update cycle, the multimodal data in the current time period is obtained from the preset data access interface, and the integrity and accuracy of the multimodal data need to be ensured to avoid data loss or damage. After cleaning the collected multimodal data and removing invalid, repeated or abnormal data, the collected multimodal data is format converted and feature extracted according to the model requirements, and the processed multimodal data is divided into training set, validation set and test set. In each update cycle, the current version of the target large model is loaded, the target large model is trained using the training set data, and the weights and parameters of the target large model are updated. At the same time, strategies such as incremental learning and transfer learning can be selected to accelerate the training process and reduce computing resource consumption. The performance of the optimized target large model, such as accuracy, recall rate, F1 score and other indicators, is evaluated on the validation set. If the performance of the optimized target large model meets expectations, the optimized target large model can be tested; otherwise, the training strategy or hyperparameters are adjusted and the optimized target large model is retrained. Finally, the performance of the optimized target large model can be further evaluated through the test set to ensure that the optimized target large model is stable and reliable in practical applications in the power industry. At the same time, the time, data source, training strategy, performance evaluation results and other information of each target large model update can be recorded. After obtaining the optimized target large model, the old version of the target large model can be replaced with the optimized target large model. During the actual operation of the power system, the performance changes of the target large model, including response time, accuracy and other indicators, are continuously monitored.If you find that the model performance has degraded or anomalies occur, you can take timely measures to repair or retrain it.

[0059] As can be seen from the above, in this embodiment, the historical multimodal data of the power industry is first obtained, and the target features corresponding to the historical multimodal data are determined based on the preset multimodal fusion strategy; then, the pre-trained initial large model is fine-tuned according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain the target large model; finally, the corresponding knowledge graph is constructed based on the preset graph database technology and the historical multimodal data, and the target large model and the knowledge graph are used to analyze the target multimodal data of the power industry collected in real time. As can be seen from the above, in this embodiment, the target features of the historical multimodal data are first determined based on the preset multimodal fusion strategy, and the initial large model is optimized using the historical multimodal data and the target features of the historical multimodal data to obtain the target large model, and then the knowledge graph is constructed using the historical multimodal data, and finally the target multimodal data is analyzed in real time through the target large model and the knowledge graph. In this way, the multimodal data fusion strategy can effectively integrate data of different modes and perform data analysis. In addition, this embodiment can perform data analysis on multimodal data through large model technology, and can combine knowledge graph technology to determine multi-dimensional data analysis results.

[0060] Based on the previous embodiment, it can be seen that the present application can analyze the multimodal data of the power industry by combining the large model technology with the knowledge graph technology. Next, in order to achieve low-latency and high-efficiency localized data processing, see Figure 2 As shown, the embodiment of the present invention further discloses a method for analyzing data in the power industry based on a large model, which may include:

[0061] Step S21: Acquire historical multimodal data of the electric power industry, and determine target features corresponding to the historical multimodal data based on a preset multimodal fusion strategy.

[0062] Step S22: fine-tune the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain a target large model.

[0063] Step S23: compress the target large model based on a preset model compression technology to obtain a corresponding lightweight model.

[0064] In this embodiment, certain application scenarios in the power industry, such as drone inspections and on-site fault diagnosis, have high requirements on the response speed of the power system, while traditional cloud computing methods have delays. In order to achieve low-latency, high-efficiency localized multimodal data processing and improve the response speed and stability of the power system, the target large model can be compressed into a corresponding lightweight model to adapt to the edge computing environment. Specifically, model compression technology includes but is not limited to model pruning, model quantization, model distillation, and weight clipping. For example, knowledge distillation technology can be used to transfer the knowledge of the target large model to a lightweight model.

[0065] Step S24: deploy the lightweight model to the target edge device so that the target edge device can directly perform data analysis on the target multimodal data collected in real time.

[0066] In this embodiment, in order to reduce data transmission delay, after compressing the target large model into a corresponding lightweight model, the lightweight model can be deployed to the target edge device. Specifically, according to the application scenario and application requirements of the power system, a suitable target edge device, such as a drone, a smart terminal, etc., is selected, and then the lightweight model is converted into a format supported by the edge device, and the lightweight model is further optimized according to the performance limitations of the edge device. For example, the storage requirements and computational complexity of the lightweight model are reduced by quantization technology, or unimportant model parameters are removed by pruning technology, and then the lightweight model is deployed to the target edge device to ensure that the lightweight model can run on the target edge device. During the operation of the lightweight model, the performance parameters of the target edge device, such as CPU usage, memory occupancy, power consumption, etc., are monitored. If performance bottlenecks or abnormal conditions are found, the model is adjusted and optimized in time. At the same time, the lightweight model can be updated regularly according to actual needs. For example, new multimodal data of the power industry is used for model retraining, or new algorithms and technologies are introduced for model optimization. By deploying the lightweight model to the target edge device, localized processing of multimodal data analysis can be achieved, thereby reducing the delay in data transmission.

[0067] For more specific processing procedures of the above steps S21 and S22, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0068] As can be seen from the above, this embodiment compresses the target large model into a corresponding lightweight model based on the preset model compression technology, and deploys the lightweight model to the target edge device so that the target edge device can directly perform data analysis on the target multimodal data collected in real time. In this way, through the design of a lightweight model, some functions of the target large model can be migrated to the target edge device, thereby achieving low-latency, high-efficiency localized multimodal data processing and improving the response speed and stability of the power system.

[0069] See also Figure 3 As shown, the embodiment of the present application further provides a system framework diagram for data analysis in the power industry based on a large model. The system can adopt a layered architecture design, with the bottom layer being the data acquisition layer, the middle layer being the core algorithm layer, and the top layer being the application service layer. The data acquisition layer is responsible for obtaining original multimodal data from various sensors, monitoring equipment, and user interfaces; the core algorithm layer is the core of the entire power system, including multimodal data fusion, large model training and optimization, knowledge graph construction and maintenance, and intelligent decision support; and the application service layer is an interface for end users, providing corresponding intelligent services according to different business needs, including intelligent dispatching systems, power transmission inspection systems, and customer service systems. Specifically, the intelligent dispatching system can generate a fault handling plan based on the data analysis results of the power industry based on the large model, and the dispatcher can view the suggestions through the power system interface, and make corrections and improvements. The power system can also automatically call the existing dispatching system to perform corresponding operations. In this way, not only can the accuracy and handling efficiency of fault diagnosis be greatly improved, but also human errors can be reduced and the overall stability of the power system can be improved. The power transmission inspection system can use large models to improve the ability to identify defects and hidden dangers. It can also inspect the transmission lines by combining drones, visualization devices and other equipment. The power system can also automatically identify the defect type and generate corresponding inspection reports. In this way, it can not only improve the inspection efficiency, but also timely discover and deal with potential problems, and improve the quality and efficiency of power transmission operation and maintenance. The intelligent customer service system based on large models can handle multiple rounds of customer conversations, understand customer demands, and automatically generate work order information. In addition, the power system can also provide personalized service suggestions to improve customer satisfaction, thereby improving the overall customer service experience. As can be seen from the above, by integrating advanced large model technology with multimodal data analysis and processing technology, it is possible to solve the complex problems faced by the power industry in multiple fields such as dispatching, transmission, and marketing, improve the operating efficiency and safety of the entire power system, and realize the intelligent management and optimization of the power system.

[0070] Accordingly, see Figure 4 As shown, the embodiment of the present application also provides a power industry data analysis device based on a large model, which may include:

[0071] The target feature determination module 11 is used to obtain historical multimodal data of the power industry and determine the target features corresponding to the historical multimodal data based on a preset multimodal fusion strategy;

[0072] A large model fine-tuning module 12, configured to fine-tune the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data, so as to obtain a target large model;

[0073] The data analysis module 13 is used to construct a corresponding knowledge graph based on the preset graph database technology and the historical multimodal data, and use the target big model and the knowledge graph to perform data analysis on the target multimodal data of the power industry collected in real time.

[0074] As can be seen from the above, in this application, the historical multimodal data of the power industry is first obtained, and the target features corresponding to the historical multimodal data are determined based on the preset multimodal fusion strategy; then, the pre-trained initial large model is fine-tuned according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain the target large model; finally, the corresponding knowledge graph is constructed based on the preset graph database technology and the historical multimodal data, and the target large model and the knowledge graph are used to analyze the target multimodal data of the power industry collected in real time. As can be seen from the above, in this application, the target features of the historical multimodal data are first determined based on the preset multimodal fusion strategy, and the initial large model is optimized using the historical multimodal data and the target features of the historical multimodal data to obtain the target large model, and then the historical multimodal data is used to construct the knowledge graph, and finally the target multimodal data is analyzed in real time through the target large model and the knowledge graph. In this way, the multimodal data fusion strategy can effectively integrate data of different modes and perform data analysis. In addition, the present application can perform data analysis on multimodal data through large model technology, and can combine knowledge graph technology to determine multi-dimensional data analysis results.

[0075] In some specific implementations, the target feature determination module 11 may include:

[0076] A feature extraction unit, used to determine a feature extraction method corresponding to each modal data, and extract features of each modal data corresponding to the historical multimodal data based on each feature extraction method;

[0077] A target feature determination unit is used to encode the features of each modal data corresponding to the historical multimodal data based on the preset multimodal fusion strategy to obtain the target features corresponding to the historical multimodal data, so that the pre-trained initial large model can learn the target features corresponding to the historical multimodal data based on a preset multi-head attention mechanism.

[0078] In some specific implementations, the large model fine-tuning module 12 may include:

[0079] A large model fine-tuning unit is used to fine-tune the pre-trained initial large model according to the historical multimodal data and the learning results of the target features corresponding to the historical multimodal data based on the preset multi-head attention mechanism to obtain the target large model.

[0080] In some specific implementations, the data analysis module 13 may include:

[0081] A multimodal data acquisition unit, used to acquire the target multimodal data collected in real time through a preset data access interface;

[0082] A decision result acquisition unit, used for inputting the target multimodal data into the target large model, and acquiring the decision result output by the target large model;

[0083] A first data analysis unit, configured to perform data analysis on the target multimodal data using the knowledge graph to obtain corresponding multimodal data analysis results;

[0084] A data analysis result determination unit is used to determine a target data analysis result of the target multimodal data based on the decision result and the multimodal data analysis result.

[0085] In some specific implementations, the large model-based power industry data analysis device may further include:

[0086] A multimodal data updating module, used to update the multimodal data in the knowledge graph based on the target multimodal data;

[0087] A multimodal data deletion module is used to delete the corresponding multimodal data in the knowledge graph if it is determined that there is data to be deleted in the knowledge graph based on the timestamps corresponding to each multimodal data in the knowledge graph and preset error detection rules.

[0088] In some specific implementations, the large model-based power industry data analysis device may further include:

[0089] A large model optimization module is used to update and optimize the target large model by a preset online learning method and a preset incremental learning method to obtain an optimized target large model; or, to obtain multimodal data within a current time period at every preset update period, and to update and optimize the target large model based on the multimodal data within the current time period to obtain an optimized target large model;

[0090] Accordingly, the data analysis module 13 may include:

[0091] The second data analysis unit is used to use the optimized target big model and the knowledge graph to perform data analysis on the target multimodal data of the power industry collected in real time.

[0092] In some specific implementations, the large model-based power industry data analysis device may further include:

[0093] A large model compression module, used to compress the target large model based on a preset model compression technology to obtain a corresponding lightweight model;

[0094] The third data analysis module is used to deploy the lightweight model to the target edge device so that the target edge device can directly perform data analysis on the target multimodal data collected in real time.

[0095] Furthermore, the present application also discloses an electronic device. Figure 5 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the large model-based power industry data analysis method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment can specifically be an electronic computer.

[0096] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0097] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0098] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the large model-based power industry data analysis method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

[0099] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned large-model-based power industry data analysis method disclosed above is implemented. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0100] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0101] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0102] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0103] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0104] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data analysis method for the power industry based on a large model, characterized in that: include: Acquire historical multimodal data of the power industry, and determine target features corresponding to the historical multimodal data based on a preset multimodal fusion strategy; Fine-tuning the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain a target large model; A corresponding knowledge graph is constructed based on the preset graph database technology and the historical multimodal data, and the target big model and the knowledge graph are used to perform data analysis on the target multimodal data of the power industry collected in real time.

2. The power industry data analysis method based on a large model according to claim 1 is characterized in that: The determining the target features corresponding to the historical multimodal data based on a preset multimodal fusion strategy includes: Determine a feature extraction method corresponding to each modal data, and extract features of each modal data corresponding to the historical multimodal data based on each feature extraction method; Based on the preset multimodal fusion strategy, the features of each modal data corresponding to the historical multimodal data are encoded to obtain the target features corresponding to the historical multimodal data, so that the pre-trained initial large model can learn the target features corresponding to the historical multimodal data based on the preset multi-head attention mechanism.

3. The power industry data analysis method based on a large model according to claim 2 is characterized in that: The fine-tuning of the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data to obtain a target large model includes: The pre-trained initial large model is fine-tuned according to the historical multimodal data and the learning results of the target features corresponding to the historical multimodal data based on the preset multi-head attention mechanism to obtain the target large model.

4. The power industry data analysis method based on a large model according to claim 1 is characterized in that: The use of the target big model and the knowledge graph to analyze the target multimodal data of the power industry collected in real time includes: Acquire the target multimodal data collected in real time through a preset data access interface; Inputting the target multimodal data into the target macro model, and obtaining a decision result output by the target macro model; Using the knowledge graph to perform data analysis on the target multimodal data to obtain corresponding multimodal data analysis results; A target data analysis result of the target multimodal data is determined based on the decision result and the multimodal data analysis result.

5. The power industry data analysis method based on a large model according to claim 1 is characterized in that: After using the target big model and the knowledge graph to analyze the target multimodal data of the power industry collected in real time, the method further includes: Updating the multimodal data in the knowledge graph based on the target multimodal data; If it is determined that there is data to be deleted in the knowledge graph based on the timestamps corresponding to each of the multimodal data in the knowledge graph and preset error detection rules, the corresponding multimodal data in the knowledge graph is deleted.

6. The power industry data analysis method based on a large model according to any one of claims 1 to 5, characterized in that: Also includes: The target large model is updated and optimized by a preset online learning method and a preset incremental learning method to obtain an optimized target large model; Or, acquiring multimodal data in a current time period every preset update period, and updating and optimizing the target macro model based on the multimodal data in the current time period to obtain an optimized target macro model; Accordingly, the target large model and the knowledge graph are used to perform data analysis on the target multimodal data of the power industry collected in real time, including: The optimized target big model and the knowledge graph are used to perform data analysis on the target multimodal data of the power industry collected in real time.

7. The power industry data analysis method based on a large model according to claim 1 is characterized in that: Also includes: Compressing the target large model based on a preset model compression technology to obtain a corresponding lightweight model; The lightweight model is deployed to a target edge device so that the target edge device can directly perform data analysis on the target multimodal data collected in real time.

8. A data analysis device for the power industry based on a large model, characterized in that: include: A target feature determination module is used to obtain historical multimodal data of the power industry and determine the target features corresponding to the historical multimodal data based on a preset multimodal fusion strategy; A large model fine-tuning module, used for fine-tuning the pre-trained initial large model according to the historical multimodal data and the target features corresponding to the historical multimodal data, so as to obtain a target large model; The data analysis module is used to construct a corresponding knowledge graph based on the preset graph database technology and the historical multimodal data, and use the target big model and the knowledge graph to perform data analysis on the target multimodal data of the power industry collected in real time.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the large model-based power industry data analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the large model-based power industry data analysis method as described in any one of claims 1 to 7.

Images