Data aggregation method and system based on energy big data

Abstract

The invention provides a data aggregation method and system based on energy big data. The data aggregation method based on the energy big data comprises the steps of obtaining modal information corresponding to to-be-archived energy data; classifying the energy data according to the modal information to obtain the category of the energy data; determining an archiving priority of the energy data according to pre-stored archiving requirements and categories; determining a mapping vector between the energy data and the archiving demand according to the category, the priority and the modal information; wherein the mapping vector is used for indicating an association relationship between the energy data and the archiving demand; and archiving the energy data according to the mapping vector. The invention relates to the technical field of energy data analysis, and can improve the accuracy of data aggregation based on energy big data.

Description

Technical Field

[0001] The present application relates to the technical field of energy data analysis, and in particular to a data aggregation method and system based on energy big data. Background Art

[0002] At present, with the development of big data technology, more and more enterprises and organizations tend to use data analysis methods to assist and supervise the process nodes and production processes of traditional production industries to improve production efficiency. In the energy industry, energy big data processing systems are usually used to analyze and archive structured data. However, this method has insufficient processing capabilities for unstructured data and lacks analysis of the correlation between different energy data, resulting in low accuracy of data analysis and data archiving results. Summary of the invention

[0003] In view of the above, it is necessary to propose a data aggregation method and system based on energy big data to solve the technical problem of low accuracy of data aggregation based on energy big data.

[0004] The present application provides a data aggregation method based on energy big data, which is applied to electronic devices, and the method includes: obtaining energy data to be archived, and obtaining modal information corresponding to the energy data; classifying the energy data according to the modal information to obtain the category of the energy data; wherein the category is used to characterize the position of the energy data in the energy industry; determining the archiving priority of the energy data according to pre-stored archiving requirements and the category, and the archiving priority is used to indicate the importance of the energy data in the energy industry; determining a mapping vector between the energy data and the archiving requirements according to the category, the priority and the modal information; wherein the mapping vector is used to indicate the association relationship between the energy data and the archiving requirements; and archiving the energy data according to the mapping vector.

[0005] In some embodiments, classifying the energy data according to the modal information to obtain the category of the energy data includes: encoding the energy data to obtain a first code; encoding the modal information to obtain a second code; determining a coding vector corresponding to the energy data according to the first code and the second code; inputting the coding vector into a pre-trained first model to obtain a category vector of the energy data; wherein the category vector is used to indicate the category of the energy data.

[0006] In some embodiments, the method also includes training the first model, and the training of the first model includes: encoding pre-acquired sample energy data and corresponding sample modal information to obtain sample codes; determining the confidence of the sample codes based on the sample energy data and the sample modal information; based on the sample codes, determining the corresponding prediction category vector according to a pre-built initial classification model; determining a first loss value of the initial classification model according to the label category vector corresponding to the sample code, the prediction category vector and the confidence; continuously updating the initial classification model based on the back propagation algorithm until the first loss value meets a preset condition, stopping updating the initial classification model, and obtaining the first model.

[0007] In some embodiments, determining the confidence of the sample encoding based on the sample energy data and the sample modal information includes: performing semantic analysis on the energy data to obtain semantic information of the energy data; determining the degree of match between the semantic information and the sample modal information; and determining the confidence of the sample encoding based on the degree of match.

[0008] In some embodiments, determining the first loss value of the initial classification model according to the label category vector corresponding to the sample encoding, the predicted category vector, and the confidence comprises: ; Wherein, Loss1 represents the first loss value of the initial classification model; S represents the confidence; i represents the index of the category; n represents the number of categories; A i represents the probability value corresponding to the i-th category in the label category vector; B i Represents the probability value corresponding to the i-th category in the predicted category vector.

[0009] In some embodiments, determining the mapping vector between the energy data and the archiving requirement based on the category, the priority and the modal information includes: generating prompt data based on the category, the modal information and a pre-stored prompt template; inputting the prompt data into a pre-trained generative model to obtain an initial mapping vector between the energy data and the archiving requirement; and determining the mapping vector between the energy data and the archiving requirement based on the priority and the initial mapping vector.

[0010] In some embodiments, the method also includes training the generative model, and the training of the generative model includes: generating sample prompt data based on pre-acquired label categories, sample modal information, and pre-stored prompt templates; inputting the sample prompt data into a pre-built initial generative model to obtain a prediction mapping vector between the energy data and the archiving requirements; determining a second loss value of the initial generative model based on the label mapping vector and the prediction mapping vector corresponding to the sample modal information; updating the initial generative model based on a back-propagation algorithm until the second loss value meets a preset condition, stopping updating the initial generative model, and obtaining the generative model.

[0011] In some embodiments, determining the second loss value of the initial generative model according to the label mapping vector and the prediction mapping vector corresponding to the sample modality information includes: ; Wherein, Loss2 represents the second loss value of the initial generative model; j represents the index of the dimension in the label mapping vector; m represents the number of dimensions in the label mapping vector; C j represents the mapping value corresponding to the jth dimension in the label mapping vector; D j Represents the mapping value corresponding to the j-th category in the predicted mapping vector.

[0012] An embodiment of the present application also provides a data aggregation system based on energy big data, the system comprising an electronic device and a server; the electronic device is used to obtain energy data to be archived, and obtain modal information corresponding to the energy data; the electronic device is used to classify the energy data according to the modal information to obtain the category of the energy data; wherein the category is used to characterize the position of the energy data in the energy industry; the electronic device is used to determine the archiving priority of the energy data according to pre-stored archiving requirements and the category, and the archiving priority is used to indicate the importance of the energy data in the energy industry; the electronic device is used to determine the mapping vector between the energy data and the archiving requirement according to the category, the priority and the modal information; wherein the mapping vector is used to indicate the association between the energy data and the archiving requirement; the electronic device is used to archive the energy data according to the mapping vector.

[0013] In some embodiments, the electronic device is also used to encode the energy data to obtain a first code; the electronic device is used to encode the modal information to obtain a second code; the electronic device is used to determine the encoding vector corresponding to the energy data based on the first code and the second code; the electronic device is used to input the encoding vector into a pre-trained first model to obtain a category vector of the energy data; wherein the category vector is used to indicate the category of the energy data.

[0014] It can be seen from the above technical solutions that the embodiment of the present application classifies the energy data to be archived according to the modal information of the energy data, obtains the category of the energy data, and thus characterizes the position of the energy data to be archived in the energy industry based on the quantitative data, and provides quantitative data support for subsequent data archiving. Then, the archiving priority of the energy data is determined according to the archiving requirements and categories to indicate the importance of the energy data in the energy industry. The association between the energy data and the archiving requirements can be corrected based on the archiving priority, and then the mapping vector between the energy data and the archiving requirements is determined according to the category, priority and modal information, so as to indicate the association between the energy data and the archiving requirements based on the quantitative data, and finally the energy data is archived according to the mapping vector. In this way, the importance of energy data of different modalities and different types can be determined during the data archiving process, and the energy data can be structured based on the mapping vector, which can improve the accuracy of data aggregation based on energy big data. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is an application scenario diagram of a data aggregation method based on energy big data provided in one embodiment of the present application.

[0016] Figure 2 This is a flow chart of a data aggregation method based on energy big data provided in one embodiment of the present application.

[0017] Figure 3 It is a flowchart of a method for training a first model provided in one embodiment of the present application.

[0018] Figure 4 It is a flowchart of a method for training a second model provided in one embodiment of the present application.

[0019] Figure 5 It is a functional module diagram of a data aggregation system based on energy big data provided in one embodiment of the present application.

[0020] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to more clearly understand the purpose, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth to facilitate a full understanding of the present application, and the embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments.

[0022] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used herein in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0024] The embodiment of the present application provides a data aggregation method based on energy big data, which can be applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0025] An electronic device can be any electronic product that can interact with customers, such as personal computers, tablet computers, smart phones, personal digital assistants (PDAs), game consoles, Internet Protocol Television (IPTV), smart wearable devices, etc.

[0026] The electronic device may also include a network device and / or a client device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0027] The networks where electronic devices are located include but are not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0028] like Figure 1 The diagram shows an application scenario of a data aggregation method based on energy big data provided by an embodiment of the present application. The data aggregation method based on energy big data provided by the present application can be applied to an electronic device 100. The electronic device 100 is communicatively connected to a server 200. The server 200 is used to store energy data to be archived and corresponding modal information. The electronic device 100 is used to obtain energy data to be archived, and to obtain modal information corresponding to the energy data; the energy data is classified according to the modal information to obtain the category of the energy data; wherein the category is used to characterize the position of the energy data in the energy industry. The electronic device 100 is also used to determine the archiving priority of the energy data according to the pre-stored archiving requirements and the category, and the archiving priority is used to indicate the importance of the energy data in the energy industry; according to the category, the priority and the modal information, a mapping vector between the energy data and the archiving requirements is determined; wherein the mapping vector is used to indicate the association between the energy data and the archiving requirements; and the energy data is archived according to the mapping vector.

[0029] like Figure 2 As shown, it is a flow chart of a data aggregation method based on energy big data provided by an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. A data aggregation method based on energy big data provided by an embodiment of the present application includes the following steps.

[0030] S20, obtaining energy data to be archived, and obtaining modal information corresponding to the energy data.

[0031] In one embodiment of the present application, the energy data to be archived is used to record and describe information in multiple fields in the energy industry. Specifically, energy data can be energy production and supply data, such as the production, import and export volume, and reserve volume of various energy sources. Energy production data can be used to characterize the mining and production of energy resources, and supply data can be used to characterize the supply capacity and stability of energy. Energy data can also be energy consumption and demand data, such as energy consumption, energy consumption structure, and energy demand forecasts of various departments and industries. Energy consumption data can be used to evaluate energy utilization efficiency, guide energy conservation, and optimize energy structure. Energy data can also be energy price data, such as market prices, price trends, and price indices of energy. Energy price data is of great reference significance for the operation of the energy market and the business decisions of energy companies. Energy data can also be energy emission data, such as carbon dioxide emissions and pollutant emissions of energy. Energy emission data is an important indicator for measuring the impact of the energy industry on the environment and evaluating environmental policies. Energy data can also be energy policy and technical data, such as energy policies and regulations of various countries, research and development and application of energy technologies, etc. Energy policy and technical data can help analyze the policy environment and technological progress of the energy market. In addition, from the perspective of data types, energy data is usually collected and recorded through energy equipment, sensors and measuring instruments. It includes data on energy production, transmission, distribution and use. These data usually have the characteristics of time series data, that is, they are arranged in chronological order and can record and reflect characteristics and trends that change over time. From the perspective of application scenarios, energy data can be used for energy production optimization, energy consumption monitoring, energy demand forecasting, energy conservation and emission reduction monitoring, energy market analysis, intelligent energy management systems, energy supply chain optimization, energy efficiency evaluation and many other aspects.

[0032] In one embodiment of the present application, energy data of multiple modes refers to data obtained from different fields or perspectives for the same description object in the energy industry. Among them, energy data of multiple modes covers multiple types, and each type of data can be regarded as a mode. Specifically, the modal information corresponding to energy data can be a text mode, including various energy-related reports, documents, records, etc., such as energy policy documents, equipment operation logs, energy transaction records, etc., which can provide background information, rule descriptions and historical data in the energy field. Modal information can also be an image mode. In the energy field, image data usually comes from surveillance cameras, drone photography, satellite images, etc. For example, energy data of image mode can be an energy equipment image obtained by photographing power lines, oil and gas pipelines, wind turbines and other equipment, which is used to monitor equipment status, identify safety hazards, etc. Modal information can also be a video mode, and energy data of video mode can be a continuous image sequence, which is used to provide richer image information in the time dimension, which can be used to monitor equipment operation, personnel operation, etc., as well as for recording and tracing safety events. Modal information can also be an audio mode, including equipment sound monitoring, environmental noise analysis, etc. For example, by analyzing the operating sound of the equipment, it is possible to determine whether the equipment is operating normally or has a fault. Modal information can also be sensor modal. The energy data of the sensor modal includes the measurement values ​​of various physical quantities such as temperature, pressure, flow, vibration, etc., which can be used to monitor the operating status of the equipment in real time and warn of potential faults.

[0033] S21, classifying the energy data according to the modal information to obtain a category of the energy data; wherein the category is used to characterize the position of the energy data in the energy industry.

[0034] In one embodiment of the present application, since the energy data to be archived corresponds to multiple modalities, and the energy data to be archived includes structured and unstructured data, in order to determine the quantitative relationship between each type of energy data when archiving the energy data, and to facilitate the subsequent data retrieval and data mining process to ensure the accuracy of the data retrieval results or data mining results, the energy data can be first classified according to the energy data and the corresponding modal information to determine the position of the energy data in the energy industry, and provide quantitative data support for the archiving of the energy data. Exemplarily, the category of energy data can be upstream, midstream or downstream. When the category of energy data is upstream, it indicates that the energy data is data generated by the upstream industry in the energy industry; when the category of energy data is downstream, it indicates that the energy data is data generated by the downstream industry in the energy industry.

[0035] Exemplarily, the modal information corresponding to energy production data includes, but is not limited to, sensor data (e.g., temperature, pressure, flow, etc.), image data (e.g., the operating status of energy equipment), sound data (e.g., abnormal sounds during the operation of energy equipment), etc. Since energy production data originates from the source of the energy industry (e.g., coal mining, oil refining, wind power generation, solar power generation, etc.), energy production data is usually located in the upstream of the energy industry and can reflect the energy production capacity. By analyzing energy production data, the efficiency, cost and potential risk points of energy production can be obtained.

[0036] In one embodiment of the present application, when data aggregation based on energy big data is performed, different modalities and different types of energy data can be integrated to provide data support for real-time monitoring of energy production conditions and prediction of demand changes.

[0037] In order to indicate the position of energy data in the energy industry according to the category of energy data, the category of energy data can be determined based on a pre-trained first model. Specifically, classifying energy data according to modal information to obtain the category of energy data includes: encoding energy data to obtain a first code; encoding modal information to obtain a second code; determining a coding vector corresponding to the energy data according to the first code and the second code; inputting the coding vector to the pre-trained first model to obtain a category vector of energy data; wherein the category vector is used to indicate the category of energy data.

[0038] Specifically, the energy data and the modal information can be encoded respectively according to a preset encoding algorithm to obtain a first encoding corresponding to the energy data and a second encoding corresponding to the modal information. Among them, the preset encoding algorithm can be a bag of words model, a word frequency inverse document frequency algorithm, or a word embedding algorithm, which is not limited in this application. The first encoding and the second encoding can be spliced ​​to obtain the encoding vector corresponding to the energy data to ensure that the encoding vector is used to represent the complete information in the energy data and the modal information; the spliced ​​first encoding and the second encoding can also be input into a pre-trained autoencoder model, and the vector output by the autoencoder model is used as the encoding vector corresponding to the energy data, so that the dimensions of the first encoding and the second encoding can be reduced, and the key information in the energy data and the modal information can be ensured in the encoding vector, thereby ensuring the subsequent classification accuracy while improving the efficiency of classifying the energy data. Among them, the first model can be a pre-trained model with data classification function, and the first model input data is the encoding vector corresponding to the energy data, and the output data of the first model is the category vector corresponding to the energy data. The category vector has values ​​of multiple dimensions, each dimension corresponds to a category, and the value of any dimension is used to represent the probability that the energy data belongs to the category corresponding to the dimension. Specifically, the category corresponding to the highest value in the category vector can be determined as the category corresponding to the energy data.

[0039] In one embodiment of the present application, in order to improve the accuracy of classifying energy data, the first model may be trained based on pre-acquired sample data. The sample data includes sample energy data and corresponding sample modal information, and the sample data also includes a corresponding label category vector. The label category vector is used to indicate the category of the sample energy data. For a specific method of training the first model based on sample data, please refer to Figure 3 The corresponding detailed description.

[0040] S22, determining an archiving priority of the energy data according to a pre-stored archiving requirement and the category, wherein the archiving priority is used to indicate the importance of the energy data in the energy industry.

[0041] In one embodiment of the present application, determining the archiving priority of energy data based on pre-stored archiving requirements and the category of energy data includes: performing semantic analysis on the archiving requirements to obtain semantic coding corresponding to the archiving requirements; determining the similarity between the semantic coding and the category vector; and determining the archiving priority of the energy data based on the similarity. The archiving requirement may be a requirement for long-term retention of multimodal or multi-dimensional data during data management. The archiving requirement is usually based on multiple aspects, including energy industry cost control, energy system performance optimization, energy data security and compliance, and future data access and utilization.

[0042] S23, determining a mapping vector between the energy data and the archiving requirement according to the category, the priority and the modal information; wherein the mapping vector is used to indicate an association relationship between the energy data and the archiving requirement.

[0043] In one embodiment of the present application, determining a mapping vector between energy data and archiving requirements based on the category of energy data, the priority of energy data and the modal information corresponding to the energy data includes: generating prompt data based on the category, modal information and a pre-stored prompt template; inputting the prompt data into a pre-trained generative model to obtain an initial mapping vector between energy data and archiving requirements; and determining a mapping vector between energy data and archiving requirements based on the priority and the initial mapping vector.

[0044] The pre-stored prompt template is used to format the category and modal information of the energy data. For example, the content of the prompt template may be "Task prompt: Please determine the mapping relationship between energy data and archiving requirements based on the following category information and modal information; Output prompt: Please output a mapping vector to characterize the above mapping relationship; Category information:; Modal information:; Archiving requirements:; Priority:". The generative model may be a GPT model or an LLM model, which is not limited in this application.

[0045] In one embodiment of the present application, the input data of the generative model is the prompt data, and the output data of the generative model is the initial mapping vector, which is used to characterize the association between the energy data predicted by the generative model and the archiving requirement. In order to improve the accuracy of the association, the initial mapping vector can be corrected according to the priority corresponding to the energy data. Specifically, the product of the priority and the initial mapping vector can be determined as the mapping vector corresponding to the energy data.

[0046] S24, archiving the energy data according to the mapping vector.

[0047] In one embodiment of the present application, for any archiving requirement, the information in the archiving requirement (e.g., text information, image information, etc.) can be confirmed as the first node; for any energy data and corresponding modal information, the energy data and modal information can be confirmed as the second node; the mapping vector can be determined as the edge weight between the first node and the second node. In this way, energy data can be archived in a structured manner, thereby improving the accuracy of data aggregation based on energy big data.

[0048] It can be seen from the above technical solutions that the embodiment of the present application classifies the energy data to be archived according to the modal information of the energy data, obtains the category of the energy data, and thus characterizes the position of the energy data to be archived in the energy industry based on the quantitative data, and provides quantitative data support for subsequent data archiving. Then, the archiving priority of the energy data is determined according to the archiving requirements and categories to indicate the importance of the energy data in the energy industry. The association between the energy data and the archiving requirements can be corrected based on the archiving priority, and then the mapping vector between the energy data and the archiving requirements is determined according to the category, priority and modal information, so as to indicate the association between the energy data and the archiving requirements based on the quantitative data, and finally the energy data is archived according to the mapping vector. In this way, the importance of energy data of different modalities and different types can be determined during the data archiving process, and the energy data can be structured based on the mapping vector, which can improve the accuracy of data aggregation based on energy big data.

[0049] like Figure 3 , is a flow chart of a method for training a first model provided in an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for training a first model provided in an embodiment of the present application includes the following steps.

[0050] S30, encoding the pre-acquired sample energy data and the corresponding sample modal information to obtain a sample code.

[0051] In one embodiment of the present application, the method for encoding the pre-acquired sample energy data and the corresponding sample modal information is the same as the method for encoding the energy data and modal information in step S21, and will not be repeated here.

[0052] S31, determining the confidence of the sample encoding according to the sample energy data and the sample modal information.

[0053] In one embodiment of the present application, according to the sample energy data and the sample modal information, determining the confidence of the sample encoding includes: performing semantic analysis on the energy data to obtain the semantic information of the energy data; determining the matching degree between the semantic information and the sample modal information; and determining the confidence of the sample encoding according to the matching degree. Among them, the semantic information of the energy data can be quantitative data, which is used to characterize the industrial information contained in the energy data. For example, the industrial information in the energy data that can be characterized by the semantic information, wherein the industrial information can characterize the energy supply and resource reserves, and can also characterize the energy consumption of different industries, and can also characterize the trend of energy prices, etc. Specifically, the energy data can be analyzed using a natural language processing model to obtain the semantic information of the energy data, and the matching degree between the semantic information and the modal information can be determined according to a preset matching degree calculation method. Among them, the preset matching degree calculation method can be a Euclidean distance algorithm, a cosine similarity algorithm, or a Hamming distance algorithm, and this application does not limit this.

[0054] In one embodiment of the present application, the confidence of the sample coding can be determined based on the matching degree of semantic information and modal information. Among them, the higher the matching degree of semantic information and modal information, the higher the similarity between the information contained in the energy data and the corresponding sample modal information, and the higher the confidence of the sample energy coding corresponding to the sample energy data; the lower the matching degree of semantic information and modal information, the lower the similarity between the information contained in the energy data and the corresponding sample modal information, and the lower the confidence of the sample energy coding corresponding to the sample energy data. Specifically, the matching degree of semantic information and sample modal information can be determined as the confidence of the corresponding sample coding.

[0055] S32: Based on the sample code and a pre-built initial classification model, determine a corresponding prediction category vector.

[0056] In one embodiment of the present application, the sample code can be input into a pre-built initial classification model to obtain a predicted category vector output by the initial classification model. The predicted category vector has values ​​of multiple dimensions, each dimension corresponds to a category, and the value of any dimension is used to characterize the probability that the sample energy data corresponding to the sample code belongs to the category corresponding to the dimension. Specifically, it can be determined that the category corresponding to the highest value in the predicted category vector is the category corresponding to the sample energy data. The initial classification model can be a multi-layer perceptron model, a convolutional neural network model, or a recurrent neural network model, and the present application does not limit this.

[0057] S33: Determine a first loss value of the initial classification model according to the label category vector corresponding to the sample encoding, the predicted category vector, and the confidence level.

[0058] In one embodiment of the present application, in order to quantitatively evaluate the performance of the initial classification model, the first loss value of the initial classification model can be determined according to the label category vector, the predicted category vector and the confidence corresponding to the sample encoding. The performance of the initial classification model is evaluated based on the first loss value, thereby providing data support for the subsequent update of the initial classification model. Specifically, the method for determining the first loss value satisfies the following relationship:

[0059] ;

[0060] Wherein, Loss1 represents the first loss value of the initial classification model; S represents the confidence; i represents the index of the category; n represents the number of categories; A i represents the probability value corresponding to the i-th category in the label category vector; B i Represents the probability value corresponding to the i-th category in the predicted category vector.

[0061] In one embodiment of the present application, a higher first loss value indicates that a higher degree of difference between the predicted category vector output by the initial classification model and the label category vector is, and a lower accuracy of the output result of the initial classification model is.

[0062] S34, continuously updating the initial classification model based on the back propagation algorithm until the first loss value meets a preset condition, stopping updating the initial classification model, and obtaining the first model.

[0063] In one embodiment of the present application, the initial classification model can be continuously updated based on the back propagation algorithm until the first loss value meets the preset condition, and the initial classification model is stopped from being updated to obtain a first model for classifying energy data. The preset condition may be that the first loss value is less than or equal to a preset termination threshold. When the first loss value is less than or equal to the preset termination threshold, it indicates that the difference between the predicted category vector output by the initial classification model and the label category vector is small, and the accuracy of the output result of the initial classification model is high. Exemplarily, the termination threshold may be 0.1.

[0064] like Figure 4 , is a flow chart of a method for training a second model provided in an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for training a second model provided in an embodiment of the present application includes the following steps.

[0065] S40, generating sample prompt data based on the pre-acquired label category, sample modality information and pre-stored prompt template.

[0066] In one embodiment of the present application, the method for generating sample prompt data based on pre-acquired label categories, sample modal information, and pre-stored prompt templates is the same as the method for obtaining prompt data in step S23, and will not be repeated here.

[0067] S41, inputting the sample prompt data into a pre-built initial generative model to obtain a prediction mapping vector between the energy data and the archiving demand.

[0068] In one embodiment of the present application, the sample prompt data can be input into a pre-built initial generative model to obtain a prediction mapping vector output by the initial generative model. The prediction mapping vector is used to quantitatively characterize the correlation between the energy data predicted by the initial generative model and the archiving requirements. The initial generative model can be a GPT model or an LLM, etc., which is not limited in the present application.

[0069] S42: Determine a second loss value of the initial generative model according to the label mapping vector corresponding to the sample modal information and the prediction mapping vector.

[0070] In one embodiment of the present application, in order to quantitatively evaluate the performance of the initial generative model, the second loss value of the initial generative model can be determined according to the label mapping vector and the prediction mapping vector corresponding to the sample modal information. The performance of the initial generative model is evaluated based on the second loss value, thereby providing data support for the subsequent update of the initial generative model. Specifically, the method for determining the second loss value satisfies the following relationship:

[0071] ;

[0072] Wherein, Loss2 represents the second loss value of the initial generative model; j represents the index of the dimension in the label mapping vector; m represents the number of dimensions in the label mapping vector; C j represents the mapping value corresponding to the jth dimension in the label mapping vector; D j Represents the mapping value corresponding to the j-th category in the predicted mapping vector.

[0073] In one embodiment of the present application, a higher second loss value indicates that a higher degree of difference between the prediction mapping vector output by the initial generative model and the label mapping vector is present, and a lower accuracy of the output result of the initial generative model is present.

[0074] S43, updating the initial generative model based on a back propagation algorithm until the second loss value meets a preset condition, stopping updating the initial generative model, and obtaining the generative model.

[0075] In one embodiment of the present application, the initial generative model can be continuously updated based on the back propagation algorithm until the second loss value meets the preset condition, and the initial generative model is stopped from being updated to obtain a second model for predicting the correlation between energy data and archiving requirements. The preset condition may be that the second loss value is less than or equal to a preset termination threshold. When the second loss value is less than or equal to the preset termination threshold, it indicates that the difference between the prediction mapping vector output by the initial generation model and the label mapping vector is small, and the accuracy of the output result of the initial generation model is high. Exemplarily, the termination threshold may be 0.1.

[0076] See also Figure 5 , Figure 5 1 is a functional module diagram of a data aggregation system based on energy big data provided in one embodiment of the present application. The data aggregation system based on energy big data 500 includes an electronic device 100 and a server 200. The module / unit referred to in the present application refers to a series of computer-readable instruction segments that can be executed by the processor 13 and can complete fixed functions, which are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0077] The electronic device 100 is used to obtain the energy data to be archived from the server 200, and to obtain the modal information corresponding to the energy data.

[0078] The electronic device 100 is used to classify the energy data according to the modal information to obtain a category of the energy data; wherein the category is used to characterize the position of the energy data in the energy industry.

[0079] The electronic device 100 is used to determine the archiving priority of the energy data according to the pre-stored archiving requirements and the category, and the archiving priority is used to indicate the importance of the energy data in the energy industry.

[0080] The electronic device 100 is used to determine a mapping vector between the energy data and the archiving requirement according to the category, the priority and the modal information; wherein the mapping vector is used to indicate an association relationship between the energy data and the archiving requirement.

[0081] The electronic device 100 is used to archive the energy data according to the mapping vector.

[0082] See also Figure 6, is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 100 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 is used to execute the computer-readable instructions stored in the memory to implement a data aggregation method based on energy big data described in any of the above embodiments.

[0083] In one embodiment of the present application, the electronic device 100 further includes a bus, a computer program stored in the memory 12 and executable on the processor 13, such as a data aggregation program based on energy big data.

[0084] Figure 6 Only the electronic device 100 having the memory 12 and the processor 13 is shown, and those skilled in the art can understand that Figure 6 The structure shown does not constitute a limitation on the electronic device 100 , and the electronic device 100 may include fewer or more components than shown in the figure, or combine some components, or arrange the components differently.

[0085] Combination Figure 2 The memory 12 in the electronic device 100 stores a plurality of computer-readable instructions to implement the data aggregation method based on energy big data, and the processor 13 can execute the plurality of instructions to implement: obtaining energy data to be archived, and obtaining modal information corresponding to the energy data; classifying the energy data according to the modal information to obtain the category of the energy data; wherein the category is used to characterize the position of the energy data in the energy industry; determining the archiving priority of the energy data according to the pre-stored archiving requirements and the category, and the archiving priority is used to indicate the importance of the energy data in the energy industry; determining a mapping vector between the energy data and the archiving requirements according to the category, the priority and the modal information; wherein the mapping vector is used to indicate the association relationship between the energy data and the archiving requirements; and archiving the energy data according to the mapping vector.

[0086] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0087] Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 100 and does not constitute a limitation on the electronic device 100. The electronic device 100 may be a bus-type structure or a star-type structure. The electronic device 100 may also include more or less other hardware or software than shown in the diagram, or a different arrangement of components. For example, the electronic device 100 may also include input and output devices, network access devices, etc.

[0088] It should be noted that the electronic device 100 is only an example, and other existing or future electronic products that are suitable for the present application should also be included in the protection scope of the present application and included here by reference.

[0089] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium can be non-volatile or volatile. The readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 100, such as a mobile hard disk of the electronic device 100. In other embodiments, the memory 12 can also be an external storage device of the electronic device 100, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device 100. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 100, such as a code of a data aggregation program based on energy big data, but also can be used to temporarily store data that has been output or is to be output.

[0090] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 100, and uses various interfaces and lines to connect various components of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing programs or modules stored in the memory 12 (for example, executing data aggregation programs based on energy big data, etc.), and calling data stored in the memory 12.

[0091] The processor 13 executes the operating system of the electronic device 100 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned data aggregation method embodiments based on energy big data, for example Figure 2 Steps shown.

[0092] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 100.

[0093] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute a part of a data aggregation method based on energy big data described in various embodiments of the present application.

[0094] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware devices to complete through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0095] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory and other memory, etc.

[0096] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0097] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. Figure 6Only one arrow is used in the figure, but it does not mean that there is only one bus or one type of bus. The bus is configured to realize the connection and communication between the memory 12 and at least one processor 13, etc.

[0098] An embodiment of the present application also provides a computer-readable storage medium (not shown), in which computer-readable storage medium is stored computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the data aggregation method based on energy big data described in any of the above embodiments.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0100] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0102] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the specification can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A data aggregation method based on energy big data, applied to electronic equipment, characterized in that: The method comprises: Acquiring energy data to be archived, and acquiring modal information corresponding to the energy data; Classifying the energy data according to the modal information to obtain a category of the energy data; wherein the category is used to characterize the position of the energy data in the energy industry; Determining an archiving priority of the energy data according to a pre-stored archiving requirement and the category, wherein the archiving priority is used to indicate the importance of the energy data in the energy industry; Determine a mapping vector between the energy data and the archiving requirement according to the category, the priority and the modal information; wherein the mapping vector is used to indicate an association relationship between the energy data and the archiving requirement; The energy data is archived according to the mapping vector.

2. The data aggregation method based on energy big data according to claim 1, characterized in that: The energy data is classified according to the modal information to obtain the category of the energy data, including: Encoding the energy data to obtain a first code; Encoding the modal information to obtain a second code; Determining a coding vector corresponding to the energy data according to the first coding and the second coding; The encoding vector is input into a pre-trained first model to obtain a category vector of the energy data; wherein the category vector is used to indicate the category of the energy data.

3. The data aggregation method based on energy big data according to claim 2 is characterized in that: The method further includes training the first model, wherein the training the first model includes: Encoding the pre-acquired sample energy data and the corresponding sample modal information to obtain a sample code; Determining the confidence of the sample encoding according to the sample energy data and the sample modal information; Based on the sample code, according to a pre-built initial classification model, determining a corresponding prediction category vector; Determining a first loss value of the initial classification model according to the label category vector corresponding to the sample encoding, the predicted category vector, and the confidence; The initial classification model is continuously updated based on a back propagation algorithm until the first loss value meets a preset condition, and then the updating of the initial classification model is stopped to obtain the first model.

4. The data aggregation method based on energy big data according to claim 3 is characterized in that: Determining the confidence of the sample encoding according to the sample energy data and the sample modal information includes: Performing semantic analysis on the energy data to obtain semantic information of the energy data; Determining a degree of match between the semantic information and the sample modality information; The confidence level of the sample encoding is determined according to the matching level.

5. The data aggregation method based on energy big data according to claim 3 is characterized in that: The determining the first loss value of the initial classification model according to the label category vector corresponding to the sample encoding, the predicted category vector, and the confidence comprises: ； Wherein, Loss1 represents the first loss value of the initial classification model; S represents the confidence; i represents the index of the category; n represents the number of categories; A i represents the probability value corresponding to the i-th category in the label category vector; B i Represents the probability value corresponding to the i-th category in the predicted category vector.

6. The data aggregation method based on energy big data according to claim 1, characterized in that: The determining, according to the category, the priority and the modal information, a mapping vector between the energy data and the archiving requirement comprises: Generate prompt data based on the category, the modality information, and a pre-stored prompt template; Inputting the prompt data into a pre-trained generative model to obtain an initial mapping vector between the energy data and the archiving requirements; Based on the priority and the initial mapping vector, a mapping vector between the energy data and the archiving requirement is determined.

7. The data aggregation method based on energy big data according to claim 6, characterized in that: The method further includes training the generative model, wherein the training the generative model includes: Generate sample prompt data based on pre-acquired label categories, sample modality information, and pre-stored prompt templates; Inputting the sample prompt data into a pre-built initial generative model to obtain a prediction mapping vector between the energy data and the archiving demand; Determining a second loss value of the initial generative model according to the label mapping vector corresponding to the sample modality information and the prediction mapping vector; The initial generative model is updated based on a back-propagation algorithm until the second loss value meets a preset condition, and then the updating of the initial generative model is stopped to obtain the generative model.

8. The data aggregation method based on energy big data according to claim 7, characterized in that: The determining, according to the label mapping vector and the prediction mapping vector corresponding to the sample modality information, a second loss value of the initial generative model comprises: ； Wherein, Loss2 represents the second loss value of the initial generative model; j represents the index of the dimension in the label mapping vector; m represents the number of dimensions in the label mapping vector; C j represents the mapping value corresponding to the jth dimension in the label mapping vector; D j Represents the mapping value corresponding to the j-th category in the predicted mapping vector.

9. A data aggregation system based on energy big data, characterized in that: The system includes an electronic device and a server; The electronic device is used to obtain energy data to be archived from the server, and to obtain modal information corresponding to the energy data; The electronic device is used to classify the energy data according to the modal information to obtain a category of the energy data; wherein the category is used to characterize the position of the energy data in the energy industry; The electronic device is used to determine the archiving priority of the energy data according to the pre-stored archiving requirements and the category, wherein the archiving priority is used to indicate the importance of the energy data in the energy industry; The electronic device is used to determine a mapping vector between the energy data and the archiving requirement according to the category, the priority and the modality information; wherein the mapping vector is used to indicate an association relationship between the energy data and the archiving requirement; The electronic device is used to archive the energy data according to the mapping vector.

10. The data aggregation system based on energy big data according to claim 9, characterized in that: The electronic device is also used to encode the energy data to obtain a first code; The electronic device is used to encode the modal information to obtain a second code; The electronic device is used to determine the encoding vector corresponding to the energy data according to the first encoding and the second encoding; The electronic device is used to input the encoding vector into a pre-trained first model to obtain a category vector of the energy data; wherein the category vector is used to indicate the category of the energy data.

Images