Metadata automatic extraction method oriented to energy field

Abstract

The invention discloses an automatic metadata extraction method and device for the energy field, equipment and a storage medium, and belongs to the field of energy data management, and the method comprises the steps that first metadata and analysis data of energy data are determined; inputting the first metadata into a first language model to obtain a plurality of cue words of the energy data; and inputting the plurality of prompt words and the analysis data of the energy data into a second language model to obtain second metadata of the energy data, so that a data structure of the second metadata corresponds to an energy type corresponding to the energy data. According to the method and the device, the diversity and the complexity of the energy data are adapted, the metadata extraction requirements of different energy types are met, and the adaptability and the flexibility of metadata extraction are improved. The problems of low metadata extraction efficiency and low adaptability of the extracted metadata in the energy field are solved.

Description

Technical Field

[0001] The present application relates to the field of energy data management, and specifically to an automatic metadata extraction method, device, equipment and storage medium for the energy field. Background Art

[0002] In the process of building an energy digital network in the energy field, the automatic extraction of metadata becomes the key to achieving efficient data management and utilization. In the case of the coexistence of various data sources and standards, the characteristics of energy data such as distribution, heterogeneity and cross-subject complexity are particularly prominent. For example, in different energy fields (such as solar energy and hydrogen energy), there are significant differences in the data content of metadata.

[0003] At present, metadata extraction is mainly divided into two categories: one is to obtain metadata manually, and the other is to automatically generate metadata with the help of technical means, which usually relies on metadata extraction tools.

[0004] However, manual extraction methods have obvious inefficiencies, and metadata generated with the help of existing automated tools can only extract limited types of metadata information, making it difficult to meet users' needs for metadata extraction in specific fields. Summary of the invention

[0005] The present application aims to provide a method, device, equipment and storage medium for automatic metadata extraction in the energy field, which at least solves the problems of low efficiency of metadata extraction in the energy field and low adaptability of the extracted metadata.

[0006] In a first aspect, the present application discloses an automatic metadata extraction method for the energy field, including: Determine first metadata of energy data, and determine parsed data of the energy data; the first metadata has a preset data structure; Inputting the first metadata into a first language model to obtain a plurality of prompt words for the energy data; the generated plurality of prompt words correspond to energy types corresponding to the energy data; The plurality of prompt words and parsed data of the energy data are input into a second language model to obtain second metadata of the energy data, so that a data structure of the second metadata corresponds to an energy type corresponding to the energy data.

[0007] In a second aspect, the embodiment of the present application further discloses an automatic metadata extraction device for the energy field, comprising: A data extraction module, used to determine first metadata of energy data and determine parsed data of the energy data; the first metadata has a preset data structure; a prompt word generation module, configured to input the first metadata into a first language model to obtain a plurality of prompt words for the energy data; the generated plurality of prompt words correspond to energy types corresponding to the energy data; The metadata generation module is used to input the multiple prompt words and parsed data of the energy data into a second language model to obtain second metadata of the energy data, so that the data structure of the second metadata corresponds to the energy type corresponding to the energy data.

[0008] In a third aspect, an embodiment of the present application further discloses an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application further discloses a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In summary, in the embodiment of the present application, the prompt words generated by the first language model can accurately reflect the specific types and characteristics of energy data. Through these prompt words, it is convenient to accurately identify and understand the context and field characteristics of energy data, and improve the adaptability of metadata extraction; and then the second language model is guided by the prompt words to extract more detailed and professional metadata information from the parsed data, so that the extracted second metadata not only has high accuracy, but also can cover the professional metadata requirements of various subdivided scenes in the energy field. Therefore, based on the method of the embodiment of the present application, the inefficiency and high cost of manual extraction are avoided, and the efficiency of metadata extraction is greatly improved. At the same time, the use of the language model ensures that the extracted metadata has high accuracy and consistency. While adapting to the diversity and complexity of energy data, it meets the metadata extraction requirements of different energy types, and improves the adaptability and flexibility of metadata extraction. Solve the problems of low efficiency of metadata extraction in the energy field and low adaptability of extracted metadata. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In the attached picture: Figure 1 This is a flowchart of the steps of an automatic metadata extraction method for the energy field provided by an embodiment of the present application; Figure 2 It is a flowchart of the steps of another method for automatic metadata extraction for the energy field provided by an embodiment of the present application; Figure 3 This is the initial construction process of the second language model in the embodiment of the present application; Figure 4It is a program logic diagram provided according to an embodiment of the present application; Figure 5 It is a block diagram of an automatic metadata extraction device for the energy field provided by an embodiment of the present application; Figure 6 is a block diagram of an electronic device according to an embodiment of the present application; Figure 7 It is a block diagram of an electronic device of another embodiment provided by the embodiments of the present application. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0013] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0014] Figure 1 This is an automatic metadata extraction method for the energy field provided in an embodiment of the present application.

[0015] The method may include the following steps: Step 101: determine first metadata of energy data, and determine parsed data of the energy data.

[0016] The first metadata has a preset data structure.

[0017] In some embodiments of the present application, in order to ensure that the energy data has a structured and standardized format in the subsequent processing process, the first metadata of the energy data will be determined, and the parsed data of the energy data will be determined, wherein the first metadata has a preset data structure. The preset data structure is usually some metadata that has been formulated under the prior art and does not match the specific needs. In this way, it will provide effective data support for the input of the model in the subsequent steps.

[0018] In a specific example, a user may need to process data from the coal industry. The user first collects and organizes the data, including raw data and related metadata information. This process provides a reliable foundation for subsequent metadata extraction.

[0019] Step 102: input the first metadata into a first language model to obtain a plurality of prompt words of energy data.

[0020] The generated multiple prompt words correspond to the energy types corresponding to the energy data.

[0021] In some embodiments of the present application, in order to utilize the language understanding ability of the first language model to generate prompt words that conform to a specific type of energy data, the first metadata is input into the first language model, and multiple prompt words are generated through the processing of the model, and these prompt words match the energy type corresponding to the energy data, such as the role definition and task description of the energy data, and the requirements for the metadata field described by the user, or input and output samples, etc. The prompt words are important information to guide the model to extract metadata, helping the model to better understand and parse the content of the energy data. This can lay the foundation for the accurate metadata extraction of the second language model.

[0022] In a specific example, data from the solar industry will be processed. After converting this data into structured first metadata, it is input into the first language model. The model processes the input metadata and generates multiple prompt words, such as "solar radiation intensity", "photovoltaic cell efficiency", "installation angle", etc. These prompt words correspond to the specific types and characteristics of solar data. Through this process, users can obtain a set of prompt words that accurately reflect the characteristics of solar data, providing a reliable basis for subsequent metadata extraction.

[0023] Step 103 : inputting the plurality of prompt words and the parsed data of the energy data into a second language model to obtain second metadata of the energy data, so that the data structure of the second metadata corresponds to the energy type corresponding to the energy data.

[0024] In some embodiments of the present application, in order to utilize the powerful processing capability of the second language model to extract more detailed and professional metadata information from energy data, the generated multiple prompt words and parsed data are input into the second language model together, and the second language model parses and extracts the second metadata of the energy data based on these input information. These prompt words guide the model to understand the context and domain characteristics of the energy data. In this way, more detailed and professional second metadata corresponding to the energy data can be obtained, thereby improving the accuracy and adaptability of the data.

[0025] In a specific example, a user processes data in the hydrogen energy industry. The user inputs the structured first metadata and the generated prompt words into the second language model. The model uses the prompt words to guide the parsing of the data and extracts second metadata from the hydrogen energy data, such as "hydrogen purity", "storage pressure", "transportation temperature", etc. These extracted second metadata correspond to the specific types and characteristics of hydrogen energy, ensuring the accuracy and professionalism of the data. Through this process, users can obtain high-quality metadata information for further data management and utilization.

[0026] In summary, in the embodiment of the present application, the prompt words generated by the first language model can accurately reflect the specific types and characteristics of energy data. Through these prompt words, it is convenient to accurately identify and understand the context and field characteristics of energy data, and improve the adaptability of metadata extraction; and then the second language model is guided by the prompt words to extract more detailed and professional metadata information from the parsed data, so that the extracted second metadata not only has high accuracy, but also can cover the professional metadata requirements of various subdivided scenes in the energy field. Therefore, based on the method of the embodiment of the present application, the inefficiency and high cost of manual extraction are avoided, and the efficiency of metadata extraction is greatly improved. At the same time, the use of the language model ensures that the extracted metadata has high accuracy and consistency. While adapting to the diversity and complexity of energy data, it meets the metadata extraction requirements of different energy types, and improves the adaptability and flexibility of metadata extraction. Solve the problems of low efficiency of metadata extraction in the energy field and low adaptability of extracted metadata.

[0027] Figure 2 This is another method for automatically extracting metadata for the energy field provided in an embodiment of the present application.

[0028] The method may include the following steps: Step 201: determine first metadata of energy data, and determine parsed data of the energy data.

[0029] The first metadata has a preset data structure.

[0030] The method shown in this step has been explained in step 101 and will not be repeated here.

[0031] Optionally, in order to determine the first metadata of the energy data, step 201 includes the following sub-steps: Sub-step 2011: extracting a first description text of the energy data from the energy data.

[0032] In some embodiments of the present application, in order to ensure that the subsequent metadata extraction process can be based on detailed and accurate description texts, the first description text related to the energy data is extracted. These description texts contain key information and features of the energy data, which is convenient for subsequent model processing. The description text generated by this step can provide basic information support for subsequent steps, ensuring the comprehensiveness and accuracy of metadata extraction.

[0033] In a specific example, the user needs to process a batch of data from the coal industry. The user extracts relevant descriptive texts from these data, such as coal type name, calorific value, ash content, etc. These descriptive texts contain the main characteristics and important information of the coal data. Through this process, the user obtains a set of detailed descriptive texts, which provides the necessary basic data for subsequent metadata extraction.

[0034] Sub-step 2012, when the amount of information in the first description text is less than or equal to a preset information amount threshold, the first description text is enhanced to update the first description text.

[0035] In some embodiments of the present application, in order to ensure that the description text has sufficient information so that it can provide sufficient data support for metadata extraction, when it is detected that the information amount of the first description text is less than or equal to a preset information amount threshold, the description text will be enhanced. Information enhancement includes adding or supplementing more relevant data information to make the description text more detailed and comprehensive. The information amount threshold refers to the standard used to measure whether the description text contains sufficient information. The clear technical effect brought about by executing this step is that the updated first description text contains more information, thereby improving the accuracy and completeness of metadata extraction in subsequent steps.

[0036] In a specific example, the user found that the description text extracted from the wind energy data was insufficient in information and might only contain two data points: "wind speed" and "wind direction". The user enhanced the description text and added more relevant information such as "wind force level" and "wind energy conversion efficiency". Through this process, the user was able to obtain an updated description text with more information, providing a more comprehensive data foundation for subsequent metadata extraction. Ultimately, the updated description text ensured the accuracy and completeness of the metadata extraction process.

[0037] Sub-step 2013, when the information amount of the first description text is greater than the information amount threshold, keyword extraction is performed on the description text to obtain multiple keywords in the description text.

[0038] In some embodiments of the present application, in order to effectively extract key information from a large amount of information and facilitate subsequent data processing and analysis, when it is detected that the amount of information in the first description text is greater than a preset information amount threshold, keyword extraction is performed on the description text. Keyword extraction refers to identifying and extracting key words that best represent the content of the text from the text. The information amount threshold is a standard for measuring whether the description text contains too much information. The keywords generated in this way can effectively summarize the main content of the description text, which is convenient for subsequent metadata encapsulation.

[0039] In a specific example, the user processed a batch of solar energy data description texts and found that these texts contained a large amount of information, such as "solar radiation intensity", "photovoltaic panel efficiency", "installation angle", "weather conditions", etc. Since the amount of information was greater than the preset threshold, the user extracted keywords from the description texts and identified and extracted keywords such as "radiation intensity", "panel efficiency", and "installation angle". Through this process, the user can obtain a set of keywords that effectively summarize the main content of the description text, providing a concise and accurate information basis for subsequent metadata packaging.

[0040] Sub-step 2014: encapsulate the obtained multiple keywords according to the data structure to obtain first metadata.

[0041] In some embodiments of the present application, in order to structure the extracted key information and facilitate subsequent data processing and analysis, the multiple keywords obtained previously are encapsulated according to a predefined data structure to obtain the first metadata. The data structure of metadata, that is, the metadata schema, refers to a way of organizing and storing data to ensure that the data can be processed and used consistently and accurately. In this way, the first metadata that meets the preset standards will be generated, thereby improving the consistency and integrity of the data.

[0042] In a specific example, a user processes a batch of wind energy data and has previously extracted multiple keywords from the description text, such as "wind speed", "wind direction", "power generation", etc. The user encapsulates these keywords according to a defined data structure, such as a table or JSON format, to generate the first metadata. Through this process, the user can obtain a set of structured first metadata, which provides a reliable data foundation for subsequent metadata extraction and analysis.

[0043] Optionally, in order to determine the parsed data of the energy data, step 201 includes the following sub-steps: Sub-step 2015 , when the energy data is structured data, determining the text data extracted from the energy data as parsed data.

[0044] In some embodiments of the present application, in order to directly use the text data in the structured data as parsed data and simplify the processing flow, when it is confirmed that the energy data is structured data, the text data extracted therefrom will be determined as parsed data. Structured data refers to data organized and stored according to a predefined model and format. In this way, the existing text data can be directly used as parsed data, which reduces additional data processing work and improves processing efficiency and accuracy.

[0045] In a specific example, the user needs to process a set of structured data in the power industry. These data include pre-defined fields such as power station name, installed capacity, and power generation. The user extracts the corresponding text data from these structured data, such as "Power station name: XX power station", "Installed capacity: 500MW", "Power generation: 300GWh", etc., and directly identifies them as parsed data. Through this process, users can simplify the data processing steps and quickly obtain the required parsed data.

[0046] Sub-step 2016, when the energy data includes unstructured data, the unstructured data is parsed to obtain a second description text of the energy data, and the second description text is packaged with the unstructured data in the energy data to serve as parsed data.

[0047] In some embodiments of the present application, in order to ensure that unstructured data can be effectively parsed and utilized, thereby improving the overall integrity and availability of the data, when the energy data contains unstructured data, data parsing will be performed on these unstructured data to obtain a second description text. Unstructured data refers to free-format data that is not organized according to a predefined model. Data parsing is the process of converting these free-format data into more structured and understandable text. Subsequently, the obtained second description text is encapsulated with the original unstructured data as parsed data. The clear technical effect brought about by executing this step is that the consistency and integrity of the data are enhanced, ensuring the effective use of unstructured data in metadata extraction.

[0048] In a specific example, the user processes a batch of unstructured data in the oil industry, including free-format daily operation records, maintenance reports, etc. The user parses these unstructured data and extracts key description texts, such as "maintenance time", "equipment status", "operator", etc. The extracted description text is packaged together with the original unstructured data to generate parsed data. Through this process, the user can convert unstructured data into structured information, which is convenient for subsequent metadata extraction and analysis.

[0049] Step 202: input the first metadata into a first language model to obtain a plurality of prompt words of energy data.

[0050] The generated multiple prompt words correspond to the energy types corresponding to the energy data.

[0051] The method shown in this step has been explained in step 102 and will not be repeated here.

[0052] Step 203 : inputting the plurality of prompt words and the parsed data of the energy data into a second language model to obtain second metadata of the energy data, so that the data structure of the second metadata corresponds to the energy type corresponding to the energy data.

[0053] The method shown in this step has been explained in step 103 and will not be repeated here.

[0054] Step 204: In response to the adjustment operation on the second metadata, the first language model and / or the second language model is adjusted to update the second metadata according to the adjusted first language model and / or the second language model.

[0055] In some embodiments of the present application, in order to ensure the accuracy and adaptability of the second metadata, it can be dynamically adjusted according to actual needs, and the first language model and / or the second language model will be adjusted in response to the user's adjustment operation on the second metadata. These adjustments can be optimized based on user feedback and needs to ensure that the metadata generated by the model better meets the actual needs of the user. By adjusting the model parameters and training data, the accuracy and flexibility of metadata extraction can be further improved. The clear technical effect brought about by executing this step is that the updated second metadata can better reflect the specific characteristics and user needs of the energy data, and improve the applicability and accuracy of the data.

[0056] In a specific example, the user may find that the extracted metadata information of solar energy data is not completely accurate or does not meet expectations, for example, the information of "photovoltaic panel type" is not detailed enough. The user adjusts the second metadata and feeds back the adjustment information to the system. Based on the user's feedback, the system optimizes and adjusts the parameters and training data of the first language model and / or the second language model. Through this adjustment process, the second metadata regenerated by the model will be more detailed and accurate, such as adding detailed classification information of "photovoltaic panel type". Ultimately, the updated metadata better meets the needs of users and improves the practicality and credibility of the data.

[0057] Optionally, step 204 includes the following sub-steps: Sub-step 2041 , in response to the adjustment operation on the second metadata, obtaining the adjustment data of the second metadata.

[0058] In some embodiments of the present application, in order to obtain the user's feedback on the modification of the second metadata so as to further optimize the model, the adjustment operation performed by the user on the second metadata will be responded to, thereby generating relevant adjustment data. The adjustment data refers to the specific information reflecting the user's modification or optimization of the metadata. The adjustment data generated in this way can provide basic data for subsequent model adjustments, ensuring that the model can better adapt to user needs.

[0059] In a specific example, the user found that some metadata information in the extracted natural gas industry data was not detailed enough, such as the missing "gas purity" information. The user adjusted these metadata and added detailed "gas purity" information. In response to this adjustment operation, the system generated adjusted data containing the newly added "gas purity" information. Through this process, the user can ensure that all key information is included in the metadata, providing accurate feedback for subsequent model optimization. Ultimately, the generated adjusted data ensures that the model can better adapt to actual needs and improves the accuracy and completeness of the metadata.

[0060] Sub-step 2042: adjusting the first language model and / or the second language model according to the adjustment data of the second metadata, so as to update the second metadata according to the adjusted first language model and / or the second language model.

[0061] In some embodiments of the present application, in order to ensure that the model can be adaptively optimized and improved, so as to more accurately extract metadata information, at least one of the first language model and the second language model will be adjusted according to the adjustment data of the obtained second metadata. The adjustment process may involve retraining the model or modifying model parameters to reflect user feedback and needs. Through this adjustment, the model can better adapt to and parse energy data. In this way, more accurate and adaptable second metadata will be generated through the updated model, further improving the accuracy of metadata extraction and user satisfaction.

[0062] In a specific example, the user adjusted the extracted metadata information of the natural gas industry and added new data fields, such as "gas density" and "transmission pressure". Based on these adjusted data, the system adjusted at least one of the first language model and the second language model, such as retraining the model or adjusting parameter settings. Through this adjustment process, the metadata regenerated by the model will more accurately reflect the user's needs, including the new data fields.

[0063] Optionally, for the adjustment process of the first language model, sub-step 2042 includes the following sub-steps: Sub-step 20421, determining the adjustment data of the first metadata according to the adjustment data of the second metadata.

[0064] In some embodiments of the present application, in order to ensure that the optimization and training of the first language model are based on the latest user needs and data adjustments, the corresponding adjustment data of the first metadata will be determined based on the adjustment data of the second metadata. The adjustment data refers to the specific information after the user modifies or optimizes the metadata. In this way, it is ensured that the first language model can more accurately reflect the user's needs in subsequent training. The adjustment data of the first metadata generated in this way will provide a basis for the training and optimization of the subsequent model.

[0065] In a specific example, a user adjusts the second metadata extracted from the natural gas industry, such as adding new data fields "gas purity" and "pipeline pressure". Based on these adjustment information, the system determines the relevant first metadata adjustment data, including updated field descriptions and relationships. This ensures that the training data of the first language model contains the latest requirements and adjustment content.

[0066] Sub-step 20422: Use the adjusted data of the first metadata as first training data to train the first language model to obtain a trained first language model, and return to step 202.

[0067] In some embodiments of the present application, in order to optimize the first language model through continuous training so that it can better adapt to the adjusted needs of the user, the determined first metadata adjustment data will be used as the first training data to retrain the first language model, and then the process of step 202 will be re-executed, that is, the first metadata is input into the first language model to obtain multiple prompt words for energy data. Training refers to further learning and optimizing the model using new data, aiming to improve the performance and accuracy of the model. In this way, the first language model can be updated and better understand and process the metadata extraction task. In this way, a trained first language model will be obtained, which can generate prompt words more accurately, thereby improving the effect of metadata extraction in subsequent steps.

[0068] In a specific example, the user adjusted the data of the natural gas industry and added new metadata fields such as "gas composition" and "transmission conditions". The system used these adjusted metadata as training data to retrain the first language model. Through this process, the user can enable the first language model to better learn and understand the new data structure and field requirements. Ultimately, the trained first language model can generate more accurate and adaptable prompt words, which helps improve the performance of the second language model in the metadata extraction process, and further improves the accuracy and comprehensiveness of metadata extraction.

[0069] Optionally, for the adjustment process of the second language model, sub-step 2042 includes the following sub-steps: Sub-step 20423: updating the plurality of prompt words according to the adjustment data of the second metadata.

[0070] In some embodiments of the present application, in order to ensure that the prompt words can accurately reflect the latest user needs and data adjustments, multiple prompt words will be updated according to the adjustment data of the second metadata. Updating the prompt words means regenerating or modifying the prompt words according to the adjustment data to make them more adaptable to the new data content and extraction requirements. The prompt words are important information to guide the model to extract metadata, which can help the model understand and parse data. The prompt words updated in this way can provide more accurate information support for subsequent model training and metadata extraction.

[0071] In a specific example, the user adjusted the data of the natural gas industry and added new metadata fields such as "gas composition" and "transmission conditions". The system updated the corresponding prompt words based on these adjusted data and generated new prompt words such as "composition analysis", "transmission pressure", "storage conditions", etc. Through this process, users can ensure that the prompt words reflect the latest needs and data content, providing an accurate information basis for subsequent model training.

[0072] Sub-step 20424, using the updated prompt word as the second training data to train the second language model to obtain a trained second language model, and then returning to step 203.

[0073] In some embodiments of the present application, in order to continuously optimize the second language model so that it can better process and parse the latest data requirements, the updated prompt words are used as the second training data to retrain the second language model, and then the process of step 203 is re-executed, that is, multiple prompt words and parsed data of energy data are input into the second language model to obtain the second metadata of energy data. The training process refers to further learning and optimizing the model using the updated prompt word data to ensure that the model can more accurately reflect the latest data content and extraction requirements. Through this continuous training, the second language model can maintain efficient and accurate metadata extraction capabilities. The more accurate and efficient second language model obtained in this way improves the overall effect of metadata extraction.

[0074] In a specific example, the user adjusted the data of the natural gas industry and updated the prompt words, such as adding "gas composition analysis" and "pressure conditions". The system uses these updated prompt words as new training data to retrain the second language model. Through this process, the user ensures that the model can learn and adapt to the new prompt words, improving its understanding and parsing ability of the data. Ultimately, the trained second language model can extract professional metadata of the natural gas industry more accurately and efficiently, providing a more reliable information foundation for subsequent data application and management.

[0075] Step 205: store the second metadata together with the energy data.

[0076] In some embodiments of the present application, in order to ensure that all data can be centrally managed and effectively utilized, the extracted second metadata is stored in the system together with the original energy data. This storage method can ensure the integrity and traceability of the data. By storing the two types of data together, it is convenient for users to conduct comprehensive queries and analysis of the data. The clear technical effect brought about by executing this step is that the consistency and integrity of the data are enhanced and the efficiency of data management is improved.

[0077] In a specific example, users have extracted detailed metadata information from hydrogen industry data. Users store these metadata together with the corresponding hydrogen raw data in the data warehouse. Through this process, users can ensure that all hydrogen-related data is concentrated in one storage system, which is convenient for future data query, analysis and application. Ultimately, this common storage method not only ensures the integrity and consistency of the data, but also improves the efficiency of data management and utilization.

[0078] Optionally, step 205 includes the following sub-steps: Sub-step 2051: generating identification data of the energy data according to the second metadata, so that the identification data corresponds to the energy data one by one.

[0079] In some embodiments of the present application, in order to ensure that each energy data has a unique identifier for subsequent data management and tracing, corresponding identification data will be generated based on the second metadata to ensure that these identification data can correspond one-to-one with the energy data. Identification data refers to specific information that uniquely identifies each energy data. The clear technical effect brought about by executing this step is that identification data for uniquely identifying each energy data is generated, which improves the management and retrieval efficiency of data.

[0080] In a specific example, a user processes a batch of data from the power industry and extracts secondary metadata, such as power generation, installed capacity, etc. The system generates unique identification data based on these metadata, such as using UUID (Universally Unique Identifier) ​​or other identification methods. These identification data ensure that each power data record has a unique identifier. Through this process, users can easily manage and trace each data record.

[0081] Sub-step 2052: storing the second metadata in a metadata database, storing the energy data in an energy database, and establishing a mapping relationship between the metadata database and the energy database according to the identification data.

[0082] In some embodiments of the present application, in order to achieve separate storage and management of metadata and energy data, and to establish an association between the two through identification data, so as to facilitate efficient retrieval and management of data, the extracted second metadata will be stored in the metadata database, the corresponding energy data will be stored in the energy database, and a mapping relationship will be established between the two databases based on the generated identification data. The mapping relationship refers to the use of identification data to connect the metadata in the metadata database with the actual data in the energy database. This will enable efficient management of metadata and energy data, and enhance data traceability and query efficiency.

[0083] In a specific example, a user processes a batch of photovoltaic power station data. The extracted metadata such as "power station name" and "installed capacity" are stored in the metadata database, while the actual energy data such as "real-time power generation" and "equipment status" are stored in the energy database. The system uses unique identification data such as UUID to establish a mapping relationship between the metadata in the metadata database and the energy data in the energy database. Through this process, users can easily query and manage the data of photovoltaic power stations. Ultimately, the established mapping relationship ensures efficient retrieval and management of data, while improving the integrity and traceability of the data.

[0084] Optionally, in some embodiments of the present application, a fine-tuned language model is used as the second language model.

[0085] The fine-tuned language model can obtain more accurate language understanding and information extraction capabilities in a specific field through further training on data in that field. This enables the model to more accurately identify and parse relevant metadata information when processing energy data; at the same time, the fine-tuning process enables the model to adapt to the specific needs and characteristics of the energy field, thereby improving the adaptability and accuracy of metadata extraction. This targeted optimization can still maintain high extraction efficiency and accuracy when energy data is complex and diverse; in addition, the use of a fine-tuned language model can also reduce the difficulty of processing unstructured data while ensuring data processing performance. By fine-tuning the model, it can better handle terminology and data structures unique to the energy field, thereby improving the overall performance of data parsing and processing; and because the fine-tuned language model has been optimized on data sets in related fields, its performance in performing metadata extraction tasks is better than that of a general language model that has not been fine-tuned. This advantage ensures that the model can provide high-quality metadata extraction results in different energy types and complex data environments. Therefore, choosing to use a fine-tuned language model as the second language model not only improves the accuracy and efficiency of metadata extraction, but also enhances the model's adaptability to specific needs in the energy field, thereby ensuring the efficiency, accuracy and reliability of the metadata extraction process.

[0086] like Figure 3 As shown, it is a specific process of obtaining the initial fine-tuning model when the fine-tuning model is used as the second language model in the embodiment of the present application: R1: Data Acquisition. The process of data acquisition includes collecting relevant data from databases or data sources in various energy fields. The reason for performing data acquisition is to collect basic data for training the model. After executing this step, a large amount of initial data for fine-tuning the model can be obtained, providing basic data support for subsequent processing and training.

[0087] R2: Data Processing. The process of data processing includes removing noise data, handling missing values, and standardizing data formats. The reason for performing data processing is to clean and organize the collected data to ensure the quality and consistency of the data. Through the processing steps, clean and structured data can be obtained to ensure that the data quality meets the requirements of the training model.

[0088] R3: Data Annotation. The process of data annotation includes annotating the data manually or automatically, and marking the important features and attributes of the data. The reason for performing data annotation is to add labels to the data so that the model can learn how to extract meaningful meta-information from the data. The labeled data can be used as an effective sample for training the model to improve the learning effect and accuracy of the model.

[0089] R4: Data Augmentation. The process of data augmentation includes various transformation operations on existing data, such as data expansion, noise addition, and data augmentation techniques. The reason for performing data augmentation is to expand the diversity of the training data set and improve the generalization ability of the model. Through data augmentation, the types of training data can be enriched and the performance of the model in different scenarios can be improved.

[0090] R5: Model Fine-Tuning. The process of model fine-tuning includes retraining the pre-trained model using processed and annotated data. The reason for performing model fine-tuning is to further train the basic model with data from a specific field to improve its performance in that field. Through fine-tuning, the model can better adapt to the data characteristics of a specific field and improve the professionalism and accuracy of the model.

[0091] R6: Model Generation. The process of model generation includes saving the fine-tuned model and deploying it to the system. The reason for performing model generation is to complete the construction of the fine-tuned model so that it can be applied to the actual metadata extraction task. The generated fine-tuned model can be used as a second language model to play a role in metadata extraction of energy data.

[0092] like Figure 4 As shown, it is the program execution logic under the method disclosed in the embodiment of the present application, which specifically completes the whole process of storing energy data based on metadata by executing the following steps: S1 User uploads data: Users start uploading energy data that needs to be packaged into digital objects. These data can include various types of energy data such as coal, electricity, oil, and gas.

[0093] S2 data analysis: After the data is uploaded, the system parses the user data and converts it into a standard format that meets the model input requirements. This step ensures that the data can be processed and understood correctly.

[0094] in: S2.1 Metadata format upload: Users upload the metadata format (schema), describe the specific requirements for metadata extraction, and specify the metadata fields that need to be extracted.

[0095] S2.2 Calling the first language model to parse the metadata format: The system parses the metadata format uploaded by the user by calling the first language model. The parsed format generates standard prompt words, including role definition, task description, metadata field requirements described by the user, input and output samples, etc.

[0096] S3 Data Parsing: The system performs detailed parsing of the uploaded data and converts the data into a structured format for subsequent processing.

[0097] S4 Prompt word generation: Generate a set of prompt words based on the parsed data and metadata format. These prompt words are used to guide the second language model to extract metadata.

[0098] S5 calls the second language model to generate metadata: The system passes the prompt words and parsed data to the second language model, and the large model parses the data according to the prompt words to generate structured metadata. These metadata meet user needs and industry standards.

[0099] S6 User Review: Users review the generated metadata to ensure its accuracy and completeness. If any problems are found, users can make adjustments.

[0100] S7 User Adjustment: Users adjust metadata based on actual needs to ensure that it meets expectations. The adjustment information will be fed back to the system for model optimization.

[0101] S8 Digital Object Packaging: Finally, the adjusted and reviewed metadata enters the digital object packaging process to ensure the uniqueness and traceability of the data.

[0102] in: S8.1 The identity resolution system generates a unique identifier: The system calls the identity resolution system to generate a unique identifier for each digital object to ensure the uniqueness of the digital object.

[0103] S8.2 The metadata associated with the identifier is stored in the registry: After the generated metadata is associated with the unique identifier, it is stored in the system's built-in metadata registry in a standardized format, supporting efficient indexing and query functions.

[0104] S8.3 Identify and associate entities and store them in the warehouse: The system associates the corresponding data entity with the unique identifier and stores it in the system's built-in data warehouse to achieve complete encapsulation of energy digital objects and support long-term storage and management.

[0105] In summary, in the embodiment of the present application, the prompt words generated by the first language model can accurately reflect the specific types and characteristics of energy data. Through these prompt words, it is convenient to accurately identify and understand the context and field characteristics of energy data, and improve the adaptability of metadata extraction; and then the second language model is guided by the prompt words to extract more detailed and professional metadata information from the parsed data, so that the extracted second metadata not only has high accuracy, but also can cover the professional metadata requirements of various subdivided scenes in the energy field. Therefore, based on the method of the embodiment of the present application, the inefficiency and high cost of manual extraction are avoided, and the efficiency of metadata extraction is greatly improved. At the same time, the use of the language model ensures that the extracted metadata has high accuracy and consistency. While adapting to the diversity and complexity of energy data, it meets the metadata extraction requirements of different energy types, and improves the adaptability and flexibility of metadata extraction. Solve the problems of low efficiency of metadata extraction in the energy field and low adaptability of extracted metadata.

[0106] refer to Figure 5 , which shows an automatic metadata extraction device 30 for the energy field provided by an embodiment of the present application, including: The data extraction module 301 is used to determine the first metadata of the energy data and determine the parsed data of the energy data; the first metadata has a preset data structure; A prompt word generation module 302 is used to input the first metadata into a first language model to obtain a plurality of prompt words for energy data; the generated plurality of prompt words correspond to energy types corresponding to the energy data; The metadata generation module 303 is used to input multiple prompt words and parsed data of the energy data into a second language model to obtain second metadata of the energy data, so that the data structure of the second metadata corresponds to the energy type corresponding to the energy data.

[0107] Optionally, the data extraction module 301 includes: A description extraction submodule, used for extracting a first description text of the energy data from the energy data; An information enhancement submodule, configured to enhance the information of the first description text to update the first description text when the information amount of the first description text is less than or equal to a preset information amount threshold; A keyword submodule, used for extracting keywords from the description text to obtain a plurality of keywords in the description text when the information amount of the first description text is greater than the information amount threshold; The first metadata submodule is used to encapsulate the obtained multiple keywords according to the data structure to obtain first metadata.

[0108] Optionally, the data extraction module 301 includes: A first parsing submodule, for determining text data extracted from the energy data as parsed data when the energy data is structured data; The second parsing submodule is used to parse the non-structured data to obtain a second description text of the energy data when the energy data contains non-structured data, and to encapsulate the second description text with the non-structured data in the energy data as parsed data.

[0109] Optionally, the device 30 further includes: The model training module is used to adjust the first language model and / or the second language model in response to the adjustment operation on the second metadata, so as to update the second metadata according to the adjusted first language model and / or the second language model.

[0110] Optionally, the model training module includes: A second metadata adjustment submodule, configured to obtain adjustment data of the second metadata in response to an adjustment operation on the second metadata; The model training submodule is used to adjust the first language model and / or the second language model according to the adjustment data of the second metadata, so as to update the second metadata according to the adjusted first language model and / or the second language model.

[0111] Optionally, the model training submodule includes: A first adjustment unit, configured to determine adjustment data of the first metadata according to adjustment data of the second metadata; The first training unit is used to train the first language model using the adjustment data of the first metadata as the first training data to obtain the trained first language model, and return to the step of inputting the first metadata into the first language model to obtain multiple prompt words of the energy data.

[0112] Optionally, the model training submodule includes: A second adjustment unit, configured to update the plurality of prompt words according to the adjustment data of the second metadata; The second training unit is used to train the second language model using the updated prompt words as second training data to obtain a trained second language model, and return to the step of inputting multiple prompt words and parsed data of energy data into the second language model to obtain second metadata of the energy data, so that the data structure of the second metadata corresponds to the energy type corresponding to the energy data.

[0113] Optionally, the device 30 further includes: The data storage module is used to store the second metadata together with the energy data.

[0114] Optionally, the data storage module includes: an identification generating submodule, configured to generate identification data of the energy data according to the second metadata, so that the identification data corresponds to the energy data one by one; The data storage submodule is used to store the second metadata in the metadata database, store the energy data in the energy database, and establish a mapping relationship between the metadata database and the energy database according to the identification data.

[0115] In summary, in the embodiment of the present application, the prompt words generated by the first language model can accurately reflect the specific types and characteristics of energy data. Through these prompt words, it is convenient to accurately identify and understand the context and field characteristics of energy data, and improve the adaptability of metadata extraction; and then the second language model is guided by the prompt words to extract more detailed and professional metadata information from the parsed data, so that the extracted second metadata not only has high accuracy, but also can cover the professional metadata requirements of various subdivided scenes in the energy field. Therefore, based on the method of the embodiment of the present application, the inefficiency and high cost of manual extraction are avoided, and the efficiency of metadata extraction is greatly improved. At the same time, the use of the language model ensures that the extracted metadata has high accuracy and consistency. While adapting to the diversity and complexity of energy data, it meets the metadata extraction requirements of different energy types, and improves the adaptability and flexibility of metadata extraction. Solve the problems of low efficiency of metadata extraction in the energy field and low adaptability of extracted metadata.

[0116] Reference Figure 6 , the electronic device 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output (I / O) interface 512 , a sensor component 514 , and a communication component 516 .

[0117] The processing component 502 generally controls the overall operation of the electronic device 500, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.

[0118] The memory 504 is used to store various types of data to support the operation of the electronic device 500. Examples of such data include instructions for any application or method operating on the electronic device 500, contact data, phone book data, messages, pictures, multimedia, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0119] The power supply component 506 provides power to the various components of the electronic device 500. The power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 500.

[0120] The multimedia component 508 includes an interface that provides an output interface between the electronic device 500 and the user. In some embodiments, the interface may include a liquid crystal display (LCD) and a touch panel (TP). If the interface includes a touch panel, the interface may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the electronic device 500 is in an operating mode, such as a shooting mode or a multimedia mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0121] The audio component 510 is used to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), and when the electronic device 500 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is used to receive an external audio signal. The received audio signal can be further stored in the memory 504 or sent via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0122] The input / output I / O interface 512 provides an interface between the processing component 502 and the peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0123] The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the electronic device 500. For example, the sensor assembly 514 can detect the open / closed state of the electronic device 500, the relative positioning of components, such as the display and keypad of the electronic device 500, and the sensor assembly 514 can also detect the position change of the electronic device 500 or a component of the electronic device 500, the presence or absence of user contact with the electronic device 500, the orientation or acceleration / deceleration of the electronic device 500, and the temperature change of the electronic device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0124] The communication component 516 is used to facilitate wired or wireless communication between the electronic device 500 and other devices. The electronic device 500 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0125] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement the methods provided in the embodiments of the present application.

[0126] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, and the instructions can be executed by a processor 520 of an electronic device 500 to perform the above method. For example, the non-transitory storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0127] Figure 7FIG. 6 is a block diagram of an electronic device 600 according to another embodiment of the present invention. For example, the electronic device 600 may be provided as a server.

[0128] Reference Figure 7 , the electronic device 600 includes a processing component 622, which further includes one or more processors, and a memory resource represented by a memory 632, for storing instructions that can be executed by the processing component 622, such as an application. The application stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 622 is configured to execute instructions to perform the method provided in the embodiment of the present application.

[0129] The electronic device 600 may also include a power supply component 626 configured to perform power management of the electronic device 600, a wired or wireless network interface 650 configured to connect the electronic device 600 to a network, and an input / output (I / O) interface 658. The electronic device 600 may operate based on an operating system stored in the memory 632, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0130] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0131] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for automatic metadata extraction in the energy field, characterized in that: include: Determining first metadata of energy data, and determining parsed data of the energy data; The first metadata has a preset data structure; Inputting the first metadata into a first language model to obtain a plurality of prompt words for the energy data; The generated plurality of prompt words correspond to energy types corresponding to the energy data; The plurality of prompt words and parsed data of the energy data are input into a second language model to obtain second metadata of the energy data, so that a data structure of the second metadata corresponds to an energy type corresponding to the energy data.

2. The method according to claim 1, characterized in that The first metadata of energy data is determined, including: extracting a first description text of the energy data from the energy data; When the amount of information of the first description text is less than or equal to a preset information amount threshold, performing information enhancement on the first description text to update the first description text; When the information amount of the first description text is greater than the information amount threshold, performing keyword extraction on the description text to obtain a plurality of keywords in the description text; The obtained multiple keywords are packaged according to the data structure to obtain the first metadata.

3. The method according to claim 1, characterized in that The method further comprises: In response to the adjustment operation on the second metadata, the first language model and / or the second language model are adjusted to update the second metadata according to the adjusted first language model and / or the second language model.

4. The method according to claim 3, characterized in that In response to the adjustment operation on the second metadata, adjusting the first language model and / or the second language model to update the second metadata according to the adjusted first language model and / or the second language model includes: In response to an adjustment operation on the second metadata, obtaining adjustment data of the second metadata; The first language model and / or the second language model are adjusted according to the adjustment data of the second metadata, so as to update the second metadata according to the adjusted first language model and / or the second language model.

5. The method according to claim 4, characterized in that The adjusting the first language model and / or the second language model according to the adjustment data of the second metadata, so as to update the second metadata according to the adjusted first language model and / or the second language model, includes: determining the adjustment data of the first metadata according to the adjustment data of the second metadata; The first language model is trained using the adjusted data of the first metadata as first training data to obtain the trained first language model, and the step of inputting the first metadata into the first language model to obtain a plurality of prompt words for the energy data is returned to.

6. The method according to claim 4, characterized in that The adjusting the first language model and / or the second language model according to the adjustment data of the second metadata, so as to update the second metadata according to the adjusted first language model and / or the second language model, includes: updating the plurality of prompt words according to the adjustment data of the second metadata; The updated prompt words are used as second training data to train the second language model to obtain the trained second language model, and the step of inputting the multiple prompt words and parsed data of the energy data into the second language model to obtain second metadata of the energy data is returned to, so that the data structure of the second metadata corresponds to the energy type corresponding to the energy data.

7. The method according to claim 1, characterized in that The method further comprises: The second metadata is stored together with the energy data.

8. An automatic metadata extraction device for the energy field, characterized in that: include: A data extraction module, configured to determine first metadata of energy data and determine parsed data of the energy data; The first metadata has a preset data structure; a prompt word generation module, configured to input the first metadata into a first language model to obtain a plurality of prompt words for the energy data; The generated plurality of prompt words correspond to energy types corresponding to the energy data; The metadata generation module is used to input the multiple prompt words and parsed data of the energy data into a second language model to obtain second metadata of the energy data, so that the data structure of the second metadata corresponds to the energy type corresponding to the energy data.

9. An electronic device, characterized in that: include: A processor, a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 7.

Images