Data resource classification and level-to-level management method, system and device for energy field and medium

Abstract

The invention discloses a data resource classification and level-to-level management method, system and device oriented to the energy field and a medium, and belongs to the technical field of data management in the energy field. The method comprises the following steps: acquiring a data management range comprising unstructured data and structured data; constructing an initial data directory comprising a plurality of levels according to the data management range; after the unstructured data are converted into structured data, all the structured data are stored in a pre-constructed database, and the structured data comprise data entities and keyword vectors; based on the keyword vector, classifying data entities in all the structured data by adopting a large model technology and a similarity algorithm, and storing a classification result into a corresponding classification hierarchy in the initial data directory to obtain a complete data directory; and based on the complete data directory, grading the data entities which do not reach the preset security classification to obtain a grading result. Unification of data classification and grading standards is realized, and the management efficiency is improved.

Description

Technical Field

[0001] This invention belongs to the field of data management technology in the energy sector, specifically relating to a method, system, device, and medium for classifying and hierarchically managing data resources in the energy sector. Background Technology

[0002] With the advancement of digital transformation in the energy industry, various energy companies have generated massive amounts of data during their production and operations. However, for most companies, this data is scattered across different business systems (such as power plant monitoring systems, contract systems, and smart construction site systems), and current data management methods have several problems:

[0003] (1) Data misalignment: Departments within an enterprise often manage data according to their own business needs, resulting in inconsistent data formats and rules at the company level, making cross-departmental data integration and business collaboration extremely difficult.

[0004] (2) Compliance is difficult to guarantee: Energy data contains a large amount of sensitive information, such as user personal information, power plant design parameters, and enterprise operating data. However, the existing management methods lack a systematic compliance review mechanism. Some enterprises have problems such as collecting user information without clear authorization and not protecting sensitive data in a graded manner, which may lead to serious legal consequences.

[0005] (3) Data potential is not developed: Due to the lack of effective classification and grading of existing data management, energy companies have difficulty in clearly grasping the full picture of their own data resources, resulting in the inability to accurately identify high-value data and cross-domain data relationships.

[0006] (4) Lack of dynamic maintenance: When energy companies adjust their business strategies and upgrade their system architecture, the data resource catalog is not updated in a timely manner, and the metadata is seriously deviated, resulting in the data catalog being disconnected from the actual data status and affecting the accuracy of subsequent data applications. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system, device and medium for the classification and hierarchical management of data resources in the energy field, which overcomes the defects of inconsistent standards, serious data silos, difficulty in ensuring compliance, insufficient value mining and lack of dynamic maintenance in the existing data management of the energy field.

[0008] This invention provides the following technical solution:

[0009] Firstly, a data resource classification and hierarchical management method for the energy sector is provided, including: acquiring the data management scope including unstructured data and structured data;

[0010] An initial data catalog comprising several levels is constructed based on the aforementioned data management scope;

[0011] After converting the unstructured data into structured data, all the structured data is stored in a pre-built database. The structured data includes data entities and keyword vectors.

[0012] Based on the keyword vector, large model technology and similarity algorithm are used to classify the data entities in all structured data, and the classification results are stored in the corresponding classification level in the initial data directory to obtain the complete data directory;

[0013] Based on the complete data catalog, data entities that do not meet the preset security level are classified, and the classification results are obtained.

[0014] As an optional technical solution of the present invention, both the unstructured data and the structured data include business modules, business domains and business domain subdomains;

[0015] The initial data catalog includes business segment level, business domain level, and business domain subdomain level.

[0016] As an optional technical solution of the present invention, the conversion of unstructured data into structured data includes:

[0017] The unstructured data includes image-type unstructured data. For the image-type unstructured data, the OCR method is used to perform text parsing on the image-type unstructured data to obtain text parsing results. The text parsing results are then input into a pre-acquired large language model, and data entities are extracted based on preset prompt words.

[0018] The extracted data entities are converted into structured data.

[0019] As an optional technical solution of the present invention, the step of classifying data entities in all structured data based on keyword vectors and using a similarity algorithm includes:

[0020] Calculate the distance between the keyword vector of each structured data and the name vector of all business domain subdomains, sort the distances from smallest to largest, and select the top M distances, represented as:

[0021] ;

[0022] in, Indicates the first Each business domain subdomain level, Indicates the first Keyword vectors Indicates the first Business domain subdomain level With the The distance between elements in the keyword vector, where M represents the minimum length of the keyword vector for all structured data. This indicates taking the first M distances;

[0023] The first M distances are summed, and the structured data is classified into the business domain subdomain level corresponding to the maximum summation result. The data entities corresponding to the structured data are then used as the data entity level.

[0024] As an optional technical solution of the present invention, the step of storing the classification results in the corresponding classification level of the initial data directory to obtain the complete data directory includes:

[0025] The complete data catalog includes business segment hierarchy, business domain hierarchy, business domain subdomain hierarchy, business theme hierarchy, data entity hierarchy, and attribute hierarchy;

[0026] Using preset prompts, the field attributes of structured data are divided into full Pinyin, first letter of Pinyin, full Pinyin of English words, and first letter of English words. The field attributes, their corresponding structured data, and keyword vectors are input into a large language model to obtain the attribute hierarchy.

[0027] A large language model is used to cluster data entities at the data entity level to obtain the business theme level.

[0028] As an optional technical solution of the present invention, the step of classifying data entities that do not reach the preset security level to obtain the classification result includes:

[0029] The score S of data entities that do not meet the preset security level is calculated using a pre-built data classification and scoring model, and is expressed as:

[0030] S= ;

[0031] in, Represents the data openness coefficient. Indicates the time-related decay coefficient. Indicates the economic loss coefficient. Indicates the influence range coefficient. Indicates the severity level of the threat;

[0032] Based on the score S and the preset score threshold, the classification result of data entities that do not reach the preset security level is determined.

[0033] Secondly, a data resource classification and grading management system for the energy sector is provided, including:

[0034] The data acquisition module is used to acquire data management scope including both unstructured and structured data;

[0035] The directory building module is used to build an initial data directory including several levels according to the data management scope;

[0036] The conversion module is used to convert the unstructured data into structured data and then store all the structured data in a pre-built database. The structured data includes data entities and keyword vectors.

[0037] The classification module is used to classify the data entities in all structured data based on the keyword vector, using large model technology and similarity algorithm, and store the classification results in the corresponding classification level in the initial data directory to obtain the complete data directory;

[0038] The classification module is used to classify data entities that do not meet the preset security level based on the complete data catalog, and obtain the classification result.

[0039] Thirdly, an apparatus is provided, comprising: a memory, a processor, and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect.

[0040] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0041] Compared with the prior art, the beneficial effects of the present invention are:

[0042] This invention provides a data resource classification and grading management method for the energy sector. By developing a data catalog adapted to the energy sector, it achieves unified enterprise-level data classification and grading standards, ensuring consistent understanding of data across departments and improving internal collaboration efficiency. Data that does not meet the preset confidentiality level is classified to protect sensitive information. Attached Figure Description

[0043] Figure 1 This is a flowchart of a data resource classification and hierarchical management method for the energy sector, as described in this embodiment of the invention. Detailed Implementation

[0044] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0045] Example 1

[0046] This embodiment provides a method for classifying and hierarchically managing data resources in the energy sector.

[0047] like Figure 1 As shown, the specific steps include the following:

[0048] Step 1: Obtain the scope of data management, including both unstructured and structured data.

[0049] Both the unstructured and structured data include business segments, business domains, and business domain subdomains.

[0050] Step 2: Construct an initial data catalog with several levels based on the data management scope.

[0051] The initial data directory includes business segment hierarchy, business domain hierarchy, and business domain subdomain hierarchy. The business segment hierarchy, business domain hierarchy, and business domain subdomain hierarchy have been confirmed in the initial data directory.

[0052] Step 3: After converting the unstructured data into structured data, store all the structured data in a pre-built database. The structured data includes data entities and keyword vectors.

[0053] The unstructured data includes image-type unstructured data. For the image-type unstructured data, the OCR method is used to perform text parsing on the image-type unstructured data to obtain text parsing results. The text parsing results are then input into a pre-acquired large language model, and data entities are extracted based on preset prompt words.

[0054] The extracted data entities are converted into structured data. In this embodiment, the extracted data entities are organized into SQL statements and stored in the database.

[0055] In this embodiment, for unstructured data that is not suitable for OCR or large model technology analysis, entries can be manually added to the data directory.

[0056] Step 4: Based on the keyword vector, use large model technology and similarity algorithm to classify the data entities in all structured data, and store the classification results in the corresponding classification level in the initial data directory to obtain the complete data directory.

[0057] Calculate the distance between the keyword vector of each structured data and the name vector of all business domain subdomains, sort the distances from smallest to largest, and select the top M distances, represented as:

[0058] ;

[0059] in, Indicates the first Each business domain subdomain level, Indicates the first Keyword vectors Indicates the first Business domain subdomain level With the The distance between elements in the keyword vector, where M represents the minimum length of the keyword vector for all structured data. This indicates taking the first M distances;

[0060] The first M distances are summed, and the structured data is classified into the business domain subdomain level corresponding to the maximum summation result. The data entities corresponding to the structured data are then used as the data entity level.

[0061] The complete data catalog includes business segment level, business domain level, business domain subdomain level, business theme level, data entity level, and attribute level.

[0062] By using preset prompts, the field attributes of structured data are divided into full Pinyin spelling, first letter of Pinyin spelling, full Pinyin spelling of English words, and first letter of English words. The field attributes, their corresponding structured data, and keyword vectors are then input into a large language model to obtain the attribute hierarchy.

[0063] A large language model is used to cluster data entities at the data entity level to obtain the business theme level.

[0064] Step 5: Based on the complete data catalog, classify the data entities that do not reach the preset security level to obtain the classification results.

[0065] The score S of data entities that do not meet the preset security level is calculated using a pre-built data classification and scoring model, and is expressed as:

[0066] S= ;

[0067] in, Represents the data openness coefficient. Indicates the time-related decay coefficient. Indicates the economic loss coefficient. Indicates the influence range coefficient. Indicates the severity level of the threat; = , , , , , Indicates the age of the data. This indicates the company's estimated economic losses following the data breach. This indicates the percentage of company employees directly affected by a data breach out of the company's total workforce.

[0068] Based on the score S and a preset score threshold, the classification results of data entities that do not reach the preset security level are determined, and the compliance of the classified data is manually verified. In this embodiment, the score... As a scoring threshold, data with a score less than 1 is classified as the lowest level of internal protection, while data with a score greater than 7 is classified as the highest level of internal protection.

[0069] In this embodiment, a data leakage emergency plan is configured for each level of the complete data catalog. Simultaneously, version management is implemented for the complete data catalog, employing an event-driven update strategy and automatic updates in conjunction with changes to the data platform. The version number format is "major version.minor version.revision number," and the change rules are as follows: a major version number change indicates a significant adjustment to the energy data classification standards; a minor version number change indicates important improvements to the energy data management functions or the addition of new energy data items; and a revision number change indicates the fixing of minor errors in the energy data catalog or minor adjustments.

[0070] Example 2

[0071] This embodiment provides a data resource classification and grading management system for the energy sector, including:

[0072] The data acquisition module is used to acquire data management scope including both unstructured and structured data;

[0073] The directory building module is used to build an initial data directory including several levels according to the data management scope;

[0074] The conversion module is used to convert the unstructured data into structured data and then store all the structured data in a pre-built database. The structured data includes data entities and keyword vectors.

[0075] The classification module is used to classify the data entities in all structured data based on the keyword vector, using large model technology and similarity algorithm, and store the classification results in the corresponding classification level in the initial data directory to obtain the complete data directory;

[0076] The classification module is used to classify data entities that do not meet the preset security level based on the complete data catalog, and obtain the classification result.

[0077] Example 3

[0078] This embodiment provides an apparatus, including: a memory, a processor, and a computer program;

[0079] The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in Example 1.

[0080] Example 4

[0081] This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in Embodiment 1.

[0082] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for classifying and hierarchically managing data resources in the energy sector, characterized in that, include: To acquire the scope of data management, including both unstructured and structured data; An initial data catalog comprising several levels is constructed based on the aforementioned data management scope; After converting the unstructured data into structured data, all the structured data is stored in a pre-built database. The structured data includes data entities and keyword vectors. Based on the keyword vector, large model technology and similarity algorithm are used to classify the data entities in all structured data, and the classification results are stored in the corresponding classification level in the initial data directory to obtain the complete data directory; Based on the complete data catalog, data entities that do not meet the preset security level are classified, and the classification results are obtained.

2. The data resource classification and hierarchical management method for the energy sector according to claim 1, characterized in that, Both the unstructured data and the structured data include business modules, business domains, and business domain subdomains; The initial data catalog includes business segment level, business domain level, and business domain subdomain level.

3. The data resource classification and hierarchical management method for the energy sector according to claim 1, characterized in that, The process of converting unstructured data into structured data includes: The unstructured data includes image-type unstructured data. For the image-type unstructured data, the OCR method is used to perform text parsing on the image-type unstructured data to obtain text parsing results. The text parsing results are then input into a pre-acquired large language model, and data entities are extracted based on preset prompt words. The extracted data entities are converted into structured data.

4. The data resource classification and hierarchical management method for the energy sector according to claim 2, characterized in that, The method of classifying data entities in all structured data based on keyword vectors and using a similarity algorithm includes: Calculate the distance between the keyword vector of each structured data and the name vector of all business domain subdomains, sort the distances from smallest to largest, and select the top M distances, represented as: ； in, Indicates the first Each business domain subdomain level, Indicates the first Keyword vectors Indicates the first Business domain subdomain level With the The distance between elements in the keyword vector, where M represents the minimum length of the keyword vector for all structured data. This indicates taking the first M distances; The first M distances are summed, and the structured data is classified into the business domain subdomain level corresponding to the maximum summation result. The data entities corresponding to the structured data are then used as the data entity level.

5. The data resource classification and hierarchical management method for the energy sector according to claim 4, characterized in that, The step of storing the classification results in the corresponding classification level of the initial data directory to obtain the complete data directory includes: The complete data catalog includes business segment hierarchy, business domain hierarchy, business domain subdomain hierarchy, business theme hierarchy, data entity hierarchy, and attribute hierarchy; Using preset prompts, the field attributes of structured data are divided into full Pinyin, first letter of Pinyin, full Pinyin of English words, and first letter of English words. The field attributes, their corresponding structured data, and keyword vectors are input into a large language model to obtain the attribute hierarchy. A large language model is used to cluster data entities at the data entity level to obtain the business theme level.

6. The data resource classification and hierarchical management method for the energy sector according to claim 1, characterized in that, The process of classifying data entities that do not meet the preset security level to obtain classification results includes: The score S of data entities that do not meet the preset security level is calculated using a pre-built data classification and scoring model, and is expressed as: S= ； in, Represents the data openness coefficient. Indicates the time-related decay coefficient. Indicates the economic loss coefficient. Indicates the influence range coefficient. Indicates the severity level of the threat; Based on the score S and the preset score threshold, the classification result of data entities that do not reach the preset security level is determined.

7. A data resource classification and hierarchical management system for the energy sector, characterized in that, include: The data acquisition module is used to acquire data management scope including both unstructured and structured data; The directory building module is used to build an initial data directory including several levels according to the data management scope; The conversion module is used to convert the unstructured data into structured data and then store all the structured data in a pre-built database. The structured data includes data entities and keyword vectors. The classification module is used to classify the data entities in all structured data based on the keyword vector, using large model technology and similarity algorithm, and store the classification results in the corresponding classification level in the initial data directory to obtain the complete data directory; The classification module is used to classify data entities that do not meet the preset security level based on the complete data catalog, and obtain the classification result.

8. An apparatus, characterized in that, include: Memory, processor, and computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the method described in any one of claims 1-6.

Images