Data fusion discovery method, system and equipment for energy digital network and medium

Abstract

The invention discloses an energy digital network-oriented data fusion discovery method, system and device and a medium. The method comprises the following steps: acquiring each energy digital object oriented to an energy digital network; combining the feature attributes extracted from the metadata of each energy digital object to obtain a text statement set of each energy digital object; utilizing a pre-training language model to extract semantic features of the energy digital object according to the text statement set, and generating a corresponding semantic feature vector; clustering the semantic feature vectors, performing pairwise similarity calculation on energy digital objects in each cluster in an optimal clustering result, and combining the energy digital objects with similarity to form a similar energy digital object combination set; fusing the energy digital objects in each combination in the similar energy digital object combination set to construct a new energy digital object; and complementation of cross-domain energy data is realized.

Description

Technical Field

[0001] The present invention relates to the field of energy digital processing, and in particular to a data fusion discovery method, system, device and medium for energy digital networks. Background Art

[0002] The Digital Object Architecture (DOA) is a logical extension of the existing Internet architecture. It supports the data interoperability requirements between different systems, rather than just transmitting digital information from one location on the Internet to another. The basic element in the DOA is the digital object, which is a unified abstraction of data resources in any information system. A complete digital object can be divided into three parts: identification, metadata, and entity. Among them, identification is the unique identity tag of the digital object, metadata is the descriptive information related to the digital object and the application and business, and the digital object entity is the actual content of the data resource. The energy digital network based on digital objects forms a new type of energy digital network across subjects and heterogeneous networks by designing a unified access protocol for cross-subject digital objects, establishing access links between multi-subject energy digital objects, realizing active access and addressing access of energy digital objects, and providing platform-based and technical data circulation support capabilities.

[0003] Energy data is large in scale, widely distributed, and changes rapidly. It has the characteristics of multi-source heterogeneity and data islands. Joint search between different data is time-consuming and ineffective. Based on the encapsulation of energy digital objects, the multi-source heterogeneity problem of multi-subject energy data is solved. The construction of energy digital network solves the data island problem between multi-subject energy data. The knowledge graph technology for energy digital objects can aggregate energy digital objects on the energy digital network and the link relationship between objects into knowledge, making energy digital objects easier to understand, calculate and apply, and can achieve rapid response and reasoning of energy data.

[0004] However, on this basis, it is impossible to solve the problem of identifying and merging similar data sets in the energy digital network, and thus it is impossible to realize the automatic discovery of energy data. This has greatly affected the efficiency of energy data processing and analysis, especially the search between cross-domain energy data, and cannot meet the current accuracy and efficiency requirements. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a data fusion discovery method, system, device and medium for energy digital networks, which can realize automatic identification of similar energy digital objects, and construct new energy digital objects based on similar energy digital objects, thereby providing data fusion discovery for energy digital networks.

[0006] The present invention is achieved through the following technical solutions: The present invention provides a data fusion discovery method for energy digital networks, comprising: Obtain various energy digital objects for the energy digital network; Merging characteristic attributes extracted from metadata of each energy digital object to obtain a text sentence set of each energy digital object; Using a pre-trained language model, extracting semantic features of the energy digital object according to the text sentence set, and generating a corresponding semantic feature vector; Clustering the semantic feature vectors, performing pairwise similarity calculation on the energy digital objects in each cluster in the optimal clustering result, and combining the energy digital objects with similarity to form a similar energy digital object combination set; The energy digital objects in each combination of similar energy digital objects are merged to construct a new energy digital object.

[0007] The present invention also provides a data fusion discovery system for energy digital networks, comprising: An acquisition module, used for acquiring various energy digital objects for the energy digital network; A feature attribute extraction module, used to merge the feature attributes extracted from the metadata of each energy digital object to obtain a text sentence set of each energy digital object; A semantic feature extraction module, using a pre-trained language model, extracts semantic features of energy digital objects according to the text sentence set, and generates corresponding semantic feature vectors; The clustering module clusters the semantic feature vectors, performs pairwise similarity calculation on the energy digital objects in each cluster in the optimal clustering result, and combines the energy digital objects with similarity to form a similar energy digital object combination set; The fusion discovery module fuses the energy digital objects in each combination of similar energy digital objects to construct a new energy digital object.

[0008] The present invention also provides a computer device, comprising: Memory for storing computer programs; A processor is used to implement the data fusion discovery method for energy digital networks when executing the computer program.

[0009] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the data fusion discovery method for energy digital networks is implemented.

[0010] The present invention also provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the steps of the data fusion discovery method for energy digital networks are implemented.

[0011] Compared with the prior art, the present invention has the following beneficial technical effects: The data fusion discovery method of the present invention is oriented to the energy digital network, extracts energy digital objects and their attribute features, uses natural language processing technology to extract semantic features, cooperates with machine learning technology to analyze the semantic similarity between energy digital objects, and constructs new energy digital objects through the fusion of similar energy digital objects, and finally registers them in the energy digital network, thereby realizing automatic discovery of energy data. The method of the present invention optimizes the data search and exploration process, realizes the complementarity of cross-domain energy data, improves the efficiency and accuracy of energy data processing and analysis, promotes the release of energy data value, and supports intelligent decision-making in the energy field. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flowchart of the data fusion discovery method described in an example of the present invention.

[0013] Figure 2 It is a structural block diagram of the data fusion discovery system described in an example of the present invention.

[0014] Figure 3 It is a flowchart of the practical application of the data fusion discovery method described in the example of the present invention.

[0015] Figure 4 It is a structural schematic diagram of a computer device described in an example of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0018] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0019] In the present invention, "module", "device", "system" and the like refer to related entities applied to computers, such as hardware, a combination of hardware and software, software or software in execution, etc. In detail, for example, an element can be, but is not limited to, a process, a processor, an object, an executable element, an execution thread, a program and / or a computer running on a processor. In addition, an application or script program running on a server, a server can all be an element. One or more elements can be in an execution process and / or thread, and an element can be localized on a computer and / or distributed between two or more computers, and can be operated by various computer-readable media. An element can also communicate through local and / or remote processes according to a signal with one or more data packets, for example, a signal from a data that interacts with another element in a local system, a distributed system, and / or a network on the Internet through a signal to interact with other systems.

[0020] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include" and "comprise" include not only those elements, but also other elements that are not explicitly listed, or also include elements inherent to such processes, methods, articles or equipment. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or equipment that includes the elements.

[0021] In the existing technology, centralized data indexing and search technologies are difficult to apply to situations where data is physically dispersedly stored. It is necessary to explore distributed data search and discovery technologies to support data recommendations for business scenarios.

[0022] The method of the present invention utilizes the complementary relationship between different energy data, especially between cross-domain energy data, and can obtain more comprehensive information through mutual combination. For example, in the field of energy consumption, by integrating the consumption data of multiple energy sources, it is possible to easily obtain a full amount of energy consumption information, thereby improving the accuracy of data analysis applications in the field of energy consumption. Since there is a certain correlation between different data sources, more accurate information can be obtained by mutual combination. Similarity measurement is a method for calculating the degree of similarity between data, and is the basis for data similarity fusion; according to the similarity measurement results, similar data items are fused to generate a more comprehensive and accurate data set. Among them, energy digital objects are created based on energy data and inherit the relevant characteristics of energy data. Therefore, by calculating the feature similarity of energy digital objects, similar energy digital objects can be fused and new energy digital objects can be constructed to achieve automatic discovery of energy data.

[0023] Specifically, Figure 1 As shown, a data fusion discovery method for energy digital networks includes: S1. Acquire various energy digital objects for the energy digital network; Facing the energy digital network, it is actually facing the energy digital network system. Therefore, it is possible to traverse the energy digital objects through the access interface to obtain each energy digital object and its metadata information; thus, it is possible to obtain relevant description information of the energy digital objects, applications and services, providing a data basis for subsequent operations.

[0024] S2. Merging the characteristic attributes extracted from the metadata of each energy digital object to obtain a text sentence set of each energy digital object; When extracting characteristic attributes from metadata, it is necessary to analyze the metadata of the energy data object, and then select "subject, subject classification, industry category, industry subcategory, business keyword, usage keyword, location, time information" etc. as the characteristic attributes of the energy digital object based on the contribution of the metadata to the characteristic identification of the energy data object, which are used to judge the similarity between energy digital objects.

[0025] After the characteristic attributes are selected, the characteristic attributes of each energy digital object are merged respectively to form a text sentence representing the energy digital object and a text sentence set of each energy digital object.

[0026] S3, using a pre-trained language model to extract semantic features of the energy digital object according to the text sentence set, and generate a corresponding semantic feature vector; Before using the pre-trained language model for natural language processing, it is necessary to perform text segmentation on the text sentence set of the energy digital object to obtain the corresponding text feature vector, and then use the pre-trained language model to calculate and extract the text feature vector of the segmented text sentence to achieve semantic feature extraction of the energy digital object and generation of feature vectors.

[0027] S4, clustering the semantic feature vectors, performing pairwise similarity calculation on the energy digital objects in each cluster in the optimal clustering result, and combining the energy digital objects with similarity to form a similar energy digital object combination set; Clustering is an unsupervised learning method used to divide data instances into multiple groups so that the distance between instances in the same group is small and the distance between the same groups is large. The data is operated by iteratively updating the cluster center and assigning data points to the nearest cluster center. The goal of the clustering algorithm is to minimize the internal distance and maximize the spacing, so as to classify similar energy digital objects. Therefore, the energy digital objects in each cluster in the optimal clustering result can be similar to each other and the energy digital objects that can be fused can be found.

[0028] S5. Fusion the energy digital objects in each combination of similar energy digital objects to construct a new energy digital object.

[0029] By integrating similar energy digital objects, new energy digital objects can be constructed and then registered in the energy digital network, thereby realizing a one-time discovery of the energy digital objects. By repeating steps S1 to S5, continuous and sustained discovery and updating of energy digital objects can be realized in the energy digital network.

[0030] During fusion, the metadata information of the energy digital objects in each combination is first fused and updated, and the data sources are fused by building an association table, aligning the data, and disambiguating the data to generate a new identifier. A new energy digital object is constructed based on the fused metadata information, data source, and identifier.

[0031] In actual applications, it is possible to recommend and integrate user data according to needs, integrate existing data objects according to customer needs, and form data objects that meet customer needs; and realize the induction, concentration and linking of energy data objects.

[0032] In a preferred embodiment of the present invention, a pre-training process of a pre-training model is provided; the pre-training language model is an ALBERT pre-training model, and is trained using the following method: Use the word segmentation tool to segment the text sentences of each energy digital object, extract each word and generate a feature attribute word list; According to the feature attribute vocabulary, the text sentences of each energy digital object are converted into a training data set in binary file format; Take the training dataset in binary file format as input, train the ALBERT pre-trained model, and generate the ALBERT pre-trained model.

[0033] In this preferred embodiment, the binary format preferably adopts the tfrecord format, which is a binary file format recommended by the TensorFlow framework and is mainly used for storing and transmitting large-scale data sets, such as images, text or voice data.

[0034] Specifically include: The first step is to traverse the energy digital objects through the access interface of the energy digital network system to obtain each energy digital object and its metadata information; analyze the metadata of the energy data object, and according to the contribution of the metadata to the feature identification of the energy data object, select "subject, subject classification, industry category, industry subcategory, business keyword, usage keyword, location, time information" as the characteristic attributes of the energy digital object to judge the similarity between energy digital objects.

[0035] Create an energy digital object attribute matrix based on energy digital objects and their characteristic attributes :

[0036] Among them, id is the coding identifier of the energy digital object, and attr is the attribute vector of the energy digital object, which is defined as follows:

[0037] Where i represents the i-th energy digital object, They respectively represent the characteristic attributes of energy digital objects, such as "subject, subject classification, industry category, industry subcategory, business keyword, usage keyword, location, time information" and so on.

[0038] The second step is to merge the characteristic attributes of the energy digital object to form a text sentence that represents the energy digital object. :

[0039] Use SPM (Sentence Piece Model) word segmentation tool to analyze the text sentences of each energy digital object Perform text segmentation, extract each word and generate a feature attribute vocabulary; According to the characteristic attribute word list, the text sentences of each energy digital object are Convert the training data set into tfrecord format; The training dataset in tfrecord format is used as input to train the ALBERT pre-trained model and generate an ALBERT pre-trained model for semantic feature extraction of energy digital objects.

[0040] In a preferred embodiment of the present invention, the semantic feature vectors are clustered, specifically including clustering using a K-means algorithm, determining the clustering value in the K-means clustering by calculating the average silhouette coefficient method, and obtaining an optimal clustering result; The silhouette coefficient SC is as follows:

[0041] Among them, a represents the average distance between the energy digital object and other energy digital objects in the cluster, and b represents the minimum value of the average distance between the energy digital object and other clusters; The range of SC is [-1,1]. According to the SC change trend, the clustering result with the largest SC value is selected as the optimal clustering result.

[0042] In a preferred embodiment of the present invention, the energy digital objects in each cluster in the optimal clustering result are subjected to pairwise similarity calculation, and the energy digital objects with similarity are combined to form a similar energy digital object combination set, which specifically includes: The cosine similarity of the semantic feature vectors of any two energy digital objects is calculated according to the following formula: :

[0043] Among them, x and y are the semantic feature vectors of two energy digital objects, ||x||, ||y|| represents the modulus of x and y vectors; If the cosine similarity value is in the interval [0.8, 1], the two energy digital objects are considered similar; the interval range [0.8, 1] can be obtained based on expert experience; Count the energy digital objects with similarities and combine them to form a combination set of similar energy digital objects :

[0044] in, It represents the combination of energy digital objects with similarity, and t is the number of combinations.

[0045] Corresponding to the above data fusion method, the present invention also provides a data fusion discovery system for energy digital networks, such as Figure 2 As shown, it includes, An acquisition module 201 is used to acquire various energy digital objects for the energy digital network; The feature attribute extraction module 202 is used to merge the feature attributes extracted from the metadata of each energy digital object to obtain a text sentence set of each energy digital object; The semantic feature extraction module 203 uses a pre-trained language model to extract the semantic features of the energy digital object according to the text sentence set and generates a corresponding semantic feature vector; The clustering module 204 clusters the semantic feature vectors, performs pairwise similarity calculation on the energy digital objects in each cluster in the optimal clustering result, and combines the energy digital objects with similarity to form a similar energy digital object combination set; The fusion discovery module 205 fuses the energy digital objects in each combination in the similar energy digital object combination set to construct a new energy digital object.

[0046] In a preferred embodiment of the system of the present invention, the clustering module is further used to perform clustering using a K-means algorithm, and to determine the clustering value in the K-means clustering by calculating the average silhouette coefficient method to obtain the optimal clustering result; The silhouette coefficient SC is as follows:

[0047] Among them, a represents the average distance between the energy digital object and other energy digital objects in the cluster, and b represents the minimum value of the average distance between the energy digital object and other clusters; The range of SC is [-1,1]. According to the SC change trend, the clustering result with the largest SC value is selected as the optimal clustering result.

[0048] In a preferred embodiment of the system of the present invention, the clustering module is further used to calculate the cosine similarity of the semantic feature vectors of any two energy digital objects according to the following formula: :

[0049] Among them, x and y are the semantic feature vectors of two energy digital objects, ||x||, ||y|| represents the modulus of x and y vectors; According to expert experience, if the value of cosine similarity is in the interval [0.8, 1], two energy digital objects are considered similar; Count the energy digital objects with similarities and combine them to form a combination set of similar energy digital objects :

[0050] in, It represents the combination of energy digital objects with similarity, and t is the number of combinations.

[0051] In a preferred example of the system described in the present invention, the fusion discovery module is also used to fuse and update the metadata information of the energy digital objects in each combination, fuse the data sources by building an association table, aligning the data, and disambiguating the data, generate a new identifier, build a new energy digital object based on the fused metadata information, data source and identifier, and register the new energy digital object in the energy digital network to realize energy data fusion discovery.

[0052] In the actual application process of the present invention, the following steps can be followed, wherein the first two steps are the training process of the pre-trained model, which is necessary when used for the first time, but does not need to be repeated each time the data fusion is discovered subsequently. It only needs to be executed when the model needs to be evolved and updated.

[0053] (1) Extraction of energy digital objects and generation of attribute matrix. For the energy digital network system, traverse the energy digital objects through the access interface to obtain each energy digital object and its metadata information; analyze the metadata of the energy data object, and select "subject, subject classification, industry category, industry subcategory, business keyword, usage keyword, location, time information" as the characteristic attributes of the energy digital object based on the contribution of the metadata to the feature identification of the energy data object, which is used to judge the similarity between energy digital objects. When obtaining energy digital objects, the best situation is to obtain all energy digital objects and their metadata information, which is conducive to the judgment of data similarity and the discovery of new energy digital objects.

[0054] (2) Training of the ALBERT pre-trained model for feature extraction. The feature attributes of energy digital objects are merged to form text sentences that represent energy digital objects; the text sentences of each energy digital object are segmented using the SPM (Sentence Piece Model) word segmentation tool. Perform text segmentation, extract each word and generate a feature attribute word list; according to the feature attribute word list, the text sentences of each energy digital object are Convert the training dataset into tfrecord format; use the training dataset in tfrecord format as input to train the ALBERT pre-trained model and generate the ALBERT pre-trained model.

[0055] (3) Extraction of semantic features of energy digital objects. For the energy digital network system, traverse the energy digital objects through the access interface, obtain the newly added energy digital objects, and add the attributes of the newly added energy digital objects to the energy digital object attribute matrix; merge the characteristic attributes of each energy digital object to form a text sentence set of each energy digital object; use the SPM word segmentation tool to perform text segmentation on the text sentence set of the energy digital object, calculate and extract the text feature vector of the text sentence that has been segmented based on the ALBERT pre-trained model, and realize the semantic feature extraction of energy digital objects and the generation of feature vectors.

[0056] (4) Clustering based on semantic features of energy digital objects. Based on the semantic feature vectors of energy digital objects, the K-means algorithm is used for clustering; the clustering value in the K-means clustering is determined by calculating the average silhouette coefficient method.

[0057] (5) Calculate the similarity of energy digital objects. Based on the semantic feature vectors of energy digital objects, calculate the cosine similarity of the semantic features of any two energy digital objects in each cluster after clustering, and count the energy digital objects with similarities to form a combination set of similar energy digital objects.

[0058] (6) Construction and registration of new energy digital objects. Energy digital objects within similar energy digital object combinations are integrated. First, metadata information is integrated and updated. Then, data sources are integrated by building association tables, aligning data, and eliminating data ambiguity. Finally, new identifiers are generated. New energy digital objects are then constructed and registered in the energy digital network to achieve automatic discovery of energy data.

[0059] Specifically, Figure 3 As shown, the following steps are included.

[0060] Step 1: Extract energy digital objects and generate attribute matrix.

[0061] Facing the energy digital network system, the energy digital objects are traversed through the access interface to obtain each energy digital object and its metadata information; the metadata of the energy data object is analyzed, and according to the contribution of the metadata to the feature identification of the energy data object, "subject, subject classification, industry category, industry subcategory, business keyword, usage keyword, location, time information" are selected as the characteristic attributes of the energy digital object to judge the similarity between energy digital objects.

[0062] Create an energy digital object attribute matrix based on energy digital objects and their characteristic attributes :

[0063] Among them, id is the coding identifier of the energy digital object, and attr is the attribute vector of the energy digital object, which is defined as follows:

[0064] Where i represents the i-th energy digital object , , , , , , , , which respectively represent the characteristic attributes of energy digital objects such as "subject, subject classification, industry category, industry subcategory, business keyword, use keyword, location, time information" and so on.

[0065] Step 2: Training of the ALBERT pre-trained model for feature extraction.

[0066] Merge the characteristic attributes of energy digital objects to form text sentences that represent energy digital objects :

[0067] Use SPM (Sentence Piece Model) word segmentation tool to analyze the text sentences of each energy digital object Perform text segmentation, extract each word and generate a feature attribute vocabulary; According to the characteristic attribute word list, the text sentences of each energy digital object are Convert the training data set into tfrecord format; Take the training dataset in tfrecord format as input, train the ALBERT pre-trained model, and generate the ALBERT pre-trained model.

[0068] Step 3: Extract semantic features of energy digital objects.

[0069] Facing the energy digital network system, the energy digital objects are traversed through the access interface to obtain the newly added energy digital objects, and the attributes of the newly added energy digital objects are added to the energy digital object attribute matrix according to step 1. middle; Merge the characteristic attributes of each energy digital object separately to form a text statement set of each energy digital object :

[0070] Where m is the number of energy digital objects; Use the SPM word segmentation tool to perform text segmentation on the text sentence set of the energy digital object, and calculate and extract the text sentences that have been segmented based on the ALBERT pre-trained model generated in step 2 The text feature vector of energy digital objects is extracted to realize the semantic feature extraction and feature vector generation.

[0071] Step 4: Clustering based on semantic features of energy digital objects.

[0072] Based on the semantic feature vector of the energy digital object obtained in step 3, clustering is performed using the K-means algorithm; The clustering value in K-means clustering is determined by calculating the average silhouette coefficient method. The calculation method of the silhouette coefficient SC is as follows:

[0073] Where a represents the average distance between the energy digital object and other energy digital objects in the cluster, and b represents the minimum value of the average distance between the energy digital object and other clusters; The range of SC is [-1,1]. According to the SC change trend, the clustering result with the largest SC value is selected as the optimal result.

[0074] Step 5: Calculate the similarity of energy digital objects.

[0075] Based on the clustering results of step 4, the similarity between the energy digital objects in the cluster is calculated. Specifically, the semantic feature vectors of the energy digital objects obtained in step 3 are used to calculate the cosine similarity of the semantic features of any two energy digital objects. , the formula is as follows:

[0076] Where x and y are the semantic feature vectors of two energy digital objects, ||x||, ||y|| represents the modulus of the x and y vectors; According to expert experience, if the value of cosine similarity is in the interval [0.8, 1], two energy digital objects are considered similar; Count the energy digital objects with similarities and combine them to form a combination set of similar energy digital objects :

[0077] in It represents the combination of energy digital objects with similarity, and t is the number of combinations.

[0078] Step 6: Construction and registration of new energy digital objects.

[0079] Combination set of digital objects with similar energy sources , for each combination The energy digital objects in the network are integrated. First, the metadata information is integrated and updated. Then, the data sources are integrated by building association tables, data alignment, data disambiguation, etc. Finally, a new identifier is generated, and then a new energy digital object is built and registered in the energy digital network to realize automatic discovery of energy data.

[0080] In one embodiment of the present invention, a computer device 400 is also provided, such as Figure 4 As shown, including: Memory 401, used for storing computer programs; The processor 402 is used to implement a data fusion discovery method for energy digital networks as described in any one of the examples of the present invention when executing the computer program.

[0081] The system may further include a computer program 403 stored in the memory 401 and executable on the at least one processor 402 and at least one communication bus 404 .

[0082] The memory 401 can be used to store the computer program 403, and the processor 402 implements the steps of the data fusion discovery method for energy digital network described in any one of the examples by running or executing the computer program stored in the memory 401 and calling the data stored in the memory 401. The memory 401 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data) created according to the use of the computer device 400, etc. In addition, the memory 401 can include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0083] The at least one processor 402 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 402 may be a microprocessor or any conventional processor, etc. The processor 402 is the control center of the computer device 400, and uses various interfaces and lines to connect various parts of the entire computer device 400.

[0084] The memory 401 in the computer device 400 stores a plurality of instructions to implement a data fusion discovery method for energy digital networks, and the processor 402 can execute the plurality of instructions to implement: If the module / unit integrated in the computer device 400 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory).

[0085] In one example of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, a data fusion discovery method for energy digital networks as described in any one of the examples of the present invention is implemented.

[0086] A computer program product is also provided in one example of the present invention, including a computer program, which, when executed by a processor, implements the steps of a data fusion discovery method for energy digital networks as described in any one of the examples of the present invention.

[0087] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0089] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0090] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A data fusion discovery method for energy digital networks, characterized in that: include, Obtain various energy digital objects for the energy digital network; Merging characteristic attributes extracted from metadata of each energy digital object to obtain a text sentence set of each energy digital object; Using a pre-trained language model, extracting semantic features of the energy digital object according to the text sentence set, and generating a corresponding semantic feature vector; Clustering the semantic feature vectors, performing pairwise similarity calculation on the energy digital objects in each cluster in the optimal clustering result, and combining the energy digital objects with similarity to form a similar energy digital object combination set; The energy digital objects in each combination of similar energy digital objects are merged to construct a new energy digital object.

2. A data fusion discovery method for energy digital network according to claim 1, characterized in that: The characteristic attributes extracted from the metadata of each energy digital object are selected based on the contribution of the metadata to the characteristic identification of the energy data object, including the subject, subject classification, industry category, industry subcategory, business keyword, use keyword, location and time information.

3. The data fusion discovery method for energy digital network according to claim 1 is characterized in that: The text statement set of each energy digital object includes a text statement of each energy digital object, and the text statement of each energy digital object is represented as follows: ； in, is the text statement of the energy digital object, i represents the i-th energy digital object, They respectively represent the characteristic attributes of the subject, subject classification, industry category, industry subcategory, business keyword, usage keyword, location, and time information of the i-th energy digital object.

4. The data fusion discovery method for energy digital network according to claim 1 is characterized in that: The pre-trained language model is a lightweight bidirectional encoder representation method ALBERT pre-trained model, and is trained using the following method: Use the word segmentation tool to segment the text sentences of each energy digital object, extract each word and generate a feature attribute word list; According to the feature attribute vocabulary, the text sentences of each energy digital object are converted into a training data set in binary file format; Take the training dataset in binary file format as input, train the ALBERT pre-trained model, and generate the ALBERT pre-trained model.

5. The data fusion discovery method for energy digital network according to claim 1 is characterized in that: The clustering of the semantic feature vectors specifically includes: clustering using a K-means algorithm, determining a clustering value in the K-means clustering by calculating an average silhouette coefficient method, and obtaining an optimal clustering result; The silhouette coefficient SC is as follows: Among them, a represents the average distance between the energy digital object and other energy digital objects in the cluster, and b represents the minimum value of the average distance between the energy digital object and other clusters; The range of SC is [-1,1], and the clustering result with the largest SC value is selected as the optimal clustering result.

6. The data fusion discovery method for energy digital network according to claim 1 is characterized in that: The method of performing pairwise similarity calculation on the energy digital objects in each cluster in the optimal clustering result and combining the energy digital objects with similarity to form a similar energy digital object combination set specifically includes: The cosine similarity of the semantic feature vectors of any two energy digital objects is calculated according to the following formula: : Among them, x and y are the semantic feature vectors of two energy digital objects, ||x||, ||y|| represents the modulus of x and y vectors; If the value of cosine similarity is in the interval [0.8, 1], it is determined that the two energy digital objects are similar; Count the energy digital objects with similarities and combine them to form a combination set of similar energy digital objects : in, It represents the combination of energy digital objects with similarity, and t is the number of combinations.

7. The data fusion discovery method for energy digital network according to claim 1 is characterized in that: The step of fusing the energy digital objects in each combination of similar energy digital objects to construct a new energy digital object specifically includes: Merge and update the metadata information of each energy digital object in the combination. Data sources are integrated by building association tables, aligning data, and eliminating data ambiguity. Generate a new ID, Construct new energy digital objects based on the fused metadata information, data sources and identifiers. Register new energy digital objects into the energy digital network to realize energy data fusion discovery.

8. A data fusion discovery system for energy digital networks, characterized in that: include, An acquisition module, used for acquiring various energy digital objects for the energy digital network; A feature attribute extraction module, used to merge the feature attributes extracted from the metadata of each energy digital object to obtain a text sentence set of each energy digital object; A semantic feature extraction module, using a pre-trained language model, extracts semantic features of energy digital objects according to the text sentence set, and generates corresponding semantic feature vectors; The clustering module clusters the semantic feature vectors, performs pairwise similarity calculation on the energy digital objects in each cluster in the optimal clustering result, and combines the energy digital objects with similarity to form a combination set of similar energy digital objects; The fusion discovery module fuses the energy digital objects in each combination of similar energy digital objects to construct a new energy digital object.

9. The data fusion discovery system for energy digital network according to claim 8, characterized in that: The clustering module is also used to perform clustering using the K-means algorithm, determine the clustering value in the K-means clustering by calculating the average silhouette coefficient method, and obtain the optimal clustering result; The silhouette coefficient SC is as follows: Among them, a represents the average distance between the energy digital object and other energy digital objects in the cluster, and b represents the minimum value of the average distance between the energy digital object and other clusters; The range of SC is [-1,1], and the clustering result with the largest SC value is selected as the optimal clustering result.

10. The data fusion discovery system for energy digital network according to claim 8, characterized in that: The clustering module is also used to calculate the cosine similarity of the semantic feature vectors of any two energy digital objects according to the following formula: : Among them, x and y are the semantic feature vectors of two energy digital objects, ||x||, ||y|| represents the modulus of x and y vectors; If the value of cosine similarity is in the interval [0.8, 1], the two energy digital objects are considered similar; Count the energy digital objects with similarities and combine them to form a combination set of similar energy digital objects : in, It represents the combination of energy digital objects with similarity, and t is the number of combinations.

11. The data fusion discovery system for energy digital network according to claim 8, characterized in that: The fusion discovery module is further used to: Merge and update the metadata information of each energy digital object in the combination. Data sources are integrated by building association tables, aligning data, and eliminating data ambiguity. Generate a new ID, Construct new energy digital objects based on the fused metadata information, data sources and identifiers. Register new energy digital objects into the energy digital network to realize energy data fusion discovery.

12. A computer device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement a data fusion discovery method for energy digital networks as described in any one of claims 1 to 7 when executing the computer program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for data fusion discovery for energy digital networks according to any one of claims 1 to 7 is implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of a data fusion discovery method for energy digital networks described in any one of claims 1 to 7 are implemented.

Images