A data analysis method and system based on data grading

Abstract

The present application relates to a kind of data analysis method and system based on data grading, belong to data mining and analysis technical field, solve the problem that semantic relationship is not complete and associated data availability is not high in existing data analysis. Including the event data of power system is obtained, including image and its text content;According to text feature extractor, each word and text feature code in text content are extracted;Using image feature extractor, each target and image feature code in image are extracted;Based on knowledge graph embedding, the embedding vector corresponding to the word and target existing in knowledge graph is obtained;According to the similarity between embedding vector, relationship feature vector is constructed;According to relationship feature vector, text feature code and image feature code, obtain multi-modal feature vector, input into grading identification module, obtain the level of each event data;According to CLIP model and clustering algorithm, the association between event data of the same level is analyzed. Accurate data analysis is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data mining and analysis, and particularly relates to a data analysis method and system based on data classification. BACKGROUND

[0002] The digital transformation of enterprises cannot be achieved without high-quality data management and analysis. Currently, new technologies such as machine learning and artificial intelligence are increasingly being applied in data management and analysis. These new technologies can help enterprises better understand and apply data, improving data quality and management efficiency. Through machine learning and data mining, important information and patterns in data can be quickly identified and extracted, making it easier to handle complex data management tasks. Automated data management and analysis tools can help enterprises quickly identify and utilize key information in data to make better business decisions and uncover potential opportunities.

[0003] Currently, most data asset inventory work in the industry is done manually using Excel tools for recording. However, a large number of devices are used during the operation of a converter station, and more control systems are used for management in the converter station, making the data volume of power enterprises extremely large. On the other hand, the operation of the converter station generates a variety of structured and unstructured data, as well as a large amount of associated data such as video data, energy data, and weather data. These factors make the characteristics of power data different from other data, with characteristics such as multiple data types and large capacity. If data inventory is done manually, it is not only time-consuming and labor-intensive, but also has the possibility of misjudgment and omission for associated data of different structures, such as abnormal alarms collected by monitoring videos and historical faults recorded in text.

[0004] In addition, the prior art does not introduce external information, which cannot accurately extract the characteristics between data, resulting in incomplete semantic relationships, and does not pay attention to the importance of data. Analyzing data containing different levels affects data analysis and query speed, and establishes many weakly related relationships, making the data association generate a lot of information that is not useful, affecting user use. SUMMARY

[0005] In view of the above analysis, the embodiments of the present application aim to provide a data analysis method and system based on data classification to solve the problems of incomplete semantic relationships and low availability of associated data in existing data analysis.

[0006] In one aspect, the embodiments of the present application provide a data analysis method based on data classification, comprising the following steps:

[0007] Obtain event data of a power system, each event data including an image and its text content;

[0008] According to the text feature extractor, extract each word and text feature code in the text content; use the image feature extractor to extract each target and image feature code in the image;

[0009] Based on the knowledge graph embedding, the words and targets existing in the knowledge graph are put into the text node set and the image node set respectively, and the embedding vectors of each node are obtained; according to the similarity between the embedding vectors of each node, the relationship feature vector of each node is constructed;

[0010] According to the relationship feature vector, the text feature code and the image feature code, the multi-modal feature vector of each event data is obtained, which is input into the hierarchical recognition module to obtain the level of each event data; according to the CLIP model and the clustering algorithm, the association relationship between the event data of the same level is analyzed.

[0011] Based on the further improvement of the above method, the relationship feature vector is a four-dimensional vector composed of four groups of relationship values; the four groups of relationship values are the sum of the similarity of each group in the four groups divided according to the positive and negative values of the similarity and the set to which the compared nodes belong.

[0012] Based on the further improvement of the above method, the method further comprises: setting the relationship feature vector of the word and target not existing in the knowledge graph as a four-dimensional vector composed of zero.

[0013] Based on the further improvement of the above method, according to the relationship feature vector, the text feature code and the image feature code, the multi-modal feature vector of each event data is obtained, which includes:

[0014] The relationship feature vectors of each node in the text node set are spliced at the head of the text feature code; the relationship feature vectors of each node in the image node set are spliced at the head of the image feature code; the spliced two kinds of feature vectors are respectively passed through two linear layers to output feature representations of the same dimension; the two kinds of feature representations of the same event data are connected and combined to obtain the multi-modal feature vector.

[0015] Based on the further improvement of the above method, after the hierarchical recognition module receives the multi-modal feature vector of each event data, it is first projected through a linear layer and then obtained through a softmax layer to obtain the level of each event data; the level includes core, important and general.

[0016] Based on the further improvement of the above method, according to the CLIP model and the clustering algorithm, the association relationship between the event data of the same level is analyzed, which includes:

[0017] The image and text content of the event data of the same level are input into the pre-trained CLIP model to obtain the fusion feature vector input into the softmax layer;

[0018] The fusion feature vectors are classified by using a clustering algorithm, and a strong correlation relationship is established between event data of the same class according to a classification result.

[0019] Further improvement based on the above method, the text feature encoding and the image feature encoding have the same length, which is obtained by respectively passing the output of the last hidden layer in the text feature extractor and the image feature extractor through two linear layers, regularization processing and an activation function in turn.

[0020] Further improvement based on the above method, the knowledge graph embedding is obtained by using a TransE model to obtain embedding vectors of each entity in a Freebase knowledge graph dataset.

[0021] Further improvement based on the above method, the text feature extractor uses a pre-trained Bert model, and the image feature extractor uses a pre-trained Vision Transformer model.

[0022] On the other hand, an embodiment of the present application provides a data analysis system based on data classification, comprising:

[0023] An event data acquisition module is configured to acquire event data of a power system, each piece of event data including an image and text content thereof;

[0024] An event feature extraction module is configured to extract each word and text feature encoding in the text content by using a text feature extractor, and extract each target and image feature encoding in the image by using an image feature extractor;

[0025] A joint relationship extraction module is configured to put each word and target existing in a knowledge graph into a text node set and an image node set respectively based on knowledge graph embedding, and obtain embedding vectors of each node; and construct a relationship feature vector of each node according to the similarity between the embedding vectors of each node.

[0026] A data classification analysis module is configured to obtain a multi-modal feature vector of each piece of event data according to the relationship feature vector, the text feature encoding and the image feature encoding, and input the multi-modal feature vector into a classification recognition module to obtain a level of each piece of event data; and analyze a correlation relationship between event data of the same level according to a CLIP model and a clustering algorithm.

[0027] Compared with the prior art, the present application can at least realize one of the following beneficial effects: reasonably introducing external information, supplementing the semantic relationship of images and texts in event data according to the similarity of embedding vectors between external information, realizing comprehensive consideration of event data characteristics, and improving the accuracy of event data classification of the power system; according to the level of event data, the specified level data or the same level data is selectively associated and analyzed, the data processing scale is reduced, and the loss of system performance caused by data analysis is reduced; through clustering analysis of the fusion feature vector, a strong correlation relationship of the same level data is established, and the availability of the data is improved.

[0028] In the present application, the above technical solutions can be combined with each other to realize more preferred combination solutions. Other features and advantages of the present application will be described in the subsequent specification, and some advantages will become apparent from the specification, or will be understood by implementing the present application. The purposes and other advantages of the present application can be realized and obtained from the contents specifically pointed out in the specification and the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0029] The accompanying drawings are included to provide a further understanding of the present application, and are incorporated herein and constitute a part of the application. The same reference numerals in the drawings refer to the same elements throughout.

[0030] Figure 1 A data analysis method based on data classification in embodiment 1 of the present application. DETAILED DESCRIPTION

[0031] The preferred embodiments of the present application will be specifically described below in conjunction with the drawings, wherein the drawings constitute a part of the present application, and are used to explain the principles of the embodiments of the present application, and are not used to limit the scope of the present application.

[0032] Embodiment 1

[0033] In one specific embodiment of the present application, a data analysis method based on data classification is disclosed, as shown in Figure 1 The method comprises the following steps:

[0034] S11, acquiring event data of a power system, each event data comprising an image and text content thereof.

[0035] It should be noted that each key workstation in the converter station of the power company, such as an operator workstation, an oil chromatography workstation, and an electric energy metering workstation, is installed with a monitoring camera, and image data of each key event is collected through a video monitoring system, and is preprocessed to a uniform size, and at least one image data corresponds to one text content.

[0036] Exemplarily, the key event is: oil chromatographic workstation gas chromatograph maintenance, the collected image data is the maintenance scene image, and there are 3 pieces of text content corresponding to 3 pieces of text content: abnormal reading of the first gas chromatograph of the oil chromatographic workstation in June 2018; on June 3, a person from outside registered to enter the oil chromatographic workstation; and a staff member maintained the gas chromatograph.

[0037] S12, according to the text feature extractor, each word and text feature code in the text content is extracted; and by using the image feature extractor, each target and image feature code in the image is extracted.

[0038] It should be noted that the text feature extractor uses a pre-trained Bert model, and the input vector of the Bert model is composed of Token embedding, segment embedding and position embedding. Among them, Token embedding is obtained by tokenizing each word {w0, w1,..., w t} in each piece of text content using WordPiece, and at the same time, a classification mark [CLS] is added in front of the sentence sequence to obtain the final representation of the text feature, which is used for subsequent classification, and a mark [SEP] is added between sentences to distinguish the context of the sentences; segment embedding is used to represent the sentence corresponding to each word; position embedding is to label each word to determine the specific position of the word in the sequence.

[0039] The Bert model uses multi-layer Transformer as the main framework to learn rich context information. In this step, the output of the last hidden layer of the text feature extractor is taken, and then sequentially passes through two linear layers, regularization processing and activation function to obtain token encoding as text feature code E text , which is represented as follows:

[0040] E text =Tr{f1{w0,w1,...,w t}}=(x1,x2,...,x n ) Formula (1)

[0041] Wherein, f1 represents text preprocessing operation, Tr represents multi-layer Transformer processing, and n represents the length of the text feature code.

[0042] It should be noted that the image feature extractor uses a pre-trained VisionTransformer (ViT) model, which first divides the input image into equal image blocks {v0, v1,..., v M} sequence, and then projects them to a low-dimensional space through a linear projection layer to convert these image blocks into a vector sequence, and then inputs these vector sequences into the Transformer encoder for processing. Exemplarily, the size of the input image is 224x224, and it is divided into 16x16 image blocks.

[0043] Similarly, the output of the last hidden layer of the image feature extractor is taken and sequentially passed through two linear layers, regularization and activation functions to obtain the encoded feature vector as the image feature encoding representation E image :

[0044] E image = Tr{f2{v0,v1,...,v m}} = (y1,y2,...,y n ) Formula (2)

[0045] Wherein, f2 represents the pre-processing operation of the image block, Tr represents the multi-layer Transformer processing, and the length n of the image feature encoding is the same as the length of the text feature encoding.

[0046] Further, according to the output result of the image feature extractor, each target in the image is identified. Exemplarily, in the image of the gas chromatograph maintenance of the oil chromatographic workstation, the targets include the gas chromatograph and the person.

[0047] S13, based on the knowledge graph embedding, the segmented words and targets existing in the knowledge graph are respectively put into the text node set and the image node set, and the embedding vectors of each node are obtained; according to the similarity between the embedding vectors of each node, the relationship feature vector of each node is constructed.

[0048] It should be noted that the knowledge graph (knowledge graph, KG) is a technology that uses a graph model to describe knowledge and model the relationship between things. Knowledge graph embedding (knowledge graph embedding, KGE) is a widely used knowledge representation method. The main idea is to embed entities and relationships in the knowledge graph into a continuous vector space, so as to quantify their similarity through a scoring function. Freebase is a large and multi-domain knowledge graph dataset, containing more than 250 million entities, collecting a lot of entity attribute and relationship attribute information.

[0049] In this step, the knowledge graph embedding is to obtain the embedding vectors of each entity in the Freebase knowledge graph dataset using the TransE model. The TransE model is also called a translation model, which is a method that can represent entities and relationships on a large-scale multi-relation dataset in the form of embedding.

[0050] It should be noted that the text feature extractor and image feature extractor in step S12 use each word in each text content and the name of each object in the image as the entity name to identify whether it exists in the knowledge graph. The word and object existing in the knowledge graph are placed into the text node and image node sets, respectively, and the embedding vector of each node is obtained. In other words, the word and object in the node set can be aligned and matched with the knowledge graph.

[0051] According to the cosine similarity, for a node v i , get it and other nodes {v1,v2,...,v k The similarity between {d i1 ,d i2 ,...,d ik}.

[0052] Considering that the similarity range is between [-1,1], positive and negative values ​​have different meanings, the node set includes the text node set V T and image node set V I , so {d i1 ,d i2 ,...,d ik According to the positive and negative values ​​of similarity and the sets to which the compared nodes belong, the nodes are divided into four groups. The sum of the similarities of each group is used to calculate four relationship values, forming a four-dimensional vector as the node v i The relationship feature vector is denoted as R i =(r i0 ,r i1 ,r i2 ,r i3 ), the specific calculation formula is as follows:

[0053]

[0054] It should be noted that the relationship feature vectors between the word segmentation and the target that do not exist in the knowledge graph are set to a four-dimensional vector R consisting of zeros. z =(0,0,0,0).

[0055] Through the above steps, each m word segment in each text content and each g target in each image corresponds to a relational feature vector.

[0056] Compared with the existing technology, this embodiment introduces external information with the help of a large, multi-domain knowledge graph, supplements the semantic relationship between images and text in event data based on the similarity of the embedding vectors between external information, realizes the comprehensive consideration of event data characteristics, and improves the accuracy of data analysis.

[0057] S14, according to the relationship feature vector, the text feature encoding and the image feature encoding, obtaining the multi-modal feature vector of each event data, inputting into the hierarchical recognition module to obtain the level of each event data; according to the CLIP model and the clustering algorithm, the correlation between the event data of the same level is analyzed.

[0058] It should be noted that the embodiment formulates the event data hierarchical classification standard by referring to the business standards of the national power industry and professional field, and the use of data of each part of the power grid. For historical event data, experts label the level according to the standard, and the level includes three levels: core, important and general. The hierarchical recognition module includes a linear layer and a softmax classification layer, and is trained using a cross-entropy loss function to obtain a trained hierarchical recognition module.

[0059] In the actual task, according to step S11, the newly added or to be recognized event data is obtained, and according to steps S12 and S13, the feature encoding, image feature encoding and relationship feature vector are obtained. The multi-modal feature vector of each event data is obtained through the following steps, including:

[0060] The relationship feature vectors of the nodes in the text node set are spliced at the head of the text feature encoding; the relationship feature vectors of the nodes in the image node set are spliced at the head of the image feature encoding; the spliced two kinds of feature vectors are respectively input into two linear layers to output feature representations of the same dimension; and the two kinds of feature representations of the same event data are connected and combined to obtain a multi-modal feature vector.

[0061] The multi-modal feature vector of each event data is input into the hierarchical recognition module, projected through a linear layer first, and then obtained through a softmax layer to obtain the level of each event data.

[0062] It should be noted that the hierarchical recognition module outputs the confidence degree of each level for each event data, which is represented by a confidence index, and the value range is [0, 1]. The level with the maximum confidence degree is taken as the recognition result. If the maximum confidence index is less than 0.9, manual secondary review is performed, so as to realize accurate data grading and data classification.

[0063] The embodiment determines the level of the event data, selectively analyzes and processes the data of the specified level, eliminates the processing of data of different levels, reduces the scale of data processing, reduces the loss of system performance caused by data analysis, and has a positive significance for maintaining the high performance of the system.

[0064] Preferably, the event data of the core or important level is subjected to strong correlation analysis, the data range of the analysis is reduced, and the information required by the user is quickly searched; in addition, the data of the same level is analyzed, and the user with lower authority can only query the event data of the ordinary level and the associated data.

[0065] In this step, the CLIP (Contrastive Language-Image Pre-Training) model is used to learn the semantic relationship between images and texts in the same level of event data, and a fusion feature vector is obtained.

[0066] In the sample set for training the CLIP model, the image dataset uses ImageNet and COCO datasets, and the text dataset uses Wikipedia and BookCorpus. The CLIP model includes two main components: a convolutional neural network (CNN) for processing images and a Transformer model for processing texts. The CLIP model is trained using a contrastive learning method. The purpose of contrastive learning is to learn how to map similar image and text samples into a similar embedding space, while mapping dissimilar samples into a distant embedding space. The CLIP model uses different contrastive loss functions to achieve this goal, such as the NT-Xent contrastive loss function.

[0067] In this step, the CLIP model and clustering algorithm are used to analyze the correlation between event data of the same level, including:

[0068] The images and text content of the same level of event data are input into the pre-trained CLIP model to obtain the fusion feature vector of the input softmax layer. The clustering algorithm is used to classify the fusion feature vector, and based on the classification result, a strong correlation is established between event data of the same class.

[0069] It should be noted that the softmax layer is used to obtain the final output result of the CLIP model, but this step does not require the output result, but rather the feature vector used to identify the output result, in order to perform clustering analysis. After establishing a strong correlation between event data of the same level, when a user searches for a core event data, the core event data with a strong correlation is automatically output, improving the usability of the data. In addition, when constructing the data directory of the power system, relevant data can also be stored or correlated based on the analysis results, helping enterprises quickly identify and utilize key information in the data, in order to better make business decisions and explore potential opportunities.

[0070] Compared with the prior art, the data analysis method based on data grading provided by the embodiment reasonably introduces external information, supplements the semantic relationship of images and texts in event data according to the similarity of embedding vectors between the external information, realizes comprehensive consideration of the characteristics of event data, and improves the accuracy of event data grading of the power system; according to the level of event data, important or core data is selectively selected for correlation analysis, the data processing scale is reduced, and the loss of system performance caused by data analysis is reduced; by clustering analysis of the fusion feature vector, a strong correlation is established, and the availability of data is improved.

[0071] Embodiment 2

[0072] Another embodiment of the application discloses a data analysis system based on data grading, so as to realize the data analysis method based on data grading in the embodiment 1. The specific implementation of each module is referred to the corresponding description in the embodiment 1. The system comprises:

[0073] An event data acquisition module is configured to acquire event data of a power system, each event data comprising an image and text content thereof;

[0074] An event feature extraction module is configured to extract each word and text feature code in the text content by using a text feature extractor; and extract each target and image feature code in the image by using an image feature extractor;

[0075] A joint relationship extraction module is configured to place each word and target existing in a knowledge graph into a text node set and an image node set respectively based on knowledge graph embedding, and acquire embedding vectors of each node; and construct a relationship feature vector of each node according to the similarity between the embedding vectors of each node;

[0076] A data grading analysis module is configured to obtain a multi-modal feature vector of each event data according to the relationship feature vector, the text feature code and the image feature code, and input the multi-modal feature vector into a grading identification module to obtain a level of each event data; and analyze the correlation relationship between event data of a core level by using a CLIP model and a clustering algorithm.

[0077] Since the embodiment is related to the aforementioned data analysis method based on data grading, the two can be mutually referred to, and thus repeated description is not given here. Since the system embodiment and the method embodiment have the same principles, the system embodiment also has the corresponding technical effects of the method embodiment.

[0078] Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiment methods can be completed by instructing the relevant hardware by a computer program, and the program can be stored in a computer readable storage medium. The computer readable storage medium is a disk, an optical disk, a read-only memory, a random access memory, etc.

[0079] The above description is merely preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A data analysis method based on data classification, characterized in that: The following steps are involved: Acquire event data of the power system, where each event data includes an image and its text content; Using the text feature extractor, extract each word and text feature code from the text content; using the image feature extractor, extract each target and image feature code from the image; Based on knowledge graph embedding, the word segments and targets existing in the knowledge graph are placed into a text node set and an image node set, respectively, and the embedding vector of each node is obtained. Based on the similarity between the embedding vectors of each node, a relationship feature vector of each node is constructed. The relationship feature vector is a four-dimensional vector composed of four groups of relationship values. The relationship values ​​of the four groups are the sum of the similarities of the four groups divided according to the positive and negative values ​​of the similarity and the sets to which the compared nodes belong. The relationship feature vectors of the word segments and targets not existing in the knowledge graph are set to a four-dimensional vector composed of zeros. According to the relationship feature vector, text feature code and image feature code, a multimodal feature vector of each event data is obtained, and the vector is input into a hierarchical recognition module to obtain the level of each event data; according to the CLIP model and clustering algorithm, the correlation relationship between event data of the same level is analyzed; the multimodal feature vector of each event data is obtained according to the relationship feature vector, text feature code and image feature code, including: splicing the relationship feature vector of each node in the text node set into the text feature code header; splicing the relationship feature vector of each node in the image node set into the image feature code header; passing the two spliced ​​feature vectors through two linear layers respectively, and outputting feature representations of the same dimension; and connecting and combining the two feature representations of the same event data to obtain a multimodal feature vector.

2. The data analysis method based on data classification according to claim 1, characterized in that: After receiving the multimodal feature vectors of each event data, the hierarchical recognition module first projects the data through a linear layer and then passes it through a softmax layer to obtain the level of each event data; the levels include core, important and general.

3. The data analysis method based on data classification according to claim 1 or 2, characterized in that: The CLIP model and clustering algorithm are used to analyze the correlation between event data at the same level, including: Input the image and text content of event data at the same level into the pre-trained CLIP model to obtain the fused feature vector of the input softmax layer; The clustering algorithm is used to classify the fused feature vectors, and based on the classification results, a strong correlation is established for event data of the same category.

4. The data analysis method based on data classification according to claim 1, characterized in that: The text feature code and the image feature code have the same length and are obtained by respectively passing the output of the last hidden layer in the text feature extractor and the image feature extractor through two linear layers, regularization processing and activation function in sequence.

5. The data analysis method based on data classification according to claim 1, characterized in that: The knowledge graph embedding is to obtain the embedding vector of each entity in the Freebase knowledge graph dataset using the TransE model.

6. The data analysis method based on data classification according to claim 1, characterized in that: The text feature extractor adopts a pre-trained Bert model, and the image feature extractor adopts a pre-trained Vision Transformer model.

7. A data analysis system based on data classification, characterized in that: include: An event data acquisition module is used to acquire event data of the power system. Each event data includes an image and its text content. The event feature extraction module is used to extract each word segmentation and text feature code in the text content based on the text feature extractor; and to extract each target and image feature code in the image using the image feature extractor; The joint relationship extraction module is used to embed the word segments and targets in the knowledge graph into a text node set and an image node set, respectively, and obtain the embedding vector of each node; construct a relationship feature vector for each node based on the similarity between the embedding vectors of each node; the relationship feature vector is a four-dimensional vector composed of four groups of relationship values; the relationship values ​​of the four groups are the sum of the similarities of the four groups divided according to the positive and negative values ​​of the similarity and the sets to which the compared nodes belong; the relationship feature vectors of the word segments and targets that do not exist in the knowledge graph are set to four-dimensional vectors composed of zeros; The data hierarchical analysis module is used to obtain a multimodal feature vector of each event data based on the relationship feature vector, text feature code and image feature code, and pass it into the hierarchical identification module to obtain the level of each event data; according to the CLIP model and clustering algorithm, the correlation relationship between event data of the same level is analyzed; the multimodal feature vector of each event data is obtained based on the relationship feature vector, text feature code and image feature code, including: splicing the relationship feature vector of each node in the text node set into the text feature code header; splicing the relationship feature vector of each node in the image node set into the image feature code header; passing the two spliced ​​feature vectors through two linear layers respectively, and outputting feature representations of the same dimension; and connecting and combining the two feature representations of the same event data to obtain a multimodal feature vector.

Images