A text semantic communication method based on joint knowledge graph learning

Abstract

The application relates to a text semantic communication method based on joint knowledge graph learning, which comprises the following steps: completing the alignment of entities, relations and entity attributes in a knowledge graph library between user terminals through a secure hash algorithm; using a generative adversarial network to unify the embedding representation of different user terminal knowledge graph libraries; a user terminal compresses a to-be-transmitted text into semantic triples and matches a similar semantic triple set from a local knowledge graph library; the similar semantic triples are encoded and sent to a receiving terminal; the receiving terminal decodes the received message to obtain a similar semantic triple set; the similarity between the similar semantic triple and the local knowledge graph library is calculated by using a string similarity algorithm, the most similar triple is selected as the recovered semantic triple; and a fine-tuned semantic recovery module is used to convert the recovered semantic triple into a recovered text and recover the to-be-transmitted text, so that the safety and privacy of semantic transmission can be improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of wireless communication, and particularly relates to a text semantic communication method based on joint knowledge graph learning. BACKGROUND

[0002] Benefiting from the progress of microelectronics and artificial intelligence technology, deep learning (DL), natural language processing (NLP) technology recently emerged and played an important role in the transformation of traditional communication technology to the future. As an intelligent communication scheme, semantic communication is proposed, which, unlike traditional communication, emphasizes the successful delivery of message semantics rather than the accurate bit stream transmission in traditional communication. Semantic communication can further compress data while preserving meaning by extracting the meaning of data, filtering out useless, irrelevant and unimportant information, and processing data within the semantic domain. In addition, semantic communication is expected to be robust to poor channel environments, i.e. low signal to noise ratio (SNR) regions, which is very suitable for applications requiring high reliability. These factors prompt us to develop intelligent semantic communication systems by considering the semantics behind digital bits to improve the accuracy and efficiency of communication.

[0003] Semantic communication based on knowledge graph is a recently emerging communication paradigm, and knowledge graph is a data structure that stores human knowledge in a graph structure composed of entities and relationships. Using knowledge graph as a tool to improve communication efficiency and accuracy improves the explainability of semantic communication under deep learning architecture. The key role of knowledge graph in the communication system is to provide semantic representation of information, making semantic encoding not only efficient and accurate but also robust to communication errors. However, existing semantic communication models cannot fully utilize data from different users and different knowledge domains as background knowledge base while protecting the privacy of exchanged data. Each user maintains a private knowledge graph library, and as the public demand for privacy protection increases, most users are reluctant to share their knowledge graph library. Under the premise of protecting user privacy, a more loosely coupled and principled way to share their knowledge graph library and benefit multiple parties should be considered. SUMMARY

[0004] In order to solve the problems in the background art and ensure the privacy of users, the application provides a text semantic communication method based on joint knowledge graph learning, comprising:

[0005] S1: Each user end respectively uses knowledge graph embedding technology to convert its own knowledge graph library into low-dimensional vector representation, and calculates the secure hash value of entities, relationships and entity attributes in the knowledge graph library through a secure hash algorithm;

[0006] S2: The entities, relationships and entity attributes in the knowledge graph library are aligned through the secure hash values of entities, relationships and entity attributes between user terminals;

[0007] S3: An adversarial generative network is constructed, and the generator and the discriminator are respectively arranged at two different user terminals, and the embedding representations of the knowledge graph libraries of different user terminals are unified through training the adversarial generative network;

[0008] S4: The sending user terminal inputs the text to be transmitted into a semantic extraction model for semantic information recognition, constructs semantic triples according to the recognized semantic information, performs similarity matching on the semantic triples according to the embedding representation of the local knowledge graph library to extract a similar semantic triple set, and performs source coding and channel coding on the extracted similar semantic triples and sends the coded information to the receiving terminal;

[0009] S5: The receiving user terminal receives the message sent by the sending terminal, performs channel decoding and source decoding on the received message to obtain a similar semantic triple set, calculates the similarity between the similar semantic triples and the local knowledge graph library by using a string similarity algorithm, selects the most similar triple as the recovered semantic triple, and converts the recovered semantic triple into a recovered text by using a fine-tuned semantic recovery module.

[0010] The present application has at least the following beneficial effects

[0011] The present application provides a text semantic communication method based on joint knowledge graph learning, aiming at the problem that the traditional semantic communication system cannot fully utilize data from different users and different fields. By using a knowledge extractor as a semantic encoder, the source signal is input into a semantic extraction model for semantic information recognition, and the knowledge triple information is extracted from the input text and compressed into a triple semantic representation.

[0012] In the joint knowledge graph learning stage, in order to improve the quality of a single knowledge graph of multiple cross-domain knowledge graphs in a multi-user system, ensure the privacy of the knowledge graph of each user terminal, and fully utilize the local data of the user terminal, an adversarial generative network is introduced, the generator and the discriminator are located at different sides of a pair of knowledge graphs, the generator is trained to convert the knowledge data at the generator end into a similar form of the knowledge data at the discriminator end, so that the updated knowledge data has a similar embedding vector representation as the local one, and the original data is not exposed.

[0013] By introducing the adversarial mechanism of the generative adversarial network, the generated knowledge is more in line with logical and semantic requirements, and can fully utilize the local data of users to improve the quality of the knowledge graph without infringing on the privacy of user data. Through privacy protection technology, all parties can safely share and integrate data in the knowledge graph to produce a more complete and rich knowledge graph without worrying about the leakage of sensitive data. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 A schematic diagram of a communication system of the present application;

[0015] Figure 2 A schematic diagram of the architecture for embedding representation of unified different user-end knowledge graph libraries using generative adversarial networks in the present application;

[0016] Figure 3 A comparison chart of information transmission bits for different communication systems of the present application;

[0017] Figure 4 A model performance chart of BLEU scores of different transmission models under different SNRs. DETAILED DESCRIPTION

[0018] The embodiments of the present application will be described in detail below with specific reference to specific examples. Those skilled in the art can easily understand other advantages and effects of the present application from the disclosure of the present specification. The present application can also be implemented or applied by different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the diagrams provided in the following examples only illustrate the basic concept of the present application in a schematic manner, and the following examples and features in the examples can be combined with each other without conflict.

[0019] In embodiment 1, the present application provides a text semantic communication method based on joint knowledge graph learning, comprising: training an adversarial generative network joint knowledge graph learning update to obtain a more rich local knowledge graph; constructing and training a semantic extraction model and a semantic recovery module, inputting the trained text semantic communication with the to-be-transmitted text data to obtain the recovered text.

[0020] In embodiment 2, please refer to Figures 1-4 The present application provides a text semantic communication method based on joint knowledge graph learning, comprising:

[0021] S1: Each user-end respectively uses knowledge graph embedding technology to convert its own knowledge graph library into low-dimensional vector representation, and calculates the secure hash value of the entity, relationship and entity attribute in the knowledge graph library through a secure hash algorithm;

[0022] S2: Users align entities, relationships, and entity attributes in the knowledge graph database using secure hash values ​​of entities, relationships, and entity attributes.

[0023] S3: Construct an adversarial generative network system, setting the generator and discriminator on two different user terminals respectively, and unify the embedding representation of knowledge graph databases on different user terminals by training the adversarial generative network;

[0024] S4: The sending user terminal inputs the text to be transmitted into the semantic extraction model for semantic information recognition; and constructs semantic triples based on the recognized semantic information; performs similarity matching on the semantic triples based on the embedding representation of the local knowledge graph library to extract a set of similar semantic triples; performs source coding and channel coding on the extracted similar semantic triples, and sends the encoded information to the receiving end;

[0025] S5: The receiving end receives messages sent by the sending end, performs channel decoding and source decoding on the received messages to obtain a set of similar semantic triples; uses a string similarity algorithm to calculate the similarity between the similar semantic triples and the local knowledge graph database, selects the most similar triples as the restored semantic triples; and uses a fine-tuned semantic restoration module to convert the restored semantic triples into restored text.

[0026] In one embodiment, such as Figure 2 As shown, it is used to realize end-to-end semantic communication between the sender and receiver. It includes: a sender, a channel, and a receiver. The sender mainly includes functions such as text semantic extraction, semantic encoding, and channel encoding; the channel is responsible for information transmission; the receiver mainly includes functions such as channel decoding and text recovery based on a text generation model.

[0027] Specifically, the TransE model is used to represent entities and relations in a knowledge graph. The TransE model is a vector embedding-based knowledge graph representation method that maps entities and relations in a knowledge graph to a low-dimensional vector space, achieving a structured representation of knowledge. In the TransE model, each knowledge graph is represented as a set of facts in the form of triplets, i.e., (h, r, t), where h represents the head entity, r represents the relation, and t represents the tail entity. The TransE model assumes that for each triplet, if h + r ≈ t, then the triplet is true. Therefore, TransE learns the low-dimensional vector representation of each entity and relation by optimizing the loss function, such that the head entity vector h plus the relation vector r is as close as possible to the tail entity vector t. A distance function is used to define the objective function. For a given triplet (h, r, t), the model's loss function is:

[0028] L(h,r,t)=||h+rt||

[0029] By minimizing this loss function, TransE is able to learn embedding representations.

[0030] Specifically, as shown in Figure 1 The sending user end owns its own private knowledge graph library, and in order to share or align entities and relations, the parties need to jointly update knowledge in a secure and privacy-protected manner.In the knowledge graph alignment process, a secure hash method is used, and for each entity in the knowledge graph, a unique hash value is generated using a secure hash function.In the experiment, the SHA-256 algorithm is used.After completing the hash value generation, the two parties compare the hash value lists and find the common hash values, that is, the entities corresponding to these hash values in the respective knowledge graphs are the same, and all matching hash values will be used as aligned entities.

[0031] In order to share and align entities and relations, the parties need to jointly update knowledge in a secure and privacy-protected manner.For this purpose, the application adopts a GAN-based knowledge graph alignment mechanism, which enables the knowledge graphs of the two parties to update and expand knowledge without revealing private data.

[0032] Preferably, the embedding representation of the unified knowledge graph library of different user ends by training the generative adversarial network comprises:

[0033] The knowledge graph in the local knowledge graph library is converted into a low-dimensional vector representation by using the knowledge graph embedding technology;

[0034] The generator receives the low-dimensional vector representation from the local knowledge graph library as input and outputs the fused embedding representation;

[0035] The discriminator receives the embedding representation generated by the generator or the low-dimensional vector representation from the local knowledge graph library as input and outputs a probability value representing the authenticity of the data; wherein the loss function L G of the generator G is:

[0036]

[0037] The loss function L D of the discriminator D is:

[0038]

[0039] Wherein, G() represents the generator, D() represents the discriminator;n represents the number of training samples;c m represents the sample input into the generator;y k represents the sample of the local knowledge graph library;log represents the natural logarithm;

[0040] After the training of the generative adversarial network is completed, the embedding representation generated by the generator is used to update the embedding representation in the local knowledge base of the user end, so as to unify the embedding representations of different user end knowledge graph bases.

[0041] Specifically, given two knowledge graph bases KG(g i , g j ) with aligned entities E i ∩E j and relations R i ∩R j , a generative adversarial network (GAN) is used to unify the embedding representations of the aligned entities and relations. The generator and the discriminator are located on different sides of a pair of knowledge graphs, i.e., on two clients respectively. More specifically, the generator is placed in g i , and the discriminator is placed in g j. The generator converts the embedding of the aligned entities from g i to g j , and the discriminator distinguishes between the synthetic embedding of the generator and the local real embedding in g j . After GAN training, the synthetic embedding can learn features from both KGs, so it can replace the original embedding of E i ∩E j and R i ∩R j in g i and g j as refined and unified embeddings. One side of the generator is responsible for converting its private knowledge graph data into a form that the other side cannot directly distinguish. During the generation process, the generator generates corresponding knowledge fragments based on the aligned entities and their neighborhood relations. The loss function of the generator is defined as:

[0042]

[0043] One side of the discriminator is responsible for determining whether the generated data is real data, i.e., whether the generated data matches the knowledge in its own graph. Through adversarial training with the generator, the discriminator gradually improves its ability to distinguish aligned knowledge. The loss function of the discriminator is defined as:

[0044]

[0045] where G() represents the generator, D() represents the discriminator; n represents the number of training samples; x m represents the sample input to the generator; y k represents the sample of the local knowledge graph base; log represents the natural logarithm.

[0046] Through adversarial training of the generator and the discriminator, the generated data gradually tends to be real data that the discriminator cannot distinguish. The generator side is g iThe adjacent entities and connections of the aligned entities are used to generate embeddings, which are then added to the discriminator's knowledge graph as entities and relationships. This joint update mechanism achieves alignment and knowledge sharing between the two knowledge graphs through continuous iteration of the adversarial generative network.

[0047] Preferably, the step of performing similarity matching of semantic triples based on the embedding representation of the local knowledge graph includes:

[0048] The semantic triples are transformed into low-dimensional vector representations using knowledge graph embedding technology. The Euclidean distance between the low-dimensional vector representation of the semantic triples and the embedded representation of triples in the local knowledge graph library is calculated. A distance threshold ε is set. If the distance is less than the threshold, the triple is added to the set of similar semantic triples.

[0049] Preferably, the knowledge graph embedding technology includes the TransE algorithm, whose loss function is as follows:

[0050] L(h, r, t) = ||h + rt||

[0051] Where h represents the embedding vector of the head entity, r represents the embedding vector of the relation, and t represents the embedding vector of the tail entity; the optimization objective is to minimize the loss function until the loss function converges, thus obtaining a low-dimensional vector representation of the triples.

[0052] Preferably, the step of inputting the text to be transmitted into the semantic extraction model for semantic information recognition includes: analyzing the input text using natural language processing technology, identifying entities in the input text using the Named Entity Recognition (NER) model, predicting the relationships between the identified entities using a relation classification model, and then constructing semantic triples.

[0053] Preferably, the encoded information x from the transmitting end includes:

[0054] x=C(S(m))

[0055] Where S represents the source encoder and C represents the channel encoder.

[0056] Preferably, the message y sent by the sender and received by the receiver includes:

[0057] y = wx + n

[0058] Where x is the encoded information from the transmitting end, n is an independent and identically distributed vector with a circularly symmetric Gaussian distribution, and w ~ N(0, δ). 2 I) indicates that w follows N(0, δ) 2 I) Probability distribution, δ 2 I represents the average noise power of the channel, and I is the identity matrix.

[0059] Preferably, the channel and source decoding of the received information comprises:

[0060]

[0061] wherein, denotes the semantic triple information decoded by the receiving end; S -1 denotes the source decoder; C -1 denotes the channel decoder. In this embodiment, the source encoder adopts Huffman encoding, the channel encoder adopts turbo encoding, and the decoding mode is the corresponding encoding mode.

[0062] In this embodiment, the sending end obtains the text information to be transmitted, and uses a named entity recognition model (a semantic extraction model) to obtain an entity-relation-entity triple set (a semantic triple) according to the text information to be transmitted. The recognized semantic triple information is represented by an embedding vector according to a knowledge representation model (TransE algorithm) of a local knowledge graph, the triple vector is matched with a knowledge graph library vector, and the triple vector in the local database is reserved to obtain semantic expression information conforming to the local database.

[0063] The semantic information is represented by the knowledge graph, which removes the redundancy in the transmitted information, thereby improving the effectiveness of the application. After receiving the transmitted knowledge triple, the receiving end selects the triple with the highest similarity as the semantic information according to the knowledge graph. The text information is recovered using a knowledge graph text generation model (a semantic recovery module), and the application adopts a T5 model pre-trained by a large amount of data to realize the knowledge graph text generation function.

[0064] The research scenario of the application is in semantic communication. In order to simulate the process of updating the semantic communication system model on a single machine, a local data set needs to be allocated to each user end participating in training. The data set used in this experiment is the WebNLG data set. The data set covers the triple and sentence information of airports, artists, astronauts, athletes, buildings, celestial bodies, cities, comic characters, food, transportation methods, monuments, politicians, sports teams, universities, literary works, movies, scientists and music works.

[0065] The triple information in the WebNLG data set is uniformly divided into 10 subsets. Each subset contains triples from different fields in the WebNLG data set, ensuring that each subset covers diverse field knowledge such as people, places, events, and works. Through this division method, the universality and representativeness of the data in each subset are ensured, which is convenient for subsequent cross-domain joint knowledge update.

[0066] For each knowledge graph, we calculate the number of relations, entities and triples, and divide the triples into training set, validation set and test set according to the settings in the TransE model in the ratio of 90:5:5, and conduct triple classification test experiment to train the knowledge graph embedding representation. The default dimension of the embedding vector is set to d = 100, the training round is set to 300, the learning rate is set to 0.01, and the batch size is set to 100. In the triple classification experiment, negative samples are generated by destroying the head or tail entity, and the ratio of negative samples to positive samples is set to 1:1.

[0067] For the 10 client knowledge graphs, joint update is performed according to the full connection joint update strategy, two knowledge graphs are selected for joint update in each round, and the update of all knowledge graphs is performed in sequence, the goal is to ensure that each knowledge graph g i Can be updated with the rest of g j The adversarial generation network training round is set to 300, the batch size is set to 32, the learning rate is set to 0.01, and the momentum is set to 0.9. Among them, the generator receives the embedding vectors of the aligned entities of the two knowledge graphs, performs linear transformation on the input entity embedding vector E, and uses matrix multiplication W x X to transform the embedding space. G = W E Where W is the weight matrix to be learned, and E is the input aligned entity embedding vector. The generator outputs the transformed vector as the input of the discriminator.

[0068] The task of the discriminator is to distinguish between embeddings from real knowledge graphs and embeddings generated by the generator. The discriminator is a multi-layer perceptron that includes three fully connected layers and a normalization layer. After the activation function sigmoid, a binary classification probability value is output, indicating the probability that the input embedding vector is a real embedding, to determine whether the input vector is generated or from real data.

[0069] The receiving end semantic recovery adopts the T5 model pre-trained by a large amount of data to realize the knowledge graph text generation method. Based on the WebNLG dataset, the training dataset and the validation dataset of the text generation model are obtained, the parameters of the triple text generation model are fine-tuned, the learning rate of the model is set to 2e-5, the training batch size is 16, and the model is trained for a total of 40 epochs. Finally, the model outputs the training parameters, and the system uses the training parameters to realize triple recovery sentence information.

[0070] The experiment adopts the T5 model pre-trained by a large amount of data to realize the knowledge graph text generation function. Figure 3 The relationship between the bit length to be sent and the sentence length is shown. Especially for long sentences, the method of the experiment has obtained a very high compression rate in the system. In order to explore the influence of different encoding and decoding methods on performance, such as Figure 4As shown, the relationship between SNR and BLEU score under the AWGN channel is shown, and the polyline marked with a circular symbol represents the semantic communication system model, the polyline marked with a triangular symbol represents the semantic communication system model without joint knowledge graph learning, and the polyline marked with a square symbol represents the communication system combined with the Huffman source coding mode and the Turbo channel coding method. Based on the comparison and analysis of the images, it can be concluded that the proposed model performs better at low SNR and performs worse than the traditional communication system in the high SNR region. This is because the proposed semantic recovery method attempts to recover the same semantics rather than the same sentence structure.

[0071] Based on the comparison and analysis of the images, first, the proposed model performs better at low SNR and performs worse than the traditional communication system in the high SNR region. This is because the proposed semantic recovery method attempts to recover the same semantics rather than the same sentence structure. Second, the semantic communication system model with joint knowledge graph learning improves the recovery quality in semantic recovery by integrating and aligning multiple knowledge graphs to extract entity relationships from a wider data source. In contrast, the semantic communication system model without joint knowledge graph learning has poor text recovery effect due to the lack of some triplets caused by insufficient coverage of entity relationships. The experimental results show that the performance of the jointly updated knowledge graph in the text recovery task has a significant advantage, not only improving the extraction quality of knowledge triplets, but also showing better semantic accuracy in the recovered text.

[0072] Finally, it is pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the purpose and scope of the technical solutions, which should be covered in the scope of the claims of the present application.

Claims

1. A text semantic communication method based on joint knowledge graph learning, characterized in that, include: S1: Each user terminal uses knowledge graph embedding technology to transform its own knowledge graph library into a low-dimensional vector representation, and calculates the secure hash values ​​of entities, relations and entity attributes in the knowledge graph library through a secure hash algorithm; S2: Users align entities, relationships, and entity attributes in the knowledge graph database using secure hash values ​​of entities, relationships, and entity attributes. S3: Construct an adversarial generative network system, setting the generator and discriminator on two different user terminals respectively, and unify the embedding representation of knowledge graph databases on different user terminals by training the adversarial generative network; S4: The sending user terminal inputs the text to be transmitted into the semantic extraction model for semantic information recognition; and constructs semantic triples based on the recognized semantic information; performs similarity matching on the semantic triples based on the embedding representation of the local knowledge graph library to extract a set of similar semantic triples; performs source coding and channel coding on the extracted similar semantic triples, and sends the encoded information to the receiving end; S5: The receiving end receives messages sent by the sending end, performs channel decoding and source decoding on the received messages to obtain a set of similar semantic triples; uses a string similarity algorithm to calculate the similarity between the similar semantic triples and the local knowledge graph database, selects the most similar triples as the restored semantic triples; and uses a fine-tuned semantic restoration module to convert the restored semantic triples into restored text.

2. The text semantic communication method based on joint knowledge graph learning according to claim 1, characterized in that, The method of unifying the embedding representation of different user-side knowledge graph databases through training an adversarial generative network includes: The knowledge graph in the local knowledge graph database is transformed into a low-dimensional vector representation using knowledge graph embedding technology. The generator takes a low-dimensional vector representation from a local knowledge graph library as input and outputs a fused embedded representation. The discriminator receives the embedded representation generated by the generator or the low-dimensional vector representation from the local knowledge graph as input, and outputs a probability value representing the authenticity of the data; where the generator's loss function L G for: The loss function L of the discriminator D for: Where G() represents the generator, D() represents the discriminator; n represents the number of training samples; x m y represents the sample input to the generator; k This represents a sample from the local knowledge graph database; log represents the natural logarithm. After the generative adversarial network is trained, the embedding representations generated by the generator are used to update the embedding representations in the local knowledge base on the user end, so as to unify the embedding representations of different user end knowledge graph bases.

3. The text semantic communication method based on joint knowledge graph learning according to claim 2, characterized in that, The step of performing similarity matching on semantic triples based on the embedding representation of the local knowledge graph includes: The semantic triples are transformed into low-dimensional vector representations using knowledge graph embedding technology. The Euclidean distance between the low-dimensional vector representation of the semantic triples and the embedded representation of triples in the local knowledge graph library is calculated. A distance threshold ε is set. If the distance is less than the threshold, the triple is added to the set of similar semantic triples.

4. The text semantic communication method based on joint knowledge graph learning according to claim 3, characterized in that, The knowledge graph embedding technology includes the TransE algorithm, whose loss function is as follows: L(h,r,t)=‖h+rt‖ Where h represents the embedding vector of the head entity, r represents the embedding vector of the relation, and t represents the embedding vector of the tail entity; the optimization objective is to minimize the loss function until the loss function converges, thus obtaining a low-dimensional vector representation of the triples.

5. A text semantic communication method based on joint knowledge graph learning according to claim 2, characterized in that, The step of inputting the text to be transmitted into the semantic extraction model for semantic information recognition includes: analyzing the input text using natural language processing technology, identifying entities in the input text using the Named Entity Recognition (NER) model, predicting the relationships between the identified entities using a relation classification model, and then constructing semantic triples.

6. The text semantic communication method based on joint knowledge graph learning according to claim 1, characterized in that, The encoded information x from the transmitting end includes: x=C(S(m)) Where S represents the source encoder and C represents the channel encoder.

7. The text semantic communication method based on joint knowledge graph learning according to claim 1, characterized in that, The message y sent by the sender and received by the receiver includes: y = wx + h Where x is the encoded information from the transmitting end, n is an independent and identically distributed vector of a circularly symmetric Gaussian distribution, and w ~ N(0,δ) 2 I) indicates that w follows N(0,δ) 2 I) Probability distribution, δ 2 I represents the average noise power of the channel, and I is the identity matrix.

8. A text semantic communication method based on joint knowledge graph learning according to claim 1, characterized in that, The channel and source decoding of the received information includes: in, This represents the semantic triplet information decoded at the receiving end; S -1 Indicates source decoder; C -1 This indicates the channel decoder.

Images