A knowledge graph enhancement-based semantic communication method, electronic device, and medium

Abstract

This invention provides a semantic communication method based on knowledge graph enhancement. The method is implemented at a sender and a receiver, specifically including: extracting semantic information from a sent sentence using a semantic encoder at the sender and representing it as a first semantic vector; encoding the first semantic vector into a transmission signal using a channel encoder; transmitting the transmission signal through a physical channel to obtain a received signal; upon receiving the received signal, the receiver decodes the received signal using a channel decoder to obtain a second semantic vector; the receiver uses a knowledge extractor to find semantically related triples from a knowledge base that are associated with the second semantic vector and embeds them into a knowledge vector; and the semantic decoder uses the channel to decode the second semantic vector and the knowledge vector to obtain the received sentence.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and semantic communication, and in particular to a semantic communication method, electronic device, and medium based on knowledge graph enhancement. Background Technology

[0002] With the rapid development of deep learning and natural language processing, a new form of communication—semantic communication—has become a research hotspot in the field of communication. Unlike traditional communication, semantic communication emphasizes the accurate transmission of semantics between the sender and receiver, rather than the accuracy at the symbolic level.

[0003] Current research on semantic communication largely focuses on joint source-channel coding schemes based on deep learning. A key assumption of these studies is that the sender and receiver share common prior knowledge. However, this language model-based approach still lacks the ability to understand and reason about knowledge, making it difficult to fully exploit the implicit prior knowledge within complex sentences.

[0004] To enhance a system's ability to understand and reason about knowledge, some studies have proposed incorporating knowledge graphs into semantic communication. Knowledge graphs represent human knowledge as a graph-like structure composed of entities and relations, allowing machine learning models to explicitly process common sense. In a knowledge graph, each piece of knowledge is abstracted as a (entity-relation-entity) triple. Current work has combined knowledge graphs with semantic communication. These studies share a common characteristic: using knowledge triples as semantic carriers, with information semantically encoded into triples and transmitted at the sender. However, the expressive power of knowledge triples is limited, and the knowledge in a knowledge base is often insufficient to represent all the semantic information in a sentence. Using only these triples as semantic carriers for communication inevitably leads to additional semantic loss. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a semantic communication method, electronic device, and medium based on knowledge graph enhancement.

[0006] To achieve the above technical objectives, the technical solution of the present invention is as follows: A first aspect of the present invention provides a semantic communication method based on knowledge graph enhancement, the method being implemented based on a sending end and a receiving end, specifically including:

[0007] Step 1: Extract semantic information from the sent sentence using the semantic encoder at the transmitting end and represent it as a first semantic vector; then encode the first semantic vector into a transmitted signal using the channel encoder.

[0008] Step 2: Transmit the transmitted signal through a physical channel to obtain the received signal;

[0009] Step 3: After receiving the received signal, the receiving end decodes the received signal using a channel decoder to obtain the second semantic vector;

[0010] Step 4: The receiving end uses a knowledge extractor to find triples that are semantically related to the second semantic vector from the knowledge base and embeds them as knowledge vectors.

[0011] Step 5: The semantic decoder uses the channel to decode the second semantic vector and the knowledge vector obtained in step 4 to obtain the received sentence.

[0012] Furthermore, the semantic encoder is a Transformer-based encoder, and the channel encoder is implemented using a fully connected layer.

[0013] Furthermore, the process of transmitting the transmitted signal through a physical channel to obtain the received signal can be represented as follows:

[0014] y = Hx + n

[0015] In the formula, y is the received signal, x is the transmitted signal, H is the channel matrix, and n is Gaussian white noise.

[0016] Furthermore, the channel decoder in step three is implemented using a fully connected layer.

[0017] Furthermore, the receiving end is used to maintain a knowledge base, where each fact is represented as a knowledge triple, which is entity-relation-entity.

[0018] Furthermore, step four specifically involves:

[0019] First, an encoder model consisting of L stacked Transformer encoders is used to encode the second semantic vector obtained from channel decoding, resulting in the embedding representation z of the second semantic vector. (L) ;

[0020] Then, a multi-label classification model is used to calculate the index vector t of the knowledge triples related to the second semantic vector, i.e., t = sigmoid(z (L) W t +b t );in i = 1...n t ;n t W represents the number of triples in the knowledge base. t and b t These are the parameters of the multi-label classifier model; for all semantic relevance thresholds greater than or equal to the user-defined threshold, t... i Find the triple {m} with index i in the knowledge base. i};

[0021] The relevant knowledge triples {m} predicted by the multi-label classification model are used to... i It is embedded into a knowledge vector.

[0022] Furthermore, the semantic relevance threshold is 0.5.

[0023] Furthermore, the semantic decoding process in step five also includes concatenating the second semantic vector and the knowledge vector.

[0024] A second aspect of the present invention provides an electronic device including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described knowledge graph-based semantic communication method.

[0025] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described knowledge graph-based semantic communication method.

[0026] Compared with existing technologies, the advantages of this invention are as follows: This invention proposes a knowledge graph-based semantic communication method that utilizes knowledge graphs to enhance the performance of semantic communication. The extracted knowledge triples can provide additional prior knowledge for the semantic decoder, thereby improving the accuracy and efficiency of semantic decoding. This invention only applies the knowledge graph at the receiving end, thus avoiding additional requirements on the sending end and preventing the introduction of artificial semantic losses. This invention is not limited to the standard Transformer structure and can also be used for various Transformer variants. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of the semantic communication method based on knowledge graph enhancement proposed in an embodiment of the present invention;

[0029] Figure 2 This is a schematic block diagram of a knowledge extractor provided in an embodiment of the present invention;

[0030] Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0032] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0033] like Figure 1 As shown, this invention provides a semantic communication method based on knowledge graph enhancement, implemented using a sender and a receiver. The content transmitted in semantic communication can be abstracted as a sent sentence of length N. Where s i This represents the i-th token in the sentence.

[0034] Step 1: Through the semantic encoder S in the sending end β (·) Extract the sent sentence The semantic information is obtained and represented as a first semantic vector h, where the semantic encoding process is represented as h = S β (s); then, the channel encoder C α (·) Encode the first semantic vector h into a transmitted signal x that is transmitted over the physical channel, where the channel coding process is represented as x = C α (h).

[0035] The semantic encoder is a Transformer-based encoder, and the channel encoder is implemented using a fully connected layer.

[0036] Step 2: Transmit the transmitted signal x through a physical channel to obtain the received signal y. The transmission process is represented as: y = Hx + n, where H is the channel matrix and n is Gaussian white noise.

[0037] Step 3: After receiving the received signal y, the receiving end first uses a channel decoder. The received signal y is decoded to obtain the second semantic vector. The channel decoding process is represented as follows:

[0038] The channel decoder is implemented using a fully connected layer.

[0039] Step 4: The receiving end maintains a knowledge base, where each fact is represented as a knowledge triple, which is entity-relation-entity. The receiving end utilizes a knowledge extractor K. θ (·) Search the knowledge base for the second semantic vector Semantically related triples are embedded as knowledge vector k.

[0040] like Figure 2 As shown, the knowledge extractor in step four is specifically:

[0041] First, the second semantic vector obtained by decoding the channel is obtained using an encoder model consisting of L stacked Transformer encoders. Encode it. Assume z (l-1) It is the output of the (l-1)th encoder layer, where z (0) Equivalent to the second semantic vector The self-attention mechanism of the l-th layer can then be represented as:

[0042]

[0043] in, and d is the projection matrix of the l-th layer. k It refers to the dimension of the model. (l-1) Through residual connections and the calculated Attention(z) result (l-1) The sums are then performed, followed by layer normalization, i.e., a (l) =LayerNorm(Attention(z) (l-1) )+z (l-1) )

[0044] Among them, a (l) For the output, LayerNorm(·) represents the layer normalization operation. The calculated output a (l) By providing a feedforward neural network in and These are the parameters of the feedforward layer in the l-th encoder. Next, we perform residual connections and normalization operations z. (l) =LayerNorm(FFN(a (l) )+a (l) Following the above scheme, L-layer coding yields the embedded representation z of the channel decoding vector. (L) .

[0045] Then, a multi-label classification model is used to calculate the index vector t of the knowledge triples related to vector semantics, i.e., t = sigmoid(z (L) W t +b t ).in i = 1...n t n t W represents the number of triples in the knowledge base. t and b t These are the parameters of a multi-label classifier model. For The triplet m corresponding to index i i It is predicted to be semantically relevant. For all t i ≥0.5, find the triplet {m} corresponding to index i in the knowledge base. i}

[0046] The relevant knowledge triples {m} predicted by the multi-label classification model i} is embedded into a knowledge vector k = f k ({m i In}), where f k (·) indicates the embedding process. Subsequently, the knowledge vector and the decoding vector are concatenated and fed into the semantic decoder.

[0047] The knowledge extractor model can be trained using gradient descent. A weighted binary cross entropy (BCE) is used as the loss function, which can be expressed as:

[0048]

[0049] Among them, t i ∈{0,1} represents the training labels. This is the predicted output of equation (11). i It is the weight of the i-th position, which is related to the hyperparameter w. For t i =0, w i =w; otherwise w i =1-w.

[0050] It is worth noting that the decoders applicable to this knowledge extractor are not limited to the traditional Transformer architecture, but can also be applied to different Transformer variants.

[0051] Step 5: Knowledge-Enhanced Semantic Decoder Decoding the second semantic vector using the channel And extract the knowledge vector k to obtain the received sentence Right now

[0052] Specifically, the semantic decoding process also includes: processing the second semantic vector. It is then concatenated with the extracted knowledge vector k.

[0053] The semantic decoder is a Transformer-based decoder.

[0054] In summary, this invention proposes a knowledge graph-based semantic communication method that leverages knowledge graphs to enhance the performance of semantic communication. The extracted knowledge triples can provide additional prior knowledge for the semantic decoder, thereby improving the accuracy and efficiency of semantic decoding.

[0055] like Figure 3 As shown, this application provides an electronic device including a memory 101 for storing one or more programs and a processor 102. When the one or more programs are executed by the processor 102, they implement the method as described in any of the first aspects above.

[0056] The system also includes a communication interface 103. The memory 101, processor 102, and communication interface 103 are electrically connected directly or indirectly to each other to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used for signaling or data communication with other node devices.

[0057] The memory 101 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0058] Processor 102 can be an integrated circuit chip with signal processing capabilities. This processor 102 can be a general-purpose processor 102, including a central processing unit (CPU) 102 and a network processor 102.

[0059] (Network Processor, NP), etc.; it can also be a Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0060] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can also be implemented in other ways. The method and system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0061] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0062] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program thereon. When executed by processor 102, the computer program implements the methods described in any of the first aspects above. If the functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory 101 (ROM), random access memory 101 (RAM), magnetic disks, or optical disks.

[0063] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A semantic communication method based on knowledge graph enhancement, characterized in that, The method is implemented based on a sender and a receiver, and specifically includes: Step 1: Extract semantic information from the sent sentence using the semantic encoder at the transmitting end and represent it as a first semantic vector; then encode the first semantic vector into a transmitted signal using the channel encoder. Step 2: Transmit the transmitted signal through a physical channel to obtain the received signal; Step 3: After receiving the received signal, the receiving end decodes the received signal using a channel decoder to obtain the second semantic vector; Step 4: The receiving end uses a knowledge extractor to find the triples that are semantically related to the second semantic vector from the knowledge base and embeds them as knowledge vectors. Step four specifically involves: Firstly, the method is adopted by An encoder model consisting of stacked Transformer encoders encodes the second semantic vector obtained from channel decoding, resulting in an embedded representation of the second semantic vector. ; Then, a multi-label classification model is used to calculate the index vector of the knowledge triplet related to the second semantic vector. ,Right now ;in , , ; This indicates the number of triples in the knowledge base. and These are the parameters of the multi-label classifier model; for all semantically relevant thresholds greater than or equal to the user-defined threshold. Find the corresponding index from the knowledge base. triples ; The relevant knowledge triplet predicted by the multi-label classification model Embedded into a knowledge vector; Step 5: The semantic decoder uses the channel to decode the second semantic vector and the knowledge vector obtained in step 4 to obtain the received sentence.

2. The semantic communication method based on knowledge graph enhancement according to claim 1, characterized in that, The semantic encoder is a Transformer-based encoder, and the channel encoder is implemented using a fully connected layer.

3. The semantic communication method based on knowledge graph enhancement according to claim 1, characterized in that, The process of transmitting a signal through a physical channel and receiving a signal is represented as follows: ； In the formula, In order to receive signals, H represents the transmitted signal, H is the channel matrix, and n is Gaussian white noise.

4. The semantic communication method based on knowledge graph enhancement according to claim 1, characterized in that, The channel decoder in step three is implemented using a fully connected layer.

5. The semantic communication method based on knowledge graph enhancement according to claim 1, characterized in that, The receiving end is used to maintain a knowledge base, where each fact is represented as a knowledge triple, which is entity-relation-entity.

6. The semantic communication method based on knowledge graph enhancement according to claim 1, characterized in that, The semantic relevance threshold is 0.

5.

7. The semantic communication method based on knowledge graph enhancement according to claim 1, characterized in that, The semantic decoding process in step five also includes concatenating the second semantic vector and the knowledge vector.

8. An electronic device comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the knowledge graph-based semantic communication method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the knowledge graph-based semantic communication method as described in any one of claims 1-6.

Images