Semantic knowledge base recognition method based on heuristic features

Abstract

The invention provides a semantic knowledge base recognition method based on heuristic features, which belongs to the technical field of intelligent communication, and comprises the following steps: a base station has n versions of semantic knowledge bases, the semantic knowledge bases comprise corresponding receiving end knowledge bases and sending end knowledge bases, the base station respectively calculates a feature vector set corresponding to the receiving end knowledge base and a feature vector set corresponding to the sending end knowledge base, and when a user needs to perform downlink semantic communication with the base station, the feature vector of the receiving end knowledge base is calculated and sent to the base station in a traditional communication mode; and the base station calculates Euclidean distances between the received feature vectors and the feature vectors, selects a receiving end knowledge base corresponding to the feature vector with the minimum Euclidean distance, and matches a sending end knowledge base corresponding to the receiving end knowledge base with a user receiving end knowledge base to complete semantic knowledge base recognition. According to the invention, accurate identification and matching of the semantic knowledge base can be realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent communication, and particularly relates to a semantic knowledge base recognition method based on heuristic features. BACKGROUND

[0002] As one of the core technologies of future communication, semantic communication can break through the capacity limit of traditional communication and improve the anti-noise ability by extracting the core semantics of information, thereby significantly reducing bandwidth consumption, enhancing transmission reliability, and providing support for efficient interconnection of massive intelligent terminals.

[0003] The existing semantic communication system mainly adopts an end-to-end architecture, which may cause the problem of mismatch between the semantic knowledge bases of the receiving end and the sending end, leading to the deterioration of the performance of semantic communication. As of now, the related research on semantic knowledge base recognition and matching is still relatively lacking, and therefore it is urgent to realize the accurate recognition and matching of semantic knowledge bases based on the universality and nesting relationship of knowledge structures. SUMMARY

[0004] The purpose of the application is to provide a semantic knowledge base recognition method based on heuristic features, which can realize the accurate recognition and matching of semantic knowledge bases.

[0005] The application is implemented through the following technical solutions:

[0006] A semantic knowledge base recognition method based on heuristic features comprises the following steps:

[0007] Step S1, the base station has n versions of semantic knowledge bases, the semantic knowledge bases include corresponding receiving end knowledge bases and sending end knowledge bases, the base station respectively calculates a feature vector set corresponding to the receiving end knowledge base and a feature vector set corresponding to the sending end knowledge base wherein, represents the feature vector of the nth base station receiving end knowledge base, which is related to the embedding vector of the semantic decoding model, represents the feature vector of the nth base station sending end knowledge base, which is related to the embedding vector of the semantic encoding model;

[0008] Step S2, when a user needs to perform downlink semantic communication with the base station, the feature vector of the receiving end knowledge base of the user is calculated is sent to the base station using a traditional communication method;

[0009] Step S3, the base station calculates the received feature vector respectively with ​The Euclidean distance of each feature vector is calculated, the feature vector corresponding to the minimum Euclidean distance is selected, the base station knowledge base corresponding to the feature vector is matched with the user terminal knowledge base, and semantic knowledge base recognition is completed, wherein m1 is an integer greater than or equal to n.

[0010] Further, in the step S1, the corresponding receiver knowledge base and the transmitter knowledge base are obtained by training the semantic decoding model and the semantic encoding model based on a certain data set for a certain number of rounds.

[0011] Further, the semantic encoding model and the semantic decoding model of the base station and the user are both transformer architecture.

[0012] Further, in the step S1, the feature vector of the nth base station receiver knowledge base is represented as Wherein, i k is the index of the kth element in the word table of the transformer architecture, N is the total number of elements in the word table, e k is the embedding vector corresponding to the kth element in the word table, and ε(·) is the embedding layer of the transformer architecture.

[0013] Further, the updating method of the semantic knowledge base includes nested updating and non-nested updating, the nested updating is that when the training data set is updated, the trained model is trained by using the updated training data set, and the non-nested updating is that when the training data set is updated, a random initial model is trained by using the updated training data set.

[0014] Further, after completing the semantic knowledge base recognition, the base station encodes and transmits the downlink data based on the recognized transmitter knowledge base, and the user decodes the encoded data based on the receiver knowledge base equipped by the user, and then restores the original data.

[0015] The application also realizes the following technical solutions:

[0016] A semantic knowledge base recognition method based on heuristic features, comprising the following steps:

[0017] Step S1, the base station has n versions of semantic knowledge base, the semantic knowledge base includes corresponding receiver knowledge base and transmitter knowledge base, the base station calculates the feature vector set corresponding to the receiver knowledge base And the feature vector set corresponding to the transmitter knowledge base Wherein, The feature vector of the nth base station receiver knowledge base is represented as The feature vector of the nth base station transmitter knowledge base is represented as

[0018] Step S2, when the user numbered alpha needs to communicate with the base station in uplink semantics, the characteristic vector of the sending knowledge base is calculated is sent to the base station using the traditional communication mode ;

[0019] Step S3, the base station calculates the Euclidean distance between the received characteristic vector and each characteristic vector in , selects the characteristic vector with the minimum Euclidean distance, and matches the sending knowledge base corresponding to the characteristic vector with the user sending knowledge base, thereby completing semantic knowledge base recognition. , and the minimum integer m2 is not less than n.

[0020] Further, after completing semantic knowledge base recognition, the user encodes and transmits uplink data based on the equipped sending knowledge base, and the base station decodes based on the identified receiving knowledge base after receiving the encoded data, thereby recovering the original data.

[0021] The present application has the following beneficial effects:

[0022] 1. The base station of the present application calculates the characteristic vector set corresponding to the receiving knowledge base and the characteristic vector set corresponding to the sending knowledge base When the user needs to communicate with the base station in downlink semantics, the characteristic vector of the receiving knowledge base is calculated is sent to the base station using the traditional communication mode , the base station calculates the Euclidean distance between the received characteristic vector and each characteristic vector in , selects the characteristic vector with the minimum Euclidean distance, and matches the sending knowledge base corresponding to the characteristic vector with the user receiving knowledge base, thereby realizing precise recognition and matching of semantic knowledge base based on the universal and nested relationship of knowledge structure, and greatly compressing the data amount required for semantic knowledge base recognition. BRIEF DESCRIPTION OF DRAWINGS

[0023] The present application will be further described in detail below with reference to the accompanying drawings.

[0024] Figure 1 The flowchart of the first embodiment of the present application.

[0025] Figure 2 The broadcast scenario of the base station to multiple users.

[0026] Figure 3 The nested updating process of the semantic knowledge base.

[0027] Figure 4 The non-nested updating process of the semantic knowledge base.​

[0028] Figure 5 The image shows a simulation diagram of the BLEU score performance test of the present invention and the comparative method. Detailed Implementation

[0029] Example 1:

[0030] like Figure 1 As shown, the semantic knowledge base recognition method based on heuristic features includes the following steps:

[0031] Step S1: The base station has n versions of the semantic knowledge base, which includes the corresponding receiving-end knowledge base and the sending-end knowledge base. The base station calculates the feature vector set corresponding to the receiving-end knowledge base respectively. The set of feature vectors corresponding to the originating knowledge base in, This represents the feature vector of the knowledge base of the nth base station receiver, which is related to the embedding vector of the semantic decoding model. This represents the feature vector of the knowledge base originating from the nth base station, which is related to the embedding vector of the semantic coding model;

[0032] The corresponding receiving and sending knowledge bases are obtained by the semantic decoding model and the semantic encoding model, respectively, based on a certain dataset and trained for a certain number of rounds. Semantic knowledge base matching means that the training process of the semantic codec is completely consistent, that is, the dataset and training rounds are the same and the randomness during training is the same (e.g., fixed random number seed); semantic knowledge base mismatch means that the training process of the semantic codec is not completely consistent.

[0033] like Figure 2 As shown, when a base station wants to perform downlink semantic communication with a user, it first needs to extract representation parameters based on the user's knowledge base and send them to the base station. At this time, the base station can identify the model that can be matched with it in the stored model set based on the parameter, which is the identification of the semantic knowledge base.

[0034] Semantic knowledge bases can be categorized into complete and incomplete scenarios: a complete semantic knowledge base scenario is defined as one where the sending and receiving ends have mutually matching models and can use these models to achieve communication; otherwise, it is considered an incomplete semantic knowledge base scenario. Incompleteness of a semantic knowledge base will inevitably lead to semantic knowledge base mismatches. For example... Figure 1 As shown, a base station performs downlink broadcast semantic communication to multiple users. Each user has a different model version, but because the base station uses a broadcast transmission method, it can only use one transmission model. This will lead to a mismatch between the knowledge base of the base station and some users. In this case, although the transmitting and receiving ends have matching models, they cannot use the models to achieve effective communication. Therefore, this scenario is also called a semantic knowledge base incomplete scenario.

[0035] The updating mode of the semantic knowledge base includes nested updating and non-nested updating. The nested updating is that when the training data set is updated, the trained model is trained by using the updated training data set. The non-nested updating is that when the training data set is updated, a randomly initialized model is trained by using the updated training data set. The specific nested updating process is as shown in Figure 3 The data set A is used for initial training, at this time, the model based on random initialization is trained, and the model A is obtained after training. When the data set is updated (that is, the data set B is supplemented), the model is retrained, but at this time, the model based on the previous trained model A is no longer trained based on the random initialization. The specific non-nested updating process is as shown in Figure 4 When the data set B is supplemented and the model is retrained, the model based on random initialization is still trained, so this updating mode is called non-nested updating. In theory, when the training round tends to infinity and the learning rate is relatively large, it can be approximately considered that the two updating modes are the same. In actual situation, the nested updating is only equivalent to the local fine-tuning based on the original model. In the case of non-nested updating, the communication performance of the knowledge base when the knowledge base is not matched will be worse than that of the nested updating, because the similarity between the two models is lower.

[0036] The transmission model used in this embodiment is a DeepSC model (Deep learning enabled Semantic Communication, semantic communication based on deep learning). Based on the model, the semantic encoding model and the semantic decoding model of the base station and the user are both transformer architectures. In the architecture, the sentence to be sent is first converted into an index vector by means of a word table Wherein L is the number of words in the sentence to be sent, and then s is input into the embedding layer ε(·) to be mapped into an embedding vector Wherein each word is mapped into a vector with a dimension of E. This is a key step in the semantic communication process, which embodies the semantic understanding and representation process of the model for the sentence. The distance between the embedding matrices can represent the similarity between the knowledge bases to some extent. However, the order of magnitude of the floating point elements in the embedding matrix is usually millions, so the amount of data to be transmitted needs to be compressed.

[0037] In the present application, a feature vector is extracted from the embedding layer as a label of the whole model based on the heuristic rule of semantic updating. Generally, the more similar the label is, the more similar the model will be, and the performance of the two models used as communication transceivers will be better. Specifically, the feature vector of the nth base station receiver knowledge base is represented as Wherein, i kis the index of the kth element in the word table of the transformer architecture, N is the total number of elements in the word table, e k is the embedding vector corresponding to the kth element of the word table, and ε(·) is the embedding layer of the transformer architecture of the base station receiver knowledge base. The feature vectors of the base station transmitter knowledge base, the user receiver knowledge base, and the user transmitter knowledge base are calculated using the above formula.

[0038] Step S2, the user end only has one receiver knowledge base, and the user needs to communicate with the base station in downlink semantics. The feature vector of the receiver knowledge base is calculated is sent to the base station using traditional communication methods.

[0039] Step S3, the base station calculates the received feature vector respectively, and selects the feature vector with the smallest Euclidean distance, and matches the transmitter knowledge base corresponding to the receiver knowledge base with the user receiver knowledge base, to complete semantic knowledge base recognition.

[0040] Step S4, after completing semantic knowledge base recognition, the base station encodes and transmits the downlink data based on the identified transmitter knowledge base, and the user receives the encoded data and decodes it based on the receiver knowledge base equipped, and then restores the original data, where m1 is an integer not less than n.

[0041] Embodiment Two:

[0042] The difference between this embodiment and embodiment one is that this embodiment is used for uplink semantic communication to the base station, which specifically includes the following steps:

[0043] The semantic knowledge base recognition method based on heuristic features includes the following steps:

[0044] Step S1, the base station has n versions of semantic knowledge base, and the semantic knowledge base includes corresponding receiver knowledge base and transmitter knowledge base. The base station calculates the feature vector set corresponding to the receiver knowledge base and the feature vector set corresponding to the transmitter knowledge base wherein represents the feature vector of the nth base station receiver knowledge base, which is related to the embedding vector of the semantic decoding model, represents the feature vector of the nth base station transmitter knowledge base, which is related to the embedding vector of the semantic encoding model.

[0045] Step S2, for only one transmitter knowledge base, the user with number α needs to communicate with the base station in uplink semantics. The feature vector of the transmitter knowledge base is calculated is sent to the base station using traditional communication methods.​​ transmitted to the base station;

[0046] Step S3, the base station calculates the received feature vector respectively, and the Euclidean distance of each feature vector in the corresponding to the minimum Euclidean distance, and matches the corresponding sender-receiver knowledge base with the user sender knowledge base, completing semantic knowledge base recognition;

[0047] Step S4, the user encodes and transmits the uplink data based on the equipped sender knowledge base, and the base station decodes based on the identified receiver knowledge base after receiving the encoded data, and then restores the original data, wherein m2 is an integer not less than n.

[0048] As Figure 5 The BLEU score performance test simulation diagram of the present application and the comparative method (Huffman+LDPC+4QAM). The simulation conditions are: the sentences with the number of words between 4 and 30 in the English data set of the European Parliament are used as the data set for training and testing, and the total number of sentences is 73472. The parameters of the DeepSC model are shown in Table 1, wherein the number of neurons in the output layer is 22234, i.e. the total number of words in the word table obtained by data preprocessing. During training, the AWGN channel is uniformly selected between 5 and 10 dB, the number of training rounds is set to 80, the learning rate is 10 -4 , the batch size is 128, the optimizer uses the Adam optimizer, the weight decay is set to 5x10 -4 , and the betas are set to (0.9, 0.98). The performance test index uses the Bilingual Evaluation Understudy (BLEU) score, and the default size of the calculation unit is 1. The computer equipment parameters used in the simulation are Intel(R) Xeon(R) Silver 4110 CPU@2.10GHz and NVIDIA GeForce RTX 2080Ti.

[0049] Table 1

[0050]

[0051] Consider the downlink semantic communication scenario using nested update. First, the knowledge bases M1~M12 are trained, wherein the knowledge base M1 is trained based on the first 15000 sentences of the European Parliament English dataset for 80 rounds, and each subsequent knowledge base is accumulated and trained for 80 rounds in units of 2500 sentences. The test set is the 45001~50000 sentences of the European Parliament English dataset. It is assumed that the base station only saves the receiving knowledge bases M3, M6, M9, and M12, and the user's receiving knowledge base is one of M1~M12. The user calculates the feature vector of the corresponding receiving knowledge base and sends it to the base station, which will further calculate the Euclidean distance based on As shown in Table 2, the Euclidean distance between the feature vectors corresponding to M1~M12 and the feature vectors corresponding to the base station receiving knowledge bases is shown.

[0052] Table 2

[0053]

[0054] Taking the user's receiving knowledge base M2 as an example, it can be seen from Table 2 that the receiving knowledge base corresponding to the feature vector with the smallest Euclidean distance in the base station is M3. Then, the corresponding BLEU score performance test is performed, and the test results are shown in Figure 5 . Among them, the colored solid line is the BLEU performance when using the identified receiving knowledge base for communication, the colored dashed line is the BLEU performance when using other non-identified receiving knowledge bases for communication, and the black solid line is the comparative scheme (i.e., Huffman+LDPC+4QAM, wherein the LDPC code length K=6 and the code rate R=3 / 8). In the present application, each word after encoding will form 8 symbols; while in the comparative scheme, each word after encoding will form an average of about 12.68 symbols. At this time, the frequency band utilization rate (unit: Baud / Hz) of the present application is more than 1.5 times that of the comparative scheme (i.e., the number of symbols required for transmission can be compressed to less than 2 / 3 of the traditional scheme under the condition of achieving the same communication effect), and the identified semantic decoding model and semantic encoding model pair can achieve not less than the BLEU performance of the traditional scheme when the backup is less than 11.8dB and 14.2dB, respectively.

[0055] As can be seen from the results of Table 2, as the difference between the versions of the knowledge bases increases, the Euclidean distance between the feature vectors also shows an increasing trend. Combined with the results of Figure 5 , it can be seen that the present application can effectively identify the model pair with relatively better performance in most cases.

[0056] The present application can greatly compress the amount of data required for identification, for example, the feature vector can be compressed by V=22234 times compared to directly transmitting the embedding matrix (V is the total number of words in the word table).

[0057] The above merely provides the preferred embodiment of the present application, and therefore cannot limit the scope of the application. Any equivalent changes and modifications made in accordance with the patent scope and the content of the specification should still fall within the scope of the present application.

Claims

1. A semantic knowledge base recognition method based on heuristic features, characterized in that: Includes the following steps: Step S1: The base station has n versions of the semantic knowledge base, which includes the corresponding receiving-end knowledge base and the sending-end knowledge base. The base station calculates the feature vector set corresponding to the receiving-end knowledge base respectively. The set of feature vectors corresponding to the originating knowledge base in, This represents the feature vector of the knowledge base of the nth base station receiver, which is related to the embedding vector of the semantic decoding model. This represents the feature vector of the knowledge base originating from the nth base station, which is related to the embedding vector of the semantic coding model; Step S2: When a user needs to conduct downlink semantic communication with the base station, calculate the feature vector of its receiving knowledge base. Using traditional communication methods Send to the base station; Step S3: The base station calculates the received feature vector. respectively with The Euclidean distance of each feature vector is used to select the receiving knowledge base corresponding to the feature vector with the smallest Euclidean distance. The sending knowledge base corresponding to this receiving knowledge base is then matched with the user's receiving knowledge base to complete the semantic knowledge base recognition.

2. The semantic knowledge base recognition method based on heuristic features according to claim 1, characterized in that: In step S1, the corresponding receiving knowledge base and sending knowledge base are obtained by semantic decoding model and semantic encoding model respectively through a certain number of training rounds based on a certain dataset.

3. The semantic knowledge base recognition method based on heuristic features according to claim 2, characterized in that: The semantic encoding and semantic decoding models for both the base station and the user are based on the transformer architecture.

4. A semantic knowledge base recognition method based on heuristic features according to claim 1, 2, or 3, characterized in that: In step S1, the feature vector of the nth base station receiving end knowledge base is represented as follows: Among them, i k Let be the index of the k-th element in the vocabulary of the transformer architecture, where N is the total number of elements in the vocabulary, and e is the index of the k-th element. k Let ε be the embedding vector corresponding to the k-th element of the vocabulary, and ε(·) be the embedding layer of the transformer architecture.

5. A semantic knowledge base recognition method based on heuristic features according to claim 1, 2, or 3, characterized in that: The semantic knowledge base is updated in two ways: nested updates and non-nested updates. Nested updates involve using the updated training dataset to train the already trained model when the training dataset is updated. Non-nested updates involve using the updated training dataset to train a randomly initialized model when the training dataset is updated.

6. A semantic knowledge base recognition method based on heuristic features according to claim 1, 2, or 3, characterized in that: After completing the semantic knowledge base recognition, the base station encodes and transmits downlink data based on the recognized sending knowledge base. After receiving the encoded data, the user decodes it based on its equipped receiving knowledge base to recover the original data.

7. A semantic knowledge base recognition method based on heuristic features, characterized in that: Includes the following steps: Step S1: The base station has n versions of the semantic knowledge base, which includes the corresponding receiving-end knowledge base and the sending-end knowledge base. The base station calculates the feature vector set corresponding to the receiving-end knowledge base respectively. The set of feature vectors corresponding to the originating knowledge base in, This represents the feature vector of the knowledge base of the nth base station receiver, which is related to the embedding vector of the semantic decoding model. This represents the feature vector of the knowledge base originating from the nth base station, which is related to the embedding vector of the semantic coding model; Step S2: When user number α needs to perform uplink semantic communication with the base station, calculate the feature vector of its originating knowledge base. Using traditional communication methods Send to the base station; Step S3: The base station calculates the received feature vector. respectively with The Euclidean distance of each feature vector is used to select the originating knowledge base corresponding to the feature vector with the smallest Euclidean distance, and the receiving knowledge base corresponding to the originating knowledge base is matched with the user's originating knowledge base to complete the semantic knowledge base recognition. Here, m2 is an integer not less than n.

8. The semantic knowledge base recognition method based on heuristic features according to claim 7, characterized in that: After semantic knowledge base recognition is completed, the user encodes and transmits the uplink data based on its equipped sending-end knowledge base. After receiving the encoded data, the base station decodes it based on its recognized receiving-end knowledge base to recover the original data.

Images