Voice semantic deep learning communication system
By introducing a speech semantic deep learning communication system into the wireless communication system, using deep semantic optimization technology, the problem of redundancy and susceptibility to interference in traditional wireless communication is solved, efficient and stable speech semantic communication is achieved, and the machine is improved The efficiency and applicability of semantic interactions.
Patent Information
- Application Number
- CN202410577920.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-10
- Publication Date
- 2024-11-29
AI Technical Summary
Traditional wireless communication systems ignore semantic information in data transmission, resulting in redundancy and susceptibility to interference, and deep learning methods are difficult to control and interpret flexibly, and are difficult to integrate with existing communication frameworks.
Design a speech semantic deep learning communication system, including a knowledge base module and a semantic communication module, through semantic encoding, channel encoding, modulation and detection error correction submodule, use deep semantic optimization to achieve the extraction and transmission of key semantic information, and reduce semantic errors , and adapt to different pronunciation and semantic scenarios.
It significantly improves the quality and robustness of wireless voice communication, improves the efficiency and applicability of machine semantic interaction, and reduces the difficulty and cost of R&D.
Smart Images

Figure FT_1 
Figure FT_2
Abstract
Description
Technical Field
[0001] The present invention relates to the field of semantic communication technology, and in particular to a method and system for realizing more efficient and robust wireless semantic communication based on the redesign of key modules of a traditional wireless communication system. Background Art
[0002] Traditional digital wireless communication mainly relies on bit streams for data transmission, ignoring the semantic information of the transmitted content. The data in this method is seriously redundant and is easily affected by various interferences during the communication process, resulting in errors. In contrast, semantic communication captures semantic features by establishing a knowledge base, which can identify potential links between semantically related content representations, thereby achieving redundancy removal and error correction. For example, synonyms and near-synonyms can represent the same semantics, thereby effectively saving bandwidth resources.
[0003] Recently, end-to-end semantic communication using deep learning methods has been widely studied. This fully learned and complex method is difficult to flexibly control and interpret, and is completely dependent on fixed conditions during the training phase. Once the environment changes, its functionality is limited. In addition, the end-to-end architecture is difficult to utilize existing communication framework standards and is difficult to integrate with existing systems.
[0004] Some works have proposed to improve the performance of semantic systems by expanding and updating the knowledge base, but they have not fully considered the potential advantages provided by traditional key communication modules such as modulation and error detection. These module designs have been tested in practice for a long time. How to reshape them in a targeted manner will be an effective way to improve the efficiency of semantic communication. Directly using and upgrading existing modules will help reduce the difficulty and cost of research and development. Summary of the invention
[0005] The technical problem solved by the present invention is: to address the problem of noise interference in wireless voice communication, a speech semantic deep learning communication system is proposed to achieve high-quality wireless voice communication.
[0006] The technical solution of the present invention is: a speech semantic deep learning communication system, which proposes a machine semantic communication system and method based on deep semantic optimization. The system includes a knowledge base module and a semantic communication module. The knowledge base module establishes a machine semantic knowledge base containing semantic representations such as machine tasks and objects. The semantic communication module includes a semantic coding submodule, a channel coding submodule, a modulation submodule, and a detection and error correction submodule. The semantic coding submodule performs semantic segmentation on the source content according to the knowledge base, and selects and encodes the source content according to the importance of each semantic part. This can effectively extract and transmit key semantic information and reduce transmission resource consumption. The channel coding submodule introduces knowledge base information into the encoding logic, identifies semantically important information, and takes priority protection measures. This can minimize semantic damage in the event of transmission errors. The modulation submodule is redesigned based on the knowledge base to minimize error propagation at the semantic level rather than the traditional bit level. The detection and error correction submodule combines the knowledge base information to realize detection and error correction based on semantic constraints and improve robust performance.
[0007] In addition, the system can also consider the importance of semantics to allocate resources such as channels and power, and realize semantic priority transmission. Through the above technical means, the invention realizes efficient and stable machine semantic communication based on deep semantic optimization. It can significantly improve the application effect of machine semantic interaction.
[0008] Furthermore, the knowledge base module models and abstracts the speech semantic structure and features through corpus learning and deep learning. The knowledge base submodule will continuously enrich and update the knowledge base content to adapt to different speech semantic scenarios. It provides shared semantic references for other submodules and supports semantic-based communication functions.
[0009] Furthermore, the semantic encoding submodule performs semantic segmentation on the speech signal based on the knowledge base and extracts key semantic features. It selectively encodes semantic information according to the importance of different semantic parts. The semantic encoding submodule uses a deep learning neural network model to learn semantic expression capabilities from speech signals and achieve efficient semantic feature extraction and encoding.
[0010] Furthermore, the channel coding submodule introduces knowledge base information into the traditional channel coding logic to identify the semantic features of the key parts. It customizes the allocation of channel resources with higher protection priority for these important semantic features, enhancing their robustness in transmission.
[0011] Furthermore, the modulation submodule redesigns and optimizes the traditional modulation method based on the knowledge base. It aims to minimize the propagation of semantic errors through learnable mapping relationships, thereby improving the semantic quality of speech.
[0012] Furthermore, the detection and error correction submodule uses the semantic information constraints provided by the knowledge base to propose a new signal detection method. It can also effectively locate and correct transmission errors at the semantic level based on semantic features.
[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention proposes a machine semantic communication system based on a deep semantic knowledge base. The system achieves semantic priority transmission by systematically optimizing each key module, using semantic segmentation, priority protection, minimum error design and other means, thereby significantly improving the efficiency and quality performance of semantic interaction between machines. Compared with the prior art, deep optimization is achieved, effectively improving the efficiency, stability and applicability of machine semantic communication. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to make the technical solution of the present invention clear, some drawings will be selected below to illustrate the present invention. The following drawings are only some examples of the present invention; users can obtain other similar drawings based on the drawings. Among them: Figure 1 This is the speech semantics deep learning communication flow chart of the present invention. Figure 2 An exemplary diagram of source channel coding and semantics-based modulation according to the present invention. DETAILED DESCRIPTION
[0015] The following will be a more in-depth and complete description of the technical solutions in the implementation cases of the present invention. The implementation cases are part of the implementation cases. Users can obtain other implementation cases based on the present invention without creative work. The "other implementation cases" all belong to the scope of protection of the present invention.
[0016] like Figure 1 As shown, a speech semantic deep learning communication system comprises the following steps: (1) Establish a knowledge base model that includes speech and semantic features; (2) Using the knowledge base to semantically encode speech signals; (3) Redesign the modulation strategy based on the knowledge base; (4) Implement channel coding based on semantic features; (5) Decoding and restoring the source speech through semantic constraints;
[0017] According to one embodiment of the present invention, a large amount of speech corpus is used for training, a CNN model containing multiple convolutional layers and pooling layers is constructed, and speech semantic representation is extracted by self-learning.
[0018] Based on the above implementation, the method for establishing a knowledge base model containing speech semantic features in the above step (1) is as follows: collect a large amount of speech corpus, each speech sample contains a speech signal and corresponding phonemes and vocabulary annotations. Design a deep learning model structure suitable for speech processing tasks, using a convolutional neural network. The network contains multiple layers of one-dimensional convolutional layers to extract local features, and a pooling layer to obtain global spatial features. The SGD optimization algorithm is used for model training. The model uses a large number of speech annotation samples to learn the expression ability of speech features, realize automatic extraction of speech semantic representation, and thus build a database containing speech semantic knowledge.
[0019] In the above step (2), the method of using the knowledge base to semantically encode speech signals is established as follows: perform multimodal fusion preprocessing on the input source data to extract cross-modal semantic dependencies. According to the task requirements and matching with the knowledge base, give adaptive importance tags to semantic units. Design a customized extraction network with a built-in attention mechanism to achieve adaptive semantic segmentation and filtering. Encode the filtered semantic fragments with fixed length or variable length. Consider the compactness constraints within synonymous semantics. Use hash mapping structures for implicit encoding and transmission of low-importance semantics. Meet the requirements of real-time load adjustment. Take the example learning method to continuously optimize the extraction-encoding network to adapt to new semantic types and application scenarios.
[0020] In the above step (3), the method of redesigning the modulation strategy based on the knowledge base is as follows: by constructing a semantic knowledge base and a channel model, an end-to-end learning method is used to jointly train the semantic coding network and the modulation mapping network. The semantic coding network uses the knowledge base to extract and encode the semantic features of the source data, and the modulation mapping network learns to map the semantic code to the modulation constsllation point based on the semantic features and channel state information to minimize the semantic error derived from the modulation transmission. In this way, connections are established between different semantic features and between semantic features and modulation signals, so that the modulation method can be used to transmit semantic information while taking into account communication performance. This method can better meet the needs of semantic information transmission.
[0021] In the above step (4), the method for implementing channel coding based on semantic features is as follows: first, different protection levels are marked for each semantic feature according to the semantic importance. Then, it uses a deep learning model to identify the semantic features in the source data and encodes them. On this basis, the encoded semantic features are imported into the LDPC encoding process as information words of the LDPC codeword. According to the protection level of the semantic features, they are mapped to different positions of the codeword, thereby implementing differentiated protection. In this way, important semantic features will be preferentially mapped to highly protected positions of the LDPC codeword, realizing selective protection for different semantic features. In addition, when decoding the channel, the introduction of semantic knowledge also helps to improve the error correction capability. The use of this channel coding method based on semantic features can further improve the reliability and spectrum efficiency of semantic transmission.
[0022] In the above step (5), the method of restoring the source speech by decoding through semantic constraints is as follows: the receiver first uses the pre-built speech knowledge base to perform semantic detection and identify the semantic features in the received speech signal. Then, it imports the semantic detection results into the LDPC decoding process, and combines the detected semantic constraints to guide the LDPC decoder to find estimated code words that meet the semantic requirements. For example, if the detected semantics is "hello", the decoder will give priority to possible code words that contain the "hello" semantic fragment. Compared with LDPC decoding that only considers traditional error statistics, this method introduces top-level semantic constraints while ensuring the traditional decoding logic. In this way, it is possible to find a more realistic reconstruction of the source signal, thereby recovering the semantic information loss caused by channel errors to a certain extent, and ultimately improving the ability to restore the quality of the source speech.
[0023] In summary, the present invention proposes a speech semantic deep learning communication system. A speech semantic knowledge base model using a convolutional neural network structure is constructed to realize automatic extraction of speech semantic features. A semantic coding method based on differential coding of a semantic knowledge base is proposed to compress and encode semantic related information in the source speech. A learning modulation mapping relationship is designed to combine the minimum semantic error and transmission error to improve the modulation strategy. Semantic knowledge is introduced into LDPC coding to identify important semantics and achieve priority protection. A semantic constraint decoding method is used to reconstruct the source speech and improve the decoding quality. The method optimizes key modules such as coding, modulation, and encoding through deep learning, and different modules work together. Compared with the end-to-end method, its modular design is easier to apply in practice. In addition, compared with the traditional method: the transmission efficiency and compression effect are significantly improved; the transmission quality of the semantic error is minimized is guaranteed; the system has strong adaptability and can cope with changes in complex wireless environments.
[0024] The above is only one embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A speech semantic deep learning communication system, characterized in that: include: Coding module and decoding module; the coding module uses a new semantic coding method to encode the source data to obtain an encrypted data stream; the decoding module uses a new semantic measurement method to decode and restore the received data stream; The system utilizes a trained updated deep learning model to achieve better capabilities in removing ambient noise and transmission noise.
2. The system according to claim 1, characterized in that The encoding module redesigns the traditional hybrid automatic repeat request method to encode the source data; the decoding module decodes the received data stream according to the redesigned hybrid automatic repeat request method.
3. The system according to claim 1, characterized in that The encoding module redesigns the traditional modulation method to encode the source data; the decoding module decodes the received data stream according to the redesigned modulation method.
4. The system according to claim 2, characterized in that The speech semantic deep learning communication system can automatically adjust the deep learning model according to environmental changes, so as to better cope with the changing wireless environment.
5. The system according to claim 1, characterized in that The deep learning model adopts a convolutional neural network structure to extract semantic features of source data.
6. The system according to claim 5, characterized in that: The convolutional neural network includes multiple layers of convolutional layers and pooling layers to extract increasingly rich semantic features.
7. The system according to claim 1, characterized in that: The deep learning model adopts a recursive neural network structure to model the semantic dependency relationship between source data.
Citation Information
Cited By
Self-adaptive voice semantic communication method based on hierarchical time sequence importance
CN121619609A
