Semantic communication method, electronic device and storage medium

Abstract

The present application provides a semantic communication method, an electronic device and a storage medium, which realize the processing of real-time signals by controlling the extraction range of semantic features of transmission signals, thereby realizing real-time transmission of semantic communication.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a semantic communication method, electronic device and storage medium. Background Art

[0002] Semantic communication is different from traditional communication. It aims to transmit grammatical data and is a new type of communication. In semantic communication scenarios with real-time requirements, such as real-time mobile calls, real-time audio and video conferences, and online Internet of Vehicles, end-to-end semantic communication transmission cannot be applied to semantic communication scenarios with real-time requirements because it cannot process and transmit real-time signals. Summary of the invention

[0003] In view of this, the purpose of the present application is to propose a semantic communication method, an electronic device and a storage medium.

[0004] Based on the above objectives, the present application provides a semantic communication method, including:

[0005] Acquire transmission signal;

[0006] Extracting semantic features within a certain range of the transmission signal to obtain local semantic features;

[0007] The local semantic features are encoded to obtain an encoded signal, and the encoded signal is input into a channel for communication transmission.

[0008] Optionally, the acquiring the transmission signal specifically includes:

[0009] The transmission signal is processed in blocks to obtain a transmission signal having a plurality of signal blocks.

[0010] Optionally, extracting the semantic features within a certain range of the transmission signal to obtain the local semantic features includes:

[0011] The semantic features in the signal block are extracted to obtain the local semantic features.

[0012] Optionally, extracting the semantic features within a certain range of the transmission signal to obtain the local semantic features includes:

[0013] Obtaining an intermediate value according to the transmission signal;

[0014] Based on the intermediate value, the semantic features within a certain range of the transmission signal are extracted to obtain the local semantic features.

[0015] Optionally, extracting the semantic features within a certain range of the transmission signal to obtain the local semantic features includes:

[0016] Get cache semantic features;

[0017] Obtaining an intermediate value according to the cache semantic feature and the transmission signal;

[0018] Based on the intermediate value, the semantic features within a certain range of the transmission signal are extracted to obtain the local semantic features.

[0019] Optionally, extracting the semantic features within a certain range of the transmission signal based on the intermediate value to obtain the local semantic features includes:

[0020] Based on the pre-constructed semantic feature extraction matrix, the attention distribution of the semantic feature is obtained according to the intermediate value;

[0021] Obtaining an attention value of the semantic feature according to the attention distribution of the semantic feature;

[0022] Obtaining a first intermediate feature according to the attention value of the semantic feature;

[0023] According to the first intermediate feature, a second intermediate feature is obtained;

[0024] The local semantic feature is obtained according to the first intermediate feature and the second intermediate feature.

[0025] Optionally, after obtaining the local semantic feature according to the first intermediate feature and the second intermediate feature, the method further includes:

[0026] The local semantic features are cached.

[0027] Optionally, inputting the coded signal into a channel for communication transmission comprises:

[0028] The coded signal is input into a channel so that the channel outputs a coded signal with noise. The coded signal with noise is decoded and semantic features are restored to obtain a received signal.

[0029] Based on the above purpose, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any of the above embodiments is implemented.

[0030] Based on the above purpose, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in any of the above embodiments.

[0031] From the above description, it can be seen that the semantic communication method, electronic device and storage medium provided by the present application realize the processing of real-time signals by controlling the extraction range of semantic features of transmission signals, thereby realizing real-time transmission of semantic communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 A schematic diagram of an exemplary communication system based on the semantic communication method according to an embodiment of the present application is shown.

[0034] Figure 2 A flowchart of an exemplary semantic communication method according to an embodiment of the present application is shown.

[0035] Figure 3 A schematic diagram of an exemplary semantic communication method according to an embodiment of the present application is shown.

[0036] Figure 4 A schematic diagram of an exemplary electronic device according to an embodiment of the present application is shown.

[0037] Figure 5 A diagram showing simulation comparison results of the semantic communication method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0039] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0040] As described in the background technology, end-to-end semantic communication transmission cannot be applied to semantic communication scenarios with real-time requirements because it cannot process and transmit real-time signals. In related technologies, when processing transmission signals, only a complete source data (for example, a sentence or a voice) can be processed, and the processing range of the transmission signal cannot be freely controlled, resulting in poor flexibility in signal processing.

[0041] The present application provides a semantic communication method, electronic device and storage medium. By flexibly controlling the calculation range of signal semantic information, the semantic features of real-time signals can be extracted, transmitted and restored under wireless channel conditions with certain noise and fading, while ensuring the transmission accuracy, reducing communication overhead and reducing the communication processing delay at the transmitting and receiving ends.

[0042] Figure 1 A schematic diagram of an exemplary communication system based on the semantic communication method according to an embodiment of the present application is shown.

[0043] like Figure 1 As shown, in order to realize the communication transmission of real-time speech signals, after obtaining the signal s, the signal s is divided into blocks to obtain a signal m with several signal blocks. The signal m is input into a streaming semantic encoder based on Transformer (a machine learning model) to extract the semantic feature b of the signal. Among them, streaming means real-time speech signal. The semantic encoder controls the activation range of neurons through the designed masked attention mechanism to enable the network to have the ability to extract streaming semantic information. The semantic feature b is input into the channel encoder for channel coding to obtain the signal x. The signal x is input into the wireless channel for transmission to obtain a signal y=hx+n with channel noise, where h is the channel coefficient and n is the channel noise. The signal y is input into the channel decoder for channel decoding to obtain the signal The signal The input is a Transformer-based streaming semantic decoder to recover the semantic features, and the real-time speech signal frames that are continuously recovered are integrated and reconstructed to obtain the recovered real-time speech signal. Among them, the structure of the Transformer-based semantic decoder is basically consistent with the structure of the Transformer-based semantic encoder.

[0044] In the process of training the streaming semantic communication system, it is carried out in an end-to-end manner. The embodiment of the present application is illustrated by taking real-time speech recovery as an example. Training the entire Transformer-based streaming semantic communication system requires the design of an efficient loss function to optimize the network parameters of the entire transceiver so that the network eventually converges. The loss function of the embodiment of the present application is divided into two parts, namely, calculating the original signal s and the reconstructed signal in the time domain. The minimum mean square error (TMSE) of the time domain sampling values, and the calculation of the signal s and the reconstructed signal in the frequency domain The mean absolute error (FMAE) of the short-time Fourier transform is expressed as follows:

[0045]

[0046] Among them, L TMSE is the minimum mean square error, N is the number of time domain sampling points of the speech signal s, s n is the nth time domain sampling value of the speech signal s, To reconstruct the signal The nth time domain sampling value.

[0047]

[0048] Among them, L FMAE is the mean absolute error, T is the number of frames of speech signal s, F is the number of frequency domain sampling points of speech signal s, S(t,f) is the discrete Fourier transform frequency domain value of speech signal s at the fth sampling point in frame t, S is the short-time Fourier transform of signal s, To reconstruct the signal The short-time Fourier transform of The real and imaginary parts of .

[0049] Through the above formula, the total loss function can be obtained as:

[0050]

[0051] Among them, L TFME is the total loss function, θ T represents the parameters of the sender, θ R represents the parameters of the receiving end, and λ is the weighting factor that controls the weights of these two items.

[0052] The parameters of the entire network transceiver are optimized and updated by the stochastic gradient descent method, and the expression is:

[0053]

[0054] Among them, θ is the model parameter, i is the model parameter iteration at the i-th step, η is the learning rate of model training, It is the differential function operator.

[0055] With the above loss function, the embodiment of the present application jointly trains the streaming semantic codec and the channel codec, and uses the stochastic gradient descent method to finally converge the model parameters.

[0056] Figure 2 A flowchart of an exemplary semantic communication method according to an embodiment of the present application is shown. The method may include the following steps.

[0057] In step 201, a transmission signal is obtained. When a voice signal is transmitted, the content of the transmission is usually a complete source data. Therefore, to achieve real-time voice signal transmission, after the transmission signal is obtained, the transmission signal needs to be processed in blocks.

[0058] In some embodiments, the acquiring of the transmission signal may specifically include: performing block processing on the transmission signal to obtain a transmission signal having a plurality of signal blocks. The signal after the block processing has a plurality of signal blocks. The signal having the plurality of signal blocks is then input into a streaming semantic encoder based on a Transformer to extract semantic features.

[0059] In step 203, semantic features within a certain range of the transmission signal are extracted to obtain local semantic features. Signal feature extraction and encoding are performed through a Transformer-based streaming semantic encoder.

[0060] The Transformer-based streaming semantic codec of the embodiment of the present application is mainly composed of three parts: an adaptive mask attention network, a multi-head self-attention network, and a feedforward neural network, and an enhanced memory node module is added for caching previously calculated semantic information; the channel codec is composed of multiple fully connected layers.

[0061] In some embodiments, the extracting of semantic features within a certain range of the transmission signal to obtain local semantic features may further include: extracting semantic features within the signal block to obtain the local semantic features. The semantic features of the signal within the signal block in the signal are extracted to obtain local semantic features. The purpose of dividing the signal into blocks is also to achieve real-time processing of the signal, rather than only processing a whole segment of the signal as in the related art.

[0062] In some embodiments, extracting the semantic features within a certain range of the transmission signal to obtain the local semantic features may further include: obtaining an intermediate value based on the transmission signal; and extracting the semantic features within a certain range of the transmission signal based on the intermediate value to obtain the local semantic features.

[0063] In the embodiment of the present application, the streaming semantic communication system based on the Transformer network mainly realizes the semantic extraction and restoration reconstruction of real-time signals. In order to make the model have a streaming effect, the embodiment of the present application designs an adaptive mask attention network for the streaming semantic codec, which is described in detail below.

[0064] Since the structure of the semantic decoder based on the Transformer network is basically consistent with the structure of the semantic encoder based on the Transformer, the embodiment of the present application only introduces the structure of the semantic encoder. In the streaming semantic encoder based on the Transformer, the speech signal s obtained after preprocessing is added to the position encoding vector to obtain an embedding vector m (for example, signal m), which is then input into a flexible masked attention network. In the signal block of signal m, local semantic features are extracted in the signal block based on the intermediate value. Among them, the intermediate value in the process of extracting local semantic features can be calculated according to the following formula:

[0065] m=s+pos

[0066] Among them, pos is the position coding information of the speech signal.

[0067] Q,K,V=Linear(m,(W Q ,W K ,W V ))

[0068] Among them, W Q ,W K ,W V is the weight of the linear mapping, Q, K, B are the intermediate values ​​used to extract the semantic features of the signal, and Linear means linear mapping of the embedded vector.

[0069] In some embodiments, based on the intermediate value, the semantic features within a certain range of the transmission signal are extracted to obtain the local semantic features, including: based on a pre-constructed semantic feature extraction matrix, according to the intermediate value, obtaining the attention distribution of the semantic features; according to the attention distribution of the semantic features, obtaining the attention value of the semantic features; according to the attention value of the semantic features, obtaining a first intermediate feature; according to the first intermediate feature, obtaining a second intermediate feature; according to the first intermediate feature and the second intermediate feature, obtaining the local semantic feature.

[0070] In the embodiment of the present application, after the intermediate value is calculated, the attention distribution of the adaptive mask attention network and the multi-head attention value are calculated according to the intermediate value and the pre-constructed semantic feature extraction matrix. The attention value of the speech signal in a specific range is calculated, so that the network pays more attention to the speech signal frame containing important semantic information. The calculation process of the adaptive mask attention network is as follows:

[0071]

[0072] Among them, α A (Q, K) is the attention distribution of the signal semantic features, A is the pre-constructed semantic feature extraction matrix (e.g., mask attention matrix), exp is the exponential function operation, i, j, k are the index values ​​of the corresponding parameters, T is the matrix transpose operation, and Q, K are the intermediate values.

[0073] z A (Q,K,V)=α A (Q,K)V

[0074] Among them, z A (Q, K, V) is the attention value of the semantic feature, α A (Q, K) is the attention distribution of the signal semantic features, and V is the median value.

[0075]

[0076] Among them, Multihead(Q,K,V) is the first intermediate feature (for example, multi-head attention value), Concatenate the attention values ​​of multiple semantic features, I = 1, 2, …, n.

[0077] hidden=Activate(Linear(Multihead(Q,K,V)))W H

[0078] Among them, hidden is the second intermediate feature (for example, the hidden feature of the middle layer of the network), Activate is the activation function, and W H is the network matrix parameter learned by the network.

[0079] out=hidden+Linear(Multihead(Q,K,V))

[0080] Among them, out is the local semantic feature of the output.

[0081] In the embodiment of the present application, when constructing the semantic feature extraction matrix, the construction rules can be set as follows:

[0082] 1. Voice frame signals belonging to the same block can access each other.

[0083] 2. For two frame signals of different blocks, if the distance between them is less than the set history window length, the frame signal belonging to the current block can access the frame signal of the history block.

[0084] 3. For two frame signals of different blocks, the frame signal of the current block cannot access the future frame signal.

[0085] Based on the above rules, the constructed semantic feature extraction matrix diagram is as follows: Figure 3 As shown in the figure, the semantic feature extraction matrix is ​​a matrix composed of 0s and 1s. When extracting the key semantic features of real-time speech signals, the matrix can be used to flexibly control the range of the calculated specific attention distribution, thereby enabling the model to have the ability of streaming semantic extraction.

[0086] In some embodiments, after obtaining the local semantic feature according to the first intermediate feature and the second intermediate feature, the method may further include: caching the local semantic feature. In order to meet the low latency requirements in real-time communication scenarios and reduce communication overhead, an enhanced memory node is further used to cache historical semantic features to reduce computational complexity.

[0087] In some embodiments, the extracting of semantic features within a certain range of the transmission signal to obtain local semantic features may further include: acquiring cached semantic features; obtaining an intermediate value based on the cached semantic features and the transmission signal; and extracting semantic features within a certain range of the transmission signal based on the intermediate value to obtain the local semantic features.

[0088] Specifically, the attention function of the Transformer is used to calculate the attention output of the current input signal (for example, the local semantic features extracted at the current moment). The attention output result is cached, recorded as Mn (representing the cached result of the nth block) and saved in the enhanced memory node. When extracting semantic features at the next moment, the semantic features cached in the enhanced memory node are used to extract new semantic features, which reduces communication overhead.

[0089] In the embodiment of the present application, the attention function of the Transformer is used to connect the semantic features of the cached enhanced memory node and the signal input at this moment, and together perform the calculation of the attention feature output value of the input signal at this moment. In this case, when calculating the intermediate values ​​Q, K, and V extracted by the semantic feature, the following formula is used:

[0090] Q=W Q [C n ,F n ,dn ]

[0091] K=W K [M n ,F n ,C n ]

[0092] V=W V [M n ,F n ,C n ]

[0093] Among them, Mn is the cached semantic feature calculated at the last moment of the cache, Cn is the currently input transmission signal, Fn is the future voice signal allowed to be input, and dn is the summary query vector obtained after Cn is pooled.

[0094] After calculating the intermediate value, the local semantic features are extracted through the adaptive mask attention network.

[0095] Specifically, the attention distribution of the adaptive mask attention network and the multi-head attention value are calculated according to the intermediate value and the pre-built semantic feature extraction matrix. The attention value of the speech signal in a specific range is calculated to make the network pay more attention to the speech signal frames containing important semantic information. The calculation process of the adaptive mask attention network is as follows:

[0096]

[0097] Among them, α A (Q, K) is the attention distribution of the signal semantic features, A is the pre-constructed semantic feature extraction matrix (e.g., mask attention matrix), exp is the exponential function operation, i, j, k are the index values ​​of the corresponding parameters, T is the matrix transpose operation, and Q, K are the intermediate values.

[0098] z A (Q,K,V)=α A (Q,K)V

[0099] Among them, z A (Q, K, V) is the attention value of the semantic feature, α A (Q, K) is the attention distribution of the signal semantic features, and V is the median value.

[0100]

[0101] Among them, Multihead(Q,K,V) is the first intermediate feature (for example, multi-head attention value), Concatenate the attention values ​​of multiple semantic features, I = 1, 2, …, n.

[0102] hidden=Activate(Linear(Multihead(Q,K,V)))W H

[0103] Among them, hidden is the second intermediate feature (for example, the hidden feature of the middle layer of the network), Activate is the activation function, and W H is the network matrix parameter learned by the network.

[0104] out=hidden+Linear(Multihead(Q,K,V))

[0105] Among them, out is the local semantic feature of the output.

[0106] The entire Transformer-based streaming encoder flow chart is summarized as follows:

[0107] The preprocessed speech signal is input into the masked attention network based on the Transformer encoder, and the masked attention output value is calculated, and then the layer is normalized. The normalized result is input into the self-attention network for local semantic feature extraction to obtain the self-attention output value, and then the layer is normalized. The layer normalization result is input into the feedforward mapping network and activated by an activation function (for example, ReLU function), and the result is cached in the enhanced memory node. Repeat the previous steps of semantic feature extraction, and obtain the semantic features of the input continuous real-time speech signal after processing by the multi-layer encoder module.

[0108] In step 205, the local semantic features are encoded to obtain a coded signal, and the coded signal is input into a channel for communication transmission. After the extracted local semantic features are input into an encoder for encoding to obtain a coded signal, the coded signal is input into a channel for transmission. The noisy coded signal output by the channel is decoded and the semantic features are restored to obtain a received signal.

[0109] In some embodiments, inputting the encoded signal into a channel for communication transmission may further include: inputting the encoded signal into a channel so that the channel outputs an encoded signal with noise, and the encoded signal with noise is decoded and semantic features are restored to obtain a received signal.

[0110] In the embodiment of the present application, except that the last layer of the decoder is the output layer, which realizes the integration and output of the speech signal sampling values, the calculation process of the streaming semantic decoder based on Transformer is basically the same as that of the encoder. Combining the above two modules, the embodiment of the present application realizes a streaming semantic communication system in a real-time communication scenario, so that the real-time signal can be extracted, transmitted and restored in a streaming semantic information manner, and through the caching mechanism, the communication overhead is further reduced while ensuring the recovery performance, and the end-to-end communication delay is reduced.

[0111] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0112] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0113] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the semantic communication method described in any of the above embodiments is implemented.

[0114] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0115] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0116] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0117] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0118] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0119] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0120] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0121] The electronic device of the above embodiment is used to implement the corresponding semantic communication method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0122] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the semantic communication method described in any of the above embodiments.

[0123] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0124] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the semantic communication method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0125] The technical effects of the embodiments of the present application are as follows.

[0126] The simulation results of the non-streaming semantic communication system (Ful-SemAudio), the streaming semantic communication system with future frames (F-SemAudio), and the streaming semantic communication system without future frames (L-SemAudio) are shown in Table 1. It can be understood that since the Ful-SemAudio model is not used in real-time communication scenarios, the parameters shown in Table 1 do not exist, so the simulation results of the model are not shown in Table 1. Figure 5The simulation comparison results of three communication systems are shown, including: Proposed Ful-SemAudio (offline), Proposed L-SemAudio (online), Proposed F-SemAudio (online), where offline refers to a non-real-time communication system, that is, an offline communication scenario, and online refers to a real-time communication system used in an online communication scenario. STOI is the short-time objective intelligibility, and SNR is the signal-to-noise ratio, in dB. Combined with Table 1 and Figure 5 It can be seen from the simulation comparison result diagram that the semantic communication method provided in the embodiment of the present application can achieve better results.

[0127] Table 1: Simulation comparison table of the semantic communication method provided in the embodiment of the present application

[0128]

[0129] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0130] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0131] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0132] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A semantic communication method, characterized in that: include: Acquire transmission signal; Extracting semantic features within a certain range of the transmission signal to obtain local semantic features; Encoding the local semantic features to obtain a coded signal, and inputting the coded signal into a channel for communication transmission; The acquiring of the transmission signal specifically includes: Processing the transmission signal in blocks to obtain a transmission signal having a plurality of signal blocks; The step of extracting the semantic features within a certain range of the transmission signal to obtain the local semantic features further includes: A Transformer-based streaming semantic encoder is used to extract semantic features within the signal block to obtain local semantic features; wherein the Transformer-based streaming semantic encoder includes an adaptive mask attention network, a multi-head self-attention network, a feedforward neural network and an enhanced memory node module, and the enhanced memory node module is used to cache semantic information previously calculated; Wherein, the adaptive masked attention network is used to calculate the attention value of the speech signal in a specific range; The step of extracting semantic features within a certain range of the transmission signal to obtain local semantic features includes: Obtaining an intermediate value according to the transmission signal; Based on the intermediate value, extracting semantic features within a certain range of the transmission signal to obtain the local semantic features; The extracting the semantic features within a certain range of the transmission signal based on the intermediate value to obtain the local semantic features includes: Based on the pre-constructed semantic feature extraction matrix, the attention distribution of the semantic feature is obtained according to the intermediate value; Obtaining an attention value of the semantic feature according to the attention distribution of the semantic feature; Obtaining a first intermediate feature according to the attention value of the semantic feature; According to the first intermediate feature, a second intermediate feature is obtained; The local semantic feature is obtained according to the first intermediate feature and the second intermediate feature.

2. The method according to claim 1, characterized in that The extracting semantic features within a certain range of the transmission signal to obtain local semantic features includes: Get cache semantic features; Obtaining an intermediate value according to the cache semantic feature and the transmission signal; Based on the intermediate value, the semantic features within a certain range of the transmission signal are extracted to obtain the local semantic features.

3. The method according to claim 1, characterized in that After obtaining the local semantic feature according to the first intermediate feature and the second intermediate feature, the method further includes: The local semantic features are cached.

4. The method according to claim 1, characterized in that: The step of inputting the coded signal into a channel for communication transmission comprises: The coded signal is input into a channel so that the channel outputs a coded signal with noise. The coded signal with noise is decoded and semantic features are restored to obtain a received signal.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

6. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 4.

Images