Vector matrix determination method and system, storage medium, and electronic device
By encoding the message into a three-dimensional codeword matrix and converting it into a vector matrix using a fully connected network group, the problem that semantic communication systems cannot integrate classical communication is solved, the ability to process sentences of different lengths is realized, and the application scenarios and data compression potential is expanded.
Patent Information
- Application Number
- PCT/CN2023/142953
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-18
- Filing Date
- 2023-12-28
- Publication Date
- 2024-11-21
AI Technical Summary
The existing semantic communication system does not involve classic data encoding, and requires the same length of transmitted sentences, which makes it impossible to integrate the classic communication system, limiting the application scenarios of semantic communication and the potential for data information compression.
By encoding the message to be transmitted into a three-dimensional codeword matrix and converting it using a fully connected network group, a vector matrix with the same codeword length is obtained, and the input of the semantic encoding module is realized to blend classical encoding and semantic encoding.
It realizes the integration of classical coding with semantic coding of semantic communication system, and can process sentences to be transmitted of different lengths, expanding the application scenarios of semantic communication and the potential for data information compression.
Smart Images

Figure CN2023142953_21112024_PF_FP_ABST
Abstract
Description
Method, system, storage medium and electronic device for determining vector matrix
[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on May 18, 2023, with application number 202310567601.X and invention name “Method, system, storage medium and electronic device for determining vector matrix”, the entire contents of which are incorporated by reference into this disclosure. Technical Field
[0002] The present disclosure relates to the field of communications, and in particular, to a method, system, storage medium, and electronic device for determining a vector matrix. Background Art
[0003] In related fields, a deep learning-based semantic communication (DeepSC) model is commonly used in semantic communication systems. The structure of this model is shown in Figure 1.
[0004] In addition to the aforementioned DeepSC model, there are many other semantic communication models, such as DeepSC-L, the Universal Transformer-based semantic communication system, and the variable coding rate semantic communication system. These models require fixed-length one-hot vectors as input for semantic encoding.
[0005] In addition, existing semantic communication systems generally adopt a machine learning (ML) framework, which uses the powerful nonlinear fitting capabilities of artificial neural networks (ANNs) to extract the semantic redundancy behind the information. However, due to the complexity of natural language, current semantic communication system implementations basically adopt a joint source-channel coding (JSCC) scheme, in which source coding and channel coding are performed jointly and cannot be separated. And because ANNs can only process fixed-length data, such system implementations cannot process variable-length information. This defect makes it difficult for semantic communication systems to integrate with classical communication systems, and also greatly limits the data information compression potential of semantic communication.
[0006] In summary, existing semantic communication systems do not utilize classical data encoding and require that the transmitted sentences be of the same length. These transmission requirements significantly limit the application scenarios of semantic communication and hinder the integration of semantic communication with classical communication. It should be noted that there are currently no examples of integrating semantic encoding with classical encoding.
[0007] Regarding related technologies, existing semantic communication systems do not involve classical data encoding and require the transmitted sentences to be of the same length, which leads to the problem that semantic communication systems cannot integrate classical communication. No effective solution has been proposed yet.
[0008] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology.
[0009] Summary of the Invention
[0010] The embodiments of the present disclosure provide a method, system, storage medium and electronic device for determining a vector matrix, so as to at least solve the problem in the related art that existing semantic communication systems do not involve classical data encoding and require the transmitted sentences to be of the same length, which leads to the inability of semantic communication systems to integrate classical communication.
[0011] According to one aspect of an embodiment of the present disclosure, a method for determining a vector matrix is provided, comprising: encoding a message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codewords in the regular codeword matrix have the same codeword length, and the codewords in the irregular codeword matrix have different codeword lengths, wherein the codeword matrix is a three-dimensional codeword matrix; and converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0012] In an exemplary embodiment, the fully connected network group includes multiple fully connected networks with different input lengths and the same output lengths, the second dimension size of the two-dimensional parameters of the multiple fully connected networks is related to the number of words in the vocabulary corresponding to the natural language used when training the fully connected networks, and the vector matrix serves as the input of the semantic encoding module, wherein the vector matrix is a three-dimensional vector matrix.
[0013] In an exemplary embodiment, before converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: obtaining the codeword length of each codeword in the codeword matrix; and determining the codeword length as the first dimension size of the two-dimensional parameter of the fully connected network corresponding to each codeword.
[0014] In an exemplary embodiment, the codeword matrix is converted by a fully connected network group to obtain a vector matrix with the same codeword length, including: inputting each codeword in the codeword matrix into the fully connected network corresponding to each codeword, and determining the vector A corresponding to each codeword in the vector matrix by the following formula: A=W i h+b i ;
[0015] Wherein, h represents each codeword in the codeword matrix, W i and b i represents the two-dimensional parameters of the fully connected network corresponding to the i-th codeword in the codeword matrix, where W i and b i The first dimension size of each is the codeword length of the i-th codeword; the vector matrix is obtained according to the position of each codeword in the codeword matrix and the vector A corresponding to each codeword.
[0016] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: obtaining a probability vector corresponding to each third-dimensional vector of the vector matrix; determining a target vector based on the probability vector, wherein the target vector is a vector in which the maximum element in each third-dimensional vector is set to a first value; and combining multiple target vectors into a target matrix.
[0017] In an exemplary embodiment, determining a target vector based on the probability vector includes: determining noise data corresponding to each third-dimensional vector based on the probability vector; and determining the target vector based on the noise data and the probability vector, wherein the noise data is used to make the target vector not a constant value.
[0018] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: directly inputting the vector matrix into a first fully connected network of a semantic encoding module, wherein the first dimension size of the two-dimensional parameters of the first fully connected network is related to the number of words in the vocabulary.
[0019] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: grouping according to the third dimension direction of the vector matrix to obtain multiple groups of sub-vector matrices; converting the multiple groups of sub-vector matrices into multiple sub-target matrices, and determining the target matrix based on the multiple sub-target matrices.
[0020] In an exemplary embodiment, the codeword matrix is converted through a fully connected network group to obtain a vector matrix of codewords with the same length, including: obtaining a maximum codeword length in the codeword matrix; filling a second value at a target position of the codeword matrix according to the maximum codeword length to obtain a padded codeword matrix in which the codeword lengths are all the maximum codeword lengths; and converting each codeword in the padded codeword matrix through the fully connected network group to obtain a vector matrix in which the codeword lengths are all the same.
[0021] In an exemplary embodiment, the present invention includes: when the encoding is encoded by an arithmetic coding algorithm or an LZ coding algorithm, obtaining a two-dimensional codeword matrix corresponding to the encoding; obtaining a three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix, replacing the first dimension and the second dimension of the three-dimensional codeword matrix to obtain an input matrix, wherein the third dimension size of the input matrix is the maximum codeword length of the codeword matrix; determining a second fully connected network in which the first dimension size of the two-dimensional parameter is the maximum codeword length and the second dimension size of the two-dimensional parameter is related to the number of words in the vocabulary, and converting each codeword in the input matrix through the second fully connected network to obtain a vector matrix with the same codeword length.
[0022] In an exemplary embodiment, encoding the message to be transmitted into a codeword matrix includes at least one of the following: encoding the message to be transmitted into a codeword matrix using a Huffman coding algorithm; encoding the message to be transmitted into a codeword matrix using an arithmetic coding algorithm; encoding the message to be transmitted into a codeword matrix using an LZ coding algorithm; encoding the message to be transmitted into a codeword matrix using a Huffman coding algorithm, and truncating the codeword matrix according to a target length to form a lossy coded codeword matrix.
[0023] In an exemplary embodiment, the first value is a positive number, and the second value is a value smaller than the first value.
[0024] According to another aspect of the embodiments of the present disclosure, a vector matrix determination system is also provided, including: an encoding module, configured to encode a message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codewords in the regular codeword matrix have the same codeword length, and the codewords in the irregular codeword matrix have different codeword lengths, wherein the codeword matrix is a three-dimensional codeword matrix; a data semantic conversion module, connected to the encoding module, configured to convert the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0025] In an exemplary embodiment, the system further includes: a semantic encoding module, connected to the data semantic conversion module, configured to replace the embedding layer of the semantic encoding module with a first fully connected network, and receive the vector matrix through the first fully connected network.
[0026] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the above-mentioned method for determining the vector matrix when running.
[0027] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the vector matrix through the computer program.
[0028] Through the present disclosure, the message to be transmitted is encoded into a codeword matrix; the codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length. In other words, by encoding the message to be transmitted into a three-dimensional codeword matrix and converting the obtained codeword matrix into a vector matrix with the same codeword length, the technical problem that the existing semantic communication systems in the related art do not involve classical data coding and require the transmitted sentences to be of the same length, which in turn causes the semantic communication system to be unable to integrate classical communication, is solved. The technical effect of integrating classical coding with the semantic coding of the existing semantic communication system is achieved, and by integrating classical coding and semantic coding, sentences to be transmitted of different lengths can be processed. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:
[0030] FIG1 is a schematic diagram of the DeepSC model structure in related art;
[0031] FIG2 is a hardware structure block diagram of a computer terminal for a method for determining a vector matrix according to an embodiment of the present disclosure;
[0032] FIG3 is a flow chart of a method for determining a vector matrix according to an embodiment of the present disclosure;
[0033] FIG4 is a schematic diagram of the architecture of a semantic coding system in the related art;
[0034] FIG5 is a schematic diagram illustrating the difference between classic source coding and semantic coding according to an embodiment of the present disclosure (I);
[0035] FIG6 is a schematic diagram (II) illustrating the difference between classical source coding and semantic coding according to an embodiment of the present disclosure;
[0036] FIG7 is a schematic diagram (III) illustrating the difference between classical source coding and semantic coding according to an embodiment of the present disclosure;
[0037] FIG8 is a schematic diagram (four) illustrating the difference between classic source coding and semantic coding according to an embodiment of the present disclosure;
[0038] FIG9 is a schematic diagram illustrating the difference between classic source coding and semantic coding according to an embodiment of the present disclosure (V);
[0039] FIG10 is a schematic diagram of semantic information according to an embodiment of the present disclosure;
[0040] FIG11 is a schematic diagram of the structure of a system for fusing classic Huffman coding and semantic coding according to an embodiment of the present disclosure;
[0041] FIG12 is a schematic diagram of the structure of a system for fusing classic Huffman coding and improved semantic coding according to an embodiment of the present disclosure;
[0042] FIG13 is a schematic diagram of the structure of a classic Huffman coding and semantic coding fusion system after adding a “Many-Hot” vector conversion module according to an embodiment of the present disclosure;
[0043] FIG14 is a schematic diagram of the structure of a system for fusing classic Huffman coding and semantic coding using a padding mechanism according to an embodiment of the present disclosure;
[0044] FIG15 is a schematic diagram of the structure of an arithmetic coding and semantic coding fusion system according to an embodiment of the present disclosure;
[0045] FIG16 is a schematic diagram of the structure of a system for fusing LZ coding and semantic coding according to an embodiment of the present disclosure;
[0046] FIG17 is a schematic diagram of the structure of a multi-language semantic encoding and decoding system according to an embodiment of the present disclosure;
[0047] FIG18 is a structural block diagram of a system for determining a vector matrix according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0048] In order to enable those skilled in the art to better understand the solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.
[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0050] The method embodiments provided in the embodiments of the present disclosure can be executed in a computer terminal or a similar computing device. Taking running on a computer terminal as an example, FIG2 is a hardware structure block diagram of a computer terminal of the method for determining a vector matrix in an embodiment of the present disclosure. As shown in FIG2 , the computer terminal may include one or more (only one is shown in FIG2 ) processors 202 (the processor 202 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD)) and a memory 204 for storing data. In an exemplary embodiment, the above-mentioned computer terminal may also include a transmission device 206 and an input and output device 208 for communication functions. It can be understood by those skilled in the art that the structure shown in FIG2 is only for illustration and does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include more or fewer components than those shown in FIG2 , or have a different configuration with the same functions as those shown in FIG2 or more functions than those shown in FIG2 .
[0051] The memory 204 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the vector matrix in the embodiment of the present disclosure. The processor 202 executes various functional applications and data processing by running the computer program stored in the memory 204, that is, implementing the above-mentioned method. The memory 204 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 204 may further include a memory remotely located relative to the processor 202, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.
[0052] Transmission device 206 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a communications provider of a computer terminal. In one embodiment, transmission device 206 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 206 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0053] FIG3 is a flow chart of a method for determining a vector matrix according to an embodiment of the present disclosure. As shown in FIG3 , the method includes the following steps:
[0054] Step S302: Encode the message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single codeword or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix. The codewords in the regular codeword matrix have the same codeword length, and the codewords in the irregular codeword matrix have different codeword lengths. The codeword matrix is a three-dimensional codeword matrix.
[0055] Step S304: transform the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0056] Through the above steps, the message to be transmitted is encoded into a codeword matrix; the codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length. In other words, by encoding the message to be transmitted into a three-dimensional codeword matrix and converting the obtained codeword matrix into a vector matrix with the same codeword length, the technical problem that the existing semantic communication systems in the related art do not involve classical data coding and require the transmitted sentences to be of the same length, which in turn causes the semantic communication system to be unable to integrate classical communication, is solved. The technical effect of integrating classical coding with the semantic coding of the existing semantic communication system is achieved, and by integrating classical coding and semantic coding, sentences of different lengths to be transmitted can be processed.
[0057] In an exemplary embodiment, the fully connected network group includes multiple fully connected networks with different input lengths and the same output lengths, the second dimension size of the two-dimensional parameters of the multiple fully connected networks is related to the number of words in the vocabulary corresponding to the natural language used when training the fully connected networks, and the vector matrix serves as the input of the semantic encoding module, wherein the vector matrix is a three-dimensional vector matrix.
[0058] In an exemplary embodiment, encoding the message to be transmitted into a codeword matrix includes at least one of the following: encoding the message to be transmitted into a codeword matrix using a Huffman coding algorithm; encoding the message to be transmitted into a codeword matrix using an arithmetic coding algorithm; encoding the message to be transmitted into a codeword matrix using an LZ coding algorithm; encoding the message to be transmitted into a codeword matrix using a Huffman coding algorithm, and truncating the codeword matrix according to a target length to form a lossy coded codeword matrix.
[0059] That is, the message to be transmitted is encoded into a codeword matrix using a classical coding algorithm. The selected classical coding algorithms include but are not limited to the Huffman coding algorithm, the arithmetic coding algorithm, and the LZ coding algorithm. Optionally, the codeword matrix obtained by the Huffman coding algorithm can be truncated according to a certain length (i.e., the target length) to form a lossy coded codeword matrix.
[0060] In an exemplary embodiment, before converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: obtaining the codeword length of each codeword in the codeword matrix; and determining the codeword length as the first dimension size of the two-dimensional parameter of the fully connected network corresponding to each codeword.
[0061] After the message to be transmitted is input into the classic encoding module using the classic encoding algorithm, the classic encoding module converts the message to be transmitted and outputs it into a codeword matrix. In general, the output codeword matrix includes multiple codewords with different codeword lengths. However, it is also possible to obtain a codeword matrix including multiple codewords of the same length or a codeword matrix with only one codeword.
[0062] An optional embodiment of the present disclosure is to merge the classical coding module with the semantic coding module of the semantic communication system, and use the data-semantic information conversion module to convert the codeword matrix output by the classical coding module into a vector matrix that can be input into the classical coding module. Optionally, the data-semantic information conversion module may include a fully connected network group for converting the codeword matrix obtained by the output of the classical coding module into a vector matrix with the same codeword length, so as to facilitate input into the semantic communication system. Specifically, the fully connected network group includes a plurality of fully connected networks, and the plurality of fully connected networks may be the same as the number of codewords in the codeword matrix, and the first dimension size of the two-dimensional parameters of each of the plurality of fully connected networks is: the codeword length of the codeword to be processed corresponding to each fully connected network, for example, the first dimension size of the fully connected network corresponding to a codeword of length 1 is 1, the first dimension size of the fully connected network corresponding to a codeword of length 2 is 2, and so on.
[0063] In an exemplary embodiment, the codeword matrix is converted by a fully connected network group to obtain a vector matrix with the same codeword length, including: inputting each codeword in the codeword matrix into the fully connected network corresponding to each codeword, and determining the vector A corresponding to each codeword in the vector matrix by the following formula: A=W i h+b i ;
[0064] Wherein, h represents each codeword in the codeword matrix, W i and b i represents the two-dimensional parameters of the fully connected network corresponding to the i-th codeword in the codeword matrix, where W i and b i The first dimension size of each is the codeword length of the i-th codeword; the vector matrix is obtained according to the position of each codeword in the codeword matrix and the vector A corresponding to each codeword.
[0065] It can be understood that the fully connected network converts the codewords in the codeword matrix into vectors in the vector matrix using the above formula. Specifically, each codeword in the codeword matrix is input into the corresponding fully connected network, and each codeword is converted into a vector corresponding to each codeword using the above formula. All codewords in the codeword matrix are converted into corresponding vectors, and the obtained vectors are combined into a vector matrix according to the position of each codeword in the codeword matrix.
[0066] The two-dimensional parameter W in the above formula i and b i The size of the second dimension is related to the size of the vocabulary used in training, and the size of the vocabulary used in training is the number of words in the vocabulary corresponding to the natural language used in training. In an optional embodiment, if the language used in training is Chinese and there are 7000 words in the corresponding vocabulary, the size of the second dimension of the two-dimensional parameters of the multiple fully connected networks included in the fully connected network group is 7000. It should be noted that the obtained vector matrix is: a three-dimensional vector matrix with a fixed dimension, and the size of the third dimension is related to the size of the vocabulary used in training, and the size of the vocabulary used in training is the number of words in the vocabulary corresponding to the natural language used in training. For example, when the size of the second dimension of the two-dimensional parameter of the fully connected network group is 7000, the size of the third dimension of the vector matrix is 7000.
[0067] It should be noted that the above "7000" is only used as an example and is not used to limit the present disclosure. By analogy, the language used in training is not limited to Chinese, and the size of the vocabulary used in training is not limited to 7000 words, but is related to the actual size of the vocabulary corresponding to the language type used in training.
[0068] Since the embedding layer of the semantic encoding module requires one-hot input vectors, it's important to note that one-hot vectors are a method for converting discrete, categorical data into continuous vector representations. In a one-hot vector, each category is encoded as a vector where only one dimension has a value of 1, and all other dimensions are 0. One-hot vectors can easily convert any number of different labels into a fixed-length, dense, low-dimensional vector format. These vectors are independent of each other and have no inherent relationships. Therefore, many machine learning algorithms often encode input data into one-hot vectors for processing.
[0069] Therefore, after converting the codeword matrix to a vector matrix, optionally:
[0070] 1) The vector matrix can be further converted into a One-Hot input vector:
[0071] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: obtaining a probability vector corresponding to each third-dimensional vector of the vector matrix; determining a target vector based on the probability vector, wherein the target vector is a vector in which the maximum element in each third-dimensional vector is set to a first value; and combining multiple target vectors into a target matrix.
[0072] In an exemplary embodiment, determining a target vector based on the probability vector includes: determining noise data corresponding to each third-dimensional vector based on the probability vector; and determining the target vector based on the noise data and the probability vector, wherein the noise data is used to make the target vector not a constant value.
[0073] It can be understood that splitting the vector matrix into multiple probability vectors according to the third dimension is essentially highlighting the maximum value in each probability vector, that is, the probability value of the probability vector corresponding to the maximum value in each third-dimensional vector is the largest among the probability values corresponding to all probability vectors included in the probability vector corresponding to the third-dimensional vector.
[0074] In order to make the target vector non-constant and thus allow the gradient to be returned, a reparameterization technique can be used:
[0075] In an optional embodiment, Gumbel noise is added to the probability vector. Specifically, the corresponding noise data is determined by the third-dimensional vector, that is, the noise data corresponding to each vector element in the third-dimensional vector is calculated; each probability element in the obtained probability vector is added to each corresponding noise data, and finally a target probability vector (i.e., the target vector in the above embodiment) is obtained. All target probability vectors are combined to obtain a target matrix, i.e., a one-hot input matrix, which can be input into the embedding layer in the semantic coding module.
[0076] 2) The Embedding layer in the semantic encoding module can be replaced, thereby removing the restriction of the one-hot input vector required by the Embedding layer:
[0077] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: directly inputting the vector matrix into a first fully connected network of a semantic encoding module, wherein the first dimension size of the two-dimensional parameters of the first fully connected network is related to the number of words in the vocabulary.
[0078] In an optional embodiment of the present disclosure, the first dimension of the two-dimensional parameters of the fully connected network that replaces the Embedding layer is the same as the third dimension of the target vector. This allows the vector matrix to be directly input into the semantic encoding module, avoiding the need to convert the vector matrix into a one-hot input vector.
[0079] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: grouping according to the third dimension direction of the vector matrix to obtain multiple groups of sub-vector matrices; converting the multiple groups of sub-vector matrices into multiple sub-target matrices, and determining the target matrix based on the multiple sub-target matrices.
[0080] Grouping is performed in the direction of the third dimension of the vector matrix. Optional solutions include: grouping according to the maximum codeword length of the codeword matrix corresponding to the vector matrix, and the number of groups is related to the codeword length in the corresponding codeword matrix. A more specific optional solution is that the number of groups is related to the maximum codeword length minus the codeword length of the codeword corresponding to the vector in the vector matrix. For example, if the codeword length of the codeword corresponding to a certain vector in the vector matrix is 2, the maximum codeword length is 8, then 8-2, or 6, is the number of groups. If the vectors of the vector matrix cannot be divided equally according to the number of groups, the remaining elements can be discarded, or the group that cannot be divided equally can be used as the last group.
[0081] In an exemplary embodiment, the codeword matrix is converted through a fully connected network group to obtain a vector matrix of codewords with the same length, including: obtaining a maximum codeword length in the codeword matrix; filling a second value at a target position of the codeword matrix according to the maximum codeword length to obtain a padded codeword matrix in which the codeword lengths are all the maximum codeword lengths; and converting each codeword in the padded codeword matrix through the fully connected network group to obtain a vector matrix in which the codeword lengths are all the same.
[0082] Optionally, the codeword matrix is first padded according to the maximum codeword length in the codeword matrix to obtain a padded codeword matrix with the same codeword length.
[0083] For example, suppose the message to be transmitted is S, which consists of B sentences of length L. The length L means that there are L words. When the classic encoding module is the Huffman encoding module, the codeword matrix output by the Huffman encoding module is a three-dimensional codeword matrix with a shape of [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal, and its maximum length is recorded as max(H ij ). When the codeword matrix is padded with the second value, a padded codeword matrix with the same codeword length is obtained, and its size is [B, L, max(H ij )].
[0084] The data-semantic conversion module using the padded codeword matrix as input can then use only one parameter of size [max(H ij ), CorpusLen], can convert the padded codeword matrix into a vector matrix. It should be noted that CorpusLen is related to the vocabulary size used during training, which is the number of words in the vocabulary corresponding to the natural language used during training. In an optional embodiment of the present disclosure, the structure of the data-semantic conversion module can be simplified.
[0085] In an exemplary embodiment, the present invention includes: when the encoding is encoded by an arithmetic coding algorithm or an LZ coding algorithm, obtaining a two-dimensional codeword matrix corresponding to the encoding; obtaining a three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix, replacing the first dimension and the second dimension of the three-dimensional codeword matrix to obtain an input matrix, wherein the third dimension size of the input matrix is the maximum codeword length of the codeword matrix; determining a second fully connected network in which the first dimension size of the two-dimensional parameter is the maximum codeword length and the second dimension size of the two-dimensional parameter is related to the number of words in the vocabulary, and converting each codeword in the input matrix through the second fully connected network to obtain a vector matrix with the same codeword length.
[0086] When the classic encoding module uses an arithmetic coding algorithm or an LZ coding algorithm, the codeword matrix obtained after converting the message to be transmitted is a two-dimensional codeword matrix. Furthermore, the data-semantic information conversion module corresponding to the arithmetic coding algorithm or the LZ coding algorithm can be selected to be composed of a convolutional layer, a reshape layer, and a fully connected layer (equivalent to the second fully connected network in the above embodiment).
[0087] Therefore, obtaining the three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix includes: when the two-dimensional codeword matrix has a shape of [B,H i ], where H i It represents the length of the arithmetic codeword obtained by encoding the i-th sentence, and the longest length is recorded as max(H i ), fill the two-dimensional code matrix to obtain a size of [B, max(H i )], a three-dimensional codeword matrix can be obtained based on the two-dimensional filling codeword matrix when certain conditions are met.
[0088] Optionally, under certain conditions, a three-dimensional codeword matrix is obtained based on the two-dimensional padded codeword matrix, including: passing the two-dimensional padded codeword matrix through a convolutional layer with a layer channel number L and a kernel size of 1*1, when the padding number P of the convolutional layer and the stride S satisfy the following relationship:
[0089] Among them, P H ,S H Indicates the number of padding and step length in the column direction, P W ,S W Indicates the number of padding and step size in the row direction. The three-dimensional matrix corresponding to the two-dimensional padding codeword matrix can be obtained through the above convolution layer, and the size is [L, B, max(H i )].
[0090] Furthermore, the first dimension and the second dimension of the obtained three-dimensional codeword matrix are permuted to obtain an input matrix, and input into the matrix with a parameter size of [max(H i ), CorpusLen], and finally a vector matrix of size [B, L, CorpusLen] can be obtained. The scheme for inputting the vector matrix into the semantic encoding module can be referred to other embodiments of the present disclosure and will not be repeated here.
[0091] In the above exemplary embodiment, the first value is a positive number, and the second value is a value smaller than the first value. For example, if the first value is a positive number 2, the second value should be a value smaller than 2.
[0092] Next, the method for determining the vector matrix is further described in conjunction with the following embodiments.
[0093] It should be noted that, as shown in Figure 1, the DeepSC model operates as follows: the message that the source needs to transmit is marked as S, and the message S consists of B sentences of length L. The length L means that it has L words (English), then S can be expressed as:
[0094] where {w ij ; i=1,2,...,Bj=1,2,...,L} represents the jth word in the i-th sentence.
[0095] The message S is then encoded as a one-hot vector of size [B, L, CorpusLen], where CorpusLen represents the number of different words in the message, and then passes through an Embedding layer, which stores the parameter W of shape [CorpusLen, E]. Embedding After the Embedding layer, we get a word distribution vector E of size [B, E]. E = SW Embedding .
[0096] In natural language processing (NLP), word distributed vectors can represent the correlation between word meanings. That is, they can be regarded as codewords after considering word meanings, which is the first step in semantic encoding, representing the original message S using the most efficient physical carrier (word distributed vector).
[0097] Then we need to calculate the position information of each word in the word vector E. The calculation formula is as follows:
[0098] Where pos = [1, 2, ..., L], n = [1, 2, ..., E]. And add it element by element with the calculated word vector E to get the new word vector that takes position information into account, which is still represented by E here:
[0099] in, Represents element-wise addition.
[0100] The word distribution vectors E, which take positional information into account, are then passed through the Transformer Encoder to calculate their correlation. As shown in Figure 4, the Transformer Encoder used in the semantic communication system includes three encoders (i.e., the encoders in Figure 4). The structure of each encoder is shown in the dashed box on the right side of Figure 4, consisting of an 8-head attention layer (i.e., the multi-head attention layer in Figure 4) and two fully connected layers.
[0101] It should be noted that the FNN in FIG4 is a feedforward neural network (FNN for short).
[0102] The word vector E is first input into the multi-head attention layer. The 8-head attention layer divides the word vector into 8 equal parts, and then uses the parameters W Q ={W Q 1 , W Q 2 ,...,W Q 8}, W K ={W K 1 , W K 2 ,...,W K 8}, W V ={W V 1 , W V 2 ,...,W V 8}.
[0103] Embed the original word vector into the subspace and get three feature vectors: Q = {Q 1 , Q 2 ,...,Q 8}=SW Q T , K={K 1 , K 2 , ..., K 8}=SW KT , V={V 1 , V 2 ,...,V 8}=SW V T .
[0104] Then, we perform an attention operation on these three feature vectors to get the output. The shape of the output vector O is consistent with the input word vector E:
[0105] The output vector O is then normalized through the residual connection and layer normalization to finally obtain the attention output H. The attention output also needs to pass through two fully connected layers: Z = Relu (OW1 + b1) W2 + b2.
[0106] The vector Z transformed by the fully connected layer undergoes a residual connection and layer normalization again, ultimately yielding the output vector M. Through these steps, one encoder encodes the message S. Since three encoders are used here, this process needs to be repeated twice to obtain the final output, also denoted as M. The attention mechanism described above completes the second step of semantic encoding, extracting some grammatical association information.
[0107] The output vector M is then passed through a dense layer, which performs channel coding to obtain the channel transmission vector X: X = Relu (ZW Dense +b Dense ).
[0108] After vector X is transmitted through the AWGN channel, Y is obtained, which is then decoded at the destination. The decoding process is basically the inverse of the encoding process, so it will not be described in detail here.
[0109] Based on the above description, an optional embodiment of the present disclosure combines a classic encoding scheme to develop a joint encoding module for data and semantics.
[0110] Specifically, in order to solve the technical problem that the existing semantic communication systems in the relevant technologies do not involve classical data coding and require the transmitted sentences to be of the same length, which leads to the inability of the semantic communication system to integrate classical communication, and to further improve the source coding in semantic communication, an optional embodiment of the present disclosure proposes a semantic communication coding module for variable-length coding input. The input of the coding module can be the source coding of the classical coding system, such as Huffman coding or arithmetic coding, and the codewords of these classical codes are further optimized within the system so that they can be further optimized and improved according to the semantic redundant information implied by the context, and finally sent to the receiver through the channel on the air interface, while the semantic decoding system of the receiver can remain unchanged.
[0111] The following should be explained:
[0112] Semantic communication is a new communication paradigm that has emerged in recent years. Based on this paradigm, the communication system will not only consider the redundancy existing in the encoded data, that is, the data redundancy compressed by source coding and the data redundancy added by channel coding to combat noise, but also the redundancy existing in the natural language meaning represented by the encoded data, which is called semantic redundancy.
[0113] Classical source coding only considers data redundancy. As shown in Figure 5, each black dot represents a codeword, and there is no correlation between them. However, classic source coding does not consider the semantics represented by these codewords. In semantic space, however, as shown in Figure 6, the meanings represented by these codewords may be related. For example, the words "cat" and "dog" in the figure share some common characteristics, such as being small animals and both being kept as domestic pets. In some contexts, they can overlap, as shown in Figures 7 and 8. Depending on the context and the meaning of the codewords, they can be further compressed. However, the correlation between natural language is not like that between numbers. Natural language is more complex and can be nonlinear, so the final diagram may be something like Figure 9.
[0114] Due to the complexity of natural language, it can be non-stationary, possess memory, and be nonlinear. Therefore, it does not satisfy the additivity conditions of the Shannon entropy formula. A simple example is "taking a sentence out of context." Therefore, semantic communication systems cannot be described using the classic Shannon model. This is why semantic communication systems are implemented using the ANN-based JSCC architecture. To better describe the data and semantics integrated encoding module proposed in this disclosure, it is first necessary to provide a detailed definition of the new concepts in semantic communication systems.
[0115] Semantic information S: The semantic information of a message M is defined as the sum of the semantic information contained in the data e carrying the message M and the semantic information contained in the data e (where ) and the amount of semantic association information between them, superposition in the semantic space:
[0116] Here Representing high-dimensional operations, it is a temporary representation when the specific functional form of semantic feature information and semantic association information cannot be determined. Where e represents the set of individual data that constitute the message M, which belongs to a certain physical carrier E, such as English or Chinese, that is,
[0117] Semantic association information function cor: It represents the semantic information generated by the arrangement of multiple single data due to corresponding grammatical rules under the current physical carrier, which is represented by the cor function.
[0118] Semantic feature information function λ: represents the remaining information that can characterize the data meaning characteristics in the semantic information after extracting the associated information generated by the single data arrangement under the current physical carrier.
[0119] The above definition can be simply represented by Figure 10. Semantic information is high-dimensional information, and the simplified diagram here only shows two dimensions. In fact, semantic feature information and semantic association information may be high-dimensional, and the operations between them are also high-dimensional operations.
[0120] It is important to note that the semantic association information here (at least) has two characteristics:
[0121] The first feature is meaning association information, that is, the association between word meanings. This type of information is similar to word vector representation in deep learning, such as Word2Vec.
[0122] Another feature is grammatical association information, that is, the association between contexts. This type of information is similar to the language model in natural language processing.
[0123] The disclosed embodiments consider semantic encoding to be a two-step process. The first step is to transform the message M, based on semantic association information, so that the same semantics can be represented using the most efficient physical medium, with "most efficient" meaning the least amount of data information. The second step is to extract the syntactic association information contained in the transformed information representation, so that the remaining syntactic association information is sufficient for decoding.
[0124] Optionally, the above embodiment is described below in conjunction with specific embodiments:
[0125] Example 1: Huffman coding and semantic coding fusion solution:
[0126] Optional embodiment 1 of the present disclosure relates to a method for fusing Huffman coding with semantic coding, and more particularly to a module in the communications field that converts classical source coding into semantic coding, referred to as a data-semantic information conversion module. The aforementioned device can be applied to a transmitting device. This module not only considers the data redundancy compression of the original message but also maps it to semantic redundancy, enabling semantic encoding and compression of semantic redundancy in Huffman codes of unequal lengths.
[0127] This embodiment includes a Huffman coding and semantic coding fusion device. The system structure is shown in Figure 11. The device includes: a Huffman coding module 111, a data-semantic information conversion module 112, and a semantic coding module 113. The above modules can be one or more dedicated or general hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above functions.
[0128] Furthermore, the main function of the Huffman encoding module is to perform Huffman encoding. Assuming that the incoming message is S, the message S consists of B sentences of length L, where length L means there are L words (English is used as an example here), then S can be expressed as:
[0129] where {w ij ; i=1,2,...,Bj=1,2,...,L} represents the jth word in the i-th sentence.
[0130] The message matrix S first passes through the Huffman encoding module, which is mainly composed of a Huffman tree. This module reduces the number of bits required to represent the data and allows each word in the message matrix S to be represented by codewords of varying lengths. That is, the message matrix S is expanded into a Huffman codeword matrix H, whose shape is [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal. For the convenience of description, the longest Huffman codeword length is recorded as max(H ij ).
[0131] Furthermore, the data-semantic conversion module addresses the conflict between semantically relevant information and data redundancy. It combines Huffman coding with semantic coding to achieve feature mapping and balanced encoding. Furthermore, this module includes two submodules: a redundancy mapping module and a one-hot vector generation module.
[0132] 1) The redundant mapping module establishes a mapping relationship between data encoding and semantic encoding. The mapping relationship is composed of a group of fully connected layers (equivalent to the fully connected network group in the above embodiment), and the parameter size of each fully connected layer in the group (equivalent to the fully connected network in the above embodiment) is consistent with the length H of the codeword after Huffman encoding. ijAnd it is related to the pre-set output length CorpusLen. CorpusLen is a positive integer. The final output matrix is recorded as A code .
[0133] Specifically, regarding the redundant mapping module:
[0134] Generally speaking, the more frequently a word appears in message S, the shorter its Huffman codeword is. However, the more structured information it contains with other words in natural language, the wider the semantic range it can represent, and the richer its semantic association information is. In other words, the richer its semantic association information is, the more semantic redundant information may need to be extracted during the semantic encoding stage. Conversely, the less frequently a word appears, the longer its Huffman codeword is, but the less semantic association information it contains, and the less semantic redundant information may need to be extracted during the semantic encoding stage. The data-semantic conversion module will not only consider the data information redundancy of message S, but also the semantic association information redundancy of message S, and then attempt to establish a mapping between the two to find a balanced coding point.
[0135] After the Huffman codeword matrix is input into the data-semantic information conversion module, it will be converted into a codeword that takes into account structured information (that is, semantic association information) using a fully connected layer based on its codeword length. The size of the parameters contained in the fully connected layer here is determined by the Huffman codeword length. Huffman codewords of different lengths h will be mapped to vectors of the same dimension: A = W Huffman i h+b Huffman i ,
[0136] Among them, W Huffman i with b Huffman i Represents parameters of different dimensions, where the first dimension is the same as the length of the corresponding Huffman codeword, for example, the length H ij The Huffman codeword is 1, and the input fully connected layer parameter size is [1, CorpusLen], and the length is H ij For a Huffman codeword of 2, the input parameter size of the fully connected layer is [2, CorpusLen], and so on. The longest Huffman codeword is recorded as max(H ij ).
[0137] For a length of H ij Although the Huffman code lengths are different, after the data-semantic information conversion module, the vector matrix A composed of vector A is obtained. codeThe size of is [B, L, CorpusLen], and the length of each codeword is expanded to CorpusLen, which represents the structured relationship between it and other codewords. Thus, the data redundancy and semantic association information redundancy are mapped through the data-semantic information conversion module.
[0138] 2) The one-hot vector generation module can output a vector B in which one element is 1 and the other elements are 0. In addition, the reparameterization technique is used. Furthermore, the reparameterization technique can meet the gradient backpropagation conditions during network training. In order to illustrate how this technique solves the gradient propagation problem and the data changes in this embodiment, an example is given here. For example, A code The third dimension vector in the matrix is denoted as π, and its length is CorpusLen. It is transformed as follows:
[0139] Specifically:
[0140] Pair vector matrix A code Perform a Softmax operation and apply it to the one-hot vector input of the original semantic encoder (i.e., the semantic encoding module). Here, we can directly use the argmax function to select the largest element in each row of A and set it to 1, thus forming a new one-hot vector B. However, it should be noted that since the argmax function itself is not differentiable, the gradient cannot be propagated backward during training, resulting in the inability to update the parameters in the data-semantic information conversion module. Therefore, reparameterization techniques such as the Gumbel-Softmax Trick can be used to achieve the above process. The specific optional implementation process is as follows:
[0141] A code The third dimension vector in the matrix is denoted as π, and its length is CorpusLen. First, perform Softmax on the vector π to obtain the probability of each element appearing:
[0142] π i The probability of occurrence p i Add Gumbel noise:
[0143] Among them G i is called Gumbel noise: G i =-log(-log(U i )),U i ~U(0,1),
[0144] This noise is used to make the returned results Not fixed. In the end we get The vector is a one-hot vector. However, the argmax function cannot be differentiated in this formula, so the Softmax function is used instead:
[0145] Among them, τ is called the scaling factor. When τ is small enough, x π It is close to a One-Hot vector and can be reverse-derivative.
[0146] According to the above options The vector composed of , which can be used as the input of the semantic encoding module.
[0147] Furthermore, the new one-hot vector B is finally passed through the embedding of the semantic coding module to obtain a vector matrix of size [B, L, E] for subsequent semantic coding operations.
[0148] After the above operations, during the joint training of the semantic codec, the Huffman coding module will store data redundancy information, the data-semantic information conversion module will store the correspondence between data redundancy and semantic association information, and the semantic coding module will store the extracted semantic association information, thereby completing the integrated encoding process of semantics and data.
[0149] The innovation of optional embodiment 1 of the present disclosure lies in that a data-semantic conversion module is constructed to represent the mapping between data redundancy and semantic redundancy, which can be jointly considered during the encoding process to obtain the optimal encoding; multiple fully connected networks are combined to form a network group, which realizes unified processing of classic codewords of different encoding lengths; and reparameterization techniques are used to convert classic coding representations that cannot be backpropagated into probabilities, so that the overall system can be optimized through the backpropagation algorithm.
[0150] Example 2: Direct fusion of Huffman coding and improved semantic coding:
[0151] This embodiment includes a method for improving the design of a semantic encoding module, thereby further optimizing the design of a data-semantic information conversion module, thereby reducing the complexity of the overall device implementation.
[0152] The system structure of the optional embodiment 2 of the present disclosure is shown in FIG12 :
[0153] FIG12 is a schematic diagram of the structure of a system integrating classic Huffman coding and improved semantic coding according to an embodiment of the present disclosure. Based on FIG11 , the embedding layer in the original semantic coding module is replaced with a fully connected layer to connect it with the data-semantic information conversion module. FIG12 further includes: a Huffman coding module 111, a data-semantic information conversion module 112, and an improved semantic coding module 123. These modules may be one or more dedicated or general-purpose hardware modules, separate or integrated chips, memories, etc., with corresponding functions or technical features. They may also be one or more embedded or non-embedded dedicated or general-purpose software, software modules, software code, operating systems, or a combination of one or more hardware and software components, with the aforementioned functions.
[0154] Optional embodiment 2 of the present disclosure is obtained on the basis of optional embodiment 1 of the present disclosure, and differs from optional embodiment 1 of the present disclosure in that:
[0155] The data-semantic conversion module only includes redundant mapping modules, and there is no need to use reparameterization techniques to transform the matrix A. code Transform the vector in and directly output the matrix A code .
[0156] The Embedding layer of the semantic encoding module is replaced by a fully connected layer, so the matrix A can be used directly code as input.
[0157] Optionally, the following specifically describes the execution process of the improved solution in optional embodiment 2 of the present disclosure:
[0158] In the optional embodiment 2 of the present disclosure, it is also assumed that the incoming message is S, and the message S consists of B sentences with a length of L. The length L means that it has L words (English), which is expressed as:
[0159] After the message matrix S passes through the Huffman encoding module and the data-semantic conversion module, the vector matrix A of size [B, L, CorpusLen] is finally obtained. code , but due to A code It is not a One-Hot vector itself, so it cannot be directly used as input by the semantic encoding module.
[0160] The fundamental reason is that the Embedding layer requires a one-hot input vector, which can be regarded as an input with a probability of 1 in only one case. The purpose of the Embedding layer is to reduce the amount of calculation. If the amount of calculation is not considered, a fully connected layer with a parameter size of [CorpusLen, E] can be directly used to replace the Embedding layer. In this way, the vector matrix A code It can be directly used as the input of the semantic encoding module. After passing through the fully connected layer, a vector matrix of size [B, L, E] is obtained for subsequent semantic encoding operations.
[0161] The innovation of optional embodiment 2 of the present disclosure is that, based on the Embedding implementation principle, a fully connected layer is used instead of the Embedding layer to store word distributed vectors, which expands its input data requirements. Therefore, there is no need to use a One-Hot vector generation module in the data-semantic information conversion module, which not only simplifies the design structure, but also includes all semantic association information corresponding to the data encoding into the calculation scope of the fusion coding system.
[0162] Example 3: Huffman coding transformation and semantic coding fusion solution:
[0163] Optional embodiment 3 of the present disclosure includes an improved "Many-Hot" vector conversion module, which converts the output of the data-semantic information conversion module into a legal input of the semantic coding module, while taking into account the anti-correlation relationship between data coding and semantic coding, making the overall design of the system more reasonable.
[0164] FIG13 is a schematic diagram of the structure of a classic Huffman coding and semantic coding fusion system after adding a "Many-Hot" vector conversion module according to an embodiment of the present disclosure, including: a Huffman coding module 111, a data-semantic information conversion module 112, a semantic coding module 113, and a "Many-Hot" vector conversion module 134. The modules can be one or more dedicated or general-purpose hardware modules, separate or integrated chips, memories, etc., with corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general-purpose software, software modules, software code, operating systems, or a combination of one or more hardware and software with the above functions.
[0165] Optional embodiment 3 of the present disclosure is obtained on the basis of embodiment 1, and differs from embodiment 1 in that a “Many-Hot” vector conversion module is added after the data-semantic information conversion module.
[0166] The "Many-Hot" vector conversion module establishes the anti-correlation mapping relationship that may exist between data encoding and semantic encoding. And the output matrix A of the data-semantic conversion module can be converted to code Further transformation is performed to become a "Many-Hot" vector.
[0167] Furthermore, the anti-correlation mapping relationship means that the more frequently a word appears in the original message S, the shorter its Huffman codeword is, the more information it may contain that needs semantic encoding, and the greater the weight it will be considered during semantic encoding. Vice versa.
[0168] Furthermore, the “Many-Hot” vectors are grouped, and the One-Hot vector generation module operation described in Example 1 is performed on each group.
[0169] The grouping operation is performed according to the matrix A code The third dimension, for each vector, corresponds to the third dimension of the Huffman codeword matrix, that is, the dimension of the codeword. The number of groups is inversely related to the length of the corresponding Huffman codeword, that is, if the corresponding codeword length is H ij , then the number of groups is max(H ij )-H ij If it cannot be divided equally, the remaining elements can be discarded, or the group that cannot be divided equally can be used as the last group.
[0170] The one-hot vector generation module operation may pass the grouping vectors through the one-hot vector generation module in embodiment 1 respectively and then concatenate them.
[0171] Optionally, the following specifically describes the execution process of optional embodiment 3 of the present disclosure:
[0172] Also assume that the incoming message is S, which consists of B sentences of length L, where length L means it has L words (English), expressed as:
[0173] The message matrix S first passes through the Huffman encoding module to obtain the Huffman codeword matrix H, whose shape is [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal, and the longest length is recorded as max(H ij ).
[0174] After the codeword matrix H passes through the data-semantic conversion module, a vector matrix A of size [B, L, CorpusLen] is obtained. code , but due to Acode It is not a One-Hot vector itself, so it cannot be directly used as input by the semantic encoding module.
[0175] At the same time, the more frequently a word appears in the original message S, the shorter its Huffman codeword is, but it may contain more semantically relevant information, making semantic encoding more important. Conversely, the less frequently a word appears, the longer its Huffman codeword is, but it may contain less semantically relevant information, making semantic encoding less important. Therefore, there may be an inverse correlation between data redundancy and semantic redundancy.
[0176] Therefore, in order to reflect the anti-correlation between data redundancy and semantic redundancy, and also to transform the vector matrix A code To become the legal input of the semantic encoding module, a "Many-Hot" vector conversion module is designed here, which operates on the data in two steps.
[0177] First, the vector matrix A code , perform grouping operations on them, and grouping is performed on each vector according to its third dimension, which corresponds to the third dimension of the Huffman codeword matrix, that is, the dimension of storing codewords. The number of groups is inversely related to the corresponding Huffman codeword length, that is, if the corresponding codeword length is H ij , then the number of groups is max(H ij )-H ij In short, the number of groups is proportional to the original codeword length H ij If the grouping is not possible, the remaining elements can be discarded, or the group that cannot be equally divided can be used as the last group. This grouping method corresponds to the inverse correlation between data redundancy and semantic redundancy.
[0178] Then, the grouped vectors are respectively subjected to the reparameterization technique described in the optional embodiment 1 of the present disclosure to transform them into one-hot vectors, and then these vectors are connected to finally obtain "many-hot" vectors, which can be used as legal input by the semantic encoding module.
[0179] The innovation of the optional embodiment 3 of the present disclosure is that the vector matrix A representing the probability code Group coding is performed, and finally a "Many-Hot" vector that can reflect different semantic redundant relationships is obtained as the input of the semantic coding module.
[0180] Example 4: Huffman coding and semantic coding fusion scheme using a padding mechanism:
[0181] Optional embodiment 4 of the present disclosure includes an improved padding mechanism Huffman encoding method, which allows it to be input into a data-semantic information conversion module that only includes a fully connected network, greatly simplifying the design complexity of the module and saving storage and computing resources.
[0182] Figure 14 is a schematic diagram of the structure of a system that integrates classical Huffman coding and semantic coding using a padding mechanism according to an embodiment of the present disclosure. A Huffman coding module 111, a padding module 144, and a data-semantic information conversion module 142 are added before the original semantic coding module 113, thereby achieving the integration of the classical coding and semantic coding systems. The modules can be one or more dedicated or general-purpose hardware modules, separate or integrated chips, memories, etc., with corresponding functions or technical features. They can also be one or more embedded or non-embedded dedicated or general-purpose software, software modules, software code, operating systems, or a combination of one or more hardware and software components that have the above functions.
[0183] This embodiment is obtained on the basis of the embodiment 1, and differs from the embodiment 1 in that a filling module is added before the data-semantic information conversion module.
[0184] The filling module can convert the codeword matrix H and output a regular matrix of the same length. The regular matrix is a matrix in which the elements that do not originally exist in the codeword matrix are filled with 0. The shape of the 0-filled matrix is [B, L, max(H ij )-H ij ], is a matrix of irregular shape.
[0185] Optionally, the following specifically describes the execution process of optional embodiment 4 of the present disclosure:
[0186] Assume that the incoming message is S, which consists of B sentences of length L, where length L means it has L words (English is used as an example here), then S can be expressed as:
[0187] where {w ij ; i=1,2,...,Bj=1,2,...,L} represents the jth word in the i-th sentence.
[0188] The message matrix S first passes through the Huffman encoding module to obtain the Huffman codeword matrix H, whose shape is [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal, and its maximum length is recorded as max(H ij ).
[0189] In order to reduce the complexity of system design and reduce storage space overhead, the Huffman codewords of unequal length in the codeword matrix H are filled with 0, and the length of the codeword matrix H is max(H ij ) codeword, the codeword matrix is recorded as H', and its size is [B,L,max(H ij )]0The shape of the filling matrix is [B,L,max(H ij )-H ij ], is a matrix of irregular shape.
[0190] The codeword matrix H' is then input into the data-semantic information conversion module, which consists of a fully connected network with a parameter size of [max(H ij ),CorpusLen], so the vector matrix A can be output code , whose size is [B,L,CorpusLen].
[0191] Finally, the vector matrix A can be converted into code Input into the semantic encoding module.
[0192] The innovation of optional embodiment 4 of the present disclosure is that the padding mechanism can remove the input restriction of the data-semantic encoding module. Previously, a regular matrix input was required, and the padding mechanism can use an irregular vector matrix as its input.
[0193] Example 5: Arithmetic coding and semantic coding fusion solution:
[0194] This embodiment includes a method for fusing arithmetic coding and semantic coding, so that arithmetic coding can also be fused with semantic coding.
[0195] Figure 15 is a structural diagram of the arithmetic coding and semantic coding fusion system according to an embodiment of the present disclosure. An arithmetic coding module 151 and a data-semantic information conversion module 152 are added before the original semantic coding module 113 to realize the fusion of the classical coding and semantic coding systems. The modules can be one or more dedicated or general hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above functions.
[0196] Optionally, the following specifically describes the execution process of optional embodiment 5 of the present disclosure:
[0197] Assume that the incoming message is S, which consists of B sentences of length L, where L means it has L words (in English), expressed as:
[0198] The message matrix S is first passed through the arithmetic coding module to reduce the number of bits of the data, and each sentence in the message matrix S is represented by codewords of different lengths, and their lengths are not equal. The codeword matrix obtained after arithmetic coding is represented as H, and its shape is [B,H i ], where H i It represents the length of the arithmetic codeword obtained by encoding the i-th sentence, and the longest length is recorded as max(H i ).
[0199] The codeword matrix H then passes through a data-semantic conversion module, which establishes a mapping between arithmetic coding and semantic coding.
[0200] The establishment step is divided into three steps: the first step is to calculate the longest length max(H i ) fills all codeword vectors with 0 to obtain a new codeword matrix H. The second step is to pass a convolution layer with L channels and K×K kernel size. When the filling number P and step size S meet certain conditions, the size of the codeword matrix H passing through the convolution layer becomes [L, B, max(H i )]. In the third step, the codeword matrix H is dimensionally permuted and then passed through a fully connected layer with a parameter size related to the preset parameter CorpusLen. Finally, the matrix A with a size related to CorpusLen can be obtained. code .
[0201] Based on the establishment steps, the following describes an optional implementation scheme for establishing a mapping between arithmetic coding and semantic coding:
[0202] In this module, first, according to the longest length max(H i ) fill all codeword vectors with 0 to obtain a new codeword matrix H, whose size is [B, max(H i )], and then pass through a convolutional layer with L channels and a kernel size of 1*1. When its padding number P and stride S satisfy the following relationship:
[0203] Among them, P H ,S H Indicates the number of padding and step length in the column direction, P W ,S W Indicates the padding number and step size in the row direction. The codeword matrix H through this convolutional layer becomes [L, B, max(H i )] (equivalent to the input matrix in the above embodiment).
[0204] Change the size to [L,B,max(H i)] is permuted on the first and second dimensions of the codeword matrix, and then a parameter size of [max(H i ), CorpusLen], the fully connected layer can finally get the input matrix A of size [B,L,CorpusLen] code (equivalent to the vector matrix in the above embodiment).
[0205] The matrix A code The input semantic coding module can refer to the solutions in optional embodiments 1, 2 and 3 of the present disclosure, which will not be repeated here.
[0206] The semantic encoding module can perform semantic encoding on the input message, thereby completing semantic communication.
[0207] The innovation of optional embodiment 5 of the present disclosure is that, based on the characteristics of convolutional neural networks, arithmetic coding is incorporated into the data and semantic fusion coding module.
[0208] Example 6: LZ coding and semantic coding fusion solution:
[0209] This embodiment includes a method for fusing LZ coding and semantic coding, so that LZ coding can also be fused with semantic coding. The system structure is shown in FIG16 .
[0210] Figure 16 is a structural diagram of the LZ coding and semantic coding fusion system according to an embodiment of the present disclosure. An LZ coding module 161 and a data-semantic information conversion module 162 are added before the original semantic coding module 113 to realize the fusion of the classical coding and semantic coding systems. The modules can be one or more dedicated or general hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above functions.
[0211] This embodiment is obtained on the basis of the embodiment 5, and differs from the embodiment 5 in that the arithmetic coding module is replaced by the LZ coding module.
[0212] The message matrix S is first passed through the LZ encoding module to reduce the number of bits of the data. LZ encoding will create a new dictionary to encode different sentences, and the encoding lengths are not equal. The codeword matrix obtained after LZ encoding is represented as H, and its shape is [B,H i ], where H i It represents the length of the arithmetic codeword obtained by encoding the i-th sentence, and the longest length is recorded as max(H i ).
[0213] The remaining modules are similar to those in Example 5.
[0214] The following describes in detail the implementation process of the LZ coding and semantic coding fusion method:
[0215] Alternatively, assume that the incoming message is S, which consists of B sentences of length L, where L means it has L words (in English), expressed as:
[0216] The message matrix S first passes through the LZ encoding module. LZ encoding creates a new dictionary to encode different sentences, and the encoding lengths are not equal. The codeword matrix obtained after LZ encoding is represented as H, and its shape is [B,H i ], where H i Indicates the length of the LZ codeword obtained by encoding the i-th sentence, and the longest length is recorded as max(H i ).
[0217] The subsequent encoding process is consistent with the process described in optional embodiment 5 of the present disclosure and will not be repeated here.
[0218] The innovation of optional embodiment 6 of the present disclosure is that, based on the characteristics of convolutional neural networks, LZ coding is incorporated into the data and semantic fusion coding module.
[0219] Example 7: Multi-language semantic encoding and decoding solution:
[0220] This embodiment includes a semantic encoding and decoding method for multiple languages. This embodiment takes English transmission and Chinese reception as an example to describe a semantic encoding and decoding device for multiple languages.
[0221] Based on the above model, by changing the training method of the semantic encoder and decoder, a multilingual semantic encoding and decoding solution can be implemented. The key idea behind the fusion of classical encoding and semantic encoding is that the binary sequence of a natural language encoded sequentially can still retain its semantic association information. In fact, numbers themselves are a form of natural language, and the associations between the encoded numbers are still preserved. For example, for someone who knows Chinese but no English, translating Chinese into English is like encoding Chinese using the 26 English letters. Although the form is different, it still retains the association information unique to natural languages. The same is true for encoding Chinese using binary sequences.
[0222] Based on the above principles, when the training data at both ends of the semantic communication system is changed, the data encoding may change due to different natural language forms, but the semantic encoding will not be unable to extract its inherent semantic association information due to different encoding methods.
[0223] This embodiment takes English transmission and Chinese reception as an example to describe a semantic encoding and decoding solution for multiple languages. The system structure of this solution is shown in FIG17 .
[0224] Figure 17 is a schematic structural diagram of a multi-language semantic encoding and decoding system according to an embodiment of the present disclosure, including: a classic encoding module 171 (including but not limited to Huffman encoding, arithmetic encoding, and LZ encoding in the above-mentioned embodiments), a data-semantic information conversion module 172 (including but not limited to any data-semantic information conversion module in the above-mentioned embodiments), a semantic encoding module 173 (equivalent to any semantic encoding module in the above-mentioned embodiments), a channel 174, and a semantic decoding module 175. The modules can be one or more dedicated or general-purpose hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or can be one or more embedded or non-embedded dedicated or general-purpose software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above-mentioned functions.
[0225] Furthermore, based on the system shown in Figure 17, the English input refers to inputting English as a source of information during the communication process. Correspondingly, English is required as input corpus during the system training process.
[0226] The classic coding module refers to a Huffman coding module, an arithmetic coding module or an LZ coding module, and the data coding information is stored in the classic coding module.
[0227] The data-semantic information conversion module refers to the data encoding to semantic encoding mapping module in Examples 1-6. Different structures determine the design complexity of the overall system. The data-semantic information conversion module stores the mapping relationship between data encoding and semantic encoding.
[0228] The semantic encoding module refers to a semantic association information extractor built using Transformer Encoder, which can store English semantic encoding information.
[0229] The channel refers to a normal wireless or wired communication channel.
[0230] The semantic decoding module refers to a semantic association information assignment module built using Transformer Decoder, which can store semantic decoding information of Chinese.
[0231] The Chinese output means that during the communication process, the destination end can receive Chinese with the same semantics as the English sent by the source end.
[0232] The innovation of this embodiment is that, based on the fusion method of classical encoding and semantic encoding, bilingual corpus can be used when implementing the training system, thereby realizing a multilingual semantic communication system.
[0233] It should be noted that the above optional embodiments 1 to 7 can be implemented separately or in combination with each other, and the present disclosure does not limit this.
[0234] Furthermore, the above optional embodiments of the present disclosure have the following novel features:
[0235] 1) A semantic-data conversion module that integrates Huffman coding. Based on a fully connected network, this module processes classical codewords of unequal lengths, maps data redundancy to semantic redundancy, and outputs the one-hot vector required by the semantic coding module.
[0236] 2) An improvement to the embedding in the semantic encoder is proposed. Based on the embedding principle, it can be replaced by a fully connected layer, thereby removing the format restriction of the input data of the semantic encoding module.
[0237] 3) A "Many-Hot" vector creation method is proposed. Based on the One-Hot vector principle and the correspondence between data redundancy in classical encoding and semantic redundancy in semantic encoding, One-Hot vectors are implemented separately in a grouping manner. Finally, the "Many-Hot" vector is obtained by concatenating them. This vector can better reflect the correspondence between data redundancy and semantic redundancy.
[0238] 4) A padding mechanism-based rule for processing codewords of unequal lengths is proposed. Based on the padding mechanism and the Softmax operation principle, codewords of unequal lengths can be directly input into the semantic-data conversion module, eliminating the input data format restriction of the module.
[0239] 5) A semantic-data conversion module that integrates arithmetic coding and LZ coding. This module is based on a convolutional neural network and implements the processing of classical codewords of unequal lengths, completing the mapping from data redundancy to semantic redundancy, and outputting the one-hot vector required by the semantic coding module.
[0240] 6) A multilingual transmission system based on a classical encoding scheme is proposed. Based on the basic principles of Transformer and the nature of semantic association information, different languages can be used when training data and semantic integration modules, thus realizing a multilingual semantic communication system.
[0241] In summary, the optional embodiments of the present disclosure propose a multilingual encoding method that fuses data and semantics. This method can receive variable-length codewords from a classic source coding system and further compress them using the semantic redundancy contained in contextual information. This achieves the combined manipulation of data and semantic redundancy, integrating the classic source coding system with the semantic coding system. At the same time, the semantic decoding module at the destination can remain unchanged, reducing the overall design complexity of the system.
[0242] In this embodiment, a vector matrix determination system is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.
[0243] FIG18 is a block diagram of a system for determining a vector matrix according to an embodiment of the present disclosure. As shown in FIG18 , the system for determining a vector matrix includes:
[0244] an encoding module 182 configured to encode a message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single codeword or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, wherein the codewords in the regular codeword matrix have the same codeword length, and the codewords in the irregular codeword matrix have different codeword lengths, wherein the codeword matrix is a three-dimensional codeword matrix;
[0245] The data semantic conversion module 184 is connected to the encoding module and is configured to convert the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0246] Through the above system, the message to be transmitted is encoded into a codeword matrix; the codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length. In other words, by encoding the message to be transmitted into a three-dimensional codeword matrix and converting the obtained codeword matrix into a vector matrix with the same codeword length, the technical problem that existing semantic communication systems in the related art do not involve classical data coding and require the transmitted sentences to be of the same length, which in turn causes the semantic communication system to be unable to integrate classical communication, is solved. The technical effect of integrating classical coding with the semantic coding of existing semantic communication systems is achieved, and by integrating classical coding and semantic coding, sentences of different lengths to be transmitted can be processed.
[0247] In an exemplary embodiment, the system further includes: a semantic encoding module, connected to the data semantic conversion module, configured to replace the embedding layer of the semantic encoding module with a first fully connected network, and receive the vector matrix through the first fully connected network.
[0248] In an exemplary embodiment, the fully connected network group includes multiple fully connected networks with different input lengths and the same output lengths, the second dimension size of the two-dimensional parameters of the multiple fully connected networks is related to the number of words in the vocabulary corresponding to the natural language used when training the fully connected networks, and the vector matrix serves as the input of the semantic encoding module, wherein the vector matrix is a three-dimensional vector matrix.
[0249] In an exemplary embodiment, the encoding module 182 is further configured to perform at least one of the following: encoding the message to be transmitted into a codeword matrix through a Huffman encoding algorithm; encoding the message to be transmitted into a codeword matrix through an arithmetic encoding algorithm; encoding the message to be transmitted into a codeword matrix through an LZ encoding algorithm; encoding the message to be transmitted into a codeword matrix through a Huffman encoding algorithm, and truncating the codeword matrix according to a target length to form a lossy encoded codeword matrix.
[0250] That is, the message to be transmitted is encoded into a codeword matrix using a classical coding algorithm. The selected classical coding algorithms include but are not limited to the Huffman coding algorithm, the arithmetic coding algorithm, and the LZ coding algorithm. Optionally, the codeword matrix obtained by the Huffman coding algorithm can be truncated according to a certain length (i.e., the target length) to form a lossy coded codeword matrix.
[0251] In an exemplary embodiment, the data semantic conversion module 184 is further configured to obtain the codeword length of each codeword in the codeword matrix; and determine the codeword length as the first dimension size of the two-dimensional parameter of the fully connected network corresponding to each codeword.
[0252] After the message to be transmitted is input into the classic encoding module using the classic encoding algorithm, the classic encoding module converts the message to be transmitted and outputs it as a codeword matrix. In general, the output codeword matrix includes multiple codewords with different codeword lengths. However, it is also possible to obtain a codeword matrix including multiple codewords of the same length or a codeword matrix with only one codeword.
[0253] An optional embodiment of the present disclosure is to merge the classical coding module with the semantic coding module of the semantic communication system, and use the data-semantic information conversion module to convert the codeword matrix output by the classical coding module into a vector matrix that can be input into the classical coding module. Optionally, the data-semantic information conversion module may include a fully connected network group, which is used to convert the codeword matrix obtained by the output of the classical coding module into a vector matrix with the same codeword length, so as to facilitate input into the semantic communication system. Specifically, the fully connected network group includes a plurality of fully connected networks, and the plurality of fully connected networks may be the same as the number of codewords in the codeword matrix, and the first dimension size of the two-dimensional parameters of each of the plurality of fully connected networks is: the codeword length of the codeword to be processed corresponding to each fully connected network, for example, the first dimension size of the fully connected network corresponding to the codeword of length 1 is 1, the first dimension size of the fully connected network corresponding to the codeword of length 2 is 2, and so on.
[0254] In an exemplary embodiment, the data semantic conversion module 184 is further configured to input each codeword in the codeword matrix into the fully connected network corresponding to each codeword, and determine the vector A corresponding to each codeword in the vector matrix by the following formula: A=W i h+b i ;
[0255] Wherein, h represents each codeword in the codeword matrix, W i and b i represents the two-dimensional parameters of the fully connected network corresponding to the i-th codeword in the codeword matrix, where W i and b i The first dimension size of each is the codeword length of the i-th codeword; the vector matrix is obtained according to the position of each codeword in the codeword matrix and the vector A corresponding to each codeword.
[0256] It can be understood that the fully connected network converts the codewords in the codeword matrix into vectors in the vector matrix using the above formula. Specifically, each codeword in the codeword matrix is input into the corresponding fully connected network, and each codeword is converted into a vector corresponding to each codeword using the above formula. All codewords in the codeword matrix are converted into corresponding vectors, and the obtained vectors are combined into a vector matrix according to the position of each codeword in the codeword matrix.
[0257] The two-dimensional parameter W in the above formula i and b iThe size of the second dimension is related to the size of the vocabulary used in training, and the size of the vocabulary used in training is the number of words in the vocabulary corresponding to the natural language used in training. In an optional embodiment, if the language used in training is Chinese and there are 7000 words in the corresponding vocabulary, the size of the second dimension of the two-dimensional parameters of the multiple fully connected networks included in the fully connected network group is 7000. It should be noted that the obtained vector matrix is: a three-dimensional vector matrix with a fixed dimension, and the size of the third dimension is related to the size of the vocabulary used in training, and the size of the vocabulary used in training is the number of words in the vocabulary corresponding to the natural language used in training. For example, when the size of the second dimension of the two-dimensional parameter of the fully connected network group is 7000, the size of the third dimension of the vector matrix is 7000.
[0258] It should be noted that the above "7000" is only used as an example and is not used to limit the present disclosure. By analogy, the language used in training is not limited to Chinese, and the size of the vocabulary used in training is not limited to 7000 words, but is related to the actual size of the vocabulary corresponding to the language type used in training.
[0259] Since the embedding layer of the semantic encoding module requires one-hot input vectors, it's important to note that one-hot vectors are a method for converting discrete, categorical data into continuous vector representations. In a one-hot vector, each category is encoded as a vector where only one dimension takes a value of 1, and all other dimensions are 0. One-hot vectors can conveniently convert any number of different labels into a fixed-length, dense, low-dimensional vector format. These vectors are independent of each other and have no inherent relationships. Therefore, many machine learning algorithms often encode input data into one-hot vectors for processing.
[0260] Therefore, after converting the codeword matrix to a vector matrix, optionally:
[0261] 1) The vector matrix can be further converted into a One-Hot input vector:
[0262] In an exemplary embodiment, the data semantic conversion module 184 is further configured to obtain a probability vector corresponding to each third-dimensional vector of the vector matrix; determine a target vector based on the probability vector, wherein the target vector is a vector that sets the maximum element in each third-dimensional vector to the first value; and combine multiple target vectors into a target matrix.
[0263] In an exemplary embodiment, the data semantic conversion module 184 is further configured to determine the noise data corresponding to each third-dimensional vector based on the probability vector; and determine the target vector based on the noise data and the probability vector, wherein the noise data is used to make the target vector not a constant value.
[0264] It can be understood that splitting the vector matrix into multiple probability vectors according to the third dimension is essentially highlighting the maximum value in each probability vector, that is, the probability value of the probability vector corresponding to the maximum value in each third-dimensional vector is the largest among the probability values corresponding to all probability vectors included in the probability vector corresponding to the third-dimensional vector.
[0265] In order to make the target vector non-constant and thus allow the gradient to be returned, a reparameterization technique can be used:
[0266] In an optional embodiment, Gumbel noise is added to the probability vector. Specifically, the corresponding noise data is determined by the third-dimensional vector, that is, the noise data corresponding to each vector element in the third-dimensional vector is calculated; each probability element in the obtained probability vector is added to each corresponding noise data, and finally a target probability vector (i.e., the target vector in the above embodiment) is obtained. All target probability vectors are combined to obtain a target matrix, i.e., a one-hot input matrix, which can be input into the embedding layer in the semantic coding module.
[0267] 2) The Embedding layer in the semantic encoding module can be replaced, thereby removing the restriction of the one-hot input vector required by the Embedding layer:
[0268] In an exemplary embodiment, the data semantic conversion module 184 is further configured to directly input the vector matrix into the first fully connected network of the semantic encoding module, wherein the first dimension size of the two-dimensional parameters of the first fully connected network is related to the number of words in the vocabulary.
[0269] In an optional embodiment of the present disclosure, the first dimension of the two-dimensional parameters of the fully connected network that replaces the Embedding layer is the same as the third dimension of the target vector. This allows the vector matrix to be directly input into the semantic encoding module, avoiding the need to convert the vector matrix into a one-hot input vector.
[0270] In an exemplary embodiment, the data semantic conversion module 184 is further configured to group the vector matrix according to the third dimension direction to obtain multiple groups of sub-vector matrices; convert the multiple groups of sub-vector matrices into multiple sub-target matrices, and determine the target matrix based on the multiple sub-target matrices.
[0271] Grouping is performed in the direction of the third dimension of the vector matrix. Optional solutions include: grouping according to the maximum codeword length of the codeword matrix corresponding to the vector matrix, and the number of groups is related to the codeword length in the corresponding codeword matrix. A more specific optional solution is that the number of groups is related to the maximum codeword length minus the codeword length of the codeword corresponding to the vector in the vector matrix. For example, if the codeword length of the codeword corresponding to a certain vector in the vector matrix is 2, the maximum codeword length is 8, then 8-2, or 6, is the number of groups. If the vectors of the vector matrix cannot be divided equally according to the number of groups, the remaining elements can be discarded, or the group that cannot be divided equally can be used as the last group.
[0272] In an exemplary embodiment, the data semantic conversion module 184 is further configured to obtain a maximum codeword length in the codeword matrix; fill a second value at a target position in the codeword matrix according to the maximum codeword length to obtain a padded codeword matrix in which all codeword lengths are the maximum codeword lengths; and convert each codeword in the padded codeword matrix through a fully connected network group to obtain a vector matrix in which all codeword lengths are the same.
[0273] Optionally, the codeword matrix is first padded according to the maximum codeword length in the codeword matrix to obtain a padded codeword matrix with the same codeword length.
[0274] For example, suppose the message to be transmitted is S, which consists of B sentences of length L. The length L means that there are L words. When the classic encoding module is the Huffman encoding module, the codeword matrix output by the Huffman encoding module is a three-dimensional codeword matrix with a shape of [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal, and its maximum length is recorded as max(H ij ). When the codeword matrix is padded with the second value, a padded codeword matrix with the same codeword length is obtained, and its size is [B, L, max(H ij )].
[0275] The data-semantic conversion module using the padded codeword matrix as input can then use only one parameter of size [max(H ij ), CorpusLen], can convert the padded codeword matrix into a vector matrix. It should be noted that CorpusLen is related to the vocabulary size used during training, which is the number of words in the vocabulary corresponding to the natural language used during training. In an optional embodiment of the present disclosure, the structure of the data-semantic conversion module can be simplified.
[0276] In an exemplary embodiment, the data semantic conversion module 184 is further configured to obtain a two-dimensional codeword matrix corresponding to the encoding when the encoding is encoded by an arithmetic coding algorithm or an LZ coding algorithm; obtain a three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix, permute the first dimension and the second dimension of the three-dimensional codeword matrix to obtain an input matrix, wherein the third dimension size of the input matrix is the maximum codeword length of the codeword matrix; determine a second fully connected network in which the first dimension size of the two-dimensional parameter is the maximum codeword length and the second dimension size of the two-dimensional parameter is related to the number of words in the vocabulary, and convert each codeword in the input matrix through the second fully connected network to obtain a vector matrix with the same codeword length.
[0277] When the classic encoding module uses an arithmetic coding algorithm or an LZ coding algorithm, the codeword matrix obtained after converting the message to be transmitted is a two-dimensional codeword matrix. Furthermore, the data-semantic information conversion module corresponding to the arithmetic coding algorithm or the LZ coding algorithm can be selected to be composed of a convolutional layer, a reshape layer, and a fully connected layer (equivalent to the second fully connected network in the above embodiment).
[0278] Therefore, obtaining the three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix includes: when the two-dimensional codeword matrix has a shape of [B,H i ], where H i It represents the length of the arithmetic codeword obtained by encoding the i-th sentence, and the longest length is recorded as max(H i ), fill the two-dimensional code matrix to obtain a size of [B, max(H i )], a three-dimensional codeword matrix can be obtained based on the two-dimensional filling codeword matrix when certain conditions are met.
[0279] Optionally, under certain conditions, a three-dimensional codeword matrix is obtained based on the two-dimensional padded codeword matrix, including: passing the two-dimensional padded codeword matrix through a convolutional layer with a layer channel number L and a kernel size of 1*1, when the padding number P of the convolutional layer and the stride S satisfy the following relationship:
[0280] Among them, P H ,S H Indicates the number of padding and step length in the column direction, P W ,S W Indicates the number of padding and step size in the row direction. The three-dimensional matrix corresponding to the two-dimensional padding codeword matrix can be obtained through the above convolution layer, and the size is [L, B, max(H i )].
[0281] Furthermore, the first dimension and the second dimension of the obtained three-dimensional codeword matrix are permuted to obtain an input matrix, and input into the matrix with a parameter size of [max(H i ), CorpusLen], a second fully connected network can ultimately obtain a vector matrix of size [B, L, CorpusLen]. The scheme for inputting the vector matrix into the speech coding module can be referred to other embodiments of the present disclosure and will not be repeated here.
[0282] In the above exemplary embodiment, the first value is a positive number, and the second value is a value smaller than the first value. For example, if the first value is a positive number 2, the second value should be a value smaller than 2.
[0283] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a readable storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present disclosure.
[0284] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.
[0285] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.
[0286] An embodiment of the present disclosure further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.
[0287] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:
[0288] S1, encoding a message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codewords in the regular codeword matrix have the same codeword length, and the codewords in the irregular codeword matrix have different codeword lengths, wherein the codeword matrix is a three-dimensional codeword matrix;
[0289] S2, transforming the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0290] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0291] Optionally, in this embodiment, the above-mentioned electronic device can also be configured to execute the above-mentioned step S1 through a computer program, encoding the message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codewords in the regular codeword matrix have the same codeword length, and the codewords in the irregular codeword matrix have different codeword lengths, wherein the codeword matrix is a three-dimensional codeword matrix; S2, converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0292] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.
[0293] Obviously, those skilled in the art should understand that the modules or steps of the present disclosure described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices, they can be implemented using program code executable by the computing device, and thus, they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present disclosure is not limited to any particular combination of hardware and software.
[0294] The foregoing description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art will readily appreciate that various modifications and variations of the present disclosure are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present disclosure shall be included within the scope of protection of the present disclosure.
Claims
1. A method for determining a vector matrix, comprising: Encoding the message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codeword lengths of the codewords in the regular codeword matrix are all the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix; The codeword matrix is transformed through a fully connected network group to obtain a vector matrix with the same codeword length.
2. The method for determining a vector matrix according to claim 1, wherein: The fully connected network group includes multiple fully connected networks with different input lengths and the same output lengths, the second dimension size of the two-dimensional parameters of the multiple fully connected networks is related to the number of words in the vocabulary corresponding to the natural language used when training the fully connected networks, and the vector matrix is used as the input of the semantic encoding module, wherein the vector matrix is a three-dimensional vector matrix.
3. The method for determining a vector matrix according to claim 2, wherein: Before converting the codeword matrix through the fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: Obtaining the codeword length of each codeword in the codeword matrix; The codeword length is determined as the first dimension size of the two-dimensional parameters of the fully connected network corresponding to each codeword.
4. The method for determining a vector matrix according to claim 3, wherein: The codeword matrix is transformed through a fully connected network group to obtain a vector matrix with the same codeword length, including: Each codeword in the codeword matrix is input into the fully connected network corresponding to each codeword, and the vector A corresponding to each codeword in the vector matrix is determined by the following formula: A=W i h+b i ; Wherein, h represents each codeword in the codeword matrix, W i and b i represents the two-dimensional parameters of the fully connected network corresponding to the i-th codeword in the codeword matrix, where W i and b i The first dimension size of is the codeword length of the i-th codeword; The vector matrix is obtained according to the position of each codeword in the codeword matrix and the vector A corresponding to each codeword.
5. The method for determining a vector matrix according to claim 1, wherein: After converting the codeword matrix through the fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: Obtain a probability vector corresponding to each third-dimensional vector of the vector matrix; Determine a target vector according to the probability vector, wherein the target vector is a vector in which the maximum element in each third-dimensional vector is set to a first value; A plurality of said target vectors are combined into a target matrix.
6. The method for determining a vector matrix according to claim 5, wherein: Determining a target vector according to the probability vector includes: Determine the noise data corresponding to each third-dimensional vector according to the probability vector; The target vector is determined according to the noise data and the probability vector, wherein the noise data is used to make the target vector not a constant value.
7. The method for determining a vector matrix according to claim 2, wherein: After converting the codeword matrix through the fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: The vector matrix is directly input into a first fully connected network of a semantic encoding module, wherein a first dimension size of a two-dimensional parameter of the first fully connected network is related to the number of words in the vocabulary.
8. The method for determining a vector matrix according to claim 1, wherein: After converting the codeword matrix through the fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: Grouping is performed according to the third dimension direction of the vector matrix to obtain multiple groups of sub-vector matrices; The multiple groups of sub-vector matrices are converted into multiple sub-target matrices, and the target matrix is determined according to the multiple sub-target matrices.
9. The method for determining a vector matrix according to claim 1, wherein: The codeword matrix is transformed through a fully connected network group to obtain a vector matrix with the same codeword length, including: Obtaining a maximum codeword length in the codeword matrix; Fill a second value at a target position of the codeword matrix according to the maximum codeword length to obtain a filled codeword matrix whose codeword lengths are all the maximum codeword lengths; Each codeword in the padding codeword matrix is converted through the fully connected network group to obtain a vector matrix with the same codeword length.
10. The method for determining a vector matrix according to claim 2, wherein: include: In the case where the encoding is performed by an arithmetic encoding algorithm or an LZ encoding algorithm, obtaining a two-dimensional codeword matrix corresponding to the encoding; Obtaining a three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix, replacing the first dimension and the second dimension of the three-dimensional codeword matrix to obtain an input matrix, wherein the size of the third dimension of the input matrix is the maximum codeword length of the codeword matrix; A second fully connected network is determined in which the first dimension size of the two-dimensional parameter is the maximum codeword length, and the second dimension size of the two-dimensional parameter is related to the number of words in the vocabulary, and each codeword in the input matrix is converted through the second fully connected network to obtain a vector matrix with the same codeword length.
11. The method for determining a vector matrix according to claim 1, wherein: The message to be transmitted is encoded into a codeword matrix, which includes at least one of the following: Encoding the message to be transmitted into the codeword matrix by using a Huffman coding algorithm; Encoding the message to be transmitted into the codeword matrix by an arithmetic coding algorithm; The message to be transmitted is encoded into the codeword matrix by using the LZ encoding algorithm. The message to be transmitted is encoded into the codeword matrix through the Huffman coding algorithm, and the codeword matrix is truncated according to the target length to form a lossy coded codeword matrix.
12. The method for determining a vector matrix according to claim 5 or 9, wherein: The first value is a positive number, and the second value is a value smaller than the first value.
13. A vector matrix determination system, comprising: A coding module, configured to encode a message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codeword lengths of the codewords in the regular codeword matrix are all the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix; The data semantic conversion module is connected to the encoding module and is configured to convert the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
14. The vector matrix determination system according to claim 13, wherein: The system also includes: a semantic encoding module, which is connected to the data semantic conversion module and is configured to replace the embedding layer of the semantic encoding module with a first fully connected network, and receive the vector matrix through the first fully connected network.
15. A computer-readable storage medium, wherein a computer program is stored in the storage medium, wherein: The computer program is arranged to execute the method as claimed in any one of claims 1 to 12 when executed.
16. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the method according to any one of claims 1 to 12 through the computer program.
Citation Information
Patent Citations
Vector matrix determination method and system, storage medium and electronic device
CN119011687A
One-to-many demand analysis and recognition method based on deep learning
CN111158640A
Information source coding and decoding method and device based on context semantics
CN112836506A
One-to-many multi-user semantic communication model and communication method
CN115309869A
Semantic encoding method, semantic encoding device, semantic decoding method and semantic decoding device
CN115955297A
