Vector matrix determination method and system, storage medium and electronic device
By encoding the message into a three-dimensional codeword matrix and performing full-connection network conversion, the problem that semantic communication system cannot handle variable-length information is solved, the integration of classical coding and semantic coding is achieved, and the application scenario and data information compression potential is expanded.
Patent Information
- Application Number
- CN202310567601.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-18
- Publication Date
- 2024-11-22
AI Technical Summary
The existing semantic communication system does not involve classic data encoding, and requires the same length of transmitted sentences, which makes it impossible to integrate the classic communication system, limiting the application scenarios of semantic communication and the potential for data information compression.
By encoded the message to be transmitted into a three-dimensional codeword matrix and converting it with a fully connected network group, a vector matrix with the same codeword length is obtained, which solves the problem that the semantic communication system cannot process variable-length information, and realizes the classic Fusion of encoding and semantic encoding.
It realizes the integration of classical coding with semantic coding of semantic communication system, and can process sentences to be transmitted of different lengths, expanding the application scenarios of semantic communication and the potential for data information compression.
Smart Images

Figure CN119011687A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communications, and in particular, to a method, system, storage medium and electronic device for determining a vector matrix. Background Art
[0002] In the related field, a deep learning based semantic communication (DeepSC) model is commonly used in semantic communication systems. The structure diagram of this model is as follows: Figure 1 shown.
[0003] In addition to the above-mentioned DeepSC model, there are many semantic communication models, such as DeepSC-L, semantic communication system based on Universal Transformer, semantic communication system with variable coding rate, etc. These models require fixed-length One-Hot vectors as input for semantic encoding.
[0004] In addition, existing semantic communication systems generally use a machine learning (ML) framework and utilize the powerful nonlinear fitting capabilities of artificial neural networks (ANN) to extract the semantic redundancy behind information. However, due to the complexity of natural language, current semantic communication system implementations basically use a joint source-channel coding (JSCC) scheme, in which source coding and channel coding are performed jointly and cannot be separated. And because ANN can only process fixed-length data, such a system implementation cannot process variable-length information. This defect makes it difficult for semantic communication systems to integrate with classical communication systems, and also greatly limits the data information compression potential of semantic communication.
[0005] In summary, existing semantic communication systems do not involve classical data coding and require the length of the transmitted sentences to be the same. These transmission conditions greatly limit the application scenarios of semantic communication and are not conducive to the integration of semantic communication with classical communication. It should be noted that there is currently no case of the integration of semantic coding with classical coding.
[0006] Regarding the related technologies, the existing semantic communication systems do not involve classical data encoding and require the transmitted sentences to be of the same length, which leads to the problem that the semantic communication system cannot integrate classical communication. No effective solution has been proposed yet.
[0007] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the invention
[0008] The embodiments of the present application provide a method, system, storage medium and electronic device for determining a vector matrix, so as to at least solve the problem in the related art that the existing semantic communication systems do not involve classical data encoding and require the transmitted sentences to be of the same length, which leads to the inability of the semantic communication system to integrate classical communication.
[0009] According to one aspect of an embodiment of the present application, a method for determining a vector matrix is provided, comprising: encoding a message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, the codeword matrix includes: a regular codeword matrix, an irregular codeword matrix, the codeword lengths of the codewords in the regular codeword matrix are all the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix; converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword lengths.
[0010] In an exemplary embodiment, the fully connected network group includes multiple fully connected networks with different input lengths and the same output lengths, the second dimension size of the two-dimensional parameters of the multiple fully connected networks is related to the number of words in the vocabulary corresponding to the natural language used when training the fully connected networks, and the vector matrix serves as the input of the semantic encoding module, wherein the vector matrix is a three-dimensional vector matrix.
[0011] In an exemplary embodiment, before converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method also includes: obtaining the codeword length of each codeword in the codeword matrix; determining the codeword length as the first dimension size of the two-dimensional parameter of the fully connected network corresponding to each codeword.
[0012] In an exemplary embodiment, the codeword matrix is converted by a fully connected network group to obtain a vector matrix with the same codeword length, including: inputting each codeword in the codeword matrix into a fully connected network corresponding to each codeword, and determining a vector A corresponding to each codeword in the vector matrix by the following formula:
[0013] A=W i h+b i ;
[0014] Wherein, h represents each codeword in the codeword matrix, W i and b i represents the two-dimensional parameters of the fully connected network corresponding to the i-th codeword in the codeword matrix, where W i and b iThe first dimension size of is the codeword length of the i-th codeword; the vector matrix is obtained according to the position of each codeword in the codeword matrix and the vector A corresponding to each codeword.
[0015] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method also includes: obtaining a probability vector corresponding to each third-dimensional vector of the vector matrix; determining a target vector based on the probability vector, wherein the target vector is a vector that sets the maximum element in each third-dimensional vector to a first value; and combining multiple target vectors into a target matrix.
[0016] In an exemplary embodiment, determining a target vector based on the probability vector includes: determining noise data corresponding to each third-dimensional vector based on the probability vector; and determining the target vector based on the noise data and the probability vector, wherein the noise data is used to make the target vector not a constant.
[0017] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method also includes: directly inputting the vector matrix into a first fully connected network of a semantic encoding module, wherein the first dimension size of the two-dimensional parameters of the first fully connected network is related to the number of words in the vocabulary.
[0018] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: grouping according to the third dimension direction of the vector matrix to obtain multiple groups of sub-vector matrices; converting the multiple groups of sub-vector matrices into multiple sub-target matrices, and determining the target matrix based on the multiple sub-target matrices.
[0019] In an exemplary embodiment, the codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length, including: obtaining the maximum codeword length in the codeword matrix; filling a second value at a target position of the codeword matrix according to the maximum codeword length to obtain a filled codeword matrix with codeword lengths of the maximum codeword length; and each codeword in the filled codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length.
[0020] In an exemplary embodiment, the method includes: obtaining a two-dimensional codeword matrix corresponding to the code when the code is encoded by an arithmetic coding algorithm or an LZ coding algorithm; obtaining a three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix, replacing the first dimension and the second dimension of the three-dimensional codeword matrix to obtain an input matrix, wherein the third dimension size of the input matrix is the maximum codeword length of the codeword matrix; determining a second fully connected network in which the first dimension size of the two-dimensional parameter is the maximum codeword length and the second dimension size of the two-dimensional parameter is related to the number of words in the vocabulary, and converting each codeword in the input matrix through the second fully connected network to obtain a vector matrix with the same codeword length.
[0021] In an exemplary embodiment, encoding the message to be transmitted into a codeword matrix includes at least one of the following: encoding the message to be transmitted into a codeword matrix through a Huffman coding algorithm; encoding the message to be transmitted into a codeword matrix through an arithmetic coding algorithm; encoding the message to be transmitted into a codeword matrix through an LZ coding algorithm; encoding the message to be transmitted into a codeword matrix through a Huffman coding algorithm, and truncating the codeword matrix according to a target length to form a lossy coded codeword matrix.
[0022] In an exemplary embodiment, the first value is a positive number, and the second value is a value smaller than the first value.
[0023] According to another aspect of the embodiment of the present application, a vector matrix determination system is also provided, including: an encoding module, used to encode a message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codeword lengths of the codewords in the regular codeword matrix are all the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix; a data semantic conversion module, connected to the encoding module, used to convert the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0024] In an exemplary embodiment, the system further includes: a semantic encoding module, connected to the data semantic conversion module, for replacing the embedding layer of the semantic encoding module with a first fully connected network, and receiving the vector matrix through the first fully connected network.
[0025] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining the vector matrix when running.
[0026] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the vector matrix through the computer program.
[0027] Through the present application, the message to be transmitted is encoded into a codeword matrix; the codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length. In other words, by encoding the message to be transmitted into a three-dimensional codeword matrix and converting the obtained codeword matrix into a vector matrix with the same codeword length, the technical problem that the existing semantic communication systems in the related art do not involve classical data coding and require the transmitted sentences to be of the same length, which leads to the inability of the semantic communication system to integrate classical communication, is solved. The technical effect of integrating classical coding with the semantic coding of the existing semantic communication system is achieved, and then by integrating classical coding and semantic coding, sentences to be transmitted of different lengths can be processed. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The exemplary embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0029] Figure 1 It is a schematic diagram of the DeepSC model structure in related technologies;
[0030] Figure 2 It is a hardware structure block diagram of a computer terminal of the method for determining a vector matrix in an embodiment of the present application;
[0031] Figure 3 is a flowchart of a method for determining a vector matrix according to an embodiment of the present application;
[0032] Figure 4 It is a schematic diagram of the architecture of a semantic coding system in the related art;
[0033] Figure 5 Schematic diagram (I) showing the difference between classical source coding and semantic coding according to an embodiment of the present application;
[0034] Figure 6 Schematic diagram (II) showing the difference between classical source coding and semantic coding according to an embodiment of the present application;
[0035] Figure 7 Schematic diagram (III) showing the difference between classical source coding and semantic coding according to an embodiment of the present application;
[0036] Figure 8Schematic diagram (IV) showing the difference between classical source coding and semantic coding according to an embodiment of the present application;
[0037] Fig. 9 Schematic diagram (V) showing the difference between classical source coding and semantic coding according to an embodiment of the present application;
[0038] Fig.10 is a schematic diagram of semantic information according to an embodiment of the present application;
[0039] Fig.11 It is a schematic diagram of the structure of a system for fusing classic Huffman coding and semantic coding according to an embodiment of the present application;
[0040] Fig.12 It is a structural diagram of a system for fusing classic Huffman coding and improved semantic coding according to an embodiment of the present application;
[0041] Fig.13 It is a structural diagram of the classic Huffman coding and semantic coding fusion system according to the embodiment of the present application after adding a "Many-Hot" vector conversion module;
[0042] Fig.14 It is a structural diagram of a system for fusing classic Huffman coding and semantic coding using a padding mechanism according to an embodiment of the present application;
[0043] Fig.15 is a structural diagram of an arithmetic coding and semantic coding fusion system according to an embodiment of the present application;
[0044] Fig.16 is a structural diagram of an LZ coding and semantic coding fusion system according to an embodiment of the present application;
[0045] Fig.17 is a structural diagram of a multi-language semantic encoding and decoding system according to an embodiment of the present application;
[0046] Fig.18 It is a structural block diagram of a system for determining a vector matrix according to an embodiment of the present application. DETAILED DESCRIPTION
[0047] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0048] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0049] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 2 : is a hardware structure block diagram of a computer terminal for determining a vector matrix according to an embodiment of the present application. Figure 2 As shown, the computer terminal may include one or more ( Figure 2 Only one is shown in the figure) processor 202 (processor 202 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD)) and a memory 204 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 206 and an input / output device 208 for communication functions. It can be understood by those skilled in the art that Figure 2 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 2 More or fewer components as shown, or with Figure 2 Equivalent functions or comparisons shown Figure 2 A different configuration with more features is shown.
[0050] The memory 204 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the vector matrix in the embodiment of the present application. The processor 202 executes various functional applications and data processing by running the computer program stored in the memory 204, that is, to implement the above method. The memory 204 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 204 may further include a memory remotely arranged relative to the processor 202, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0051] The transmission device 206 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 206 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 206 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0052] Figure 3 is a flow chart of a method for determining a vector matrix according to an embodiment of the present application, such as Figure 3 As shown, the steps of the method include:
[0053] Step S302, encoding the message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codeword lengths of the codewords in the regular codeword matrix are all the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix;
[0054] Step S304: transform the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0055] Through the above steps, the message to be transmitted is encoded into a codeword matrix; the codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length. That is to say, by encoding the message to be transmitted into a three-dimensional codeword matrix and converting the obtained codeword matrix into a vector matrix with the same codeword length, the technical problem that the existing semantic communication systems in the related technology do not involve classical data coding and require the transmitted sentences to be of the same length, which leads to the inability of the semantic communication system to integrate classical communication, is solved. The technical effect of integrating classical coding with the semantic coding of the existing semantic communication system is achieved, and then by integrating classical coding and semantic coding, sentences to be transmitted of different lengths can be processed.
[0056] In an exemplary embodiment, the fully connected network group includes multiple fully connected networks with different input lengths and the same output lengths, the second dimension size of the two-dimensional parameters of the multiple fully connected networks is related to the number of words in the vocabulary corresponding to the natural language used when training the fully connected networks, and the vector matrix serves as the input of the semantic encoding module, wherein the vector matrix is a three-dimensional vector matrix.
[0057] In an exemplary embodiment, encoding the message to be transmitted into a codeword matrix includes at least one of the following: encoding the message to be transmitted into a codeword matrix through a Huffman coding algorithm; encoding the message to be transmitted into a codeword matrix through an arithmetic coding algorithm; encoding the message to be transmitted into a codeword matrix through an LZ coding algorithm; encoding the message to be transmitted into a codeword matrix through a Huffman coding algorithm, and truncating the codeword matrix according to a target length to form a lossy coded codeword matrix.
[0058] That is, the message to be transmitted is encoded into a codeword matrix by a classical coding algorithm, and the selected classical coding algorithm includes but is not limited to the Huffman coding algorithm, the arithmetic coding algorithm, and the LZ coding algorithm. Optionally, the codeword matrix obtained by the Huffman coding algorithm can also be truncated according to a certain length (i.e., the target length) to form a lossy coded codeword matrix.
[0059] In an exemplary embodiment, before converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method also includes: obtaining the codeword length of each codeword in the codeword matrix; determining the codeword length as the first dimension size of the two-dimensional parameter of the fully connected network corresponding to each codeword.
[0060] After the message to be transmitted is input into the classic coding module using the classic coding algorithm, the classic coding module converts the message to be transmitted and outputs it into a codeword matrix. Generally, the output codeword matrix includes multiple codewords, and the codeword lengths of the codewords are not equal, but it is also possible to obtain a codeword matrix including multiple codewords with the same codeword length or a codeword matrix with only one codeword.
[0061] An optional embodiment of the present application is to merge the classic coding module with the semantic coding module of the semantic communication system, and use the data-semantic information conversion module to convert the codeword matrix output by the classic coding module into a vector matrix that can be input into the classic coding module. Optionally, the data-semantic information conversion module may include a fully connected network group for converting the codeword matrix obtained by the output of the classic coding module into a vector matrix with the same codeword length, so as to facilitate input into the semantic communication system. Specifically, the fully connected network group includes a plurality of fully connected networks, and the plurality of fully connected networks may be the same as the number of codewords in the codeword matrix, and the first dimension size of the two-dimensional parameters of each fully connected network in the plurality of fully connected networks is: the codeword length of the codeword to be processed corresponding to each fully connected network, for example, the first dimension size of the fully connected network corresponding to the codeword of length 1 is 1, and the first dimension size of the fully connected network corresponding to the codeword of length 2 is 2, and so on.
[0062] In an exemplary embodiment, the codeword matrix is converted by a fully connected network group to obtain a vector matrix with the same codeword length, including: inputting each codeword in the codeword matrix into a fully connected network corresponding to each codeword, and determining a vector A corresponding to each codeword in the vector matrix by the following formula:
[0063] A=W i h+b i ;
[0064] Wherein, h represents each codeword in the codeword matrix, W i and b i represents the two-dimensional parameters of the fully connected network corresponding to the i-th codeword in the codeword matrix, where W i and b i The first dimension size of is the codeword length of the i-th codeword; the vector matrix is obtained according to the position of each codeword in the codeword matrix and the vector A corresponding to each codeword.
[0065] It can be understood that the fully connected network converts the codewords in the codeword matrix into vectors in the vector matrix through the above formula. Specifically, each codeword in the codeword matrix is input into the corresponding fully connected network, and each codeword is converted into a vector corresponding to each codeword through the above formula. All codewords in the codeword matrix are converted into corresponding vectors, and the obtained vectors are combined into a vector matrix according to the position of each codeword in the codeword matrix.
[0066] The two-dimensional parameter W in the above formula i and b i The size of the second dimension is related to the size of the vocabulary used in training, and the size of the vocabulary used in training is the number of words in the vocabulary corresponding to the natural language used in training. In an optional embodiment, if the language used in training is Chinese, and there are 7000 words in the corresponding vocabulary, the size of the second dimension of the two-dimensional parameters of the multiple fully connected networks included in the fully connected network group is 7000. It should be noted that the obtained vector matrix is: a three-dimensional vector matrix with a fixed dimension, and the size of the third dimension is related to the size of the vocabulary used in training, and the size of the vocabulary used in training is the number of words in the vocabulary corresponding to the natural language used in training. For example, when the size of the second dimension of the two-dimensional parameters of the fully connected network group is 7000, the size of the third dimension of the vector matrix is 7000.
[0067] It should be noted that the above "7000" is only used as an example and is not used to limit the present application. By analogy, the language used in training is not limited to Chinese, and the size of the vocabulary used in training is not limited to 7000 words, but is related to the actual size of the vocabulary corresponding to the language type used in training.
[0068] Since the embedding layer of the semantic encoding module requires a One-Hot input vector, it should be noted that the One-Hot vector is a method of converting discrete, categorical data into a continuous vector representation. In a One-Hot vector, each category is encoded into a vector, in which only one dimension has a value of 1, and the other dimensions are 0. Using a One-Hot vector, any number of different labels can be easily converted into a fixed-length, dense, and low-dimensional vector format, and these vectors are independent of each other and have no intrinsic relationship. Therefore, in many machine learning algorithms, input data is often encoded into a One-hot vector format and processed.
[0069] Therefore, after converting the codeword matrix to a vector matrix, optionally:
[0070] 1) The vector matrix can be further converted into a One-Hot input vector:
[0071] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method also includes: obtaining a probability vector corresponding to each third-dimensional vector of the vector matrix; determining a target vector based on the probability vector, wherein the target vector is a vector that sets the maximum element in each third-dimensional vector to a first value; and combining multiple target vectors into a target matrix.
[0072] In an exemplary embodiment, determining a target vector based on the probability vector includes: determining noise data corresponding to each third-dimensional vector based on the probability vector; and determining the target vector based on the noise data and the probability vector, wherein the noise data is used to make the target vector not a constant.
[0073] It can be understood that splitting the vector matrix into multiple probability vectors according to the third dimension essentially highlights the maximum value in each probability vector, that is, the probability value of the probability vector corresponding to the maximum value in each third-dimensional vector is the largest among the probability values corresponding to all probability vectors included in the probability vector corresponding to the third-dimensional vector.
[0074] In order to make the target vector non-constant and allow the gradient to be returned, a reparameterization technique can be used:
[0075] In an optional embodiment, Gumbel noise is added to the probability vector, specifically: the corresponding noise data is determined by the third dimensional vector, that is, the noise data corresponding to each vector element in the third dimensional vector is calculated; each probability element in the obtained probability vector is added to each corresponding noise data, and finally the target probability vector (that is, the target vector in the above embodiment) is obtained. All the target probability vectors are combined to obtain a target matrix, that is, a One-Hot input matrix, which can be input into the Embedding layer in the semantic coding module.
[0076] 2) The Embedding layer in the semantic encoding module can be replaced, thereby removing the restriction of the One-Hot input vector required by the Embedding layer:
[0077] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method also includes: directly inputting the vector matrix into a first fully connected network of a semantic encoding module, wherein the first dimension size of the two-dimensional parameters of the first fully connected network is related to the number of words in the vocabulary.
[0078] In an optional embodiment of the present application, the first dimension size of the two-dimensional parameters of the fully connected network replacing the Embedding layer is the same as the third dimension size of the target vector. Therefore, the vector matrix can be directly input into the semantic encoding module, avoiding the need to convert the vector matrix into a one-hot input vector.
[0079] In an exemplary embodiment, after converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: grouping according to the third dimension direction of the vector matrix to obtain multiple groups of sub-vector matrices; converting the multiple groups of sub-vector matrices into multiple sub-target matrices, and determining the target matrix based on the multiple sub-target matrices.
[0080] According to the third dimension direction of the vector matrix, the optional scheme includes: grouping according to the maximum codeword length of the codeword matrix corresponding to the vector matrix, and the number of groups is related to the codeword length in the corresponding codeword matrix. A more specific optional scheme is that the number of groups is related to the maximum codeword length minus the codeword length of the codeword corresponding to the vector in the vector matrix. For example, if the codeword length of the codeword corresponding to a certain vector in the vector matrix is 2, the maximum codeword length is 8, and 8-2, that is, 6, is the number of groups. If the vectors of the vector matrix cannot be divided equally according to the number of groups, the remaining elements can be discarded, or the groups that cannot be divided equally can be used as the last group.
[0081] In an exemplary embodiment, the codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length, including: obtaining the maximum codeword length in the codeword matrix; filling a second value at a target position of the codeword matrix according to the maximum codeword length to obtain a filled codeword matrix with codeword lengths of the maximum codeword length; and each codeword in the filled codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length.
[0082] Optionally, the codeword matrix is first padded according to the maximum codeword length in the codeword matrix to obtain a padded codeword matrix with the same codeword length.
[0083] For example, suppose the message to be transmitted is S, and the message S consists of B sentences of length L, where L means there are L words. When the classic encoding module is the Huffman encoding module, the codeword matrix output by the Huffman encoding module is a three-dimensional codeword matrix with a shape of [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal, and its maximum length is recorded as max(H ij). When the codeword matrix is filled with the second value, a filled codeword matrix with the same codeword length is obtained, and its size is [B, L, max(H ij )].
[0084] The data-semantic conversion module using the padded codeword matrix as input can then use only one parameter size of [max(H ij ), CorpusLen], the fill codeword matrix can be converted into a vector matrix. It should be noted that CorpusLen is related to the vocabulary size used in training, and the vocabulary size used in training is the number of words in the vocabulary corresponding to the natural language used in training. In an optional embodiment of the present application, the structure of the data-semantic conversion module can be made simpler.
[0085] In an exemplary embodiment, the method includes: obtaining a two-dimensional codeword matrix corresponding to the code when the code is encoded by an arithmetic coding algorithm or an LZ coding algorithm; obtaining a three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix, replacing the first dimension and the second dimension of the three-dimensional codeword matrix to obtain an input matrix, wherein the third dimension size of the input matrix is the maximum codeword length of the codeword matrix; determining a second fully connected network in which the first dimension size of the two-dimensional parameter is the maximum codeword length and the second dimension size of the two-dimensional parameter is related to the number of words in the vocabulary, and converting each codeword in the input matrix through the second fully connected network to obtain a vector matrix with the same codeword length.
[0086] When the classical coding module adopts an arithmetic coding algorithm or an LZ coding algorithm, the codeword matrix obtained after converting the message to be transmitted is a two-dimensional codeword matrix. Then, the data-semantic information conversion module corresponding to the arithmetic coding algorithm or the LZ coding algorithm can be selected to be composed of a convolutional layer, a Reshape layer, and a fully connected layer (equivalent to the second fully connected network of the above embodiment).
[0087] Therefore, obtaining the three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix includes: in the two-dimensional codeword matrix having a shape of [B,H i ], where H i It represents the length of the arithmetic codeword obtained by encoding the i-th sentence, and the longest length is recorded as max(H i ), fill the two-dimensional codeword matrix to obtain a size of [B, max(H i )], under certain conditions, a three-dimensional codeword matrix can be obtained based on the two-dimensional filling codeword matrix.
[0088] Optionally, under certain conditions, a three-dimensional codeword matrix is obtained according to the two-dimensional padding codeword matrix, including: passing the two-dimensional padding codeword matrix through a convolutional layer with a layer channel number L and a kernel size of 1*1, when the padding number P of the convolutional layer satisfies the following relationship with the step size S:
[0089]
[0090] Where P H ,S H Indicates the number of padding and stride in the column direction, P W ,S W Indicates the padding number and step size in the row direction. The three-dimensional matrix corresponding to the two-dimensional padding codeword matrix can be obtained through the above convolutional layer, with a size of [L, B, max(H i )].
[0091] Further, the first dimension and the second dimension of the obtained three-dimensional codeword matrix are permuted to obtain an input matrix, and input into the parameter size [max(H i ), CorpusLen], and finally a vector matrix of size [B, L, CorpusLen] can be obtained. The scheme of inputting the vector matrix into the semantic encoding module can refer to other embodiments of the present application, which will not be described here.
[0092] In the above exemplary embodiment, the first value is a positive number, and the second value is a value smaller than the first value. For example, when the first value is a positive number 2, the second value should be a value smaller than 2.
[0093] Next, the method for determining the vector matrix is further described in conjunction with the following embodiments.
[0094] It should be noted that if Figure 1 As shown in Figure 2, the operation of the DeepSC model is as follows: the message that the source needs to transmit is marked as S, and the message S consists of B sentences with a length of L. The length L means that it has L words (English), then S can be expressed as:
[0095]
[0096] where {w ij ; i=1,2,...,Bj=1,2,...,L} represents the jth word in the i-th sentence.
[0097] The message S is then encoded as a One-Hot vector of size [B,L,CorpusLen], where CorpusLen represents the number of different words contained in the message, and then passes through an Embedding layer, which stores a parameter W of shape [CorpusLen,E].Embedding , after the Embedding layer, we get a word distributed vector E of size [B,E].
[0098] E=SW Embedding .
[0099] In natural language processing (NLP), word distributed vectors can represent the correlation between word meanings, that is, they can be regarded as codewords after considering the word meanings, that is, the first step of semantic encoding, representing the original message S with the most efficient physical carrier (word distributed vector).
[0100] Then we need to calculate the position information of each word in the word vector E. The calculation formula is as follows:
[0101]
[0102] Where pos = [1, 2, ..., L], n = [1, 2, ..., E]. And add it to the calculated word vector E element by element to get the new word vector that takes into account the position information, which is still represented by E here:
[0103]
[0104] in, Represents element-by-element addition.
[0105] Afterwards, the word distribution vector E that takes into account the position information is passed through the Transformer Encoder to calculate the correlation between them. Figure 4 As shown, the Transformer Encoder used in the semantic communication system contains three encoders (i.e. Figure 4 Encoder), the structure of each Encoder is as follows Figure 4 As shown in the dashed box on the right, it contains an 8-head attention layer (i.e. Figure 4 The multi-head attention layer in ) and two fully connected layers.
[0106] It should be noted that: Figure 4 The FNN in it is: Feedforward neural network (FNN for short).
[0107] The word vector E is first input into the multi-head attention layer. The 8-head attention layer divides the word vector into 8 equal parts, and then uses the following parameters respectively:
[0108] W Q = {W Q 1 , W Q2 , ..., W Q 8},
[0109] W K = {W K 1 , W K 2 , ..., W K 8},
[0110] W V = {W V 1 , W V 2 , ..., W V 8}.
[0111] Embed the original word vector into the subspace and get three feature vectors:
[0112] Q={Q 1 , Q 2 , ..., Q 8 = SW Q T ,
[0113] K={K 1 , K 2 , ..., K 8 = SW K T ,
[0114] V={V 1 , V 2 , ..., V 8 = SW V T .
[0115] Then, the three feature vectors are subjected to attention operation to obtain the output. The shape of the output vector O is consistent with the input word vector E:
[0116]
[0117] The output vector O is then normalized through the residual connection and layer, and finally the attention output H is obtained. The attention output also needs to pass through two fully connected layers:
[0118] Z=Re lu(OW1+b1)W2+b2.
[0119] The vector Z transformed by the fully connected layer is again subjected to residual connection and layer normalization, and finally the output vector M is obtained. Through the above steps, one encoder encodes the message S. Since three encoders are used here, the above process needs to be performed twice to obtain the final output, which is also recorded as M. Through the above attention mechanism operation, the second step of semantic encoding is completed, that is, a part of grammatical association information is extracted.
[0120] The output vector M is then first passed through a dense layer, which performs channel coding on it to obtain the channel transmission vector X:
[0121] X=Re lu(ZW Dense +b Dense ).
[0122] After vector X is transmitted through the AWGN channel, Y is obtained, and then decoded at the destination. The decoding process is basically the inverse process of encoding, so it will not be described in detail here.
[0123] Based on the above description, the optional embodiment of the present application combines the classic encoding scheme to develop a joint encoding module for data and semantics.
[0124] Specifically, in order to solve the technical problem that the existing semantic communication systems in the relevant technologies do not involve classical data coding and require the transmitted sentences to be of the same length, which leads to the inability of the semantic communication system to integrate classical communication, and to further improve the source coding in semantic communication, an optional embodiment of the present application proposes a semantic communication coding module for variable-length coding input. The input of the coding module can be the source coding of the classical coding system, such as Huffman coding or arithmetic coding, and the codewords of these classical codes are further optimized within the system so that they can be further optimized and improved according to the semantic redundant information implied by the context, and finally sent to the receiver through the channel on the air interface, while the semantic decoding system of the receiver can remain unchanged.
[0125] The following should be explained:
[0126] Semantic communication is a new communication paradigm that has emerged in recent years. Based on this paradigm, the communication system will not only consider the redundancy existing in the encoded data, that is, the data redundancy compressed by source coding and the data redundancy added by channel coding to combat noise, but also the redundancy existing in the natural language meaning represented by the encoded data, which is called semantic redundancy.
[0127] In classical source coding, only data redundancy is considered, such as Figure 5 As shown in , each black dot represents a codeword, and there is no correlation between them. However, the classical source coding does not consider the semantics represented by these codewords. In the semantic space, Figure 6As shown in the figure, the meanings represented by these code words may be related. For example, "cat" and "dog" in the figure have some common characteristics, such as they are small animals and can be kept as domestic pets. They can overlap in some contexts, such as Figure 7 , Figure 8 As shown. Depending on the context and the meaning of the codeword, it can be further compressed, but the correlation between natural languages is not like that of numbers. Natural languages are more complex and may be nonlinear, so the final diagram may be Fig. 9 .
[0128] Due to the complexity of natural language, it may be non-stationary, have memory, and be nonlinear, so it does not meet the additivity condition for the Shannon entropy formula to be valid. The simplest example is "quoting out of context". Therefore, the semantic communication system cannot be described by the classic Shannon model, which is why the semantic communication system is implemented using the ANN-based JSCC architecture. In order to better describe the data and semantics integrated coding module proposed in this application, it is first necessary to define the new concepts in the semantic communication system in detail.
[0129] Semantic information S: The semantic information of a message M is defined as the sum of the semantic information contained in the data e carrying the message M and the data e (where ) and the amount of semantic association information between them, superposition in the semantic space:
[0130]
[0131] Here It represents high-dimensional operations and is a temporary representation when the specific functional form of semantic feature information and semantic association information cannot be determined. Where e represents the set of individual data that make up the message M, which belongs to a physical carrier E, such as English or Chinese, that is,
[0132] Semantic association information function cor: It indicates the semantic information generated by the arrangement of multiple single data due to corresponding grammatical rules under the current physical carrier, which is expressed by the cor function.
[0133] Semantic feature information function λ: represents the remaining information that can characterize the data meaning characteristics in the semantic information after the associated information generated by the single data arrangement is extracted under the current physical carrier.
[0134] The above definition can be used with Fig.10 Simple representation. Semantic information is high-dimensional information. The simple diagram here only represents two dimensions. In fact, the semantic feature information and semantic association information may be high-dimensional, and the operations between them are also high-dimensional operations.
[0135] It should be noted that the semantic association information here (at least) has two characteristics:
[0136] The first feature is the meaning association information, that is, the association between word meanings. This type of information is similar to the word vector representation in deep learning, such as Word2Vec.
[0137] Another feature is grammatical association information, that is, the association between contexts. This type of information is similar to the language model in natural language processing.
[0138] The embodiment of the present application considers that semantic encoding is divided into two steps: the first step is to transform the message M, so that the same semantics can be represented using the most efficient physical carrier according to the meaning association information, and "most efficient" means the least amount of data information. The second step is to extract the grammatical association information contained in the information representation after the transformation, so that the remaining grammatical association information is sufficient for decoding.
[0139] Optionally, the above embodiment is described below in conjunction with specific embodiments:
[0140] Embodiment 1: Huffman coding and semantic coding fusion solution
[0141] Optional embodiment 1 of the present application relates to a method for fusing Huffman coding and semantic coding, and in particular to a module for converting classic source coding into semantic coding in the field of communication, which is called a data-semantic information conversion module, and the above device can be applied to a sending end device. In this module, not only the data redundancy compression of the original message is considered, but also it is mapped with the semantic redundancy, so that Huffman codes of unequal length can also be semantically encoded and compressed with semantic redundancy.
[0142] This embodiment includes a Huffman coding and semantic coding fusion device, the system structure is as follows Fig.11 As shown, the device includes: a Huffman encoding module 111, a data-semantic information conversion module 112, and a semantic encoding module 113. The above modules can be one or more special or general hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or one or more embedded or non-embedded special or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above functions.
[0143] Furthermore, the main function of the Huffman encoding module is to perform Huffman encoding. Assuming that the incoming message is S, the message S consists of B sentences with a length of L, where the length L means that there are L words (English is used as an example here), then S can be expressed as:
[0144]
[0145] where {w ij ; i=1,2,...,Bj=1,2,...,L} represents the jth word in the i-th sentence.
[0146] The message matrix S first passes through the Huffman encoding module, which is mainly composed of a Huffman tree. This module reduces the number of bits required to represent the data, and can use codewords of different lengths to represent each word in the message matrix S. That is, the message matrix S is expanded into a Huffman codeword matrix H, whose shape is [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal. For ease of description, the longest Huffman codeword length is recorded as max(H ij ).
[0147] Furthermore, the data-semantic conversion module is to solve the contradiction between semantic association information and data redundancy information. The data-semantic conversion module jointly considers Huffman coding and semantic coding, realizes feature mapping of the two, and performs balanced coding. Furthermore, the module includes two submodules: a redundant mapping module and a One-Hot vector generation module.
[0148] 1) The redundant mapping module establishes a mapping relationship between data coding and semantic coding. The mapping relationship is composed of a group of fully connected layers (equivalent to the fully connected network group in the above embodiment), and the parameter size of each fully connected layer in the group (equivalent to the fully connected network in the above embodiment) is proportional to the length H of the codeword after Huffman coding. ij And the preset output length CorpusLen. CorpusLen is a positive integer. The final output matrix is recorded as A code .
[0149] Specifically, regarding the redundant mapping module:
[0150] In general, the more frequently a word appears in message S, the shorter its Huffman codeword is, but the more structured information it may contain with other words in natural language, and the wider the semantic range it can represent, that is, the richer its semantic association information is, and the more redundant semantic information may need to be extracted in the semantic encoding stage. Conversely, the less frequently a word appears, the longer its Huffman codeword is, but the less semantic association information it may have, and the less redundant semantic information may need to be extracted in the semantic encoding stage. The data-semantic conversion module will not only consider the data information redundancy of message S, but also the semantic association information redundancy of message S, and then try to establish a mapping between the two to find a balanced coding point.
[0151] After the Huffman codeword matrix is input into the data-semantic information conversion module, it will be converted into a codeword that takes into account structural information (that is, semantic association information) according to its codeword length using a fully connected layer. The size of the parameters contained in the fully connected layer here is determined by the length of the Huffman codeword. Huffman codewords h of different lengths will be mapped to vectors of the same dimension:
[0152] A=W Huffman i h+b Huffman i ,
[0153] Among them, W Huffman i With b Huffman i Represents parameters of different dimensions, where the first dimension is the same as the length of the corresponding Huffman codeword, for example, the length H ij The Huffman codeword is 1, and the input fully connected layer parameter size is [1, CorpusLen], and the length is H ij The Huffman codeword with a value of 2 has a parameter size of [2,CorpusLen] for the input full connection layer, and so on. The longest Huffman codeword has a length of max(H ij ).
[0154] For a length of H ij Although the Huffman code lengths are different, after passing through the data-semantic information conversion module, the vector matrix A composed of vector A is obtained. code The size of is [B, L, CorpusLen], and the length of each codeword is expanded to CorpusLen, which represents the structured relationship between it and other codewords. Thus, the data redundancy is mapped to the semantic association information redundancy through the data-semantic information conversion module.
[0155] 2) The one-hot vector generation module can output a vector B in which one element is 1 and the other elements are all 0. In addition, the reparameterization technique is used. Further, the reparameterization technique can meet the gradient back propagation conditions during network training. In order to illustrate how this technique solves the gradient propagation problem and the data changes in this embodiment, an example is given here. For example, A code The third dimension vector in the matrix is denoted as π, and its length is CorpusLen. It is transformed as follows:
[0156] Specifically:
[0157] Pair vector matrix A code Do Softmax and correspond it to the One-Hot vector input of the original semantic encoder (i.e., the semantic encoding module). Here, the argmax function can be directly used to select the largest element in each row of A and set it to 1, thereby forming a new One-Hot vector B. However, it should be noted that since the argmax function itself is not differentiable, the gradient cannot be propagated backward during the training process, causing the parameters in the data-semantic information conversion module to be unable to be updated. Therefore, a reparameterization technique can be used here, such as the Gumbel-Softmax Trick to implement the above process. The specific optional implementation process is:
[0158] A code The third dimension vector in the matrix is denoted as π, and its length is CorpusLen. First, Softmax is performed on the vector π to obtain the probability of each element appearing:
[0159]
[0160] π i The probability of occurrence p i Add Gumbel noise:
[0161]
[0162] Among them G i is called Gumbel noise:
[0163] G i =-log(-log(U i )),U i ~U(0,1),
[0164] This noise is used to make the returned results Not fixed. In the end we get The vector is a One-Hot vector. However, the argmax function cannot be derived in this formula, so the Softmax function is used instead:
[0165]
[0166] Among them, τ is called the scaling factor. When τ is small enough, x π It is close to a One-Hot vector and can be reversely derived.
[0167] According to the above options, x πi The vector composed of , which can be used as the input of the semantic encoding module.
[0168] Furthermore, the new One-Hot vector B is finally passed through the Embedding of the semantic encoding module to obtain a vector matrix of size [B, L, E] for subsequent operations of semantic encoding.
[0169] After the above operations, during the joint training of the semantic codec, the Huffman coding module will store data redundancy information, the data-semantic information conversion module will store the correspondence between data redundancy and semantic association information, and the semantic coding module will store the extracted semantic association information, thereby completing the integrated encoding process of semantics and data.
[0170] The innovation of optional embodiment 1 of the present application lies in that a data-semantic conversion module is constructed to represent the mapping between data redundancy and semantic redundancy, which can be considered jointly during the encoding process to obtain the optimal encoding; multiple fully connected networks are combined to form a network group, which realizes unified processing of classic codewords of different coding lengths; and re-parameterization techniques are used to convert classic coding representations that cannot be back-propagated into probabilities, so that the overall system can be optimized through a back-propagation algorithm.
[0171] Embodiment 2: Direct fusion scheme of Huffman coding and improved semantic coding
[0172] This embodiment includes a method for improving the design of a semantic encoding module, thereby further optimizing the design of a data-semantic information conversion module, thereby reducing the complexity of the overall device implementation.
[0173] The system structure of the optional embodiment 2 of the present application is as follows Fig.12 As shown:
[0174] Fig.12 is a schematic diagram of the structure of the fusion system of the classic Huffman coding and the improved semantic coding according to the embodiment of the present application. Fig.11 Based on this, the Embedding layer in the original semantic encoding module is replaced with a fully connected layer to connect it with the previous data-semantic information conversion module. Fig.12The Huffman coding module 111, the data-semantic information conversion module 112, and the improved semantic coding module 123 are included. The modules can be one or more dedicated or general hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above functions.
[0175] Optional embodiment 2 of the present application is obtained on the basis of optional embodiment 1 of the present application, and differs from optional embodiment 1 of the present application in that:
[0176] The data-semantic conversion module only includes redundant mapping modules, and there is no need to use reparameterization techniques to transform the matrix A. code Transform the vector in and directly output the matrix A code .
[0177] The Embedding layer of the semantic encoding module is replaced by a fully connected layer, so the matrix A can be used directly code as input.
[0178] Optionally, the following specifically describes the execution process of the improved solution in the optional embodiment 2 of the present application:
[0179] In the optional embodiment 2 of the present application, it is also assumed that the incoming message is S, and the message S consists of B sentences with a length of L, where the length L means that it has L words (English), expressed as:
[0180]
[0181] After the message matrix S passes through the Huffman encoding module and the data-semantic conversion module, the vector matrix A of size [B, L, CorpusLen] is finally obtained. code , but due to A code It is not a One-Hot vector itself, so it cannot be directly used as input by the semantic encoding module.
[0182] The essential reason is that the Embedding layer requires a One-Hot input vector, which can be regarded as an input with a probability of 1 in only one case. The Embedding layer is used to reduce the amount of calculation. Without considering the amount of calculation, a fully connected layer with a parameter size of [CorpusLen, E] can be directly used to replace the Embedding layer. In this way, the vector matrix A code It can be directly used as the input of the semantic encoding module. After passing through the fully connected layer, a vector matrix of size [B, L, E] is obtained for subsequent operations of semantic encoding.
[0183] The innovation of optional embodiment 2 of the present application lies in that, based on the Embedding implementation principle, a fully connected layer is used instead of the Embedding layer to store word distributed vectors, thereby expanding its input data requirements. Therefore, there is no need to use a One-Hot vector generation module in the data-semantic information conversion module, which not only simplifies the design structure, but also incorporates all semantic association information corresponding to the data encoding into the calculation scope of the fusion coding system.
[0184] Embodiment 3: Huffman coding transformation and semantic coding fusion solution
[0185] Optional embodiment 3 of the present application includes an improved "Many-Hot" vector conversion module, which converts the output of the data-semantic information conversion module into a legal input of the semantic coding module, while taking into account the anti-correlation relationship between data coding and semantic coding, making the overall design of the system more reasonable.
[0186] Fig.13 It is a structural diagram of the classic Huffman coding and semantic coding fusion system according to the embodiment of the present application after adding the "Many-Hot" vector conversion module, including: Huffman coding module 111, data-semantic information conversion module 112, semantic coding module 113, "Many-Hot" vector conversion module 134. The module can be one or more dedicated or general hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above functions.
[0187] Optional embodiment 3 of the present application is obtained on the basis of embodiment 1, and differs from embodiment 1 in that a “Many-Hot” vector conversion module is added after the data-semantic information conversion module.
[0188] The "Many-Hot" vector conversion module establishes the possible anti-correlation mapping relationship between data encoding and semantic encoding. And the output matrix A of the data-semantic conversion module can be code After further transformation, it becomes a "Many-Hot" vector.
[0189] Furthermore, the anti-correlation mapping relationship means that the more frequently a word appears in the original message S, the shorter the Huffman codeword is, the more information that may be required for semantic coding is contained, and the greater the weight of consideration for the word in semantic coding is.
[0190] Furthermore, the “Many-Hot” vectors are grouped, and the One-Hot vector generation module operation described in Example 1 is performed on each group.
[0191] The grouping operation is performed according to the matrix A code The third dimension, realized for each vector, corresponds to the third dimension of the Huffman codeword matrix, i.e., the dimension storing the codeword. The number of groups is inversely related to the length of the corresponding Huffman codeword, i.e., if the corresponding codeword length is H ij , then the number of groups is max(H ij )-H ij If it cannot be equally divided, the remaining elements can be discarded, or the group that cannot be equally divided can be used as the last group.
[0192] The One-Hot vector generation module operation may pass the grouping vectors through the One-Hot vector generation module in Embodiment 1 respectively, and then concatenate them.
[0193] Optionally, the execution process of optional embodiment 3 of the present application is described in detail below:
[0194] Also assume that the incoming message is S, which consists of B sentences of length L, where the length L means that it has L words (English), expressed as:
[0195]
[0196] The message matrix S first passes through the Huffman encoding module to obtain the Huffman codeword matrix H, whose shape is [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal, and the longest length is recorded as max(H ij ).
[0197] After that, the codeword matrix H passes through the data-semantic conversion module to obtain a vector matrix A of size [B, L, CorpusLen] code , but due to A code It is not a One-Hot vector itself, so it cannot be directly used as input by the semantic encoding module.
[0198] At the same time, the more frequently a word appears in the original message S, the shorter the Huffman codeword is, but it may contain more semantically related information, so semantic coding needs to be considered more. Conversely, the less frequently a word appears, the longer the Huffman codeword is, but it may contain less semantically related information, so semantic coding needs to be considered less. Therefore, there may be an anti-correlation relationship between data redundancy and semantic redundancy.
[0199] Therefore, in order to reflect the anti-correlation between data redundancy and semantic redundancy, and also to transform the vector matrix A code To become the legal input of the semantic encoding module, a "Many-Hot" vector conversion module is designed here, and its operation on data is divided into two steps.
[0200] First, the vector matrix A code , perform grouping operations on them, and grouping is performed on each vector according to its third dimension, which corresponds to the third dimension of the Huffman codeword matrix, that is, the dimension of storing codewords. The number of groups is inversely related to the corresponding Huffman codeword length, that is, if the corresponding codeword length is H ij , then the number of groups is max(H ij )-H ij In short, the number of groups is proportional to the original codeword length H ij If it is impossible to divide equally, the remaining elements can be discarded, or the group that cannot be divided equally can be used as the last group. This grouping method corresponds to the anti-correlation relationship between data redundancy and semantic redundancy.
[0201] Then, the grouped vectors are respectively subjected to the reparameterization technique described in the optional embodiment 1 of the present application to transform them into One-Hot vectors, and then these vectors are connected to finally obtain "Many-Hot" vectors, which can be used as legal input by the semantic encoding module.
[0202] The innovation of the optional embodiment 3 of the present application is that the vector matrix A representing the probability code Group encoding is performed, and finally a "Many-Hot" vector that can reflect different semantic redundant relationships is obtained as the input of the semantic encoding module.
[0203] Embodiment 4: Huffman coding and semantic coding fusion scheme using padding mechanism
[0204] Optional embodiment 4 of the present application includes an improved padding mechanism Huffman encoding method, so that it can be input into a data-semantic information conversion module that only includes a fully connected network, greatly simplifying the design complexity of the module and saving storage and computing resources.
[0205] Fig.14It is a structural diagram of a system for fusion of classic Huffman coding and semantic coding using a padding mechanism according to an embodiment of the present application, in which a Huffman coding module 111, a padding module 144, and a data-semantic information conversion module 142 are added before the original semantic coding module 113, thereby realizing the fusion of the classic coding and semantic coding systems. The module may be one or more dedicated or general hardware modules, separate or integrated chips, memories, etc. having corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software having the above functions.
[0206] This embodiment is obtained on the basis of the embodiment 1, and differs from the embodiment 1 in that a filling module is added before the data-semantic information conversion module.
[0207] The filling module can convert the codeword matrix H and output a regular matrix of the same length. The regular matrix is a matrix in which the elements that originally do not exist in the codeword matrix are filled with 0. The shape of the 0-filled matrix is [B, L, max(H ij )-H ij ], is an irregularly shaped matrix.
[0208] Optionally, the execution process of optional embodiment 4 of the present application is described in detail below:
[0209] Assume that the incoming message is S, which consists of B sentences of length L, where length L means it has L words (English is used as an example here), then S can be expressed as:
[0210]
[0211] where {w ij ; i=1,2,...,Bj=1,2,...,L} represents the jth word in the i-th sentence.
[0212] The message matrix S first passes through the Huffman encoding module to obtain the Huffman codeword matrix H, whose shape is [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal, and its maximum length is recorded as max(H ij ).
[0213] In order to reduce the complexity of system design and reduce storage space overhead, the Huffman codewords of unequal lengths in the codeword matrix H are filled with 0s to obtain codewords with the same length as max(H ij ) codeword, the codeword matrix is recorded as H', and its size is [B,L,max(Hij )]0The shape of the filled matrix is [B,L,max(H ij )-H ij ], is an irregularly shaped matrix.
[0214] The codeword matrix H' is then input into the data-semantic information conversion module, which consists of a fully connected network with a parameter size of [max(H ij ),CorpusLen], so the vector matrix A can be output code , whose size is [B,L,CorpusLen].
[0215] Finally, the vector matrix A can be converted into code Input into the semantic encoding module.
[0216] The innovation of optional embodiment 4 of the present application is that the filling mechanism can be used to remove the input restrictions of the data-semantic coding module. Previously, a regular matrix input was required, and the filling mechanism can use an irregular vector matrix as its input.
[0217] Example 5: Arithmetic Coding and Semantic Coding Fusion Solution
[0218] This embodiment includes a method for fusing arithmetic coding and semantic coding, so that arithmetic coding can also be fused with semantic coding.
[0219] Fig.15 It is a structural diagram of the arithmetic coding and semantic coding fusion system according to the embodiment of the present application. An arithmetic coding module 151 and a data-semantic information conversion module 152 are added before the original semantic coding module 113 to realize the fusion of the classical coding and semantic coding systems. The modules can be one or more dedicated or general hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above functions.
[0220] Optionally, the execution process of optional embodiment 5 of the present application is described in detail below:
[0221] Assume that the incoming message is S, which consists of B sentences of length L, where L means it has L words (in English), expressed as:
[0222]
[0223] The message matrix S is firstly reduced in the number of bits of the data through the arithmetic coding module, and each sentence in the message matrix S is represented by codewords of different lengths, and their lengths are not equal. The codeword matrix obtained after arithmetic coding is represented as H, and its shape is [B,H i ], where H i It represents the length of the arithmetic codeword obtained by encoding the i-th sentence, and the longest length is recorded as max(H i ).
[0224] The codeword matrix H is then passed through a data-semantic conversion module, which establishes a mapping between arithmetic coding and semantic coding.
[0225] The establishment step is divided into three steps: the first step is to calculate the longest length max(H i ) fills all codeword vectors with 0 to obtain a new codeword matrix H. The second step is to pass a convolutional layer with L channels and a kernel size of K×K. When the filling number P and the step size S meet certain conditions, the size of the codeword matrix H passing through the convolutional layer becomes [L, B, max(H i )]. In the third step, the codeword matrix H is dimensionally permuted and then passed through a fully connected layer with a parameter size related to the preset parameter CorpusLen. Finally, the matrix A with a size related to CorpusLen can be obtained. code .
[0226] Based on the establishment steps, an optional implementation scheme for establishing a mapping between arithmetic coding and semantic coding is described below:
[0227] In this module, firstly, according to the longest length max(H i ) fills all codeword vectors with 0 to obtain a new codeword matrix H, whose size is [B, max(H i )], and then pass through a convolutional layer with L channels and a kernel size of 1*1. When its padding number P and step size S satisfy the following relationship:
[0228]
[0229] Where P H ,S H Indicates the number of padding and stride in the column direction, P W ,S W Indicates the padding number and step size in the row direction. The size of the codeword matrix H after this convolutional layer becomes [L, B, max (H i )](equivalent to the input matrix in the above embodiment).
[0230] Change the size to [L,B,max(Hi )], the first and second dimensions of the codeword matrix are permuted, and then a parameter size of [max(H i ),CorpusLen], and finally we can get the input matrix A of size [B,L,CorpusLen] code (equivalent to the vector matrix in the above embodiment).
[0231] The matrix A code The input semantic coding module can refer to the solutions in optional embodiments 1, 2 and 3 of the present application, which will not be repeated here.
[0232] The semantic encoding module can perform semantic encoding on the input message, thereby completing semantic communication.
[0233] The innovation of optional embodiment 5 of the present application is that, based on the characteristics of convolutional neural networks, arithmetic coding is incorporated into the data and semantic fusion coding module.
[0234] Embodiment 6: LZ coding and semantic coding fusion solution
[0235] This embodiment includes a method for fusing LZ coding and semantic coding, so that LZ coding can also be fused with semantic coding. The system structure is as follows: Fig.16 shown.
[0236] Fig.16 It is a structural diagram of the LZ coding and semantic coding fusion system according to an embodiment of the present application. An LZ coding module 161 and a data-semantic information conversion module 162 are added before the original semantic coding module 113 to achieve the fusion of the classical coding and semantic coding systems. The modules can be one or more dedicated or general hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above functions.
[0237] This embodiment is obtained on the basis of Embodiment 5, and differs from Embodiment 5 in that: an LZ encoding module is used to replace an arithmetic encoding module.
[0238] The message matrix S First, the LZ encoding module reduces the number of bits used to represent the data. LZ encoding creates a new dictionary to encode different sentences, and the encoding lengths are not equal. The codeword matrix obtained after LZ encoding is represented as H, and its shape is [B,H i ], where H i Indicates i The length of the arithmetic codeword obtained by encoding the sentence, the longest length is recorded as max(Hi ).
[0239] The remaining modules are similar to those in Example 5.
[0240] The following specifically describes the implementation process of the LZ coding and semantic coding fusion method:
[0241] Optionally, assume that the incoming message is S, which consists of B sentences of length L, where the length L means that it has L words (in English), expressed as:
[0242]
[0243] The message matrix S first passes through the LZ encoding module. LZ encoding creates a new dictionary to encode different sentences, and the encoding lengths are not equal. The codeword matrix obtained after LZ encoding is represented as H, and its shape is [B,H i ], where H i It indicates the length of LZ codeword obtained by encoding the i-th sentence, and the longest length is recorded as max(H i ).
[0244] The subsequent encoding process is consistent with the process described in optional embodiment 5 of the present application, and will not be repeated here.
[0245] The innovation of optional embodiment 6 of the present application is that, based on the characteristics of convolutional neural networks, LZ coding is incorporated into the data and semantic fusion coding module.
[0246] Example 7: Multi-language semantic encoding and decoding solution
[0247] This embodiment includes a semantic encoding and decoding method for multiple languages. This embodiment takes English transmission and Chinese reception as an example to describe a semantic encoding and decoding device for multiple languages.
[0248] Based on the above model, by changing the training method of the semantic codec, a multi-language semantic coding and decoding scheme can be implemented. The key idea of the fusion of the above classical coding and semantic coding is that it is believed that the binary sequence of the natural language encoded in sequence can still retain its semantic association information. In fact, numbers themselves are also a natural language, and the encoded numbers still retain association information. For example, for a person who knows Chinese but does not know English at all, translating Chinese into English is to encode Chinese with 26 English letters. Although the form is different, it still retains the unique association information of natural language. The same is true for encoding Chinese in binary sequences.
[0249] Based on the above principles, when the training data at both ends of the semantic communication system is changed, the data encoding may change due to different natural language forms, but the semantic encoding will not be unable to extract its inherent semantic association information due to different encoding methods.
[0250] This embodiment takes English transmission and Chinese reception as an example to describe the semantic encoding and decoding solution for multiple languages. The system structure of this solution is as follows: Fig.17 shown.
[0251] Fig.17 It is a structural diagram of a multi-language semantic encoding and decoding system according to an embodiment of the present application, including: a classic encoding module 171 (including but not limited to Huffman encoding, arithmetic encoding, and LZ encoding in the above embodiments), a data-semantic information conversion module 172 (including but not limited to any data-semantic information conversion module in the above embodiments), a semantic encoding module 173 (equivalent to any semantic encoding module in the above embodiments), a channel 174, and a semantic decoding module 175. The modules can be one or more dedicated or general hardware modules, separate or integrated chips, memories, etc. with corresponding functions or technical features, or one or more embedded or non-embedded dedicated or general software, software modules, software codes, operating systems, or a combination of one or more hardware and software with the above functions.
[0252] Based on Fig.17 In the system shown, the English input refers to inputting English as a source of information during the communication process. Correspondingly, English is required as input corpus during the training process of the system.
[0253] The classic coding module refers to a Huffman coding module, an arithmetic coding module or an LZ coding module, and the data coding information is stored in the classic coding module.
[0254] The data-semantic information conversion module refers to the data coding to semantic coding mapping module in Examples 1-6. Different structures determine the design complexity of the overall system. The data-semantic information conversion module stores the mapping relationship between data coding and semantic coding.
[0255] The semantic encoding module refers to a semantic association information extractor built using Transformer Encoder, which can store English semantic encoding information.
[0256] The channel refers to a normal wireless or wired communication channel.
[0257] The semantic decoding module refers to a semantic association information assignment module built using Transformer Decoder, which can store the semantic decoding information of Chinese.
[0258] The Chinese output means that during the communication process, the destination end can receive Chinese with the same semantics as the English sent by the source end.
[0259] The innovation of this embodiment is that, based on the fusion method of classical encoding and semantic encoding, bilingual corpus can be used when implementing the training system, thereby realizing a multi-language semantic communication system.
[0260] It should be noted that the above optional embodiments 1 to 7 can be implemented separately or in combination with each other, and this application does not limit this.
[0261] Furthermore, the above optional embodiments of the present application have the following novel features:
[0262] 1) A semantic-data conversion module integrating Huffman coding is built on a fully connected network. This module processes classical codewords of unequal lengths, completes the mapping from data redundancy to semantic redundancy, and outputs the One-Hot vector required by the semantic coding module.
[0263] 2) An improvement on the Embedding in the semantic encoder is proposed. Based on the Embedding principle, it can be replaced by a fully connected layer, thereby removing the format restriction of the input data of the semantic encoding module.
[0264] 3) A “Many-Hot” vector creation method is proposed. Based on the One-Hot vector principle, the correspondence between data redundancy of classical coding and semantic redundancy of semantic coding, One-Hot vectors are implemented separately by grouping, and finally connected to obtain a “Many-Hot” vector, which can better reflect the correspondence between data redundancy and semantic redundancy.
[0265] 4) A regular processing method for codewords of unequal length based on a padding mechanism is proposed. Based on the padding mechanism and the Softmax operation principle, codewords of unequal length can be directly input into the semantic-data conversion module, eliminating the input data format restriction of the module.
[0266] 5) A semantic-data conversion module that integrates arithmetic coding and LZ coding. This module is based on a convolutional neural network and realizes the processing of classical codewords of unequal lengths, completes the mapping from data redundancy to semantic redundancy, and outputs the One-Hot vector required by the semantic coding module.
[0267] 6) A multilingual transmission system based on the classical coding scheme is proposed. Based on the basic principles of Transformer and the nature of semantic association information, different languages can be used when training data and semantic integration modules, thus realizing a multilingual semantic communication system.
[0268] In summary, the optional embodiment of the present application proposes a coding method for multiple languages that integrates data and semantics. This method can receive variable-length codewords of the classic source coding system, and can further compress them using the semantic redundancy contained in the context information, thereby realizing the joint manipulation of data redundancy and semantic redundancy, and integrating the classic source coding system and the semantic coding system. At the same time, the semantic decoding module of the destination can remain unchanged, reducing the overall design complexity of the system.
[0269] In the present embodiment, a vector matrix determination system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the equipment described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.
[0270] Fig.18 is a structural block diagram of a system for determining a vector matrix according to an embodiment of the present application. Fig.18 As shown, the vector matrix determination system includes:
[0271] The encoding module 182 is used to encode the message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix. The codeword lengths of the codewords in the regular codeword matrix are the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix;
[0272] The data semantic conversion module 184 is connected to the encoding module and is used to convert the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0273] Through the above system, the message to be transmitted is encoded into a codeword matrix; the codeword matrix is converted through a fully connected network group to obtain a vector matrix with the same codeword length. In other words, by encoding the message to be transmitted into a three-dimensional codeword matrix and converting the obtained codeword matrix into a vector matrix with the same codeword length, the technical problem that the existing semantic communication systems in the related technology do not involve classical data coding and require the transmitted sentences to be of the same length, which leads to the inability of the semantic communication system to integrate classical communication, is solved. The technical effect of integrating classical coding with the semantic coding of the existing semantic communication system is achieved, and then by integrating classical coding and semantic coding, sentences to be transmitted of different lengths can be processed.
[0274] In an exemplary embodiment, the system further includes: a semantic encoding module, connected to the data semantic conversion module, for replacing the embedding layer of the semantic encoding module with a first fully connected network, and receiving the vector matrix through the first fully connected network.
[0275] In an exemplary embodiment, the fully connected network group includes multiple fully connected networks with different input lengths and the same output lengths, the second dimension size of the two-dimensional parameters of the multiple fully connected networks is related to the number of words in the vocabulary corresponding to the natural language used when training the fully connected networks, and the vector matrix serves as the input of the semantic encoding module, wherein the vector matrix is a three-dimensional vector matrix.
[0276] In an exemplary embodiment, the encoding module 182 is also used to perform at least one of the following: encoding the message to be transmitted into a codeword matrix through a Huffman encoding algorithm; encoding the message to be transmitted into a codeword matrix through an arithmetic encoding algorithm; encoding the message to be transmitted into a codeword matrix through an LZ encoding algorithm; encoding the message to be transmitted into a codeword matrix through a Huffman encoding algorithm, and truncating the codeword matrix according to a target length to form a lossy encoded codeword matrix.
[0277] That is, the message to be transmitted is encoded into a codeword matrix by a classical coding algorithm, and the selected classical coding algorithm includes but is not limited to the Huffman coding algorithm, the arithmetic coding algorithm, and the LZ coding algorithm. Optionally, the codeword matrix obtained by the Huffman coding algorithm can also be truncated according to a certain length (i.e., the target length) to form a lossy coded codeword matrix.
[0278] In an exemplary embodiment, the data semantic conversion module 184 is further used to obtain the codeword length of each codeword in the codeword matrix; and determine the codeword length as the first dimension size of the two-dimensional parameter of the fully connected network corresponding to each codeword.
[0279] After the message to be transmitted is input into the classic coding module using the classic coding algorithm, the classic coding module converts the message to be transmitted and outputs it into a codeword matrix. Generally, the output codeword matrix includes multiple codewords, and the codeword lengths of the codewords are not equal, but it is also possible to obtain a codeword matrix including multiple codewords with the same codeword length or a codeword matrix with only one codeword.
[0280] An optional embodiment of the present application is to merge the classic coding module with the semantic coding module of the semantic communication system, and use the data-semantic information conversion module to convert the codeword matrix output by the classic coding module into a vector matrix that can be input into the classic coding module. Optionally, the data-semantic information conversion module may include a fully connected network group, which is used to convert the codeword matrix obtained by the output of the classic coding module into a vector matrix with the same codeword length, so as to facilitate input into the semantic communication system. Specifically, the fully connected network group includes a plurality of fully connected networks, and the plurality of fully connected networks may be the same as the number of codewords in the codeword matrix, and the first dimension size of the two-dimensional parameters of each of the plurality of fully connected networks is: the codeword length of the codeword to be processed corresponding to each fully connected network, for example, the first dimension size of the fully connected network corresponding to the codeword of length 1 is 1, and the first dimension size of the fully connected network corresponding to the codeword of length 2 is 2, and so on.
[0281] In an exemplary embodiment, the data semantic conversion module 184 is further used to input each codeword in the codeword matrix into a fully connected network corresponding to each codeword, and determine the vector A corresponding to each codeword in the vector matrix by the following formula:
[0282] A=W i h+b i ;
[0283] Wherein, h represents each codeword in the codeword matrix, W i and b i represents the two-dimensional parameters of the fully connected network corresponding to the i-th codeword in the codeword matrix, where W i and b i The first dimension size of is the codeword length of the i-th codeword; the vector matrix is obtained according to the position of each codeword in the codeword matrix and the vector A corresponding to each codeword.
[0284] It can be understood that the fully connected network converts the codewords in the codeword matrix into vectors in the vector matrix through the above formula. Specifically, each codeword in the codeword matrix is input into the corresponding fully connected network, and each codeword is converted into a vector corresponding to each codeword through the above formula. All codewords in the codeword matrix are converted into corresponding vectors, and the obtained vectors are combined into a vector matrix according to the position of each codeword in the codeword matrix.
[0285] The two-dimensional parameter W in the above formula i and b iThe size of the second dimension is related to the size of the vocabulary used in training, and the size of the vocabulary used in training is the number of words in the vocabulary corresponding to the natural language used in training. In an optional embodiment, if the language used in training is Chinese, and there are 7000 words in the corresponding vocabulary, the size of the second dimension of the two-dimensional parameters of the multiple fully connected networks included in the fully connected network group is 7000. It should be noted that the obtained vector matrix is: a three-dimensional vector matrix with a fixed dimension, and the size of the third dimension is related to the size of the vocabulary used in training, and the size of the vocabulary used in training is the number of words in the vocabulary corresponding to the natural language used in training. For example, when the size of the second dimension of the two-dimensional parameters of the fully connected network group is 7000, the size of the third dimension of the vector matrix is 7000.
[0286] It should be noted that the above "7000" is only used as an example and is not used to limit the present application. By analogy, the language used in training is not limited to Chinese, and the size of the vocabulary used in training is not limited to 7000 words, but is related to the actual size of the vocabulary corresponding to the language type used in training.
[0287] Since the embedding layer of the semantic encoding module requires a one-hot input vector, it should be noted that a one-hot vector is a method of converting discrete, categorical data into a continuous vector representation. In a one-hot vector, each category is encoded into a vector, in which only one dimension has a value of 1, and the other dimensions are all 0. One-hot vectors can be used to easily convert any number of different labels into a fixed-length, dense, and low-dimensional vector format, and these vectors are independent of each other and have no intrinsic relationship. Therefore, in many machine learning algorithms, input data is often encoded into a one-hot vector format and processed.
[0288] Therefore, after converting the codeword matrix to a vector matrix, optionally:
[0289] 1) The vector matrix can be further converted into a One-Hot input vector:
[0290] In an exemplary embodiment, the data semantic conversion module 184 is also used to obtain a probability vector corresponding to each third-dimensional vector of the vector matrix; determine a target vector based on the probability vector, wherein the target vector is a vector that sets the maximum element in each third-dimensional vector to a first value; and combine multiple target vectors into a target matrix.
[0291] In an exemplary embodiment, the data semantic conversion module 184 is further used to determine the noise data corresponding to each third-dimensional vector based on the probability vector; determine the target vector based on the noise data and the probability vector, wherein the noise data is used to make the target vector not a constant value.
[0292] It can be understood that splitting the vector matrix into multiple probability vectors according to the third dimension essentially highlights the maximum value in each probability vector, that is, the probability value of the probability vector corresponding to the maximum value in each third-dimensional vector is the largest among the probability values corresponding to all probability vectors included in the probability vector corresponding to the third-dimensional vector.
[0293] In order to make the target vector non-constant and allow the gradient to be returned, a reparameterization technique can be used:
[0294] In an optional embodiment, Gumbel noise is added to the probability vector, specifically: the corresponding noise data is determined by the third dimensional vector, that is, the noise data corresponding to each vector element in the third dimensional vector is calculated; each probability element in the obtained probability vector is added to each corresponding noise data, and finally the target probability vector (that is, the target vector in the above embodiment) is obtained. All the target probability vectors are combined to obtain a target matrix, that is, a One-Hot input matrix, which can be input into the Embedding layer in the semantic coding module.
[0295] 2) The Embedding layer in the semantic encoding module can be replaced, thereby removing the restriction of the One-Hot input vector required by the Embedding layer:
[0296] In an exemplary embodiment, the data semantic conversion module 184 is also used to directly input the vector matrix into the first fully connected network of the semantic encoding module, wherein the first dimension size of the two-dimensional parameters of the first fully connected network is related to the number of words in the vocabulary.
[0297] In an optional embodiment of the present application, the first dimension size of the two-dimensional parameters of the fully connected network replacing the Embedding layer is the same as the third dimension size of the target vector. Therefore, the vector matrix can be directly input into the semantic encoding module, avoiding the need to convert the vector matrix into a one-hot input vector.
[0298] In an exemplary embodiment, the data semantic conversion module 184 is also used to group the vector matrix according to the third dimension direction to obtain multiple groups of sub-vector matrices; convert the multiple groups of sub-vector matrices into multiple sub-target matrices, and determine the target matrix based on the multiple sub-target matrices.
[0299] According to the third dimension direction of the vector matrix, the optional scheme includes: grouping according to the maximum codeword length of the codeword matrix corresponding to the vector matrix, and the number of groups is related to the codeword length in the corresponding codeword matrix. A more specific optional scheme is that the number of groups is related to the maximum codeword length minus the codeword length of the codeword corresponding to the vector in the vector matrix. For example, if the codeword length of the codeword corresponding to a certain vector in the vector matrix is 2, the maximum codeword length is 8, and 8-2, that is, 6, is the number of groups. If the vectors of the vector matrix cannot be divided equally according to the number of groups, the remaining elements can be discarded, or the groups that cannot be divided equally can be used as the last group.
[0300] In an exemplary embodiment, the data semantic conversion module 184 is also used to obtain the maximum codeword length in the codeword matrix; fill a second value at a target position of the codeword matrix according to the maximum codeword length to obtain a filled codeword matrix whose codeword lengths are all the maximum codeword lengths; and convert each codeword in the filled codeword matrix through a fully connected network group to obtain a vector matrix whose codeword lengths are all the same.
[0301] Optionally, the codeword matrix is first padded according to the maximum codeword length in the codeword matrix to obtain a padded codeword matrix with the same codeword length.
[0302] For example, suppose the message to be transmitted is S, and the message S consists of B sentences of length L, where L means there are L words. When the classic encoding module is the Huffman encoding module, the codeword matrix output by the Huffman encoding module is a three-dimensional codeword matrix with a shape of [B, L, H ij ], where H ij The length of the Huffman codeword obtained by encoding the jth word of the i-th sentence is unequal, and its maximum length is recorded as max(H ij ). When the codeword matrix is filled with the second value, a filled codeword matrix with the same codeword length is obtained, and its size is [B, L, max(H ij )].
[0303] The data-semantic conversion module using the padded codeword matrix as input can then use only one parameter size of [max(H ij ), CorpusLen], the fill codeword matrix can be converted into a vector matrix. It should be noted that CorpusLen is related to the vocabulary size used in training, and the vocabulary size used in training is the number of words in the vocabulary corresponding to the natural language used in training. In an optional embodiment of the present application, the structure of the data-semantic conversion module can be made simpler.
[0304] In an exemplary embodiment, the data semantic conversion module 184 is also used to obtain a two-dimensional codeword matrix corresponding to the encoding when the encoding is encoded by an arithmetic coding algorithm or an LZ coding algorithm; obtain a three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix, and replace the first dimension and the second dimension of the three-dimensional codeword matrix to obtain an input matrix, wherein the third dimension size of the input matrix is the maximum codeword length of the codeword matrix; determine a second fully connected network in which the first dimension size of the two-dimensional parameter is the maximum codeword length, and the second dimension size of the two-dimensional parameter is related to the number of words in the vocabulary, and convert each codeword in the input matrix through the second fully connected network to obtain a vector matrix with the same codeword length.
[0305] When the classical coding module adopts an arithmetic coding algorithm or an LZ coding algorithm, the codeword matrix obtained after converting the message to be transmitted is a two-dimensional codeword matrix. Then, the data-semantic information conversion module corresponding to the arithmetic coding algorithm or the LZ coding algorithm can be selected to be composed of a convolutional layer, a Reshape layer, and a fully connected layer (equivalent to the second fully connected network of the above embodiment).
[0306] Therefore, obtaining the three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix includes: in the two-dimensional codeword matrix having a shape of [B,H i ], where H i It represents the length of the arithmetic codeword obtained by encoding the i-th sentence, and the longest length is recorded as max(H i ), fill the two-dimensional codeword matrix to obtain a size of [B, max(H i )], under certain conditions, a three-dimensional codeword matrix can be obtained based on the two-dimensional filling codeword matrix.
[0307] Optionally, under certain conditions, a three-dimensional codeword matrix is obtained according to the two-dimensional padding codeword matrix, including: passing the two-dimensional padding codeword matrix through a convolutional layer with a layer channel number L and a kernel size of 1*1, when the padding number P of the convolutional layer satisfies the following relationship with the step size S:
[0308]
[0309] Where P H ,S H Indicates the number of padding and stride in the column direction, P W ,S W Indicates the padding number and step size in the row direction. The three-dimensional matrix corresponding to the two-dimensional padding codeword matrix can be obtained through the above convolutional layer, with a size of [L, B, max(H i )].
[0310] Further, the first dimension and the second dimension of the obtained three-dimensional codeword matrix are permuted to obtain an input matrix, and input into the parameter size [max(H i ), CorpusLen], and finally a vector matrix of size [B, L, CorpusLen] can be obtained. The scheme of inputting the vector matrix into the speech coding module can refer to other embodiments of the present application, which will not be described in detail here.
[0311] In the above exemplary embodiment, the first value is a positive number, and the second value is a value smaller than the first value. For example, when the first value is a positive number 2, the second value should be a value smaller than 2.
[0312] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a readable storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.
[0313] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0314] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0315] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0316] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:
[0317] S1, encode the message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codeword lengths of the codewords in the regular codeword matrix are all the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix;
[0318] S2, transforming the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0319] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0320] Optionally, in this embodiment, the above-mentioned electronic device can also be configured to execute the above-mentioned step S1 through a computer program, and encode the message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix. The codeword lengths of the codewords in the regular codeword matrix are all the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix; S2, converting the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
[0321] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0322] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
[0323] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for determining a vector matrix, characterized in that: include: Encoding the message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, and the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codeword lengths of the codewords in the regular codeword matrix are all the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix; The codeword matrix is transformed through a fully connected network group to obtain a vector matrix with the same codeword length.
2. The method for determining a vector matrix according to claim 1, characterized in that: The fully connected network group includes multiple fully connected networks with different input lengths and the same output lengths, the second dimension size of the two-dimensional parameters of the multiple fully connected networks is related to the number of words in the vocabulary corresponding to the natural language used when training the fully connected networks, and the vector matrix is used as the input of the semantic encoding module, wherein the vector matrix is a three-dimensional vector matrix.
3. The method for determining a vector matrix according to claim 2, characterized in that: Before converting the codeword matrix through the fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: Obtaining the codeword length of each codeword in the codeword matrix; The codeword length is determined as the first dimension size of the two-dimensional parameters of the fully connected network corresponding to each codeword.
4. The method for determining a vector matrix according to claim 3, characterized in that: The codeword matrix is transformed through a fully connected network group to obtain a vector matrix with the same codeword length, including: Each codeword in the codeword matrix is input into the fully connected network corresponding to each codeword, and the vector A corresponding to each codeword in the vector matrix is determined by the following formula: A=W i h+b i ; Wherein, h represents each codeword in the codeword matrix, W i and b i represents the two-dimensional parameters of the fully connected network corresponding to the i-th codeword in the codeword matrix, where W i and b i The first dimension size of is the codeword length of the i-th codeword; The vector matrix is obtained according to the position of each codeword in the codeword matrix and the vector A corresponding to each codeword.
5. The method for determining a vector matrix according to claim 1, characterized in that: After converting the codeword matrix through the fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: Obtain a probability vector corresponding to each third-dimensional vector of the vector matrix; Determine a target vector according to the probability vector, wherein the target vector is a vector in which the maximum element in each third-dimensional vector is set to a first value; A plurality of said target vectors are combined into a target matrix.
6. The method for determining a vector matrix according to claim 5, characterized in that: Determining a target vector according to the probability vector includes: Determine the noise data corresponding to each third-dimensional vector according to the probability vector; The target vector is determined according to the noise data and the probability vector, wherein the noise data is used to make the target vector not a constant value.
7. The method for determining a vector matrix according to claim 2, characterized in that: After converting the codeword matrix through the fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: The vector matrix is directly input into a first fully connected network of a semantic encoding module, wherein a first dimension size of a two-dimensional parameter of the first fully connected network is related to the number of words in the vocabulary.
8. The method for determining a vector matrix according to claim 1, characterized in that: After converting the codeword matrix through the fully connected network group to obtain a vector matrix with the same codeword length, the method further includes: Grouping is performed according to the third dimension direction of the vector matrix to obtain multiple groups of sub-vector matrices; The multiple groups of sub-vector matrices are converted into multiple sub-target matrices, and the target matrix is determined according to the multiple sub-target matrices.
9. The method for determining a vector matrix according to claim 1, characterized in that: The codeword matrix is transformed through a fully connected network group to obtain a vector matrix with the same codeword length, including: Obtaining a maximum codeword length in the codeword matrix; Fill a second value at a target position of the codeword matrix according to the maximum codeword length to obtain a filled codeword matrix whose codeword lengths are all the maximum codeword lengths; Each codeword in the padding codeword matrix is converted through the fully connected network group to obtain a vector matrix with the same codeword length.
10. The method for determining a vector matrix according to claim 2, characterized in that: include: In the case where the encoding is performed by an arithmetic encoding algorithm or an LZ encoding algorithm, obtaining a two-dimensional codeword matrix corresponding to the encoding; Obtaining a three-dimensional codeword matrix corresponding to the two-dimensional codeword matrix, replacing the first dimension and the second dimension of the three-dimensional codeword matrix to obtain an input matrix, wherein the size of the third dimension of the input matrix is the maximum codeword length of the codeword matrix; A second fully connected network is determined in which the first dimension size of the two-dimensional parameter is the maximum codeword length, and the second dimension size of the two-dimensional parameter is related to the number of words in the vocabulary, and each codeword in the input matrix is converted through the second fully connected network to obtain a vector matrix with the same codeword length.
11. The method for determining a vector matrix according to claim 1, characterized in that: The message to be transmitted is encoded into a codeword matrix, which includes at least one of the following: Encoding the message to be transmitted into the codeword matrix by using a Huffman coding algorithm; Encoding the message to be transmitted into the codeword matrix by an arithmetic coding algorithm; The message to be transmitted is encoded into the codeword matrix by using the LZ encoding algorithm. The message to be transmitted is encoded into the codeword matrix through the Huffman coding algorithm, and the codeword matrix is truncated according to the target length to form a lossy coded codeword matrix.
12. The method for determining a vector matrix according to claim 5 or 9, characterized in that: The first value is a positive number, and the second value is a value smaller than the first value.
13. A vector matrix determination system, characterized in that: include: A coding module, used for encoding a message to be transmitted into a codeword matrix, wherein the codeword matrix includes: a single or multiple codewords, the codeword matrix includes: a regular codeword matrix and an irregular codeword matrix, the codeword lengths of the codewords in the regular codeword matrix are all the same, and the codeword lengths of the codewords in the irregular codeword matrix are different, wherein the codeword matrix is a three-dimensional codeword matrix; A data semantic conversion module is connected to the encoding module and is used to convert the codeword matrix through a fully connected network group to obtain a vector matrix with the same codeword length.
14. The vector matrix determination system according to claim 13, characterized in that: The system also includes: a semantic encoding module, connected to the data semantic conversion module, used to replace the embedding layer of the semantic encoding module with a first fully connected network, and receive the vector matrix through the first fully connected network.
15. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 12 when executed.
16. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the method according to any one of claims 1 to 12 through the computer program.
Citation Information
Cited By
Vector matrix determination method and system, storage medium, and electronic device
EP4604005A1
Vector matrix determination method and system, storage medium, and electronic device
WO2024234673A1
