Fine-grained hybrid automatic retransmission method and device based on semantic base

Abstract

The embodiment of the invention provides a fine-grained hybrid automatic retransmission method and device based on a semantic base, a semantic knowledge base is constructed based on an explicit semantic base, a sending end generates verification information for error detection based on the fine-grained semantic base, and a receiving end performs noisy semantic error detection on the recovered semantic base by using the verification information, so that the accuracy of the fine-grained hybrid automatic retransmission is improved. And when the semantic base is judged to have a semantic error, the sending end is requested to retransmit the wrong semantic base, so that information retransmission refined to semantic base granularity is realized, the semantic error detection accuracy can be ensured, the retransmission overhead is reduced, the transmission delay is reduced, and the effectiveness and reliability of communication in a complex channel environment are realized.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication technology, and in particular to a semantic-based fine-grained hybrid automatic retransmission method and device. Background Art

[0002] Semantic communication focuses on the transmission of the "meaning" of information and has been widely studied in recent years. Compared with traditional grammatical communication, which is based on Shannon's information theory and focuses on the correct reception of each transmitted symbol, semantic communication focuses on the content and meaning of the transmitted information rather than the coding symbols, which can fully meet communication needs while improving communication efficiency. Current semantic communication does not consider the problem of transmission errors caused by interference, and does not provide error detection and correction processing methods, which affects the quality of communication. Summary of the invention

[0003] In view of this, the purpose of the embodiments of the present application is to propose a semantic-based fine-grained hybrid automatic retransmission method and device to solve the transmission error problem of semantic communication.

[0004] Based on the above purpose, the embodiment of the present application provides a semantic-based fine-grained hybrid automatic retransmission method, which is applied to a sending end, including:

[0005] Extract semantic features from raw information;

[0006] According to the semantic features, multiple semantic bases that meet the preset task requirements and semantic representations composed of indexes of multiple semantic bases are matched from a preset semantic base knowledge base;

[0007] Channel-encoding the semantic representation according to a preset encoding method to obtain an encoded semantic representation;

[0008] Sending the encoded semantic representation to a receiving end via a channel;

[0009] Generate verification information based on multiple semantic bases;

[0010] The verification information is sent to the receiving end, so that the receiving end verifies the received information based on the verification information.

[0011] Optionally, generating verification information based on multiple semantic bases includes:

[0012] Inputting a plurality of semantic bases into a preset context estimation module, and the context estimation module outputs a probability estimation result of each semantic base; wherein the probability estimation result includes a mean value;

[0013] Calculate the mean correction value between the mean of each semantic base and the corresponding semantic base;

[0014] The mean correction value of each semantic base is input into a preset verification information encoding module, and the verification information encoding module outputs verification information.

[0015] Optionally, the semantic base knowledge base is constructed based on an explicit semantic base, and the method for generating the explicit semantic base includes:

[0016] Extract semantic features from existing information;

[0017] According to a preset clustering rule, an explicit semantic base is extracted from the semantic features.

[0018] The embodiment of the present application also provides a semantic-based fine-grained hybrid automatic retransmission method, which is applied to a receiving end and includes:

[0019] receiving semantic information and verification information;

[0020] performing channel decoding on the semantic information to obtain a decoded semantic representation;

[0021] According to the decoded semantic representation, multiple semantic bases are restored by matching from a preset semantic base knowledge base;

[0022] Verifying the restored multiple semantic bases using the verification information to obtain verified semantic bases;

[0023] Reconstruct the original information based on the verified semantic basis.

[0024] Optionally, the using the verification information to verify the restored multiple semantic bases to obtain verified semantic bases includes:

[0025] Inputting the restored multiple semantic bases into a preset context estimation module, and the context estimation module outputs a probability estimation result of each restored semantic base; wherein the probability estimation result includes a mean and a variance;

[0026] Inputting the verification information into a preset verification information decoding module, and the verification information decoding module outputs the mean correction value of each semantic basis;

[0027] Determining the final mean of each restored semantic base according to the mean of each restored semantic base and the mean correction value of the corresponding semantic base;

[0028] Conduct hypothesis testing based on the final mean and variance of each restored semantic base to obtain the test results of each semantic base;

[0029] If the verification result contains a semantic basis that needs to be retransmitted, the verification result is sent to the sending end, so that the sending end resends the semantic basis that needs to be retransmitted according to the verification result.

[0030] Optionally, the method further includes:

[0031] Receiving retransmitted semantic information; wherein the retransmitted semantic information is obtained after the sending end retransmits the semantic base to be retransmitted based on the verification result and transmits it through the channel;

[0032] The next round of error detection is performed based on maximum ratio combining of the retransmitted semantic information and the semantic information.

[0033] Optionally, performing maximum ratio combining based on the retransmitted semantic information and the semantic information includes:

[0034] Decoding the retransmitted semantic information to obtain a retransmitted semantic representation;

[0035] The index corresponding to the semantic base in the retransmitted semantic representation is combined with the index of the received semantic base by maximum ratio.

[0036] Optionally, the hypothesis test is performed based on the final mean and variance of each restored semantic base to obtain the test result of each semantic base, including:

[0037] Based on the final mean and variance of each semantic basis, the significance threshold adaptively generated by the binary error detection module is used to perform hypothesis testing and obtain the test results.

[0038] The embodiment of the present application further provides a semantic-based fine-grained hybrid automatic retransmission device, which is applied to a transmitting end and includes:

[0039] Extraction module, used to extract semantic features from raw information;

[0040] A matching module is used to match multiple semantic bases that meet the preset task requirements and a semantic representation composed of indexes of multiple semantic bases from a preset semantic base knowledge base according to semantic features;

[0041] A coding module, used for performing channel coding on the semantic representation according to a preset coding method to obtain a coded semantic representation;

[0042] A generation module, used for generating verification information based on multiple semantic bases;

[0043] The sending module is used to send the encoded semantic representation to the receiving end via the channel; and send the verification information to the receiving end, so that the receiving end verifies the received information based on the verification information.

[0044] The embodiment of the present application further provides a semantic-based fine-grained hybrid automatic retransmission device, which is applied to a receiving end and includes:

[0045] A receiving module, used for receiving semantic information and verification information;

[0046] A decoding module, used for decoding the semantic information to obtain a decoded semantic representation;

[0047] A matching module, used for matching and restoring a plurality of semantic bases from a preset semantic base knowledge base according to the decoded semantic representation;

[0048] A verification module, used to verify the restored multiple semantic bases using the verification information to obtain verified semantic bases;

[0049] The reconstruction module is used to reconstruct the original information based on the verified semantic basis.

[0050] From the above description, it can be seen that the fine-grained hybrid automatic retransmission method and device based on semantic basis provided in the embodiment of the present application builds a semantic knowledge base based on an explicit semantic basis, the sending end generates verification information for error detection based on the fine-grained semantic basis, and the receiving end uses the verification information to perform noisy semantic error detection on the recovered semantic basis. When it is determined that there is a semantic error in the semantic basis, the sending end is requested to retransmit the erroneous semantic basis, thereby realizing information retransmission refined to the semantic basis granularity, which can ensure the accuracy of semantic error detection, reduce retransmission overhead, reduce transmission delay, and achieve the effectiveness and reliability of communication in complex channel environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 This is a schematic diagram of a method flow chart applied to a sending end according to an embodiment of the present application;

[0053] Figure 2 This is a schematic diagram of a method flow chart applied to a receiving end according to an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a system model of an embodiment of the present application;

[0055] Figure 4 A schematic diagram of an error detection and retransmission model according to an embodiment of the present application;

[0056] Figure 5 This is an example diagram of error diffusion in an embodiment of the present application;

[0057] Figure 6 Schematic diagram of curves showing changes in image reconstruction perceived quality with the number of retransmissions under different methods of the embodiments of the present application;

[0058] Figure 7 Schematic diagram of a curve showing a change in multi-scale structural similarity of image reconstruction with the number of retransmissions under different methods of the embodiments of the present application;

[0059] Figure 8 Schematic diagram of a curve showing a change in total transmission overhead with the number of retransmissions under different methods of an embodiment of the present application;

[0060] Fig. 9 This is a structural block diagram of a device at the transmitting end of an embodiment of the present application;

[0061] Fig.10 This is a structural block diagram of a receiving device of an embodiment of the present application;

[0062] Fig.11 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0064] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0065] In the related technologies, semantic communication can significantly improve service quality and communication efficiency compared with traditional grammatical communication. However, the related semantic communication system does not provide error detection and correction mechanisms, which will reduce the communication quality due to interference factors. The error detection mode of the traditional hybrid automatic retransmission method focuses on bit errors, ignoring the meaning of information and communication intention, and cannot be applied to semantic communication for error detection and correction.

[0066] In view of this, an embodiment of the present application provides a fine-grained hybrid automatic retransmission method based on semantic basis, constructs a semantic knowledge base of explicit semantic basis, adopts a hybrid automatic retransmission framework to verify the fine-grained semantic basis, and improves the reliability and communication quality of semantic communication in a noisy environment.

[0067] The technical solution of the present application is further described in detail below through specific embodiments.

[0068] like Figure 1 , 3 As shown, the embodiment of the present application provides a semantic-based fine-grained hybrid automatic retransmission method, which is applied to a sending end, and the method includes:

[0069] S101: extracting semantic features from raw information;

[0070] In this embodiment, for the original information of the sender, according to the application scenario and task requirements, a semantic feature extraction module is used to extract semantic features from the original information. The original information includes but is not limited to image information, text information, audio information, etc. In different application scenarios, based on specific task requirements, a corresponding semantic feature extraction module is pre-trained, and the semantic feature extraction module is used to extract features from the original information to obtain semantic features.

[0071] S102: According to the semantic features, a plurality of semantic bases that meet the preset task requirements and a semantic representation composed of indexes of the plurality of semantic bases are matched from a preset semantic base knowledge base;

[0072] In this embodiment, the semantic base knowledge base is constructed based on an explicit semantic base (Semantic Base, Seb), and the sending end and the receiving end have synchronized semantic base knowledge bases to ensure that both parties have the same semantic understanding of the same information and ensure the normal progress of semantic communication. Based on the extracted semantic features, the semantic base knowledge base is used to match multiple semantic bases corresponding to the semantic features and a semantic representation composed of indexes of multiple semantic bases. The matching method of the semantic features in the semantic base knowledge base can be determined according to the preset task requirements. The matched semantic base can be at least one semantic base that meets the task requirements, and / or a semantic base set composed of multiple semantic bases combined in a specific way; each semantic base is encoded according to a specific index encoding method to obtain the index corresponding to each semantic base, and the semantic representation is obtained according to the indexes of the matched multiple semantic bases.

[0073] In some implementations, a method for generating an explicit semantic base includes:

[0074] Extract semantic features from existing information;

[0075] According to the preset clustering rules, the semantic features are clustered using the preset clustering algorithm to obtain the explicit semantic base.

[0076] In this embodiment, the method for generating an explicit semantic base is as follows: N is the amount of existing information, and a set of semantic features are extracted from the existing information using the semantic feature extraction module. N p is the number of semantic features in the existing information, is the semantic feature, c, h, w are the dimensions of the feature, is the nth item of the i-th existing information p semantic features; then, according to the preset clustering rules, the semantic features are clustered using the clustering algorithm Clustering is performed to obtain a set of semantic base Among them, K is the preset number of semantic bases.

[0077] In some embodiments, a semantic base knowledge base is constructed based on the generated explicit semantic base, and the semantic base knowledge base is synchronously transmitted to the sending end and the receiving end. The existing information is, for example, image information, text information, audio information, etc. that have been transmitted between the sending end and the receiving end. Optionally, the semantic base knowledge base can be constructed based on the partial order relationship of the semantic base.

[0078] In some implementations, the method for generating an explicit semantic basis is implemented based on a semantic basis generation module of a deep learning network model, and the loss function of the semantic basis generation module is:

[0079]

[0080] Among them, L D is the loss function of the original information distortion reconstructed at the receiving end, using mean square error, L reg is the regular term of clustering operation, sg(·) is the gradient truncation operation, I is the original information, is the original information reconstructed by the receiving end, and F is the semantic feature extracted from the original information. Since the clustering operation is not differentiable, the direct estimation is used to approximate the gradient. This regularization term is used to solve the problem of feature space expansion caused by the direct estimation. Since the semantic basis S is the cluster center and does not contain trainable parameters, the gradient of the semantic basis is truncated.

[0081] S103: performing channel coding on the semantic representation according to a preset coding method to obtain a coded semantic representation;

[0082] S104: sending the encoded semantic representation to a receiving end via a channel;

[0083] In this embodiment, the semantic representation is input into a preset channel coding module, and the channel coding module performs channel coding on the semantic representation according to a preset coding method to generate a coded semantic representation, and the coded semantic representation is sent to the receiving end through a channel. Among them, channel coding and channel decoding are implemented based on a deep neural network model, and the network structure is flexibly configured according to specific application scenarios, source modalities, transmission requirements, etc.; channel coding and channel decoding can also be implemented based on non-machine learning methods; the channel coding and decoding method can be implemented using any known coding method, and the coding and decoding method will not be specifically explained.

[0084] S105: Generate verification information based on multiple semantic bases;

[0085] S106: Send the verification information to the receiving end, so that the receiving end verifies the received information based on the verification information.

[0086] In this embodiment, verification information is generated based on multiple semantic bases, and the verification information is sent to the receiving end through an error-free channel, so as to be used for information verification between the sending end and the receiving end, thereby improving communication reliability.

[0087] The method for generating verification information includes:

[0088] Inputting multiple semantic bases into a preset context estimation module, and the context estimation module outputs a probability estimation result of each semantic base; the probability estimation result includes a mean and a variance;

[0089] Calculate the mean correction value between the mean of each semantic base and the corresponding semantic base;

[0090] The mean correction value of each semantic base is input into a preset verification information encoding module, and the verification information encoding module outputs verification information.

[0091] In this embodiment, the sender generates verification information based on multiple semantic bases generated by the original information. The multiple semantic bases are input into the context estimation module, and the context estimation module outputs the probability estimation result of each semantic base, the probability estimation result includes the mean and variance, and the difference between the mean of each semantic base and the corresponding semantic base is calculated as the mean correction value of the semantic base. The mean correction value of each semantic base is input into the verification information encoding module, and the verification information encoding module encodes the mean correction value to obtain verification information.

[0092] In some embodiments, the verification information encoding module removes redundant information based on the input mean correction value, performs compression encoding, and obtains the verification information. At the receiving end, the verification information is decoded and reconstructed using a corresponding verification information decoding module to restore the verification information.

[0093] like Figure 2 , 4As shown in , 5, the embodiment of the present application provides a fine-grained hybrid automatic retransmission method based on semantic base, which is applied to a receiving end, and the method includes:

[0094] S201: receiving semantic information and verification information;

[0095] S202: performing channel decoding on the semantic information to obtain a decoded semantic representation;

[0096] In this embodiment, the receiving end receives semantic information and verification information from the channel, inputs the semantic information into a preset channel decoding module, and the channel decoding module performs channel decoding on the semantic information according to a preset decoding method to obtain a decoded semantic representation, and performs verification based on the decoded semantic representation using the verification information. Among them, the decoding method of the channel decoding module and the encoding method of the channel encoding module are matching encoding and decoding methods, so that the channel decoding module can decode and restore the information before encoding.

[0097] S203: According to the decoded semantic representation, multiple restored semantic bases are matched from a semantic base knowledge base;

[0098] In this embodiment, the receiving end determines the indexes of multiple semantic bases in the semantic representation based on the decoded semantic representation, and uses the semantic base knowledge base to match the corresponding multiple semantic bases based on the indexes of the multiple semantic bases as the recovered multiple semantic bases. Due to the presence of factors such as channel interference, the multiple semantic bases recovered by the receiving end may contain errors.

[0099] S204: Verify the restored multiple semantic bases using the verification information to obtain multiple verified semantic bases;

[0100] In this embodiment, after the restored multiple semantic bases are obtained, the restored multiple semantic bases are verified using the verification information, and the method includes:

[0101] Inputting the restored multiple semantic bases into a preset context estimation module, and the context estimation module outputs the probability estimation results of each restored semantic base; the probability estimation results include mean and variance;

[0102] Input the verification information into a preset verification information decoding module, and the verification information decoding module outputs the mean correction value of each semantic basis;

[0103] Determining the final mean of each restored semantic base according to the mean of each restored semantic base and the mean correction value of the corresponding semantic base;

[0104] Conduct hypothesis testing based on the final mean and variance of each restored semantic base to obtain the test results of each semantic base;

[0105] If the verification result contains a semantic basis that needs to be retransmitted, the verification result is sent to the sending end, so that the sending end retransmits the semantic basis that needs to be retransmitted according to the verification result.

[0106] In this embodiment, after obtaining the recovered multiple semantic bases, the recovered multiple semantic bases are input into a context estimation module, and the context estimation module outputs probability estimation results of the recovered multiple semantic bases, and the probability estimation results include the mean and variance of each semantic base; the received verification information is input into a verification information decoding module, and the verification information decoding module decodes the verification information to obtain the mean correction value of each recovered semantic base, the mean of each semantic base output by the context estimation module is superimposed with the mean correction value of the corresponding recovered semantic base to obtain the final mean of each semantic base, and a hypothesis test is performed based on the final mean and variance of each semantic base to obtain a test result, and the verification result is determined according to the test result.

[0107] In some embodiments, the context estimation module at the transmitting end and the context estimation module at the receiving end have the same network parameters. The check information decoding module and the check information encoding module are respectively an encoder and a decoder constructed based on a CNN network.

[0108] In some embodiments, the context estimation module at the receiving end outputs the probability estimation result in, is a mean value set composed of the means of each semantic base recovered, and the check information is decoded by the check information decoding module to obtain a mean correction value set composed of the mean correction values ​​of each semantic base recovered The mean value is set The mean and mean correction value set of each semantic basis in The mean correction values ​​of the corresponding semantic bases in the mean set are added to obtain the final mean of each restored semantic base. For example, the mean of the ith semantic base in the mean set is added to the mean correction value of the ith semantic base in the mean correction value set to obtain the final mean of the ith semantic base.

[0109] After determining the final mean of each semantic base, a hypothesis test is performed based on the final mean and variance of each semantic base. The hypothesis test can be expressed as:

[0110]

[0111] in, is the i-th semantic basis recovered, is the final mean of the ith semantic base. The null hypothesis H0 indicates that the semantic base Through the test, there is no semantic error; the alternative hypothesis H1 indicates that the semantic basis There is a semantic error and retransmission is required.

[0112] In the hypothesis testing process, the semantic basis set consisting of multiple semantic basis recovered is calculated The test statistic The calculation method is:

[0113]

[0114] Among them, z i is the restored i-th semantic basis The test statistic, σ i is the restored i-th semantic basis The standard deviation, N p is the number of semantic bases recovered.

[0115] Each restored semantic base is tested, that is, the test statistic of each restored semantic base is judged based on the corresponding significance threshold, and whether the semantic base has a semantic error is determined according to the judgment result. That is, this embodiment constructs a hypothesis test problem by analyzing the statistical characteristic difference between noisy semantics and semantics not contaminated by noise, obtains the test statistic of each semantic base in the noisy semantics, and further adaptively generates the judgment threshold of each semantic element by analyzing the relevant information in the test statistic, thereby realizing error detection based on fine-grained semantic bases and improving retransmission efficiency.

[0116] In some methods, hypothesis testing of fine-grained semantic bases can be implemented based on the error detection and retransmission module. The error detection and retransmission module includes a masked convolution layer and a point-by-point convolution layer, wherein the convolution kernel size of the masked convolution layer is 5*5, and the central element is blocked. Through convolution operation, the value probability modeling of each central semantic base is realized based on the corresponding surrounding 24 semantic bases; the point-by-point convolution layer is used to achieve the effect of deepening the network fitting ability under the premise of ensuring causal constraints. The network structure of the error detection and retransmission module can effectively realize context-based semantic probability modeling, and because it is completely composed of convolutional networks, it has good parallel computing capabilities, ensuring the efficiency of calculations.

[0117] In some embodiments, a hypothesis test is performed based on the final mean and variance of each semantic base to obtain a test result, including:

[0118] Based on the final mean and variance of each semantic basis, the significance threshold adaptively generated by the binary error detection module is used to perform hypothesis testing and obtain the test results.

[0119] In this embodiment, Figure 5As shown in the figure, considering that the test statistics Z of the restored multiple semantic bases have error diffusion phenomenon, that is, an error in a semantic base will not only cause the test statistic corresponding to the semantic base to rise, but also cause the test statistics corresponding to the adjacent semantic bases that are mutually contextual to fluctuate. The method of using a single significance threshold for judgment may result in misjudgment. Therefore, in order to offset the impact of error diffusion, a binary error detection module is used to adaptively generate a significance threshold based on the test statistic Z, and the adaptive significance threshold is used to make a judgment on the hypothesis test to improve the accuracy of error detection.

[0120] In some embodiments, the binary error detection module is based on a supervised learning method. It is obtained by constructing sample features and corresponding labels to train a preset machine learning model, that is, constructing an erroneous semantic base and its corresponding label, an adjacent semantic base of the erroneous semantic base and its corresponding label, and a correct semantic base and its corresponding label to train the model. The binary error detection module obtained after training can obtain accurate test results by judging with an appropriate significance threshold.

[0121] In some implementations, verifying the restored multiple semantic bases using verification information further includes:

[0122] Receiving retransmitted semantic information; wherein the retransmitted semantic information is obtained after the sending end retransmits the semantic base to be retransmitted based on the verification result and transmits it through the channel;

[0123] The next round of error detection is performed based on the maximum ratio combination of the retransmitted semantic information and the received semantic information.

[0124] In this embodiment, after the test results of each semantic base are determined by hypothesis testing, the test results of all semantic bases are sent to the transmitting end. The transmitting end determines the semantic base that needs to be retransmitted based on the received test results, determines the corresponding index based on the semantic base that needs to be retransmitted, forms a semantic representation based on the index of the retransmitted semantic base, and sends the semantic representation to the receiving end after channel encoding. The receiving end receives the retransmitted semantic information, obtains the retransmitted semantic representation after decoding, performs maximum ratio merging on the index corresponding to the retransmitted semantic base and the index of the received semantic base, restores the semantic base based on the merged index, that is, obtains the merged semantic base through matching the semantic base knowledge base, and performs the next round of testing based on the merged semantic base until it is detected that the accuracy of the restored semantic base reaches the preset requirement, or the upper limit of the number of retransmissions is reached.

[0125] In some embodiments, the receiving end recovers the semantic basis set based on the verification information Perform error detection and obtain fine-grained error detection results Among them, e i,(t) Semantic base The test result includes the existence of semantic errors or the absence of semantic errors, and t is the number of transmissions. The receiving end will test the result E (t) Sent to the sender. The sender receives the test result E (t) After that, the semantic base S with semantic errors is determined (t) , for semantic bases S with semantic errors (t) After processing, it is resent to the receiving end. The receiving end receives the retransmitted semantic base The corresponding index is compared with the received semantic base The indexes of the received semantic base are combined by maximum ratio, and then the next round of error detection process is carried out, that is, the index of the semantic base received for the tth time is combined by maximum ratio with the indexes of the same semantic base received from the 1st to the t-1th time, and then a subsequent check is performed. Error detection and retransmission are performed according to the above process until the semantic accuracy of the verification reaches the preset accuracy threshold, or the upper limit of the number of retransmissions is reached, and the information verification is completed.

[0126] In some embodiments, an error detection and retransmission process is implemented using an error detection and retransmission module based on a deep learning network, wherein the loss function of the error detection and retransmission module is:

[0127] L=L ctx +L comp +L BC (4)

[0128] Among them, L ctx is the loss of the context estimation module, L comp To check the loss of the encoding module and the decoding module, L BC To compensate for the loss of the binary error detection module, the context estimation module, the check coding module, the check decoding module and the binary error detection module are jointly trained to realize the error detection and retransmission function of the error detection and retransmission module.

[0129] For the context estimation module, the loss L ctx , defined as the estimated probability Q S The cross entropy with the semantic basis set S, the loss function is:

[0130]

[0131] Where M is the set of final means of the restored semantic bases, ∑ 2 is the set of variances of the recovered semantic bases, and S is the set of semantic bases sent by the sender. The context estimation module fits the semantic probability distribution that is not contaminated by noise by minimizing the cross entropy, and obtains the test statistics of each semantic base contained in it by substituting it into the calculated noisy semantics, which can effectively support the implementation of the error detection mechanism.

[0132] For the loss of verification information Lcomp , that is, the loss function of the check encoding module and the check decoding module, expressed as:

[0133]

[0134] in, is the distortion measure of the verification information, max(-log2P(b)-b th ,0) is the verification information b exceeds the preset threshold b th Part of the expenses. ctx is the mean set of the mean values ​​of the semantic bases output by the context estimation module at the sending end, M + The mean correction value set is composed of the mean correction values ​​of each semantic basis encoded by the verification information encoding module of the sending end.

[0135] For the binary error detection module, the loss L BC , defined as the test result Q E With error pattern E gt The cross entropy of , the loss function is:

[0136]

[0137] f BC (·) is the binary error detection module, Z is the test statistic of each semantic basis recovered, and w is the weight parameter controlling the binary category, which can improve the recall / precision of error detection by increasing / decreasing it.

[0138] S205: Reconstruct the original information based on the verified semantic base.

[0139] In this embodiment, the receiving end verifies the restored multiple semantic bases to obtain verified semantic bases, and reconstructs the original information based on the verified semantic bases, thereby realizing semantic information transmission that meets task requirements between the sending end and the receiving end.

[0140] In some implementations, taking the original information as image information as an example, the image information to be transmitted by the sending end is The sending end uses the semantic feature extraction module to extract semantic features from image information. Among them, N p is the number of semantic features of image information; the semantic feature F is matched with the semantic base knowledge base to obtain a semantic base set consisting of multiple semantic bases The number of semantic bases is consistent with the number of semantic features. The index of each semantic base is determined according to multiple semantic bases to obtain the semantic representation. For N pThe method comprises the following steps: obtaining an index of a semantic base; performing channel coding on the semantic representation by using a channel coding module to obtain an encoded semantic representation; and sending the encoded semantic representation to a receiving end through a channel.

[0141] After the sender obtains the semantic basis set S, it inputs the semantic basis set S into the context estimation module. The context estimation module models the context association contained in S and outputs the probability estimation result of each semantic basis s in the semantic basis set S. The probability estimation result of semantic basis s is for An element of is the mean set of semantic bases, is the mean corresponding to the semantic base s1, is the standard deviation set of semantic bases, σ1 corresponds to the standard deviation of semantic base s1; calculate the mean set M ctx The mean of each semantic base in and the mean correction value of the corresponding semantic base in the semantic base set S are calculated, that is, the mean of the semantic base is subtracted from the semantic base to obtain the mean correction value of the semantic base, the mean correction value of each semantic base is input into the verification information encoding module, the verification information encoding module is used to encode the mean correction value of each semantic base to obtain verification information b, and the verification information is transmitted to the receiving end through an error-free channel.

[0142] The receiving end receives the semantic information and uses the channel decoding module to decode the semantic information to obtain the decoded semantic representation. Match the decoded semantic representation with the semantic base knowledge base to obtain a semantic base set consisting of multiple restored semantic bases. The recovered multiple semantic bases are verified using the received verification information to obtain the verified multiple semantic bases, and then the semantic reconstruction module is used to reconstruct the image information based on the verified multiple semantic bases.

[0143] The above is an example of the general process of semantic communication using image information. The same processing logic applies to other modalities such as text information and voice information, except that the dimensions of the parameters are different. Its dimension is 3×H×W; for speech information, the Mel cepstral coefficient can be used to express it as Among them, K is the dimension of Mel-frequency cepstral coefficients, and T is the number of speech frames; accordingly, the dimensions of parameters such as feature extraction module, semantic features and semantic basis can also be adjusted accordingly.

[0144] The fine-grained hybrid automatic retransmission method based on semantic basis provided in the embodiment of the present application builds a semantic knowledge base based on an explicit semantic basis, the sending end generates verification information for error detection based on the fine-grained semantic basis, the receiving end performs noisy semantic error detection on the restored semantic basis using the verification information, and when it is determined that there is a semantic error in the semantic basis, the sending end is requested to retransmit the erroneous semantic basis, thereby realizing information retransmission refined to the granularity of the semantic basis, which can ensure the accuracy of semantic error detection, reduce retransmission overhead, reduce transmission delay, and achieve the effectiveness and reliability of communication in complex channel environments.

[0145] The following is a description of the effects of the method of the present application in conjunction with specific embodiments. Taking the transmission of image information as an example, considering that the channel coefficients follow a standard complex Gaussian distribution The Rayleigh fading channel with a signal-to-noise ratio of 3.5 dB is simulated, and a 1 / 2 code rate block error correction code LDPC is used, and the channel coding and modulation method uses 4QAM. The method of the present application (denoted as Ours in the figure) is compared with the traditional hybrid automatic retransmission semantic base information transmission framework (Ours, CRC), the traditional hybrid automatic retransmission JPEG2000, BPG image format, and the image transmission scheme based on machine learning (Balle18 and VQ23). Figure 6-8 As shown in the figure, the method of the present application is superior to other traditional methods in terms of image reconstruction perceptual quality (LPIPS, Learned Perceptual Image PatchSimilarity), image reconstruction multi-scale structural similarity (MS-SSIM, Multi-Scale StructuralSimilarity Index), total transmission overhead and other indicators.

[0146] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0147] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] like Fig. 9As shown, the embodiment of the present application provides a semantic-based fine-grained hybrid automatic retransmission device, which is applied to a sending end and includes:

[0149] Extraction module, used to extract semantic features from raw information;

[0150] A matching module is used to match multiple semantic bases that meet the preset task requirements and a semantic representation composed of indexes of multiple semantic bases from a preset semantic base knowledge base according to semantic features;

[0151] The encoding module is used to perform channel encoding on the semantic representation according to a preset encoding method to obtain an encoded semantic representation;

[0152] A generation module, used for generating verification information based on multiple semantic bases;

[0153] The sending module is used to send the encoded semantic representation to the receiving end via the channel; and send the verification information to the receiving end, so that the receiving end verifies the received information based on the verification information.

[0154] like Fig.10 As shown, the embodiment of the present application provides a semantic-based fine-grained hybrid automatic retransmission device, which is applied to a receiving end, including:

[0155] A receiving module, used for receiving semantic information and verification information;

[0156] A decoding module, used to decode the semantic information to obtain a decoded semantic representation;

[0157] A matching module, used for matching a plurality of restored semantic bases from a semantic base knowledge base according to the decoded semantic representation;

[0158] A verification module, used to verify the restored multiple semantic bases using the verification information to obtain verified semantic bases;

[0159] The reconstruction module is used to reconstruct the original information based on the verified semantic basis.

[0160] For the convenience of description, the above devices are described in terms of functions divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0161] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0162] Fig.11A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0163] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0164] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0165] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0166] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0167] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0168] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0169] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0170] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0171] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0172] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0173] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0174] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present disclosure.

Claims

1. A semantic-based fine-grained hybrid automatic retransmission method, applied to a transmitter, characterized in that: include: Extract semantic features from raw information; According to the semantic features, multiple semantic bases that meet the preset task requirements and semantic representations composed of indexes of multiple semantic bases are matched from a preset semantic base knowledge base; Channel-encoding the semantic representation according to a preset encoding method to obtain an encoded semantic representation; Sending the encoded semantic representation to a receiving end via a channel; Generate verification information based on multiple semantic bases; The verification information is sent to the receiving end, so that the receiving end verifies the received information based on the verification information.

2. The method according to claim 1, characterized in that The generating of verification information based on multiple semantic bases includes: Inputting a plurality of semantic bases into a preset context estimation module, and the context estimation module outputs a probability estimation result of each semantic base; wherein the probability estimation result includes a mean value; Calculate the mean correction value between the mean of each semantic base and the corresponding semantic base; The mean correction value of each semantic base is input into a preset verification information encoding module, and the verification information encoding module outputs verification information.

3. The method according to claim 1, characterized in that: The semantic base knowledge base is constructed based on an explicit semantic base, and the method for generating the explicit semantic base includes: Extract semantic features from existing information; According to a preset clustering rule, an explicit semantic base is extracted from the semantic features.

4. A semantic-based fine-grained hybrid automatic retransmission method, applied to a receiving end, characterized in that: include: receiving semantic information and verification information; performing channel decoding on the semantic information to obtain a decoded semantic representation; According to the decoded semantic representation, multiple semantic bases are restored by matching from a preset semantic base knowledge base; Verifying the restored multiple semantic bases using the verification information to obtain verified semantic bases; Reconstruct the original information based on the verified semantic basis.

5. The method according to claim 4, characterized in that The method of verifying the restored multiple semantic bases using the verification information to obtain the verified semantic bases includes: Inputting the restored multiple semantic bases into a preset context estimation module, and the context estimation module outputs a probability estimation result of each restored semantic base; wherein the probability estimation result includes a mean and a variance; Inputting the verification information into a preset verification information decoding module, and the verification information decoding module outputs the mean correction value of each semantic basis; Determining the final mean of each restored semantic base according to the mean of each restored semantic base and the mean correction value of the corresponding semantic base; Conduct hypothesis testing based on the final mean and variance of each restored semantic base to obtain the test results of each semantic base; If the verification result contains a semantic basis that needs to be retransmitted, the verification result is sent to the sending end, so that the sending end resends the semantic basis that needs to be retransmitted according to the verification result.

6. The method according to claim 5, characterized in that Also includes: Receiving retransmitted semantic information; wherein the retransmitted semantic information is obtained after the sending end retransmits the semantic base to be retransmitted based on the verification result and transmits it through the channel; The next round of error detection is performed based on maximum ratio combining of the retransmitted semantic information and the semantic information.

7. The method according to claim 6, characterized in that Performing maximum ratio combining based on the retransmitted semantic information and the semantic information, including: Decoding the retransmitted semantic information to obtain a retransmitted semantic representation; The index corresponding to the semantic base in the retransmitted semantic representation is combined with the index of the received semantic base by maximum ratio.

8. The method according to claim 5, characterized in that The hypothesis test is performed based on the final mean and variance of each restored semantic base to obtain the test results of each semantic base, including: Based on the final mean and variance of each semantic basis, the significance threshold adaptively generated by the binary error detection module is used to perform hypothesis testing and obtain the test results.

9. A semantic-based fine-grained hybrid automatic retransmission device, applied to a transmitting end, characterized in that: include: Extraction module, used to extract semantic features from raw information; A matching module is used to match multiple semantic bases that meet the preset task requirements and a semantic representation composed of indexes of multiple semantic bases from a preset semantic base knowledge base according to semantic features; A coding module, used for performing channel coding on the semantic representation according to a preset coding method to obtain a coded semantic representation; A generation module, used for generating verification information based on multiple semantic bases; The sending module is used to send the encoded semantic representation to the receiving end via the channel; and send the verification information to the receiving end, so that the receiving end verifies the received information based on the verification information.

10. A semantic-based fine-grained hybrid automatic retransmission device, applied to a receiving end, characterized in that: include: A receiving module, used for receiving semantic information and verification information; A decoding module, used for decoding the semantic information to obtain a decoded semantic representation; A matching module, used for matching and restoring a plurality of semantic bases from a preset semantic base knowledge base according to the decoded semantic representation; A verification module, used to verify the restored multiple semantic bases using the verification information to obtain verified semantic bases; The reconstruction module is used to reconstruct the original information based on the verified semantic basis.

Images