A fine-grained hybrid automatic repeat request method and device based on a semantic base

Abstract

The embodiment of the application provides a fine-grained hybrid automatic repeat method and device based on a semantic base, a semantic knowledge base is constructed based on an explicit semantic base, a sending end generates check information for error detection based on a fine-grained semantic base, a receiving end performs noisy semantic error detection on a recovered semantic base by using the check information, and when it is judged that the semantic base has semantic errors, the sending end is requested to retransmit the erroneous semantic base, so that information retransmission is realized to the granularity of the semantic base, the semantic error detection accuracy is ensured, the retransmission cost is reduced, the transmission delay is reduced, and the effectiveness and reliability of communication in a complex channel environment are realized.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of communication, in particular to a fine-grained hybrid automatic repeat method and device based on semantic base. BACKGROUND

[0002] Semantic communication focuses on the "meaning" transmission of information, and has been widely studied in recent years. Compared with traditional syntax communication based on Shannon information theory, semantic communication focuses on the correct reception of each transmission symbol, and focuses on the content meaning of the transmitted information rather than the encoded symbol, which can improve the communication efficiency while fully meeting the communication demand. The current semantic communication does not consider the problem of transmission error caused by interference, and does not provide a processing method for error detection and correction, which affects the communication quality. SUMMARY

[0003] Therefore, the embodiment of the present application aims to provide a fine-grained hybrid automatic repeat method and device based on semantic base to solve the problem of transmission error of semantic communication.

[0004] To achieve the above purpose, the embodiment of the present application provides a fine-grained hybrid automatic repeat method based on semantic base, applied to a sending end, comprising:

[0005] extracting semantic features from original information;

[0006] According to the semantic features, a plurality of semantic bases satisfying the preset task requirements and a semantic representation composed of indexes of the plurality of semantic bases are matched from a preset semantic base knowledge base;

[0007] The semantic representation is channel encoded according to a preset encoding mode to obtain an encoded semantic representation;

[0008] The encoded semantic representation is sent to a receiving end through a channel;

[0009] Based on the plurality of semantic bases, a check information is generated;

[0010] The check information is sent to the receiving end, so that the receiving end checks the received information based on the check information.

[0011] Optionally, the generating of the check information based on the plurality of semantic bases comprises:

[0012] The plurality of semantic bases are input into a preset context estimation module, and a probability estimation result of each semantic base is output by the context estimation module; wherein the probability estimation result comprises a mean value;

[0013] The mean value of each semantic base and the mean value correction value between the corresponding semantic base are calculated;

[0014] Input the mean correction value of each semantic base into a preset check information encoding module, and output check information from the check information encoding module.

[0015] Optionally, the semantic base knowledge base is constructed based on explicit semantic bases, and a method for generating the explicit semantic bases comprises:

[0016] extracting semantic features from existing information;

[0017] extracting explicit semantic bases from the semantic features according to a preset clustering rule.

[0018] The application also provides a fine-grained hybrid automatic repeat request method based on semantic bases, which is applied to a receiving end and comprises:

[0019] receiving semantic information and check information;

[0020] performing channel decoding on the semantic information to obtain decoded semantic representations;

[0021] According to the decoded semantic representations, a plurality of recovered semantic bases are matched from a preset semantic base knowledge base;

[0022] performing check on the plurality of recovered semantic bases by using the check information to obtain checked semantic bases;

[0023] reconstructing original information based on the checked semantic bases.

[0024] Optionally, the performing check on the plurality of recovered semantic bases by using the check information to obtain checked semantic bases comprises:

[0025] inputting the plurality of recovered semantic bases into a preset context estimation module, and outputting probability estimation results of the recovered semantic bases from the context estimation module; wherein the probability estimation results comprise a mean and a variance;

[0026] inputting the check information into a preset check information decoding module, and outputting mean correction values of each semantic base from the check information decoding module;

[0027] determining final means of the recovered semantic bases according to the means of the recovered semantic bases and the mean correction values of the corresponding semantic bases;

[0028] performing hypothesis testing based on the final means and the variances of the recovered semantic bases to obtain testing results of the semantic bases;

[0029] if the testing results exist semantic bases that need to be retransmitted, sending the testing results to a sending end, so that the sending end retransmits the semantic bases that need to be retransmitted according to the testing results.

[0030] Optionally, the method further comprises:

[0031] receiving retransmission semantic information; wherein the retransmission semantic information is obtained after the sending end retransmits the semantic bases that need to be retransmitted based on the check result and transmits the retransmission semantic information through a channel;

[0032] performing maximum ratio combining based on the retransmission semantic information and the semantic information to perform next round error detection.

[0033] Optionally, the maximum ratio combining based on the retransmission semantic information and the semantic information comprises:

[0034] decoding the retransmission semantic information to obtain retransmission semantic representation;

[0035] performing maximum ratio combining on the index corresponding to the semantic base in the retransmission semantic representation and the index of the received semantic base.

[0036] Optionally, the hypothesis testing based on the final mean and variance of each semantic base to obtain the test result of each semantic base comprises:

[0037] performing hypothesis testing based on the final mean and variance of each semantic base and the significance threshold value adaptively generated by the binary error detection module to obtain the test result.

[0038] Embodiments of the present application also provide a fine-grained hybrid automatic retransmission device based on semantic bases, applied to a sending end, comprising:

[0039] an extraction module configured to extract semantic features from original information;

[0040] a matching module configured to match a plurality of semantic bases and semantic representation composed of indexes of the plurality of semantic bases that meet preset task requirements from a preset semantic base knowledge base according to the semantic features;

[0041] an encoding module configured to perform channel encoding on the semantic representation according to a preset encoding mode to obtain encoded semantic representation;

[0042] a generation module configured to generate check information based on the plurality of semantic bases;

[0043] a sending module configured to send the encoded semantic representation to a receiving end through a channel and send the check information to the receiving end, so that the receiving end checks the received information based on the check information.

[0044] Embodiments of the present application also provide a fine-grained hybrid automatic retransmission device based on semantic bases, applied to a receiving end, comprising:

[0045] a receiving module configured to receive semantic information and check information;

[0046] a decoding module, configured to decode the semantic information to obtain a decoded semantic representation;

[0047] a matching module, configured to match, according to the decoded semantic representation, a plurality of recovered semantic bases from a preset semantic base knowledge base;

[0048] a checking module, configured to check the plurality of recovered semantic bases by using the checking information to obtain checked semantic bases;

[0049] a reconstruction module, configured to reconstruct the original information based on the checked semantic bases.

[0050] As can be seen from the above, the method and device for fine-grained hybrid automatic repeat based on semantic bases provided by the embodiments of the present application construct a semantic knowledge base based on explicit semantic bases, the sending end generates checking information for error detection based on fine-grained semantic bases, the receiving end uses the checking information to perform noisy semantic error detection on the recovered semantic bases, and when it is determined that the semantic bases have semantic errors, the sending end is requested to retransmit the erroneous semantic bases, thereby realizing information retransmission refined to the granularity of semantic bases, ensuring the accuracy of semantic error detection, reducing the retransmission overhead, reducing the transmission delay, and realizing the effectiveness and reliability of communication in a complex channel environment. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0052] Figure 1 a method flowchart for the method of the embodiments of the present application applied to the sending end;

[0053] Figure 2 a method flowchart for the method of the embodiments of the present application applied to the receiving end;

[0054] Figure 3 a system model schematic diagram of the embodiments of the present application;

[0055] Figure 4 a detection and retransmission model schematic diagram of the embodiments of the present application;

[0056] Figure 5 an error diffusion example diagram of the embodiments of the present application;

[0057] Figure 6 a curve diagram of the change of the perceptual quality of image reconstruction with the number of retransmissions under different methods of the embodiments of the present application;

[0058] Figure 7 Fig. 3 is a schematic diagram of a curve of the multi-scale structural similarity of image reconstruction under different methods of embodiments of the present application versus the number of retransmissions;

[0059] Figure 8 Fig. 4 is a schematic diagram of a curve of the total transmission overhead under different methods of embodiments of the present application versus the number of retransmissions;

[0060] Figure 9 Fig. 5 is a block diagram of a device structure of a sending end of embodiments of the present application;

[0061] Figure 10 Fig. 6 is a block diagram of a device structure of a receiving end of embodiments of the present application;

[0062] Figure 11 Fig. 7 is a block diagram of an electronic device structure of embodiments of the present application. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, further detailed explanations will be given below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0064] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood as the common meanings understood by those skilled in the art to which the present disclosure belongs. The terms "first", "second", and similar terms used in the embodiments of the present application do not represent any order, number, or importance, but are only used to distinguish different components. The terms "include", "contain", and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, without excluding other elements or objects. The terms "connect" or "connected" and similar terms do not mean only physical or mechanical connections, but can also include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like only represent relative positional relationships, which can change accordingly when the absolute positions of the described objects change.

[0065] In the related art, semantic communication can significantly improve the quality of service and communication efficiency compared to traditional syntax communication. The related semantic communication system does not provide error detection and correction mechanisms, and is affected by interference factors, which reduces the communication quality. The error detection mode of the traditional hybrid automatic retransmission method focuses on bit errors and ignores information meaning and communication intent, and cannot be applied to semantic communication for error detection and correction.

[0066] Therefore, the embodiments of the present application provide a fine-grained hybrid automatic retransmission method based on a semantic base, construct a semantic knowledge base of an explicit semantic base, use a hybrid automatic retransmission framework to verify the fine-grained semantic base, and improve the reliability and communication quality of semantic communication in a noisy environment.

[0067] The technical solutions of the present application are further described in detail below through specific embodiments.

[0068] As shown in Figure 1 , 3 The embodiment of the present application provides a fine-grained hybrid automatic repeat request method based on semantic base, which is applied to a sending end, and the method comprises the following steps:

[0069] S101: extracting semantic features from original information;

[0070] In the embodiment, for the original information of the sending end, according to the application scene and task requirement, a semantic feature extraction module is used to extract semantic features from the original information. The original information includes but is not limited to image information, text information, audio information, etc. In different application scenes, based on specific task requirements, a corresponding semantic feature extraction module is pre-trained, and the semantic feature extraction module is used to extract features from the original information to obtain semantic features.

[0071] S102: according to the semantic features, matching a plurality of semantic bases and a semantic representation composed of indexes of the plurality of semantic bases that meet the pre-set task requirement from a pre-set semantic base knowledge base;

[0072] In the embodiment, the semantic base knowledge base is constructed based on explicit semantic base (Semantic Base, Seb), and the sending end and the receiving end have a synchronous semantic base knowledge base to ensure that the semantic understanding of the same information by both parties is the same and to ensure the normal progress of semantic communication. Based on the extracted semantic features, the semantic base knowledge base is used to match a plurality of semantic bases and a semantic representation composed of indexes of the plurality of semantic bases corresponding to the semantic features. The matching method of the semantic features in the semantic base knowledge base can be determined according to the pre-set task requirement, the matched semantic bases can be at least one semantic base meeting the task requirement and / or a semantic base set combined in a specific way from a plurality of semantic bases; each semantic base is encoded according to a specific index encoding method to obtain an index corresponding to each semantic base, and a semantic representation is obtained according to the indexes of the matched plurality of semantic bases.

[0073] In some embodiments, the method for generating explicit semantic bases comprises:

[0074] extracting semantic features from existing information;

[0075] According to a pre-set clustering rule, a pre-set clustering algorithm is used to cluster the semantic features to obtain explicit semantic bases.

[0076] In the embodiment, the method for generating explicit semantic bases is specifically: for the existing information N is the number of existing information, and a semantic feature extraction module is used to extract a group of semantic features from the existing information N p is the number of semantic features in the existing information, is the semantic feature, c, h, w are the dimensions of the feature, is the n p th semantic feature of the i th existing information; then, according to a preset clustering rule, a clustering algorithm is used to cluster the semantic features , wherein K is the number of preset semantic bases.

[0077] In some manners, a semantic base knowledge base is constructed based on the generated explicit semantic bases, and the semantic base knowledge base is synchronously transmitted to the sending end and the receiving end. The existing information is, for example, image information, text information, audio information and the like that has been transmitted between the sending end and the receiving end. Optionally, the semantic base knowledge base can be constructed based on the partial order relation of the semantic bases.

[0078] In some embodiments, the method for generating explicit semantic bases is implemented based on a semantic base generation module of a deep learning network model, and a loss function of the semantic base generation module is:

[0079]

[0080] , wherein L D is a loss function of distortion of the original information reconstructed by the receiving end, a mean square error is adopted, L reg is a regularization term of clustering operation, sg(·) is a gradient truncation operation, I is the original information, is the original information reconstructed by the receiving end, F is a semantic feature extracted from the original information. Since the clustering operation is not derivable, a straight-through estimation is adopted to approximate the gradient, and the regularization term is used to solve the feature space expansion problem caused by the straight-through estimation; since the semantic base S is the clustering center and does not contain trainable parameters, the gradient of the semantic base is truncated.

[0081] S103: Channel encoding is performed on the semantic representation according to a preset encoding mode, to obtain an encoded semantic representation;

[0082] S104: The encoded semantic representation is sent to the receiving end through a channel;

[0083] In this embodiment, the semantic representation is input into a preset channel coding module, the channel coding module encodes the semantic representation according to a preset encoding mode to generate an encoded semantic representation, and the encoded semantic representation is sent to the receiving end through a channel. The channel coding and channel decoding are implemented based on a deep neural network model, and the network structure is flexibly configured according to specific application scenarios, signal source modalities, transmission requirements, etc.; the channel coding and channel decoding can also be implemented based on a non-machine learning method; the channel coding and decoding mode can be implemented by using any known encoding mode, and the encoding and decoding mode is not specifically explained.

[0084] S105: generating check information based on the plurality of semantic bases;

[0085] S106: sending the check information to the receiving end to enable the receiving end to check the received information based on the check information.

[0086] In this embodiment, the check information is generated based on the plurality of semantic bases, and the check information is sent to the receiving end through an error-free channel, which is used for information checking between the sending end and the receiving end, thereby improving the communication reliability.

[0087] The method for generating the check information includes:

[0088] The plurality of semantic bases are input into a preset context estimation module, and the context estimation module outputs a probability estimation result of each semantic base; the probability estimation result includes a mean value and a variance;

[0089] A mean value correction value between the mean value of each semantic base and the corresponding semantic base is calculated.

[0090] The mean value correction value of each semantic base is input into a preset check information encoding module, and the check information encoding module outputs the check information.

[0091] In this embodiment, the plurality of semantic bases generated by the sending end based on the original information are used to generate the check information. The plurality of semantic bases are input into a context estimation module, and the context estimation module outputs a probability estimation result of each semantic base, which includes a mean value and a variance. A difference value between the mean value of each semantic base and the corresponding semantic base is calculated as a mean value correction value of the semantic base. The mean value correction value of each semantic base is input into a check information encoding module, and the check information encoding module encodes the mean value correction value to obtain the check information.

[0092] In some ways, the check information encoding module removes redundant information and performs compression encoding based on the input mean value correction value to obtain the check information. In the receiving end, the check information is decoded and reconstructed by using a corresponding check information decoding module to recover the check information.

[0093] As Figure 2 , 4, 5, the embodiment of the application provides a kind of based on semantic base's fine-grained hybrid automatic repeat method, it is applied to receiving end, method includes:

[0094] S201: receiving semantic information and check information;

[0095] S202: semantic information is channel decoded, and the semantic representation after decoding is obtained;

[0096] In the embodiment, the receiving end receives semantic information and check information from the channel, inputs the semantic information into the preset channel decoding module, and decodes the semantic information according to the preset decoding mode by the channel decoding module to obtain the semantic representation after decoding, and checks based on the semantic representation after decoding using the check information. The decoding mode of the channel decoding module is matched with the encoding mode of the channel encoding module, so that the channel decoding module can decode and recover the information before encoding.

[0097] S203: according to the semantic representation after decoding, a plurality of semantic bases recovered are matched from semantic base knowledge base;

[0098] In the embodiment, the receiving end determines the index of the plurality of semantic bases in the semantic representation according to the semantic representation after decoding, and matches the corresponding plurality of semantic bases using the semantic base knowledge base according to the index of the plurality of semantic bases, as the plurality of semantic bases recovered. Due to the existence of channel interference and other factors, there may be errors in the plurality of semantic bases recovered by the receiving end.

[0099] S204: check the plurality of semantic bases recovered using check information, and obtain the plurality of semantic bases after checking;

[0100] In the embodiment, after obtaining the plurality of semantic bases recovered, the plurality of semantic bases recovered are checked using the check information, and the method includes:

[0101] Input the plurality of semantic bases recovered into the preset context estimation module, and output the probability estimation result of each semantic base recovered by the context estimation module;The probability estimation result includes mean and variance;

[0102] Input check information into the preset check information decoding module, and output the mean correction value of each semantic base by the check information decoding module;

[0103] According to the mean of each semantic base recovered and the mean correction value of the corresponding semantic base, determine the final mean of each semantic base recovered;

[0104] Based on the final mean and variance of each semantic base recovered, hypothesis testing is carried out, and the test result of each semantic base is obtained;

[0105] If the check result exists a semantic base needing retransmission, the check result is sent to the sending end, so that the sending end retransmits the semantic base needing retransmission according to the check result.

[0106] In this embodiment, after the plurality of recovered semantic bases are obtained, the plurality of recovered semantic bases are input into a context estimation module, and a probability estimation result of the plurality of recovered semantic bases is output by the context estimation module, the probability estimation result including a mean value and a variance of each semantic base; the received check information is input into a check information decoding module, and a mean value correction value of each recovered semantic base is obtained by decoding the check information by the check information decoding module, the mean value of each semantic base output by the context estimation module is superimposed with the mean value correction value of the corresponding recovered semantic base to obtain a final mean value of each semantic base, and hypothesis testing is performed based on the final mean value and the variance of each semantic base to obtain a check result, and the check result is determined according to the check result.

[0107] In some ways, the context estimation module of the sending end and the context estimation module of the receiving end have the same network parameters. The check information decoding module and the check information encoding module are respectively an encoder and a decoder constructed based on a CNN network.

[0108] In some ways, the context estimation module of the receiving end outputs a probability estimation result wherein, is a mean value set composed of the mean values of the recovered semantic bases, and the mean value correction value set composed of the mean value correction values of the recovered semantic bases is obtained by decoding the check information by the check information decoding module The mean value set is superimposed with the mean value correction value of the corresponding semantic base in the mean value correction value set to obtain the final mean value of each recovered semantic base, for example, the mean value of the i th semantic base in the mean value set is superimposed with the mean value correction value of the i th semantic base in the mean value correction value set to obtain the final mean value of the i th semantic base.

[0109] After the final mean value of each semantic base is determined, hypothesis testing is performed based on the final mean value and the variance of each semantic base, and the hypothesis testing can be represented as:

[0110]

[0111] wherein, is the i th recovered semantic base, is the final mean value of the i th semantic base. The original hypothesis H0 indicates that the semantic base There is no semantic error through testing; the alternative hypothesis H1 indicates that the semantic base There is a semantic error and retransmission is needed.

[0112] In the hypothesis testing process, a semantic basis set composed of the recovered plurality of semantic bases is calculated a test statistic The calculation method is:

[0113]

[0114] wherein z i is a test statistic of the i-th recovered semantic basis σ i is a standard deviation of the i-th recovered semantic basis N p is the number of recovered semantic bases.

[0115] The test is performed on each recovered semantic basis, that is, the test statistic of each recovered semantic basis is judged based on the corresponding significance threshold, and it is determined according to the judgment result whether there is a semantic error in the semantic basis. That is, in this embodiment, by analyzing the difference in statistical characteristics between the noisy semantics and the semantics not contaminated by noise, a hypothesis testing problem is constructed, the test statistic of each semantic basis in the noisy semantics is obtained, and further by analyzing the relevant information in the test statistic, the judgment threshold of each semantic element is adaptively generated, so as to realize the error detection and correction verification based on the fine-grained semantic basis, and improve the retransmission efficiency.

[0116] In some ways, the hypothesis test on the fine-grained semantic basis can be realized based on the error detection and retransmission module. The error detection and retransmission module includes a mask convolution layer and a point-by-point convolution layer, wherein the convolution kernel size of the mask convolution layer is 5*5, and the center element is blocked, and the value probability of each center semantic basis is modeled based on the corresponding 24 surrounding semantic bases through convolution operation; the point-by-point convolution layer is used to deepen the network fitting ability under the premise of ensuring the causality constraint. The network structure of the error detection and retransmission module can effectively realize the context-based semantic probability modeling, and since it is completely composed of convolution network, it has good parallel computing ability, which ensures the efficiency of the calculation.

[0117] In some embodiments, the hypothesis test is performed based on the final mean and variance of each semantic basis to obtain a test result, including:

[0118] Based on the final mean and variance of each semantic basis, the significance threshold adaptively generated by the binary error detection module is used to perform hypothesis test to obtain a test result.

[0119] In this embodiment, as Figure 5As shown, considering the error diffusion phenomenon of the test statistics Z of the recovered multiple semantic bases, that is, the error of a certain semantic base not only causes the test statistics corresponding to the semantic base to rise, but also causes the test statistics corresponding to the adjacent semantic bases which are the context of the semantic base to fluctuate, the way of using a single significance threshold for judgment may cause misjudgment. Therefore, in order to offset the influence of error diffusion, the binary error detection module generates a significance threshold based on the test statistics Z, and the adaptive significance threshold is used for hypothesis testing to improve the correctness of error detection.

[0120] In some ways, the binary error detection module is trained based on a supervised learning method by constructing the features and corresponding labels of samples, that is, constructing the error semantic bases and corresponding labels, the adjacent semantic bases of the error semantic bases and corresponding labels, the correct semantic bases and corresponding labels, and training the model to obtain the binary error detection module after training, which can obtain accurate test results with a suitable significance threshold.

[0121] In some embodiments, the recovered multiple semantic bases are verified using the check information, which further includes:

[0122] Receiving the retransmitted semantic information; wherein the retransmitted semantic information is obtained after the sending end retransmits the semantic bases that need to be retransmitted based on the check result and transmits them through the channel;

[0123] Based on the retransmitted semantic information and the received semantic information, maximum ratio combining is performed for the next round of error detection.

[0124] In this embodiment, after determining the test results of each semantic base by hypothesis testing, the test results of all semantic bases are sent to the sending end. The sending end determines the semantic bases that need to be retransmitted according to the received test results, determines the corresponding indexes according to the semantic bases that need to be retransmitted, forms a semantic representation according to the indexes of the retransmitted semantic bases, and sends the semantic representation to the receiving end after channel encoding. The receiving end receives the retransmitted semantic information, decodes it to obtain the retransmitted semantic representation, performs maximum ratio combining on the indexes of the retransmitted semantic bases and the received semantic bases, recovers the semantic bases based on the combined indexes, that is, matches the combined semantic bases through the semantic base knowledge base, and performs the next round of test based on the combined semantic bases until the accuracy of the recovered semantic bases reaches the preset requirement or the upper limit of the number of retransmissions is reached.

[0125] In some ways, the receiving end performs error detection on the recovered semantic base set based on the check information to obtain a fine-grained error detection result wherein e i,(t) is a semantic base , the test result includes the existence of semantic errors or the non-existence of semantic errors, t is the number of transmissions. The receiving end transmits the test result E (t) to the sending end. The sending end receives the test result E (t) , and then determines the semantic base S (t) with semantic errors. After processing the semantic base S (t) with semantic errors, it is retransmitted to the receiving end. The receiving end receives the retransmitted semantic base with the corresponding index, and the index is maximum ratio combined with the index of the same semantic base that has been received, and then the next round of error detection process is performed, that is, the index of the semantic base received for the tth time is maximum ratio combined with the index of the same semantic base received for the 1st to t-1th time, and subsequent verification is performed. According to the above process, error detection and retransmission are performed until the semantic accuracy of the check reaches the preset accuracy threshold or the upper limit of the number of retransmissions is reached, and the information check is completed.

[0126] In some ways, the error detection and retransmission module based on the deep learning network is used to realize the error detection and retransmission process, wherein the loss function of the error detection and retransmission module is:

[0127] L = L ctx + L comp + L BC (4)

[0128] Wherein, L ctx is the loss of the context estimation module, L comp is the loss of the check encoding module and the check decoding module, and L BC is the loss of the binary error detection module. The context estimation module, the check encoding module, the check decoding module and the binary error detection module are jointly trained to realize the error detection and retransmission function of the error detection and retransmission module.

[0129] For the loss L ctx of the context estimation module, the cross entropy of the estimated probability Q S and the semantic base set S is defined, and the loss function is:

[0130]

[0131] Wherein, M is the final mean set of each semantic base recovered, and ∑ 2 is the variance set of each semantic base recovered, and S is the semantic base set sent by the sending end. The context estimation module fits the semantic probability distribution not contaminated by noise by minimizing the cross entropy, and obtains the test statistics of each semantic base contained in the noisy semantics by substituting the calculation, which can effectively support the implementation of the error detection mechanism.

[0132] For the loss Lcomp The loss function of the check encoding module and the check decoding module is represented as:

[0133]

[0134] wherein, is a distortion measure of the check information, max(-log2P(b)-b th , 0) is a check information b exceeding a preset threshold b th The overhead of the M ctx is a mean value set of the mean values of each semantic base output by the context estimation module of the sending end, M + is a mean value set of the mean value correction values of each semantic base encoded by the check information encoding module of the sending end.

[0135] For the loss L BC of the binary error detection module, the loss function is defined as the cross entropy of the verification result Q E and the error pattern E gt .

[0136]

[0137] f BC (·) is the binary error detection module, Z is the verification statistic of each semantic base recovered, and w is a weight parameter for controlling the binary category, which can improve the recall rate / accuracy of error detection by increasing / decreasing.

[0138] S205: Reconstruct the original information based on the checked semantic base.

[0139] In this embodiment, the receiving end obtains the checked semantic base after checking the recovered multiple semantic bases, reconstructs the original information based on the checked semantic base, and realizes the transmission of semantic information between the sending end and the receiving end that meets the task requirements.

[0140] In some embodiments, taking the original information as image information as an example, the image information to be transmitted by the sending end is The sending end extracts semantic features F from the image information by using the semantic feature extraction module. wherein, N p is the number of semantic features of the image information; the semantic features F are matched with the semantic base knowledge base to obtain a semantic base set composed of multiple semantic bases The number of semantic bases is consistent with the number of semantic features, the index of each semantic base is determined according to the multiple semantic bases, and the semantic representation is obtained. is the N pan index of the semantic bases; channel encoding the semantic representation by using a channel encoding module to obtain an encoded semantic representation; and transmitting the encoded semantic representation to a receiving end through a channel.

[0141] After the transmitting end obtains the semantic base set S, the semantic base set S is input into a context estimation module, the context estimation module models the context association contained in S, and outputs the probability estimation result of each semantic base s in the semantic base set S The probability estimation result of the semantic base s is is an element in S, where M is a mean value set of the semantic bases, and is the mean value corresponding to the semantic base s1. is a standard deviation set of the semantic bases, and ctx The mean value of each semantic base in S is corrected by subtracting the mean value of the corresponding semantic base in S, that is, the mean value of the semantic base is subtracted from the mean value of the semantic base to obtain the mean value correction value of the semantic base. The mean value correction values of the semantic bases are input into a check information encoding module, and the check information encoding module is used to encode the mean value correction values of the semantic bases to obtain check information b. The check information is transmitted to the receiving end through an error-free channel.

[0142] The receiving end receives the semantic information, decodes the semantic information by using a channel decoding module, and obtains the decoded semantic representation The decoded semantic representation is matched with the semantic base knowledge base to obtain a semantic base set composed of the recovered multiple semantic bases The recovered multiple semantic bases are checked by using the received check information to obtain checked multiple semantic bases, and the image information is reconstructed based on the checked multiple semantic bases by using a semantic reconstruction module

[0143] The above is a general process of semantic communication taking image information as an example. Text information and voice information and other modal information also apply the above processing logic, only the dimensions of the parameters are different. For example, for image information the dimension is 3xHxW; for voice information, the mel-frequency cepstral coefficient can be expressed as where K is the dimension of the mel-frequency cepstral coefficient, and T is the number of frames of the voice; accordingly, the dimensions of the parameters such as the feature extraction module, the semantic feature, and the semantic base are adjusted accordingly.

[0144] ​The method provided in the embodiment of the application is based on a semantic base, a semantic knowledge base is constructed based on an explicit semantic base, the sending end generates check information for error detection based on a fine-grained semantic base, the receiving end performs noisy semantic error detection on the recovered semantic base by using the check information, when it is determined that the semantic base has semantic errors, the sending end is requested to retransmit the erroneous semantic base, the information retransmission is realized to the granularity of the semantic base, the semantic error detection accuracy can be ensured, the retransmission overhead is reduced, the transmission delay is reduced, and the effectiveness and reliability of communication in a complex channel environment are realized.

[0145] The effects of the method of the application are described below in combination with specific embodiments. Taking the transmission of image information as an example, the channel coefficients are simulated to follow a standard complex Gaussian distribution The channel coefficients are simulated to follow a standard complex Gaussian distribution Figures 6-8 As shown in the figure, in terms of image reconstruction perceptual quality (LPIPS, Learned Perceptual Image Patch Similarity), image reconstruction multi-scale structural similarity (MS-SSIM, Multi-Scale Structural Similarity Index), and total transmission overhead, the method of the application is superior to other traditional methods.

[0146] It should be noted that the method of the embodiment of the application can be executed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scenario and completed by multiple devices in cooperation. In the distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiment of the application, and the multiple devices can interact with each other to complete the method.

[0147] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order other than that described in the embodiments and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing can be advantageous or possible.

[0148] As Figure 9As shown, this application provides a fine-grained hybrid automatic repeater based on semantic bases, applied at the transmitting end, including:

[0149] The extraction module is used to extract semantic features from the raw information;

[0150] The matching module is used to match multiple semantic bases that meet the preset task requirements and the semantic representation composed of the indexes of multiple semantic bases from the preset semantic base knowledge base based on semantic features.

[0151] The encoding module is used to perform channel coding on the semantic representation according to a preset coding method to obtain the encoded semantic representation;

[0152] The generation module is used to generate verification information based on multiple semantic bases;

[0153] The transmitting module is used to transmit the encoded semantic representation to the receiving end via the channel; and to transmit verification information to the receiving end so that the receiving end can verify the received information based on the verification information.

[0154] like Figure 10 As shown, this application provides a fine-grained hybrid automatic repeater based on semantic bases, applied at the receiving end, including:

[0155] The receiving module is used to receive semantic information and verification information.

[0156] The decoding module is used to decode semantic information to obtain the decoded semantic representation;

[0157] The matching module is used to match and recover multiple semantic bases from the semantic base knowledge base based on the decoded semantic representation;

[0158] The verification module is used to verify the recovered semantic bases using verification information to obtain the verified semantic bases.

[0159] The reconstruction module is used to reconstruct the original information based on the verified semantic base.

[0160] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware.

[0161] The apparatus described above is used to implement the corresponding methods in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0162] Figure 11A more specific electronic device hardware structure schematic diagram provided by the embodiment is shown, and the device can include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 for internal communication.

[0163] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present specification.

[0164] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1020 and called and executed by the processor 1010.

[0165] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0166] The communication interface 1040 is used to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0167] The bus 1050 includes a channel for transmitting information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0168] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary to implement the embodiments of the present application, and does not have to contain all the components shown in the figure.

[0169] The electronic device of the above embodiment is used to implement the corresponding method in the foregoing embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0170] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0171] Those skilled in the art should understand that the above discussion of any embodiment is only exemplary and is not intended to imply that the scope of the present disclosure (including claims) is limited to these examples; under the idea of the present disclosure, the above embodiments or technical features in different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the embodiments of the present application as described above. In order to be brief, they are not provided in detail.

[0172] Additionally, to simplify the description and discussion, and so as not to obscure the embodiments of the application being presented, the well-known functions or constructions of integrated circuit (IC) chips and other components can or can not be shown in the figures and will be omitted as not to unnecessarily obscure the embodiments of the application being presented. Moreover, the apparatus can be shown in block diagram form in order to avoid obscuring the embodiments of the application being presented, and this also acknowledges the fact that the details in regard to the implementation of the apparatus of these block diagrams are highly dependent on the platform within which the embodiments of the application are to be implemented (i.e., these details should be well within the purview of one of ordinary skill in the art to consider given the specific application at hand). Where specific details are set forth in order to describe an illustrative embodiment of the disclosure, it will be apparent to one of ordinary skill in the art that the embodiments of the application can be practiced without these specific details. In other instances, detailed descriptions of well-known methods, procedures, components, circuits and the like can omit so as not to unnecessarily obscure aspects of the embodiments of the application. It will also be apparent to one of ordinary skill in the art that embodiments of the application, as described herein, can be implemented in a number of different platforms to satisfy specified performance requirements.

[0173] While the present disclosure has been described with respect to a limited number of embodiments, it will be apparent to those skilled in the art that many alternate, modifications, and variations will be suggested by the foregoing description and that the embodiments of the present disclosure are thereby not to be restricted.

[0174] It is therefore intended that the present disclosure cover all such alternatives, modifications and variations as can fall within the broad scope of the appended claims. Accordingly, all expedients or combinations of expedients that are within the scope of the appended claims are intended to be embraced therein.

Claims

1. A fine-grained hybrid automatic retransmission method based on semantic bases, applied at the sending end, characterized in that, include: Extract semantic features from raw information; Based on semantic features, multiple semantic bases that meet the preset task requirements are obtained by matching from the preset semantic base knowledge base, and a semantic representation composed of the indexes of multiple semantic bases. The semantic representation is channel-coded according to a preset coding method to obtain the encoded semantic representation; The encoded semantic representation is transmitted to the receiving end via the channel; Verification information is generated based on multiple semantic bases; The verification information is sent to the receiving end so that the receiving end can verify the received information based on the verification information; When the receiving end verifies that there is a semantic base that needs to be retransmitted, it retransmits the semantic base that needs to be retransmitted to the receiving end based on the verification result, so that the receiving end can perform the next round of error detection based on the received information and the retransmitted semantic base.

2. The method according to claim 1, characterized in that, The generation of verification information based on multiple semantic bases includes: Multiple semantic bases are input into a preset context estimation module, which outputs the probability estimation results of each semantic base; wherein, the probability estimation results include the mean. Calculate the mean of each semantic base and the corresponding mean correction value between semantic bases; The mean correction value of each semantic base is input into the preset verification information encoding module, and the verification information encoding module outputs the verification information.

3. The method according to claim 1, characterized in that, The semantic base knowledge base is constructed based on explicit semantic bases, and the methods for generating the explicit semantic bases include: Extract semantic features from existing information; According to the preset clustering rules, explicit semantic bases are extracted from the semantic features.

4. A fine-grained hybrid automatic retransmission method based on semantic bases, applied at the receiving end, characterized in that, include: Receive semantic information and verification information; The semantic information is channel decoded to obtain the decoded semantic representation; Based on the decoded semantic representation, multiple semantic bases are recovered by matching from a preset semantic base knowledge base; The verification information is used to verify the recovered semantic bases to obtain the verified semantic bases, including: if the verification result shows that there are semantic bases that need to be retransmitted, the verification result is sent to the sending end so that the sending end can retransmit the semantic bases that need to be retransmitted according to the verification result. Receive retransmitted semantic information; wherein the retransmitted semantic information is obtained by the sending end retransmitting the semantic base that needs to be retransmitted based on the verification result and transmitting it through the channel; Based on the maximum ratio merging of the retransmitted semantic information and the aforementioned semantic information, the next round of error detection is performed; The original information is reconstructed based on the verified semantic base.

5. The method according to claim 4, characterized in that, The step of verifying the recovered semantic bases using verification information to obtain verified semantic bases includes: The recovered semantic bases are input into a preset context estimation module, which outputs the probability estimation results of each recovered semantic base; wherein, the probability estimation results include the mean and variance. The verification information is input into a preset verification information decoding module, and the verification information decoding module outputs the mean correction value of each semantic base; The final mean of each recovered semantic basis is determined based on the mean of each recovered semantic basis and the corresponding mean correction value of the semantic basis. Hypothesis testing is performed based on the final mean and variance of each recovered semantic basis to obtain the verification results of each semantic basis.

6. The method according to claim 4, characterized in that, Based on the retransmitted semantic information and the maximum ratio merging of the semantic information, including: The semantic representation of the retransmission is obtained by decoding the semantic information of the retransmission. The index corresponding to the semantic base in the retransmitted semantic representation is merged with the index of the received semantic base by maximizing the ratio.

7. The method according to claim 5, characterized in that, The hypothesis testing is performed based on the final mean and variance of each semantic basis recovered, resulting in the verification results of each semantic basis, including: Based on the final mean and variance of each semantic basis, hypothesis testing is performed using the significance threshold adaptively generated by the binary error detection module to obtain the verification results.

8. A fine-grained hybrid automatic repeater based on semantic bases, applied at the transmitting end, characterized in that, include: The extraction module is used to extract semantic features from the raw information; The matching module is used to match multiple semantic bases that meet the preset task requirements and the semantic representation composed of the indexes of multiple semantic bases from the preset semantic base knowledge base based on semantic features. The encoding module is used to perform channel coding on the semantic representation according to a preset encoding method to obtain the encoded semantic representation; The generation module is used to generate verification information based on multiple semantic bases; The sending module is used to send the encoded semantic representation to the receiving end via the channel; and to send the verification information to the receiving end so that the receiving end can verify the received information based on the verification information; The retransmission module is used to retransmit the semantic base that needs to be retransmitted to the receiving end when the verification result of the received information by the receiving end is that there is a semantic base that needs to be retransmitted, so that the receiving end can perform the next round of error detection based on the received information and the retransmitted semantic base.

9. A fine-grained hybrid automatic repeater based on semantic bases, applied at a receiving end, characterized in that, include: The receiving module is used to receive semantic information and verification information. And receive retransmitted semantic information; wherein, the retransmitted semantic information is obtained by the sending end retransmitting the semantic base that needs to be retransmitted based on the verification result and transmitting it through the channel; A decoding module is used to decode the semantic information to obtain the decoded semantic representation; The matching module is used to obtain multiple restored semantic bases by matching from a preset semantic base knowledge base based on the decoded semantic representation; The verification module is used to verify the recovered multiple semantic bases using the verification information to obtain the verified semantic bases. The verification module includes: if the verification result shows that there are semantic bases that need to be retransmitted, sending the verification result to the sending end so that the sending end can retransmit the semantic bases that need to be retransmitted according to the verification result; and performing maximum ratio merging based on the retransmitted semantic information and the original semantic information to perform the next round of error detection. The reconstruction module is used to reconstruct the original information based on the verified semantic base.

Images