Intelligent communication retransmission method, device, electronic equipment and storage medium

Abstract

This invention provides an intelligent communication retransmission method, apparatus, electronic device, and storage medium, applied at a transmitting end, comprising: acquiring multi-stream semantic feature data corresponding to first modal data and second modal data; acquiring data frames through the multi-stream semantic feature data, and acquiring semantic channel reference signals through common semantic features in the multi-stream semantic feature data; transmitting the semantic channel reference signals and data frames to a receiving end, so that the receiving end inputs the data frames and semantic channel reference signals to a semantic verification decoder for decoding, and transmits an indication signal output by the semantic verification decoder to the transmitting end for indicating the channel state, the indication signal including an ACK signal and a NACK signal; and performing a retransmission operation upon receiving a NACK signal. This solves the problem in the prior art where the determination of whether retransmission is needed is only made after all transmitted data has been reconstructed, leading to resource waste and transmission delays.

Description

Technical Field

[0001] This invention relates to the field of intelligent communication technology, and in particular to an intelligent communication retransmission method, apparatus, electronic device, and storage medium. Background Technology

[0002] Semantic communication is considered a revolutionary communication technology in 6G, capable of significantly reducing the amount of data transmitted. Compared to traditional communication technologies, semantic communication processes and optimizes data at the semantic level. Its core objective is to leverage semantic information to achieve more efficient communication, minimizing errors at the semantic level rather than the bit or symbol level. Therefore, traditional bit-level-based mechanisms such as Cyclic Redundancy Check (CRC) are no longer suitable for semantic communication verification. Simultaneously, 6G is transitioning from typical scenarios to more complex, diverse, and higher-quality-of-service scenarios (such as immersive cloud XR / holographic communication / sensory interconnection / intelligent interaction), exhibiting multimodal characteristics. Existing technologies use semantic similarity detection networks to replace CRC verification.

[0003] However, semantic similarity detection networks are essentially based on semantic hybrid automatic repeat request (HARQ) transmission using a single modality (such as plain text or plain image). Figure 1 This is a schematic diagram of a retransmission mechanism for single-modal semantic communication in existing technologies. This transmission method does not consider the semantic correlation between multiple modalities and cannot be applied to multimodal scenarios. Furthermore, regardless of whether it is a large-data-volume modality or a small-data-volume modality, in existing technologies, the receiving end can only determine semantic similarity and whether retransmission is necessary after all transmitted data has been reconstructed, resulting in wasted computing resources and increased transmission latency. Summary of the Invention

[0004] This invention provides an intelligent communication retransmission method, apparatus, electronic device, and storage medium to solve the problem in the prior art where the receiving end can only determine semantic similarity and whether retransmission is needed after all transmitted data has been reconstructed, resulting in wasted computing resources and increased transmission latency.

[0005] This invention provides an intelligent communication retransmission method, comprising: acquiring multi-stream semantic feature data corresponding to first modal data and second modal data, wherein the multi-stream semantic feature data includes first modal semantic feature data, common semantic feature data, and second modal semantic feature data; acquiring a data frame through the multi-stream semantic feature data, and acquiring a semantic channel reference signal through the common semantic features in the multi-stream semantic feature data, wherein the semantic channel reference signal includes a check code and encoded common semantic features; transmitting the semantic channel reference signal and the data frame to a receiving end, such that the receiving end inputs the data frame and the semantic channel reference signal to a semantic check decoder for decoding, and transmits an indication signal output by the semantic check decoder to the transmitting end for indicating the channel state, wherein the indication signal includes an ACK signal and a NACK signal; and performing a retransmission operation upon receiving the NACK signal.

[0006] According to an intelligent communication retransmission method provided by the present invention, the step of obtaining multi-stream semantic feature data corresponding to first modal data and second modal data includes: inputting the first modal data and the second modal data into a common semantic feature extraction network to obtain the multi-stream semantic feature data output by the common semantic feature extraction network.

[0007] According to an intelligent communication retransmission method provided by the present invention, obtaining a data frame through the multi-stream semantic feature data includes: inputting the multi-stream semantic feature data into an autoencoder to obtain the data frame.

[0008] According to an intelligent communication retransmission method provided by the present invention, the step of obtaining a semantic channel reference signal through common semantic features in the multi-stream semantic feature data includes: inputting the common semantic feature data in the multi-stream semantic feature data into a semantic verification encoder to obtain the semantic channel reference signal.

[0009] According to the present invention, an intelligent communication retransmission method is provided, the method further comprising: not performing a retransmission operation when the ACK signal is received.

[0010] According to the intelligent communication retransmission method provided by the present invention, the data type of the first modal data is any one of audio data, video data, text data and point cloud data, and the data type of the second modal data is also any one of audio data, video data, text data and point cloud data, and the data types of the first modal data and the second modal data are different.

[0011] This invention also provides an intelligent communication retransmission device, comprising the following modules: a first acquisition module, configured to acquire multi-stream semantic feature data corresponding to first modal data and second modal data, wherein the multi-stream semantic feature data includes first modal semantic feature data, common semantic feature data, and second modal semantic feature data. A second acquisition module, configured to acquire a data frame through the multi-stream semantic feature data, and acquire a semantic channel reference signal through the common semantic features in the multi-stream semantic feature data, wherein the semantic channel reference signal includes a check code and encoded common semantic features. A first processing module, configured to send the semantic channel reference signal and the data frame to a receiving end, such that the receiving end inputs the data frame and the semantic channel reference signal to a semantic check decoder for decoding, and sends an indication signal output by the semantic check decoder to the sending end, wherein the indication signal includes an ACK signal and a NACK signal. A second processing module, configured to perform a retransmission operation upon receiving the NACK signal.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the intelligent communication retransmission method as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent communication retransmission method as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the intelligent communication retransmission method as described above.

[0015] The intelligent communication retransmission method, apparatus, electronic device, and storage medium provided by this invention acquire multi-stream semantic feature data corresponding to first and second modal data. After encoding the multi-stream semantic feature data, a data frame and semantic channel reference signal are generated and sent to the receiving end. These are then input to a semantic verification decoder for decoding, and an indication signal is sent. Upon receiving the NACK signal, a retransmission operation is performed. This solves the problem in existing technologies where retransmission is only determined after all transmitted data has been reconstructed, leading to resource waste and transmission delays. It enables retransmission based on signal indication before the reconstruction of large-scale modal data is complete, reducing data processing steps and latency at the receiving end. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the retransmission mechanism for single-modal semantic communication in existing technologies.

[0018] Figure 2 This is a flowchart illustrating the intelligent communication retransmission method provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the system framework of the intelligent communication retransmission method provided by the present invention.

[0020] Figure 4 This is a schematic diagram of the combination mode of semantic channel reference signal and data frame provided by the present invention.

[0021] Figure 5 This is a schematic diagram of the intelligent communication retransmission device provided by the present invention.

[0022] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] In this invention, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0024] In this invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0025] In this invention, the terms "first," "second," etc., are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, the first object can be one or more.

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] Figure 2 This is a flowchart illustrating the intelligent communication retransmission method provided by the present invention, as shown below. Figure 2 As shown, the method includes the following steps S202-S208:

[0028] Step S202: Obtain multi-stream semantic feature data corresponding to the first modality data and the second modality data. The multi-stream semantic feature data includes the first modality semantic feature data, common semantic feature data, and second modality semantic feature data.

[0029] As an optional embodiment, the modal data used in the intelligent communication retransmission method described in this application can be multiple in practical applications, and the specific implementation is consistent with the method described in this application. This application embodiment does not further limit this.

[0030] In an exemplary embodiment, step S202 can be implemented by the following step S11:

[0031] Step S11: Input the first modality data and the second modality data into the common semantic feature extraction network to obtain the multi-stream semantic feature data output by the common semantic feature extraction network.

[0032] As an optional embodiment, the common semantic feature extraction network described in this invention is essentially a deep learning network that extracts semantic features through convolutional layers.

[0033] As an optional embodiment, for the first modality data, the semantic features obtained by the common semantic feature extraction network are (first modality semantic feature data, shared features, second modality semantic feature data). Each modality data has three features, but for the first modality data, the second modality semantic feature data is 0. That is, for the first modality data, the semantic features obtained by the common semantic feature extraction network are (first modality semantic feature data, shared features, 0); similarly, for the second modality data, the semantic features obtained by the common semantic feature extraction network are (0, shared features, second modality semantic feature data).

[0034] As an optional embodiment, the common semantic feature extraction network fuses the shared features of different modal data as common semantic feature data. Therefore, the final output of the common semantic feature extraction network is (first modal semantic feature data, common semantic feature data, second modal semantic feature data).

[0035] It should be noted that the first modality semantic feature data contains important semantic information from the first modality data, and similarly, the second modality semantic feature data contains important semantic information from the second modality data. Common semantic feature data is the common information extracted from both modalities, representing the overall semantic features of the data.

[0036] In an exemplary embodiment, the data type of the first modal data is any one of audio data, video data, text data, and point cloud data, and the data type of the second modal data is also any one of audio data, video data, text data, and point cloud data, and the data types of the first modal data and the second modal data are different.

[0037] Step S204: Obtain a data frame through the multi-stream semantic feature data, and obtain a semantic channel reference signal through the common semantic features in the multi-stream semantic feature data. The semantic channel reference signal includes a check code and encoded common semantic features.

[0038] In an exemplary embodiment, in step S204 above, obtaining the data frame through the multi-stream semantic feature data can be achieved through the following step S21:

[0039] Step S21: Input the multi-stream semantic feature data into the autoencoder to obtain the data frame.

[0040] As an optional embodiment, step S21 is implemented based on a semantic coding network. The autoencoder jointly designs source coding and channel coding into a joint encoder, mapping the extracted multi-stream semantic feature data into a low-dimensional space, thereby achieving effective feature compression and reconstruction. Representing the input feature data with less information reduces the dimensionality and complexity of the data, improving data processing and storage efficiency.

[0041] In an exemplary embodiment, in step S204 above, obtaining the semantic channel reference signal through the common semantic features in the multi-stream semantic feature data can be achieved through the following step S31:

[0042] Step S31: Input the common semantic feature data in the multi-stream semantic feature data into the semantic verification encoder to obtain the semantic channel reference signal.

[0043] As an optional embodiment, step S31 above can be implemented based on a semantic check coding network. The input of the network is common semantic feature data, and the output is encoded common semantic feature data and additional bits as a semantic check code. The semantic check code is used to check the semantic consistency and integrity of the data to ensure that no errors or losses occur during data transmission. Similar to traditional CRC check, the number of check bits is determined by the length of the data frame and can be adjusted and optimized according to actual needs to achieve better check performance. This application embodiment does not further limit this.

[0044] Step S206: The semantic channel reference signal and the data frame are sent to the receiving end, so that the receiving end inputs the data frame and the semantic channel reference signal to the semantic check decoder for decoding, and sends the indication signal output by the semantic check decoder for indicating the channel state to the sending end. The indication signal includes an ACK signal and a NACK signal.

[0045] As an optional embodiment, the combination mode of semantic channel reference signals and data frames can be generated according to different actual application scenarios. For example, in scenarios where channel conditions change rapidly, a mode with denser insertion of semantic channel reference signals can be selected to better evaluate real-time channel state conditions. The specific combination method is not further limited in this application embodiment.

[0046] It should be noted that the ACK signal indicates that the channel condition is good and no semantic error occurred during transmission, meaning that retransmission is not required; conversely, the NACK signal indicates that the channel condition is poor and a semantic error occurred during transmission, instructing the sender to retransmit the corresponding modal data to prevent errors from occurring during subsequent data reconstruction.

[0047] Step S208: Upon receiving the NACK signal, perform a retransmission operation.

[0048] It should be noted that the NACK signal is a single signal packet, and the underlying data contains the data block corresponding to the data that the sender needs to retransmit. In other words, during the retransmission process, only the part with semantic errors needs to be retransmitted, instead of retransmitting all modal data, thus improving transmission efficiency.

[0049] In steps S202-S208 above, multi-stream semantic feature data corresponding to the first and second modal data are acquired. The data frames generated after encoding the multi-stream semantic feature data and the semantic channel reference signal are sent to the receiving end. These are then input to a semantic verification decoder for decoding, and an indication signal is sent. Upon receiving the NACK signal, a retransmission operation is performed. This solves the problem in existing technologies where retransmission is only determined after all transmitted data has been reconstructed, leading to resource waste and transmission delays. It enables retransmission based on signal indication before the reconstruction of large-scale modal data is complete, reducing data processing steps and latency at the receiving end.

[0050] The execution entity of each step in this method can be an intelligent communication retransmission device, which can be implemented by software and / or hardware. The device can be integrated into an electronic device, which can be a terminal device (such as a smartphone, personal computer, etc.), a server (such as a local server or cloud server, or a server cluster, etc.), a processor, or a chip, etc.

[0051] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. To better understand the above method, the following description, in conjunction with embodiments, illustrates the process, but does not limit the technical solutions of the embodiments of the present invention. Specifically:

[0052] This application proposes an intelligent communication retransmission method that utilizes the semantic correlation between modalities to assist in the early retransmission of large-scale modalities throughout the process. This enables the determination of whether to retransmit before the reconstruction of large-scale modalities is completed, effectively reducing the data processing steps and computational latency at the receiving end. At the same time, the semantic correlation between modalities will assist in the reconstruction of large-scale modalities to enhance the accuracy of data transmission.

[0053] To better understand, we will use two modal data as examples to explain the technical solutions in detail. Figure 3 This is a schematic diagram of the system framework of the intelligent communication retransmission method provided by the present invention. For example... Figure 3 As shown, the system framework includes: a common feature extraction network, a semantic encoding network, a semantic verification encoding network, a semantic verification decoding network, and a semantic channel reference signal. The following sections will further elaborate on the different components within the framework.

[0054] 1. Common Feature Extraction Network: Used to extract common information from input data of different modalities. The common feature extraction network in this invention is essentially a deep learning network that extracts features from the input data through convolutional layers.

[0055] For modality one (corresponding to the first modality data in the above embodiments), the extracted semantic features are [modality one semantic features (corresponding to the first modality semantic feature data in the above embodiments); shared features; modality two semantic features (corresponding to the second modality semantic feature data in the above embodiments)]. Each sample has three features, but for the sample data of modality one, its modality two semantic features are 0. Similarly, for the sample of modality two, the modality one semantic features are also 0.

[0056] Secondly, the shared features of different modalities are fused together as common semantic features (corresponding to the common semantic feature data in the above embodiments).

[0057] Finally, the output of the common feature extraction network is [Modal 1 semantic features (corresponding to the first modal semantic feature data in the above embodiment); common semantic features (i.e., the common semantic feature data in the above embodiment); Modal 2 semantic features (i.e., the second modal semantic feature data in the above embodiment)].

[0058] It should be noted that the modality-one semantic features contain the essential semantic information of modality one. The common semantic features are the shared information extracted from the two different modalities, representing the overall semantic features of the input data. The modality-two semantic features are similar to the modality-one semantic features, containing the essential semantic information of modality two.

[0059] 2. Semantic Coding Network: In this invention, source coding and channel coding are jointly designed as a joint encoder based on an autoencoder. The extracted multi-stream semantic feature data is mapped to a low-dimensional space, thereby achieving effective feature compression and reconstruction. Representing the input feature data with less information reduces the dimensionality and complexity of the data, improving data processing and storage efficiency.

[0060] 3. Semantic Check Encoding Network: Common semantic feature data is input to the semantic check encoding network, generating additional bits as a semantic check code, which is sent to the receiver to verify the semantic consistency and integrity of the data, ensuring no errors or data loss occur during transmission. Similar to traditional CRC checksums, the number of check bits is determined by the length of the data frame and can be adjusted and optimized according to actual needs to achieve better checksum performance. The number of check bits can be set as shown in Table 1.

[0061] Table 1: Setting the Number of Check Bits

[0062]

[0063] 4. Semantic verification decoding network: such as Figure 3As shown, the decoded common semantic feature data and semantic check code are input into the semantic check decoding network, and the output is either an ACK signal or a NACK signal (1 / 0). The ACK signal indicates that the channel condition is good and no semantic error occurred during transmission, indicating that retransmission is not required; conversely, the NACK signal indicates that the channel condition is poor and a semantic error occurred during transmission, indicating that the corresponding modal data needs to be retransmitted to prevent data reconstruction errors.

[0064] Semantic channel reference signal: Figure 3 The joint transmission of black and gray data blocks can serve as a semantic channel reference signal for evaluating channel state conditions. Considering the time-varying nature of the channel, three combination modes of the reference signal and data frames are designed as an example, such as... Figure 4 The diagram illustrates the combination modes of semantic channel reference signals and data frames provided by this invention. It should be noted that in practical applications, more combination modes can be generated based on different application scenarios, and are not limited to these three modes. For example, in scenarios where channel conditions change rapidly, a mode with denser insertion of semantic channel reference signals can be selected to better assess real-time channel state conditions.

[0065] The reconstruction and decoding networks perform the opposite operations to the feature extraction and encoding networks mentioned in this invention, and the principles will not be further explained here.

[0066] Based on the descriptions of the functions of each module above, the following analysis will focus on specific steps:

[0067] Step 1: Data Acquisition: Data in multiple modalities is acquired through sensors at the transmitting end. These modalities include, but are not limited to, audio, video, text, and point clouds.

[0068] Step 2: Feature Extraction: The common semantic feature extraction network is used to process multimodal data. When the input consists of data from two different modalities, the network will output semantic feature data from three streams (i.e., multi-stream semantic feature data), namely, modality-1 semantic feature data (corresponding to the first modality semantic feature data in the above embodiment), common semantic feature data, and modality-2 semantic feature data (corresponding to the second modality semantic feature data in the above embodiment).

[0069] Step 3: Encoding: The output multi-stream semantic feature data enters the encoding section, which transforms the feature data into a more compact and efficient representation. Specifically, the multi-stream semantic feature data is first input into an autoencoder for feature re-representation and compression. Furthermore, common semantic feature data needs to pass through a semantic verification encoder to generate additional semantic check codes. The specific number of bits is determined by the length of the data frame, and the design scheme is shown in Table 1 above.

[0070] Step 4: Transmission: Depending on the specific application scenario, different combinations of semantic channel reference signals and data frames are adopted to achieve efficient transmission in the wireless channel.

[0071] Step 5: Decoding: At the receiving end, the received data frames and semantic channel reference signals are semantically decoded.

[0072] The received semantic channel reference signal is decomposed into two parts:

[0073] (1) Common semantic features (i.e.) Figure 3 (The gray data blocks in the middle), this part of the data blocks is based on Figure 3 As shown, the decoded data contains common semantic features.

[0074] (2) Semantic check code (i.e.) Figure 3 (Black data block in the middle): This part of the data block is the semantic check bit information received by the receiving end.

[0075] These two parts of data are input into the semantic verification decoder to verify the semantic consistency and integrity of the data, thereby determining the channel state conditions.

[0076] Step 6: Based on the output of the semantic verification decoder (corresponding to the indication signal in the above embodiment), determine whether it is necessary to retransmit a certain mode data block adjacent to the semantic channel reference signal.

[0077] Specifically, the sender first determines whether to retransmit:

[0078] A. If an ACK signal is received, it means that the current channel conditions are good, and there is no need for early retransmission. The subsequent data reconstruction operation can be carried out directly.

[0079] B. If a NACK signal is received, it indicates that the current channel conditions are very poor, and an early retransmission operation is required. Further, after receiving the NACK signal, the transmitting end returns to step three to perform a data retransmission operation, retransmitting the corresponding mode data block.

[0080] Step Seven: Data Reconstruction: When reconstructing multimodal data, common semantic features can be used to assist in the reconstruction of data for each modality. For the first modality data, the assistance of common semantic features can help in the reconstruction of the semantic feature data of the first modality. That is, the first modality data can be reconstructed by analyzing and utilizing common semantic features, thereby improving the accuracy and completeness of the data. The same applies to the second modality data, and will not be elaborated here.

[0081] To better understand, the following example illustrates the multimodal transmission of audio and video in a vehicle-to-everything (V2X) scenario:

[0082] a: Vehicle-mounted sensors collect audio and video image data from the surrounding environment.

[0083] b: The audio stream and video stream are input into the common feature extraction network, which outputs three streams of semantic feature data, namely audio semantic feature data, common semantic feature data, and video image semantic feature data.

[0084] c: The output three-stream semantic feature data enters the encoding section. Audio semantic features, common semantic features, and video image semantic feature data are input into the autoencoder. Simultaneously, the common semantic features also need to pass through a semantic check encoder to generate an additional 32-bit semantic check code. The encoded common semantic features and the 32-bit semantic check code are combined as the semantic channel reference signal.

[0085] d: Based on the time-varying characteristics of channel state information in the vehicle-to-everything (V2X) scenario, a more dense semantic channel reference signal insertion mode is selected to combine it with audio and video semantic feature data and transmit it through the wireless channel.

[0086] e: The receiving end performs semantic decoding on the received semantic feature data. The semantic channel reference signal is decomposed into two parts: the decoded common semantic features; and the 32-bit semantic checksum received by the receiving end, which is input into the semantic checksum decoder.

[0087] f: Based on the output of the semantic verification decoder, determine whether to retransmit the data block of the adjacent mode of the semantic channel reference signal early. If it is ACK, retransmit the data and return to step three; otherwise, reconstruct the data.

[0088] g: When reconstructing audio and video data, audio data can be reconstructed with the assistance of common semantic features; video image data can also be reconstructed with the assistance of common semantic features.

[0089] Compared to existing technologies, this application considers a wider range of multimodal application scenarios and meets the service requirements of future 6G communication. By utilizing the semantic correlation between modalities to assist in the early retransmission of large-modal data, the processing steps at the receiving end are significantly reduced, thereby achieving the technical effects of saving computing resources and reducing communication latency.

[0090] The intelligent communication retransmission device provided by the present invention is described below. The intelligent communication retransmission device described below and the intelligent communication retransmission method described above can be referred to in correspondence.

[0091] Figure 5 This is a schematic diagram of the intelligent communication retransmission device provided by the present invention. As shown in the figure, the device includes:

[0092] The first acquisition module 502 is used to acquire multi-stream semantic feature data corresponding to the first modality data and the second modality data. The multi-stream semantic feature data includes the first modality semantic feature data, common semantic feature data, and the second modality semantic feature data.

[0093] The second acquisition module 504 is used to acquire data frames through the multi-stream semantic feature data and to acquire semantic channel reference signals through common semantic features in the multi-stream semantic feature data. The semantic channel reference signals include check codes and encoded common semantic features.

[0094] The first processing module 506 is configured to send the semantic channel reference signal and the data frame to the receiving end, so that the receiving end inputs the data frame and the semantic channel reference signal to the semantic check decoder for decoding, and sends the indication signal output by the semantic check decoder for indicating the channel state to the sending end, the indication signal including ACK signal and NACK signal.

[0095] The second processing module 508 is used to perform a retransmission operation upon receiving the NACK signal.

[0096] The aforementioned intelligent communication retransmission device acquires multi-stream semantic feature data corresponding to the first and second modal data. The encoded data frames and semantic channel reference signals generated from these multi-stream semantic feature data are sent to the receiving end. These are then input to a semantic verification decoder for decoding, and an indication signal is sent. Upon receiving the NACK signal, a retransmission operation is performed. This solves the problem in existing technologies where retransmission is only determined after all transmitted data has been reconstructed, leading to resource waste and transmission delays. It enables retransmission based on signal indication before the reconstruction of large-scale modal data is complete, reducing data processing steps and latency at the receiving end.

[0097] Optionally, the first acquisition module 502 is further configured to input the first modal data and the second modal data into a common semantic feature extraction network to obtain the multi-stream semantic feature data output by the common semantic feature extraction network.

[0098] Optionally, the second acquisition module 504 is further configured to input the multi-stream semantic feature data into the autoencoder to obtain the data frame.

[0099] Optionally, the second acquisition module 504 is further configured to input the common semantic feature data in the multi-stream semantic feature data into the semantic verification encoder to obtain the semantic channel reference signal.

[0100] Optionally, the device further includes a third processing module, configured not to perform a retransmission operation upon receiving the ACK signal.

[0101] Optionally, the data type of the first modal data can be any one of audio data, video data, text data, and point cloud data, and the data type of the second modal data can also be any one of audio data, video data, text data, and point cloud data, and the data types of the first modal data and the second modal data are different.

[0102] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute an intelligent communication retransmission method, which includes:

[0103] Obtain multi-stream semantic feature data corresponding to the first modality data and the second modality data. The multi-stream semantic feature data includes the first modality semantic feature data, common semantic feature data, and the second modality semantic feature data.

[0104] Data frames are obtained through the multi-stream semantic feature data, and semantic channel reference signals are obtained through the common semantic features in the multi-stream semantic feature data. The semantic channel reference signals include check codes and encoded common semantic features.

[0105] The semantic channel reference signal and the data frame are sent to the receiving end, so that the receiving end inputs the data frame and the semantic channel reference signal to the semantic check decoder for decoding, and sends the indication signal output by the semantic check decoder to the sending end for indicating the channel state. The indication signal includes an ACK signal and a NACK signal.

[0106] Upon receiving the NACK signal, a retransmission operation is performed.

[0107] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the intelligent communication retransmission method provided by the above methods, the method comprising:

[0109] Obtain multi-stream semantic feature data corresponding to the first modality data and the second modality data. The multi-stream semantic feature data includes the first modality semantic feature data, common semantic feature data, and the second modality semantic feature data.

[0110] Data frames are obtained through the multi-stream semantic feature data, and semantic channel reference signals are obtained through the common semantic features in the multi-stream semantic feature data. The semantic channel reference signals include check codes and encoded common semantic features.

[0111] The semantic channel reference signal and the data frame are sent to the receiving end, so that the receiving end inputs the data frame and the semantic channel reference signal to the semantic check decoder for decoding, and sends the indication signal output by the semantic check decoder to the sending end for indicating the channel state. The indication signal includes an ACK signal and a NACK signal.

[0112] Upon receiving the NACK signal, a retransmission operation is performed.

[0113] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the intelligent communication retransmission method provided by the methods described above, the method comprising:

[0114] Obtain multi-stream semantic feature data corresponding to the first modality data and the second modality data. The multi-stream semantic feature data includes the first modality semantic feature data, common semantic feature data, and the second modality semantic feature data.

[0115] Data frames are obtained through the multi-stream semantic feature data, and semantic channel reference signals are obtained through the common semantic features in the multi-stream semantic feature data. The semantic channel reference signals include check codes and encoded common semantic features.

[0116] The semantic channel reference signal and the data frame are sent to the receiving end, so that the receiving end inputs the data frame and the semantic channel reference signal to the semantic check decoder for decoding, and sends the indication signal output by the semantic check decoder to the sending end for indicating the channel state. The indication signal includes an ACK signal and a NACK signal.

[0117] Upon receiving the NACK signal, a retransmission operation is performed.

[0118] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0119] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent communication retransmission, characterized in that, Applied to the sending end, including: Obtain multi-stream semantic feature data corresponding to the first modality data and the second modality data, wherein the multi-stream semantic feature data includes the first modality semantic feature data, common semantic feature data, and the second modality semantic feature data; Data frames are obtained through the multi-stream semantic feature data, and semantic channel reference signals are obtained through the common semantic features in the multi-stream semantic feature data. The semantic channel reference signals include check codes and encoded common semantic features. The semantic channel reference signal and the data frame are sent to the receiving end, so that the receiving end inputs the data frame and the semantic channel reference signal to the semantic check decoder for decoding, and sends the indication signal output by the semantic check decoder to the sending end for indicating the channel state. The indication signal includes an ACK signal and a NACK signal. Upon receiving the NACK signal, a retransmission operation is performed.

2. The intelligent communication retransmission method according to claim 1, characterized in that, The acquisition of multi-stream semantic feature data corresponding to the first modality data and the second modality data includes: The first modality data and the second modality data are input into the common semantic feature extraction network to obtain the multi-stream semantic feature data output by the common semantic feature extraction network.

3. The intelligent communication retransmission method according to claim 1, characterized in that, The process of obtaining a data frame from the multi-stream semantic feature data includes: The multi-stream semantic feature data is input into an autoencoder to obtain the data frame.

4. The intelligent communication retransmission method according to claim 1, characterized in that, The step of obtaining the semantic channel reference signal through the common semantic features in the multi-stream semantic feature data includes: The common semantic feature data in the multi-stream semantic feature data is input into the semantic verification encoder to obtain the semantic channel reference signal.

5. The intelligent communication retransmission method according to any one of claims 1 to 4, characterized in that, The method further includes: If the ACK signal is received, no retransmission operation will be performed.

6. The intelligent communication retransmission method according to any one of claims 1 to 4, characterized in that, The data type of the first modal data is any one of audio data, video data, text data, and point cloud data, and the data type of the second modal data is also any one of audio data, video data, text data, and point cloud data, and the data types of the first modal data and the second modal data are different.

7. An intelligent communication retransmission device, applied at the transmitting end, characterized in that, include: The first acquisition module is used to acquire multi-stream semantic feature data corresponding to the first modality data and the second modality data. The multi-stream semantic feature data includes the first modality semantic feature data, common semantic feature data, and the second modality semantic feature data. The second acquisition module is used to acquire data frames through the multi-stream semantic feature data and to acquire semantic channel reference signals through common semantic features in the multi-stream semantic feature data. The semantic channel reference signals include check codes and encoded common semantic features. The first processing module is configured to send the semantic channel reference signal and the data frame to the receiving end, so that the receiving end inputs the data frame and the semantic channel reference signal to the semantic check decoder for decoding, and sends the indication signal output by the semantic check decoder for indicating the channel state to the sending end, the indication signal including ACK signal and NACK signal; The second processing module is used to perform a retransmission operation upon receiving the NACK signal.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the intelligent communication retransmission method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the intelligent communication retransmission method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent communication retransmission method as described in any one of claims 1 to 6.

Images