A feedback-aided multi-modal rateless unequal error protection system

Abstract

The application discloses a feedback-assisted multi-modal rateless unequal error protection system, comprising a source node module, a relay node module and a destination node module, wherein the source node module is used for realizing the functions of collecting, extracting, encoding and packetizing modal information; the relay node module is used for realizing the functions of identifying and unequal error protection encoding of the modal information; the destination node module is used for decoding according to the decoding rule of the unequal error protection rateless encoding after receiving the encoded packet, recovering the data packet, and recovering different modal information according to the unpacking rule; and the destination node also continuously monitors the decoding state, and feeds back the decoding state at this time to the relay node when the important level reaches the target bit error rate.

Description

Technical Field

[0001] This invention relates to the field of multimodal communication and encoding / decoding in communication, and particularly to a feedback-assisted multimodal rateless coding inequality error protection system. Background Technology

[0002] Multimodal communication is a technology that combines various modal data formats (such as images, text, and sensor data) with advanced communication technologies, aiming to transmit and process information efficiently and intelligently. This technology has enormous application potential, especially in 5G and future 6G networks. However, in multimodal communication, the importance of different modalities varies due to non-uniform information distribution, differences in fault tolerance, differences in information redundancy, variations in network environment and bandwidth limitations, and synergistic effects between modalities, leading to differences in the importance of different modal information. Even when the same modal information error occurs, the impact on the target task or information will differ depending on the modal information, giving rise to the need for unequal error protection during the transmission of different modal information. For example, in autonomous driving tasks, in adverse weather conditions, LiDAR and GPS signals may be prioritized for transmission, while camera information may be compressed or even partially discarded. In urban environments, LiDAR and cameras may contribute more, accurately perceiving surrounding vehicles and pedestrians. To ensure the reliable transmission of critical modal information, stronger error protection is often required, while the level of error protection can be appropriately reduced for auxiliary or redundant modalities. To address this situation, researchers have conducted extensive studies. Currently, the industry primarily employs resource scheduling methods, prioritizing the transmission of important modal information. However, this approach struggles to achieve targeted protection for specific modal information.

[0003] To address the aforementioned issues, this patent proposes a multimodal information transmission scheme based on application-layer unequal error protection rateless coding, taking a logical link perspective. This scheme is suitable for multimodal information transmission scenarios involving integrated terminals or multiple terminals. By using application-layer coding, it prioritizes the protection of modal information with higher reliability requirements within the multimodal information, and allows for flexible adjustment of the type of rateless coding and unequal error protection technology used for different multimodal information transmission scenarios.

[0004] Rateless coding can encode finite-length source symbols to obtain infinite-length coded symbols. During encoding, source symbols are randomly selected and XORed to obtain coded symbols. During decoding, regardless of the received coded symbols, as long as the number of received coded symbols is greater than a certain value, decoding can be completed. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a feedback-assisted multimodal rate-free unequal-code error protection system. From a logical link perspective, this system proposes an end-to-end multimodal information unequal-code transmission scheme suitable for integrated single terminals and multiple terminals. Corresponding transmission schemes are designed for different types of multimodal information, with a focus on protecting modal information with higher reliability and lower latency requirements.

[0006] To achieve the aforementioned objectives of the invention, the technical solution adopted to solve its technical problems is as follows:

[0007] A feedback-assisted multimodal rate-free coded inequality error protection system includes a source node module, a relay node module, and a destination node module, wherein:

[0008] The source node module is used to implement the functions of collecting, extracting, encoding and assembling modal information;

[0009] The relay node module is used to identify modal information and perform unequal error protection coding;

[0010] The destination node module is used to decode the data packet after receiving the encoded packet according to the decoding rules of the unequal error protection rateless encoding to recover the data packet; and to recover different modal information according to the unpacking rules; at the same time, the destination node will continuously monitor the decoding status, and when a certain importance level reaches the target bit error rate, the destination node will feed back the current decoding status to the relay node.

[0011] Furthermore, the source node module includes a modal information acquisition submodule, a modal information extraction submodule, a modal information encoding submodule, an importance level classification submodule, and a packet grouping submodule, wherein:

[0012] The modal information acquisition submodule is used to acquire corresponding modal information using different modal information acquisition devices, and to transmit the acquired modal information to the modal information extraction submodule;

[0013] The modal information extraction submodule is used to extract useful modal information from the received modal information and pass it to the modal information encoding submodule;

[0014] The modal information encoding submodule is used to perform modal information encoding on the received information, that is, to perform the corresponding source encoding, and then transmit the encoded information to the packet grouping submodule;

[0015] The importance level classification submodule is used to classify different modal information into g importance levels, i.e. g node groups, according to the reliability and recovery delay requirements of different modalities.

[0016] The packet assembly submodule is used to combine the received information into an intermediate packet according to the packet assembly rules, and then pass the intermediate packet to the relay node module.

[0017] Furthermore, the modal information includes audio, video, and pressure information.

[0018] Furthermore, the relay node module includes a modal information recognition submodule, a feedback information processing submodule, and an unequal error protection coding submodule, wherein:

[0019] The modal information identification submodule is used to identify, after the relay node receives the intermediate packet, the modal information type represented by the different intermediate packets, the data length of the different modal information, the number of data packets of the different modal information, and the storage location of the different modal information in the relay node, and then transmit it to the unequal error protection coding submodule.

[0020] The feedback information processing submodule is used to determine whether feedback information sent by the destination node has been received. If no feedback information is received, selection weights of different importance levels are set. ( The system continues to execute the functions of the unequal error protection coding submodule; if feedback information is received, it executes the adaptive weight rotation allocation and adaptive code length adjustment algorithm. After completing the above operations according to the feedback information, it enters the unequal error protection submodule.

[0021] The unequal error protection coding submodule is used to receive information from the previous module, perform unequal error protection coding according to the set selection weight and importance level, generate a coding packet and pass it to the destination node module.

[0022] Furthermore, after receiving feedback information, the relay node will temporarily "suspend" the node group that has reached the target bit error rate, that is, it will no longer select the data packets of the node group for unequal error protection coding, and will distribute the weight of the node group equally to the remaining node groups. At the same time, the relay node will also temporarily "suspend" the recovered data packets in the node group that has not reached the target bit error rate, and will no longer select the recovered data packets for further coding, based on the feedback information.

[0023] Furthermore, in the destination node module, the feedback information includes the importance level of the target bit error rate and the data packets that have been recovered.

[0024] By employing the above technical solutions, this invention has the following advantages and positive effects compared with the prior art:

[0025] 1. This invention constructs a multimodal inequality error protection scheme from the perspective of logical links. It uses an encoded transmission scheme to replace the traditional scheduling transmission scheme, which can effectively protect more important modal information.

[0026] 2. This invention proposes an adaptive weighted round-robin allocation algorithm, which can transmit node groups that have not yet reached the target bit error rate more efficiently, improve resource utilization, and effectively reduce the recovery delay of non-critical modal information.

[0027] 3. This invention proposes an adaptive code length adjustment algorithm, which can effectively reduce the number of source data packets that need to be processed in the later stages of encoding and transmission, and improve resource utilization. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0029] Figure 1 This is a schematic diagram of the structure of a feedback-assisted multimodal rate-free unequal error protection system according to the present invention. Detailed Implementation

[0030] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] In this embodiment, it is assumed that three modalities of information need to be transmitted, including audio, video, and pressure information, which are divided into three importance levels: pressure information is the most important, followed by audio, and then video. The specific encoding scheme uses LT codes from rateless coding, and the unequal error protection scheme adopts a weighted unequal error protection scheme.

[0032] This embodiment discloses a feedback-assisted multimodal rateless coding inequality error protection system, the main module steps of which are described below, and the flowchart is shown in the figure. Figure 1 As shown, it specifically includes the following modules: source node module, relay node module, and destination node module, wherein:

[0033] The source node module is used to implement the functions of collecting, extracting, encoding and assembling modal information;

[0034] Specifically, the source node module includes a modal information acquisition submodule, a modal information extraction submodule, a modal information encoding submodule, an importance level classification submodule, and a packet grouping submodule, wherein:

[0035] The modal information acquisition submodule is used to acquire pressure information using a pressure sensor, audio information using a microphone, and video information using a camera, and then transmits the acquired modal information to the modal information extraction submodule.

[0036] The modal information extraction submodule is used to extract useful modal information from the received modal information and pass it to the modal information encoding submodule;

[0037] The modal information encoding submodule is used to perform modal information encoding on the received information, that is, to perform the corresponding source encoding, such as AAC encoding for audio, and then pass the encoded information to the packet grouping submodule.

[0038] The importance level classification submodule is used to classify different modal information into three importance levels, namely three node groups, according to the reliability and recovery delay requirements of different modalities.

[0039] The packet assembly submodule is used to combine the received information into intermediate packets according to the packet assembly rules. The number of data packets for the pressure information is... The number of data packets for audio information is The number of data packets for video information is Then the intermediate packet is passed to the relay node module.

[0040] The relay node module is used to identify modal information and perform unequal error protection coding;

[0041] Specifically, the relay node module includes a modal information recognition submodule, a feedback information processing submodule, and an unequal error protection coding submodule, wherein:

[0042] The modal information identification submodule is used to identify, after the relay node receives the intermediate packet, the modal information type represented by the different intermediate packets, the data length of the different modal information, the number of data packets of the different modal information, and the storage location of the different modal information in the relay node, and then transmit it to the unequal error protection coding submodule.

[0043] The feedback information processing submodule is used to determine whether feedback information sent by the destination node has been received. If no feedback information is received, selection weights of different importance levels are set. , , The relay node continues to execute the functions of the unequal error protection coding submodule. If feedback information is received, it executes the adaptive weight rotation allocation and adaptive code length adjustment algorithm. Specifically, after receiving feedback information, the relay node will temporarily "suspend" the node group that has reached the target bit error rate, that is, it will no longer select data packets from that node group for unequal error protection coding, and will distribute the weight of that node group equally to the remaining node groups. , , Simultaneously, based on feedback information, the relay node will also temporarily "suspend" recovered data packets from node groups that have not reached the target bit error rate, and will not select recovered data packets for further encoding. After completing the above operations based on feedback information, it enters the unequal error protection submodule.

[0044] The unequal error protection coding submodule is used to perform unequal error protection coding after receiving information from the previous module. More important modal information has a higher probability of being selected for unequal error protection coding. Unequal error protection coding is performed according to the set selection weights and importance levels, and the generated coded packet is then transmitted to the destination node module.

[0045] The destination node module, upon receiving the encoded packet, decodes it according to the decoding rules of unequal error protection rateless coding to recover the data packet; and recovers different modal information according to the unpacking rules. Simultaneously, the destination node continuously monitors the decoding status, and when a certain importance level reaches the target bit error rate, the destination node feeds back the current decoding status to the relay node. Specifically, the feedback information in the destination node module includes the importance level that has reached the target bit error rate and the data packet that has been recovered.

[0046] Existing multimodal transmission schemes typically employ scheduling methods for transmission. These methods do not adequately consider the varying importance of different modalities and reduce the utilization of communication resources. This invention, from a logical link perspective, constructs a multimodal information transmission scheme suitable for different scenarios. The rateless coding scheme and unequal error protection methods can be flexibly adjusted according to different scenarios. Furthermore, by using application-layer coding, the dependence on physical layer devices is reduced, improving the reliability of multimodal information transmission. Simultaneously, adaptive weighted round-robin allocation and adaptive code length adjustment algorithms are proposed, enabling more efficient transmission of node groups that have not yet reached the target bit error rate, improving resource utilization, and effectively reducing the recovery latency of non-critical modalities.

[0047] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A feedback-assisted multimodal rate-free unequal coding error protection system, characterized in that, It includes a source node module, a relay node module, and a destination node module, wherein: The source node module is used to implement the functions of collecting, extracting, encoding and assembling modal information; The relay node module is used to identify modal information and perform unequal error protection coding; The relay node module includes a modal information recognition submodule, a feedback information processing submodule, and an unequal error protection coding submodule, wherein: The modal information identification submodule is used to identify, after the relay node receives the intermediate packet, the modal information type represented by the different intermediate packets, the data length of the different modal information, the number of data packets of the different modal information, and the storage location of the different modal information in the relay node, and then transmit it to the unequal error protection coding submodule. The feedback information processing submodule is used to determine whether feedback information sent by the destination node has been received. If no feedback information is received, selection weights of different importance levels are set. ( The system continues to execute the functions of the unequal error protection coding submodule; if feedback information is received, it executes the adaptive weight rotation allocation and adaptive code length adjustment algorithm. After completing the above operations based on the feedback information, it enters the unequal error protection coding submodule. The unequal error protection coding submodule is used to receive information from the previous module, perform unequal error protection coding according to the set selection weight and importance level, generate a coding packet and transmit it to the destination node module. After receiving feedback information, the feedback information processing submodule will temporarily "suspend" the node group that has reached the target bit error rate, that is, it will no longer select the data packets of the node group for unequal error protection coding, and will distribute the weight of the node group equally to the remaining node groups. At the same time, the relay node will also temporarily "suspend" the recovered data packets in the node group that has not reached the target bit error rate, and will no longer select the recovered data packets for further coding, based on the feedback information. The destination node module is used to decode the data packet after receiving the encoded packet according to the decoding rules of the unequal error protection rateless encoding to recover the data packet; and to recover different modal information according to the unpacking rules; at the same time, the destination node will continuously monitor the decoding status, and when a certain importance level reaches the target bit error rate, the destination node will feed back the current decoding status to the relay node.

2. The feedback-assisted multimodal rate-free coding inequality error protection system according to claim 1, characterized in that, The source node module includes a modal information acquisition submodule, a modal information extraction submodule, a modal information encoding submodule, an importance level classification submodule, and a packet grouping submodule, wherein: The modal information acquisition submodule is used to acquire corresponding modal information using different modal information acquisition devices, and to transmit the acquired modal information to the modal information extraction submodule; The modal information extraction submodule is used to extract useful modal information from the received modal information and pass it to the modal information encoding submodule; The modal information encoding submodule is used to perform modal information encoding on the received information, that is, to perform the corresponding source encoding, and then transmit the encoded information to the packet grouping submodule; The importance level classification submodule is used to classify different modal information into g importance levels, i.e. g node groups, according to the reliability and recovery delay requirements of different modalities. The packet assembly submodule is used to combine the received information into an intermediate packet according to the packet assembly rules, and then pass the intermediate packet to the relay node module.

3. The feedback-assisted multimodal rate-free coding inequality error protection system according to claim 2, characterized in that, The modal information includes audio, video, and pressure information.

4. The feedback-assisted multimodal rate-free coding inequality error protection system according to claim 1, characterized in that, In the destination node module, the feedback information includes the importance level of the target bit error rate and the data packets that have been recovered.

Images