Cross-node video and audio sharing method and system based on memory distribution

Abstract

The invention relates to the field of cross-node data sharing, two communication processes establish a memory channel based on descriptor driving based on a virtualization device driving architecture, the two communication processes cooperatively maintain an annular memory area, and a source process segments a video and audio stream to be shared into data blocks based on an image group boundary and writes the data blocks into a shared memory pool; and a consumption process obtains the descriptor from the annular control area through a polling or event-driven mechanism, directly accesses the corresponding video and audio data block in the shared memory pool according to the descriptor, realizes zero-copy data acquisition, and obtains the video and audio data through the state identification of the descriptor and a lightweight notification mechanism. And synchronizing data production and consumption between the source process and the consumption process. By constructing a zero-copy communication mechanism based on descriptor driving and an annular memory area, the delay of cross-node video and audio data sharing and the overhead of a processor are reduced, and efficient synchronization and non-copy data exchange of production and consumption processes are realized by utilizing a front-end and rear-end collaborative architecture.

Description

Technical Field

[0001] This invention relates to the field of cross-node data sharing, specifically to a cross-node audio and video sharing method and system based on memory distribution. Background Technology

[0002] In cross-node audio / video sharing scenarios, traditional solutions generally employ file copying or object store-and-forward models. The source node first writes the complete media file to centralized storage, and then the target node retrieves it via pull or download. This entire process involves multiple copies from kernel space to user space, coupled with a packet-by-packet acknowledgment mechanism based on transmission control protocols, resulting in low bandwidth utilization and high latency. When the playback client requires millisecond-level startup, the system has to pre-push multiple copies, causing a huge waste of memory and network resources. Furthermore, existing technologies mostly use fixed redundancy strategies to handle packet loss and out-of-order delivery, which cannot be dynamically adjusted according to real-time network conditions. They often send excessive redundant packets even when the packet loss rate fluctuates slightly, further exacerbating resource consumption.

[0003] The shortcomings of the above architecture are concentrated in three aspects. First, data needs to be transferred through disks or distributed object storage, resulting in significant read / write amplification, making it difficult to meet the requirements of low-latency live streaming and rapid channel switching. Second, the static configuration of redundant parameters lacks a comprehensive understanding of bandwidth, latency, and node memory availability, leading to excessive or insufficient redundant packets, affecting playback smoothness. Third, the lack of a refined memory reclamation mechanism after the session ends results in a mixture of hot and cold data residing in the shared memory pool, causing it to expand rapidly and requiring manual cleanup by operations personnel, increasing operational costs. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the problems existing in the above and / or existing memory-distributed cross-node audio and video sharing systems, the present invention is proposed.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a cross-node audio / video sharing method based on memory distribution, comprising the following steps:

[0007] The communicating processes establish a descriptor-driven memory channel based on the driver architecture of the virtualization device; the communicating processes jointly maintain at least one structured ring memory region as a carrier for cross-node data exchange;

[0008] The source process segments the audio and video stream to be shared into data blocks based on the image group boundary and writes them into the shared memory pool. The source process fills the descriptors in a predefined format to mark the metadata of the data blocks in the shared memory pool, places the descriptors in the shared ring control area, and broadcasts a quadruple of information containing the data block hash fingerprint, metadata, descriptor status identifier and lightweight notification to the cluster control plane.

[0009] The consumption process obtains descriptors from the ring control area through a polling or event-driven mechanism, and directly accesses the corresponding audio and video data blocks in the shared memory pool based on the descriptors, thereby achieving zero-copy data acquisition.

[0010] Data production and consumption between the source process and the consumer process are synchronized through the status identifier of the descriptor and the lightweight notification mechanism.

[0011] When the consumption process is a cross-node request, the cluster control plane calculates the redundancy parameters based on the received quadruple information and node topology status through a dynamic redundancy decision model, and instructs the source node to push the corresponding data block and the forward error correction redundancy package generated according to the redundancy parameters to the local shared memory pool of the requesting node through a predetermined relay link in a remote direct memory access environment.

[0012] On the requesting node side, a reorganization arbitrator is set up to perform out-of-order reordering and forward error correction decoding on the arriving data blocks. If decoding fails, a second fetch is performed from the nearest relay node. If decoding is successful, the data block is mapped to a buffer file handle accessible to the local player. After the shared session ends, the processes of both communicating parties release the data blocks in their respective shared memory pools.

[0013] As a preferred embodiment of the cross-node audio and video sharing method based on memory distribution described in this invention, the metadata in the shared memory pool includes frame size, data type, timestamp, and storage offset.

[0014] The predefined format descriptor adopts a structured data organization form, with fixed field length and field order;

[0015] The ring control area is divided into a descriptor storage area and a state synchronization area according to a preset partitioning rule, which are used to store valid descriptors and descriptor state change records, respectively.

[0016] In a preferred embodiment of the cross-node audio / video sharing method based on memory distribution described in this invention, the number R of personalized redundant packets is determined by the following formula:

[0017]

[0018] Where: p is the packet loss rate of the link network; B is the link bandwidth, B0 is the baseline bandwidth; D is the round-trip delay, D0 is the baseline delay; H is the shared memory pool remaining rate, H0 is the baseline remaining rate; t is the relative timestamp of the current session within period T; α, β, and γ are the bandwidth, delay, and memory sensitivity coefficients, respectively, which are obtained by offline training from historical session data.

[0019] As a preferred embodiment of the cross-node audio / video sharing method based on memory distribution described in this invention, the process of the reorganization arbitrator performing out-of-order reorganization and error recovery includes:

[0020] The arriving data blocks are double-checked and sorted based on the timestamp and preset sequence number in the descriptor;

[0021] If forward error correction decoding fails, then request additional redundant packets for the corresponding data block from the cluster control plane first.

[0022] If recovery still fails, a single-hop parallel secondary fetch is initiated from the shared memory pool of at least one nearby relay node. After a successful fetch, the local data block mapping is updated and the decoding process continues.

[0023] As a preferred embodiment of the cross-node audio / video sharing method based on memory distribution described in this invention, the release of data blocks in each shared memory pool adopts a cooperative two-level release strategy:

[0024] The first level uses reference counting to immediately reclaim data blocks that have been referenced by all processes;

[0025] The second level performs decaying delayed recycling based on the access frequency of data blocks, maintaining the memory mapping of high-frequency data blocks until the access frequency falls below the threshold.

[0026] To solve the above technical problems, the present invention provides the following technical solution: a cross-node audio and video sharing system based on memory distribution, characterized in that the system includes a front-end and back-end collaborative communication module, a global metadata broadcasting and link decision module, a zero-copy dynamic redundancy push module, a dual verification and error correction recovery module, and a collaborative two-level memory management module.

[0027] The front-end and back-end collaborative communication module includes a ring memory region construction unit and a descriptor production synchronization unit. The ring memory region construction unit is used to establish a structured ring memory region as a data exchange carrier between communication processes based on the virtualization device driver architecture concept. The descriptor production synchronization unit allows the source process to fill descriptors of a predefined format to mark data block metadata and place the descriptors in the ring control area. At the same time, it synchronizes data production and consumption through descriptor status identification and lightweight notification mechanism.

[0028] The global metadata broadcasting and link decision module includes a quadruple information aggregation unit and a dynamic redundancy parameter calculation unit. The quadruple information aggregation unit is used to receive and aggregate quadruple information broadcast by the source process, which includes hash fingerprint, metadata, descriptor status identifier and lightweight notification. The dynamic redundancy parameter calculation unit is used to calculate the redundancy parameters for forward error correction coding based on the quadruple information and node topology status when the consumer process is a cross-node request, through a dynamic redundancy decision model.

[0029] The zero-copy dynamic redundancy push module includes a remote direct memory access execution unit and an online redundancy package generation unit. The remote direct memory access execution unit is used to push data blocks in the shared memory pool of the source node to the local shared memory pool of the requesting node in a zero-copy manner through a predetermined relay link according to the instructions of the module. The online redundancy package generation unit is used to generate forward error correction redundancy packages in real time according to the calculated redundancy parameters during the push process and attach them to the end of the data block.

[0030] The dual verification and error correction recovery module includes a reordering and decoding unit and a redundant pull-back unit. The reordering and decoding unit is used to reorder the data blocks according to the timestamp and order through the reorganization arbitrator on the request node side and trigger forward error correction decoding. The redundant pull-back unit is used to start the error recovery process of single-hop double pull from the shared memory pool of the nearest relay node when the current decoding fails, and to complete the mapping of the data block to the local player buffer file handle after successful decoding.

[0031] The collaborative two-level memory management module includes an immediate reference reclamation unit and a hot-attenuation delayed reclamation unit. The immediate reference reclamation unit is used to immediately release the memory of data blocks referenced by all processes based on the reference counting strategy after the shared session ends. The hot-attenuation delayed reclamation unit is used to implement decay-type delayed reclamation based on the access hotness of data blocks, and maintains the memory mapping of data blocks with access hotness higher than a preset threshold to optimize resource reuse.

[0032] As a preferred embodiment of the cross-node audio / video sharing system based on memory distribution described in this invention, the calculation process performed by the dynamic redundancy parameter calculation unit is specifically as follows:

[0033] The system receives real-time data from the quadruple information aggregation unit, combines it with historical data on real-time bandwidth, transmission delay, and packet loss rate of the predetermined relay link in the node topology state, and introduces the data acquisition delay tolerance declared by the requesting node or derived by the system as a constraint. Through a dynamic redundancy decision model, it performs joint optimization calculations and outputs the number of forward error correction coding redundancy packets and specific coding strategy parameters that match the current network state and session requirements.

[0034] As a preferred embodiment of the cross-node audio / video sharing system based on memory distribution described in this invention, the specific implementation of the reordering and decoding unit includes:

[0035] A reordering reorderer based on a double-buffered queue and time window mechanism is established. The reorderer performs initial sorting and caching based on the timestamps in the data block descriptors.

[0036] An independent sequence checker performs secondary verification and final ordering on the sorted data blocks according to a preset logical sequence number;

[0037] The sorted data blocks are submitted in batches to the forward error correction decoder for decoding.

[0038] This invention provides the following technical solution: an electronic device, comprising:

[0039] One or more processors;

[0040] A storage device on which one or more programs are stored;

[0041] When the one or more programs are executed by the one or more processors, the one or more processors implement a memory-distributed cross-node audio and video sharing method.

[0042] This invention provides the following technical solution: an electronic device, comprising:

[0043] A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to implement a memory-distributed cross-node audio / video sharing method.

[0044] This invention significantly reduces latency and processor overhead in cross-node audio and video data sharing by constructing a zero-copy communication mechanism based on descriptor-driven and ring memory regions. Utilizing a front-end-back-end collaborative architecture, it achieves efficient synchronization and copy-free data exchange between production and consumption processes. Through global metadata broadcasting and dynamic redundancy decision-making, the system can adapt to network topology changes, generating optimal error correction strategies for each request and ensuring robust transmission under complex network conditions. Dual verification and intelligent rollback mechanisms effectively guarantee the integrity and sequential correctness of data in out-of-order or partially lost scenarios. Furthermore, a collaborative two-level memory management strategy intelligently balances the immediate reclamation and reuse retention of memory resources, comprehensively improving the throughput efficiency and resource utilization of audio and video stream sharing in large-scale node clusters. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0046] Figure 1 This is a flowchart of the cross-node audio and video sharing method based on memory distribution in Example 1.

[0047] Figure 2 This is an operation flowchart of the cross-node audio and video sharing system based on memory distribution in Example 2. Detailed Implementation

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0051] Example 1

[0052] Reference Figure 1 This is the first embodiment of the present invention, which provides a cross-node audio / video sharing method based on memory distribution, comprising:

[0053] The communicating processes establish a descriptor-driven memory channel based on the driver architecture of the virtualization device; the communicating processes jointly maintain at least one structured ring memory region as a carrier for cross-node data exchange;

[0054] The source process segments the audio and video stream to be shared into data blocks based on the image group boundary and writes them into the shared memory pool. The source process fills the descriptors in a predefined format to mark the metadata of the data blocks in the shared memory pool, places the descriptors in the shared ring control area, and broadcasts a quadruple of information containing the data block hash fingerprint, metadata, descriptor status identifier and lightweight notification to the cluster control plane.

[0055] Meta-information in the shared memory pool includes frame size, data type, timestamp, and storage offset;

[0056] The predefined format descriptor adopts a structured data organization form, with fixed field length and field order;

[0057] The ring control area is divided into a descriptor storage area and a state synchronization area according to a preset partitioning rule, which are used to store valid descriptors and descriptor state change records, respectively.

[0058] Redundancy parameters are determined by the following formula:

[0059]

[0060] Where: p is the packet loss rate of the link network; B is the link bandwidth, B0 is the baseline bandwidth; D is the round-trip delay, D0 is the baseline delay; H is the shared memory pool remaining rate, H0 is the baseline remaining rate; t is the relative timestamp of the current session within period T; α, β, and γ are the bandwidth, delay, and memory sensitivity coefficients, respectively, which are obtained by offline training from historical session data.

[0061] The dynamic redundancy decision model calculates redundancy parameters based on the node topology state and the quadruple information. Specifically, it includes: dynamically determining the number of redundant packets and the coding strategy for forward error correction coding based on the real-time available bandwidth, transmission latency, and historical packet loss rate of the predetermined relay link, combined with the data acquisition delay tolerance of the requesting node; the source node and the relay link perform data push and redundant packet generation in parallel under a remote direct memory access environment.

[0062] The consumption process obtains descriptors from the ring control area through a polling or event-driven mechanism, and directly accesses the corresponding audio and video data blocks in the shared memory pool based on the descriptors, thereby achieving zero-copy data acquisition.

[0063] Data production and consumption between the source process and the consumer process are synchronized through the status identifier of the descriptor and the lightweight notification mechanism.

[0064] When the consumption process is a cross-node request, the cluster control plane calculates the redundancy parameters based on the received quadruple information and node topology status through a dynamic redundancy decision model, and instructs the source node to push the corresponding data block and the forward error correction redundancy package generated according to the redundancy parameters to the local shared memory pool of the requesting node through a predetermined relay link in a remote direct memory access environment.

[0065] On the requesting node side, a reorganization arbitrator is set up to perform out-of-order reordering and forward error correction decoding on the arriving data blocks. If decoding fails, a second fetch is performed from the nearest relay node. If decoding is successful, the data block is mapped to a buffer file handle accessible to the local player. After the shared session ends, the processes of both communicating parties release the data blocks in their respective shared memory pools.

[0066] The process of reorganizing the arbitrator to perform out-of-order reorganization and error recovery includes:

[0067] The arriving data blocks are double-checked and sorted based on the timestamp and preset sequence number in the descriptor;

[0068] If forward error correction decoding fails, then request additional redundant packets for the corresponding data block from the cluster control plane first.

[0069] If recovery still fails, a single-hop parallel secondary fetch is initiated from the shared memory pool of at least one nearby relay node. After a successful fetch, the local data block mapping is updated and the decoding process continues.

[0070] A cooperative two-level release strategy is used to release data blocks in their respective shared memory pools:

[0071] The first level uses reference counting to immediately reclaim data blocks that have been referenced by all processes;

[0072] The second level performs decaying delayed recycling based on the access frequency of data blocks, maintaining the memory mapping of high-frequency data blocks until the access frequency falls below the threshold.

[0073] Example 2

[0074] Reference Figure 2 This is the second embodiment of the present invention, a cross-node audio and video sharing system based on memory distribution. The system includes a front-end and back-end collaborative communication module 100, a global metadata broadcasting and link decision module 200, a zero-copy dynamic redundancy push module 300, a dual verification and error correction recovery module 400, and a collaborative two-level memory management module 500.

[0075] The front-end and back-end collaborative communication module 100 includes a ring memory region construction unit 101 and a descriptor production synchronization unit 102. The ring memory region construction unit 101 is used to establish a structured ring memory region as a data exchange carrier between communication processes based on the virtualization device driver architecture concept. The descriptor production synchronization unit 102 allows the source process to fill descriptors of a predefined format to mark data block metadata and place the descriptors in the ring control region. At the same time, it synchronizes data production and consumption through descriptor status identification and lightweight notification mechanism.

[0076] The global metadata broadcasting and link decision module 200 includes a quadruple information aggregation unit 201 and a dynamic redundancy parameter calculation unit 202. The quadruple information aggregation unit 201 is used to receive and aggregate quadruple information broadcast by the source process, which includes hash fingerprint, metadata, descriptor status identifier and lightweight notification. The dynamic redundancy parameter calculation unit 202 is used to calculate the redundancy parameters for forward error correction coding based on the quadruple information and node topology status when the consuming process is a cross-node request, through a dynamic redundancy decision model.

[0077] The zero-copy dynamic redundancy push module 300 includes a remote direct memory access execution unit 301 and an online redundancy packet generation unit 302. The remote direct memory access execution unit 301 is used to push data blocks in the shared memory pool of the source node to the local shared memory pool of the requesting node in a zero-copy manner through a predetermined relay link according to the instructions of the module 200. The online redundancy packet generation unit 302 is used to generate forward error correction redundancy packets in real time according to the calculated redundancy parameters during the push process and attach them to the end of the data block.

[0078] The calculation process performed by the dynamic redundancy parameter calculation unit 202 is as follows:

[0079] The system receives real-time data from the quadruple information aggregation unit 201, combines it with historical data on real-time bandwidth, transmission delay, and packet loss rate of the predetermined relay link in the node topology state, and introduces the data acquisition delay tolerance declared by the requesting node or derived by the system as a constraint. Through the dynamic redundancy decision model, it performs joint optimization calculations and outputs the number of forward error correction coding redundancy packets and specific coding strategy parameters that match the current network state and session requirements.

[0080] The dual verification and error correction recovery module 400 includes a reordering and decoding unit 401 and a redundant pull-back unit 402. The reordering and decoding unit 401 is used to reorder data blocks according to timestamps and order by using a reorganization arbitrator on the request node side and trigger forward error correction decoding. The redundant pull-back unit 402 is used to start an error recovery process of single-hop double pull from the shared memory pool of the nearest relay node when the current decoding fails, and to complete the mapping of data blocks to local player buffer file handles after successful decoding.

[0081] The specific implementation methods of the reordering and decoding unit 401 include:

[0082] A reordering reorderer based on a double-buffered queue and time window mechanism is established. The reorderer performs initial sorting and caching based on the timestamps in the data block descriptors.

[0083] An independent sequence checker performs secondary verification and final ordering on the sorted data blocks according to a preset logical sequence number;

[0084] The sorted data blocks are submitted in batches to the forward error correction decoder for decoding.

[0085] The collaborative two-level memory management module 500 includes an immediate reference reclamation unit 501 and a hot-attenuation delayed reclamation unit 502. The immediate reference reclamation unit 501 is used to immediately release the memory of data blocks referenced by all processes based on a reference counting strategy after the shared session ends. The hot-attenuation delayed reclamation unit 502 is used to implement a decay-type delayed reclamation based on the access hotness of data blocks, and maintain the memory mapping of data blocks whose access hotness is higher than a preset threshold to optimize resource reuse.

[0086] Example 3

[0087] A third embodiment of the present invention provides a cross-node audio / video sharing method based on memory distribution, comprising:

[0088] To verify the actual performance of the cross-node audio and video sharing method and system based on memory distribution described in this invention, a systematic test was conducted in an experimental environment simulating a cloud media processing center.

[0089] This environment constructs a heterogeneous cluster, containing 1 source node, 3 relay nodes, and 10 request nodes.

[0090] Network links between nodes are configured in differentiated states to simulate the unevenness of real data center networks:

[0091] The bandwidth is distributed between 1Gbps and 10Gbps, the link round-trip latency is set to between 1ms and 20ms, and the random packet loss rate is controlled between 0.1% and 5%.

[0092] The test payload consisted of a 300-second 4K ultra-high-definition video stream with a constant bitrate of 50 Mbps and a fixed group of pictures (GOP) structure of 30 frames per second.

[0093] The implementation process strictly follows the steps described in this invention.

[0094] First, during the source node initialization process, based on the virtualization device driver architecture concept, a structured ring memory region containing 1024 entries is created as the control plane, and a shared memory pool with a capacity of 2GB is associated and partitioned.

[0095] The source process divides the input video stream into independent data blocks according to GOP boundaries, calculates their SHA-256 hash fingerprints, and writes each data block and its metadata (size, timestamp) to a shared memory pool.

[0096] Subsequently, the source process populates a predefined format descriptor, precisely pointing to the location of the data block in the memory pool, and marks the descriptor status as "available," while updating the ring control region through a lightweight event file descriptor mechanism.

[0097] In addition, the quadruple information, which includes hash fingerprint, timestamp, descriptor index and event notification identifier, is broadcast to the central cluster control plane in real time.

[0098] When any requesting node initiates a pull request, the cluster control plane responds immediately.

[0099] The control plane first searches the current node's topology state table to select two optimal relay links for the requesting node. Subsequently, the dynamic redundancy decision model is activated.

[0100] Upon receiving the instruction, the source node and the selected relay node initiate a remote direct memory access (RDMA) operation in parallel.

[0101] By using the memory window "stretching" technique, the target data block is directly mapped from the source node's memory pool to the requesting node's local memory pool, achieving zero-copy transmission.

[0102] In the data transmission pipeline, the system generates Reed-Solomon encoded redundant packets in real time and appends them to the end of the original data blocks for simultaneous push.

[0103] On the requesting node side, the reorganization arbitrator continues to operate.

[0104] Its internal double-buffered queue and time window mechanism first sorts the data blocks based on their timestamps.

[0105] If a continuous sequence is collected within a preset time window (such as the duration of a GOP), the forward erasure decoder is immediately triggered to recover the data. If decoding fails or a sequence number gap is detected, the arbitrator immediately requests additional redundant packets from the cluster control plane.

[0106] If the request still fails, a single-hop secondary fetch request is quickly initiated to the nearest relay node to ensure playback continuity. The raw data block after successful decoding is directly mapped to a memory file handle recognizable by the local player.

[0107] Table 1: Comparison of end-to-end latency for cross-node transmission (unit: milliseconds)

[0108] Transmission scheme Average delay 95th percentile delay Delay jitter Preparation time for first playback Traditional TCP / IP + application layer retransmission 85.2 203.5 45.7 1200 Standard RDMA (without dynamic redundancy) 32.1 78.6 18.3 850 Invention Solution 18.7 41.2 8.5 600

[0109] Table 2: Comparison of System CPU Resource Overhead (Unit: Percentage)

[0110] Transmission scheme Source node CPU utilization Request node CPU utilization Percentage of memory copy operations Traditional TCP / IP + application layer retransmission 38.5% 45.2% 22% Standard RDMA (without dynamic redundancy) 15.8% 20.4% <1% Invention Solution 12.1% 14.7% <1%

[0111] Table 3: Comparison of Network Throughput and Bandwidth Utilization

[0112] Transmission scheme Effective throughput (Gbps) Bandwidth utilization Redundant data ratio Traditional TCP / IP + application layer retransmission 7.1 71% 0% (Retransmission Only) Standard RDMA (without dynamic redundancy) 8.5 85% 0% Invention Solution 9.2 92% 3.5%

[0113] Table 4: Success rate of complete data block recovery under different packet loss rates (unit: percentage)

[0114] Average packet loss rate of the link Success rate of traditional solutions Standard RDMA success rate Success rate of the solution of this invention 1% 98.5% 99.0% 99.98% 3% 91.2% 91.8% 99.65% 5% 82.7% 83.1% 98.90%

[0115] Table 5: Comparison of memory management efficiency (500ms after shared session ends)

[0116] plan Memory instant reclamation rate High heat block retention rate Number of times memory is redistributed Immediate full release strategy 100% 0% high Fixed time window retention 60% 35% middle This invention has a two-level strategy. 95% 88% Low

[0117] Table 6: Performance Analysis of Dynamic Redundancy Decision Model (Target Delay Tolerance 100ms)

[0118] Link status (bandwidth / packet loss) Fixed redundancy strategy (10%) actual latency Actual delay of the dynamic strategy of this invention Dynamic strategy to calculate the number of redundant packets R High bandwidth (8Gbps) / Low packet loss (1%) 25ms 18ms 2 Medium bandwidth (4Gbps) / Medium packet loss (3%) 89ms (close to timeout) 52ms 5 Low bandwidth (2Gbps) / High packet loss (5%) Timeout (>100ms) 85ms 8

[0119] A comprehensive analysis of the above experimental data shows that the method and system proposed in this invention are significantly superior to existing technical solutions in terms of multi-dimensional performance indicators, demonstrating their outstanding creativity, novelty and technical advantages.

[0120] In terms of transmission efficiency and real-time performance, this invention demonstrates superior performance.

[0121] As shown in Table 1, compared with the traditional TCP / IP scheme, the average end-to-end latency of the present invention is reduced by about 78%, and by about 42% compared with the standard RDMA scheme. More importantly, the 95th percentile latency (i.e., tail latency) and latency jitter are significantly improved, which is directly due to the zero-copy mechanism eliminating the overhead of kernel protocol stack and multiple memory copying, as well as the forward-looking adaptation of the dynamic redundancy decision model to network fluctuations.

[0122] Table 6 clearly demonstrates the superiority of this model: Under moderate and severe network conditions, the fixed redundancy strategy either leads to a surge in retransmission delay due to insufficient redundancy or introduces unnecessary delay due to excessive redundancy; while the dynamic strategy of this invention, through precise calculation, ensures reliability while always keeping the delay within the tolerable threshold, achieving an intelligent balance between reliability and low latency.

[0123] In terms of system resource utilization, the advantages of this invention are also obvious.

[0124] Table 2 shows that the CPU utilization of both the source and request nodes reached its lowest level, thanks to the descriptor-based ring memory region synchronization mechanism. This mechanism replaces frequent system calls and interrupts with lightweight event notifications, resulting in minimal process synchronization overhead. Combined with true zero-copy implementation of RDMA (memory copy operations account for <1%), the system completely frees the CPU from heavy data transfer tasks, allowing it to handle more business logic or serve more concurrent streams.

[0125] Table 3 demonstrates the high efficiency of this invention from a network perspective, showing the highest effective throughput and bandwidth utilization. Despite introducing an average of 3.5% redundant data (Table 3), the overall transmission efficiency is actually improved by avoiding the throughput collapse and waiting time caused by traditional retransmission mechanisms. This proves that dynamically injected redundancy is an effective strategy for trading huge latency and deterministic benefits at the cost of minimal bandwidth.

[0126] In terms of transmission reliability and robustness, this invention achieves a qualitative leap by introducing forward error correction and intelligent backoff mechanisms.

[0127] The data in Table 4 is convincing: under different levels of link packet loss rates, the data block integrity recovery success rate of this invention is close to or exceeds 99.5%, significantly higher than that of traditional solutions. This is because traditional solutions rely on retransmission after the fact, which can easily fall into a vicious cycle of "retransmission-packet loss-retransmission" under poor network conditions; while this invention, through dynamically generated redundant packets, can instantly repair most random packet losses at the receiving end, and only initiates an efficient "secondary fetch" backoff mechanism for sudden, continuous packet loss.

[0128] This hybrid recovery mode, which prioritizes forward error correction and supplements it with re-fetching, is one of the key innovations of this invention, ensuring highly reliable delivery over unreliable networks.

[0129] Finally, in terms of intelligent and refined resource management, the collaborative two-level memory management strategy proposed in this invention demonstrates significant value.

[0130] Table 5 compares the effects of different memory reclamation strategies. While the simple immediate full release strategy is fast, it completely sacrifices the possibility of data reuse, causing subsequent requests to face repeated memory allocation and data loading overhead. The fixed-time-window retention strategy is not flexible enough.

[0131] The strategy of this invention achieves an instant recovery rate of up to 95%, while accurately retaining 88% of high-frequency data blocks through a heat decay algorithm. This allows nearby nodes to access these popular data blocks almost instantly, retrieving them directly from memory, significantly reducing overall system response latency and memory allocator pressure. This demonstrates that this invention not only focuses on the performance of a single transmission but also optimizes the lifecycle and reuse efficiency of resources from a system-wide perspective.

[0132] In summary, this embodiment demonstrates through detailed comparative data that the present invention is not a simple combination of existing technologies, but rather a systematic solution to the core technical challenge of balancing low latency, high reliability, high throughput, and low overhead in cross-node audio and video sharing. This is achieved through the organic integration and synergistic effect of multiple innovative points, including a descriptor-driven zero-copy ring communication architecture, a dynamic redundancy decision model based on real-time link status and business requirements, a dual reliability guarantee mechanism combining forward error correction and intelligent fallback, and a two-level memory management strategy that coordinates reference counting and heat decay.

Claims

1. A cross-node audio / video sharing method based on memory distribution, characterized in that, Includes the following steps: The communicating processes establish a memory channel based on the driver architecture of the virtualization device, using a descriptor-driven architecture. The two communicating processes collaboratively maintain at least one structured ring-shaped memory region as a carrier for cross-node data exchange; The source process segments the audio and video stream to be shared into data blocks based on the image group boundary and writes them into the shared memory pool. The source process fills the descriptors in a predefined format to mark the metadata of the data blocks in the shared memory pool, places the descriptors in the shared ring control area, and broadcasts a quadruple of information containing the data block hash fingerprint, metadata, descriptor status identifier and lightweight notification to the cluster control plane. The consumption process obtains descriptors from the ring control area through a polling or event-driven mechanism, and directly accesses the corresponding audio and video data blocks in the shared memory pool based on the descriptors, thereby achieving zero-copy data acquisition. Data production and consumption between the source process and the consumer process are synchronized through the status identifier of the descriptor and the lightweight notification mechanism. When the consumption process is a cross-node request, the cluster control plane calculates the redundancy parameters based on the received quadruple information and node topology status through a dynamic redundancy decision model, and instructs the source node to push the corresponding data block and the forward error correction redundancy package generated according to the redundancy parameters to the local shared memory pool of the requesting node through a predetermined relay link in a remote direct memory access environment. On the requesting node side, a reorganization arbitrator is set up to perform out-of-order reordering and forward error correction decoding on the arriving data blocks. If decoding fails, a second fetch is performed from the nearest relay node. If decoding is successful, the data block is mapped to a buffer file handle accessible to the local player. After the shared session ends, the processes of both communicating parties release the data blocks in their respective shared memory pools.

2. The cross-node audio / video sharing method based on memory distribution as described in claim 1, characterized in that: The metadata in the shared memory pool includes frame size, data type, timestamp, and storage offset; The predefined format descriptor adopts a structured data organization form, with fixed field length and field order; The ring control area is divided into a descriptor storage area and a state synchronization area according to a preset partitioning rule, which are used to store valid descriptors and descriptor state change records, respectively.

3. The cross-node audio / video sharing method based on memory distribution as described in claim 1, characterized in that: The redundancy parameter is determined by the following formula: Where: p is the packet loss rate of the link network; B is the link bandwidth, B0 is the baseline bandwidth; D is the round-trip delay, D0 is the baseline delay; H is the shared memory pool remaining rate, H0 is the baseline remaining rate; t is the relative timestamp of the current session within period T; α, β, and γ are the bandwidth, delay, and memory sensitivity coefficients, respectively, which are obtained by offline training from historical session data.

4. The cross-node audio / video sharing method based on memory distribution as described in claim 1, characterized in that... The process of the reorganization arbitrator performing out-of-order reorganization and error recovery includes: The arriving data blocks are double-checked and sorted based on the timestamp and preset sequence number in the descriptor; If forward error correction decoding fails, then request additional redundant packets for the corresponding data block from the cluster control plane first. If recovery still fails, a single-hop parallel secondary fetch is initiated from the shared memory pool of at least one nearby relay node. After a successful fetch, the local data block mapping is updated and the decoding process continues.

5. The cross-node audio / video sharing method based on memory distribution as described in claim 1, characterized in that... The release of data blocks in their respective shared memory pools adopts a cooperative two-level release strategy: The first level uses reference counting to immediately reclaim data blocks that have been referenced by all processes; The second level performs decaying delayed recycling based on the access frequency of data blocks, maintaining the memory mapping of high-frequency data blocks until the access frequency falls below the threshold.

6. A cross-node audio / video sharing system based on memory distribution, characterized in that, The system includes a front-end and back-end collaborative communication module (100), a global metadata broadcasting and link decision module (200), a zero-copy dynamic redundancy push module (300), a dual verification and error correction recovery module (400), and a collaborative two-level memory management module (500). The front-end and back-end collaborative communication module (100) includes a ring memory region construction unit (101) and a descriptor production synchronization unit (102). The ring memory region construction unit (101) is used to establish a structured ring memory region as a data exchange carrier between communication processes based on the virtualization device driver architecture concept. The descriptor production synchronization unit (102) allows the source process to fill descriptors of a predefined format to mark data block metadata and place the descriptors in the ring control region. At the same time, it synchronizes data production and consumption through descriptor status identification and lightweight notification mechanism. The global metadata broadcasting and link decision module (200) includes a quadruple information aggregation unit (201) and a dynamic redundancy parameter calculation unit (202). The quadruple information aggregation unit (201) is used to receive and aggregate quadruple information broadcast by the source process, which includes hash fingerprint, metadata, descriptor status identifier and lightweight notification. The dynamic redundancy parameter calculation unit (202) is used to calculate the redundancy parameters for forward error correction coding based on the quadruple information and node topology status when the consuming process is a cross-node request, through the dynamic redundancy decision model. The zero-copy dynamic redundancy push module (300) includes a remote direct memory access execution unit (301) and an online redundancy packet generation unit (302). The remote direct memory access execution unit (301) is used to push data blocks in the shared memory pool of the source node to the local shared memory pool of the requesting node in a zero-copy manner through a predetermined relay link according to the instructions of the module (200). The online redundancy packet generation unit (302) is used to generate forward error correction redundancy packets in real time according to the calculated redundancy parameters and attach them to the end of the data block during the push process. The dual verification and error correction recovery module (400) includes a reordering and decoding unit (401) and a redundant pull-back unit (402). The reordering and decoding unit (401) is used to reorder the data blocks according to the timestamp and order through the reorganization arbitrator on the request node side and trigger forward error correction decoding. The redundant pull-back unit (402) is used to start the error recovery process of single-hop double pull from the shared memory pool of the nearest relay node when the current decoding fails, and complete the mapping of the data block to the local player buffer file handle after successful decoding. The collaborative two-level memory management module (500) includes an immediate reference reclamation unit (501) and a hot-attenuation delayed reclamation unit (502). The immediate reference reclamation unit (501) is used to immediately release the memory of data blocks referenced by all processes based on the reference counting strategy after the shared session ends. The hot-attenuation delayed reclamation unit (502) is used to implement a decay-type delayed reclamation based on the access hotness of the data blocks, and maintain the memory mapping of data blocks whose access hotness is higher than a preset threshold to optimize resource reuse.

7. The cross-node audio / video sharing system based on memory distribution as described in claim 6, characterized in that, The calculation process performed by the dynamic redundancy parameter calculation unit (202) is as follows: The system receives real-time data from the quadruple information aggregation unit (201), combines it with historical data on real-time bandwidth, transmission delay, and packet loss rate of the predetermined relay link in the node topology state, and introduces the data acquisition delay tolerance declared by the requesting node or derived by the system as a constraint. It then performs joint optimization calculations through a dynamic redundancy decision model and outputs the number of forward error correction coding redundancy packets and specific coding strategy parameters that match the current network state and session requirements.

8. The cross-node audio / video sharing system based on memory distribution as described in claim 6, characterized in that, The specific implementation of the reordering and decoding unit (401) includes: A reordering reorderer based on a double-buffered queue and time window mechanism is established. The reorderer performs initial sorting and caching based on the timestamps in the data block descriptors. An independent sequence checker performs secondary verification and final ordering on the sorted data blocks according to a preset logical sequence number; The sorted data blocks are submitted in batches to the forward error correction decoder for decoding.

9. An electronic device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method and system as described in any one of claims 1-7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to implement the method and system of any one of claims 1-7.

Images