Multi-protocol data fusion transmission method and system in conference scenario and storage medium

Abstract

The application provides a multi-protocol data fusion transmission method and system in a conference scenario and a storage medium. The method comprises establishing a multi-protocol concurrent transmission path at a transceiving end and allocating a service level identifier to a data stream; constructing a path state vector based on a bandwidth delay product, a packet loss rate and a smooth round-trip delay, evaluating the path state through Mahalanobis distance, and enhancing detection according to the data stream level when the backup path trend is better. The multi-path state is fused through Kalman filtering, a finite state machine decoupled from the underlying congestion control is established, a session-level virtual congestion state containing a virtual congestion window and a congestion momentum is estimated, and path switching is performed when the conditions are met. Meanwhile, a flow-path weight scheduling matrix is constructed to increase the weight of high-level data streams on the better path when the network deteriorates; the initial sending window of the new path is obtained by combining the bandwidth delay product ratio, the congestion momentum and the time step projection during switching, thereby improving the conference transmission continuity and stability.

Description

Technical Field

[0001] This application belongs to the field of multi-protocol data fusion, and in particular relates to a multi-protocol data fusion transmission method, system and storage medium in a conference scenario. Background Technology

[0002] With the rapid development of mobile communications, the Internet of Things (IoT), and real-time multimedia services, heterogeneous network coexistence has become a common feature in modern communication architectures. To meet the demands of massive data transmission and the differentiated bandwidth, latency, and reliability requirements of various services, terminal devices typically need to transmit data via multiple concurrent paths using different transmission protocols. However, in real-world network environments, the bandwidth, latency, and packet loss rates of each transmission path vary significantly and fluctuate continuously. Traditional transmission mechanisms often tightly couple congestion control algorithms with path switching logic. When the quality of the primary path deteriorates and a switch to a backup path is required, the system struggles to uniformly assess and migrate session-level congestion states. The new path often needs to re-establish congestion windows or directly reuse old transmission parameters that do not match the new path's state, easily leading to reduced throughput, decreased bandwidth utilization, and increased transmission latency.

[0003] Furthermore, during concurrent multi-protocol transmission, different data streams have varying requirements for network service levels. Existing scheduling schemes typically lack the ability to integrate and perceive heterogeneous network states, and are also unable to coordinate scheduling based on service priorities. When the network environment deteriorates or multiple paths become congested, traditional methods struggle to comprehensively assess the overall session status using multi-dimensional parameters such as bandwidth, latency, and packet loss rate. They also struggle to reduce the impact of local path fluctuations on the global transmission process, thus hindering the transmission quality of high-priority services on optimal paths. In cross-path switching scenarios, existing technologies lack congestion state transition mechanisms based on the actual network characteristics of the old and new paths, making it difficult to achieve reasonable projection and smooth transition of transmission parameters, thereby affecting the overall efficiency and stability of concurrent multi-protocol transmission. Therefore, how to decouple congestion control from path switching logic and improve the continuity and smoothness of data transmission during network fluctuations and path switching has become a pressing technical problem to be solved in this field. Summary of the Invention

[0004] To address the problems of insufficient transmission continuity and stability caused by the coupling of congestion control and path switching, difficulty in integrating and perceiving network status, and lack of congestion state transition mechanism during multi-protocol concurrent transmission in existing technologies, this invention provides a multi-protocol data fusion transmission method, system, and storage medium for conference scenarios.

[0005] In a first aspect, the present invention proposes a method for multi-protocol data fusion and transmission in a conference scenario, comprising the following steps: Concurrent transmission paths using different transmission protocols are established between the transmitting and receiving ends, and a corresponding service level identifier is assigned to each data stream; a path state vector containing bandwidth-delay product, packet loss rate, and smooth round-trip time is constructed; the Mahalanobis distance between each path state vector and a preset ideal state vector is calculated; when the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector shows a better decreasing trend than that of the main path, the detection intensity is increased by combining the service level identifier of the data stream to be assigned to the backup path or assisted by the backup path for transmission testing. By fusing the state vectors of each path using the Kalman filter algorithm, a finite state machine decoupled from the congestion control of each path is maintained to estimate a virtual congestion state representing the overall network status of the session. The virtual congestion state includes a virtual congestion window and congestion momentum. When the Mahalanobis distance of any backup path is continuously less than the product of the Mahalanobis distance of the main path and a preset proportional threshold, and the continuous satisfaction time reaches a preset duration threshold, the finite state machine is triggered to perform a path switching operation. A data flow and path weight scheduling matrix is ​​constructed. When the virtual congestion state indicates network deterioration, priority is given to ensuring the transmission weight of high-service-level data flows on the target transmission path. When performing the path switching operation, based on the ratio of the measured bandwidth-delay product of the new primary path and the old primary path, the congestion momentum, and the time step, the sending window allocated to the old primary path before the switch is projected as the initial sending window of the new primary path.

[0006] On the other hand, the present invention also proposes a multi-protocol data fusion transmission system for conference scenarios, comprising the following modules: The calculation module is used to establish concurrent transmission paths using different transmission protocols between the transceiver and the receiver, assign a corresponding service level identifier to each data stream; construct a path state vector including bandwidth-delay product, packet loss rate, and smooth round-trip time; calculate the Mahalanobis distance between each path state vector and a preset ideal state vector; when the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector shows a better decreasing trend than that of the main path, the detection intensity is increased by combining the service level identifier of the data stream to be assigned to the backup path or assisted by the backup path for transmission testing. The fusion module is used to fuse the state vectors of each path using a Kalman filter algorithm, maintain a finite state machine decoupled from the congestion control of each path, estimate a virtual congestion state representing the overall network status of the session, the virtual congestion state including a virtual congestion window and congestion momentum; when the Mahalanobis distance of any backup path is continuously less than the product of the Mahalanobis distance of the main path and a preset proportion threshold, and the continuous satisfaction time reaches a preset duration threshold, the finite state machine is triggered to perform a path switching operation. The construction module is used to construct a data flow and path weight scheduling matrix. When the virtual congestion state indicates network deterioration, it prioritizes the transmission weight of high-service-level data flows on the target transmission path. When performing the path switching operation, based on the ratio of the measured bandwidth-delay product of the new primary path and the old primary path, the congestion momentum, and the time step, the sending window allocated to the old primary path before the switch is projected as the initial sending window of the new primary path.

[0007] This invention establishes concurrent transmission paths using different protocols at the transceiver ends and assigns service level identifiers to data streams. It constructs path state vectors by combining bandwidth-delay product, packet loss rate, and smoothed round-trip time, and evaluates path states using Mahalanobis distance while dynamically adjusting probe intensity. By fusing multi-path states through Kalman filtering, a finite state machine decoupled from the underlying congestion control is established to estimate session-level virtual congestion states including virtual congestion windows and congestion momentum, reducing the mutual influence between path switching logic and congestion control. Furthermore, through a data stream and path weight scheduling matrix, the scheduling weight of high-service-level data streams on better paths is increased when the network deteriorates. During path switching, the initial transmission window is based on the ratio of the bandwidth-delay product of the old and new paths and the projection of congestion momentum, thereby improving switching smoothness, reducing the risk of throughput degradation, and enhancing the stability of multi-protocol fused transmission. Attached Figure Description

[0008] Figure 1 A flowchart of the first embodiment; Figure 2 A bar chart comparing the frequency of probe packets and the probe period interval under different QoS service levels. Detailed Implementation

[0009] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0010] In the first embodiment, the present invention proposes a multi-protocol data fusion and transmission method in a conference scenario, such as... Figure 1 As shown, it includes the following steps: S1. Establish concurrent transmission paths using different transmission protocols between the transmitting and receiving ends, and assign a corresponding service level identifier to each data stream; construct a path state vector containing bandwidth-delay product, packet loss rate, and smooth round-trip time; calculate the Mahalanobis distance between each path state vector and a preset ideal state vector; when the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector shows a better decreasing trend than that of the main path, increase the detection intensity by combining the service level identifier of the data stream to be assigned to the backup path or assisted in the transmission test by the backup path.

[0011] When establishing concurrent transmission paths, the sending and receiving ends create one TCP transmission path and one UDP transmission path respectively, and designate one as the primary path and the other as the backup path; at the same time, a differentiated service level identifier (QoS) field is written into the application layer data packet header for each data stream.

[0012] During data transmission, probe packets are periodically sent, and round-trip timestamps are recorded via acknowledgment messages to calculate the smoothed round-trip time (SRTT). The bottleneck bandwidth is obtained using the congestion control bandwidth estimation mechanism in the TCP path or the application layer rate sampling mechanism in the UDP path, and multiplied by the smoothed round-trip time to obtain the bandwidth-delay product (BDP). For the TCP transmission path, the number of successfully delivered packets and the number of lost packets are determined based on the ACK acknowledgment messages. For the UDP transmission path, the number of successfully delivered packets and the number of lost packets are determined based on the application layer acknowledgment information, probe response information, or received sequence number acknowledgment returned by the receiver, and the packet loss rate of the corresponding path is calculated accordingly. The bandwidth-delay product, the packet loss rate, and the smoothed round-trip time are combined to construct a one-dimensional feature array as the path state vector.

[0013] On the server side, a target baseline vector composed of the target packet loss rate, the target smooth round-trip time, and the target bandwidth-delay product is set as the preset ideal state vector. Data sets of path state vectors collected within historical time windows are merged to solve for the multidimensional variable sample covariance matrix. A preset regularization coefficient is then superimposed on the main diagonal of this covariance matrix to obtain a regularized covariance matrix. The difference vector between the current path state vector and the preset ideal state vector is calculated. This difference vector is represented as a column vector, and the transpose of the difference vector, the inverse of the regularized covariance matrix, and the difference vector are multiplied sequentially. The square root of the result yields the Mahalanobis distance.

[0014] The change in Mahalanobis distance between two adjacent time steps is recorded. The magnitude by which the Mahalanobis distance in a subsequent time step is lower than that in the previous time step is taken as the Mahalanobis distance reduction amount. The EWMA (Exponentially Weighted Moving Average) algorithm is used to calculate the reduction trend value of the Mahalanobis distance for each path. When the reduction trend value of the Mahalanobis distance for the backup path is greater than that for the primary path, the service level identifier (SBI) of the data stream to be allocated to the backup path or currently being used for auxiliary transmission testing by the backup path is extracted. The SBI value is then substituted into a preset linear mapping function to calculate the probe ratio. The default probe packet transmission frequency of the backup path is multiplied by this probe ratio to increase the detection intensity of changes in the backup path's status.

[0015] In an optional embodiment, establishing concurrent transmission paths using different transmission protocols between the transceiver ends and assigning a corresponding service level identifier to each data stream includes: Parse the media packet header of the data stream to obtain the data type; When the data type is video, a service level identifier with a value of 1 is assigned to the data stream; When the data type is audio, assign a service level identifier with a value of 2 to the data stream; When the data type is text or screen sharing, a service level identifier with a value of 3 is assigned to the data stream. The smaller the value, the higher the priority level.

[0016] When establishing concurrent transmission paths, the transmitting and receiving ends create one transmission path based on Transmission Control Protocol (TCP) and one transmission path based on User Datagram Protocol (UDP), respectively, and bind the two transmission paths to their corresponding network interfaces. Based on the initial path status evaluation results, the system determines the transmission path with better current quality or pre-designated to undertake the main transmission task as the primary path, and determines the other established transmission path that is in a probe state as the backup path. The backup path is not used as the primary data-carrying path in the current period, or only undertakes auxiliary transmission and probe tasks, and is used as a candidate switching path when the primary path quality deteriorates and the switching conditions are met.

[0017] At the intermediate layer interface between the network layer and application layer of the protocol stack at the communication transceiver end, each application layer data packet entering the network buffer is intercepted, and media header fields in the packet are read, such as the Payload Type field under the RTP protocol. When the decimal encoding corresponding to the payload type is between 96 and 127 and the mapping table identifies it as an H.264 or H.265 video stream, the QoS_flag parameter in the current TCP / UDP data stream context data structure is written to the hexadecimal value 0x01, that is, a service level identifier with a value of 1 is assigned, which is used to improve the scheduling priority of this type of data stream during bandwidth contention.

[0018] Meanwhile, if the media packet header is identified as AAC or Opus audio format encoding, the scheduler will write the value 0x02 to the QoS_flag parameter of the data stream context for this session. If the parsing reveals that the data stream is WebSocket-based text chat data or a static screen sharing stream with a frame rate of 5fps or less, a Service Class Identifier (SBI) of value 0x03 is assigned to the data stream. The underlying priority queue processing logic will periodically read the QoS_flag value of the data stream in each queue using a round-robin strategy, following the order that 0x01 takes precedence over 0x02, and 0x02 takes precedence over 0x03, and send data packets to the underlying network card to improve the network transmission stability of real-time services.

[0019] In an optional embodiment, the construction of the path state vector, which includes the bandwidth-delay product, packet loss rate, and smoothed round-trip time, includes: Probe messages are sent to each of the concurrent transmission paths according to a preset detection period; Record the time difference between sending the probe message and receiving the response message, and calculate the smooth round-trip time using an exponentially weighted moving average algorithm; The total number of packets sent and the number of packets lost within a preset time window are counted, and the ratio of the number of lost packets to the total number of packets sent is taken as the packet loss rate. The bandwidth-delay product is obtained by multiplying the smooth round-trip time by the current bottleneck bandwidth of the concurrent transmission path; The bandwidth-delay product, the packet loss rate, and the smooth round-trip time are sequentially stored in a one-dimensional feature matrix, and the feature matrix is ​​used as the path state vector.

[0020] In a multi-protocol concurrent environment, a probe period of 50 to 100 milliseconds is set on each established transmission path. It actively sends ICMP or UDP probe packets with probe sequence numbers and sending timestamps in each cycle, with the payload length limited to 64 bytes to reduce probe overhead.

[0021] After receiving the probe message, the receiving end returns a corresponding response message. The sending end calculates the round-trip time based on the sending time of the probe message and the receiving time of the corresponding response message, and smooths this time difference using an exponentially weighted moving average algorithm, according to the formula... Execution is performed, where the weight parameter α is preferably set to 0.875. In the example, if the previous smoothed value... The value was 30ms, as measured in this study. If the time is 40ms, then the updated smooth round-trip time is 31.25ms.

[0022] In the underlying sliding window protocol statistical mechanism, a preset time window with a sliding length of 1000 milliseconds is maintained. Within the preset time window, the total number of sent packets and the number of lost packets for the corresponding path are counted. For TCP transmission paths, the number of lost packets is determined based on ACK acknowledgment packets, retransmission records, or timeout unacknowledged records. For UDP transmission paths, the number of lost packets is determined based on application layer acknowledgment information, probe response information, or received sequence number acknowledgment returned by the receiver. The ratio of the number of lost packets to the total number of sent packets is used as the link packet loss rate.

[0023] The current bottleneck bandwidth BtlBw of the link is estimated. Specifically, within a preset bandwidth sampling window, the number of data bytes that have been sent and successfully acknowledged is counted, and the time interval corresponding to the completion of acknowledgment for this batch of data is recorded. The number of successfully delivered data bytes is divided by the time interval to obtain the transmission rate sample within the sampling window. A sliding window maximum value filtering process is performed on multiple consecutive transmission rate samples, and the processed transmission rate is taken as the current bottleneck bandwidth BtlBw of the link. For example, when the current bottleneck bandwidth of the link is approximately 10Mbps and the smooth round-trip time is 31.25ms, multiplying the two yields the bandwidth-delay product BDP, which is approximately 39KB. Following the fixed data structure order of BDP, Loss Rate, and SRTT, the parameters of these three dimensions are merged into a 1×3 one-dimensional feature matrix, which serves as the real-time path state vector representing the current path quality.

[0024] In an optional embodiment, calculating the Mahalanobis distance between each of the path state vectors and the preset ideal state vector includes: Obtain the path state vector of the concurrent transmission path extracted at the current moment and the preset ideal state vector; Read the historical path state vector samples of the concurrent transmission path from the cache of the preset time window, calculate the covariance matrix corresponding to the historical path state vector samples, and superimpose the preset regularization coefficient on the main diagonal of the covariance matrix to obtain the regularized covariance matrix and its inverse matrix. A difference vector is generated based on the current path state vector and the preset ideal state vector, and the difference vector is represented in column vector form; The Mahalanobis distance is obtained by sequentially multiplying the transpose of the difference vector, the inverse of the regularized covariance matrix, and the difference vector, and then taking the square root of the product.

[0025] When the path evaluation algorithm module is woken up during the 200-millisecond evaluation period, it reads the one-dimensional feature matrix of the path at the current moment from memory. As a path state vector, its elements correspond to the actual measured BDP, packet loss rate, and SRTT, respectively, and a pre-set ideal state vector is read synchronously. For example, for ultra-low latency live streaming scenarios, the ideal state vector is set to μ=[100KB,0,10ms].

[0026] A circular buffer is maintained in kernel space, recording the entire sequence of historical path state vector samples generated within the past 10 seconds. Assuming 10 samples per second, this results in 100 sample vectors. The processor then calculates the data distribution pattern of these 100 historical sample matrices, solves for the corresponding 3×3 covariance matrix S, and superimposes a regularization coefficient λ on the main diagonal of this covariance matrix S to obtain... λ is a preset constant greater than 0. The inverse matrix of this regularized covariance matrix is ​​further obtained using Gaussian-Jordan elimination or LU decomposition algorithm. The current real-time path state vector X is subtracted from the preset ideal state vector μ by vector subtraction to obtain a difference vector D representing the degree of state deviation. This difference vector D is then used in matrix operations as a 3×1 column vector. The transpose of the difference vector, the inverse of the regularized covariance matrix, and the difference vector are multiplied sequentially to obtain a non-negative scalar value. This value is then square-rooted to obtain a Mahalanobis distance scalar value that represents the deviation of the current path's overall state from the baseline.

[0027] In an optional embodiment, when the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector shows a better decreasing trend than that of the primary path, increasing the detection strength by combining the service level identifier of the data stream to be assigned to the backup path or assisted in the transmission test by the backup path includes: Read the value of the service level identifier corresponding to the data stream to be allocated to the backup path or currently being transmitted for testing with the assistance of the backup path; When the value of the service level identifier is 1, the transmission frequency of the probe message is adjusted from the base probe frequency to the first enhanced probe frequency; When the value of the service level identifier is 2, the transmission frequency of the probe message is adjusted from the base probe frequency to the second enhanced probe frequency; Wherein, the first enhanced detection frequency is higher than the second enhanced detection frequency, and the second enhanced detection frequency is higher than the reference detection frequency; Write an incrementing probe sequence number and a sending timestamp into the header of each probe message.

[0028] When the path detection decision-maker detects that the Mahalanobis distance of the backup path is decreasing compared to the primary path, it intervenes in the probe packet scheduling queue of the network interface card driver layer, extracts and identifies the Service Level Identifier (QoS_flag) in the Flow Control Block context that is preparing to or has been assigned to the backup path for transmission testing.

[0029] When the value is identified as 1, indicating that the backup path is currently or will soon carry core ultra-high-definition video services, the probe scheduler will call a high-precision timer to reduce the active packet sending period interval from the default 50 milliseconds to 16.6 milliseconds, and adjust the probe message sending frequency from the basic 20 times / second to a fixed 60 times / second, in order to identify channel state changes during video transmission gaps.

[0030] When the QoS_flag value is 2, the detector smoothly switches the packet sending interval to 25 milliseconds, effectively fixing the sending frequency to 40 times / second. To combat detection errors caused by network out-of-order delivery and replay attacks, the network interface card's underlying packet encapsulation module extends the header of each 64-byte UDP probe packet with an incrementing probe sequence number and a sending timestamp. When the Service Level Identifier (SLI) value is 3, no enhanced probe frequency adjustment is performed, and the probe packet sending frequency is maintained as the baseline probe frequency. The probe packet sending frequency and probe interval corresponding to the data stream SLI are as follows: Figure 2 As shown.

[0031] S2, by fusing the state vectors of each path using the Kalman filter algorithm, a finite state machine decoupled from the congestion control of each path is maintained to estimate a virtual congestion state representing the overall network status of the session. The virtual congestion state includes a virtual congestion window and congestion momentum. When the Mahalanobis distance of any backup path is continuously less than the product of the Mahalanobis distance of the main path and a preset proportional threshold, and the continuous satisfaction time reaches a preset duration threshold, the finite state machine is triggered to perform a path switching operation.

[0032] A Kalman filter is initialized using a pre-defined Kalman filter procedure, and the state transition matrix, observation matrix, and process noise covariance matrix are set. According to a pre-defined filtering period, the path state vectors of the main path and backup paths are obtained separately, and concatenated in path order to form a multi-path observation vector. After normalization, this vector is used as the observation input to the Kalman filter. The multi-path observation vector is used to generate candidate virtual congestion window observations, which serve as the observations used by the Kalman filter to update the virtual congestion window state.

[0033] The Kalman filter prediction step is invoked to estimate the virtual network state for the current period. Specifically, the state variables of the Kalman filter are set as a virtual congestion window and a congestion momentum; wherein, the virtual congestion window is a transmission window estimated by the session layer, used to represent the amount of data that the current session is allowed to transmit under the overall state of the primary and backup paths, and the congestion momentum is used to represent the changing trend of the virtual congestion window relative to the previous period.

[0034] Within each filtering cycle, the bandwidth-delay product, packet loss rate, and smoothed round-trip time of the primary and backup paths are read. For any path, the packet loss rate of that path is compared with a preset packet loss rate benchmark to obtain a packet loss normalization value, and a packet loss suppression coefficient is determined based on this normalization value. Similarly, the smoothed round-trip time of that path is compared with a preset delay benchmark to obtain a delay normalization value, and a delay suppression coefficient is determined based on this normalization value. A larger packet loss normalization value corresponds to a smaller packet loss suppression coefficient; a larger delay normalization value corresponds to a smaller delay suppression coefficient, and both the packet loss suppression coefficient and the delay suppression coefficient are dimensionless coefficients.

[0035] Subsequently, the bandwidth-delay product of each path is multiplied by the corresponding packet loss suppression coefficient and delay suppression coefficient to obtain the effective bearer window estimate for that path. Then, the effective bearer window estimates for the primary path and backup paths are weighted and summed to obtain candidate virtual congestion window observations. The Kalman filter performs prior predictions of the virtual congestion window and congestion momentum for the current period based on the virtual congestion window and congestion momentum of the previous period, and a preset time step. Then, the observation residual and Kalman gain are calculated using the candidate virtual window observations for the current period, and the prior prediction results are corrected based on the Kalman gain to obtain the posterior virtual congestion state for the current period. The posterior virtual congestion state includes the virtual congestion window and congestion momentum for the current period, where the virtual congestion window is represented in bytes.

[0036] A finite state machine (FSM) is instantiated in memory, comprising probe state, stable state, congestion state, and switching state. The state transition logic of the FSM is independent of the underlying TCP and UDP protocol's own congestion control logic, and is based solely on the virtual congestion window output by the Kalman filter, congestion momentum, and the Mahalanobis distance between the primary and backup paths. When the virtual congestion window continuously decreases and the congestion momentum shows a negative trend, the FSM switches to the congestion state; when the Mahalanobis distance of the backup path is consistently better than that of the primary path and meets a preset duration condition, the FSM switches to the switching state to trigger subsequent path switching and scheduling weight adjustments.

[0037] A separate high-precision timer thread is started in the system background to periodically poll and compare the current Mahalanobis distance of each path. When it is found that the Mahalanobis distance of a backup path is less than the Mahalanobis distance of the main path multiplied by a preset ratio threshold, the current system timestamp is recorded and continuous comparison begins. If this condition is met without interruption and the accumulated time of the system timestamp exceeds a preset duration threshold, the event dispatch interface of the finite state machine is called to trigger a switching event to the finite state machine, notifying the system to switch from the current state to the node performing the path switching operation.

[0038] In an optional embodiment, when the Mahalanobis distance of any of the backup paths is continuously less than the product of the Mahalanobis distance of the primary path and a preset proportional threshold, and the duration of this continuous difference reaches a preset duration threshold, the finite state machine is triggered to perform a path switching operation, including: Start a timer to periodically compare the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector, as well as the Mahalanobis distance between the path state vector of the current main path and the preset ideal state vector. When the Mahalanobis distance of the backup path is continuously less than the product of the Mahalanobis distance of the main path and a preset proportional threshold, and the continuous satisfaction time accumulated on the timer reaches a preset duration threshold, a soft interrupt signal is generated and sent to the processor to trigger a path switching operation. The processor reads the interrupt vector and changes the variable flag of the finite state machine recorded in the system memory from integer 0 to integer 1. It then schedules the newly generated data stream and the existing unconfirmed data stream in transit to the backup path for enqueueing to complete the path switching operation.

[0039] Within the dual-path decoupled transmission architecture, the main controller is equipped with a system-level high-precision timer hrtimer with a time accuracy of 100 milliseconds. Each time this timer overflows and triggers, it reads the latest calculated Mahalanobis distance parameter of the main path from the kernel data dictionary. Mahalanobis distance parameter to alternative path When detected The value is less than When the value is multiplied by a preset ratio threshold, such as 0.8, the local time counter will be incremented by 100 milliseconds. When the condition is continuously met within a smooth time window and the accumulator value reaches a preset duration threshold, such as 2000 milliseconds, the trigger function is called to send a specific soft interrupt signal to the host machine's main CPU to trigger the path switching process.

[0040] Upon receiving the soft interrupt request, the CPU jumps to the corresponding exception handling address, reads the specific interrupt vector table, performs an atomic write operation, overwrites and locks the FSM_State flag representing the finite state machine of the system session in the memory protection page, and updates the value of this flag from the integer 0 representing the stable state to the integer 1 representing the transition state. As the state machine transitions, the packet scheduling gateway stops allocating new enqueue requests to the packet queue corresponding to the original primary path and pushes the new socket data block into the packet queue bound to the backup path; for the existing data fragments that have been sent by the session layer on the old path but have not yet received confirmation from the application layer, the corresponding data fragments are retrieved from the session layer retransmission buffer and re-enqueued into the send buffer queue corresponding to the new primary path according to the encapsulation rules of the new primary path.

[0041] S3, construct a data flow and path weight scheduling matrix. When the virtual congestion state indicates network deterioration, prioritize the transmission weight of high-service-level data flows on the target transmission path. When performing the path switching operation, based on the ratio of the measured bandwidth-delay product of the new primary path and the old primary path, the congestion momentum, and the time step, project the sending window allocated to the old primary path before the switch onto the initial sending window of the new primary path.

[0042] In the data flow scheduling module, memory space is allocated and a two-dimensional floating-point array matrix is ​​initialized as the data flow and path weight scheduling matrix. The row index of this two-dimensional array corresponds to the identifier of all concurrent data flows, and the column index corresponds to the value of the internal element of each established transmission path, which represents the traffic scheduling split ratio.

[0043] When the virtual congestion window decreases over multiple consecutive filtering cycles, and the congestion momentum continuously indicates a negative trend in the virtual congestion window, the current network state is deemed to have deteriorated. At this point, the WFQ (Weighted Fair Queuing) algorithm is activated to scan all data flow objects currently connected and read the service level identifier (SSI) configured for each data flow object. The data flow with the highest priority SSI is matched to the target transmission path with the best overall path quality. In the data flow and path weight scheduling matrix, the array element values ​​corresponding to the intersection of the high-priority data flow and the target transmission path are increased to a first preset transmission weight, while the array element values ​​corresponding to the same data flow on other paths are decreased to a second preset transmission weight, where the first preset transmission weight is greater than the second preset transmission weight. Furthermore, all path weights in the row containing the data flow are normalized so that the sum of the path weights corresponding to the same data flow is 1.

[0044] When a finite state machine node initiates a path switching operation, allowing a qualified backup path to become the new primary path, the bandwidth-delay product parameter measured in real-time by the new primary path using the BBR rate measurement mechanism or application layer rate sampling mechanism at the time of the switching trigger, as well as the bandwidth-delay product parameter of the last successfully recorded data of the old primary path, are extracted. A division operation is performed to obtain the bandwidth-delay product ratio coefficient representing the capacity difference between the old and new links. The currently calculated and maintained congestion momentum value is extracted and multiplied by the system's fixed sampling time step to obtain the trend compensation window variable. The transmission window value maintained by the old primary path in the last round-trip cycle before the switching occurs is multiplied by this bandwidth-delay product ratio coefficient to achieve proportional capacity scaling. The above capacity scaling result is added to the calculated trend compensation window variable as a correction term. The summation result is used as the initial transmission control parameter for the new primary path, controlling the data transmission quota after the new primary path officially starts, thereby reducing window reconstruction overhead during cross-protocol and cross-link switching and improving switching smoothness.

[0045] In an optional embodiment, the construction of the data flow and path weight scheduling matrix, when the virtual congestion state indicates network deterioration, prioritizes the transmission weight of high-service-level data flows on the target transmission path, including: The virtual congestion window value corresponding to the virtual congestion state currently maintained in the memory is detected, and the change in the virtual congestion window between adjacent update cycles is calculated. When the change in the virtual congestion window meets the preset network deterioration judgment condition, a network deterioration trigger signal is generated; Based on the network degradation trigger signal, high service level data streams are filtered, and the target transmission path is determined according to the Mahalanobis distance and smooth round-trip time of each concurrent transmission path. In the data flow and path weight scheduling matrix, the cell corresponding to the high service level data flow and the target transmission path is written into the first preset transmission weight, the cell corresponding to the high service level data flow and the other transmission paths is written into the second preset transmission weight, and the transmission weight of the row where the high service level data flow is located is normalized so that the sum of the normalized path transmission weights is 1. Wherein, the first preset transmission weight is greater than the second preset transmission weight.

[0046] Configure the kernel's scheduling matrix generation module to scan the session-level virtual congestion state parameters residing in a specific memory address pool during each heartbeat cycle (e.g., 200 milliseconds) and extract the currently maintained global virtual congestion window. The specific byte size. This is determined by comparing it with historical values ​​recorded in the previous heartbeat cycle. If the calculation shows that the current virtual congestion window has shrunk and the difference in the decrease is greater than or equal to a preset percentage of the baseline value of the previous period, such as 20%, it is determined that the underlying bearer network has degraded or is facing a buffer queue overflow. At this time, the system's status monitor will generate and output a network deterioration trigger signal representing network degradation through the internal bus.

[0047] After successfully reading the network degradation trigger signal, a traversal mechanism is initiated to scan all flow control context structures in the current active list, filter out data flows that meet the preset high service level conditions, such as core data flows with the service level identifier QoS_flag=1; query the historical cumulative SRTT values ​​of all concurrent links in the last 1000 milliseconds, and combine them with the Mahalanobis distance values ​​of the corresponding paths to select the transmission path with the best overall path quality as the target transmission path.

[0048] To improve the transmission continuity of high-service-level data flows through resource allocation, a two-dimensional flow-path weight scheduling matrix is ​​adjusted. A first preset transmission weight, such as 0.9, is written into the array cell where the row corresponding to the high-service-level data flow intersects with the column corresponding to the target transmission path. A second preset transmission weight, such as 0.1, is written into the array cell where the row intersects with the column corresponding to the non-target transmission path. After writing the initial weights, the weights of the row are normalized. Based on the normalized path weights, the scheduler determines the transmission ratio of the high-service-level data flow between the target transmission path and the non-target transmission path, ensuring that more data packets are sent via the target transmission path with better path quality, thus achieving weighted load balancing scheduling under network degradation conditions.

[0049] In an optional embodiment, when performing the path switching operation, projecting the transmission window allocated to the old primary path before the switch onto the initial transmission window of the new primary path based on the ratio of the measured bandwidth-delay product of the new primary path and the old primary path, the congestion momentum, and the time step includes: The transmit window size allocated to the old primary path and the congestion momentum value representing the transmit window change rate are read from the registers respectively under the session-level virtual congestion state before the switchover. Extract the most recently calculated new primary path bandwidth-delay product, the last recorded old primary path bandwidth-delay product before the switch, and the time step of the congestion state update; Divide the size of the sending window allocated to the old main path by the bandwidth-delay product of the old main path to obtain a quotient value, and multiply the quotient value by the bandwidth-delay product of the new main path to obtain a first product; The second product is obtained by multiplying the preset constant, the congestion momentum value, and the time step. The first product and the second product are added together to obtain a candidate initial sending window, and the candidate initial sending window is truncated to a lower limit according to a preset minimum sending window threshold to obtain the initial sending window. The initial sending window is mapped to the sending control parameters corresponding to the new main path to complete the mapping model projection.

[0050] During the transient transition period of multipath media master-slave switching, a read request is initiated to the system register that manages the global state to extract the parameters locked by the session-level virtual congestion state machine at the critical moment of switching: one is the sending window size allocated by the old master path. The other is the congestion momentum value, which represents the rate or trend of change of the virtual congestion window. Read the latest main path bandwidth-delay product from the traffic sampling table. The bandwidth-delay product of the last recorded old primary path before the path switch occurred. And extract the currently set fixed congestion state update time step Δt.

[0051] Calculate the scaling effect of the channel capacity difference between the two paths, i.e., the sending window size of the old main path. Divide by The quotient representing the degree of window saturation is calculated, and then this quotient is multiplied by the bandwidth capacity of the new link. The first product result used for the reference projection is obtained; then momentum compensation calculation is performed, and the fixed empirical compensation constant 0.5, the previously read congestion momentum value, and the time step Δt are multiplied consecutively to obtain the second product representing the trend prediction; the first product and the second product are combined by addition to obtain the candidate initial sending window value, and the candidate initial sending window value is truncated according to the preset minimum sending window threshold to obtain the initial sending window value on the new path. When the new main path is a TCP transmission path, the initial sending window is mapped to the initial sending window parameter of the TCP congestion controller; specifically, the initial sending window is divided by the maximum segment length of the new main path to obtain the corresponding number of segments, and the number of segments is written into the congestion window variable of the TCP congestion controller.

[0052] When the new primary path is a UDP transmission path, the initial sending window is mapped to application-layer token bucket scheduling parameters. Specifically, bytes are used as the token unit of the token bucket, the initial sending window is used as the initial bucket capacity of the token bucket, the initial sending window is divided by the smooth round-trip time of the new primary path to obtain the token generation rate, and tokens are periodically replenished to the token bucket according to the token generation rate. In each scheduling period, the UDP sending scheduler determines the amount of data allowed to be sent based on the number of available tokens in the token bucket, so that the UDP path starts transmission according to the sending quota corresponding to the initial sending window.

[0053] In the second embodiment, the present invention also proposes a multi-protocol data fusion transmission system for a conference scenario, comprising the following modules: The calculation module is used to establish concurrent transmission paths using different transmission protocols between the transceiver and the receiver, assign a corresponding service level identifier to each data stream; construct a path state vector including bandwidth-delay product, packet loss rate, and smooth round-trip time; calculate the Mahalanobis distance between each path state vector and a preset ideal state vector; when the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector shows a better decreasing trend than that of the main path, the detection intensity is increased by combining the service level identifier of the data stream to be assigned to the backup path or assisted by the backup path for transmission testing. The fusion module is used to fuse the state vectors of each path using a Kalman filter algorithm, maintain a finite state machine decoupled from the congestion control of each path, estimate a virtual congestion state representing the overall network status of the session, the virtual congestion state including a virtual congestion window and congestion momentum; when the Mahalanobis distance of any backup path is continuously less than the product of the Mahalanobis distance of the main path and a preset proportion threshold, and the continuous satisfaction time reaches a preset duration threshold, the finite state machine is triggered to perform a path switching operation. The construction module is used to construct a data flow and path weight scheduling matrix. When the virtual congestion state indicates network deterioration, it prioritizes the transmission weight of high-service-level data flows on the target transmission path. When performing the path switching operation, based on the ratio of the measured bandwidth-delay product of the new primary path and the old primary path, the congestion momentum, and the time step, the sending window allocated to the old primary path before the switch is projected as the initial sending window of the new primary path.

[0054] The above description represents the preferred embodiments of the present invention. It should be noted that, for those skilled in the art, various improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for multi-protocol data fusion and transmission in a conference scenario, characterized in that, include: Concurrent transmission paths using different transmission protocols are established between the transmitting and receiving ends, and a corresponding service level identifier is assigned to each data stream; a path state vector containing bandwidth-delay product, packet loss rate, and smooth round-trip time is constructed; the Mahalanobis distance between each path state vector and a preset ideal state vector is calculated; when the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector shows a better decreasing trend than that of the main path, the detection intensity is increased by combining the service level identifier of the data stream to be assigned to the backup path or assisted by the backup path for transmission testing. By fusing the state vectors of each path using the Kalman filter algorithm, a finite state machine decoupled from the congestion control of each path is maintained to estimate a virtual congestion state representing the overall network status of the session. The virtual congestion state includes a virtual congestion window and congestion momentum. When the Mahalanobis distance of any backup path is continuously less than the product of the Mahalanobis distance of the main path and a preset proportional threshold, and the continuous satisfaction time reaches a preset duration threshold, the finite state machine is triggered to perform a path switching operation. A data flow and path weight scheduling matrix is ​​constructed. When the virtual congestion state indicates network deterioration, priority is given to ensuring the transmission weight of high-service-level data flows on the target transmission path. When performing the path switching operation, based on the ratio of the measured bandwidth-delay product of the new primary path and the old primary path, the congestion momentum, and the time step, the sending window allocated to the old primary path before the switch is projected as the initial sending window of the new primary path.

2. The method according to claim 1, characterized in that, The establishment of concurrent transmission paths using different transmission protocols between the transmitting and receiving ends, and the allocation of a corresponding service level identifier to each data stream, includes: Parse the media packet header of the data stream to obtain the data type; When the data type is video, a service level identifier with a value of 1 is assigned to the data stream; When the data type is audio, assign a service level identifier with a value of 2 to the data stream; When the data type is text or screen sharing, a service level identifier with a value of 3 is assigned to the data stream. The smaller the value, the higher the priority level.

3. The method according to claim 1, characterized in that, The construction of the path state vector, which includes the bandwidth-delay product, packet loss rate, and smooth round-trip time, includes: Probe messages are sent to each of the concurrent transmission paths according to a preset detection period; Record the time difference between sending the probe message and receiving the response message, and calculate the smooth round-trip time using an exponentially weighted moving average algorithm; The total number of packets sent and the number of packets lost within a preset time window are counted, and the ratio of the number of lost packets to the total number of packets sent is taken as the packet loss rate. The bandwidth-delay product is obtained by multiplying the smooth round-trip time by the current bottleneck bandwidth of the concurrent transmission path; The bandwidth-delay product, the packet loss rate, and the smooth round-trip time are sequentially stored in a one-dimensional feature matrix, and the feature matrix is ​​used as the path state vector.

4. The method according to claim 3, characterized in that, The calculation of the Mahalanobis distance between each path state vector and the preset ideal state vector includes: Obtain the path state vector of the concurrent transmission path extracted at the current moment and the preset ideal state vector; Read the historical path state vector samples of the concurrent transmission path from the cache of the preset time window, calculate the covariance matrix corresponding to the historical path state vector samples, and superimpose the preset regularization coefficient on the main diagonal of the covariance matrix to obtain the regularized covariance matrix and its inverse matrix. A difference vector is generated based on the current path state vector and the preset ideal state vector, and the difference vector is represented in column vector form; The Mahalanobis distance is obtained by sequentially multiplying the transpose of the difference vector, the inverse of the regularized covariance matrix, and the difference vector, and then taking the square root of the product.

5. The method according to claim 1, characterized in that, When the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector shows a better decreasing trend than that of the main path, the detection intensity is increased by combining the service level identifier of the data stream to be assigned to the backup path or assisted in the transmission test by the backup path, including: Read the value of the service level identifier corresponding to the data stream to be allocated to the backup path or currently being transmitted for testing with the assistance of the backup path; When the value of the service level identifier is 1, the transmission frequency of the probe message is adjusted from the base probe frequency to the first enhanced probe frequency; When the value of the service level identifier is 2, the transmission frequency of the probe message is adjusted from the base probe frequency to the second enhanced probe frequency; Wherein, the first enhanced detection frequency is higher than the second enhanced detection frequency, and the second enhanced detection frequency is higher than the reference detection frequency; Write an incrementing probe sequence number and a sending timestamp into the header of each probe message.

6. The method according to claim 1, characterized in that, When the Mahalanobis distance of any of the backup paths is continuously less than the product of the Mahalanobis distance of the primary path and a preset proportional threshold, and this condition is met for a continuous period of time reaching a preset duration threshold, the finite state machine is triggered to perform a path switching operation, including: Start a timer to periodically compare the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector, as well as the Mahalanobis distance between the path state vector of the current main path and the preset ideal state vector. When the Mahalanobis distance of the backup path is continuously less than the product of the Mahalanobis distance of the main path and a preset proportional threshold, and the continuous satisfaction time accumulated on the timer reaches a preset duration threshold, a soft interrupt signal is generated and sent to the processor to trigger a path switching operation. The processor reads the interrupt vector and changes the variable flag of the finite state machine recorded in the system memory from integer 0 to integer 1. It then schedules the newly generated data stream and the existing unconfirmed data stream in transit to the backup path for enqueueing to complete the path switching operation.

7. The method according to claim 1, characterized in that, The construction of the data flow and path weight scheduling matrix, when the virtual congestion state indicates network deterioration, prioritizes the transmission weight of high-service-level data flows on the target transmission path, including: The virtual congestion window value corresponding to the virtual congestion state currently maintained in the memory is detected, and the change in the virtual congestion window between adjacent update cycles is calculated. When the change in the virtual congestion window meets the preset network deterioration judgment condition, a network deterioration trigger signal is generated; Based on the network degradation trigger signal, high service level data streams are filtered, and the target transmission path is determined according to the Mahalanobis distance and smooth round-trip time of each concurrent transmission path. In the data flow and path weight scheduling matrix, the cell corresponding to the high service level data flow and the target transmission path is written into the first preset transmission weight, the cell corresponding to the high service level data flow and the other transmission paths is written into the second preset transmission weight, and the transmission weight of the row where the high service level data flow is located is normalized so that the sum of the normalized path transmission weights is 1. Wherein, the first preset transmission weight is greater than the second preset transmission weight.

8. The method according to claim 1, characterized in that, When performing the path switching operation, based on the ratio of the measured bandwidth-delay product of the new primary path and the old primary path, the congestion momentum, and the time step, the sending window allocated to the old primary path before the switch is projected as the initial sending window of the new primary path, including: The transmit window size allocated to the old primary path and the congestion momentum value representing the transmit window change rate are read from the registers respectively under the session-level virtual congestion state before the switchover. Extract the most recently calculated new primary path bandwidth-delay product, the last recorded old primary path bandwidth-delay product before the switch, and the time step of the congestion state update; Divide the size of the sending window allocated to the old primary path by the bandwidth-delay product of the old primary path to obtain a quotient value, and multiply the quotient value by the bandwidth-delay product of the new primary path to obtain a first product; The second product is obtained by multiplying the preset constant, the congestion momentum value, and the time step. The first product and the second product are added together to obtain a candidate initial sending window, and the candidate initial sending window is truncated to a lower limit according to a preset minimum sending window threshold to obtain the initial sending window. The initial sending window is mapped to the sending control parameters corresponding to the new main path to complete the mapping model projection.

9. A multi-protocol data fusion and transmission system for a conference scenario, characterized in that, include: The calculation module is used to establish concurrent transmission paths using different transmission protocols between the transceiver and the receiver, assign a corresponding service level identifier to each data stream; construct a path state vector including bandwidth-delay product, packet loss rate, and smooth round-trip time; calculate the Mahalanobis distance between each path state vector and a preset ideal state vector; when the Mahalanobis distance between the path state vector of the backup path and the preset ideal state vector shows a better decreasing trend than that of the main path, the detection intensity is increased by combining the service level identifier of the data stream to be assigned to the backup path or assisted by the backup path for transmission testing. The fusion module is used to fuse the state vectors of each path using a Kalman filter algorithm, maintain a finite state machine decoupled from the congestion control of each path, estimate a virtual congestion state representing the overall network status of the session, the virtual congestion state including a virtual congestion window and congestion momentum; when the Mahalanobis distance of any backup path is continuously less than the product of the Mahalanobis distance of the main path and a preset proportion threshold, and the continuous satisfaction time reaches a preset duration threshold, the finite state machine is triggered to perform a path switching operation. The construction module is used to construct a data flow and path weight scheduling matrix. When the virtual congestion state indicates network deterioration, it prioritizes the transmission weight of high-service-level data flows on the target transmission path. When performing the path switching operation, based on the ratio of the measured bandwidth-delay product of the new primary path and the old primary path, the congestion momentum, and the time step, the sending window allocated to the old primary path before the switch is projected as the initial sending window of the new primary path.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-8.

Images