A cross-platform audio data communication processing method and system based on multi-protocol support

Abstract

The application discloses a kind of based on multi-protocol support's cross-platform audio data communication processing method and system, it is related to data processing technical field, the method includes: dynamic coding to standardized audio data stream, obtains coding audio packet;Based on coding audio packet, construct multi-communication protocol cluster, according to the transmission capacity of source platform, the compatible adaptation degree of coding audio packet and each communication protocol is calculated, and based on compatible adaptation degree, select optimal communication protocol from multi-communication protocol cluster;According to optimal communication protocol, execute cross-platform transmission to coding audio packet, in transmission process, based on predicted communication link state, trigger communication protocol conversion mechanism, adaptively switch to alternative communication protocol to maintain transmission stability.The present application ensures that it has flexible protocol scheduling capability under the scene of communication switching, network fluctuation or transmission anomaly, significantly improves the adaptability, stability and transmission quality of audio data communication.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a cross-platform audio data communication processing method and system based on multi-protocol support. Background Technology

[0002] Audio data communication processing refers to the entire process of reliably transmitting encoded audio information over a network via digital communication and restoring it into playable voice content. It involves multiple stages such as audio encoding, packetization, protocol selection, transmission path scheduling, error control, and data restoration. Its purpose is to achieve stable transmission of audio data between different devices and network environments while ensuring sound quality and low latency.

[0003] Cross-platform audio data transmission is necessary because modern applications are widely distributed across heterogeneous systems (such as mobile devices, web applications, and embedded devices). These platforms differ in operating systems, protocol stacks, and hardware performance. To achieve seamless audio interaction between platforms, cross-platform adaptation and protocol collaboration must be supported. However, existing technologies cannot select the optimal communication protocol for cross-platform transmission, thus failing to dynamically match the best transmission channel for each platform's network environment. This can easily lead to problems such as audio latency, packet loss, or quality degradation, resulting in the inability to guarantee the continuity and stability of the audio stream under complex network conditions.

[0004] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a cross-platform audio data communication processing method and system based on multi-protocol support. It has the advantage of improving the accuracy and real-time performance of protocol selection in cross-platform audio data transmission by using the optimal communication protocol, thereby solving the problems of not being able to dynamically match the best transmission channel for the network environment of each platform, which easily leads to audio delay, packet loss, or quality degradation.

[0006] To achieve the aforementioned advantages of improving the accuracy and real-time performance of protocol selection in cross-platform audio data transmission through the optimal communication protocol, the specific technical solution adopted in this invention is as follows:

[0007] According to one aspect of the present invention, a cross-platform audio data communication processing method based on multi-protocol support is provided, the method comprising:

[0008] S1. Collect the raw audio data from the source platform and preprocess it to generate a standardized audio data stream. Then, dynamically encode the standardized audio data stream to obtain an encoded audio packet.

[0009] S2. Construct a multi-communication protocol cluster based on the encoded audio packets, calculate the compatibility and adaptability between the encoded audio packets and each communication protocol according to the transmission capability of the source platform, and select the optimal communication protocol from the multi-communication protocol cluster based on the compatibility and adaptability.

[0010] S3. Perform cross-platform transmission of encoded audio packets based on the optimal communication protocol. During transmission, trigger a communication protocol switching mechanism based on the predicted communication link status and adaptively switch to the alternative communication protocol to maintain transmission stability.

[0011] S4. After the encoded audio packet is transmitted, compare the audio data restored by the receiving end with the original audio data, and optimize the communication protocol conversion mechanism based on the comparison result.

[0012] Preferably, a multi-communication protocol cluster is constructed based on the encoded audio packets. The compatibility and adaptability of the encoded audio packets with each communication protocol are calculated according to the transmission capability of the source platform. The optimal communication protocol is selected from the multi-communication protocol cluster based on the compatibility and adaptability, including:

[0013] S21. Generate a communication protocol vector for each communication protocol in the multi-communication protocol cluster; extract the time-frequency domain features of the encoded audio packets, and combine the Mel-frequency cepstral coefficients to extract the audio perception vector of the encoded audio packets from the time-frequency domain features;

[0014] S22. Input the audio perception vector and the communication protocol vector into the dual-channel deep matching model, and analyze the compatibility and adaptation relationship between the encoded audio packet and each communication protocol through the dual-channel deep matching model.

[0015] S23. Use the branch and bound optimization algorithm to search for the optimal solution in the adaptive communication protocol cluster and use it as the optimal communication protocol, and use the suboptimal solution as the alternative communication protocol.

[0016] Preferably, the audio perception vector and the communication protocol vector are input into a dual-channel deep matching model. The dual-channel deep matching model is used to analyze the compatibility and adaptation relationship between the encoded audio packet and each communication protocol, including:

[0017] S221. Input the audio perception vector and the communication protocol vector into the audio feature channel and the protocol feature channel in the dual-channel deep matching model, respectively, and process the audio feature channel and the protocol feature channel through a shared encoder with the same structure.

[0018] S222. Extract the semantic representation of the audio perception vector and the communication protocol vector through the shared encoder, and construct an adaptation relationship judgment mechanism based on the semantic representation to judge the compatibility and adaptation relationship between the encoded audio packet and each communication protocol in the multi-communication protocol cluster.

[0019] S223. Use communication protocols that meet the compatibility and adaptation relationship as the adapted communication protocols, and integrate them to generate an adapted communication protocol cluster.

[0020] S224. Evaluate the response capability of each adapted communication protocol in historical transmission scenarios, and combine the transmission capability of the source platform to comprehensively calculate the compatibility and adaptation degree between the encoded audio packet and the adapted communication protocol.

[0021] Preferably, the adaptation relationship judgment mechanism is constructed based on semantic representation to determine the compatibility and adaptation relationship between the encoded audio packet and each communication protocol in the multi-communication protocol cluster, including:

[0022] Determine whether the semantic representations of the audio perception vector and the communication protocol vector form a corresponding synchronization vector relationship in the historical transmission scenario; if they do, determine that the encoded audio packet and the current communication protocol are directly compatible, and execute step S223; if they do not, execute the next step.

[0023] Based on historical transmission scenarios, predict whether the current communication protocol will be used by the receiving end in the future. If so, determine that the encoded audio packet and the current communication protocol are asynchronously adapted and temporarily store the current communication protocol in the cache candidate set. If not, proceed to the next step.

[0024] The system determines that the encoded audio packet does not meet the compatibility requirements of the current communication protocol and removes the current communication protocol from the multi-communication protocol cluster.

[0025] Preferably, the responsiveness of each adapted communication protocol in historical transmission scenarios is evaluated, and the compatibility between the encoded audio packet and the adapted communication protocol is comprehensively calculated in conjunction with the transmission capabilities of the source platform, including:

[0026] S2241. Each adapted communication protocol is used as a parent capsule in the capsule network, and the response capability score of the parent capsule in historical transmission scenarios is calculated. Weights are assigned to each parent capsule.

[0027] S2242. Map the audio perception vector to the prediction vector of the sub-capsule through a predefined weight matrix, and sum the prediction vectors of the sub-capsule by weight to obtain the sub-capsule weight.

[0028] S2243. Based on the parent capsule weight and child capsule weight, update the routing weight between the parent capsule and child capsule, and use the transmission capacity of the source platform as a control factor to comprehensively calculate the compatibility and adaptation degree between the encoded audio packet and the adapted communication protocol by using a weighted summation method.

[0029] Preferably, the branch-and-bound optimization algorithm is used to search for the optimal solution in the adaptive communication protocol cluster and selects it as the optimal communication protocol, while the suboptimal solution is used as a candidate communication protocol, including:

[0030] S231. Construct the root node of the search tree based on the compatibility and adaptability of the encoded audio packet and the adapted communication protocol, and set the upper and lower bounds of the search tree;

[0031] S232. Select any root node from the search tree as the starting point, expand several branches based on the currently unselected adaptive communication protocol, mark the selected or unselected state of the adaptive communication protocol for each branch, and generate several new child nodes at the same time.

[0032] S233. Based on the new child node, estimate the upper and lower bounds. If the upper bound of the new child node is less than or equal to the current global optimal solution, it means that the current branch cannot produce a better solution and should be pruned.

[0033] S234. Repeat steps S232-S233, iteratively performing expansion, upper and lower bound estimation, and pruning in the search tree until all child nodes have been traversed. Output the optimal solution as the optimal communication protocol and the suboptimal solution as the alternative communication protocol.

[0034] Preferably, cross-platform transmission of the encoded audio packets is performed according to the optimal communication protocol. During transmission, a communication protocol switching mechanism is triggered based on the predicted communication link status to adaptively switch to an alternative communication protocol to maintain transmission stability, including:

[0035] S31. Perform cross-platform transmission of encoded audio packets according to the optimal communication protocol, collect communication link status during transmission, and predict communication link status in the future time period.

[0036] S32. If the communication link status is stable in the future time period, continue to transmit according to the current optimal communication protocol. If the communication link status is delayed in the future time period, use the Markov decision model to evaluate whether the current optimal communication protocol needs to be switched to the alternative communication protocol.

[0037] S33. If the switching benefit of switching from the current optimal communication protocol to the alternative communication protocol is greater than or equal to the switching cost, the communication protocol conversion mechanism is triggered to switch the current optimal communication protocol to the alternative communication protocol. If the switching benefit is less than the switching cost, the encoding parameters of the audio encoding packet are adjusted to adapt to the deteriorated communication link state. When the communication link state is predicted to deteriorate further, the communication protocol conversion mechanism is executed.

[0038] Preferably, triggering the communication protocol switching mechanism to switch the current optimal communication protocol to an alternative communication protocol includes:

[0039] S31. Perform hash matching between the encoding features of the audio encoding packets pre-stored in the database and the protocol features of the candidate communication protocols. If the matching degree is greater than or equal to the preset threshold, it is determined that the conditions for seamless switching of communication protocols are met, and step S32 is executed. If the matching degree is less than the preset threshold, encoding feature adaptation is performed.

[0040] S32. Before switching to the alternative communication protocol, perform a hash transformation on the connection identifier of the optimal communication protocol to generate a communication token, and transmit the lightweight token to the receiving end through an out-of-band channel to reconstruct the state context of the alternative communication protocol.

[0041] S33. Divide the audio data packets into frames to obtain the content hash value, and establish an index mapping table corresponding to the content hash value on the alternative communication protocol end;

[0042] S34. When switching between alternative communication protocols, the sending end uses the hash mapping table to locate the buffer position of the audio encoded packet in the alternative communication protocol in real time.

[0043] Preferably, when switching between alternative communication protocols, after the sending end locates the buffer position of the audio encoded packet in the alternative communication protocol in real time according to the hash mapping table, the method further includes:

[0044] After switching to the alternative communication protocol and entering the new communication path, the content consistency of the audio encoding packets being transmitted is checked based on the periodic hash verification mechanism.

[0045] If the verification fails continuously, it indicates that there is a risk to audio data after the alternative communication protocol is switched. The alternative communication protocol will be reverted to the original optimal transmission protocol, and a suboptimal solution will be selected from the adapted communication protocol cluster as the new alternative communication protocol.

[0046] According to another aspect of the present invention, a cross-platform audio data communication processing system based on multi-protocol support is provided, the system comprising:

[0047] The data acquisition module is used to acquire raw audio data from the source platform, preprocess it, generate a standardized audio data stream, and dynamically encode the standardized audio data stream to obtain an encoded audio packet.

[0048] The compatibility and adaptation calculation module is used to build a multi-communication protocol cluster based on the encoded audio packets. It calculates the compatibility and adaptation degree between the encoded audio packets and each communication protocol according to the transmission capability of the source platform, and selects the optimal communication protocol from the multi-communication protocol cluster based on the compatibility and adaptation degree.

[0049] The communication protocol conversion module is used to perform cross-platform transmission of encoded audio packets according to the optimal communication protocol. During the transmission process, the communication protocol conversion mechanism is triggered based on the predicted communication link status, and the module adaptively switches to the alternative communication protocol to maintain transmission stability.

[0050] The data comparison module is used to compare the audio data restored by the receiving end with the original audio data after the encoded audio packet transmission is completed, and to optimize the communication protocol conversion mechanism based on the comparison result.

[0051] Compared with the prior art, the present invention provides a cross-platform audio data communication processing method and system based on multi-protocol support, which has the following beneficial effects:

[0052] (1) This invention extracts semantic representations through a shared encoder and judges the synchronous or asynchronous adaptation relationship between audio data and communication protocols based on historical transmission scenarios, thereby accurately selecting communication protocols that meet the adaptation conditions. At the same time, it combines the transmission capabilities and historical response characteristics of the source platform to search for the globally optimal communication protocol in the set of adapted communication protocols as the preferred channel, thereby ensuring flexible protocol scheduling capabilities in communication switching, network fluctuations or transmission anomalies, and significantly improving the adaptability, stability and transmission quality of audio data communication.

[0053] (2) The present invention utilizes the branch and bound optimization algorithm to search for the optimal solution in the adaptive communication protocol cluster. By constructing a search tree and setting upper and lower bounds, the adaptive communication protocol is expanded and pruned layer by layer. This can ensure search efficiency while avoiding invalid traversal, thereby quickly converging to the global optimal solution. In this process, not only the optimal communication protocol is obtained, but also the suboptimal solution is retained as an alternative. This not only improves the accuracy and real-time performance of protocol selection in cross-platform audio data transmission, but also provides a reliable backup path in network fluctuation or abnormal scenarios.

[0054] (3) This invention combines link state prediction and protocol adaptive switching mechanism. When the communication link is about to enter an unstable state, the Markov decision model dynamically evaluates the switching benefits and costs and determines whether to trigger the communication protocol conversion mechanism to achieve smooth switching of encoded audio packets between multiple protocols. Through hash matching mechanism and communication token transmission, the protocol state docking is quickly completed. Combined with hash mapping table, the audio data packet buffer position is accurately located to ensure low latency and packet loss-free transmission during protocol switching, thereby significantly improving the stability and service continuity of cross-platform audio data transmission. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1This is a flowchart of a cross-platform audio data communication processing method based on multi-protocol support according to an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of a cross-platform audio data communication processing system based on multi-protocol support according to an embodiment of the present invention;

[0058] Figure 3 This is a flowchart illustrating the analysis of compatibility and adaptation relationships in a cross-platform audio data communication processing method based on multi-protocol support according to an embodiment of the present invention.

[0059] In the picture:

[0060] 1. Data acquisition module; 2. Compatibility and adaptation calculation module; 3. Communication protocol conversion module; 4. Data comparison module. Detailed Implementation

[0061] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0062] According to embodiments of the present invention, a cross-platform audio data communication processing method and system based on multi-protocol support is provided.

[0063] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, according to an embodiment of the present invention, a cross-platform audio data communication processing method based on multi-protocol support is provided, the method comprising:

[0064] S1. Collect the raw audio data from the source platform and preprocess it to generate a standardized audio data stream. Then, dynamically encode the standardized audio data stream to obtain an encoded audio packet.

[0065] It should be noted that the raw audio data from the source platform is collected and preprocessed to generate a standardized audio data stream. This standardized audio data stream is then dynamically encoded to obtain an encoded audio packet, which includes:

[0066] After acquiring raw audio data from the source platform, preprocessing operations such as noise reduction, endpoint detection, and speech enhancement are first performed to improve audio quality. Then, based on the transmission conditions of the target platform (such as bandwidth, latency, and packet loss rate), a suitable audio encoder (such as Opus, AAC, or Speex) and corresponding encoding parameters (such as bitrate, frame length, and compression ratio) are dynamically selected. Finally, the compression encoding process of the standardized audio data stream is completed, and encoded audio packets with a transmittable structure are output for subsequent cross-platform communication.

[0067] S2. Construct a multi-communication protocol cluster based on the encoded audio packets, calculate the compatibility and adaptability between the encoded audio packets and each communication protocol according to the transmission capability of the source platform, and select the optimal communication protocol from the multi-communication protocol cluster based on the compatibility and adaptability.

[0068] This involves constructing a multi-communication protocol cluster based on encoded audio packets, calculating the compatibility between the encoded audio packets and each communication protocol based on the transmission capabilities of the source platform, and selecting the optimal communication protocol from the multi-communication protocol cluster based on the compatibility.

[0069] S21. Generate a communication protocol vector for each communication protocol in the multi-communication protocol cluster; extract the time-frequency domain features of the encoded audio packets, and combine the Mel-frequency cepstral coefficients to extract the audio perception vector of the encoded audio packets from the time-frequency domain features.

[0070] It should be noted that the communication protocol undergoes structural abstraction, extracting its key parameters (such as protocol type, transmission mode, message structure, QoS characteristics, etc.), which are then quantified and mapped to a unified vector space to construct the communication protocol vector. Specifically, the extraction of time-frequency domain features from coded audio packets includes:

[0071] Step 1: Pre-emphasize the original audio signal to improve the signal-to-noise ratio of the high-frequency part, and then perform frame segmentation and windowing (such as using Hamming window) to maintain the local stationarity of the signal; apply Fast Fourier Transform to each frame to convert the time domain signal into a frequency domain signal, obtain the amplitude spectrum and power spectrum of each frame, and extract the spectral envelope, main frequency distribution and overall energy change based on this.

[0072] Step 2: Calculate the short-time energy in each frame as a time-domain feature representing the signal strength, and the zero-crossing rate (i.e., the number of times the signal changes near zero) to reflect the frequency variation trend of the signal. These time-frequency joint features provide basic support for subsequent Mel frequency cepstral coefficient extraction and audio perception vector modeling.

[0073] Step 3: Combine the spectral information with the Mel filter bank processing, calculate the Mel frequency cepstral coefficients, and fuse the time domain and frequency domain information to form a multi-dimensional audio perception vector describing the audio perception characteristics, which is used for subsequent compatibility modeling and matching with communication protocol vectors.

[0074] It should be noted that Mel frequency cepstral coefficients are an audio feature extraction method that mimics the human ear's perception mechanism. By converting the audio signal from the time domain to the frequency domain and mapping it to the Mel scale, and then performing logarithmic compression and discrete cosine transform, low-dimensional feature vectors representing the perceptual characteristics of the speech signal, such as timbre and pitch, are extracted. By compressing high-dimensional spectral information while retaining the key information most relevant to human hearing, subsequent audio recognition, matching, and perceptual modeling are more robust and semantically discriminative.

[0075] S22. Input the audio perception vector and the communication protocol vector into the dual-channel deep matching model, and analyze the compatibility and adaptation relationship between the encoded audio packet and each communication protocol through the dual-channel deep matching model.

[0076] Among them, such as Figure 3 As shown, the audio perception vector and communication protocol vector are input into a dual-channel deep matching model. The dual-channel deep matching model is used to analyze the compatibility and adaptation relationship between the encoded audio packet and each communication protocol, including:

[0077] S221. Input the audio perception vector and the communication protocol vector into the audio feature channel and the protocol feature channel in the dual-channel deep matching model, respectively, and process the audio feature channel and the protocol feature channel through a shared encoder with the same structure.

[0078] It should be noted that the architecture of the dual-channel deep matching model consists of two parallel input channels, which are used to receive audio perception vectors and communication protocol vectors respectively. Each channel contains a shared encoder with the same structure. This encoder typically uses stacked convolutional neural networks (CNN), recurrent neural networks (RNN), or transformer structures for deep semantic abstraction.

[0079] During the input phase, the audio feature channel receives an audio perception vector composed of Mel frequency cepstral coefficients, while the communication protocol channel receives a communication protocol vector generated by protocol parameters. The two channels are processed by an encoder with shared weights to extract high-dimensional feature representations in a unified semantic space.

[0080] The key to a shared encoder lies in using a completely consistent network structure and weight parameters, enabling the model to learn comparable semantic mappings across different modalities. This ensures the effectiveness of feature alignment and matching between audio and protocols. Specifically, achieving a shared encoder using a completely consistent network structure and weight parameters includes:

[0081] During the model design phase, a unified encoder structure (such as a multi-layer convolutional neural network, bidirectional LSTM, or Transformer sub-layer) is defined, and the encoder module is constructed as a single instance. Then, in the model, the same encoder instance is specified as its embedded sub-network for the audio feature channel and the communication protocol feature channel, respectively. This means that the two channels share the same weight matrix and structural path during forward propagation.

[0082] During training, the encoder's parameter gradients are fed back from the joint loss function of the two channels and are updated uniformly through a backpropagation, thus keeping the weights synchronized and non-differentiated. In terms of implementation, this can be achieved through the "weight sharing" mechanism in deep learning frameworks (such as PyTorch or TensorFlow) or by explicitly referencing the same sub-network object. This strategy allows the audio perception vector and the communication protocol vector to be mapped to a consistent feature space.

[0083] S222. Extract the semantic representation of the audio perception vector and the communication protocol vector through the shared encoder, and construct an adaptation relationship judgment mechanism based on the semantic representation to judge the compatibility and adaptation relationship between the encoded audio packet and each communication protocol in the multi-communication protocol cluster.

[0084] The mechanism for determining compatibility between encoded audio packets and each communication protocol in a multi-communication protocol cluster, based on semantic representation, includes:

[0085] Determine whether the semantic representations of the audio perception vector and the communication protocol vector form a corresponding synchronization vector relationship in the historical transmission scenario; if they do, determine that the encoded audio packet and the current communication protocol are directly compatible, and execute step S223; if they do not, execute the next step.

[0086] Based on historical transmission scenarios, predict whether the current communication protocol will be used by the receiving end in the future. If so, determine that the encoded audio packet and the current communication protocol are asynchronously adapted and temporarily store the current communication protocol in the cache candidate set. If not, proceed to the next step.

[0087] The system determines that the encoded audio packet does not meet the compatibility requirements of the current communication protocol and removes the current communication protocol from the multi-communication protocol cluster.

[0088] It should be noted that the communication protocol is screened layer by layer through the adaptation relationship judgment mechanism based on semantic representation. First, it is determined whether the audio perception vector and the communication protocol vector form a synchronous vector relationship in the historical transmission scenario. If they do, it is directly identified as an adapted protocol. If they do not, it is predicted whether the communication protocol will be used by the receiving end in the future based on the historical scenario. If the prediction result is yes, it is marked as asynchronous adaptation and stored in the cache candidate set. If the conditions are still not met, the protocol is removed, and finally a cluster of highly matched adapted communication protocols is obtained.

[0089] The adaptation relationship judgment mechanism realizes multi-level dynamic screening of communication protocols. It not only uses historical data to improve the accuracy of adaptation judgment, but also enhances the system's foresight and adaptability through the prediction mechanism, thereby significantly improving the rationality, real-time performance and robustness of protocol selection in cross-platform audio data transmission.

[0090] S223. Use communication protocols that meet the compatibility and adaptation relationship as the adapted communication protocols, and integrate them to generate an adapted communication protocol cluster.

[0091] S224. Evaluate the response capability of each adapted communication protocol in historical transmission scenarios, and combine the transmission capability of the source platform to comprehensively calculate the compatibility and adaptation degree between the encoded audio packet and the adapted communication protocol.

[0092] The evaluation process includes assessing the responsiveness of each adapted communication protocol in historical transmission scenarios, and combining this with the transmission capabilities of the source platform to comprehensively calculate the compatibility between the encoded audio packet and the adapted communication protocol.

[0093] S2241. Each adapted communication protocol is used as a parent capsule in the capsule network, and the response capability score of the parent capsule in historical transmission scenarios is calculated. Weights are assigned to each parent capsule.

[0094] S2242. Map the audio perception vector to the prediction vector of the sub-capsule through a predefined weight matrix, and sum the prediction vectors of the sub-capsule by weight to obtain the sub-capsule weight.

[0095] S2243. Based on the parent capsule weight and child capsule weight, update the routing weight between the parent capsule and child capsule, and use the transmission capacity of the source platform as a control factor to comprehensively calculate the compatibility and adaptation degree between the encoded audio packet and the adapted communication protocol by using a weighted summation method.

[0096] It should be noted that the following section, in conjunction with specific implementation methods, further explains the evaluation of the response capability of each adapted communication protocol in historical transmission scenarios, and, in conjunction with the transmission capability of the source platform, comprehensively calculates the compatibility and adaptation degree between the encoded audio packet and the adapted communication protocol:

[0097] The response capability score for each adapted communication protocol is calculated based on historical transmission scenarios, and then normalized to obtain the parent capsule weight.

[0098] ;

[0099] In the formula, For the first i The first father capsule (i.e., the first) i The weight of each adapted communication protocol indicates its relative importance in historical scenarios; γ i For the first i The response capability score of each parent capsule is calculated based on a comprehensive analysis of indicators such as latency, packet loss rate, and throughput in historical transmission scenarios. N To accommodate the total number of communication protocols; This indicates that the response capability score is indexed to enhance differentiation. This represents the normalization factor, which makes the sum of the weights of all parent capsules equal to 1.

[0100] The audio perception vector is mapped to a prediction vector through the weight matrix of each sub-capsule, and then weighted and normalized to obtain the sub-capsule weights:

[0101] ;

[0102] In the formula, For the first j The normalized weights of each sub-capsule represent the importance of the audio features in that dimension; Let be the norm of the j-th prediction vector, usually the Euclidean norm, which represents the activation strength of the subcapsule; Indicates the first j The predicted vector of each sub-capsule; M This represents the total number of subcapsules, corresponding to the number of feature dimensions of the audio perception vector; e The base of the natural logarithm; No. l The norm of each prediction vector; Indicates the first l The predicted vector of each sub-capsule, where... j For the target subcapsule index, l This is the index variable in the normalized summation.

[0103] Combining the parent capsule weight, child capsule weight, and the source platform's transmission capacity factor η They jointly update the routing weights between parent and child capsules, ultimately using the fusion formula:

[0104] ;

[0105] In the formula, Score i Indicates the encoded audio packet and the first i The compatibility and adaptability of each communication protocol; η This represents the source platform's transmission capacity adjustment factor, whose weights are dynamically adjusted based on bandwidth, latency, and stability. Indicates the first i Individual capsule weight; Indicates the first j The weight of each sub-capsule reflects the importance of the audio feature dimension; M This indicates the number of subcapsule (i.e., audio feature dimensions); P i Indicates the first i The feature vectors of each parent capsule represent the feature semantics of a certain communication protocol; Represents the first obtained by mapping audio perception vectors. j Predicted vectors for each sub-capsule; Indicates the first iThe father capsule and the first j Cosine similarity between individual capsules measures the degree of semantic matching.

[0106] S23. Use the branch and bound optimization algorithm to search for the optimal solution in the adaptive communication protocol cluster and use it as the optimal communication protocol, and use the suboptimal solution as the alternative communication protocol.

[0107] Among these, the branch-and-bound optimization algorithm is used to search for the optimal solution in the adapted communication protocol cluster and select it as the optimal communication protocol, while the suboptimal solution is used as a candidate communication protocol, including:

[0108] S231. Construct the root node of the search tree based on the compatibility and adaptability of the encoded audio packet and the adapted communication protocol, and set the upper and lower bounds of the search tree;

[0109] S232. Select any root node from the search tree as the starting point, expand several branches based on the currently unselected adaptive communication protocol, mark the selected or unselected state of the adaptive communication protocol for each branch, and generate several new child nodes at the same time.

[0110] S233. Based on the new child node, estimate the upper and lower bounds. If the upper bound of the new child node is less than or equal to the current global optimal solution, it means that the current branch cannot produce a better solution and should be pruned.

[0111] It should be noted that in the branch and bound optimization algorithm, upper and lower bound estimation is the core mechanism for evaluating the optimal potential of the current subproblem (node), where:

[0112] The upper bound estimate represents the maximum possible fitness that the child node can achieve, optimistically estimating its potential. If the upper bound value UBnode is less than or equal to the current fitness score, it means that no matter how the branch is expanded, it will not exceed the current optimal solution, and therefore it can be pruned.

[0113] The lower bound estimate represents the minimum fitness that a child node can guarantee in its current state. It is a conservative estimate and is usually used to determine whether the current optimal solution needs to be updated or to provide search inspiration. If the lower bound of a child node, LBnode, is higher than the fitness score, it means that it is very likely to contain a new, better solution, and the node should be expanded first.

[0114] Upper and lower bounds can usually be quickly estimated using heuristic strategies, for example:

[0115] The upper bound UBnode = current fitness level + estimated sum of the maximum fitness level among the remaining unselected communication protocols.

[0116] The lower bound LBnode = current fitness level + estimated sum of minimum or average fitness level values ​​in the remaining protocols.

[0117] Combining these two valuations allows for more intelligent control of the search path and a rapid approach to the optimal and suboptimal communication protocol selection, thereby improving overall optimization efficiency and matching accuracy.

[0118] S234. Repeat steps S232-S233, iteratively performing expansion, upper and lower bound estimation, and pruning in the search tree until all child nodes have been traversed. Output the optimal solution as the optimal communication protocol and the suboptimal solution as the alternative communication protocol.

[0119] It should be noted that the branch and bound optimization algorithm is used to find the optimal solution in combinatorial optimization problems. Its core idea is to represent the problem as a search tree, generate subproblems through "branches", estimate the optimal probability of each subproblem through "bounding", and prune branches that cannot produce the optimal solution. By introducing the branch and bound optimization algorithm, the compatibility fitness space is systematically searched. It no longer relies on exhaustive traversal or greedy strategies, but uses upper and lower bound estimation and pruning mechanisms to intelligently skip redundant schemes that cannot produce better solutions, thereby significantly improving the search efficiency of the optimal communication protocol. At the same time, it retains the suboptimal solution as an alternative communication protocol to enhance the robustness and fault tolerance of the system. It can quickly switch under dynamic network conditions or changes in the receiver protocol to ensure that the encoded audio packet always matches the most suitable communication protocol to obtain the best transmission performance.

[0120] S3. Perform cross-platform transmission of encoded audio packets based on the optimal communication protocol. During transmission, trigger a communication protocol switching mechanism based on the predicted communication link status, and adaptively switch to the alternative communication protocol to maintain transmission stability.

[0121] Among these measures, cross-platform transmission of encoded audio packets is performed based on the optimal communication protocol. During transmission, a communication protocol switching mechanism is triggered based on the predicted communication link status to adaptively switch to an alternative communication protocol to maintain transmission stability. This includes:

[0122] S31. Perform cross-platform transmission of encoded audio packets according to the optimal communication protocol, collect communication link status during transmission, and predict communication link status in the future time period.

[0123] S32. If the communication link status is stable in the future time period, continue transmission according to the current optimal communication protocol. If the communication link status is delayed in the future time period, use the Markov decision model to evaluate whether the current optimal communication protocol needs to be switched to the alternative communication protocol.

[0124] It should be noted that Markov decision models are used to describe how an agent chooses the optimal action based on the current state to obtain the maximum long-term benefit in an environment with stochastic decision-making behavior. The principle is based on the Markov property, that is, "the information of the current state is sufficient to determine the distribution of future states." In each decision step, the model uses a quadruple (...) S , A , P , R Characterization: S It is a set of states. A It is a set of actions. P It is the state transition probability function. R It is an immediate reward function, where the agent rewards based on the current state. s ∈ S Select Action a ∈ A With probability P ( s ′∣ s , a Transition to the next state s And receive a reward R ( s , a ), through the policy function π ( a | s To plan the optimal behavioral path and maximize the cumulative expected benefits (such as total discount rewards), the Markov decision model is used to quantitatively evaluate the long-term benefits and immediate costs of protocol switching, so as to achieve adaptive optimal selection of communication protocols in dynamic network environments, which avoids the additional overhead caused by frequent switching and can respond to network state changes in a timely manner.

[0125] S33. If the switching benefit of switching from the current optimal communication protocol to the alternative communication protocol is greater than or equal to the switching cost, the communication protocol conversion mechanism is triggered to switch the current optimal communication protocol to the alternative communication protocol. If the switching benefit is less than the switching cost, the encoding parameters of the audio encoding packet are adjusted to adapt to the deteriorated communication link state. When the communication link state is predicted to deteriorate further, the communication protocol conversion mechanism is executed.

[0126] The mechanism that triggers the communication protocol switching to switch the current optimal communication protocol to an alternative communication protocol includes:

[0127] The encoding features of the audio encoding packets pre-stored in the database are hash-matched with the protocol features of the candidate communication protocols. If the matching degree is greater than or equal to a preset threshold, it is determined that the conditions for seamless switching of communication protocols are met, and step S32 is executed. If the matching degree is less than the preset threshold, encoding feature adaptation is performed.

[0128] Before switching to the alternative communication protocol, the connection identifier of the optimal communication protocol is hashed to generate a communication token. The lightweight token is then transmitted to the receiving end through an out-of-band channel to reconstruct the state context of the alternative communication protocol.

[0129] The audio data packets are divided into frames to obtain content hash values, and an index mapping table corresponding to the content hash values ​​is established on the alternative communication protocol side.

[0130] When switching between alternative communication protocols, the sending end uses a hash mapping table to locate the buffer position of the audio encoded packet in the alternative communication protocol in real time.

[0131] After switching to the alternative communication protocol and entering the new communication path, the content consistency of the audio encoding packets being transmitted is checked based on the periodic hash verification mechanism.

[0132] If the verification fails continuously, it indicates that there is a risk to audio data after the alternative communication protocol is switched. The alternative communication protocol will be reverted to the original optimal transmission protocol, and a suboptimal solution will be selected from the adapted communication protocol cluster as the new alternative communication protocol.

[0133] It should be noted that a communication token is generated by hashing the connection identifier of the current optimal communication protocol. This lightweight token is then transmitted to the receiving end via an out-of-band channel to quickly reconstruct the state context of the alternative communication protocol before the switch, ensuring that the switch does not cause an interruption.

[0134] The audio data packets are then divided into frames and content hash values ​​are generated. A mapping index table between frame hash values ​​and buffer locations of alternative protocols is established, laying the foundation for locating audio data in the future. When a protocol switch occurs, the sending end can quickly locate the buffer area of ​​the current audio frame in the alternative protocol according to the hash mapping table, avoiding data retransmission and playback delay.

[0135] After the switch, the system continuously checks the content consistency of the audio packets being transmitted through a periodic hash verification mechanism to ensure that the audio data maintains semantic integrity in the new communication path. If the verification fails continuously, it indicates that there may be data misalignment or protocol adaptation anomaly. The system will automatically fall back to the original optimal protocol and reselect the suboptimal solution from the adaptation cluster as the new alternative protocol, and re-execute the alternative communication protocol switching process.

[0136] S4. After the encoded audio packet is transmitted, compare the audio data restored by the receiving end with the original audio data, and optimize the communication protocol conversion mechanism based on the comparison result.

[0137] It should be noted that after the encoded audio packet transmission is complete, the audio data reconstructed at the receiving end is compared with the original audio data, and the communication protocol conversion mechanism is optimized based on the comparison result, including:

[0138] The audio data restored by the receiving end is compared frame-by-frame or content hash-by-content with the original audio data to accurately evaluate the restoration accuracy of the communication protocol in the actual transmission process. Secondly, based on the comparison results, the error rate, content offset or data loss are statistically analyzed to evaluate the stability and robustness of the current protocol under the actual transmission link. Finally, the evaluation parameters in the communication protocol conversion mechanism (such as switching threshold, benefit calculation function, link fault tolerance threshold) are corrected in reverse by combining error indicators, thereby achieving dynamic optimization.

[0139] According to another embodiment of the invention, such as Figure 2 As shown, a cross-platform audio data communication processing system based on multi-protocol support is also provided, the system comprising:

[0140] Data acquisition module 1 is used to acquire raw audio data from the source platform and preprocess it to generate a standardized audio data stream, and dynamically encode the standardized audio data stream to obtain an encoded audio packet;

[0141] The compatibility and adaptation calculation module 2 is used to build a multi-communication protocol cluster based on the encoded audio packets, calculate the compatibility and adaptation degree between the encoded audio packets and each communication protocol according to the transmission capability of the source platform, and select the optimal communication protocol from the multi-communication protocol cluster based on the compatibility and adaptation degree.

[0142] The communication protocol conversion module 3 is used to perform cross-platform transmission of encoded audio packets according to the optimal communication protocol. During the transmission process, the communication protocol conversion mechanism is triggered based on the predicted communication link status, and the communication protocol is adaptively switched to the alternative communication protocol to maintain transmission stability.

[0143] The data comparison module 4 is used to compare the audio data restored by the receiving end with the original audio data after the encoded audio packet transmission is completed, and to optimize the communication protocol conversion mechanism based on the comparison result.

[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cross-platform audio data communication processing method based on multi-protocol support, characterized in that, The method includes: S1. Collect the raw audio data from the source platform and preprocess it to generate a standardized audio data stream. Then, dynamically encode the standardized audio data stream to obtain an encoded audio packet. S2. Construct a multi-communication protocol cluster based on encoded audio packets. Calculate the compatibility between the encoded audio packets and each communication protocol based on the transmission capabilities of the source platform. Then, select the optimal communication protocol from the multi-communication protocol cluster based on the compatibility. This specifically includes: S21. Generate a communication protocol vector for each communication protocol in the multi-communication protocol cluster; extract the time-frequency domain features of the encoded audio packets, and combine the Mel-frequency cepstral coefficients to extract the audio perception vector of the encoded audio packets from the time-frequency domain features; S22. Input the audio perception vector and the communication protocol vector into the dual-channel deep matching model, and analyze the compatibility and adaptation relationship between the encoded audio packet and each communication protocol through the dual-channel deep matching model, including: S221. Input the audio perception vector and the communication protocol vector into the audio feature channel and the protocol feature channel in the dual-channel deep matching model, respectively, and process the audio feature channel and the protocol feature channel through a shared encoder with the same structure. S222. Extract the semantic representation of the audio perception vector and the communication protocol vector through the shared encoder, and construct an adaptation relationship judgment mechanism based on the semantic representation to judge the compatibility and adaptation relationship between the encoded audio packet and each communication protocol in the multi-communication protocol cluster. S223. Use communication protocols that meet the compatibility and adaptation relationship as the adapted communication protocols, and integrate them to generate an adapted communication protocol cluster. S224. Evaluate the response capability of each adapted communication protocol in historical transmission scenarios, and combine the transmission capability of the source platform to comprehensively calculate the compatibility and adaptation degree between the encoded audio packet and the adapted communication protocol. S23. Use the branch and bound optimization algorithm to search for the optimal solution in the adaptive communication protocol cluster and use it as the optimal communication protocol, and use the suboptimal solution as the candidate communication protocol. S3. Perform cross-platform transmission of encoded audio packets based on the optimal communication protocol. During transmission, trigger a communication protocol switching mechanism based on the predicted communication link status and adaptively switch to the alternative communication protocol to maintain transmission stability. S4. After the encoded audio packet is transmitted, compare the audio data restored by the receiving end with the original audio data, and optimize the communication protocol conversion mechanism based on the comparison result.

2. The cross-platform audio data communication processing method based on multi-protocol support according to claim 1, characterized in that, The mechanism for determining compatibility between the encoded audio packet and each communication protocol in the multi-communication protocol cluster, based on semantic representation, includes: Determine whether the semantic representations of the audio perception vector and the communication protocol vector form a corresponding synchronization vector relationship in the historical transmission scenario; if they do, determine that the encoded audio packet and the current communication protocol are directly compatible, and execute step S223; if they do not, execute the next step. Based on historical transmission scenarios, predict whether the current communication protocol will be used by the receiving end in the future. If so, determine that the encoded audio packet and the current communication protocol are asynchronously adapted and temporarily store the current communication protocol in the cache candidate set. If not, proceed to the next step. The system determines that the encoded audio packet does not meet the compatibility requirements of the current communication protocol and removes the current communication protocol from the multi-communication protocol cluster.

3. The cross-platform audio data communication processing method based on multi-protocol support according to claim 2, characterized in that, The evaluation of the response capability of each adapted communication protocol in historical transmission scenarios, combined with the transmission capability of the source platform, and the comprehensive calculation of the compatibility between the encoded audio packet and the adapted communication protocol, includes: S2241. Each adapted communication protocol is used as a parent capsule in the capsule network, and the response capability score of the parent capsule in historical transmission scenarios is calculated. Weights are assigned to each parent capsule. S2242. Map the audio perception vector to the prediction vector of the sub-capsule through a predefined weight matrix, and sum the prediction vectors of the sub-capsule by weight to obtain the sub-capsule weight. S2243. Based on the parent capsule weight and child capsule weight, update the routing weight between the parent capsule and child capsule, and use the transmission capacity of the source platform as a control factor to comprehensively calculate the compatibility and adaptation degree between the encoded audio packet and the adapted communication protocol by using a weighted summation method.

4. The cross-platform audio data communication processing method based on multi-protocol support according to claim 3, characterized in that, The step of using the branch-and-bound optimization algorithm to search for the optimal solution in the adaptive communication protocol cluster and using it as the optimal communication protocol, while using the suboptimal solution as a candidate communication protocol, includes: S231. Construct the root node of the search tree based on the compatibility and adaptability of the encoded audio packet and the adapted communication protocol, and set the upper and lower bounds of the search tree; S232. Select any root node from the search tree as the starting point, expand several branches based on the currently unselected adaptive communication protocol, mark the selected or unselected state of the adaptive communication protocol for each branch, and generate several new child nodes at the same time. S233. Based on the new child node, estimate the upper and lower bounds. If the upper bound of the new child node is less than or equal to the current global optimal solution, it means that the current branch cannot produce a better solution and should be pruned. S234. Repeat steps S232-S233, iteratively performing expansion, upper and lower bound estimation, and pruning in the search tree until all child nodes have been traversed. Output the optimal solution as the optimal communication protocol and the suboptimal solution as the alternative communication protocol.

5. The cross-platform audio data communication processing method based on multi-protocol support according to claim 1, characterized in that, The method of performing cross-platform transmission of encoded audio packets according to the optimal communication protocol, and triggering a communication protocol switching mechanism based on the predicted communication link status during transmission to adaptively switch to an alternative communication protocol to maintain transmission stability, includes: S31. Perform cross-platform transmission of encoded audio packets according to the optimal communication protocol, collect communication link status during transmission, and predict communication link status in the future time period. S32. If the communication link status is stable in the future time period, continue to transmit according to the current optimal communication protocol. If the communication link status is delayed in the future time period, use the Markov decision model to evaluate whether the current optimal communication protocol needs to be switched to the alternative communication protocol. S33. If the switching benefit of switching from the current optimal communication protocol to the alternative communication protocol is greater than or equal to the switching cost, the communication protocol conversion mechanism is triggered to switch the current optimal communication protocol to the alternative communication protocol. If the switching benefit is less than the switching cost, the encoding parameters of the audio encoding packet are adjusted to adapt to the deteriorated communication link state. When the communication link state is predicted to deteriorate further, the communication protocol conversion mechanism is executed.

6. The cross-platform audio data communication processing method based on multi-protocol support according to claim 5, characterized in that, The triggering communication protocol switching mechanism switches the current optimal communication protocol to an alternative communication protocol, including: The encoding features of the audio encoding packets pre-stored in the database are hash-matched with the protocol features of the candidate communication protocols. If the matching degree is greater than or equal to a preset threshold, it is determined that the conditions for seamless switching of communication protocols are met, and step S32 is executed. If the matching degree is less than the preset threshold, encoding feature adaptation is performed. Before switching to the alternative communication protocol, the connection identifier of the optimal communication protocol is hashed to generate a communication token. The lightweight token is then transmitted to the receiving end through an out-of-band channel to reconstruct the state context of the alternative communication protocol. The audio data packets are divided into frames to obtain content hash values, and an index mapping table corresponding to the content hash values ​​is established on the alternative communication protocol side. When switching between alternative communication protocols, the sending end uses a hash mapping table to locate the buffer position of the audio encoded packet in the alternative communication protocol in real time.

7. The cross-platform audio data communication processing method based on multi-protocol support according to claim 6, characterized in that, When switching between alternative communication protocols, the sending end locates the buffer position of the audio encoded packet in the alternative communication protocol in real time according to the hash mapping table, and the process further includes: After switching to the alternative communication protocol and entering the new communication path, the content consistency of the audio encoding packets being transmitted is checked based on the periodic hash verification mechanism. If the verification fails continuously, it indicates that there is a risk to audio data after the alternative communication protocol is switched. The alternative communication protocol will be reverted to the original optimal transmission protocol, and a suboptimal solution will be selected from the adapted communication protocol cluster as the new alternative communication protocol.

8. A cross-platform audio data communication processing system based on multi-protocol support, used to implement the cross-platform audio data communication processing method based on multi-protocol support as described in any one of claims 1-7, characterized in that, The system includes: The data acquisition module is used to acquire raw audio data from the source platform, preprocess it, generate a standardized audio data stream, and dynamically encode the standardized audio data stream to obtain an encoded audio packet. The compatibility and adaptation calculation module is used to build a multi-communication protocol cluster based on the encoded audio packets. It calculates the compatibility and adaptation degree between the encoded audio packets and each communication protocol according to the transmission capability of the source platform, and selects the optimal communication protocol from the multi-communication protocol cluster based on the compatibility and adaptation degree. The communication protocol conversion module is used to perform cross-platform transmission of encoded audio packets according to the optimal communication protocol. During the transmission process, the communication protocol conversion mechanism is triggered based on the predicted communication link status, and the module adaptively switches to the alternative communication protocol to maintain transmission stability. The data comparison module is used to compare the audio data restored by the receiving end with the original audio data after the encoded audio packet transmission is completed, and to optimize the communication protocol conversion mechanism based on the comparison result.

Images