Audio unvarnished transmission method and system and readable storage medium

Abstract

The invention discloses an audio unvarnished transmission method, system and device and a readable storage medium, and the method comprises the steps that a cloud host receives a data transmission request sent by a virtual earphone, and determines a transmission type and a transmission direction; if the data transmission request is of an isochronous transmission type and the transmission direction is output, inserting the data transmission request into a local timing completion queue, and extracting audio data from the data transmission request; executing a completion response on the request in the timing completion queue according to the target rate; and sending the audio data to the cloud host client so as to play the audio data by using a physical earphone of the cloud host client. In the application, the processing and response processes of the audio data and the data transmission request of the isochronous transmission type are separated, the working link can be effectively shortened, and the high timeliness of isochronous transmission can be ensured, so that the stable transmission of the audio data is ensured, and the functional characteristics of the USB earphone of the cloud host client can be brought into full play in a cloud computer scene.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to an audio transparent transmission method, system and readable storage medium. Background Art

[0002] Cloud computers share and virtualize computing resources through the network, allowing users to access powerful remote computing resources through lightweight terminal devices. However, in terms of supporting high-end USB headset devices, if a fixed-configuration virtual sound card driver is used, and the client interacts with the physical device through the system API for audio streams, it is difficult to fully utilize the functional characteristics of local high-end USB headsets.

[0003] Through the USB device transparent transmission technology, the local USB headset physical device is completely transparently transmitted to the cloud computer streaming host, and the corresponding virtual USB headset device is generated. In this way, at the theoretical level, the functional characteristics of the local high-end USB headset can be effectively exerted. However, in practical applications, there is indeed a delay problem that is not easy to ignore. Specifically, when the host sends a URB request of the isochronous transmission type to the virtual USB headset device to the client, the client needs to process the audio data first, and after determining that the audio data processing is completed, it will request a response to the cloud host. This causes the URB request to be processed, resulting in problems such as request response delays due to the long working link, which in turn causes the system to determine that the isochronous transmission is abnormal and reset the audio endpoint of the headset device, causing the interruption of audio data transmission, which will seriously affect the user experience of the client's physical USB headset device.

[0004] In summary, how to effectively ensure that the USB headset of the client fully exerts its functional characteristics in the cloud computer scenario is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] The purpose of this application is to provide an audio transparent transmission method, system and readable storage medium, which can fully utilize the functional characteristics of the client's USB headset in a cloud computer scenario.

[0006] In order to solve the above technical problems, this application provides the following technical solutions:

[0007] An audio transparent transmission method, applied to a cloud host, includes:

[0008] Receive a data transmission request sent by the virtual headset, and determine the transmission type and transmission direction;

[0009] If the data transmission request is of isochronous transmission type and the transmission direction is output, inserting the data transmission request into a local timing completion queue and extracting audio data from the data transmission request;

[0010] For the requests in the timed completion queue, executing completion responses according to the target rate;

[0011] The audio data is sent to the cloud host client so that the audio data can be played using a physical headset of the cloud host client.

[0012] Preferably, executing completion responses for requests in the timed completion queue at a target rate includes:

[0013] Determining a delayed procedure call timer using the target rate;

[0014] For the requests in the timing completion queue, the delayed procedure call timer is used to complete the response.

[0015] Preferably, it also includes:

[0016] Receiving a rate adjustment notification sent by the cloud host client in the event of an abnormal audio data processing situation;

[0017] The target rate is adjusted to maintain the playback quality of the audio data.

[0018] An audio transparent transmission method, applied to a cloud host client, includes:

[0019] Receive the audio data sent by the cloud host after executing the above-mentioned audio transparent transmission method, and insert the audio data into the tail of the audio data queue for buffering;

[0020] Obtaining processing state statistical information, and adjusting the audio data queue using the processing state statistical information;

[0021] Taking out audio data for single transmission from the audio data queue, and constructing an isochronous transmission request corresponding to the local system using the taken out audio data;

[0022] The isochronous transmission request is sent to the physical headset to play the audio data using the physical headset.

[0023] Preferably, it also includes:

[0024] Using the processing status statistical information, determining whether the audio data processing is abnormal;

[0025] If so, a rate adjustment notification is sent to the cloud host to maintain the playback quality of the audio data.

[0026] Preferably, obtaining processing status statistical information includes:

[0027] Get the evaluation parameters corresponding to the current audio pass-through;

[0028] The evaluation parameter is determined as the processing status statistical information.

[0029] Preferably, obtaining the evaluation parameters corresponding to the current audio transparent transmission includes:

[0030] Based on the processing strategy and processing progress of the received audio data, updating statistical parameters;

[0031] In combination with the calibration strategy and the statistical parameters, the state parameters and the configuration parameters are updated;

[0032] The updated statistical parameters, state parameters and configuration parameters are determined as the evaluation parameters.

[0033] Preferably, adjusting the audio data queue using the processing state statistical information includes:

[0034] Determining whether a condition for discarding audio data is met by using the processing status statistical information;

[0035] If yes, determining the discard quantity using the processing status statistical information;

[0036] The audio data matching the discard quantity is read from the head of the audio data queue, and the exclusive audio data is discarded.

[0037] An audio transparent transmission system, comprising:

[0038] A cloud host and a cloud host client; the cloud host and the cloud host client are in communication connection;

[0039] The cloud host is used to execute the steps of the above-mentioned audio transparent transmission method,

[0040] The cloud host client is used to execute the steps of the above-mentioned audio transparent transmission method.

[0041] A readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned audio transparent transmission method are implemented.

[0042] Apply the method provided in the embodiment of the present application to receive a data transmission request sent by a virtual headset and determine the transmission type and transmission direction; if the data transmission request is an isochronous transmission type and the transmission direction is output, insert the data transmission request into a local timed completion queue and extract audio data from the data transmission request; for the requests in the timed completion queue, execute a completion response at a target rate; and send the audio data to a cloud host client so that the audio data can be played using the physical headset of the cloud host client.

[0043] In this application, after the cloud host obtains the data transmission request sent by the virtual headset, it first determines its transmission type and transmission direction. When it is clear that the data transmission request is of isochronous transmission type and the transmission direction is output, the data transmission request can be directly inserted into the local timed completion queue. At the same time, the audio data is extracted from the data transmission request. For the requests in the timed completion queue, the completion response is executed at the target rate. For the extracted audio data, it is sent directly to the cloud host client so that the audio data can be played using the physical headset of the cloud host client.

[0044] That is to say, when the cloud host receives the data transmission request sent by the virtual headset, it first determines whether it is an isochronous transmission type and whether the transmission direction of its data transmission request is output. If it is an isochronous transmission type and the transmission of the data transmission request is output, it indicates that the data transmission request carries audio data that needs to be transmitted to the cloud host client side for playback. In order to avoid problems such as request response delay caused by too long working links, in this application, the data transmission request is directly inserted into the local timed completion queue, and the response processing is completed at the target rate. For the audio data that needs to be transmitted to the cloud host client, it is directly extracted and sent to the cloud host client so that the audio data can be played based on the physical headset of the cloud host client. That is, separating the processing and response process of the audio data from the data transmission request of the isochronous transmission type can effectively shorten the working link, ensure the high timeliness of the isochronous transmission, and thus ensure the stable transmission of the audio data, so that the USB headset of the client can give full play to its functional characteristics in the cloud computer scenario.

[0045] Correspondingly, the embodiments of the present application also provide an audio transparent transmission system and a readable storage medium corresponding to the above-mentioned audio transparent transmission method, which have the above-mentioned technical effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 This is a flowchart of an implementation method of an audio transparent transmission method in an embodiment of the present application;

[0048] Figure 2 This is a flowchart of another audio transparent transmission method in an embodiment of the present application;

[0049] Figure 3This is a specific implementation flow chart of the audio transparent transmission method in the embodiment of the present application;

[0050] Figure 4 This is a structural diagram of an audio transparent transmission system in an embodiment of the present application;

[0051] Figure 5 This is a schematic diagram of the specific structure of an audio transparent transmission system in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0053] Please refer to Figure 1 , Figure 1 This is a flow chart of an audio transparent transmission method in an embodiment of the present application. The method can be applied to a cloud host. The method includes the following steps:

[0054] S101, receiving a data transmission request sent by a virtual headset, and determining a transmission type and a transmission direction.

[0055] In this embodiment, the local USB headset physical device of the cloud host client can be completely transparently transmitted to the cloud computer streaming host through the USB device transparent transmission technology, and a corresponding virtual USB headset device can be generated, and the virtual USB headset device can be used as the default audio playback and sound recording device. For the convenience of description, in this article, the local USB headset physical device of the cloud host client is referred to as a physical headset, and the virtual USB headset device on the cloud host side is referred to as a virtual headset.

[0056] The data transmission request may specifically be a URB request. The URB (USB Request Block) request is a data structure implemented in the USB driver in the system kernel, and is used to organize each data transmission request of the USB device driver.

[0057] When users use the cloud host, such as opening games, music, videos and other applications that generate audio output, the system will send the audio data to the virtual USB headset device in the form of URB requests.

[0058] After receiving the URB request, the cloud host can determine whether it is an isochronous transfer type URB by checking whether Urb->UrbHeader.Function is equal to URB_FUNCTION_ISOCH_TRANSFER. If Urb->UrbHeader.Function is equal to URB_FUNCTION_ISOCH_TRANSFER, it indicates that the URB request is an isochronous transfer type.

[0059] If it is an isochronous transfer type, then continue to determine whether Urb->TransferFlags & USBD_TRANSFER_DIRECTION_IN is equal to 0, and determine whether the transmission direction of the current operating device is OUT (output). Among them, Urb->TransferFlags & USBD_TRANSFER_DIRECTION_IN is equal to 0, that is, the corresponding transmission direction is output. OUT means that the audio data needs to be transmitted to the physical headset, which is the endpoint of the speaker interface; IN means that data needs to be obtained from the headset, which is the endpoint of the microphone interface.

[0060] S102: If the data transmission request is of isochronous transmission type and the transmission direction is output, insert the data transmission request into a local timing completion queue, and extract audio data from the data transmission request.

[0061] When it is clear that the data transmission request is of isochronous transmission type and the transmission direction is output, the data transmission request can be directly inserted into the local timing completion queue, and the audio data can be extracted from the data transmission request.

[0062] It should be noted that in this embodiment, inserting the data transmission request into the local timed completion queue and extracting the audio data from the data transmission request are independent of each other and are not limited to a certain order. In practical applications, they can be executed in parallel to reduce latency.

[0063] The audio data includes audio content data and description information of the audio content data.

[0064] S103: Execute completion responses for requests in the timed completion queue at a target rate.

[0065] In the embodiment of the present application, for the request in the timed completion object, the completion response can be directly executed according to the target rate.

[0066] It should be noted that, generally, the completion response of a URB request requires that the audio data corresponding to the request be processed before the request response is processed. In this embodiment, the request in the timed completion queue is completed directly on the cloud host without waiting for the audio data to be processed.

[0067] In a specific implementation of the present application, for requests in the timed completion queue, executing a completion response at a target rate includes:

[0068] Using the target rate, determine the delay process call timer;

[0069] For requests in the timed completion queue, a delayed procedure call timer is used to complete the response.

[0070] For ease of description, the above steps are combined for explanation below.

[0071] The response completion process of the URB request may be specifically as follows: first record the current request reception time, determine whether there is a set DPC (Deferred Procedure Call) timer, if not, create and set the DPC timer to complete the URB request, and mark the existing DPC timer, the trigger time T1 defaults to the default value. The default value may be the standard completion time of a single request minus the default adjustment value, which may be set based on the problem of insufficient timer precision.

[0072] For example, the trigger time T1 may be set to 19520 us, which is obtained by subtracting the default adjustment value of 480 us from the standard completion time of a single request of 20000 us. The source of this time is based on the analysis of the URB sending method of the system isochronous transmission type. Specifically, when audio data is sent to the virtual USB headset device, the system will first continuously send two URBs containing 10ms audio data. When a request is completed, it will immediately continue to send a URB request containing 10ms audio data, so that there are always two URB requests being processed, so that there will be no idle interval when the audio is transmitted to the headset. It can be concluded that the interval between each completed URB and the previous one must be kept at 10ms, and the completion time of a single URB request is kept at 20ms, that is, 20000us. The reduction of 480us is due to the insufficient timer accuracy and other time-consuming execution processing problems in the Windows system. It can be set through the system function ExSetTimerResolution. The reduction value is a relatively stable initial value that has been tested. Under normal circumstances, 100 URB requests will be sent and completed in 1s, otherwise too slow or too slow completion will trigger the system's abnormal judgment, thereby performing ABORT and RESET operations on the transmission endpoint, resulting in transmission interruption and destroying the continuous integrity of audio playback. If it exists, the URB request is inserted into the end of the DPC task queue and processed by the set DPC timer callback.

[0073] Then, the DPC timer callback is triggered. First, the current URB request is completed with the default success status S. Then, it is determined whether the DPC task queue is empty. If it is empty, it is marked that there is no set DPC timer. Otherwise, a new URB request is taken from the head of the DPC task queue, and the time that has passed for this URB request is calculated. The judgment is made to make adjustments to set the time when the DPC callback trigger of the URB request is completed. This can be used to eliminate the problem of inaccurate trigger completion time caused by the accuracy of the system timer. The specific operation is: get the current time minus the time when the request is received to obtain the time consumed by the current URB request. If it is greater than 8000us and less than 12000us, it is calibrated to the default time T1 of the trigger minus the consumed time. Otherwise, the default DPC task completion time T2 of 9520us is used (the standard completion time of a single request is 20000us divided by 2, and then the default adjustment value of 480us is subtracted). Finally, this time is used to set the DPC timer of the URB request.

[0074] S104: Send the audio data to the cloud host client so that the audio data can be played using a physical headset of the cloud host client.

[0075] When sending audio data to the cloud host client, the audio data can be constructed as an audio data item and inserted into the tail of the audio data sending queue corresponding to the virtual USB headset device, and the corresponding sending thread is notified to process. The sending thread takes an audio data item from the head of the audio data sending queue and determines whether the current queue length exceeds a preset value, such as 10 (this value can be adjusted according to the actual application scenario). If it exceeds, the audio data is discarded to solve the problem of untimely audio data transmission due to network anomalies, program anomalies and other problems, thereby accumulating a large amount of outdated data and affecting normal use. If it does not exceed, the audio data is sent to the cloud host client.

[0076] It should be noted that step S103 and step S104 can be performed independently of each other without interfering with each other.

[0077] In a specific implementation of the present application, it also includes:

[0078] Receive rate adjustment notifications sent by the cloud host client in the event of abnormal audio data processing;

[0079] The target rate is adjusted to maintain the playback quality of the audio data.

[0080] For ease of description, the above steps are combined for explanation below.

[0081] When the physical earphones of the cloud host client are transmitted to the cloud host, the audio data in the isochronous transmission type URB request of the virtual earphones is sent to the cloud host client. The URB request is completed directly locally at a preset constant rate, and the cloud host client performs audio data processing operations such as buffering and frame dropping on the audio data.

[0082] The cloud host client determines the current processing status and adjusts the cloud host's constant completion rate (target rate) time to ensure the stability of audio data buffering and processing, and sends the audio data to the physical USB headset device in standard mode. During the processing, the request completion status is determined, and then based on the request completion status, it is decided whether to continue sending or notify the host of transmission abnormalities for recovery.

[0083] When the cloud host receives the rate adjustment notification, it can adjust the target rate to maintain the playback quality of the audio data. The target rate can be increased or decreased according to the specific details of the rate adjustment notification, and the adjustment range can also correspond to the specific situation of the notification. I will not go into details here.

[0084] Apply the method provided in the embodiment of the present application to receive a data transmission request sent by a virtual headset and determine the transmission type and transmission direction; if the data transmission request is an isochronous transmission type and the transmission direction is output, insert the data transmission request into a local timed completion queue and extract audio data from the data transmission request; for the requests in the timed completion queue, execute a completion response at a target rate; and send the audio data to a cloud host client so that the audio data can be played using the physical headset of the cloud host client.

[0085] In this application, after the cloud host obtains the data transmission request sent by the virtual headset, it first determines its transmission type and transmission direction. When it is clear that the data transmission request is of isochronous transmission type and the transmission direction is output, the data transmission request can be directly inserted into the local timed completion queue. At the same time, the audio data is extracted from the data transmission request. For the requests in the timed completion queue, the completion response is executed at the target rate. For the extracted audio data, it is sent directly to the cloud host client so that the audio data can be played using the physical headset of the cloud host client.

[0086] That is to say, when the cloud host receives the data transmission request sent by the virtual headset, it first determines whether it is an isochronous transmission type and whether the transmission direction of its data transmission request is output. If it is an isochronous transmission type and the transmission of the data transmission request is output, it indicates that the data transmission request carries audio data that needs to be transmitted to the cloud host client side for playback. In order to avoid problems such as request response delay caused by too long working links, in this application, the data transmission request is directly inserted into the local timed completion queue, and the response processing is completed at the target rate. For the audio data that needs to be transmitted to the cloud host client, it is directly extracted and sent to the cloud host client so that the audio data can be played based on the physical headset of the cloud host client. That is, separating the processing and response process of the audio data from the data transmission request of the isochronous transmission type can effectively shorten the working link, ensure the high timeliness of the isochronous transmission, and thus ensure the stable transmission of the audio data, so that the USB headset of the client can give full play to its functional characteristics in the cloud computer scenario.

[0087] Please refer to Figure 2 , Figure 2 This is a flowchart of another audio transparent transmission method in an embodiment of the present application. The method can be applied to a cloud host client. The method includes the following steps:

[0088] S201, receiving the cloud host to execute Figure 1 The audio data is sent after the audio transparent transmission method is shown, and the audio data is inserted into the end of the audio data queue for buffering.

[0089] In the present application, when the cloud host sends audio data to the cloud host client, it can be sent in the form of audio data items.

[0090] After receiving the audio data, the cloud host client can insert the audio data into the end of the audio data queue for buffering.

[0091] S202: Obtain processing status statistical information, and use the processing status statistical information to adjust the audio data queue.

[0092] The processing status statistical information, namely the statistical information of the processing status of the received audio data by the cloud host client, includes statistical parameters obtainable by direct counting, configuration parameters used for calibration, and status parameters.

[0093] The audio data queue is adjusted, including discarding the audio data therein and retaining the audio data therein to wait for being sent to the physical headset.

[0094] Among them, obtaining processing status statistics includes:

[0095] Get the evaluation parameters corresponding to the current audio pass-through;

[0096] The evaluation parameters are determined as processing status statistics.

[0097] Among them, obtaining the evaluation parameters corresponding to the current audio transparent transmission includes:

[0098] Based on the processing strategy and processing progress of the received audio data, updating statistical parameters;

[0099] Combined with the calibration strategy and statistical parameters, the state parameters and configuration parameters are updated;

[0100] The updated statistical parameters, state parameters and configuration parameters are determined as evaluation parameters.

[0101] Among them, statistical parameters are parameters that are directly obtained through statistics, such as direct counting, or directly based on the buffer size buffer data; state parameters can be specifically the current processing state, which can be subdivided into multiple state conditions; evaluation parameters can be used by the calibration strategy to determine whether to adjust the parameters used for the target rate.

[0102] The audio data queue is adjusted using the processing status statistics information, including:

[0103] Using the processing status statistical information, determining whether the audio data discarding condition is met;

[0104] If so, the number of discards is determined using processing status statistics;

[0105] The audio data matching the discard quantity is read from the head of the audio data queue, and the exclusive audio data is discarded.

[0106] By discarding them starting from the head, you can ensure the timeliness of the physical headphones.

[0107] In a specific implementation of the present application, it also includes:

[0108] Using processing status statistics, determine whether audio data processing is abnormal;

[0109] If yes, a rate adjustment notification is sent to the cloud host to maintain the playback quality of the audio data.

[0110] In this way, the request response of the cloud host can match the audio data processing rate of the cloud host client, thereby ensuring the playback quality of the physical headphones.

[0111] S203: Take out audio data for single transmission from the audio data queue, and construct an isochronous transmission request corresponding to the local system using the taken out audio data.

[0112] S204: Send an isochronous transmission request to the physical headset to play audio data using the physical headset.

[0113] For ease of understanding, the following is a combination of specific parameters and Figure 3 , combine the above steps to explain.

[0114] First, the meaning of the relevant policy values ​​and the preset default values ​​are explained: the current transmission state value is recorded as S1 (the initial value is N), the number of isochronous transmission requests currently sent to the physical headset for processing is recorded as C1 (the initial value is 0), the number of isochronous transmission requests submitted to the headset device during normal transmission is recorded as C2 (the default fixed value is 2), the current number of buffered audio data is recorded as C3 (the initial value is 0, which is equivalent to the length of the buffer queue), the number of times the current number of buffered data is 0 consecutively is recorded as C4 (the initial value is 0), the maximum number of data that can be buffered is recorded as M1 (the default fixed value is 17), and the maximum number of buffered data is recorded as M2 (the default fixed value is 17). The maximum number of buffered data is recorded as M2 (the default fixed value is M1 / 2+1), the maximum number of consecutive empty buffered data is recorded as M3 (the default fixed value is 800), the number of currently buffered audio data that can trigger submission to the headset is recorded as C5 (the default fixed value is M2+C2), the value when the last buffered data volume was in the dangerous range is C6 (the initial value is 0), the floating value of the buffered audio data volume in the safe range is C7 (the default fixed value is M1 / 2 / 2), the minimum value of the buffered audio data volume in the safe range is M4 (the default fixed value is M1 / 2+1- C7), and the maximum value of the buffered audio data volume in the safe range is M5 (the default fixed value is M1 / 2+1+ C7), the number of times the current buffered data exceeds M5 continuously is recorded as C8 (initial value is 0), the maximum number of times the current buffered data exceeds M5 continuously is recorded as M6 (default fixed value is 800), the floating value C9 when the buffered audio data volume is in the dangerous range (default fixed value is 2), the increment used to compare with the last record when the current buffered audio data volume is in the dangerous range is recorded as C10 (initial value is 0), the default value 1 of the increment used to compare with the last record when the current buffered audio data volume is in the dangerous range is recorded as D1 (default fixed value is 2), the floating value C9 when the buffered audio data volume is in the dangerous range is recorded as D1 (default fixed value is 2), the floating value C9 when the buffered audio data volume is in the dangerous range is recorded as D1 (default fixed value is 2), the floating value C9 when the buffered audio data volume is in the dangerous range is recorded as D1 (default fixed value is 2). The default value 2 of the increment of the record comparison is recorded as D2 (the default fixed value is 1), the number of times the current buffered data number is continuously equal to M1 or equal to 0 is recorded as C11 (the initial value is 0), the number of times the current buffered data number is continuously less than M4 is recorded as C12 (the initial value is 0), the maximum number of times the current buffered data number is continuously less than M4 is recorded as M7 (the default fixed value is 800), the increase or decrease value of the request completion time of the isochronous transmission URB type notifying the host side is C13 (the default is 20, the unit is us), the increase or decrease value of the request completion time of the isochronous transmission URB type notifying the host side is C14 (the default is 10, the unit is us).

[0115] Whenever an audio data item is sent to the cloud host client: the audio data is first buffered and frame loss is processed, which specifically includes the following steps:

[0116] First, insert the audio data into the tail of the audio data queue for buffering, and then determine whether C1 is greater than or equal to C2. If it is not, mark the current processing state as A. If it is, mark the current processing state as B. Continue to determine whether C3 is greater than M1. If it is, discard M2 audio data in the buffered data queue. Considering the timeliness, discard them starting from the head of the queue and set S1 to O1.

[0117] Then, the current C3 value and the current audio data transmission and processing status are determined to calibrate the amount of buffered data to keep it stable, so as to reduce the probability of the problem that the audio playback is not smooth due to the empty buffer and the playback is stuck due to the discarded audio data caused by the full buffer. The specific calibration strategy is as follows:

[0118] 1. When the current processing state is A, increment the value of C4 and determine whether it is greater than M3. If so, reset the value of C4 to 0 and set S1 to F1. Continue to determine whether C1 is 0 and C3 is less than C5. If so, set C6 to C3. If not, set the current processing state to C.

[0119] 2. When the current processing state is B, the steps are as follows:

[0120] (1) Determine whether C3 is greater than M5. If not, set the value of C8 to 0 and execute step (2). If yes, increment the value of C8.

[0121] ①. Continue to determine whether C3 is greater than or equal to the value of M1 minus C9. If so, increment C8 again and set C10 to D2. If not, set C10 to D1.

[0122] ②. Continue to determine whether C3 is equal to M1. If so, increment the value of C11. If not, set C11 to 0.

[0123] ③. Determine whether C8 is greater than M6. If not, no subsequent processing is performed. If so, continue to determine whether C3-C6 is greater than or equal to C10 or whether C11 is greater than M6. If so, set S1 to O2, and finally set C6 to C3, and C8 and C11 to 0.

[0124] (2) Determine whether C3 is less than M4. If not, set the value of C12 to 0 without further processing. If yes, increment the value of C12.

[0125] ①. Determine whether C3 is less than or equal to the value of C9. If so, increment C12 again and set C10 to D2. If not, set C10 to D1.

[0126] ②. Determine whether C3 is equal to 0. If so, increment the value of C11. If not, set C11 to 0.

[0127] ③. Determine whether C12 is greater than M7. If not, no subsequent processing is performed. If yes, continue to determine whether C6-C3 is greater than or equal to C10 or whether C11 is greater than M7. If yes, set S1 to F2, and finally set C6 to C3, and C12 and C11 to 0.

[0128] (3) Determine whether C3 is equal to 0. If so, increment C4. If not, set C4 to 0.

[0129] ① Continue to determine whether C4 is greater than M3. If so, set C4 to 0 and S1 to F1.

[0130] 3. Determine the value of S1:

[0131] (1) When S1 is O1 or F1, it means that the buffered data has been discarded because the number of buffered data exceeds the maximum limit or the buffered data has been empty for a long time. At this time, the audio processing speed sent to the physical USB headset is significantly lower than or higher than the speed at which the host system sends audio data to the virtual USB headset, resulting in unstable buffered data volume. The host processing rate needs to be calibrated to be consistent with the client, and the corresponding virtual USB headset on the host side is informed to adjust the time to complete the request. Specifically, when S1 is equal to O1, the host side T1 is added to C13, and when S1 is equal to F1, the host side T1 is subtracted from C13, and finally S1 is reset to N.

[0132] (2) When S1 is O2 or F2, it means that the number of buffered data has been in a dangerous range for a long time and is showing an aggravating trend (becoming empty or full). At this time, the audio processing speed sent to the physical USB headset is slightly lower or higher than the speed at which the host system sends audio data to the virtual USB headset, or there is a short-term processing rate fluctuation, resulting in the amount of buffered data not being in a stable and balanced state. The host processing rate needs to be calibrated to be consistent with the client, and the corresponding virtual USB headset on the host is informed to slightly adjust the time to complete the request. Specifically, when S1 is equal to O2, the host T1 is added to C14, and when S1 is equal to F2, the host T1 is subtracted from C14, and finally S1 is reset to N.

[0133] The audio data is taken out from the audio data buffer queue, constructed into an isochronous transfer request corresponding to the client system and sent to the physical USB headset device, specifically the following steps:

[0134] 1. Preset N1 to 0. First, determine whether the current processing status is C. If so, calculate the number of requests N1 that need to be sent to the USB headset. The calculation method is: first calculate the value of C2 minus C1, and compare it with C3. If (C2-C1) is greater than C3, then N1 is C3, otherwise N1 is (C2-C1). This is to keep consistent with the way the host system sends audio data to the virtual USB headset, so that the USB headset always has two URB requests being processed, so that there will be no idle interval when the audio is transmitted to the headset.

[0135] 2. Take out N1 data from the head of the audio data buffer queue in turn, and construct each data and its description information into a request for the USB isochronous transfer type supported by the current system. For example, if the LINUX system interacts with the device through LIBUSB, it constructs the isochronous transfer request structure provided by LIBUSB and submits it to the device. If the WINDOWS system is used, it constructs the corresponding URB structure and sends it to the device, and determines the corresponding sending status value. If successful, the value of C1 is incremented, otherwise S1 is set to E.

[0136] The abnormal notification processing will determine the current S1 value. The specific steps are as follows:

[0137] 1. Whenever a request sent to the physical USB headset is completed, first decrement the value of C1, then determine the current request completion status. If it is not successful, set the S1 status to E.

[0138] 2. Determine the current state of S1. If it is E, notify the host side that the corresponding virtual USB headset device has abnormal transmission state. The error state needs to be returned when the URB request is completed next time, so that the system can perform abnormal recovery processing (such as ABORT, RESET port or reset device, etc.), and finally reset S1 to N. If it is not E, continue to determine when the number of buffered data is greater than 0. If the determination is established, take a data from the head of the buffered data queue, and construct the data and its description information into a request of the USB isochronous transmission type supported by the current system and send it to the device, and then determine the corresponding transmission status value. If successful, the value of C1 is incremented, otherwise S1 is set to E.

[0139] The above policy values ​​and their default preset values ​​can be adjusted and optimized according to the actual usage scenario, and the examples are only for ease of understanding. In addition, this application is also applicable to the processing optimization of transparent microphone audio data, which will not be repeated here.

[0140] This application implements an audio solution based on virtual headset transmission in a cloud computer environment to ensure the real-time and stability of audio data. In addition, the invention can ensure the smoothness of audio playback through dynamic rate adjustment and frame loss compensation mechanism under different network environments and hardware conditions. Combined with technical means such as audio buffer management and abnormal recovery, the entire audio transmission link has a higher fault tolerance capability, effectively improving the user experience. It can be applied to scenarios such as remote office and cloud gaming that require high-quality audio interaction, providing users with a remote audio solution that is close to the local device experience, and giving full play to the functions of high-end USB headset devices.

[0141] Corresponding to the above method embodiment, the embodiment of the present application further provides an audio transparent transmission system. The audio transparent transmission system described below and the audio transparent transmission method described above can be referred to each other.

[0142] See also Figure 4 As shown, the system includes:

[0143] A cloud host 100 and a cloud host client 200; the cloud host and the cloud host client have a communication connection;

[0144] Cloud host, used to execute Figure 1 The steps of the audio transparent transmission method shown are:

[0145] The cloud host client is used to execute Figure 2 The steps of the audio transparent transmission method are shown.

[0146] Please refer to Figure 5 ,In this system, it mainly includes URB (USB Request Block) request response and data separation module, ,audio data buffering and frame dropping module, dual-end processing rate calibration module, USB ,isochronous transmission sending module, and request completion and exception notification module.

[0147] When the URB request response and data separation module receives the isochronous transmission URB request type of the virtual USB device, it obtains the audio data with the transmission direction of OUT and sends it to the corresponding client. At the same time, it inserts the URB request into the timing completion queue and completes the request at a standard and constant rate to ensure that the system's isochronous transmission abnormality judgment is not triggered.

[0148] After receiving the audio data sent by the host, the client audio data buffering and frame dropping module determines the amount of audio data currently being processed and buffered, and performs buffering or frame dropping processing.

[0149] The dual-end processing rate calibration module determines the current data processing status. When it is too fast or too slow, or when frame loss or buffer data is empty, it notifies the host end to adjust the timing completion rate and calibrate it to a stable state in order to maintain the playback quality of the audio data.

[0150] The USB isochronous transmission sending module takes out single transmission data from the data buffer and constructs an isochronous transmission request corresponding to the client system and sends it to the physical USB headset device when the data buffer amount meets the preset value.

[0151] The request completion and exception notification module determines whether the current transmission completion status is normal. If normal, it determines whether there is buffered data in the current buffer queue. If yes, it continues to take out the single transmission data and constructs an isochronous transmission request corresponding to the client system and sends it to the physical USB headset device. Otherwise, it notifies the host side that the transmission status is abnormal and needs to inform the host side system to recover.

[0152] This system can ensure the normal use of the virtual USB headset device transmitted to the host side.

[0153] This application mainly achieves technical innovation in processing the isochronous transfer type URB of the virtual USB headset. The key lies in separating the audio data from the processing and response process of the isochronous transfer type URB request. The conventional processing method of sending the isochronous transfer type URB request of the host virtual USB headset device to the client physical USB headset device and then synchronously waiting for completion is optimized to ensure the high timeliness of isochronous transmission.

[0154] Specifically, the host system first independently processes the transmission of audio data, and adopts a more flexible strategy to manage isochronous transmission type URB requests. This separate processing method can effectively avoid link delays caused by synchronous operations and ensure stable transmission of audio data.

[0155] Through this optimization method, the host can complete the URB isochronous transfer request at a stable and constant rate, ensuring the timeliness of the transmission process. After the data is transmitted to the client, it is constructed into an isochronous transfer request corresponding to the client system and sent to the physical headset in a standard manner. This not only solves the time-consuming and unstable problem caused by the long request processing link, but also effectively avoids the system judging the isochronous transfer abnormality caused by high latency and resetting the audio endpoint of the headset device.

[0156] In addition, it also includes strategies such as audio data buffering, frame loss processing, and dynamic rate calibration, which can help further improve the stability of audio transmission and ensure the quality of audio playback. By dynamically adjusting the isochronous transmission request rate at both ends of the transparent transmission, the system can adaptively maintain efficient and low-latency transmission of audio streams under different conditions. This application can also maximize the performance of high-end headphone devices and enhance the user's audio experience.

[0157] The specific implementation of the above-mentioned audio transparent transmission method on the system can refer to the description of the above-mentioned method embodiment, which will not be repeated here.

[0158] The system provided by the embodiment of the present application is applied to receive a data transmission request sent by a virtual headset and determine the transmission type and transmission direction; if the data transmission request is an isochronous transmission type and the transmission direction is output, the data transmission request is inserted into a local timed completion queue, and audio data is extracted from the data transmission request; for the requests in the timed completion queue, a completion response is executed at a target rate; and the audio data is sent to a cloud host client so that the audio data can be played using the physical headset of the cloud host client.

[0159] In this application, after the cloud host obtains the data transmission request sent by the virtual headset, it first determines its transmission type and transmission direction. When it is clear that the data transmission request is of isochronous transmission type and the transmission direction is output, the data transmission request can be directly inserted into the local timed completion queue. At the same time, the audio data is extracted from the data transmission request. For the requests in the timed completion queue, the completion response is executed at the target rate. For the extracted audio data, it is sent directly to the cloud host client so that the audio data can be played using the physical headset of the cloud host client.

[0160] That is to say, when the cloud host receives the data transmission request sent by the virtual headset, it first determines whether it is an isochronous transmission type and whether the transmission direction of its data transmission request is output. If it is an isochronous transmission type and the transmission of the data transmission request is output, it indicates that the data transmission request carries audio data that needs to be transmitted to the cloud host client side for playback. In order to avoid problems such as request response delay caused by too long working links, in this application, the data transmission request is directly inserted into the local timed completion queue, and the response processing is completed at the target rate. For the audio data that needs to be transmitted to the cloud host client, it is directly extracted and sent to the cloud host client so that the audio data can be played based on the physical headset of the cloud host client. That is, separating the processing and response process of the audio data from the data transmission request of the isochronous transmission type can effectively shorten the working link, ensure the high timeliness of the isochronous transmission, and thus ensure the stable transmission of the audio data, so that the USB headset of the client can give full play to its functional characteristics in the cloud computer scenario.

[0161] Corresponding to the above method embodiment, the embodiment of the present application further provides a readable storage medium. The readable storage medium described below and the audio transparent transmission method described above can refer to each other.

[0162] A readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the audio transparent transmission method of the above method embodiment are implemented.

[0163] The readable storage medium may specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or other readable storage medium that can store program codes.

[0164] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0165] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0166] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0167] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms include, include or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0168] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An audio transparent transmission method, characterized in that: Applicable to cloud hosts, including: Receive a data transmission request sent by the virtual headset, and determine the transmission type and transmission direction; If the data transmission request is of isochronous transmission type and the transmission direction is output, inserting the data transmission request into a local timing completion queue and extracting audio data from the data transmission request; For the requests in the timed completion queue, executing completion responses according to the target rate; The audio data is sent to the cloud host client so that the audio data can be played using a physical headset of the cloud host client.

2. The audio transparent transmission method according to claim 1, characterized in that: For the requests in the timed completion queue, executing completion responses according to a target rate includes: Determining a delayed procedure call timer using the target rate; For the requests in the timing completion queue, the delayed procedure call timer is used to complete the response.

3. The audio transparent transmission method according to claim 1, characterized in that: Also includes: Receiving a rate adjustment notification sent by the cloud host client in the event of an abnormal audio data processing situation; The target rate is adjusted to maintain the playback quality of the audio data.

4. An audio transparent transmission method, characterized in that: Applicable to cloud host clients, including: Receiving audio data sent by the cloud host after executing the audio transparent transmission method according to any one of claims 1 to 3, and inserting the audio data into the tail of the audio data queue for buffering; Obtaining processing state statistical information, and adjusting the audio data queue using the processing state statistical information; Taking out audio data for single transmission from the audio data queue, and constructing an isochronous transmission request corresponding to the local system using the taken out audio data; The isochronous transmission request is sent to the physical headset to play the audio data using the physical headset.

5. The audio transparent transmission method according to claim 4, characterized in that: Also includes: Using the processing status statistical information, determining whether the audio data processing is abnormal; If so, a rate adjustment notification is sent to the cloud host to maintain the playback quality of the audio data.

6. The audio transparent transmission method according to claim 4, characterized in that: Get processing status statistics, including: Get the evaluation parameters corresponding to the current audio pass-through; The evaluation parameter is determined as the processing status statistical information.

7. The audio transparent transmission method according to claim 6, characterized in that: Get the evaluation parameters corresponding to the current audio pass-through, including: Based on the processing strategy and processing progress of the received audio data, updating statistical parameters; In combination with the calibration strategy and the statistical parameters, the state parameters and the configuration parameters are updated; The updated statistical parameters, state parameters and configuration parameters are determined as the evaluation parameters.

8. The audio transparent transmission method according to claim 4, characterized in that: The audio data queue is adjusted using the processing state statistical information, including: Determining whether a condition for discarding audio data is met by using the processing status statistical information; If yes, determining the discard quantity using the processing status statistical information; The audio data matching the discard quantity is read from the head of the audio data queue, and the exclusive audio data is discarded.

9. An audio transparent transmission system, characterized in that: include: A cloud host and a cloud host client; the cloud host and the cloud host client are in communication connection; The cloud host is used to perform the steps of the audio transparent transmission method according to any one of claims 1 to 3, The cloud host client is used to execute the steps of the audio transparent transmission method as described in any one of claims 4 to 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the audio transparent transmission method according to any one of claims 1 to 9 are implemented.

Images