Synchronization of audio across multiple devices

The audio aggregator driver and unit driver generate USB request blocks on the computer device, and the USB clock synchronization manager is used to ensure the synchronization of audio data of multiple USB audio devices, solving the audio inconsistency caused by the device's running speed, and improving the audio experience .

CN113874850BInactive Publication Date: 2024-11-26MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080038824.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-05-31
Filing Date
2020-04-22
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When multiple USB audio devices are synchronized, the difference in device operation speed leads to inconsistent audio data transmission, which may cause audio distortion and affect the conference experience.

Method used

Through the audio aggregator driver and unit driver on the computer device, use the USB clock synchronization manager to determine the updated frame number, generate USB request blocks, ensure that all display devices receive and output audio data at the same time, and realize cross-device synchronization of audio data. .

Benefits of technology

Ensures audio data synchronization between multiple USB audio devices, avoids audio distortion, and improves consistency and quality of the conference audio experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113874850B_ABST
    Figure CN113874850B_ABST
Patent Text Reader

Abstract

A method and apparatus for synchronizing audio between multiple display devices in communication with a computer device may include using audio data from an audio stream to determine multiple subsets of audio data for transmission to multiple display devices in communication with the computer device via a universal serial bus (USB) connection. The method and apparatus may include obtaining a current frame number of a display device render buffer from a first display device in the multiple display devices. The method and apparatus may include determining an updated frame number by adding a constant to the current frame number; and generating multiple USB request blocks containing updated frame numbers and packets having multiple subsets of audio data. The method and apparatus may include sending a USB request block to a corresponding display device in the multiple display devices.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] The present disclosure relates to a plurality of devices in communication with a computer device.

[0002] When you connect multiple devices in a room for a conference, each device can be used to emit audio. Each Audio Unit device can operate a little faster or slower than the other devices. This speed characteristic can affect the audio because when the audio is played through the device, it may not be played at the expected time, causing distortion. As a result, the audio experience can be unpleasant for individuals interacting with the device during the conference.

[0003] Therefore, there is a need in the art for improvements in synchronizing multiple independent USB audio devices. Summary of the invention

[0004] A simplified overview of one or more embodiments of the present disclosure is presented below to provide a basic understanding of such embodiments. This overview is not an extensive overview of all contemplated embodiments, and is neither intended to identify the key or essential elements of all embodiments, nor to delimit the scope of any or all embodiments. Its sole purpose is to present some concepts of one or more embodiments of the present disclosure in a simplified form as a preface to a more detailed description presented later.

[0005] An example embodiment relates to a computer device. The computer device may include a memory for storing data and instructions, at least one processor configured to communicate with the memory, and an operating system configured to communicate with the memory and the processor, wherein the operating system is operable to: determine a plurality of audio data subsets using audio data from an audio stream for transmission to a plurality of display devices in communication with the computer device via a universal serial bus (USB) connection; obtain a current frame number of a display device render buffer from a first display device of the plurality of display devices; determine an updated frame number by adding a constant to the current frame number; generate a plurality of USB request blocks, wherein each of the plurality of USB request blocks includes an updated frame number and a packet having one of the plurality of audio data subsets; and send each of the plurality of USB request blocks to a corresponding display device of the plurality of display devices.

[0006] Another example embodiment relates to a method for synchronizing audio data between multiple display devices in communication with a computer device. The method may include, at an operating system executed on the computer device, determining multiple subsets of audio data using audio data from an audio stream for transmission to multiple display devices in communication with the computer device via a universal serial bus (USB) connection. The method may include obtaining a current frame number of a display device render buffer from a first display device in the multiple display devices. The method may include determining an updated frame number by adding a constant to the current frame number. The method may include generating multiple USB request blocks, wherein each of the multiple USB request blocks includes an updated frame number and a grouping of one of the multiple audio data subsets. The method may include sending each of the multiple USB request blocks to a corresponding display device in the multiple display devices.

[0007] Another example embodiment relates to a computer-readable medium storing instructions executable by a computer device. The computer-readable medium may include at least one instruction for causing the computer device to determine multiple audio data subsets based on audio data from an audio stream to be transmitted to multiple display devices communicating with the computer device via a universal serial bus (USB) connection. The computer-readable medium may include at least one instruction for causing the computer device to obtain a current frame number of a display device rendering buffer from a first display device in a plurality of display devices. The computer-readable medium may include at least one instruction for causing the computer device to determine an updated frame number by adding a constant to the current frame number. The computer-readable medium may include at least one instruction for causing the computer device to generate multiple USB request blocks, wherein each USB request block in the multiple USB request blocks includes an updated frame number and a grouping of an audio data subset in a plurality of audio data subsets. The computer-readable medium may include at least one instruction for causing the computer device to send each USB request block in a plurality of USB request blocks to a corresponding display device in a plurality of display devices.

[0008] Other advantages and novel features related to the embodiments of the present disclosure will be set forth in part in the following description and in part will become more apparent to those skilled in the art after examining the following or learning through practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In the attached picture:

[0010] Figure 1 is a schematic block diagram of an example computer device in communication with multiple display devices according to an embodiment of the present disclosure;

[0011] Figure 2 is an example of an audio buffer according to an embodiment of the present disclosure;

[0012] Figure 3 is an example of sending a plurality of universal serial bus (USB) request blocks to a plurality of display devices according to an embodiment of the present disclosure;

[0013] Figure 4 is an example of a method for synchronizing audio across multiple display devices according to an embodiment of the present disclosure; and

[0014] Figure 5 is a schematic block diagram of an example device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] The present disclosure relates to a device and method for synchronizing audio between multiple display devices communicating with a computer device. The display device and the computer device are connected to the same universal serial bus (USB), so that a common data stream of audio data can be divided into multiple audio data subsets for each display device. Multiple audio data subsets can be provided from the computer device to all display devices using a starting frame number, so that all display devices start outputting the audio data subsets at the same time. The transmission of audio data can also occur without a starting frame number. When the starting frame number is lost, the display device can output the audio data as soon as possible, which may cause the reception of audio data to be asynchronous between devices. When audio data is transferred from the computer device to the display device, the method and device can specify a timing indicator, such as a frame number. For example, in some embodiments, the frame number can be specified in a USB request block (URB) sent to the display device. By setting the frame number, the output of audio data starts at a time that matches the frame number, which allows the audio data to be synchronized across all display devices. When synchronizing audio data, the display device can use the same USB clock as the source of the audio data clock of each display device.

[0016] Display device can have at least one display and corresponding audio input and / or audio output.Display device can be any type of display, monitor, visual presentation device, computer equipment and / or physical panel that can present information, capture audio and / or send audio.Each display device can include any number of channels (for example, loudspeaker) for capturing audio or sending audio.Each loudspeaker of display device can correspond to any number of channels.For example, loudspeaker and / or microphone can be two-channel stereo with left channel and right channel.

[0017] The device and method may include an audio aggregator driver that receives audio data from one or more applications executed on a computer device. The audio aggregator driver may generate an audio stream using the received multiple audio data, and the audio stream may be stored in an audio buffer. The size of the audio buffer may be determined by multiplying the total number of display devices communicating with the computer device by the number of audio channels supported by each display device. In addition, sorting may be applied to multiple display devices, which defines the order in which the display devices are connected and / or communicated with the computer device. The audio data may be placed in an audio stream using the sorting.

[0018] In an embodiment, the device and method may use two layers of drivers, such as an audio aggregator driver on a computer device and a plurality of unit drivers on the computer device. Each unit driver may be associated with one of a plurality of display devices, so that each display device may be represented on the computer device by the unit driver. In addition, each unit driver may communicate with a display device rendering buffer, which may receive audio data to be output via a speaker on the display device. The unit driver may transmit the audio data to the display device.

[0019] The audio aggregator driver may include a synchronization manager that can synchronize the transmission of audio data to the display device. The synchronization manager may include a clock or other timing mechanism that can be used to set timing for a unit driver on a computer device. For example, the clock may be a USB clock. The timing can be used to synchronize transmission to the display device. The synchronization manager may receive a current frame number of a display device rendering buffer from a first unit driver associated with a first display device. For example, the synchronization manager may receive a current frame number from a unit driver associated with a first display device. The current frame number may correspond to the timing of the clock. For high-speed and ultra-high-speed display devices, the data transfer rate may be a microframe. The length of a microframe is 125 microseconds, and eight consecutive microframes constitute a high-speed or ultra-high-speed frame. Thus, the length of a frame may be 1 millisecond.

[0020] The synchronization manager can determine a constant that accounts for the delay in performing device initialization to move all display devices to an operating state capable of receiving audio data for output via speakers on the display devices. The synchronization manager can calculate an updated frame number by adding the constant to the current frame number. The updated frame number can be a specified timing indicator that ensures that the audio data is provided and / or transmitted to each display device simultaneously. The audio aggregator driver can generate one or more audio data subsets based on the audio stream for transmission to each display device. The audio aggregator driver can send the audio data subsets and the updated frame number to each unit driver for transmission to the display device.

[0021] Each unit driver can generate a USB request block, which includes an updated frame number and one or more packets with a subset of audio data of a corresponding display device. The USB stack can receive the USB request block from each unit driver and can send the USB request block to the corresponding display device. The unit driver can ensure that the display device rendering buffer starts outputting the subset of audio data from the USB request block at the updated frame number. The updated frame number on each display device rendering buffer at any given point can be the same, thereby satisfying the expectation that synchronization between all display devices can occur within a microframe.

[0022] By setting the time to transmit audio data in the future to allow enough time for the display device to be initialized to an operational state, it is possible to ensure that the USB bus transmits the audio data to all display devices at exactly the same time. In this way, the audio emitted from the speakers of the display devices can be synchronized.

[0023] Reference now Figure 1 , a system 100 for use with synchronized audio across multiple display devices 106, 108, 110, 112 communicating with a computer device 102 via a USB connection is shown. The display devices 106, 108, 110, 112 and the computer device 102 can be connected to the same universal serial bus (USB). In this way, the computer device 102 can act as a USB host controller for all display devices 106, 108, 110, 112, and a common USB clock can be provided from the computer device 102 to all display devices 106, 108, 110, 112, so that all display devices 106, 108, 110, 112 can use the same USB clock as the source of the audio data clock.

[0024] Multiple display devices 106, 108, 110, 112 can be any type of display, monitor, visual presentation device, computer device and / or physical panel capable of presenting information, capturing audio and / or emitting audio. In addition, each display device 106, 108, 110, 112 can include any number of channels for capturing audio and / or emitting audio. For example, display device 106 can include a speaker 40 supporting two channels (e.g., right and left). Display device 108 can include a speaker 44 supporting two channels (e.g., right and left). Display device 110 can include a speaker 48 supporting two channels (e.g., right and left). Display device 112 can include a speaker 52 supporting two channels (e.g., right and left). Each speaker 40, 44, 48, 52 of display devices 106, 108, 110, 112 can correspond to any number of channels. Display devices 106, 108, 110, 112 can be a combination of non-computing devices, computing devices, or non-computing devices and computing devices.

[0025] Each of the plurality of display devices 106, 108, 110, 112 may be represented on the computer device 102 with a corresponding unit driver 15, 17, 19, 21. For example, unit driver 15 may be associated with display device 106, unit driver 17 may be associated with display device 108, unit driver 19 may be associated with display device 110, and unit driver 21 may be associated with display device 112. The unit drivers 15, 17, 19, 21 may communicate with the speakers 40, 44, 48, 52 of the display devices 106, 108, 110, 112.

[0026] In addition, each display device 106, 108, 110, 112 may include an audio buffer per audio device entity for capturing and / or outputting audio. For example, multiple display devices 106, 108, 110, 112 may include individual display device rendering buffers 32, 34, 36, 38 that may receive audio data 12 or audio data subsets 37, 39, 41, 43 for output via speakers 40, 44, 48, 52. The unit driver 15, 17, 19, 21 may communicate with the display device rendering buffers 32, 34, 36, 38, and for the corresponding display device rendering buffers 32, 34, 36, 38, channel-specific audio data 12 or audio data subsets 37, 39, 41, 43 may be received.

[0027] When transferring the audio data 12 or the subset of audio data 37, 39, 41, 43 from the computer device 102 to the display device render buffer 32, 34, a timing indicator, such as an updated frame number 24, 36, 38, may be used. By setting the updated frame number 24 at a time in the future, the output of the audio data 12 or the subset of audio data 37, 39, 41, 43 begins at a time that matches the updated frame number 24, which allows the audio data 12 or the subset of audio data 37, 39, 41, 43 to be synchronized across all display devices 106, 108, 110, 112, so that the audio can be output via the speakers 40, 44, 48, 52 at the same time.

[0028] Computer device 102 may include any mobile or fixed computer device that can be connected to a network. For example, computer device 102 may be a computer device such as a desktop computer or a laptop computer or a tablet computer, an Internet of Things (IoT) device, a cellular phone, a gaming device, a mixed reality or virtual reality device, a music device, a television, a navigation system, a camera, a personal digital assistant (PDA) or a handheld device, or any other computer device having wired and / or wireless connection capabilities to one or more other devices.

[0029] The computer device 102 may include an operating system 111 that may be executed by the processor 48 and / or the memory 50. The memory 50 of the computer device 102 may be configured to store data and / or computer executable instructions that define and / or are associated with the operating system 111, and the processor 48 may execute such data and / or instructions to instantiate the operating system 111. Examples of the memory 50 may include, but are not limited to, the types of memory that may be used by a computer, such as random access memory (RAM), read-only memory (ROM), tape, disk, optical disk, volatile memory, non-volatile memory, and any combination thereof. Examples of the processor 48 may include, but are not limited to, any processor specifically programmed as described herein, including a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a system on a chip (SoC), or other programmable logic or state machine.

[0030] The operating system 111 may include an audio aggregator driver 14 that may receive audio data 12 from one or more applications 10 executed on the computer device 102. The audio aggregator driver 14 may generate an audio stream 13 using the received plurality of audio data 12 and may store the audio stream 13 in an audio buffer 26. The audio aggregator drivers 14, 16 may determine a buffer size 25 of the audio buffer 26. The buffer size 25 may be calculated by multiplying the total number of display devices 106, 108, 110, 112 by the number of channels supported by each of the display devices 106, 108, 110, 112. In the example shown, each of the display devices 106, 108, 110, 112 may support two channels (e.g., the speakers 40, 44, 48, 52 may support two left and right channels). Thus, the buffer size 25 may be eight (e.g., four devices multiplied by two channels).

[0031] Sorting may be applied to display devices 106, 108, 110, 112. Sorting may have an arrangement (or relative position) of values ​​based on the order in which display devices 106, 108, 110, 112 are connected and / or communicated with computer device 102. For example, display device 106 may be a first display device, display device 108 may be a second display device, display device 110 may be a third display device, and display device 112 may be a fourth display device. Sorting may be used to place audio data 12 in audio stream 13.

[0032] The audio aggregator driver 14 may split the audio data 12 from the audio stream 13 into a plurality of audio data subsets 37, 39, 41, 43. The plurality of audio data subsets 37, 39, 41, 43 may be determined in response to the total number of display devices 106, 108, 110, 112 in communication with the computer device 102. In the illustrated example, four display devices are in communication with the computer device 102. Thus, the audio aggregation driver 14 may split the audio data 12 from the audio stream 13 into four audio data subsets 37, 39, 41, 43 so that each display device 106, 108, 110, 112 may receive one of the audio data subsets 37, 39, 41, 43. The audio aggregator driver 14 may send the audio data subsets 37, 39, 41, 43 to the respective unit drivers 15, 17, 19, 21.

[0033] The audio aggregator driver 14 may include a synchronization manager 16 that may synchronize the transmission of the audio data 12 to the display devices 106, 108, 110, 112. The synchronization manager 16 may receive information identifying the number of display devices 106, 108, 110, 112 in communication with the computer device 102, along with the ordering of the display devices 106, 108, 110, 112.

[0034] The synchronization manager 16 may include a clock 11 or other timing mechanism that may be used to set timing for the unit drivers 15, 17, 19, 21 on the computer device 102. For example, the clock 11 may be a USB clock, and the display devices 106, 108, 110, 112 may use the clock 11 as the source of the audio data clock for each display device 106, 108, 110, 112. The frame number of the display device render buffer 32, 34, 36, 38 may be related to the timing of the clock 11. In this way, the timing of the clock 11 may be used to synchronize the output of the audio data subsets 37, 39, 41, 43 on the display devices 106, 108, 110, 112 so that the audio data output begins at the same time.

[0035] The synchronization manager 16 may receive the current frame number 18 of the display device rendering buffer 32 from the unit driver 15 associated with the first display device (e.g., the display device 106). The current frame number 18 may be the current frame number of the display device rendering buffer 32 from the display device 106. The current frame number 18 may be related to the timing of the clock 11. For high-speed and ultra-high-speed display devices, the data transmission rate may be a microframe. The length of a microframe is 125 microseconds, and eight consecutive microframes constitute a high-speed or ultra-high-speed frame. Thus, the length of a frame may be 1 millisecond.

[0036] The synchronization manager 16 may determine a constant 22 that accounts for a delay in performing device initialization 20 to move all display devices 106, 108, 110, 112 to an operational state capable of receiving audio data 12 for output via the speakers 40, 44, 48, 52 on the display devices 106, 108, 110. The device initialization 20 may include instructing the unit drivers 15, 17, 19, 21 to prepare the display device render buffers 32, 34, 36, 38. The value of the constant 22 may provide sufficient time to initialize all display devices 106, 108, 110, 112 to an operational state capable of receiving audio data 12. For example, the constant 22 may be eight, which may allow eight milliseconds to move all display devices 106, 108, 110, 112 to an operational state and begin streaming audio data 12. The constant 22 may be modified as the time to move the display devices 106, 108, 110, 112 increases or decreases. Furthermore, constant 22 may increase and / or decrease as the number of display devices 106, 108, 110, 112 increases and / or decreases. Thus, constant 22 may change in response to system changes, changes in host system speed, and / or other considerations.

[0037] The synchronization manager 16 may calculate an updated frame number 24 by adding the constant 22 to the current frame number 18. The updated frame number 24 may be a specified timing indicator that may ensure that the audio data subsets 37, 39, 41, 43 outputted on each of the display devices 106, 108, 110, 112 begin simultaneously. For example, when the timing of the clock 11 matches the updated frame number 24, the display devices 106, 108, 110, 112 may begin outputting the audio data subsets 37, 39, 41, 43.

[0038] The audio aggregator driver may send the updated frame number 24 and the audio data subset 37, 39, 41, 43 to the unit drivers 15, 17, 19, 21 for transmission to the display devices 106, 108, 110, 112. Each unit driver 15, 17, 19, 21 may generate one or more USB request blocks 30, 31, 33, 35 for the corresponding display device 106, 108, 110, 112. The USB request block 30, 31, 33, 35 may include one or more packets 23, 25, 27, 29 ( Figure 3). By setting the updated frame number 24 when transmitting the USB request block 30, 31, 33, 35, the display device rendering buffer 32, 34, 369, 38 can receive the audio data subset 37, 39, 41, 43 and can wait to start outputting the audio data subset 37, 39, 41, 43 until the timing of the clock 11 matches the updated frame number 24. In this way, the output of the audio data subset 37, 39, 41, 43 synchronized across all display devices 106, 108, 110, 112 can occur by setting the time to start outputting the audio data subset 37, 39, 41, 43 using the updated frame number 24 in the future.

[0039] The USB stack 28 may receive USB request block(s) 30 , 31 , 33 , 35 from the unit drivers 15 , 17 , 19 , 21 and may send the USB request blocks 30 , 31 , 33 , 35 to the display devices 106 , 108 , 110 , 112 .

[0040] In this way, the display device rendering buffer 32, 34, 36, 38 can receive the corresponding audio data subset 37, 39, 41, 43 from the USB request block 30, 31, 33, 35 with the updated frame number 24. When the timing of the clock 11 matches the updated frame number 24, the display device 106, 108, 110, 112 can begin to output the audio data subset 37, 39, 41, 43 via the speaker 40, 44, 48, 52. The frame number 24 on each display device rendering buffer 32, 34, 36, 38 at any given point can be the same, thereby satisfying the expectation that synchronization between all display devices 106, 108, 110, 112 can occur within a microframe. When the USB request block 30, 31, 33, 35 is transmitted from the USB stack 28, the updated frame number 24 can be kept up to date by the synchronization manager 16.

[0041] By setting the time to start outputting the audio data subsets 37, 39, 41, 43 in the future to allow enough time to copy the audio data subsets 37, 39, 41, 43 to the display device rendering buffers 32, 34, 36, 38, it is possible to ensure that the USB bus transmits the audio data subsets 37, 39, 41, 43 to all display devices 106, 108, 110, 112 at exactly the same time. In this way, the audio emitted from the speakers 40, 44, 48, 52 of the display devices 106, 108, 110, 112 can be synchronized.

[0042] Reference now Figure 2 , the example audio buffer 26 may include having audio from one or more applications 10 ( Figure 1) The audio stream 13 of the received audio data 12. The audio buffer 26 may include audio streams 13 corresponding to different times / sequences of the received audio data 12.

[0043] The audio aggregator driver 14 may divide the audio buffer 26 into Figure 1 ) corresponding to the number of portions. For example, when the buffer size 25 is equal to eight, the audio buffer 26 can be divided into eight portions, where each individual portion can correspond to a single channel supported by the display devices 106, 108, 110, 112. Thus, the audio buffer 26 can include an array of eight portions of the audio data 12 (at each time / sequence).

[0044] A subset of the number of portions may be allocated to a particular display device render buffer 32, 34, 36, 38 in response to the number of channels supported by the display device 106, 108, 110, 112. For example, each display device 106, 108, 110, 112 may receive a subset of two portions in the audio buffer 26 because the display devices 106, 108, 110, 112 support two channels in the illustrated example. A plurality of audio data subsets 37, 39, 41, 43 may be generated corresponding to portions of the audio buffer 26 of the respective display device render buffers 32, 34, 36, 38.

[0045] An ordering may be applied to the number of portions so that each portion may correspond to a particular display device render buffer 32, 34, 36, 38. The ordering may have an arrangement (or relative position) of values ​​based on the order in which the display devices 106, 108, 110, 112 are connected and / or in communication with the computer device 102. For example, the display device 106 may be the first display device and the first two portions of the audio buffer 26 may be assigned to the display device render buffer 32. As such, the display device render buffer 32 may receive the audio data subset 37 from the first two portions of the audio buffer 26.

[0046] Display device 108 may be a second display device, and the next two portions of audio buffer 26 may correspond to display device render buffer 24. Display device render buffer 34 may receive audio data subset 39 from the next two portions of audio buffer 26. Display device 110 may be a third display device, and the next two portions of audio buffer 26 may be associated with display device render buffer 36. As such, display device render buffer 36 may receive audio data subset 41 from the next two portions of audio buffer 26. Display device 112 may be a fourth display device, and the last two portions of audio buffer 26 may correspond to display device render buffer 38. Display device render buffer 38 may receive audio data subset 43 from the last two portions of audio buffer 26.

[0047] Reference now Figure 3 , shows an example of sending USB request blocks 30, 31, 33, 35 to multiple display devices 106, 108, 110, 112. Each unit driver 15, 17, 19, 21 can generate a USB request block 30, 31, 33, 35 with an updated frame number 24 and a packet 23, 25, 27, 29 of audio data for transmission to the corresponding display device 106, 108, 110, 112.

[0048] For example, the unit driver 15 may send a USB request block 30 to the display device rendering buffer 32 of the display device 106, the USB request block 30 including the updated frame number 24 and one or more packets 23 having the audio data subset 37 of the display device 106. The display device rendering buffer 32 may receive the packet 23 having the audio data subset 37, and may start outputting the audio data subset 37 via the speaker 40 at the updated frame number 24. In addition, the unit driver 17 may send a USB request block 31 to the display device rendering buffer 34, the USB request block 31 including the updated frame number 24 and one or more packets 25 having the audio data subset 39 of the display device 108. The display device rendering buffer 34 may receive the packet 25 having the audio data subset 39, and may start outputting the audio data subset 39 using the speaker 44 at the updated frame number 24.

[0049] The unit driver 19 may send a USB request block 33 to the display device rendering buffer 36, the USB request block 33 including the updated frame number 24 and one or more packets 27 having the audio data subset 41 of the display device 110. The display device rendering buffer 36 may receive the packet 27 having the audio data subset 41, and may start outputting the audio data subset 41 via the speaker 48 at the updated frame number 24. The unit driver 21 may send a USB request block 35 to the display device rendering buffer 38, the USB request block 35 including the updated frame number 24 and one or more packets 29 having the audio data subset 43 of the display device 112. The display device rendering buffer 38 may receive the packet 29 having the audio data subset 43, and may start outputting the audio data subset 43 using the speaker 52 at the updated frame number.

[0050] The audio data subsets 37, 39, 41, 43 may include the same audio data 12. In this way, the display devices 106, 108, 110, 112 may output the same audio data 12. In addition, the audio data subsets 37, 39, 41, 43 may include different audio data 12, so that each display device 106, 108, 110, 112 may output different audio data 12. The audio data subsets 37, 39, 41, 43 may also include silence when one or more display devices 106, 108, 110, 112 may not need to output audio (for example, when the application is located on a different display device).

[0051] By sending the updated frame number 24 with the packets 23, 25, 27, 29 and the audio data subsets 37, 39, 41, 43 to the display device render buffers 32, 34, 36, 38, outputting the audio data subsets 37, 39, 41, 43 can begin at a time that matches the updated frame number 24, which allows the audio data subsets 37, 39, 41, 43 to be synchronized across all display devices 106, 108, 110, 112. In this way, the audio emitted from the speakers 40, 44, 48, 52 of the display devices 106, 108, 110, 112 can be synchronized.

[0052] Reference now Figure 4 , computer device 102 ( Figure 1 ) can use the example method 400 to communicate across multiple display devices 106, 108, 110, 112 in communication with a computer device 102 ( Figure 1 )Synchronous audio data 12( Figure 1 ). You can refer to the following Figure 1 The architecture discusses the actions of method 400.

[0053] At 402, method 400 may include determining a plurality of audio data subsets from an audio stream of audio data to transmit to a plurality of display devices in communication with a computer device via a USB connection. An audio aggregator driver 14 may receive audio data 12 from one or more applications 10 executing on a computer device 102. The audio aggregator driver 14 may generate an audio stream 13 using the received plurality of audio data 12 and may store the audio stream 13 in an audio buffer 26.

[0054] The audio aggregator driver 14 may determine a buffer size 25 for the audio buffer 26. The buffer size 25 may be calculated by multiplying the total number of display devices 106, 108, 110, 112 by the number of channels supported by each of the display devices 106, 108, 110, 112. In the example shown, each of the display devices 106, 108, 110, 112 may support two channels (e.g., the speakers 40, 44, 48, 52 may support two left and right channels). Thus, the buffer size 25 may be eight (e.g., four devices multiplied by two channels). The audio buffer 26 may also include a plurality of audio streams 13 corresponding to different times / sequences of the received audio data 12, such that the audio stream 13 may include an array of eight portions of the audio data 12 (at each time / sequence of the audio stream 13).

[0055] The audio aggregator driver 14 may divide the audio stream 13 into a number of portions corresponding to the buffer size 25. For example, when the buffer size 25 is equal to eight, the audio stream 13 may be divided into eight portions, where each individual portion may correspond to a single channel supported by the display devices 106, 108, 110, 112. A subset of the number of portions may be allocated to a particular display device render buffer 32, 34, 36, 38 in response to the number of channels supported by the display devices 106, 108, 110, 112. For example, each display device 106, 108, 110, 112 may receive a subset of two portions in the audio stream 13 because the display devices 106, 108, 110, 112 support two channels in the illustrated example.

[0056] The audio aggregator driver 14 may split the audio data 12 from the audio stream 13 into a plurality of audio data subsets 37, 39, 41, 43. The plurality of audio data subsets 37, 39, 41, 43 may be determined in response to the total number of display devices 106, 108, 110, 112 in communication with the computer device 102. In the example shown, four display devices are in communication with the computer device 102. Thus, the audio aggregation driver 14 may split the audio data 12 from the audio stream 13 into the four audio data subsets 37, 39, 41, 43 so that each display device 106, 108, 110, 112 may receive one of the audio data subsets 37, 39, 41, 43.

[0057] The audio data 12 from the portion of the audio stream 13 corresponding to the corresponding display device rendering buffer 32, 34, 36, 38 may be included in the audio data subsets 37, 39, 41, 43. The audio data subsets 37, 39, 41, 43 may include the same audio data 12. In this way, the display devices 106, 108, 110, 112 may output the same audio data 12. In addition, the audio data subsets 37, 39, 41, 43 may include different audio data 12, so that each display device 106, 108, 110, 112 may output different audio data 12. The audio data subsets 37, 39, 41, 43 may also include silence when one or more display devices 106, 108, 110, 112 may not need to output audio (for example, when the application is located on a different display device).

[0058] Sorting may be applied to display devices 106, 108, 110, 112. Sorting may have an arrangement (or relative position) of values ​​based on the order in which display devices 106, 108, 110, 112 are connected and / or communicated with computer device 102. For example, display device 106 may be a first display device, display device 108 may be a second display device, display device 110 may be a third display device, and display device 112 may be a fourth display device. Audio data 12 may be placed in audio stream 13 using sorting. Additionally, sorting may be used to identify a subset of the number of portions of display device render buffers 32, 34, 36, 38.

[0059] At 404, method 400 may include obtaining a current frame number of a display device render buffer from a first display device in a plurality of display devices. The audio aggregator driver 14 may include a synchronization manager 16, which may include a clock 11 or other timing mechanism that may be used by the computer device and the display devices 106, 108, 110, 112. For example, clock 11 may be a USB clock, and the display devices 106, 108, 110, 112 may use clock 11 as a source of an audio data clock for each display device 106, 108, 110, 112. The frame number of the display device render buffer 32, 34, 36, 38 may be related to the timing of clock 11. In this way, the timing of clock 11 may be used to synchronize audio data between the computer device 102 and the display devices 106, 108, 110, 112.

[0060] The synchronization manager 16 may receive the current frame number 18 of the display device rendering buffer 32 from the unit driver 15 associated with the first display device (e.g., the display device 106). The current frame number 18 may be the current frame number of the display device rendering buffer 32 from the display device 106, which corresponds to the timing of the clock 11. For high-speed and ultra-high-speed display devices, the data transmission rate may be a microframe. The length of a microframe is 125 microseconds, and eight consecutive microframes constitute a high-speed or ultra-high-speed frame. Thus, the length of a frame may be 1 millisecond.

[0061] At 406, the method 400 may include determining an updated frame number by adding a constant to the current frame number. The synchronization manager 16 may determine a constant 22 that accounts for a delay in performing device initialization 20 that prepares all display devices 106, 108, 110, 112 to receive audio data 12 for output via speakers 40, 44, 48, 52 on the display devices 106, 108, 110. The device initialization 20 may include instructing the unit drivers 15, 17, 19, 21 to prepare the display device render buffers 32, 34, 36, 38. The value of the constant 22 may provide sufficient time to initialize all display devices 106, 108, 110, 112 to an operational state capable of receiving audio data 12. For example, the constant 22 may be eight, which may allow eight milliseconds to move all display devices 106, 108, 110, 112 to an operational state and begin streaming audio data 12. Constant 22 may be modified as the time to move display devices 106, 108, 110, 112 increases or decreases. Furthermore, constant 22 may increase and / or decrease as the number of display devices 106, 108, 110, 112 increases and / or decreases. In this way, constant 22 may change in response to changes in the system, changes in system speed, and / or other considerations.

[0062] The synchronization manager 16 may calculate an updated frame number 24 by adding the constant 22 to the current frame number 18. The updated frame number 24 may be a specified timing indicator that may ensure that the audio data subsets 37, 39, 41, 43 output on each of the display devices 106, 108, 110, 112 begin simultaneously.

[0063] At 408, the method 400 may include generating a plurality of USB request blocks including the updated frame number and a packet having a plurality of audio data subsets. The audio aggregator driver may send the updated frame number 24 and the audio data subsets 37, 39, 41, 43 to the unit drivers 15, 17, 19, 21 for transmission to the display devices 106, 108, 110, 112. Each unit driver 15, 17, 19, 21 may generate one or more USB request blocks 30, 31, 33, 35 for the corresponding display device 106, 108, 110, 112. The USB request blocks 30, 31, 33, 35 may include the updated frame number 24 and one or more packets 23, 25, 27, 29 ( Figure 3 ).

[0064] For example, the unit driver 15 may generate a USB request block 30 including the updated frame number 24 and one or more packets 23 having the audio data subset 37 of the display device 106. The unit driver 17 may generate a USB request block 31 including the updated frame number 24 and one or more packets 25 having the audio data subset 39 of the display device 108. The unit driver 19 may generate a USB request block 33 including the updated frame number 24 and one or more packets 27 having the audio data subset 41 of the display device 110. The unit driver 21 may generate a USB request block 35 including the updated frame number 24 and one or more packets 29 having the audio data subset 43 of the display device 112.

[0065] At 410, the method 400 may include sending each of the plurality of USB request blocks to a corresponding display device in the plurality of display devices. The USB stack 28 may receive the USB request block(s) 30, 31, 33, 35 from the unit driver 15, 17, 19, 21. The USB request blocks 30, 31, 33, 35 of the packets 23, 25, 27, 29 with the updated frame number 24 and the audio data subset 37, 39, 41, 43 may be sent to the corresponding display device render buffer 32, 34, 36, 38.

[0066] For example, the USB request block 30 may be sent to the display device rendering buffer 32 of the display device 106, and the display device rendering buffer 32 may output the audio data subset 37 via the speaker 40 at the updated frame number 24. For example, when the timing of the clock 11 matches the updated frame number 24, the display device rendering buffer 32 may start outputting the audio data subset 37. In addition, the USB request block 31 may be sent to the display device rendering buffer 34, and the display device rendering buffer 34 may output the audio data subset 39 using the speaker 44 at the updated frame number 24. For example, when the timing of the clock 11 is equal to the updated frame number 24, the display device rendering buffer 34 may start outputting the audio data subset 39. The USB request block 33 may be sent to the display device rendering buffer 36, and the display device rendering buffer 36 may output the audio data subset 43 via the speaker 48 at the updated frame number 24. For example, when the timing of the clock 11 reaches the updated frame number 24, the display device rendering buffer 36 may start outputting the audio data subset 43. The USB request block 35 may be sent to the display device rendering buffer 38, and the display device rendering buffer 38 may output the audio data subset 43 using the speaker 52 at the updated frame number 24. For example, when the timing of the clock 11 is the same as the updated frame number 24, the display device rendering buffer 38 may start outputting the audio data subset 43.

[0067] In this way, the display device rendering buffer 32, 34, 36, 38 can receive the audio data subset 37, 39, 41, 43 from the USB request block 30, 31, 33, 35. The display device 106, 108, 110, 112 can output the received audio data subset 37, 39, 41, 43 via the speaker 40, 44, 48, 52 at the updated frame number 24. The updated frame number 24 on each display device rendering buffer 32, 34, 36, 38 at any given point can be the same, thereby satisfying the expectation that synchronization between all display devices 106, 108, 110, 112 can occur within a microframe. When the USB request block 30 is transmitted from the USB stack 28, the updated frame number 24 can be kept up to date by the synchronization manager 16.

[0068] By setting the time to start outputting the audio data subsets 37, 39, 41, 43 in the future to allow enough time to copy the audio data subsets 37, 39, 41, 43 to the display device rendering buffers 32, 34, 36, 38, it is possible to ensure that the USB bus transmits the audio data subsets 37, 39, 41, 43 to all display devices 106, 108, 110, 112 at exactly the same time. In this way, the audio emitted from the speakers 40, 44, 48, 52 of the display devices 106, 108, 110, 112 can be synchronized.

[0069] Reference now Figure 5 ,and Figure 1 In contrast, the example computer 500, which may be configured as the computer device 102 according to an embodiment, includes additional component details. In one example, the computer 500 may include a processor 56 for performing processing functions associated with one or more of the components and functions described herein. The processor 56 may include a single or multiple sets of processors or a multi-core processor. In addition, the processor 56 may be implemented as an integrated processing system and / or a distributed processing system.

[0070] The computer 500 may also include memory 58, such as for storing local versions of applications executed by the processor 56. The memory 58 may include any type of memory that may be used by a computer, such as random access memory (RAM), read-only memory (ROM), tape, magnetic disk, optical disk, volatile memory, non-volatile memory, and any combination thereof. In addition, the processor 56 may include and execute an operating system 111 ( Figure 1 ).

[0071] In addition, the computer 500 may include a communication component 60 that provides for establishing and maintaining communications with one or more parties using the hardware, software, and services described herein. The communication component 60 may host communications between components on the computer device 102 and between the computer device 102 and external devices, such as devices located on a communication network and / or devices connected serially or locally to the computer device 102. For example, the communication component 60 may include one or more buses and may also include a transmit chain component and a receive chain component associated with a transmitter and a receiver, respectively, operable to interface with external devices.

[0072] In addition, the computer 500 may include a data store 62, which may be any suitable combination of hardware and / or software, that provides mass storage of information, databases, and programs used in conjunction with the embodiments described herein. For example, the data store 62 may be a data repository for the application 10, the audio aggregator driver 14, the unit driver 15, the unit driver 17, the unit driver 19, the unit driver 21, and / or the USB stack 28.

[0073] The computer 500 may also include a user interface component 64, which is operable to receive input from a user of the computer device 102 and is also operable to generate output for presentation to the user. The user interface component 64 may include one or more input devices, including but not limited to a keyboard, a numeric keypad, a mouse, a display (e.g., it may be a touch-sensitive display), a navigation key, a function key, a microphone, a speech recognition component, any other mechanism capable of receiving input from a user, or any combination thereof. In addition, the user interface component 64 may include one or more output devices, including but not limited to a display, a speaker, a tactile feedback mechanism, a printer, any other mechanism capable of presenting output to a user, or any combination thereof.

[0074] In an embodiment, the user interface component 64 may transmit and / or receive messages corresponding to the application 10, the audio aggregator driver 14, the unit driver 15, the unit driver 17, the unit driver 19, the unit driver 21, and / or the USB stack 28. In addition, the processor 56 executes the application 10, the audio aggregator driver 14, the unit driver 15, the unit driver 17, the unit driver 19, the unit driver 21, and / or the USB stack 28, and the memory 58 or the data storage 62 may store them.

[0075] As used in this application, the terms "component", "system", etc. are intended to include computer-related entities, such as but not limited to hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be but not limited to a process, a processor, an object, an executable file, an execution thread, a program and / or a computer running on a processor. At the latest, both the application and the computer device running on a computer device can be components. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate by means of local and / or remote processes, such as according to signals with one or more data packets, such as data from a component interacting with another component in a local system, a distributed system, and / or by means of signals through networks such as the Internet and other system communications.

[0076] Furthermore, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless specified otherwise or clear from context, the phrase "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, the phrase "X employs A or B" is satisfied by any of the following: X employs A; X employs B; or X employs both A and B. Furthermore, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more" unless specified otherwise or clear from context to be directed to a singular form.

[0077] Various embodiments or features have been presented in terms of systems that may include multiple devices, components, modules, etc. It should be understood and appreciated that the various systems may include additional devices, components, modules, etc. and / or may not include all of the devices, components, modules, etc. discussed in conjunction with the figures. Combinations of these methods may also be used.

[0078] Various illustrative logic, logic blocks, and actions of the methods described in conjunction with the embodiments disclosed herein may be implemented or executed with a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, which is designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computer devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in combination with a DSP core, or any other such configuration. In addition, at least one processor may include one or more components operable to perform one or more of the steps and / or actions described above.

[0079] In addition, the steps and / or actions of the methods or algorithms described in conjunction with the embodiments disclosed herein may be directly embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium may be integrated with the processor. In addition, in some embodiments, the processor and the storage medium may reside in an ASIC. In addition, the ASIC may reside in a user terminal. Alternatively, the processor and the storage medium may reside in a user terminal as discrete components. In addition, in some embodiments, the steps and / or actions of the methods or algorithms may be present in a machine-readable medium and / or a computer-readable medium as one or any combination or a set of codes and / or instructions, which may be incorporated into a computer program product.

[0080] In one or more embodiments, the functions described can be implemented in hardware, software, firmware or any combination thereof. If implemented in software, these functions can be stored or transmitted on a computer-readable medium as one or more instructions or codes. Computer-readable media include computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that a computer can access. As an example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Disks and optical disks used herein include compact discs (CDs), laser optical disks, optical disks, digital versatile discs (DVDs), floppy disks and blue-ray discs, wherein disks usually reproduce data magnetically, and optical disks usually reproduce data optically with lasers. The combination of the above items should also be included in the scope of computer-readable media.

[0081] Although the embodiments of the present disclosure have been described in conjunction with their examples, it will be appreciated by those skilled in the art that variations and modifications may be made to the above embodiments without departing from the scope thereof. Other embodiments will be apparent to those skilled in the art from consideration of the specification or from practice of the examples disclosed herein.

Claims

1. A computer device comprising: Memory, used to store data and instructions; at least one processor configured to communicate with the memory; as well as an operating system configured to communicate with the memory and the at least one processor, wherein the operating system is operable to: determining a plurality of subsets of audio data using audio data from the audio stream for transmission to a plurality of display devices in communication with the computer device via a universal serial bus (USB) connection; Obtaining a current frame number of a display device rendering buffer from a first display device among the plurality of display devices; determining an updated frame number by adding a constant to the current frame number; generating a plurality of USB request blocks, wherein each USB request block of the plurality of USB request blocks comprises the updated frame number and a packet having one of the plurality of audio data subsets; as well as Each USB request block of the plurality of USB request blocks is sent to a corresponding display device of the plurality of display devices.

2. The computer device of claim 1, wherein the constant is calculated in response to a delay in performing device initialization, the device initialization preparing each of the plurality of display devices to receive the plurality of audio data subsets.

3. The computer device of claim 2, wherein the constant changes in response to at least one of: a change in the number of the plurality of display devices, or a change in the speed at which initialization of the devices is performed.

4. The computer device of claim 1, wherein the operating system is further operable to: In response to the ranking of the plurality of display devices, determining an order of the plurality of display devices; and The first display device is determined in response to the sequence.

5. The computer device of claim 1, wherein the operating system is further operable to: The audio stream is divided into a number of portions, wherein the number of portions is equal to the number of the plurality of display devices multiplied by a number of channels supported by the plurality of display devices.

6. The computer device of claim 5, wherein the operating system is further operable to: determine a subset of the portion of the number for each display device of the plurality of display devices, and The audio data from the subset of the number of portions is included in the plurality of audio data subsets for each of the plurality of display devices.

7. The computer device of claim 1, wherein the computer device and each of the plurality of display devices share a USB clock.

8. The computer device of claim 7, wherein the operating system is further operable to: obtain the current frame number of the display device rendering buffer in response to the timing of the USB clock.

9. The computer device of claim 1, wherein the operating system is further operable to: The updated frame number is changed after the plurality of USB request blocks are sent to each of the plurality of display devices.

10. A method for synchronizing audio data between a plurality of display devices in communication with a computer device, comprising: At an operating system executing on the computer device, using audio data from an audio stream, determining a plurality of subsets of audio data for transmission via a universal serial bus (USB) connection to a plurality of display devices in communication with the computer device; Obtaining a current frame number of a display device rendering buffer from a first display device among the plurality of display devices; determining an updated frame number by adding a constant to the current frame number; generating a plurality of USB request blocks, wherein each USB request block of the plurality of USB request blocks comprises the updated frame number and a packet having one of the plurality of audio data subsets; as well as Each USB request block of the plurality of USB request blocks is sent to a corresponding display device of the plurality of display devices.

11. The method of claim 10, wherein each of the plurality of display devices outputs the plurality of audio data subsets at the updated frame number.

12. The method of claim 10, wherein the constant is calculated in response to a delay in performing device initialization that prepares each of the plurality of display devices to receive the plurality of audio data subsets.

13. The method of claim 12, wherein the constant changes in response to at least one of: a change in the number of the plurality of display devices, or a change in the speed at which the device initialization is performed.

14. The method according to claim 10, further comprising: In response to the ranking of the plurality of display devices, determining an order of the plurality of display devices; as well as The first display device is determined in response to the sequence.

15. The method according to claim 10, further comprising: The audio stream is divided into a number of portions, wherein the number of portions is equal to the number of the plurality of display devices multiplied by a number of channels supported by the plurality of display devices.

16. The method according to claim 15, wherein the method further comprises: determining a subset of the number of portions for each display device of the plurality of display devices, and The audio data from the subset of the number of portions is included in the plurality of audio data subsets for each of the plurality of display devices.

17. The method of claim 10, wherein the computer device and each of the plurality of display devices share a USB clock.

18. The method of claim 17, wherein the current frame number of the display device render buffer is obtained in response to the timing of the USB clock.

19. The method according to claim 10, wherein the method further comprises: The updated frame number is changed after the plurality of USB request blocks are sent to each of the plurality of display devices.

20. A non-volatile computer-readable medium storing instructions executable by a computer device, comprising: at least one instruction for causing the computer device to determine, using audio data from an audio stream, a plurality of subsets of audio data for transmission via a Universal Serial Bus (USB) connection to a plurality of display devices in communication with the computer device; at least one instruction for causing the computer device to obtain a current frame number of a display device rendering buffer from a first display device among the plurality of display devices; at least one instruction for causing the computer device to determine an updated frame number by adding a constant to the current frame number; at least one instruction for causing the computer device to generate a plurality of USB request blocks, wherein each USB request block of the plurality of USB request blocks comprises the updated frame number and a packet having one of the plurality of audio data subsets; as well as At least one instruction for causing the computer device to send each USB request block of the plurality of USB request blocks to a corresponding display device of the plurality of display devices.