Audio transmission method, wireless screen mirroring system, and computer‑readable storage medium

Abstract

Embodiments of the present description provide an audio transmission method, a wireless screen mirroring system, and a computer‑readable storage medium. The method comprises: when a first universal interface of a screen mirroring device is connected to a terminal device, establishing a Display Port transmission channel connecting the screen mirroring device and the terminal device, so that the screen mirroring device can receive, by means of the Display Port transmission channel, first raw audio data collected by a speaker device; when the screen mirroring device receives a screen mirroring operation instruction and determines that the speaker device is selected in the terminal device, the screen mirroring device sending the audio data to a screen mirroring receiving end by means of a screen mirroring channel; when the screen mirroring receiving end supports invoking a peripheral function, and when no screen mirroring operation instruction is received and the speaker device is selected in the terminal device, send the audio data to the screen mirroring receiving end by invoking a peripheral channel. Data is sent to only one request source at a time, preventing the screen mirroring receiving end from receiving two audio streams from a same source, thereby improving the accuracy of audio transmission and enhancing user convenience.

Description

Audio transmission methods, wireless screen sharing systems and computer-readable storage media Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to audio transmission methods, wireless screen sharing systems, and computer-readable storage media. Background Technology

[0002] Wireless screen sharing refers to synchronizing the screen content of terminal devices such as laptops or mobile phones to a display device via a wireless network (such as a wireless LAN), enabling the display device to display the screen content of the terminal device. A wireless screen sharing device is a common device for achieving wireless screen sharing. It connects to the terminal device via a wired connection, records the terminal device's screen content, and transmits it to the wirelessly connected display device for display.

[0003] Existing screen sharing devices, in addition to their screen sharing function, also have the ability to access peripheral devices. This function typically integrates seamlessly with the receiving end (such as a smart interactive whiteboard or audio / video equipment in a conference room), allowing users to access the receiving end's own or external devices and run unified communication software on their personal devices. This eliminates the limitations of the office location and enables a better online meeting experience. Current functional devices mainly include speakers, microphones, and cameras.

[0004] However, when the screen sharing device calls the speaker on the receiving end to play sound, the speaker's sound playback function can only be displayed on the receiving end while the screen sharing is active. If the user selects the speaker corresponding to the peripheral device function of the screen sharing device when not in screen sharing mode, the sound will not play normally. This will confuse the user, making them feel that the speaker is selected but there is no sound. This creates a frustrating experience for users who believe they have selected a speaker but have no sound. Therefore, there is a desire to optimize the interactive experience of the screen sharing device calling the speaker on the receiving end to play sound.

[0005] Summary of the Invention

[0006] The main purpose of this specification is to provide an audio transmission method, a wireless screen sharing system, and a computer-readable storage medium, aiming to simplify speaker functionality and enhance the speaker's interactive experience. The technical solution is as follows:

[0007] In a first aspect, embodiments of this specification provide an audio transmission method applied to a screen sharing device. The screen sharing device includes a first general-purpose interface for connecting to a terminal device. The terminal device has a speaker device corresponding to the screen sharing device, and the speaker device is used to collect first raw audio data from the terminal device. The method includes:

[0008] When the first general interface is connected to the terminal device, a DisplayPort transmission channel is established between the screen transmitter and the terminal device, and the first raw audio data is received through the DisplayPort transmission channel;

[0009] When a screen sharing operation command is received and the terminal device selects a speaker device, the first raw audio data is processed to obtain the first audio data, and the first audio data is sent to the screen sharing receiver connected to the screen sharing device through the screen sharing channel.

[0010] When the receiving end of the screen sharing device supports calling the peripheral function and the screen sharing device does not receive the screen sharing operation instruction, in response to the terminal device selecting the speaker device, the first original audio data is processed to obtain the second audio data, and the second audio data is sent to the receiving end of the screen sharing device by calling the peripheral channel.

[0011] Secondly, embodiments of this specification provide an audio transmission method applied to a terminal device, the terminal device including a second general interface and a speaker device corresponding to the screen mirroring device, the method including:

[0012] When the second universal interface is connected to the screen sharing device, a DisplayPort transmission channel is established between the screen sharing device and the terminal device. When the speaker device is selected, the first raw audio data of the terminal device is collected based on the speaker device.

[0013] If no screen sharing operation command is received, in response to the selection of the speaker device, the speaker device is invoked to transmit the first raw audio data to the screen sharing device through the DisplayPort transmission channel, so that the screen sharing device can process the first raw audio data to obtain the second audio data, and send the second audio data to the screen sharing receiver that is communicatively connected to the screen sharing device by invoking the peripheral channel.

[0014] Thirdly, the embodiments of this specification provide a wireless screen sharing system, which includes a screen sharing device and a screen sharing receiver.

[0015] The screen sharing device is paired and connected with the screen sharing receiver to obtain the connection information and function information of the screen sharing receiver, including whether it supports calling peripheral functions.

[0016] When the screen sharing device's first general interface is connected to the terminal device, the screen sharing device is enumerated as a speaker device and presented to the terminal device to obtain the first raw audio data from the terminal device.

[0017] Based on the connection information, the screen sharing device establishes a communication connection with the screen sharing receiver and confirms whether the screen sharing receiver supports calling peripheral functions based on the communication connection.

[0018] When the receiving end of the screen sharing device supports calling the peripheral device function and the screen sharing device does not receive the screen sharing operation instruction, when the screen sharing device does not receive the screen sharing operation instruction and responds to the terminal device selecting the speaker device, the first original audio data is processed to obtain the second audio data, and the second audio data is sent to the receiving end of the screen sharing device by calling the peripheral device channel.

[0019] After decoding the second audio data, the screen-sharing receiver obtains the first original audio data and calls the speaker of the screen-sharing receiver to play the first original audio data.

[0020] Fourthly, this specification provides an audio transmission method applied to a screen sharing device. The screen sharing device has a first general interface that can be connected to a terminal device. When the terminal device is connected to the first general interface, it has a speaker device corresponding to the screen sharing device.

[0021] The methods include:

[0022] The first general interface connects to the terminal device, establishing a DisplayPort transmission channel between the screen transmitter and the terminal device;

[0023] When the unified communications software running on the terminal device does not receive the screen sharing operation command, in response to selecting the speaker device in the speaker settings of the unified communications software, the screen sharing device is invoked to collect the third raw audio data of the unified communications software from the speaker device.

[0024] The received raw audio data is encoded to obtain encoded audio data.

[0025] The third audio data is sent to the screen sharing receiver connected to the screen sharing device, so that the screen sharing receiver can decode the third audio data to obtain the third original audio data, and call the speaker of the screen sharing receiver to play the third original audio data.

[0026] Fifthly, embodiments of this specification provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0027] In this embodiment, the screen sharing device is connected to a terminal device via a first universal interface. The terminal device has a unique speaker device corresponding to the screen sharing device. This speaker device is used to collect the first raw audio data from the terminal device. When the screen sharing device is connected to the terminal device via the first universal interface, a DisplayPort transmission channel is established between the screen sharing device and the terminal device. The screen sharing device can then transmit audio data via the DisplayPort. The port transmission channel receives first raw audio data collected by the speaker device. The screen sharer can process and distribute the collected first raw audio data according to the received instructions. When the screen sharer receives a screen sharing operation instruction and determines that the speaker device is selected in the terminal device, the screen sharer processes the first raw audio data to obtain first audio data and sends the first audio data to the screen sharing receiver connected to the screen sharer through the screen sharing channel. If the screen sharing receiver supports calling peripheral functions and the screen sharer has not received a screen sharing operation instruction, when the speaker device is selected in the terminal device, the first raw audio data is processed to obtain second audio data and sent to the screen sharing receiver through the peripheral calling channel. This allows the screen sharer to use the speaker of the screen sharing receiver when it is not in a screen sharing state, and only one speaker of the screen sharer needs to be selected for the sound to play normally, thus improving the interactive experience of the screen sharer calling the speaker of the screen sharing receiver to play sound. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 is a schematic diagram of a scenario of an audio transmission method provided in an embodiment of this specification;

[0030] Figure 2 is a schematic diagram illustrating an example of an audio transmission method provided in an embodiment of this specification;

[0031] Figure 3 is a schematic diagram illustrating an example of an audio transmission method provided in an embodiment of this specification;

[0032] Figure 4 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0033] Figure 5 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0034] Figure 6 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0035] Figure 7 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0036] Figure 8 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0037] Figure 9 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0038] Figure 10 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0039] Figure 11 is a schematic diagram of the architecture of an audio transmission method provided in an embodiment of this specification;

[0040] Figure 12 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0041] Figure 13 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0042] Figure 14 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0043] Figure 15 is a flowchart illustrating an audio transmission method provided in an embodiment of this specification;

[0044] Figure 16 is a schematic diagram of the architecture of an audio transmission method provided in an embodiment of this specification. Detailed Implementation

[0045] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0046] With the rapid development of information technology and the deepening of enterprise digital transformation, electronic meetings have become an important trend. Traditional meeting models often face limitations in time, location, and resources, failing to meet rapidly changing business needs and the requirements for efficient collaboration. The application of video conferencing, online collaboration platforms, and intelligent meeting systems has made cross-regional and cross-departmental communication more convenient.

[0047] Currently, users can conduct video conferences using unified communications software running on personal devices (such as personal computers). However, due to limitations in the performance of personal devices, the quality of their cameras, microphones, and speakers is often low, resulting in a poor meeting experience in large-scale conference scenarios. Commonly used unified communications software includes Mylink Meeting software developed by Mylink, ZoomVideo Communications' ZOOM video conferencing software, Microsoft Teams developed by Microsoft, and Tencent Meeting software developed by Tencent. These software programs all have the ability to conduct audio and video calls with other participants.

[0048] Based on this, a feature has been developed for screen sharing devices to call peripheral devices. This feature typically integrates seamlessly with the receiving end (such as a smart interactive whiteboard or audio / video equipment in a conference room), allowing users to call the receiving end's own or external devices through the screen sharing device. Users can then run unified communication software on their personal devices to use the devices called by the screen sharing device, enabling them to conduct online meetings using devices with superior performance compared to their own devices. This eliminates the limitations of the office location and provides a better online meeting experience.

[0049] When using a screen sharing device, users typically plug it into a computer's USB (Universal Serial Bus) port. After the device is plugged into the computer's USB Type-A port, a corresponding speaker device is created on the computer. When a user uses the screen sharing function to share the first source audio data from the computer to the receiving end, the data is transmitted through the computer's speaker device to the screen sharing device, and then from the screen sharing device to the receiving end, where it is played through the receiving end's speakers. When a user uses the screen sharing device's peripheral device function, they usually select the corresponding speaker device in the unified communications software running on the computer. After selection, the computer transmits the second source audio data received by the unified communications software through the speaker device to the screen sharing device, and then from the screen sharing device to the receiving end, where it is played through the receiving end's speakers, such as playing the voice of another user. Compared to directly using the computer's built-in speakers, using the external speaker device accessed through the screen sharing device's peripheral device function provides a better playback experience.

[0050] Referring to Figure 1, when multiple people participate in a meeting locally, and one or more people participate remotely via online conferencing software, as shown in Figure 1, which is a schematic diagram of an audio transmission method provided in this embodiment, the offline local users include user A, user B, user C, and user D, and the online remote user includes user E. The offline local users receive visual and auditory information through a display device 104 (such as a smart conference tablet). The online remote user's (e.g., user E's) computer and one of the offline terminal devices 101 (e.g., user A's computer) both run conferencing software. The offline microphone and camera collect data and transmit it to the online remote user via the offline terminal device 101 through the conferencing software, enabling the remote user to receive visual and auditory information. The projector 103 and the display device 104 can be wirelessly connected or wired via a connecting cable 102. The connecting cable 102 can be detachably connected to the projector 103 or integrally formed with the projector 103. When User A needs to display audio and video data from their computer to local and remote users, they typically project the current audio and video data onto the smart conference tablet and share it with remote users. This way, both local and remote users can see the content displayed on User A's computer screen. In this case, User A can plug a screen sharing device into their computer and project the current audio and video data onto the smart interactive tablet, then share it through the smart conference tablet. At this point, all audio data currently playing on the computer, whether from the currently playing PowerPoint presentation or from remote participants in the meeting software, will be picked up by the speaker device corresponding to the screen sharing device on the computer. That is, the first and second original audio data are mixed together and collected by the speaker device of the screen sharing device, and then transmitted uniformly to the smart interactive tablet. When the speaker device M corresponding to the screen sharing device is selected in the meeting software, it will indeed appear to the user that the audio is being played through the smart interactive tablet's speakers.

[0051] However, in some situations, users may not want to transmit the audio and video data currently displayed on their computer (such as the initial raw audio data) to the smart interactive whiteboard. For example, if only user A is attending the meeting locally, they can directly view their own computer without transmitting to the smart interactive whiteboard. However, due to poor speaker performance on user A's computer, they cannot clearly hear the voice of the remote user. In this case, user A wants to use the speakers of the smart interactive whiteboard in the local meeting room to play the remote user's voice. User A plugs in the screen sharing device on their computer and selects the corresponding speaker device M in the meeting software. However, since the screen sharing function and the audio data for calling peripheral devices are transmitted to the receiving end through the corresponding speaker device M on the computer, the peripheral device calling function of the screen sharing device can only be used when the screen is sharing. If the receiving end's speakers cannot be used independently when the screen is not sharing, the user will not be able to play sound normally when selecting the corresponding speaker M in the meeting software, causing confusion.

[0052] In related technologies, as shown in Figure 2, which is a schematic diagram illustrating an example of an audio transmission method provided in this specification, the screen sharing device is improved so that after the screen sharing device 103 is inserted into the terminal device 101, two corresponding speaker devices are simultaneously generated, such as speaker device M1 and speaker device M2. Speaker device M1 collects the audio data transmitted during screen sharing, while speaker device M2 collects the audio data transmitted during the process of calling peripheral devices. This design requires the user to select according to the actual situation, that is, to select speaker device M1 when screen sharing is in progress, and to select speaker device M2 when only using the function of calling peripheral devices without screen sharing. This is not easy for users to distinguish and is prone to confusion and incorrect selection.

[0053] Furthermore, screen sharing devices typically have multiple modes, each corresponding to a different interface. For example, a screen sharing device can have both USB Type-A and USB Type-C modes. The USB Type-A mode supports various screen sharing functions when connected to a computer via the USB Type-A interface, while the USB Type-C mode supports the same functions. Moreover, the speaker device generated on the computer side differs for each interface. For instance, a USB Type-A screen sharing device usually generates a single speaker device on the computer side, requiring different types of speaker devices to be designed for different systems. For example, Windows systems use UAC speaker devices, while Mac systems use virtual speaker devices. For USB Type-C mode, a corresponding hardware speaker needs to be generated. Since different speaker devices already exist for different systems and interfaces, defining different speaker devices based on different functions, as described above, would make the entire audio architecture of the screen sharing device complex and redundant. Therefore, this application aims to provide a simple audio architecture that is compatible with multiple operating systems and interface modes, while being easy for users to understand and use without requiring extensive comprehension, thus improving the user experience.

[0054] Based on the above problems, this specification provides an audio transmission method. Please refer to Figure 3, which is a schematic diagram illustrating an example of an audio transmission method provided in this specification. After the screen sharing device 103 is connected to the terminal device 101 through a first general interface (not shown), the terminal device 101 displays a unique speaker device corresponding to the screen sharing device 103 (speaker device N in Figure 3). When the user selects the speaker device N, the terminal device 101 performs first raw audio data acquisition through the speaker device N and sends the acquired first raw audio data to the speaker driver instance in the screen sharing device 103. After receiving the first raw audio data, the speaker driver instance sends the first raw audio data to the audio distribution application in the screen sharing device 103. The audio distribution application then forwards the audio data to the corresponding target application as needed, such as sending it to the screen sharing application or calling the peripheral device application, to satisfy the original screen sharing and peripheral device calling functions.

[0055] Using the audio transmission method provided in this specification, after the screen sharing device is connected to the terminal device through a first general interface, the terminal device has a unique speaker device corresponding to the screen sharing device, eliminating other virtual speaker drivers or hardware speakers in the current implementation. When the user needs to use the screen sharing device function, they can clearly know that selecting the speaker device will allow them to directly use the corresponding speaker function of the screen sharing device. By selecting the speaker device, the first original audio data of the terminal device can be transmitted to the screen sharing device. The screen sharing device performs different operations according to different instructions received. For example, when a screen sharing operation instruction is received, the first original audio data is processed to obtain the first audio data, which is then sent to the screen sharing receiving end connected to the screen sharing device through the screen sharing channel. Alternatively, if the screen sharing receiving end supports calling peripheral functions, when no screen sharing operation instruction is received and the speaker device is selected, the first original audio data is processed to obtain the second audio data, which is then sent to the screen sharing receiving end through the peripheral channel. By setting a unique speaker device in the terminal device, users do not need to switch speakers when using the two different application functions of screen sharing and device calling, which reduces user confusion and improves user convenience.

[0056] The audio transmission method provided in this specification will be described in detail below with reference to specific embodiments.

[0057] Please refer to Figure 4, which is a flowchart illustrating an audio transmission method provided in an embodiment of this specification. The method may include the following steps S101-S103.

[0058] S101, when the first general interface is connected to the terminal device, a DisplayPort transmission channel is established to connect the screen transmitter and the terminal device, and the first raw audio data is received through the DisplayPort transmission channel;

[0059] In one embodiment, the screen sharing device includes a first general-purpose interface. The screen sharing device is connected to a terminal device via this first general-purpose interface. A DisplayPort (DP) transmission channel is then established between the screen sharing device and the terminal device, enabling communication between them. Specifically, the DP transmission channel can receive first raw audio data collected by a speaker device in the terminal device. It is understood that DP is a data packet-based scalable protocol used for transmitting video and audio data. DP is highly scalable and backward compatible. There are two main interface types for DP: the standard DP interface and the Type-C interface. For example, the first general-purpose interface can be USB Type-C. The first raw audio data is audio data stored in the terminal device; for example, it could be audio played on the terminal device or audio received from remote participants in the conferencing software running on the terminal device.

[0060] S102, when a screen sharing operation command is received and the terminal device selects a speaker device, the first raw audio data is processed to obtain the first audio data, and the first audio data is sent to the screen sharing receiver connected to the screen sharing device through the screen sharing channel.

[0061] In one embodiment, when the screen sharing device receives a screen sharing operation command and the terminal device selects a speaker device, it indicates that the user needs to use the screen sharing function. The acquired first raw audio data is processed (e.g., encoded) to obtain first audio data, which is then transmitted to the screen sharing receiver through a screen sharing channel. This screen sharing channel can be the transmission channel corresponding to the screen sharing application installed in the screen sharing device.

[0062] S103, when the screen sharing receiver supports calling peripheral functions and the screen sharing device does not receive a screen sharing operation instruction, in response to the terminal device selecting the speaker device, the first original audio data is processed to obtain the second audio data, and the second audio data is sent to the screen sharing receiver by calling the peripheral channel.

[0063] In one embodiment, when the screen sharing receiver supports calling peripheral devices and the screen sharing device does not receive a screen sharing operation command, in response to the terminal device selecting a speaker device, indicating that the user does not currently need to perform a screen sharing operation but needs to play audio from the terminal device through a peripheral device, the first raw audio data is processed to obtain second audio data, and the second audio data is sent to the screen sharing receiver through a peripheral device calling channel. This peripheral device calling channel can be the transmission channel corresponding to an external device calling application installed in the screen sharing device.

[0064] The screen sharing channel can be a channel established between the screen sharing device and the screen sharing receiver for transmitting screen sharing data. The screen sharing data can be audio or video data from the terminal device to be transmitted to the screen sharing receiver. The peripheral device access channel can be a channel established between the screen sharing device and the screen sharing receiver for transmitting peripheral device access data. The peripheral device access data can be acquired or played by the device built into the screen sharing receiver or an external device.

[0065] In the embodiments of this specification, a DisplayPort transmission channel is established connecting the screen sharing device and the terminal device. The screen sharing device can receive first raw audio data collected by the speaker device through the DisplayPort transmission channel. The screen sharing device can process and distribute the collected first raw audio data according to the received instructions. When the screen sharing device receives a screen sharing operation instruction and determines that the speaker device is selected in the terminal device, the screen sharing device processes the first raw audio data to obtain first audio data, and sends the first audio data to the screen sharing receiver connected to the screen sharing device through the screen sharing channel. When the screen sharing receiver supports calling peripheral functions and the screen sharing device does not receive a screen sharing operation instruction, when the speaker device is selected in the terminal device, the first raw audio data is processed to obtain second audio data, and the second audio data is sent to the screen sharing receiver through the peripheral calling channel. This allows the screen sharing device to call the speaker of the screen sharing receiver in the non-screen sharing state, and only one speaker of the screen sharing device needs to be selected for the sound to play normally, thus improving the interactive experience of the screen sharing device calling the speaker of the screen sharing receiver to play sound.

[0066] Please refer to Figure 5, which is a flowchart illustrating an audio transmission method provided in an embodiment of this specification. The method may include the following steps S201-S205.

[0067] S201, the screen sharing device is paired and connected with the screen sharing receiver to obtain the connection information and function information of the screen sharing receiver. The function information includes whether it supports calling peripheral functions.

[0068] In one embodiment, the screen sharing device is paired with the screen sharing receiver to obtain the receiver's connection and functional information. Before inserting the screen sharing device into the terminal device, it needs to be pre-paired with the receiver. This can be achieved by inserting the screen sharing device's first general interface into the receiver to obtain the receiver's connection information, including hotspot information such as the hotspot name and password. Furthermore, the receiver's functional information can also be obtained. The receiver stores fields indicating whether it supports calling peripheral functions, allowing the screen sharing device to confirm the receiver's capabilities. After pairing, the screen sharing device can be unplugged from the receiver. At this point, the screen sharing device has stored the receiver's connection information. When the screen sharing device is inserted into the terminal device and powered by the terminal device, it can wirelessly connect to the receiver based on the obtained connection information.

[0069] S202, when the first general interface of the screen sharing device is connected to the terminal device, the screen sharing device is enumerated as a speaker device and presented to the terminal device to obtain the first raw audio data in the terminal device.

[0070] In one embodiment, after the screen sharing device is connected to the terminal device, the screen sharing device is enumerated as a speaker device and displayed in the terminal device; that is, a speaker device corresponding to the screen sharing device is added to the terminal device. The first general-purpose interface can be a USB Type-C interface. Other related descriptions are given in step S101 of the embodiments described above and will not be repeated here. Of course, in other embodiments, the types of the first general-purpose interface and the second general-purpose interface can be different. For example, the first general-purpose interface can be a USB Type-C interface, and the second general-purpose interface can be a USB Type-A interface. In this case, an external interface converter can be used on the terminal device side to connect the screen sharing device to the terminal device.

[0071] S203, based on the connection information, the screen transmitter establishes a communication connection with the screen transmitter receiver, and confirms whether the screen transmitter receiver supports calling peripheral functions based on the communication connection.

[0072] In one embodiment, after connecting to the terminal device, the screen sharing device establishes a communication connection with the screen sharing receiver based on pre-acquired connection information. For example, it connects to the screen sharing receiver's hotspot based on the hotspot name and password stored in the connection information, thereby establishing a communication connection. Simultaneously, the screen sharing device reconfirms with the screen sharing receiver whether it supports calling peripheral device functions. Although the screen sharing device has already obtained the function information, the peripheral device calling function in the screen sharing receiver may not be enabled or may no longer be supported; therefore, reconfirmation is necessary.

[0073] S204, the screen sharing receiver supports calling peripheral functions. When the screen sharing device does not receive the screen sharing operation instruction and responds to the terminal device selecting the speaker device, the first original audio data is processed to obtain the second audio data, and the second audio data is sent to the screen sharing receiver by calling the peripheral channel.

[0074] In one embodiment, when the screen sharing receiver supports calling peripheral functions, but the screen sharing device does not receive the screen sharing operation instruction, and the speaker device corresponding to the screen sharing device is selected in the terminal device, it indicates that the user has a need to use the calling peripheral function. Since the calling peripheral function is implemented by the screen sharing device, the screen sharing device processes the first raw audio data to obtain the second audio data, and sends the second audio data to the screen sharing receiver by calling the peripheral channel.

[0075] S205, the screen sharing receiver decodes the second audio data to obtain the first original audio data, and calls the speaker of the screen sharing receiver to play the first original audio data.

[0076] In one embodiment, the screen sharing receiver receives the second audio data, decodes it to obtain the first original audio data, and calls the speaker of the screen sharing receiver to play the first original audio data, thereby enabling the user to use the peripheral calling function to improve the audio playback quality.

[0077] In the embodiments of this specification, the screen sharing device is paired and connected with the screen sharing receiver to obtain the connection information and functional information of the screen sharing receiver. The functional information includes whether it supports calling peripheral functions. When the first general interface of the screen sharing device is connected to the terminal device, the screen sharing device is enumerated as a speaker device and presented to the terminal device to obtain the first original audio data in the terminal device. Based on the connection information, the screen sharing device and the screen sharing receiver establish a communication connection and confirm whether the screen sharing receiver supports calling peripheral functions based on the communication connection. If the screen sharing receiver supports calling peripheral functions, when the screen sharing device does not receive a screen sharing operation command and responds to the terminal device selecting a speaker device, it processes the first original audio data to obtain second audio data and sends the second audio data to the screen sharing receiver by calling the peripheral channel. The screen sharing receiver decodes the second audio data to obtain the first original audio data and calls the speaker of the screen sharing receiver to play the first original audio data. By pairing the screen sharing device with the screen sharing receiver, the screen sharing device can automatically connect to the screen sharing receiver after connecting to the terminal device, and can determine the functions of the screen sharing receiver. When the screen sharing receiver supports calling peripheral functions, if the screen sharing device does not receive the screen sharing operation command, it will process the first raw audio data it has acquired and send it to the screen sharing receiver to realize the calling of peripheral functions.

[0078] Please refer to Figure 6, which is a flowchart illustrating an audio transmission method provided in an embodiment of this specification. The method may include the following steps S301-S303.

[0079] S301, when the second universal interface is connected to the screen sharing device, a Display Port transmission channel connecting the screen sharing device and the terminal device is established. When the speaker device is selected, the first raw audio data of the terminal device is collected based on the speaker device.

[0080] In one embodiment, the audio transmission method is described from the perspective of the terminal device. The terminal device includes a second universal interface and a speaker device corresponding to the screen sharing device. In the terminal device, the speaker device may be generated after the screen sharing device is inserted. When the second universal interface is connected to the screen sharing device, a DisplayPort transmission channel is established connecting the screen sharing device and the terminal device. The terminal device sends the first raw audio data collected by the speaker to the screen sharing device through this DisplayPort transmission channel. For other related descriptions, please refer to step S101 in the embodiments of the above specification, which will not be repeated here.

[0081] S302, in the absence of a screen sharing operation command, in response to the selection of the speaker device, the speaker device is invoked to transmit the first original audio data to the screen sharing device through the Display Port transmission channel, so that the screen sharing device can process the first original audio data to obtain the second audio data, and send the second audio data to the screen sharing receiver connected to the screen sharing device through the call of the peripheral channel.

[0082] In one embodiment, when the screen sharing device does not receive a screen sharing operation command from the end, the terminal device transmits the first raw audio data it has collected to the screen sharing device through the DisplayPort transmission channel. Then, according to the received screen sharing operation command, the screen sharing device processes the first raw audio data to obtain second audio data, and sends the second audio data to the screen sharing receiving end that is communicatively connected to the screen sharing device by calling an external channel.

[0083] S303, upon receiving a screen sharing operation command, in response to the selection of a speaker device, the speaker device is invoked to transmit the first raw audio data to the screen sharing device via the DisplayPort transmission channel, so that the screen sharing device can process the first raw audio data to obtain the first audio data, and send the first audio data to the screen sharing receiver connected to the screen sharing device via the screen sharing channel.

[0084] In one embodiment, when the screen sharing device receives a screen sharing operation command, it sends the first raw audio data to the screen sharing device via the DisplayPort transmission channel. The screen sharing device processes the first raw audio data to obtain the first audio data, and then sends the first audio data to the screen sharing receiving end via the screen sharing channel.

[0085] In the embodiments of this specification, the terminal device is connected to the screen sharing device via a second universal interface. When connected to the screen sharing device via the second universal interface, a DisplayPort transmission channel is established between the screen sharing device and the terminal device. If no screen sharing operation command is received, in response to the selection of a speaker device, the speaker device transmits first raw audio data to the screen sharing device via the DisplayPort transmission channel. The screen sharing device processes the first raw audio data to obtain second audio data and then sends the second audio data to the screen sharing receiving end communicatively connected to the screen sharing device via a peripheral channel. Simultaneously, if a screen sharing operation command is received, in response to the selection of a speaker device, the terminal device invokes the speaker device to transmit the first raw audio data to the screen sharing device via the DisplayPort transmission channel. The screen sharing device processes the first raw audio data to obtain first audio data and then sends the first audio data to the screen sharing receiving end communicatively connected to the screen sharing device via the screen sharing channel.

[0086] Please refer to Figure 7, which is a flowchart illustrating an audio transmission method provided in an embodiment of this specification. The method may include the following steps S401-S404.

[0087] S401, when the first general interface is connected to the terminal device, establishes a DisplayPort transmission channel connecting the screen transmitter and the terminal device.

[0088] In one embodiment, after the screen sharing device is connected to the second general interface of the terminal device through the first general interface, a Display Port transmission channel is established between the screen sharing device and the terminal device.

[0089] S402, In response to the connection success command, the screen mirroring device calls the speaker driver to generate a speaker driver instance;

[0090] In one embodiment, the speaker driver instance and the speaker device are generated after the two devices are interconnected. A speaker driver program is installed on the screen sharing device side. The speaker driver program can be stored in the first memory of the screen sharing device. When the two devices are successfully connected, the screen sharing device can obtain a connection success command. For example, the processor in the screen sharing device can receive the connection success command, call the speaker driver program in the first memory, and generate the speaker driver instance. Then, the first raw audio data is received from the DisplayPort transmission channel through the speaker driver instance. The speaker driver instance is for internal processing within the screen sharing device. Specifically, in the screen sharing device, the speaker driver instance and the speaker driver program can be a USB audio device (such as a UAC driver). Optionally, after generating the speaker driver instance, the first raw audio data can be received from the DisplayPort transmission channel through the speaker driver instance.

[0091] S403, the screen sharing device sends a virtual driver to the terminal device based on the DisplayPort transmission channel;

[0092] In one embodiment, the first memory stores a virtual driver program used to generate a speaker device. After the screen sharing device is connected to the terminal device, the screen sharing device sends the virtual driver program to the terminal device via a first universal interface. In other words, since a DisplayPort transmission channel is established after the first universal interface and the second universal interface are connected, the screen sharing device sends the virtual driver program to the terminal device via the DisplayPort transmission channel, so that the terminal device can generate a speaker device based on the virtual driver program. The virtual driver program can also be stored in the first memory of the screen sharing device. Specifically, the screen sharing device has a USB flash drive type device enumeration, and the virtual driver program is placed in this USB flash drive so that the terminal device can read the virtual driver program from the USB flash drive.

[0093] S404, the terminal device receives the virtual driver sent by the screen sharing device, loads the virtual driver, and generates the speaker device corresponding to the screen sharing device.

[0094] In one embodiment, after receiving the virtual driver, the terminal device loads and generates a speaker device. This speaker device is disabled when not in use and is activated by the corresponding application when needed.

[0095] In the embodiments of this specification, by generating speaker device and speaker driver instances after the screen sharing device is connected to the terminal device, the amount of data stored on both the screen sharing device and the terminal device can be reduced, avoiding increased operational burden on the terminal device or screen sharing device when not in use. Furthermore, by generating speaker devices through virtual drivers, it is not necessary to enumerate the screen sharing device as a speaker device, nor is it necessary for the screen sharing device to transmit its own device description information. The terminal device can directly install the virtual driver to complete the installation of the speaker device corresponding to the screen sharing device.

[0096] Please refer to Figure 8, which is a flowchart illustrating an audio transmission method provided in an embodiment of this specification. The method may include the following steps S501-S505.

[0097] S501, when the first general interface is connected to the terminal device, establishes a DisplayPort transmission channel connecting the screen transmitter and the terminal device;

[0098] Specifically, please refer to step S401 in the above embodiment of the specification, which will not be repeated here.

[0099] S502, in response to the connection success command, calls the speaker driver instance to receive the first raw audio data from the DisplayPort transmission channel;

[0100] In one embodiment, the screen sharing device further includes a first memory storing speaker driver instances. When the two devices are successfully connected, the screen sharing device receives a connection success command. For example, the processor within the screen sharing device receives this command and invokes the speaker driver instance in the first memory. Then, the speaker driver instance receives first raw audio data from the DisplayPort transmission channel. The speaker driver instance is for internal processing within the screen sharing device. Specifically, in the screen sharing device, the speaker driver instance can be a USB audio device (such as a UAC driver).

[0101] S503, the terminal device sends a request command to the screen sharing device;

[0102] In one embodiment, the speaker device is generated after the two devices are interconnected. When the screen sharing device is plugged into the terminal device, it performs a USB enumeration process. During this process, the screen sharing device provides device description information (such as a USB descriptor) to describe what type of USB device it is. After receiving this descriptor, the terminal device's USB driver calls the corresponding device type driver, such as the speaker driver in this solution, to generate a USB speaker. Specifically, after the two devices are connected, the terminal device sends a request command to the screen sharing device to request its device description information.

[0103] S504, the screen sharing device responds to the request command from the terminal device by sending device description information to the terminal device;

[0104] In one embodiment, after receiving the request instruction, the screen sharing device parses the request instruction and sends the device description information requested by the terminal device to the terminal device. The screen sharing device can add a USB descriptor corresponding to the USB device type to be displayed to the device description information according to the desired function.

[0105] S505, the terminal device receives the device description information sent by the screen transmitter based on the request command, and calls the corresponding speaker driver to generate a speaker device based on the device description information.

[0106] In one embodiment, the terminal device invokes the corresponding speaker driver based on the device description information. This driver configures the screen sharing device, putting it in a configured state and generating a speaker device in the terminal device. It is understood that the terminal device pre-stores the corresponding drivers required by the USB device; therefore, it can invoke the appropriate driver based on the obtained device description information of the USB device.

[0107] In the embodiments described in this specification, by generating speaker device and speaker driver instances after the screen sharing device is connected to the terminal device, the amount of data stored on both the screen sharing device and the terminal device can be reduced, avoiding increased operational burden on the terminal device or screen sharing device when not in use. By sending the device description information of the screen sharing device, there is no need to transmit the driver program, and the speaker device can be quickly installed on the terminal device.

[0108] Please refer to Figure 9, which is a flowchart illustrating an audio transmission method provided in an embodiment of this specification. The method may include the following steps S601-S604.

[0109] S601, when the first general interface is connected to the terminal device, establishes a DisplayPort transmission channel connecting the screen transmitter and the terminal device;

[0110] S602, receives first raw audio data and first raw video data through the first general interface;

[0111] Specifically, when screen sharing is required, in addition to transmitting the first raw audio data collected by the speaker device, first raw video data can also be transmitted. For example, the first raw video data can be the screen display content of the terminal device. The screen display content can include the mouse cursor, operation page, presentation file, etc. on the screen. Specifically, the terminal device can include an audio capture device and a video capture device for capturing the images (video) and audio played on the terminal device, so that the screen sharing device can receive the first raw audio data and the first raw video data through the first general interface.

[0112] S603, when a screen sharing operation command is received and the terminal device selects a speaker device, the first original audio data and the first original video data are encoded respectively to obtain the encoded first audio data and the encoded first video data.

[0113] S604, transmit the first audio data and the first video data to the screen sharing receiver that is connected to the screen sharing device through the screen sharing channel.

[0114] In one embodiment, the first original audio data and the first original video data can be encoded separately on the screen sharing device side to obtain encoded first video data and first audio data, which are then sent to the screen sharing receiving end. Since the audio data and video data are processed in the same application, the screen sharing process is more convenient and accurate.

[0115] Furthermore, in one embodiment, before the first general interface is connected to the terminal device, the method further includes:

[0116] When the first general interface is connected to the screen sharing receiver, the connection information and function information of the screen sharing receiver are obtained. The function information includes whether it supports calling peripheral functions.

[0117] Specifically, the screen sharing device and the screen sharing receiver are pre-connected via a first universal interface, and the connection information and function information of the screen sharing device are stored in the screen sharing device. Please refer to step S201 in the above embodiment, which will not be repeated here.

[0118] Furthermore, in one embodiment, before sending the first raw audio data to the screen-sharing receiver via the peripheral device call channel after confirming that the screen-sharing receiving end supports the peripheral device call function, when the terminal device selects the speaker device and does not send a screen-sharing operation command, the process includes:

[0119] A communication connection is established with the screen sharing receiver based on the application layer protocol, and the connection is used to confirm whether the screen sharing receiver supports calling peripheral functions.

[0120] Specifically, the screen sharing device and the screen sharing receiver establish a communication connection through an application layer protocol. The application layer protocol defines how application processes running on different end systems exchange messages. The screen sharing device and the screen sharing receiver establish communication using a pre-agreed application layer protocol, and determine whether the screen sharing receiver supports calling peripheral functions through the communication connection.

[0121] Furthermore, in one embodiment, the screen sharing device connects to the terminal device based on the first general interface, and also enumerates the screen sharing device as a camera device to present to the terminal device;

[0122] Once it is confirmed that the screen sharing receiver supports calling peripheral functions, when the terminal device selects the camera device, it collects the second video data sent by the screen sharing receiver, decodes the second video data to obtain the second raw video data, and sends the second raw video data to the camera device.

[0123] Specifically, the screen sharing device can also be enumerated as a camera device to acquire second video data collected by the screen sharing receiver. This second video data can be video data collected by a camera built into or connected to the screen sharing receiver. For example, it could be a live feed from a conference room. The screen sharing receiver decodes the second video data to obtain second raw video data and transmits it to the camera device in the screen sharing device, such as a USB video device.

[0124] The camera device presents the second raw video data to the terminal device via the DisplayPort transmission channel, so that the second raw video data can be accessed by the terminal device.

[0125] Specifically, the camera device presents second raw video data to the terminal device via the DisplayPort transmission channel. When the camera device is selected on the terminal device, the second raw video data can be accessed.

[0126] Furthermore, in one embodiment, the screen sharing device is connected to the terminal device based on the first general interface, and the screen sharing device is also enumerated as a microphone device and presented to the terminal device.

[0127] Once it is confirmed that the screen sharing receiver supports calling peripheral functions, when the terminal device selects the microphone device, it collects the third audio data sent by the screen sharing receiver, decodes the third audio data to obtain the second original audio data, and sends the second original audio data to the microphone device.

[0128] Specifically, the screen sharing device can also be enumerated as a microphone device in the terminal device, thereby enabling the calling of the microphone device function of the screen sharing receiver. After confirming that the screen sharing receiver supports calling peripheral functions, when the terminal device selects the microphone device, it collects the third audio data sent by the screen sharing receiver, decodes the third audio data to obtain the second raw audio data, and sends the second raw audio data to the microphone device.

[0129] The microphone device presents a second raw audio data to the terminal device via the DisplayPort transmission channel, so that the second raw audio data can be accessed by the terminal device.

[0130] Specifically, the microphone device presents the second raw audio data to the terminal device via the DisplayPort transmission channel, and the terminal device then uses this second raw audio data. It should be noted that the third audio data can be acquired in real time by the screen mirroring receiver and transmitted through the screen mirroring device, thus enabling the terminal device to receive the real-time second raw audio data stream.

[0131] Please refer to Figure 10, which is a flowchart illustrating an audio transmission method provided in an embodiment of this specification. The method may include the following steps S701-S704.

[0132] S701, the first general interface is connected to the terminal device to establish a DisplayPort transmission channel connecting the screen transmitter and the terminal device;

[0133] In one embodiment, the screen sharing device has a first general interface that can be connected to a terminal device. When the terminal device is connected to the first general interface, it has a speaker device corresponding to the screen sharing device.

[0134] S702, when the unified communications software running on the terminal device does not receive the screen sharing operation command, in response to selecting the speaker device in the speaker settings option of the unified communications software, the screen sharing device is invoked to collect the third raw audio data of the unified communications software from the speaker device.

[0135] In one embodiment, the unified communications software can be conferencing software, voice call software, or other software with screen sharing functionality. The unified communications software displays speaker settings options, allowing the user to select the desired speaker, such as the speaker built into the terminal device, an external speaker connected to the terminal device, or the speaker corresponding to the screen sharing device. When the user does not want to share their screen but selects a speaker device, the screen sharing device acquires third-party raw audio data collected by the speaker device. This third-party raw audio data can be audio data from the unified communications software, such as the voices of other users in the meeting, thereby enabling the screen sharing receiver to play the sound from the terminal device.

[0136] S703, the received third raw audio data is encoded to obtain encoded third audio data;

[0137] Specifically, the screen transmitter encodes the third raw audio data to obtain the encoded third audio data.

[0138] S704 sends the third audio data to the screen sharing receiver connected in communication with the screen sharing device, so that the screen sharing receiver can decode the third audio data to obtain the third original audio data, and call the speaker of the screen sharing receiver to play the third original audio data.

[0139] Specifically, the screen sharing device sends the third audio data to the screen sharing receiver, so that the screen sharing receiver can decode the third audio data to obtain the third original audio data and play the third original audio data, thereby realizing the calling of the peripheral device of the screen sharing receiver.

[0140] In the embodiments of this specification, the first general interface of the screen sharing device is connected to the terminal device, establishing a DisplayPort transmission channel connecting the screen sharing device and the terminal device. When the unified communication software running on the terminal device does not receive a screen sharing operation command, in response to selecting a speaker device in the speaker settings of the unified communication software, the screen sharing device is invoked to collect the third raw audio data of the unified communication software from the speaker device. The received third raw audio data is encoded to obtain encoded third audio data, which is then sent to the screen sharing receiving end. The receiving end decodes the third audio data to obtain the third raw audio data and plays it. The terminal device runs unified communication software. When the unified communication software does not receive a screen sharing operation command, if a speaker device is selected, the screen sharing device acquires the third raw audio data of the unified communication software, encodes it, and transmits it to the receiving end for playback, thus realizing the function of calling peripheral devices. For the user, this function can be achieved simply by selecting the speaker according to actual needs, improving the user's interactive experience.

[0141] Please refer to Figure 11, which is a schematic diagram of the architecture of an audio transmission method provided in this specification embodiment. As shown in Figure 11, the red arrows indicate the data transmission path when the screen sharing device projects / calls peripherals. The terminal device is connected to the screen sharing device, which is enumerated in the terminal device as a speaker, camera, and microphone. The screen sharing device includes mass storage (first memory), a USB audio device (speaker driver instance), a USB video device, a screen capture device, network audio capture devices (1 and 2), and an audio distribution application. When the speaker is selected in the terminal device, the first raw audio data is transmitted to the USB port through the audio driver. A DP data transmission channel is established between the USB port of the terminal device and the USB port of the screen sharing device. The USB audio device of the screen sharing device receives the first raw audio data from the USB port. The speaker driver instance in the screen sharing device is connected to the ALSA (Advanced Linux Sound Architecture) subsystem. The first raw audio data is transmitted to the audio distribution application through ALSA, and a general ALSA operation interface is exposed to the application layer for receiving the second raw audio data transmitted by the application layer. When the audio distribution application receives a screen sharing operation command and the speaker is selected, it transmits the first raw audio data to the network audio collector 1. The audio is encoded by an audio encoder to obtain the first audio data (audio packet), and then transmitted to the application layer of the screen sharing receiver via an application layer protocol. The audio decoder at the screen sharing receiver decodes the first raw audio data. Optionally, when the audio distribution application receives a screen sharing operation command and the speaker is selected, it can also collect the first raw video data from the terminal device, transmit the first raw video data to the screen collector, encode it using a video encoder to obtain the first video data (video packet), and then transmit the first audio data and the first video data together to the wireless access point (WIFIAP) of the screen sharing receiver via an application layer protocol. The screen sharing receiver decodes the first raw audio data and the first raw video data using a video decoder and an audio decoder respectively. The first raw audio data is played through the speaker, and the first raw video data is displayed on the monitor. When the audio distribution application does not receive a screen sharing operation command and the speaker is selected, it transmits the first raw audio data to the network audio collector 2. The second audio data (audio packet) is encoded by the audio encoder and then transmitted to the application layer of the screen sharing receiver via the application layer protocol. The first raw audio data is then decoded by the audio decoder of the screen sharing receiver. Network audio collector 1 and network audio collector 2 can be the same collector; they are only shown in Figure 11 to distinguish their different functions.Optionally, when the screen sharing receiver supports calling peripheral functions, it can acquire second raw video data from the camera via a camera acquisition device, then encode it to obtain second video data; acquire second raw audio data from the microphone via a microphone acquisition device, then encode it to obtain third audio data; transmit it to the screen sharing device application layer via an application layer protocol; then decode it to obtain second raw audio data and second raw video data via an audio decoder and a video decoder respectively; send the second audio and video data to a USB audio device via ALSA; and send the second raw video data to a USB video device via V4L2 (V4L2 is short for Video for Linux2, a kernel driver for video devices in Linux); and finally transmit it to the camera and microphone of the terminal device via a DP transmission channel, thereby enabling the terminal device to call the microphone and camera of the screen sharing receiver.

[0142] Please refer to Figure 11 for a description of the screen sharing channel and peripheral device access channel in this application embodiment. The screen sharing channel and peripheral device access channel in this application are data transmission channels used by the screen sharing device to implement the two different functions of screen sharing and peripheral device access. When using the screen sharing function, the first original audio data is transmitted from the audio distribution application to the network audio collector 1 and then encoded by the audio encoder to obtain the first audio data. The screen sharing channel is the data transmission channel that then processes the first audio data through the application layer protocol and sends it to the WIFI transceiver, which in turn sends it to the screen sharing receiving end. When transmitting through this screen sharing channel, if there is a video transmission requirement, after obtaining video data from the screen collector and encoding it with the video encoder, the screen sharing channel can also include a data transmission channel that processes the first video data through the same application layer protocol and then sends it to the WIFI transceiver, which then transmits the audio data and video data together to the screen sharing receiving end. When using the peripheral calling function, the first raw audio data is sent from the audio distribution application to the network audio collector 2, where it is encoded by the audio encoder to obtain the second audio data. The peripheral calling channel includes the second audio data being processed by the application layer protocol and then sent to the WIFI transceiver, which then transmits it to the screen sharing receiver. It is understandable that in the specific implementation, the audio encoder, the application layer protocol used, and the WIFI transceiver in the screen sharing device can all be the same; however, different modules can also be used for different data transmission channels.

[0143] In this embodiment, the screen sharing channel and the peripheral device calling channel can be implemented by designing different applications in the screen sharing device. For example, the screen sharing device includes a first memory, which stores an audio distribution application, a screen sharing application, and a peripheral device calling application. The following description, in conjunction with FIG12, illustrates the method, which may include the following steps S801-S806.

[0144] S801, the screen sharing device is connected to the terminal device through the first general interface;

[0145] In one embodiment, the wireless screen sharing system includes a screen sharing device and a screen sharing receiver. Optionally, the terminal device includes a second universal interface and a speaker device corresponding to the screen sharing device. The second universal interface is an interface in the terminal device that can connect to the first universal interface. Both the first and second universal interfaces can be USB Type-C interfaces.

[0146] The screen sharing device and the screen sharing receiver can communicate with each other. Specifically, before the screen sharing device is inserted into the terminal device, it can connect to the screen sharing receiver's general interface through its first general interface. After connection, the screen sharing receiver sends its connection information to the screen sharing device. This connection information includes the screen sharing receiver's hotspot information, such as the hotspot name and password. After pairing and connection, when the screen sharing device is inserted into the terminal device, it can establish a communication connection with the screen sharing receiver based on this connection information. This allows the terminal device to communicate with the screen sharing receiver through the screen sharing device, for example, by transmitting audio or video data from the terminal device to the screen sharing receiver for display.

[0147] An audio distribution application is an application used to manage audio data. Audio data in terminal devices is transmitted to the audio distribution application through a first general interface and managed uniformly by the audio distribution application.

[0148] A screen sharing application is a software or function used to wirelessly transmit the screen content of a device (such as a mobile phone, tablet, or computer) to other display devices (such as a TV or projector). The screen sharing application runs on the screen sharing device; once the device is connected to the terminal device, the application stored in its memory automatically runs.

[0149] A peripheral device access application is software used to manage and access external devices such as displays, microphones, and cameras. By accessing peripheral device access applications, participants can utilize the accessibility of the receiving device (such as a display, microphone, and camera), improving meeting interactivity and efficiency. Furthermore, peripheral device access applications allow users to easily access the peripheral capabilities of meeting rooms or classrooms using their own devices, reducing technical issues and setup time. Participants using devices from different platforms can seamlessly join meetings through peripheral device access applications, whether on Windows, macOS, or mobile devices. Peripheral device access applications have corresponding peripheral access channels for transmitting data to the receiving device.

[0150] Among them, the channel used to transmit screen sharing data in the audio distribution application and the screen sharing application can be used as the screen sharing channel, and the channel used to transmit data for calling peripheral devices in the audio distribution application and the application for calling peripheral devices can be used as the calling peripheral device channel.

[0151] Optionally, in one embodiment, the first memory may also store a comprehensive application and an audio distribution application. The comprehensive application combines the functions of a screen sharing application and a peripheral device calling application. When a user needs to use the screen sharing function in the comprehensive application, data is transmitted to the screen sharing receiver through the screen sharing channel. When a user needs to use the peripheral device calling function in the comprehensive application, data is transmitted to the screen sharing receiver through the peripheral device calling channel.

[0152] It is understandable that the screen sharing application and the peripheral device calling application can be presented as two separate pieces of code in the screen sharing device, which implement the screen sharing and peripheral device calling functions, or they can be presented as a single piece of code that implements both screen sharing and peripheral device calling functions simultaneously.

[0153] Among them, the channel used for transmitting screen sharing data in the audio distribution application and the integrated application can be used as the screen sharing channel, and the channel used for transmitting peripheral call data in the audio distribution application and the integrated application can be used as the peripheral call channel.

[0154] Optionally, in one embodiment, the first memory may only store a comprehensive application program. This comprehensive application program combines the functions of an audio distribution application, a screen sharing application, and a peripheral device calling application, and is capable of processing and forwarding the received first raw audio data. Specifically, when the comprehensive application program receives a screen sharing operation command and responds to the terminal device selecting a speaker device, it processes the first raw audio data to obtain first audio data, and sends the first audio data to the screen sharing receiving end communicatively connected to the screen sharing device through the screen sharing channel. If the screen sharing receiving end supports the peripheral device calling function, when the comprehensive application program does not receive a screen sharing operation command and responds to the terminal device selecting a speaker device, it processes the first raw audio data to obtain second audio data, and sends the second audio data to the screen sharing receiving end through the peripheral device calling channel.

[0155] Among them, the channel used to transmit screen sharing data in the integrated application can be used as the screen sharing channel, and the channel used to transmit data for calling peripherals in the integrated application can be used as the calling peripherals channel.

[0156] The first universal interface of the screen sharing device can be connected to the second universal interface of the terminal device. For example, by connecting the first universal interface of the screen sharing device to the second universal interface of the terminal device, a DisplayPort (DP) transmission channel is established between the screen sharing device and the terminal device, enabling communication between them. There are two main types of DP interfaces: the standard DP interface and the Type-C interface. Preferably, the first universal interface is a USB Type-C interface. Correspondingly, the second universal interface is also a USB Type-C interface. After the screen sharing device is connected to the terminal device, the terminal device starts or generates a speaker device corresponding to the screen sharing device. Specifically, this speaker device can be a UAC (USB Audio Class) device. UAC is a USB audio device category used for controlling and transmitting audio data. Since UVC defines standard video transmission protocols and data formats, the operating system usually provides built-in UVC drivers to support UVC devices, eliminating the need for additional driver installation. That is, after the screen sharing device is plugged into the terminal device, the terminal device's operating system can automatically recognize the screen sharing device and load the corresponding driver to generate a new speaker device.

[0157] In one feasible implementation, the terminal device first detects the speed type of the inserted screen sharing device to determine whether it is a high-speed or low-speed device. Then, the terminal device resets the connected device and checks whether the connected full-speed device supports high-speed mode. If it supports high-speed mode, it switches to high-speed signal mode. After the reset, the terminal device uses a default address (e.g., 0) agreed upon in the protocol to initiate initial communication with the currently connected device. This initial communication primarily obtains the type of the connected device. During this initial communication, the terminal device sends a request command to the device corresponding to the default address. This request command requests the device information of the currently connected device; in the application scenario of this solution, it requests the device information of the screen sharing device. The device information mainly describes the device type, such as HID (Human Interface Device), mass storage device, multifunction device, microphone, speakerphone, loudspeaker, display, touchscreen, projector, camera, camcorder, or webcam. It may also describe the device's operating parameters, such as rated operating voltage. Depending on the functions of the screen sharing device, the terminal device may describe one or more types. Based on the obtained device information, the terminal device calls the corresponding device type driver. For example, in this solution, the terminal device's speaker driver is called to generate the speaker device on the terminal device side.

[0158] In another feasible implementation, the terminal device can pre-install a speaker device corresponding to the screen sharing device, that is, the speaker device is generated through a virtual driver in the terminal device. After the virtual driver is installed, a speaker device is generated. This speaker device has fixed attributes and does not require additional information, such as audio sampling rate and stereo. The speaker device is disabled when it is not needed, and is started by the application when it is needed. This application can be a conferencing application or a screen sharing application on the terminal device, used to start or stop the speaker device generated by the virtual driver, and to transfer audio data to the corresponding place that needs the data (such as an audio distribution application). It is understood that the virtual driver can be pre-installed in the terminal device, or it can be sent to the terminal device after the screen sharing device is connected.

[0159] S802, in response to the selection of the speaker device of the terminal device, the first raw audio data of the terminal device is received through the speaker driver instance of the screen mirroring device;

[0160] In one embodiment, when a speaker device in the terminal device is selected, the speaker device acquires the first raw audio data of the terminal device and then transmits it to the screen sharing device via the DisplayPort transmission channel. Specifically, the first raw audio data may be received by a speaker driver instance in the screen sharing device. The speaker driver instance in the screen sharing device may be pre-installed in the screen sharing device or installed in the screen sharing device after the screen sharing device is connected to the terminal device; the specific implementation is not limited. It is understood that since this is the only new speaker device displayed to the user in the terminal device, the user can clearly know that selecting this speaker device will enable the function. After selecting this speaker device, the user can use the screen sharing function and call the peripheral device function, and both functions are implemented through this speaker device without the need to switch speaker devices. It should be noted that when the speaker device is selected, the terminal device may also not acquire the audio data of the terminal device initially, but wait for the screen sharing application and / or the peripheral device application to request audio data before acquiring the audio data. Optionally, during screen sharing, in addition to transmitting the audio data acquired by the speaker device, video data may also be transmitted; for example, the video data may be the screen display content of the terminal device. The content displayed on the screen can include the mouse cursor, the page being operated, and presentation files.

[0161] S803, the speaker driver instance sends the first raw audio data to the audio distribution application;

[0162] In one embodiment, after receiving the first raw audio data, the speaker driver instance in the screen sharing device sends the first raw audio data to the audio distribution application in the screen sharing device. For example, the speaker driver instance can be a User Account Control driver, i.e., a UAC speaker driver. The UAC speaker driver in the screen sharing device connects to the ALSA (Advanced Linux SoundArchitecture) subsystem, exposing a generic ALSA operation interface to the application layer. Specific functions of ALSA include audio device management, audio stream playback and recording, support for multi-channel audio, audio mixing, and support for MIDI devices. It also provides advanced audio processing interfaces, allowing developers to perform audio programming in applications.

[0163] S804, the audio distribution application responds to the request instruction sent by the screen sharing application and / or the calling peripheral device application, and determines the target application to which the first raw audio data is to be distributed based on the request instruction;

[0164] In one embodiment, both the screen sharing application and the peripheral device calling application need to request audio data. When a user needs to use the peripheral device calling function, the peripheral device calling application sends a request command to the audio distribution application. When a user needs to use the screen sharing function, the screen sharing application sends a request command to the audio distribution application. The audio distribution application distributes the audio data according to the received request command. The target application is either the screen sharing application or the peripheral device calling application. It should be noted that when the audio distribution application receives request commands from both the screen sharing application and the peripheral device calling application simultaneously, it prioritizes transmitting the audio data to the screen sharing application according to a pre-set priority rule. It should also be noted that the transmitted audio data can be processed or unprocessed; for example, the audio data can be encoded and / or optionally encrypted. Of course, in other embodiments, the priority of the peripheral device calling application can be set higher than that of the screen sharing application, or different strategies can be used, such as dynamically switching the priority order based on the priority of the currently open windows on the terminal device. No specific limitations are imposed.

[0165] S805, the audio distribution application distributes the first raw audio data to the target application;

[0166] In one embodiment, after determining the target application based on the request instruction, the audio distribution application sends audio data to the target application. If the target application is in the screen sharing device, the screen sharing device sends it to the corresponding screen sharing receiver. If the target application is in the terminal device, the terminal device first transmits the audio data to the screen sharing device, and then the screen sharing device transmits it to the screen sharing receiver.

[0167] S806, the target application encodes the first raw audio data to obtain the first audio data, and transmits the first audio data to the screen sharing receiver connected in communication with the screen sharing device.

[0168] In one embodiment, the target application encodes the acquired audio data to obtain a first audio data transmission to the screen sharing receiver. It is understood that, depending on the function of the target application, it may transmit only audio data or simultaneously transmit audio and video data from the terminal device to the screen sharing receiver. The screen sharing receiver is typically a peripheral device (such as a conference room or classroom display device, screen sharing box, etc.) pre-paired with the screen sharing device and communicating with it using near-field communication methods such as hotspots. Optionally, the target application may also directly transmit the first raw audio data to the screen sharing receiver without processing it.

[0169] Please refer to Figure 13, which is a flowchart illustrating another audio transmission method provided in this embodiment. The terminal device is connected to the screen sharing device. When the user selects a speaker device, taking a USB Type-C interface as an example (first and second universal interfaces), the terminal device can first enable the UAC protocol with the screen sharing device for data transmission. At this time, both the screen sharing application and the peripheral device application can send request commands to the audio distribution application in the screen sharing device. The audio distribution application confirms the target application based on the received request command. Then, the terminal device collects audio data and sends it to the audio distribution application on the screen sharing device side (the step of the speaker driver instance receiving the data first is omitted here). When the first and second universal interfaces are USB Type-C interfaces, both the screen sharing application and the peripheral device application run on the screen sharing device. The audio distribution application can distribute audio data to the screen sharing application or the peripheral device application according to the user's actual operation. Upon receiving the data, the screen sharing application or the peripheral device application sends it to the screen sharing receiver. The screen sharing receiver then uses its own or an external speaker to play the audio data.

[0170] Please refer to Figure 14, which is a flowchart illustrating an audio transmission method provided in an embodiment of this specification. As shown in Figure 14, this embodiment of the specification describes the specific flow of the audio transmission method from the perspective of a wireless screen sharing system. The method may include the following steps S901-S909.

[0171] S901, the screen sharing device is connected to the second general interface of the terminal device through the first general interface;

[0172] After the first and second universal interfaces are established, a DisplayPort transmission channel is created connecting the screen sharing device and the terminal device. Other related descriptions are given in step S101 of the embodiments described above, and will not be repeated here. Of course, in other embodiments, the types of the first and second universal interfaces can be different. For example, the first universal interface can be a USB Type-C interface, and the second universal interface can be a USB Type-A interface. In this case, an external interface converter can be connected to the terminal device to achieve the connection between the screen sharing device and the terminal device.

[0173] S902, the screen sharing device connects to the terminal device via the first general interface and runs the screen sharing application;

[0174] Specifically, in DisplayPort transmission channel mode, the screen sharing application runs directly in the screen sharing device. That is, when the screen sharing device is connected to the terminal device, the screen sharing application stored in the memory will run automatically.

[0175] S903, in response to the selection of the speaker device corresponding to the screen sharing device, the first raw audio data of the terminal device is obtained through the speaker driver instance of the screen sharing device;

[0176] S904, the speaker driver instance sends audio data to the audio distribution application;

[0177] Specifically, please refer to steps S102-S104 in the embodiments of the above specification, which will not be repeated here.

[0178] S905, the audio distribution application responds to the screen sharing request instruction sent by the screen sharing application and determines the target application to which the first original audio data is to be distributed as the screen sharing application according to the screen sharing request instruction.

[0179] In one embodiment, when the audio distribution application receives a screen sharing request instruction from the screen sharing application, it determines that the target application is the screen sharing application. In this embodiment, the screen sharing application runs on the screen sharing device, therefore the received screen sharing request instruction is obtained through internal communication of the screen sharing device. In another embodiment, the screen sharing application can run on the terminal device, in which case the screen sharing request instruction can be sent through the virtual network card of the terminal device.

[0180] S906, the audio distribution application responds to the peripheral device request instruction sent by the peripheral device request instruction and determines the target application to which the first raw audio data is to be distributed as the peripheral device request instruction;

[0181] In one embodiment, when the audio distribution application receives a request instruction to invoke a peripheral device from the application that invokes a peripheral device, it determines that the target application is the application that invokes a peripheral device. The application that invokes a peripheral device is installed in the screen sharing device, so that users do not need to install the application that invokes a peripheral device on their terminal devices; they can simply plug in the screen sharing device to use the function of invoking a peripheral device, that is, to realize hardware screen sharing.

[0182] S907, the audio distribution application responds to the screen sharing request instruction sent by the screen sharing application and the call peripheral device request instruction sent by the call peripheral device application, and determines the target application to which the first original audio data is to be distributed as the screen sharing application according to the preset request priority.

[0183] In one embodiment, when the audio distribution application receives both a screen sharing request instruction and a peripheral device call request instruction, the screen sharing request instruction has a higher priority than the peripheral device call request instruction according to the preset request priority. Therefore, the target application is determined to be the screen sharing application.

[0184] It should be noted that steps S605-S607 are parallel schemes, and the corresponding target application determination steps can be executed according to the different request instructions received by the audio distribution application.

[0185] S908, the audio distribution application distributes the first raw audio data to the target application;

[0186] In one embodiment, after identifying the target application, the audio distribution application distributes the audio data to the corresponding target application.

[0187] S909, the target application transmits the first raw audio data to the screen mirroring receiver.

[0188] In one embodiment, the target application transmits the audio data to the screen sharing receiver. Optionally, the target application may also transmit other data required for screen sharing to the screen sharing receiver while transmitting the audio data; no specific limitations are imposed.

[0189] In the embodiments of this specification, the screen sharing device is connected to a first general interface and a second general interface to establish a DisplayPort transmission channel connecting the screen sharing device and the terminal device. The first general interface of the screen sharing device is connected to the terminal device, and a screen sharing application runs. In response to the selection of a speaker device, the speaker driver instance of the screen sharing device obtains audio data from the terminal device. The terminal device transmits the audio data to the speaker driver instance of the screen sharing device through the second general interface. The speaker driver instance sends the audio data to the audio distribution application. The audio distribution application responds to the screen sharing request instruction sent by the screen sharing application and determines the target application to which the audio data is to be distributed as the screen sharing application. The audio distribution application responds to the call peripheral device request instruction sent by the call peripheral device application and determines the target application to which the audio data is to be distributed as the call peripheral device application. The audio distribution application responds to the screen sharing request instruction sent by the screen sharing application and the call peripheral device request instruction sent by the call peripheral device application, and determines the target application to which the audio data is to be distributed as the screen sharing application according to a preset request priority. The audio distribution application distributes the audio data to the target application, and the target application transmits the audio data to the screen sharing receiver.

[0190] Optionally, in one embodiment, the first universal interface is a USB Type-C interface, and the second universal interface is a USB Type-A interface. To connect the screen sharing device to the computer's USB Type-A interface, a USB Type-C to USB Type-A converter can be connected to the screen sharing device's USB Type-C interface to achieve communication with the computer's USB Type-A interface. Since the USB Type-C interface is compatible with the USB 2.0 protocol's transmission mode, and the data transmission mode of the USB 2.0 protocol differs from that of the DP protocol, this embodiment's screen sharing device incorporates another data transmission scheme based on the USB Type-A interface, allowing it to simultaneously support both DP and USB 2.0 communication protocols. When the user connects the screen sharing device and the computer using a USB Type-C to USB Type-A converter, the connection automatically switches to the USB 2.0 communication protocol's transmission mode.

[0191] When a terminal device connects to a screen sharing device via an interface converter, the terminal device can first enable the UAC protocol with the screen sharing device for data transmission. At this time, both the screen sharing application and the peripheral device application can send request commands to the audio distribution application in the screen sharing device. The audio distribution application confirms the target application based on the received request command. Then, the terminal device collects audio data and sends it to the audio distribution application on the screen sharing device side (the step of the speaker driver instance receiving the data first is omitted here). The screen sharing application runs on the terminal device and is used to collect video and / or audio data from the terminal device. The peripheral device application runs on the screen sharing device side and obtains video and / or audio data from the terminal device through an interface. The audio distribution application can distribute audio data to either the screen sharing application or the peripheral device application based on the user's actual operation. If distributed to the screen sharing application, it needs to be sent back to the screen sharing application running on the computer side through the interface. If distributed to the peripheral device application, it can be directly distributed to the peripheral device application running on the screen sharing device side without needing to be sent back to the terminal device.

[0192] Please refer to Figure 15, which is a flowchart illustrating an audio transmission method provided in an embodiment of this specification. The method may include the following steps S1001-S1015.

[0193] S1001, the screen sharing device is connected to the second general interface of the terminal device through the first general interface;

[0194] The first general interface and the second general interface are connected and use the USB data transmission channel. For other related descriptions, please refer to step S101 in the above embodiment of the specification, which will not be repeated here.

[0195] S1002, Based on the second general interface, a virtual network card is created in the terminal device, and a communication connection is established between the virtual network card and the network device;

[0196] In one embodiment, when using a USB data transmission channel, the screen sharing device includes a network device, and the terminal device includes a virtual network interface card (NIC) corresponding to the network device. Specifically, the network device can be a USB network device within the screen sharing device; similarly, the virtual NIC can also be a USB network device within the terminal device. Based on the connection between the second general interface and the virtual NIC, a virtual NIC corresponding to the network device is created in the terminal device.

[0197] S1003, the screen sharing device sends the pre-created network card driver corresponding to the network device to the terminal device;

[0198] S1004, The terminal device receives the network card driver based on the second general interface and generates a virtual network card based on the network card driver;

[0199] In one embodiment, the virtual network card can be generated by a network card driver pre-created by the screen sharing device. The screen sharing device pre-stores network card drivers corresponding to network devices and sends the network card drivers to the terminal device through a first general interface. The terminal device receives the network card drivers based on a second general interface, thereby communicating with the screen sharing device.

[0200] Among them, steps S1002 and S1003-S1004 are parallel steps, that is, the terminal device can execute step S1002 to create a network card by itself or call the virtual driver of the screen sharing device to generate a virtual network card in steps S1003-S1004.

[0201] S1005, establishes a communication connection between the virtual network card and the network device;

[0202] Specifically, a communication connection is established between the virtual network card and the network device through bridging or network address translation.

[0203] S1006, The screen sharing device sends the screen sharing application to the terminal device based on the first general interface;

[0204] In one embodiment, the first general interface is a USB Type-A interface. When the first general interface is a USB Type-A interface, the screen sharing device sends the stored screen sharing application to the terminal device through the USB communication channel established by the USB Type-A interface.

[0205] S1007, The terminal device receives and runs the screen sharing application;

[0206] In one embodiment, the terminal device receives and runs a screen sharing application sent by the screen sharing device. Specifically, the screen sharing application is used to acquire screen sharing control data and media data on the computer, and transmit the screen sharing control data and media data to the screen sharing device via a virtual network card. Screen sharing control data consists of instruction data corresponding to control operations during the screen sharing process, such as starting screen sharing, pausing screen sharing, window screen sharing, extended screen screen sharing, or privacy processing. These control operations may have different control objects. For example, starting screen sharing might be executed locally on the terminal device if the user presses the application's switch, or locally on the screen sharing device if the user presses a button on the screen sharing device. Pausing might be executed locally on the computer, while privacy processing might require execution by the screen sharing device. For control operations not executed locally, the corresponding screen sharing control data is transmitted to the screen sharing device via a virtual network card. Media data consists of audio and video data on the terminal device.

[0207] In one feasible implementation, during the transfer of the screen sharing application, the screen sharing device can be enumerated as a storage device (USB flash drive). The terminal device can display the drive letter corresponding to the type of storage device. When the terminal device detects that a user has opened the storage directory of the storage device via the drive letter, the content stored in the storage device is displayed, such as the icon corresponding to the screen sharing application. Upon receiving a double-click or right-click operation on the screen sharing application icon, the screen sharing device loads the screen sharing application into the terminal device's memory through the communication channel between the terminal device and the storage device, for execution by the terminal device's central processing unit, thereby obtaining the screen sharing control data and media data generated by the terminal device during the screen sharing process. In other embodiments, the screen sharing device's memory can work with a file transfer application installed on the terminal device to send the screen sharing application to the terminal device.

[0208] S1008, The terminal device generates a screen sharing request command based on the screen sharing application, and sends the screen sharing request command to the audio distribution application through the screen sharing application.

[0209] In one embodiment, the screen-sharing application in the terminal device can generate a screen-sharing request command, which is used to request audio data from the audio distribution application. For example, the screen-sharing application has an interface button that the user can click, and the user can trigger the sending of the screen-sharing request command by clicking the interface button. Of course, the screen-sharing device can also be equipped with a button that the user can press. When the user wants to share the screen, they can press the button, and the screen-sharing application can generate a screen-sharing request command after receiving the button operation. The screen-sharing application can send the screen-sharing request command to the virtual network card, which then sends it to the network device of the screen-sharing device, and the network device then sends it to the audio distribution application. Optionally, the terminal device sends the collected audio data and the screen-sharing request command (used to request audio data) through the USB speaker in the terminal device to the USB audio device of the screen-sharing device, and then the USB audio device sends it to the audio distribution application.

[0210] S1009, in response to the speaker device corresponding to the screen sharing device being selected, the first raw audio data of the terminal device is obtained through the speaker driver instance of the screen sharing device;

[0211] S1010, the terminal device transmits audio data to the speaker driver instance of the screen mirroring device through the second general interface;

[0212] S1011, The speaker driver instance sends the first raw audio data to the audio distribution application;

[0213] Specifically, please refer to steps S102-S104 in the embodiments of the above specification, which will not be repeated here.

[0214] S1012, the audio distribution application responds to the screen sharing request instruction sent by the screen sharing application and distributes audio data to the network device, and the network device distributes the first raw audio data to the virtual network card.

[0215] Specifically, the screen sharing device includes a network device, or a USB network device, and correspondingly, the terminal device includes a virtual network card, also called a USB network device. The two can communicate with each other. After receiving the screen sharing request command from the screen sharing application on the terminal device, the audio distribution application distributes the audio data to the network device, which then distributes the audio data to the virtual network card on the terminal device.

[0216] S1013, The terminal device receives the first raw audio data distributed by the network device based on the virtual network card;

[0217] S1014, the virtual network card forwards the first raw audio data to the screen sharing application;

[0218] Specifically, after the virtual network card in the terminal device receives the audio data, it forwards the audio data to the screen sharing application.

[0219] S1015, the screen sharing application receives the first raw audio data sent by the audio distribution application, encodes the first raw audio data to obtain the first audio data, and transmits the first audio data to the screen sharing receiver connected in communication with the screen sharing device.

[0220] Specifically, the screen sharing application receives the audio data, encodes it to obtain first audio data, and then sends the encoded first audio data to the corresponding screen sharing receiver. The screen sharing receiver is paired with the screen sharing device and can establish a communication connection with the screen sharing device when the screen sharing device is plugged into the terminal device.

[0221] Optionally, in one embodiment, receiving first raw audio data sent by an audio distribution application, calling a screen sharing application to encode the first raw audio data to obtain second audio data, and transmitting the second audio data to the screen sharing receiving end, includes:

[0222] S10151, Collect first raw video data based on screen sharing application;

[0223] Specifically, when sharing a screen, the application can transmit not only audio data collected by the speaker device but also video data. For example, the video data can be the content displayed on the terminal device's screen. The screen display content can include the mouse cursor, operation page, presentation files, etc. Specifically, the terminal device can include an audio capture device and a video capture device to capture the images (video) and audio played on the terminal device, thus obtaining the video data required for screen sharing. Understandably, the audio data received by the screen sharing application is mainly the audio collected by the speaker device, such as the user's speech audio captured by the microphone, while the audio data from the terminal device needs to be additionally captured, such as the audio data of the video or music playing on the terminal device.

[0224] S10152, The first original audio data and the first original video data are encoded based on the screen sharing application to obtain the encoded first audio data and the encoded first video data.

[0225] Specifically, when video data needs to be transmitted, the first original audio data and the first original video data can be encoded on the screen sharing application side. Since the audio data and video data are processed in the same application, the audio and video synchronization is better, making the screen sharing process more convenient and accurate, and meeting the requirement of good screen sharing synchronization.

[0226] S10153, Invoke the screen sharing application to transmit the first audio data and the first video data to the screen sharing receiver.

[0227] Specifically, after acquiring the first audio data and the first video data, the screen sharing application transmits the first audio data and the first video data together to the screen sharing receiving end. For example, this can be done via network transmission, which is a well-known method and will not be elaborated upon here.

[0228] In the embodiments described in this specification, when the first general interface is connected to the terminal device, a USB data transmission channel is established between the screen sharing device and the terminal device. After the screen sharing device is connected to the terminal device, it transmits a screen sharing application to the terminal device through the first general interface. Upon receiving the screen sharing application, the terminal device runs it and sends a screen sharing request command to the audio distribution application through the application. After receiving the request command, the audio distribution application forwards the audio data collected by the speaker device to the screen sharing application in the terminal device. The screen sharing application in the terminal device then transmits the audio data to the receiving end. The screen sharing device and the terminal device can communicate via a network device. Specifically, the screen sharing device includes a network device, and the terminal device includes a virtual network card. This virtual network card can be generated based on the network card driver sent by the screen sharing device to the terminal device, or it can be created by the terminal device itself based on the network device, thus meeting the communication needs between the screen sharing device and the terminal device. Receiving request commands and audio data from the terminal device through this transmission channel improves the transmission rate.

[0229] Please refer to Figure 16, which is a schematic diagram of the architecture of an audio transmission method provided in an embodiment of this specification. As shown in Figure 16, the blue arrows represent the data transmission path when the terminal device projects its screen / calls peripherals, and the red arrows represent the data transmission path when the screen projector projects its screen / calls peripherals. In one feasible implementation, when the first general interface is connected to the terminal device, a USB data transmission channel can also be established between the screen projector and the terminal device. In this case, the terminal device needs to communicate with the USB network device in the screen projector through a USB network device, that is, it needs to transmit audio and video data through the network card channel. The terminal device has a screen sharing function. When a screen sharing operation command is triggered on the terminal device, the audio distribution application receives the screen sharing operation command and responds by selecting the speaker device on the terminal device. It collects the first raw audio data through the first general interface and then sends it to the audio distribution application through the USB audio device and ALSA. In response to the screen sharing operation command transmitted by the terminal device, the audio distribution application sends the first raw audio data to the screen sharing USB network device. The screen sharing USB network device sends the first raw audio data to the terminal device's USB network device through the USB data transmission channel of the USB port. After that, the USB network device sends the first raw audio data to the network audio collector 3. The terminal device completes the audio encoding and obtains the first raw video data from the video storage through the screen collector. The encoded video packet (first video data) is encapsulated with the audio packet through the application layer protocol and sent back to the screen sharing device through the USB network device. After receiving the video packet, the screen sharing USB network device sends it to the screen sharing receiver through the WIFI transceiver. The screen sharing receiver decodes the video data and the first raw audio data and displays them on the display and the speaker, respectively. When the receiving end of the screen sharing device supports calling peripheral functions and the screen sharing device does not receive a screen sharing operation command, in response to the terminal device selecting a speaker device, the audio distribution application will send data to the screen sharing device's USB network device. The screen sharing device's USB network device will send the first raw audio data to the terminal device's USB network device through the USB data transmission channel of the USB port. Then, the USB network device will send the first raw audio data to the network audio collector 3. After the terminal device completes the audio encoding and obtains the second audio data, it will send it back to the screen sharing device. The screen sharing device will then send the second audio data to the WIFI transceiver, which will then forward it to the receiving end of the screen sharing device. Figure 16 shows the transmission process when the screen sharing device and the terminal device establish a DisplayPort data transmission channel. The data stream corresponding to the red arrow can be found in Figure 11 of the above-mentioned embodiment, and will not be repeated here.

[0230] This specification also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the audio transmission method as shown in the embodiments of Figures 1-16 above. For the specific execution process, please refer to the detailed description of the embodiments shown in Figures 1-16, which will not be repeated here.

[0231] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0232] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. An audio transmission method, characterized in that, The method includes: A screen sharing device, the screen sharing device including a first general interface for connecting to a terminal device, the terminal device having a speaker device corresponding to the screen sharing device, the speaker device being used to acquire first raw audio data from the terminal device; When the first general interface is connected to the terminal device, a DisplayPort transmission channel is established connecting the screen sharing device and the terminal device, and the first raw audio data is received through the DisplayPort transmission channel; When a screen sharing operation command is received and the terminal device selects the speaker device, the first original audio data is processed to obtain the first audio data, and the first audio data is sent to the screen sharing receiver connected to the screen sharing device through the screen sharing channel. When the screen sharing receiver supports calling peripheral functions and the screen sharing device does not receive the screen sharing operation instruction, in response to the terminal device selecting the speaker device, the first original audio data is processed to obtain the second audio data, and the second audio data is sent to the screen sharing receiver by calling the peripheral channel.

2. The method as claimed in claim 1, characterized in that, The screen display device also includes a first memory, which stores a speaker driver program. Receiving the first raw audio data through the DisplayPort transmission channel includes: In response to a successful connection command, the speaker driver is invoked to generate a speaker driver instance; The first raw audio data is received from the Display Port transmission channel via the speaker driver instance.

3. The method as described in claim 1, characterized in that, The screen display device also includes a first memory, which stores speaker driver instances; Receiving the first raw audio data through the DisplayPort transmission channel includes: In response to a connection success command, the speaker driver instance is invoked to receive the first raw audio data from the DisplayPort transmission channel.

4. The method as claimed in claim 1, characterized in that, When the first general interface is connected to the terminal device, it further includes: In response to a request instruction from the terminal device, device description information is sent to the terminal device so that the terminal device can call the corresponding speaker driver to generate a speaker device based on the device description information.

5. The method as claimed in claim 1, characterized in that, The screen sharing device also includes a first memory, which stores a virtual driver program. When the first general interface is connected to the terminal device, after establishing a DisplayPort transmission channel connecting the screen transmitter and the terminal device, the method further includes: The virtual driver is sent to the terminal device via the Display Port transmission channel, so that the terminal device can generate a speaker device based on the virtual driver.

6. The method as claimed in claim 1, characterized in that, Receiving the first raw audio data through the DisplayPort transmission channel includes: The first raw audio data and the first raw video data are received through the first general interface; When a screen sharing operation command is received and the terminal device selects the speaker device, sending the first original audio data to the screen sharing receiver connected to the screen sharing device via the screen sharing channel includes: When a screen sharing operation command is received and the terminal device selects the speaker device, the first original audio data and the first original video data are encoded respectively to obtain the encoded first audio data and the encoded first video data. The first audio data and the first video data are transmitted to the screen sharing receiver, which is communicatively connected to the screen sharing device, through the screen sharing channel.

7. The method as claimed in claim 1, characterized in that, Before the first general interface is connected to the terminal device, the method further includes: When the first general interface is connected to the screen sharing receiver, the connection information and function information of the screen sharing receiver are obtained, including whether the function information supports calling peripheral functions.

8. The method of claim 7, characterized in that, When the screen sharing receiver supports calling peripheral functions and the screen sharing device does not receive the screen sharing operation instruction, before sending the first original audio data to the screen sharing receiver through the peripheral calling channel in response to the terminal device selecting the speaker device, the process includes: A communication connection is established with the screen sharing receiver based on the application layer protocol, and the screen sharing receiver is confirmed to support calling peripheral functions based on the communication connection.

9. The method according to claim 1, characterized in that, The screen sharing device connects to the terminal device based on the first general interface, and also enumerates the screen sharing device as a camera device to present to the terminal device; When the screen sharing receiver supports calling peripheral functions, in response to the terminal device selecting the camera device, the transmitted data is captured. The screen receives the second video data sent by the receiving end, decodes the second video data to obtain the second original video data, and sends the second original video data to the camera device; The camera device presents the second raw video data to the terminal device via the DisplayPort transmission channel, so that the second raw video data can be accessed by the terminal device.

10. The method according to claim 1 or 9, characterized in that, Based on the connection between the screen sharing device and the terminal device via the first general interface, the screen sharing device is also enumerated as a microphone device and presented to the terminal device. When the screen sharing receiver supports calling peripheral functions, in response to the terminal device selecting the microphone device, the receiver collects the third audio data sent by the screen sharing receiver, decodes the third audio data to obtain the second original audio data, and sends the second original audio data to the microphone device. The microphone device presents the second raw audio data to the terminal device via the DisplayPort transmission channel, so that the second raw audio data can be accessed by the terminal device.

11. An audio transmission method, characterized in that, Applied to a terminal device, the terminal device including a second general interface and a speaker device corresponding to the screen sharing device, the method includes: When the second universal interface is connected to the screen sharing device, a Display Port transmission channel is established between the screen sharing device and the terminal device. When the speaker device is selected, the first raw audio data of the terminal device is collected based on the speaker device. If no screen sharing operation command is received, in response to the selection of the speaker device, the speaker device is invoked to transmit the first original audio data to the screen sharing device through the Display Port transmission channel, so that the screen sharing device can process the first original audio data to obtain the second audio data, and send the second audio data to the screen sharing receiving end that is communicatively connected to the screen sharing device by invoking the peripheral channel.

12. The audio transmission method as claimed in claim 11, characterized in that, Also includes: Upon receiving a screen sharing operation command, in response to the selection of the speaker device, the speaker device is invoked to transmit the first original audio data to the screen sharing device through the DisplayPort transmission channel, so that the screen sharing device can process the first original audio data to obtain the first audio data, and send the first audio data to the screen sharing receiving end that is communicatively connected to the screen sharing device through the screen sharing channel.

13. A wireless screen sharing system, characterized in that, The wireless screen sharing system includes a screen sharing device and a screen sharing receiver. The screen sharing device is paired and connected with the screen sharing receiver to obtain the connection information and function information of the screen sharing receiver. The function information includes whether it supports calling peripheral functions. When the first general interface of the screen sharing device is connected to the terminal device, the screen sharing device is enumerated as a speaker device and presented to the terminal device to obtain the first raw audio data in the terminal device; Based on the connection information, the screen sharing device establishes a communication connection with the screen sharing receiver, and confirms whether the screen sharing receiver supports calling peripheral functions based on the communication connection. When the screen sharing receiver supports calling peripheral functions and the screen sharing device does not receive a screen sharing operation instruction, in response to the terminal device selecting the speaker device, the first original audio data is processed to obtain the second audio data, and the second audio data is sent to the screen sharing receiver by calling the peripheral channel; The screen sharing receiver decodes the second audio data to obtain the first original audio data, and then calls the speaker of the screen sharing receiver to play the first original audio data.

14. An audio transmission method, characterized in that, The device is used in a screen sharing device, which has a first general interface for connecting to a terminal device. When the terminal device is connected to the first general interface, it has a speaker device corresponding to the screen sharing device. The methods include: The first general interface is connected to the terminal device to establish a DisplayPort transmission channel connecting the screen transmitter and the terminal device; When the unified communications software running on the terminal device does not receive a screen sharing operation command, in response to selecting the speaker device in the speaker settings option of the unified communications software, the screen sharing device is invoked to collect the third raw audio data of the unified communications software from the speaker device; The received third raw audio data is encoded to obtain encoded third audio data; The third audio data is sent to the screen sharing receiver connected in communication with the screen sharing device, so that the screen sharing receiver can decode the third audio data to obtain the third original audio data, and call the speaker of the screen sharing receiver to play the third original audio data.

15. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as claimed in any one of claims 1 to 14.

Images