Screen projection method, device and storage medium

Abstract

The application provides a screen projection method, device and storage medium. The method is applied to a screen projection sending terminal installed with a screen projection application. In a screen projection scene, an audio device currently used by the screen projection sending terminal is acquired, and an identification number corresponding to the audio device is recorded to an audio state table. Thus, when a sound output device selected by a user is determined to be a screen projection receiving terminal, audio data is acquired from the audio device identified by the identification number and sent to the screen projection receiving terminal for playing, instead of a default audio device, such as a loudspeaker, so as to ensure that, when the screen projection receiving terminal is a sound device in the screen projection process, the screen projection receiving terminal can always acquire audio data and play sound.

Description

[0001] This application is a divisional application. The original application, application number 202111523895.3, was filed on December 14, 2021. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of screen mirroring technology, and in particular to a screen mirroring method, device and storage medium. Background Technology

[0003] With the development of terminal technology, more and more terminals have screen projection functions. For example, in home, work, teaching, and gaming scenarios, the terminal can project the currently displayed screen onto a large screen, which can greatly facilitate people to view the screen content.

[0004] For screen casting scenarios where the sending end, i.e. the terminal sending the screen casting information, is a personal computer (PC device), it is currently necessary to use a screen casting application installed on the PC device to cast the audio and video content played on the PC device to the receiving end, such as a large screen device like a TV (hereinafter referred to as a large screen device).

[0005] Although PC devices can use screen mirroring applications to achieve screen mirroring, there are some problems that cannot be ignored. For example, when other audio devices, such as headphones, are connected to the PC device, the large screen device often cannot produce sound when selected as the sound output device. Summary of the Invention

[0006] To address the aforementioned technical issues, this application provides a screen projection method, device, and storage medium, aiming to ensure that, in screen projection scenarios, regardless of whether the PC device is connected to other audio devices, the large-screen device can output sound normally when selected as the audio output device.

[0007] Firstly, this application provides a screen mirroring method applied to a screen mirroring sending end, wherein the screen mirroring sending end has a screen mirroring application installed. The method includes: when audio data is played by a built-in speaker on the screen mirroring sending end, upon receiving a screen mirroring connection request, the screen mirroring sending end responds to the screen mirroring connection request and establishes a screen mirroring connection with a screen mirroring receiving end; when the user selects the screen mirroring receiving end as the audio output device, the audio data is obtained from the channel corresponding to the speaker and sent to the screen mirroring receiving end, whereby the screen mirroring receiving end plays the audio data; when headphones are connected to the screen mirroring sending end, the audio data is obtained from the channel corresponding to the headphones and sent to the screen mirroring receiving end, whereby the screen mirroring receiving end plays the audio data. Therefore, by recording the identification number of the audio device currently used by the screen projection sending end, when the sound output device is the screen projection receiving end, the screen projection sending end can obtain the audio data to be played from the channel corresponding to the audio device identified by the recorded identification number and send it to the screen projection receiving end. Thus, in the screen projection scenario, regardless of whether the screen projection sending end switches audio devices, the screen projection receiving end can receive playable audio data, that is, it can produce sound normally.

[0008] According to the first aspect, when the user selects the screen projection receiver as the audio output device, the method further includes: setting the audio state of the speaker to a mute state. This way, when the screen projection receiver emits sound, the speaker at the screen projection transmitter will not emit sound, thereby avoiding interference caused by both devices emitting sound simultaneously.

[0009] According to the first aspect, or any implementation of the first aspect above, after the headphones are connected to the screen projection sending end, the method further includes: setting the audio state of the headphones to a mute state. In this way, when the screen projection receiving end emits sound, the headphones at the screen projection sending end will not emit sound, thereby avoiding interference caused by both devices emitting sound simultaneously.

[0010] According to the first aspect, or any implementation of the first aspect above, the display interface of the screen projection sending end displays a first audio state switching control and a first volume bar control corresponding to the speaker; the method further includes: when the screen projection sending end plays the audio data by the built-in speaker at the volume value corresponding to the first volume bar control, upon receiving a click operation on the first audio state switching control, the screen projection sending end responds to the click operation by controlling the speaker to stop playing sound, and changing the icon corresponding to the first audio state switching control from a non-mute icon to a mute icon. Therefore, before the screen projection sending end and the screen projection receiving end establish a screen projection connection, whether the speaker of the screen projection sending end plays sound is closely related to the user's click operation on the first audio state switching control; clicking the first audio state switching control can modify the speaker's audio state.

[0011] According to the first aspect, or any implementation of the first aspect above, after acquiring the audio data from the channel corresponding to the speaker and sending the audio data to the screen projection receiving end for playback by the screen projection receiving end, the method further includes: upon receiving a click operation to switch the audio output device from the screen projection receiving end to the screen projection sending end, the screen projection sending end responds to the click operation by stopping the acquisition of the audio data from the channel corresponding to the speaker and keeping the speaker silent, and the icon corresponding to the first audio state switching control remains the mute icon. Therefore, during screen projection, when the audio output device is switched to the screen projection sending end, and the audio device currently used by the screen projection sending end is a built-in speaker, whether the speaker of the screen projection sending end makes a sound depends on the current state of the first audio state switching control corresponding to the speaker, thereby ensuring that whether the speaker of the screen projection sending end makes a sound in the screen projection scenario is consistent with the user's expectations.

[0012] According to the first aspect, or any implementation of the first aspect above, after stopping the acquisition of audio data from the channel corresponding to the speaker and keeping the speaker silent, the method further includes: after receiving a click operation on the first audio state switching control, the screen projection sending end responds to the click operation by controlling the speaker to play the audio data at the volume value corresponding to the first volume bar control, and changing the icon corresponding to the first audio state switching control from the mute icon to the non-mute icon.

[0013] According to the first aspect, or any implementation of the first aspect above, after acquiring the audio data from the channel corresponding to the speaker and sending the audio data to the screen projection receiving end for the screen projection receiving end to play the audio data, the method further includes: after receiving the disconnection of the screen projection connection between the screen projection sending end and the screen projection receiving end, the screen projection sending end responds to the click operation by stopping the acquisition of the audio data from the channel corresponding to the speaker and keeping the speaker silent, and the icon corresponding to the first audio state switching control remains the mute icon.

[0014] According to the first aspect, or any implementation of the first aspect above, a second audio state switching control and a second volume bar control corresponding to the headphones are displayed in the display interface of the screen projection sending end; after obtaining the audio data from the corresponding channel of the headphones and sending the audio data to the screen projection receiving end, and the screen projection receiving end plays the audio data, the method further includes: after receiving a click operation on the second audio state switching control, the screen projection sending end responds to the click operation by controlling the headphones to stop making sound, and changing the icon corresponding to the second audio state switching control from a non-mute icon to a mute icon.

[0015] According to the first aspect, or any implementation of the first aspect above, after the screen projection sending end responds to the click operation by controlling the headphones to stop producing sound and changing the icon corresponding to the second audio state switching control from a non-mute icon to a mute icon, the method further includes: after receiving a click operation to switch the sound output device from the screen projection receiving end to the screen projection sending end, the screen projection sending end responds to the click operation by stopping the acquisition of audio data from the channel corresponding to the headphones and keeping the headphones silent, while the icon corresponding to the second audio state switching control remains the mute icon. Therefore, during screen projection, when the sound output device is switched to the screen projection sending end, and the audio device currently used by the screen projection sending end is an external headphone, whether the headphones of the screen projection sending end produce sound depends on the current state of the second audio state switching control corresponding to the headphones, thus ensuring that whether the headphones of the screen projection sending end produce sound in the screen projection scenario is consistent with the user's expectations.

[0016] According to the first aspect, or any implementation of the first aspect above, after stopping the acquisition of audio data from the corresponding channel of the headphones and keeping the headphones silent, the method further includes: after receiving a click operation on the second audio state switching control, the screen projection sending end responds to the click operation by controlling the headphones to play the audio data at the volume value corresponding to the second volume bar control, and changing the icon corresponding to the second audio state switching control from the mute icon to the non-mute icon.

[0017] According to the first aspect, or any implementation of the first aspect above, after the screen projection sending end responds to the click operation to control the headphones to stop making sound and changes the icon corresponding to the second audio state switching control from a non-mute icon to a mute icon, the method further includes: after receiving the disconnection of the screen projection connection between the screen projection sending end and the screen projection receiving end, the screen projection sending end responds to the click operation to stop acquiring the audio data from the corresponding channel of the headphones and keeps the headphones silent, and the icon corresponding to the second audio state switching control remains the mute icon.

[0018] According to the first aspect, or any implementation of the first aspect above, the display interface of the screen projection sending end further displays an audio device selection list, the audio device selection list including a first audio device option and a second audio device option, the first audio device option corresponding to the speaker, and the second audio device option corresponding to the headphones; after the headphones are connected to the screen projection sending end, the audio data is obtained from the channel corresponding to the headphones and sent to the screen projection receiving end, and the screen projection receiving end plays the audio data, the method further includes: when a click operation on the first audio device option is received, the screen projection sending end responds to the click operation by obtaining the audio data from the channel corresponding to the speaker and sending the audio data to the screen projection receiving end, and the screen projection receiving end plays the audio data.

[0019] Secondly, this application provides a screen mirroring method applied to a screen mirroring sending end, wherein the screen mirroring sending end has a screen mirroring application installed. The method includes: after receiving a screen mirroring connection request, obtaining the identifier of the audio device currently used by the screen mirroring sending end as a target identifier and recording it in an audio device table; determining the sound output device selected by the user; when the sound output device is the screen mirroring sending end, playing audio data through the audio device identified by the target identifier recorded in the audio device table; when the sound output device is the screen mirroring receiving end, setting the audio device identified by the target identifier recorded in the audio device table to a mute state, obtaining the audio data from the channel corresponding to the audio device identified by the target identifier recorded in the audio device table, and sending the audio data to the screen mirroring receiving end for playback.

[0020] For example, the screen projection sending end is a PC device, and the screen projection receiving end is a large-screen device such as a TV.

[0021] Therefore, the screen mirroring solution provided in this application obtains the audio device currently used by the screen mirroring sender in the screen mirroring scenario and records the corresponding identification number of the audio device in the audio status table. In this way, when it is determined that the sound output device selected by the user is the screen mirroring receiver, audio data is obtained from the audio device identified by the identification number and sent to the screen mirroring receiver for playback, instead of the default audio device, such as a speaker. This ensures that during the screen mirroring process, when the screen mirroring receiver is the sound output device, the screen mirroring receiver can always obtain audio data and play sound.

[0022] According to the second aspect, after obtaining the identifier of the audio device currently used by the screen projection sending end as the target identifier and recording it in the audio device table, the method further includes: calling the audio device monitoring module to monitor whether the audio device currently used by the screen projection sending end has changed; when it is detected that the audio device currently used by the screen projection sending end has changed, obtaining the identifier corresponding to the changed audio device; and updating the target identifier recorded in the audio device table according to the identifier corresponding to the changed audio device.

[0023] Therefore, the screen mirroring solution provided in this application continuously monitors the audio device selected by the screen mirroring sending end by activating an audio device monitoring module during the screen mirroring process. When the audio device changes, the target identifier recorded in the audio device table is updated, ensuring that the target identifier recorded in the audio device table changes according to the actual audio device selected by the screen mirroring sending end. This guarantees that during screen mirroring, when the screen mirroring sending end is an audio output device, the audio data can be played by the changed audio device; and when the screen mirroring receiving end is an audio output device, the screen mirroring sending end can obtain audio data from the corresponding channel of the changed audio device and send it to the screen mirroring receiving end for playback. This solves the problem in existing solutions where both the screen mirroring receiving end and the screen mirroring sending end can output sound normally during the screen mirroring process, regardless of whether the screen mirroring sending end is connected to other audio devices.

[0024] According to the second aspect, or any implementation of the second aspect above, after the audio data is sent to the screen projection receiver for playback, the method further includes: after the user changes the sound output device from the screen projection receiver to the screen projection sender, playing the audio data through the audio device identified by the target identifier recorded in the audio device table.

[0025] Therefore, the screen projection solution provided in this application plays the audio data according to the audio device identified by the target identifier recorded in the audio device table. This allows the audio device playing the audio data at the screen projection sending end to match the audio device selected by the user after the sound output device is changed from the screen projection receiving end to the screen projection sending end, thereby further improving the user experience.

[0026] According to the second aspect, or any implementation of the second aspect above, after sending the audio data to the screen mirroring receiver for playback, the method further includes: after receiving a screen mirroring disconnection request, playing the audio data through the audio device identified by the target identifier recorded in the audio device table.

[0027] Therefore, the screen mirroring solution provided in this application plays the audio data according to the audio device identified by the target identifier recorded in the audio device table, so that after the screen mirroring is disconnected, the audio device playing the audio data at the screen mirroring sending end can be consistent with the one selected by the user, thereby further improving the user experience.

[0028] According to the second aspect, or any implementation of the second aspect above, after disconnecting the connection with the screen projection receiver, the method further includes: turning off the audio device monitoring module.

[0029] Therefore, the screen mirroring solution provided in this application stops monitoring changes in the audio device after screen mirroring ends, thereby effectively avoiding the occupation and waste of resources on the screen mirroring sending end.

[0030] According to the second aspect, or any implementation of the second aspect above, after receiving the screen casting connection request, the method further includes: obtaining the current audio state of the screen casting sending end as the target audio state, and recording it in the audio state table, wherein the target audio state is a muted state or a non-mute state; correspondingly, when the sound output device is the screen casting sending end, playing the audio data through the audio device identified by the target identifier recorded in the audio device table includes: when the sound output device is the screen casting sending end, playing the audio data through the audio device identified by the target identifier recorded in the audio device table, and setting the audio state of the audio device identified by the target identifier to the target audio state.

[0031] Therefore, the screen mirroring solution provided in this application obtains the audio state of the screen mirroring sender before establishing the screen mirroring connection in the screen mirroring scenario, and records the obtained audio state as the target audio state of the screen mirroring sender in the audio state table. In this way, when it is determined that the sound output device selected by the user is the screen mirroring sender, the audio state of the screen mirroring sender is directly set to the target audio state recorded in the audio state table. This ensures that in the screen mirroring scenario, when the sound switches back to the screen mirroring sender, the audio state of the screen mirroring sender can be consistent with the audio state expected by the user, thereby greatly improving the user experience.

[0032] According to the second aspect, or any implementation of the second aspect above, after obtaining the current audio state of the screen-casting terminal as the target audio state and recording it in the audio state table, the method further includes: calling the audio state monitoring module to monitor the user's click operation on the audio state switching control corresponding to the audio device identified by the target identifier; when the audio state corresponding to the audio state switching control is the mute state, after the audio state switching control is clicked, the audio state switches from the mute state to the non-mute state; when the audio state corresponding to the audio state switching control is the non-mute state, after the audio state switching control is clicked, the audio state switches from the non-mute state to the mute state; after monitoring the user's click operation on the audio state switching control corresponding to the audio device identified by the target identifier, updating the target audio state recorded in the audio state table according to the switched audio state.

[0033] Therefore, the screen mirroring solution provided in this application continuously monitors the audio status of the screen mirroring sending end by activating an audio status monitoring module during the screen mirroring process. When the audio status changes, the target audio status recorded in the audio status table is updated, ensuring that the target audio status recorded in the audio status table can be changed according to the user's actual operation. This guarantees that when screen mirroring is disconnected or the audio output device is changed to the screen mirroring sending end, the audio status set according to the target audio status recorded in the audio status table is consistent with the user's actual desired audio status. This solves the problem in existing solutions where, before or during screen mirroring, the user manually mutes the screen mirroring sending end, and after disconnecting the screen mirroring or switching from sound output from the large screen device to sound output from the screen mirroring sending end, the audio status of the screen mirroring sending end remains muted, consistent with the user's setting, instead of directly emitting sound.

[0034] According to the second aspect, or any implementation of the second aspect above, after the audio data is sent to the screen projection receiver, the method further includes: after the user changes the sound output device from the screen projection receiver to the screen projection sender, reading the target audio status recorded in the audio status table; and setting the audio status of the audio device identified by the target identifier in the screen projection sender to the target audio status.

[0035] Therefore, the screen casting solution provided in this application sets the audio status of the screen casting sender according to the target audio status recorded in the audio status table, so that after the sound output device changes from the screen casting receiver to the screen casting sender, the audio status of the screen casting sender can be consistent with the status set by the user, further improving the user experience.

[0036] According to the second aspect, or any implementation of the second aspect above, the method further includes: after receiving the screen casting disconnection request, reading the target audio status recorded in the audio status table; and setting the audio status of the audio device identified by the target identifier in the screen casting sending end to the target audio status.

[0037] Therefore, the screen mirroring solution provided in this application sets the audio status of the screen mirroring sender according to the target audio status recorded in the audio status table, so that the audio status of the screen mirroring sender can be consistent with the status set by the user after the screen mirroring is disconnected, thereby further improving the user experience.

[0038] According to the second aspect, or any implementation of the second aspect above, after disconnecting the connection with the screen projection receiver, the method further includes: turning off the audio status monitoring module.

[0039] Therefore, the screen mirroring solution provided in this application stops monitoring changes in audio status after screen mirroring ends, thereby effectively avoiding the occupation and waste of resources on the screen mirroring sending end.

[0040] Thirdly, this application provides an electronic device, which is a screen projection terminal, including a processor and a memory; wherein the memory and the processor are coupled, and the memory stores program instructions; when the program instructions are executed by the processor, the electronic device executes the instructions of the method in the first aspect or any implementation of the first aspect, or executes the instructions of the method in the second aspect or any implementation of the second aspect.

[0041] Fourthly, this application provides a computer-readable medium for storing a computer program that, when run on an electronic device, causes the electronic device to execute instructions of the method in the first aspect or any implementation thereof, or to execute instructions of the method in the second aspect or any implementation thereof.

[0042] Fifthly, this application provides a computer program product comprising a computer program that, when run on an electronic device, causes the electronic device to execute instructions of the method in the first aspect or any implementation thereof, or to execute instructions of the method in the second aspect or any implementation thereof.

[0043] Sixthly, this application provides a chip including a processing circuit and transceiver pins. The transceiver pins and the processing circuit communicate with each other via an internal connection path. The processing circuit executes instructions of the method in the first aspect or any implementation of the first aspect, or executes instructions of the method in the second aspect or any implementation of the second aspect, to control the receiving pin to receive signals and to control the transmitting pin to transmit signals.

[0044] Seventhly, this application provides a screen projection system, which includes a screen projection sending end and a screen projection receiving end.

[0045] For example, the screen projection sending end is the electronic device mentioned in the third aspect above.

[0046] For example, the screen mirroring receiver is a television or other device capable of receiving and projecting the screen mirroring information sent by the screen mirroring sender. Attached Figure Description

[0047] Figure 1 This is one of the schematic flowcharts illustrating the screen projection method provided in the embodiments of this application;

[0048] Figure 2This is a second schematic flowchart illustrating the screen projection method provided in an embodiment of this application;

[0049] Figure 3 This is an illustrative diagram of the main page of the screen mirroring sending terminal;

[0050] Figure 4 This is one of the exemplary diagrams showing a screen mirroring application initiating a screen mirroring connection;

[0051] Figure 5 This is the second illustrative diagram showing a screen mirroring application initiating a screen mirroring connection;

[0052] Figure 6 This is an example of a schematic diagram showing the interface for selecting the screen mirroring receiver;

[0053] Figure 7 This is an exemplary schematic diagram of the interface establishing a connection with the selected screen mirroring receiver;

[0054] Figure 8 This is an example illustration of the screen mirroring application interface after a successful screen mirroring connection.

[0055] Figure 9 This is an exemplary schematic diagram of the interfaces of the screen casting initiator and the screen casting receiver after a successful screen casting connection.

[0056] Figure 10 This is one of the exemplary diagrams showing the audio device for switching the screen mirroring sender;

[0057] Figure 11 This is a second illustrative diagram showing a user switching the audio device of the screen-sharing sender;

[0058] Figure 12 This is the third illustrative diagram showing a user switching the audio device of the screen-casting sender;

[0059] Figure 13 This is the fourth illustrative diagram showing a user switching the audio device of the screen-casting sender;

[0060] Figure 14 This is an example diagram illustrating how a user can mute the headphones 1 on the screen mirroring sending end.

[0061] Figure 15 This is an exemplary diagram showing the switching of the device as a sound output device;

[0062] Figure 16 This is an illustrative diagram showing the device after switching to audio output.

[0063] Figure 17This is one of the exemplary diagrams illustrating the changes in sound at the screen projection sending end and screen projection receiving end after switching the device to sound output.

[0064] Figure 18 This is the second illustrative diagram showing the changes in sound at the screen projection sending end and the screen projection receiving end after switching the device to sound output.

[0065] Figure 19 This is one of the exemplary diagrams showing the changes in sound at the screen mirroring sender and receiver after screen mirroring is disconnected;

[0066] Figure 20 This is the second illustrative diagram showing the changes in sound at the screen mirroring sender and receiver after screen mirroring is disconnected.

[0067] Figure 21 This is one of the exemplary diagrams illustrating the method called when the upper-level audio management module sets the audio status of the screen-casting sending end;

[0068] Figure 22 This is the second example of a method called when the upper-level audio management module sets the audio status of the screen-casting sending end;

[0069] Figure 23 This is one of the exemplary diagrams showing how the upper-level audio management module sets the audio status of the screen-casting sending end;

[0070] Figure 24 This is the second illustrative diagram showing how the upper-level audio management module sets the audio status of the screen-casting sending end;

[0071] Figure 25 This is the third illustrative diagram showing how the upper-level audio management module sets the audio status of the screen-casting sending end;

[0072] Figure 26 This is one of the exemplary diagrams illustrating the method invoked when the lower-level transmission module sets the audio status of the screen projection sending end;

[0073] Figure 27 This is one of the exemplary diagrams illustrating the method invoked when the lower-level transmission module sets the audio status of the screen projection sending end;

[0074] Figure 28 This is one of the exemplary diagrams showing how the audio status of the screen projection sending end is set by the lower-level transmission module;

[0075] Figure 29 This is a second illustrative diagram showing how the audio status of the screen projection sending end is set by the lower-level transmission module.

[0076] Figure 30This is the third illustrative diagram showing how the audio status of the screen projection sending end is set by the lower-level transmission module. Detailed Implementation

[0077] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0078] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0079] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0080] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0081] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0082] Before describing the technical solutions of the embodiments of this application, the electronic devices to which the technical solutions provided in the embodiments of this application are applicable will be described first.

[0083] Specifically, the technical solution provided in this embodiment is applied to the screen casting sending end, and the screen casting sending end is a PC device (hereinafter referred to as PC) with screen casting application installed.

[0084] For example, in this embodiment, the PC mentioned above can be a desktop computer or a laptop computer.

[0085] In addition, it should be noted that in this embodiment, the receiving end of the screen projection content sent by the screen projection sending end, such as video data, image data, and audio data, can be a large-screen device such as a TV or a display screen.

[0086] The implementation details of the technical solutions provided in the embodiments of this application are described below. The following implementation details are provided for ease of understanding only and are not necessary for implementing this solution.

[0087] For example, see Figure 1 The specific implementation steps of this application embodiment include:

[0088] Step 101: After receiving the screen mirroring connection request, obtain the identifier of the audio device currently used by the screen mirroring sending end as the target identifier and record it in the audio device table.

[0089] Step 102: Determine the audio output device selected by the user.

[0090] Step 103: Is the audio output device a screen mirroring sender?

[0091] Step 104: Play audio data through the audio device identified by the target identifier recorded in the audio device table.

[0092] Step 105: Set the audio device identified by the target identifier recorded in the audio device table to a mute state, obtain the audio data from the channel corresponding to the audio device identified by the target identifier recorded in the audio device table, and send the audio data to the screen projection receiving end for playback.

[0093] Therefore, the screen casting method provided in this embodiment obtains the audio device currently used by the screen casting sending end in the screen casting scenario and records the corresponding identification number of the audio device in the audio status table. In this way, when it is determined that the sound output device selected by the user is the screen casting receiving end, the audio data is obtained from the audio device identified by the identification number and sent to the screen casting receiving end for playback, instead of the default audio device, such as a speaker. This ensures that during the screen casting process, when the screen casting receiving end is the sound output device, the screen casting receiving end can always obtain audio data and play the sound.

[0094] Furthermore, it should be noted that in screen mirroring scenarios, in addition to the problems mentioned in the above embodiments where the receiving end cannot obtain audio data and play sound, or the sending end does not play audio data from the user's expected audio device, there is also the problem of the audio status of the sending end being inconsistent with the user's expectations. To solve this technical problem, see [link to relevant documentation]. Figure 2 The following implementation scheme is proposed:

[0095] Step 201: After receiving the screen casting connection request, obtain the identifier of the audio device currently used by the screen casting sending end as the target identifier and record it in the audio device table.

[0096] For example, in practical applications, a screen mirroring application may include a screen mirroring management module, an audio management module, and a transmission module.

[0097] For example, the screen mirroring management module interacts with the audio management module, and the audio management module interacts with the transmission module.

[0098] Specifically, in this embodiment, the screen casting connection request, screen casting disconnection request, and audio output device switching request involved in the screen casting scenario are all monitored by the screen casting management module.

[0099] For example, in some cases, the audio device that can be used by the screen projection sending end is, for example, a speaker built into the screen projection sending end, or headphones (Bluetooth headphones, wired headphones, etc.) or audio equipment connected to the screen projection sending end. These will not be listed one by one here, and this embodiment does not limit them.

[0100] Specifically, in this embodiment, the audio device currently used by the screen projection sending end can be any of the above-listed devices.

[0101] Furthermore, it is understood that the identification number used to identify audio devices mentioned above may be, for example, a serial number assigned by the manufacturer to the audio device, or an identification number assigned according to preset rules.

[0102] For example, in this embodiment, the screen projection sending end includes a screen projection management module, an audio management module, and a transmission module. The target identifier of the currently used audio device mentioned above can be obtained and managed by the audio management module.

[0103] It should be understood that in practical applications, in order to distinguish different audio devices, each audio device will have a unique identifier, and different audio devices may correspond to different channels. That is, there is a mapping relationship between audio devices, identifiers and channels as shown in Table 1.

[0104] Table 1. Mapping Relationship between Audio Devices, Identifiers, and Channels

[0105] speaker 10001 Channel 1 Headphone 1 10002 Channel 2 Headphones 2 10003 Channel 3 Audio 1 10004 Channel 4

[0106] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0107] For ease of explanation, let's take the contents recorded in Table 1 as an example. Assuming that after receiving the screen casting connection request, the audio management module obtains that the audio device currently used by the screen casting sender is a speaker, then the target identifier number to be recorded in the audio device table is 10001.

[0108] Accordingly, if the audio management module obtains that the audio device currently used by the screen projection sending end is Headphone 1, then the target identifier number to be recorded in the audio device table is 10002.

[0109] To better understand the technical solution provided in this embodiment, we will take a PC device as the screen projection sending end and a large-screen device such as a TV as the screen projection receiving end as an example, combined with... Figures 3 to 8 The process of initiating a screen mirroring connection request is explained.

[0110] See Figure 3 For example, the display interface of PC device 100 displays the main page of PC device 100.

[0111] Understandably, in practical applications, the main page may include one or more controls, such as video application controls, social application controls, browser controls, and PC manager application controls, which will not be listed here, and this embodiment does not impose any restrictions on them.

[0112] See also Figure 3 Specifically, in the technical solution of this embodiment, the main page of the PC device 100 includes a screen casting application control 10.

[0113] In addition, the toolbar at the bottom of the main interface of PC device 100 may include one or more controls, such as setting the audio status of PC device 100, such as adjusting to a mute or unmute state, and a sound setting control 20 that brings up the volume bar.

[0114] It should be noted that, Figure 3 The names and number of controls displayed on the PC device 100's display interface and in the following figures, as well as the names and number of controls in the pop-up interface, are merely illustrative examples and are not intended to limit the scope of this application.

[0115] See also Figure 3 When a user wants to cast the content of PC device 100 to a large-screen device such as a TV, after the user clicks the casting application control 10 on the main page, PC device 100 responds to the user's operation and displays a page to initiate a casting connection on the display interface, such as... Figure 4 As shown.

[0116] See Figure 4For example, the page that initiates the screen mirroring connection may include one or more options. For instance, the "Connect Now" option 10-1 that initiates the screen mirroring connection request allows the user to select a screen mirroring mode.

[0117] See also Figure 4 Users can choose from various screen mirroring modes, including options such as "Mirror Mode" and "Extended Mode".

[0118] Mirroring mode refers to mirroring the computer screen (PC device 100) onto the monitor of a large-screen device such as a television. In other words, after establishing a screen mirroring connection between PC device 100 and the large-screen device, the content displayed on the large-screen device's monitor is exactly the same as the content displayed on the PC device 100's screen.

[0119] The so-called extended mode refers to displaying the computer screen, i.e., the content on the screen of PC device 100, on the monitor of the receiving end, such as a large-screen device like a TV, in an extended manner. That is, after establishing a screen casting connection between PC device 100 and the large-screen device using extended mode, the monitor of the large-screen device displays both the content displayed on the screen of PC device 100 and the content of the large-screen device itself, similar to multi-screen collaboration.

[0120] In addition, it should be noted that in practical applications, the functions that the screen casting application needs to achieve can also be integrated with other functions into one application. For example, system management functions such as driver management and power management can be integrated with functions that enhance user experience, such as screen casting and multi-screen collaboration, into one application.

[0121] For example, the application could be called "My Computer" or "Computer Manager", etc., and this embodiment does not limit it.

[0122] See Figure 5 For example, when a user clicks the "Computer Interconnection" option displayed in the function list, the PC device 100 responds to the user's action, and the display interface on the right will show something like this. Figure 4 The page shown is for initiating a screen mirroring connection.

[0123] See also Figure 4 or Figure 5 When the user selects the mirroring mode and clicks the "Connect Now" option 10-1, the PC device 100 responds to the user's action by displaying the device selection window 10-2 on the display interface, such as... Figure 6 As shown.

[0124] Specifically, the device selection window 10-2 displays currently available devices, i.e., large-screen devices that can be used as screen mirroring receivers, such as... Figure 5As shown, the currently available devices are TV1, TV2 and TV3.

[0125] See also Figure 6 The device selection window 10-2 can also include an option to "find other display devices". When none of the currently displayed available devices are the device the user wants to select, the user can click the "find other display devices" option to jump to the page for finding other display devices, and the PC device 100 will enable the search function to find available large-screen devices nearby.

[0126] Understandably, in practical application scenarios, the prerequisite for PC device 100 to establish a screen mirroring connection with a large screen device can be either a wired connection or a wireless connection.

[0127] For example, regarding the method of wireless connection, it could be that both devices are connected to the same WLAN network, or that they are successfully paired via Bluetooth.

[0128] Therefore, in the screen mirroring scenario, when the PC device 100 and the large screen device establish a wireless connection, after the user clicks the "Connect Now" option 10-1, the PC device 100 also needs to determine whether Bluetooth or WLAN is enabled on its own device.

[0129] Accordingly, if the PC device 100 does not have Bluetooth or WLAN enabled, the PC device 100 will respond to the user's operation by enabling Bluetooth or WLAN before jumping to the device selection window 10-2, and then jump to the device search interface.

[0130] Accordingly, after the search operation is completed, the system will jump to the device selection window 10-2, which displays the available devices found.

[0131] Furthermore, it is understood that in practical applications, the operation of turning on Bluetooth or WLAN can be performed by the PC device 100 itself, or it can be done by jumping to the interface for turning on Bluetooth or WLAN and being turned on by the user. This embodiment does not limit this.

[0132] See also Figure 6 For example, when a user clicks the option for TV 1, PC device 100 responds to the user's action by displaying connection window 10-3 on the display interface, such as... Figure 7 As shown.

[0133] See Figure 7 For example, the connection window 10-3 may include a cancel option 10-4.

[0134] For example, when the user clicks the cancel option 10-4, the PC device 100, in response to the user's action, will return to... Figure 6 The interface shown.

[0135] For example, after PC device 100 establishes a screen mirroring connection with TV 1, the display interface of PC device 100 will change to Figure 8 The style shown.

[0136] For example, the display interface showing a successful screen mirroring connection may include one or more options. For instance... Figure 8 The options shown include disconnection option 10-5, audio output device selection option 10-6, and options for users to select screen mirroring modes, such as "mirror mode" and "extended mode".

[0137] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0138] Step 202: Obtain the current audio status of the screen projection sending end as the target audio status and record it in the audio status table.

[0139] Specifically, the target audio status of the screen-casting end mentioned above is obtained and managed by the audio management module.

[0140] It should be noted that in this embodiment, the audio state is divided into a silent state and a non-silent state. Therefore, the target audio state mentioned above can be, for example, a silent state or a non-silent state.

[0141] Furthermore, in practical applications, in order to facilitate rapid identification within the machine, the target audio status recorded in the audio status table can be represented by a pre-agreed status value (status code).

[0142] For example, in some cases, "0" can be used to represent the mute state and "1" can be used to represent the non-mute state.

[0143] Furthermore, it is understood that, specifically in this embodiment, when there are multiple audio devices available for the screen projection sending end to choose from, such as the speaker, headphones 1, headphones 2 and audio 1 shown in Table 1, in order to accurately know the audio status of each audio device, a mapping relationship as shown in Table 2 can be established.

[0144] Table 2. Mapping Relationship Between Audio Devices and Audio Status

[0145] speaker 10001 Channel 1 mute mode 0 Headphone 1 10002 Channel 2 Non-silent state 1 Headphones 2 10003 Channel 3 mute mode 0 Audio 1 10004 Channel 4 mute mode 0

[0146] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0147] Step 203: Determine the audio output device selected by the user.

[0148] See also Figure 8 When the user selects "TV1" as the audio output device, the audio output device is determined to be the screen casting receiver, meaning that the sound is emitted from the screen casting receiver.

[0149] Accordingly, when the user selects "local device" as the audio output device, the audio output device is determined to be the screen projection sending end. That is, the audio is output from the screen projection sending end.

[0150] Step 204: Is the audio output device a screen mirroring sender?

[0151] Specifically, if the determined audio output device is the screen projection sender, then proceed with step 205; otherwise, proceed with step 206.

[0152] Step 205: Play the audio data through the audio device identified by the target identifier recorded in the audio device table, and set the audio status of the audio device identified by the target identifier to the target audio status.

[0153] Referring again to Table 1, for example, if the target identifier recorded in the audio device table is "10001", according to the mapping relationship recorded in Table 1, the audio device identified by the target identifier is a speaker. Then, when the sound output device is the screen projection sending end, the audio data is played by the speaker.

[0154] Referring to Table 2, for example, if the target audio state recorded in the audio state table has a value of "0", according to the mapping relationship recorded in Table 2, the target audio state of the speaker is muted. In this way, the audio state of the speaker is set to muted, and the screen projection sending end, as the sound output device, will be muted, that is, it will not make any sound. At this time, the screen of the screen projection receiving end and the screen of the screen projection sending end display the same picture, but neither device makes any sound.

[0155] Referring to Table 2, for example, if the target audio state recorded in the audio state table is "1", according to the mapping relationship recorded in Table 2, the target audio state of the speaker is non-mute. Then, the audio state of the speaker is set to non-mute, so the screen projection sending end will produce sound as a sound output device. At this time, the screen of the screen projection receiving end and the screen of the screen projection sending end display the same picture, and the screen projection receiving end does not produce sound.

[0156] Step 206: Set the audio device identified by the target identifier recorded in the audio device table to a mute state, obtain the audio data from the channel corresponding to the audio device identified by the target identifier recorded in the audio device table, and send the audio data to the screen projection receiving end for playback.

[0157] It should be noted that when the audio output device is the screen mirroring receiver, regardless of whether the target audio state recorded in the audio status table is muted or not, the screen mirroring sender cannot produce sound after the screen mirroring is established. Therefore, the screen mirroring application in the screen mirroring sender needs to call the interface for setting the screen mirroring sender's sound to mute the screen mirroring sender, and then send the audio data to the screen mirroring receiver through the aforementioned transmission module. In this way, the screen mirroring receiver can play the audio data after receiving it, that is, the screen mirroring receiver can produce sound.

[0158] Furthermore, it should be understood that, as described above, different audio devices may correspond to different channels, and the audio data originates from the channel corresponding to the audio device currently used by the screen mirroring sender. Therefore, the audio data sent to the screen mirroring receiver is obtained from the channel corresponding to the audio device identified by the target identifier recorded in the audio device table, and then the obtained audio data is sent to the screen mirroring receiver for playback.

[0159] Referring to Table 1, for example, if the target identifier recorded in the audio device table is "10001", according to the mapping relationship recorded in Table 1, the audio device identified by the target identifier is a speaker, and the channel corresponding to the speaker is channel 1. Therefore, when sending audio data to the screen projection receiver, the audio data is specifically obtained from channel 1, and then the obtained audio data is sent to the screen projection receiver for playback.

[0160] Accordingly, if the target identifier recorded in the audio device table is "10002", according to the mapping relationship recorded in Table 1, the audio device identified by the target identifier is Headphone 1, and the channel corresponding to Headphone 1 is Channel 2. Therefore, when sending audio data to the screen projection receiver, the audio data is specifically obtained from Channel 2, and then the obtained audio data is sent to the screen projection receiver for playback.

[0161] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0162] For ease of explanation, this embodiment assumes that after establishing a screen mirroring connection between the sending and receiving ends, the receiving end is the default audio output device. That is, the default scenario is as follows: Figure 9 As shown.

[0163] See Figure 9 For example, after a screen mirroring connection is established between PC device 100 (screen mirroring sender) and TV 200 (screen mirroring receiver), if TV 200 is an audio output device, in mirroring mode, the screen of PC device 100 will display image 30, and the screen of TV 200 will display the same content as image 30, for example... Figure 9 The image 30' displayed on the screen of TV 200 is silent while PC device 100 is silent and TV 200 is audible.

[0164] Understandably, the process of acquiring audio data sent to the TV 200 is described in step 206 above, and will not be repeated here.

[0165] In addition, it should be noted that when the audio output device is the screen mirroring receiver, the operation of setting the screen mirroring sender to mute is implemented by the system, not modified by the user. Therefore, this state is not recorded in the audio status table and will not interfere with the user's operation.

[0166] Furthermore, it is understandable that in practical applications, the audio data sent by the transmission module to the TV 200 is usually an audio data packet after audio encoding processing. This can both avoid the loss of data frames caused by direct transmission of audio data and avoid the direct exposure of user privacy by audio data.

[0167] Furthermore, in practical applications, the protocols followed by PC device 100 and TV 200 may differ when establishing a screen mirroring connection using a screen mirroring application. Therefore, the encoding method can be determined based on the protocol type corresponding to the audio data, and then the audio data can be encoded using the determined encoding method.

[0168] Specifically, when the transmission module sends audio data to the screen mirroring receiver, such as the TV 200 mentioned above, it first needs to determine the protocol type corresponding to the audio data; then, it determines the target encoding method of the audio data according to the protocol type; next, it encodes the audio data according to the target encoding method to obtain an audio data packet; finally, it sends the audio data packet to the screen mirroring receiver.

[0169] Accordingly, when sending the audio data packet to the screen mirroring receiver, the protocol type is also sent to the screen mirroring receiver.

[0170] In this way, the receiving end of the screen mirroring, such as TV 200, can know which decoding method needs to be used to decode the audio data packets, thus ensuring the accuracy of the decoding results.

[0171] Furthermore, it should be noted that, regarding the determination of the target encoding method of the audio data based on the aforementioned protocol type, it can be determined, for example, by using a codec mapping relationship table pre-negotiated between the screen casting sender and screen casting receiver. That is, the encoding method corresponding to the protocol type is searched in the codec mapping relationship table, and then the found encoding method is determined as the target encoding method.

[0172] In this way, the sending and receiving ends of the screen projection do not need to directly transmit the encoding method, thereby avoiding the interception of the content transmitted between them and ensuring that the audio data is not stolen by criminals. This is especially effective in ensuring the security of audio and video conferencing.

[0173] Furthermore, it should be noted that in practical application scenarios, after the PC device 100 establishes a screen mirroring connection with the large screen device, during the screen mirroring process, the user may manually change the audio device used by the PC device 100, or a higher-priority audio device may be connected to the PC device 100. Therefore, in order to ensure that the target identifier recorded in the audio device table can be dynamically updated according to actual changes, it is necessary to monitor whether the audio device currently used by the PC device 100 has changed.

[0174] In this specific embodiment, in order to monitor whether the audio device has changed, the audio device monitoring module provided by the system used by the PC device 100 is used.

[0175] Specifically, after step 201 is executed, PC device 100 can call the aforementioned audio device monitoring module, which in turn monitors whether the audio device currently used by PC device 100 has changed.

[0176] Accordingly, when the audio device monitoring module detects a change in the audio device currently used by the screen projection sending end, it obtains the identifier corresponding to the changed audio device; then it sends the obtained identifier to the audio management module, which updates the target identifier recorded in the audio device table according to the identifier corresponding to the changed audio device.

[0177] Furthermore, it should be noted that in practical applications, headphones have higher priority than speakers and audio equipment. That is, when headphones are connected to PC device 100, audio data will be output preferentially from the corresponding channel of the headphones. Therefore, the aforementioned high-priority audio device access to PC device 100 could be, for example, when the audio device currently used by PC device 100 is a speaker, and a Bluetooth or wired headset has established a connection with PC device 100.

[0178] For ease of explanation, the monitoring of audio equipment changes by the audio equipment monitoring module and the updating of target identifiers by the audio management module are described below in conjunction with... Figures 10 to 13 Please provide an explanation.

[0179] See Figure 10 For example, during the screen mirroring process, if a user wants to switch the audio device currently used by the PC device 100 from the speaker to the headphones 1, the user can first click the sound settings control 20.

[0180] Accordingly, after the user clicks the sound settings control 20, the PC device 100, in response to the user's action, will display the sound settings window 20-1, such as... Figure 11 As shown.

[0181] See also Figure 11 For example, the sound settings window 20-1 displays an audio device selection control 20-2, the currently used audio device, such as a speaker, and an audio status switching control 20-3 corresponding to the speaker and a volume bar control 20-4 for adjusting the volume.

[0182] See also Figure 11 When the user clicks the audio device selection control 20-2, the PC device 100 responds to the user's action by displaying a list of selectable playback devices, such as... Figure 12 As shown.

[0183] See Figure 12 For example, the list of playback devices would display all audio devices available for selection by PC device 100, such as... Figure 12 The headphones 1 and speaker.

[0184] See also Figure 12 When the user clicks the Headphones 1 option, the PC device 100 responds to the user's action by switching the currently used audio device from the speakers to Headphones 1. The content displayed in the sound settings window 20-1 after the switch is as follows: Figure 13 As shown.

[0185] Understandably, Figures 10 to 13 In the implementation shown, the speaker and headphones 1 share an audio state switching control 20-3 and a volume bar control 20-4 for adjusting the volume.

[0186] Specifically, in this situation, when the audio device currently used by PC device 100 is a speaker, the operation of the audio state switching control 20-3 and the volume bar control 20-4 for adjusting the volume is essentially adjusting the audio state and volume of the speaker.

[0187] Accordingly, when the audio device currently used by PC device 100 is headphones 1, the operation of the audio state switching control 20-3 and the volume bar control 20-4 for adjusting the volume is essentially adjusting the audio state and volume of headphones 1.

[0188] Understandably, in some examples, this approach can be implemented by maintaining a mapping relationship between the current audio state of the audio state switching control 20-3, the current volume value of the volume bar control 20-4, and the currently used audio device. Then, when a change in the audio device is detected, the audio state displayed by the audio state switching control 20-3 and the volume value displayed by the volume bar control 20-4 can be adjusted accordingly.

[0189] Furthermore, it is understandable that in other implementation scenarios, the audio state switching control 20-3 and the volume bar control 20-4 for adjusting the volume can be set according to the number of audio devices that the PC device 100 can select. In this way, when the audio devices change, the audio state and volume value can be set by the corresponding audio state switching control 20-3 and the volume bar control 20-4 for adjusting the volume.

[0190] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0191] In addition, it should be noted that in actual application scenarios, after the PC device 100 establishes a screen mirroring connection with the large screen device, that is, during the screen mirroring process, the user may also manually modify the audio status of the audio device currently used by the PC device 100. Therefore, in order to ensure that the status value of the target audio status recorded in the audio status table can be dynamically updated according to the user's actual modification, it is necessary to monitor the audio status switching control.

[0192] Specifically, in this embodiment, in order to monitor whether the user has clicked the audio status switching control corresponding to the audio device currently used by the PC device 100, the audio status monitoring module provided by the system used by the PC device 100 is utilized.

[0193] Specifically, after performing step 202 above, PC device 100 can call the aforementioned audio status monitoring module, which in turn monitors the user's click operation on the audio status switching control corresponding to the audio device currently used by PC device 100.

[0194] It should be noted that when the audio state corresponding to the audio state switching control is the aforementioned mute state, after the audio state switching control is clicked, the audio state switches from the mute state to the aforementioned non-mute state.

[0195] Accordingly, when the audio state corresponding to the audio state switching control is the non-mute state, the audio state switches from the non-mute state to the mute state after the audio state switching control is clicked.

[0196] Therefore, by using the audio status monitoring module to monitor the click operation of the audio status switching control, after monitoring the user's click operation on the audio status switching control corresponding to the audio device identified by the target identifier, the target audio status recorded in the audio status table can be updated according to the switched audio status.

[0197] For example, if the audio device currently used by PC device 100 before screen mirroring is a speaker, and the target audio state of the speaker is "0", then after screen mirroring is connected, if the audio device has not changed, the speaker's audio state will change to non-mute after the user clicks the audio state switching control. At this time, the audio state table will be updated, for example, by changing the recorded state value corresponding to the speaker from "0" to "1".

[0198] Accordingly, before the screen mirroring connection, the audio device currently used by PC device 100 is a speaker. When the target audio state of the speaker is "0", after the screen mirroring connection is established, if the audio device changes to headphones and the target audio state of the headphones is "1", then after the user clicks the audio state switching control, the audio state of the headphones will change to mute. At this time, the audio state table will be updated, for example, by changing the recorded state value corresponding to the headphones from "1" to "0".

[0199] To better understand the modification of the audio status of the audio device currently used by PC device 100, the following explanation will take the example of headphones 1 being used by PC device during screen mirroring.

[0200] See also Figure 13 For example, during the screen mirroring process, if the user wants to change the audio status of the headset 1 (which was in a non-mute state before the change), the user can click the audio status switching control 20-3.

[0201] Accordingly, the audio status monitoring module detects the click operation, and the PC device 100 responds to the user's operation by changing the icon style of the audio status switching control 20-3. Figure 13 The style shown becomes Figure 14 The icon style of the audio status switching control 20-3 and the icon style of the sound setting control 20 will also change. Figure 13 The style shown becomes Figure 14The icon style of the sound setting control 20. That is, it indicates that the target audio state of the headphones 1 has changed from non-mute to mute. At this time, the state value recorded in the audio state table corresponding to the headphone 1 will be changed from "1" representing non-mute state to "0" representing mute state.

[0202] In this way, the target audio status of the audio device currently used by the screen mirroring sender, as recorded in the audio status table, can be changed according to the user's actual operation. This ensures that when screen mirroring is disconnected or the audio output device is changed to the screen mirroring sender, the audio status set according to the target audio status of the currently used audio device recorded in the audio status table is consistent with the audio status that the user actually wants to set.

[0203] In practical applications, the audio status of the audio device currently used by PC device 100 is set according to the target audio status recorded in the audio status table corresponding to the currently used audio device. For example, it can be divided into scenarios of switching the sound output device to PC device 100 and scenarios of disconnecting the screen projection connection.

[0204] For example, in the scenario of switching the sound output device to PC device 100, the implementation process is as shown in steps 207 and 208.

[0205] Step 207: After the user changes the audio output device from the screen mirroring receiver to the screen mirroring sender, read the target audio status recorded in the audio status table.

[0206] Step 208: Set the audio status of the audio device identified by the target identifier in the screen projection sending end to the target audio status.

[0207] See Figure 15 For example, after the user clicks the sound output device selection option 10-6, the PC device 100 will display the sound output device selection control 10-7 in response to the user's operation.

[0208] For example, in some cases, such as scenarios where audio data is played either by the screen-casting sender or the screen-casting receiver, the audio output device selection control 10-7 may include the currently connected screen-casting receiver, for example... Figure 15 TV 1 in the middle, and the screen casting sending end, for example Figure 15 The local machine in the middle.

[0209] For example, in other instances, such as when audio data can also be played from other third-party audio devices, like speakers, the sound output device selection control 10-7 may include, in addition to... Figure 15 The TV 1 shown here, the unit, may also include available third-party audio devices.

[0210] See also Figure 15 If the user selects the local option displayed in the sound output device selection control 10-7, the sound output device will switch from sound output from TV 1 to sound output from PC device 100. In response to the user's action, the content displayed in the sound output device option 10-6 on PC device 100 will change accordingly. Figure 15 TV 1 in the middle has been changed to Figure 16 The local machine in the middle.

[0211] In existing solutions, that is, solutions that do not use the screen casting method provided in this embodiment, after the user switches the audio output device from the TV 200 to the PC device 100, regardless of whether the user modifies the audio status of the audio device currently used by the PC device during the screen casting process, the PC device 100 will produce sound while the TV 200 will not.

[0212] For example Figure 17 As shown, the sound settings option 20 displays the audio device currently used by the PC device 100. For example, the audio status of the headphones 1 is muted. However, in the scenario where the PC device 100 is a sound output device, in the existing implementation, the PC device 100 may play audio data through the default speaker, that is, it will still emit sound.

[0213] Therefore, it can be seen that in the existing solution, when the audio output device is switched from TV 200 to PC device 100 during the screen casting process, the audio device used by PC device 100 to play audio data is not only inconsistent with the user's expectations, but the audio status is also inconsistent with the user's expectations.

[0214] Specifically, in the technical solution provided in this embodiment, since the target identifier recorded in the audio device table and the target audio status in the audio status table are updated in real time according to the user's operation, and during the screen casting process, after the sound output device switches from the TV 200 to the PC device 100, the audio data is played by the audio device corresponding to the target identifier recorded in the audio device table, and the audio status of the audio device playing the audio data is consistent with the target audio status recorded in the audio status table, after the sound output device switches from the TV 200 to the PC device 100, the audio device playing the audio data on the PC device 100, and the audio status of the audio device playing the audio data, can be consistent with the user's expectations, thereby solving the problem of inconsistent audio devices and audio status in the existing solution.

[0215] For example, suppose the target identifier recorded in the audio device table is the identifier of headphone 1, see [link to audio device table]. Figure 18 The sound settings option 20 displays the audio status of PC device 100, which is also the audio status of headphones 1.

[0216] See also Figure 18 For example, the audio state of the headphones 1 is muted. Using the technical solution provided in this embodiment, in the scenario where the PC device 100 is a sound output device, the PC device 100 will not make any sound as a sound output device, but will be in a muted state. That is, in this scenario, both the PC device 100 and the TV 200 only display the picture and do not make any sound.

[0217] Therefore, based on the technical solution provided in this embodiment, after the sound output device is switched from the TV 200 to the PC device 100, the audio device of the PC device 100 playing audio data and the audio status of the audio device playing audio data can be consistent with the user's expectations, thereby solving the problem of inconsistent audio devices and audio status in the existing solution.

[0218] Regarding the scenario of disconnecting the screen mirroring connection mentioned above, the implementation process can be as follows: After receiving the screen mirroring disconnection request, the screen mirroring management module notifies the audio management module to read the target identifier recorded in the audio device table and the target audio status of the audio device corresponding to the target identifier recorded in the audio status table, and then sets the audio status of the audio device identified by the target identifier in the screen mirroring sending end to the target audio status.

[0219] See also Figure 16 For example, after the user clicks the disconnect option 10-5, the PC device 100 will disconnect the screen mirroring connection from the TV 200 in response to the user's operation.

[0220] Understandably, since the screen mirroring information provided by PC device 100 will not be sent to TV 200 through the transmission module after the screen mirroring is disconnected, TV 200 will not display the content displayed on PC device 100's screen, and TV 200 will not play the audio data provided by PC device 100.

[0221] In existing solutions that do not use the screen casting method provided in this embodiment, after the user clicks the disconnect option 10-5, regardless of whether the user has modified the audio status of the audio device currently used by the PC device 100 during the screen casting process, the system will directly set the default audio device of the PC device 100 after disconnecting the screen casting, such as the speakers, while the TV 200 will not make any sound.

[0222] For example Figure 19As shown, sound settings option 20 displays that the audio device currently used by PC device 100 is the headphone 1, and the audio status is muted. After disconnecting the screen mirroring from TV 200, since the system is unaware that the user changed the currently used audio device from the speaker to the headphone 1 and modified the audio status of the headphone 1 during the screen mirroring process, PC device 100 will control the speaker to play audio data and will still emit sound. Obviously, the existing solution will cause the audio device and audio status of PC device 100 playing audio data to be inconsistent with the user's expectations.

[0223] Specifically, in the technical solution provided in this embodiment, since the target identifier recorded in the audio device table and the target audio status recorded in the audio status table corresponding to the target identifier are updated in real time according to the user's operation, and after the screen mirroring is disconnected, the audio device corresponding to the target identifier recorded in the audio device table plays the audio data, and the audio status of the audio device playing the audio data is consistent with the target audio status recorded in the audio status table, after the sound output device is switched from the TV 200 to the PC device 100, the audio device playing the audio data on the PC device 100, and the audio status of the audio device playing the audio data can be consistent with the user's expectations, thereby solving the problem of inconsistent audio devices and audio status in the existing solution.

[0224] For example, suppose the target identifier recorded in the audio device table is the identifier of headphone 1, see [link to audio device table]. Figure 20 The sound settings option 20 displays the audio status of PC device 100, which is also the audio status of headphones 1.

[0225] See also Figure 20 For example, the audio state of the headphones 1 is in a muted state. Using the technical solution provided in this embodiment, after the PC device 100 disconnects the screen projection connection with the TV 200, the audio state of the headphones 1 will be set to the target audio state recorded in the audio state table, that is, the muted state. In this scenario, the PC device 100 only displays the picture and does not make any sound, while the TV 200 neither displays the picture displayed on the screen of the PC device 100 nor makes any sound.

[0226] In addition, it should be noted that since the audio device monitoring module is constantly listening for changes in the audio device during the screen mirroring process, and the audio status monitoring module is constantly listening for user clicks on the audio status switching controls corresponding to the currently used audio device, it is necessary to close the audio device monitoring module and the audio status monitoring module after the screen mirroring ends, i.e., after disconnecting the screen mirroring, to stop monitoring for changes in the audio device and to stop monitoring for changes in the audio status, thereby avoiding the occupation and waste of PC device resources.

[0227] Therefore, before establishing a screen mirroring connection, the system first obtains the audio device currently used by the PC device 100, and records the identifier of the currently used audio device as the target identifier in the audio device table. At the same time, it obtains the current audio status, and records the current audio status as the target audio status in the audio status table. This allows the system to know which audio device on the PC device 100 is in which audio status during the screen mirroring process. Consequently, when the screen mirroring is disconnected or the PC device 100 is used as the audio output device, the PC device 100 can play audio data from the audio device that the user expects, and the audio status of the audio device playing the audio data can be consistent with the user's expectations.

[0228] Furthermore, by recording the identifier of the audio device currently used by PC device 100 as the target identifier in the audio device table, during the screen casting process, when the audio output device is the screen casting receiver, i.e., TV 200, PC device 100 can obtain audio data from the channel corresponding to the audio device identified by the target identifier recorded in the audio device table and send it to TV 200. This ensures that during the screen casting process, no matter how the user switches the audio device used by the screen casting sender, TV 2, as the audio output device, can obtain playable audio data and thus play the sound.

[0229] Furthermore, it should be noted that in practical applications, if the user clicks the volume control 20-4 to adjust the volume, the PC device will respond to the user's operation by changing the volume of the output audio data. Thus, if the audio output device is a large-screen device, since the audio data received from the PC device is obtained based on the adjusted volume encoding, the audio data played on the large-screen device will be synchronized with the volume adjusted by the volume control 20-4. For example, when the user increases the volume using the volume control 20-4, the volume of the audio data played on the large-screen device will also increase, and when the user decreases the volume using the volume control 20-4, the volume of the audio data played on the large-screen device will also decrease.

[0230] Furthermore, in practical applications, the technical solution provided in this embodiment can be implemented in two ways.

[0231] Method 1.1: During the screen casting process, the audio status of the screen casting sending end is set by the audio management module.

[0232] See Figure 21 For example, to better understand the technical solution provided in this embodiment, the PC is muted before screen mirroring, and then audio is played for screen mirroring (when connecting the large screen device for the first time, such as...). Figure 21The following scenario illustrates the process of switching the sound from TV 1 to the PC, then switching back to TV 1, and finally disconnecting the screen mirroring connection.

[0233] See also Figure 21 For example, when a user clicks to cast the screen, the upper-level screen casting management module (e.g., MultiCastMgr) notifies the audio management module (e.g., AudioControl) through the audioContol.Init() method; then the audio management module obtains the identifier of the currently used audio device through SaveMuteState(), saves the identifier as the initial value, and calls SetDeviceMute() to mute the currently used audio device. For example, if the currently used audio device is a speaker, the speaker is muted; if the currently used audio device is headphone 1, headphone 1 is muted. After that, ListenDeviceChange() is started to monitor whether the audio sending device has changed.

[0234] It should be noted that, in practical applications, the order in which the audio module calls SetDeviceMute() to mute the currently used audio device and calls ListenDeviceChange() to start monitoring whether the audio device used by the PC has changed is not important.

[0235] See also Figure 21 For example, during the execution of the above operations, the upper layer of the product will notify the corresponding protocol layer, that is, the audio output device of the transmission module is the large screen (TV1). At this time, after receiving the upper layer notification, the transmission module obtains the currently used audio device through audioService().

[0236] Accordingly, in response to the request from audioService(), the audio management module will send the channel information corresponding to the audio device identified by the target identifier recorded in the audio device table to the transmission module via DeviceVoiceMode().

[0237] Accordingly, after receiving the channel information sent by the audio management module, the transmission module will obtain audio data from the channel pointed to by the channel information, and use SpeakerENcoder() to encode and decode the obtained audio data, and then package and send it to TV1 so that TV1 can decode the audio and output sound.

[0238] See also Figure 21For example, when the sound is switched to the local device, the upper-level screen mirroring management module notifies the audio management module through the audioContol.Init() method. The upper-level product notifies the transmission module not to encode, decode, and package the audio and send it to TV 1, that is, to stop encoding and call Audiostop(). In this way, TV 1 cannot receive audio data packets and cannot decode and obtain audio data for playback, so TV 1 will be muted.

[0239] Meanwhile, the audio management module uses the HAudioDevice() method to query the audio device currently in use by the PC, that is, to determine it by querying the target identifier recorded in the audio device table.

[0240] Accordingly, when the audio device is a speaker, the speaker is set to output sound using SetDeviceMute(); when the audio device is headphone 1, headphone 1 is set to output sound using SetDeviceMute().

[0241] Next, when switching from local audio to large screen audio, the process is similar to the process executed when the large screen is the default audio source during the initial screen mirroring connection, except that ListenDeviceChange() does not need to be called again.

[0242] See also Figure 21 For example, when the screen mirroring connection is disconnected, the audio management module notifies the transmission module to disconnect the screen mirroring connection with TV 1 via Devicestop(), and simultaneously disables the audio device monitoring module that monitors whether the PC's audio device has changed via StopNotifyDeviceChange(). At the same time, the audio management module uses HAudioDevice() to query the audio device currently used by the PC.

[0243] Accordingly, when the audio device is a speaker, the speaker is set to output sound using SetDeviceMute(); when the audio device is headphone 1, headphone 1 is set to output sound using SetDeviceMute().

[0244] In addition, the audio management module will also notify the transmission module not to encode, decode, and package the audio data before sending it to TV 1, i.e., stop encoding, and call Audiostop(). As a result, TV 1 will not receive the audio data packets and will not be able to decode and play the audio data, so TV 1 will be muted.

[0245] Furthermore, it is understandable that during the screen mirroring process, ListenDeviceChange() will be running continuously, and when it detects a change in the audio device, it will send the changed audio device identifier to the audio management module to update the target identifier in the audio device table.

[0246] Method 1.2: During the screen casting process, the audio management module sets the audio status of the screen casting sending end based on the audio status monitored by the audio status monitoring module.

[0247] See Figure 22 For example, when a user clicks to cast the screen, the upper-level screen casting management module notifies the audio management module through the audioContol.Init() method; then the audio management module obtains the identifier and audio status of the currently used audio device through SaveMuteState(), saves the identifier and audio status of the currently used audio device as initial values, and calls SetDeviceMute() to mute the currently used audio device. For example, if the currently used audio device is a speaker, the speaker is muted; if the currently used audio device is headphone 1, headphone 1 is muted. After that, ListenDeviceChange() is started to monitor whether the audio sending device has changed, and ListenVolChange() is started to monitor whether the audio status of the currently used audio device has changed.

[0248] It should be noted that in practical applications, the audio module's operations of calling SetDeviceMute() to mute the currently used audio device, calling ListenDeviceChange() to enable monitoring of whether the audio device used by the PC has changed, and calling ListenVolChange() to monitor whether the audio status of the currently used audio device has changed, do not distinguish the execution order.

[0249] See also Figure 22 For example, during the execution of the above operations, the upper layer of the product will notify the corresponding protocol layer, that is, the audio output device of the transmission module is the large screen (TV1). At this time, after receiving the upper layer notification, the transmission module obtains the currently used audio device through audioService().

[0250] Accordingly, in response to the request from audioService(), the audio management module will send the channel information corresponding to the audio device identified by the target identifier recorded in the audio device table to the transmission module via DeviceVoiceMode().

[0251] Accordingly, after receiving the channel information sent by the audio management module, the transmission module will obtain audio data from the channel pointed to by the channel information, and use SpeakerENcoder() to encode and decode the obtained audio data, and then package and send it to TV1 so that TV1 can decode the audio and output sound.

[0252] See also Figure 22For example, when the sound is switched to the local device, the upper-level screen mirroring management module notifies the audio management module through the audioContol.Init() method. The upper-level product notifies the transmission module not to encode, decode, and package the audio and send it to TV 1, that is, to stop encoding and call Audiostop(). In this way, TV 1 cannot receive audio data packets and cannot decode and obtain audio data for playback, so TV 1 will be muted.

[0253] Meanwhile, the audio management module uses the HAudioDevice() method to query the audio device currently used by the PC and the audio status of that audio device. That is, it determines the device by querying the target identifier recorded in the audio device table and the audio status of the corresponding audio device recorded in the audio status table.

[0254] Accordingly, when the audio device is a speaker and the audio state is not muted, SetDeviceMute() is used to set the speaker to play sound; when the audio device is a speaker and the audio state is muted, SetDeviceMute() is used to mute the speaker.

[0255] Accordingly, when the audio device is headphone 1 and the audio state is not muted, setDeviceMute() is used to set headphone 1 to play sound; when the audio device is headphone 1 and the audio state is muted, setDeviceMute() is used to mute headphone 1.

[0256] Next, when switching from local audio to large screen audio, the process is similar to the process executed when the large screen is the default audio source during the initial screen mirroring connection, except that ListenDeviceChange() and ListenVolChange() do not need to be called again.

[0257] See also Figure 22 For example, when the screen mirroring connection is disconnected, the audio management module notifies the transmission module to disconnect the screen mirroring connection with TV 1 via Devicestop(). Simultaneously, it disables the audio device monitoring module (which monitors whether the PC's audio device has changed) via StopNotifyDeviceChange(), and the audio status monitoring module (which monitors whether the audio status has changed) via StopNotifyVolChange(). At the same time, the audio management module uses HAudioDevice() to query the PC's currently used audio device and its corresponding audio status.

[0258] Accordingly, when the audio device is a speaker and the audio state is not muted, SetDeviceMute() is used to set the speaker to play sound; when the audio device is a speaker and the audio state is muted, SetDeviceMute() is used to mute the speaker.

[0259] Accordingly, when the audio device is headphone 1 and the audio state is not muted, setDeviceMute() is used to set headphone 1 to play sound; when the audio device is headphone 1 and the audio state is muted, setDeviceMute() is used to mute headphone 1.

[0260] In addition, the audio management module will also notify the transmission module not to encode, decode, and package the audio data before sending it to TV 1, i.e., stop encoding, and call Audiostop(). As a result, TV 1 will not receive the audio data packets and will not be able to decode and play the audio data, so TV 1 will be muted.

[0261] Furthermore, it is understandable that during the screen mirroring process, ListenDeviceChange() and ListenVolChange() will be running continuously, and when a change in the audio device is detected, the changed audio device identifier will be sent to the audio management module to update the target identifier in the audio device table.

[0262] See also Figure 22 For example, if ListenDeviceChange() detects a change in the audio device currently used by the PC during screen mirroring, it sends the identifier of the switched audio device, such as deviceId, to the audio management module. The audio management module then modifies the target identifier recorded in the audio device table using SaveMuteState(). Simultaneously, it uses the switched audio device identifier deviceId as a query parameter to query the audio status of the audio device corresponding to that identifier deviceId, as monitored by the audio status monitoring module, using FindDeviceMuteState(). After retrieving the audio status, it sends it to the audio management module via VolChagestate().

[0263] To better understand Figure 21 and 22 The following describes the screen projection processing implemented based on the technical solution provided in this embodiment in three scenarios. Figure 22 Taking the screen mirroring scenario shown as an example, where both the audio device monitoring module and the audio status monitoring module are activated simultaneously, combined with... Figures 23 to 25 Please provide a detailed explanation.

[0264] See Figure 23For example, the screen casting sender is referred to as PC and the screen casting receiver is referred to as TV 1.

[0265] The PC has a screen mirroring application installed, which includes a screen mirroring management module, an audio management module, and a transmission module.

[0266] In addition, the system used by the PC provides an audio status monitoring module.

[0267] Assuming that when PC and TV1 establish a screen mirroring connection for the first time, the audio output device is set to TV1 by default, meaning that the sound will come from TV1.

[0268] See also Figure 23 For example, when a user clicks... Figure 4 or Figure 5 When the "Connect Now" option 10-1 is selected, the screen casting management module will receive a screen casting connection request to TV 1, i.e., step 301.

[0269] Next, the screen mirroring management module will execute step 302, sending an initialization command to the audio management module.

[0270] Accordingly, after receiving the initialization command sent by the screen mirroring management module, the audio management module will execute step 303 to perform initialization operations, such as by calling the audioContol.Init() interface.

[0271] Next, after initialization, the audio management module will obtain the audio device currently in use on the local machine, as well as the audio status of that audio device.

[0272] In practical applications, the audio management module can obtain the audio device currently in use on the local machine and the audio status of that device through a pre-compiled program interface.

[0273] For example, in this embodiment, the program interface is referred to as SaveMuteState().

[0274] The audio management module can obtain the audio device currently used by the PC and the audio status of that device by calling SaveMuteState().

[0275] Accordingly, if the audio management module obtains in step 304 that the currently used audio device is a speaker and the speaker's audio state is muted, then step 305 will be executed next. Specifically, a mapping relationship will be established between the speaker's identifier obtained in step 304 and the speaker's audio state, the state value corresponding to the muted state, such as "0", for subsequent use.

[0276] In addition, it is understandable that in practical applications, it is also necessary to record the obtained speaker identification number in the audio device table and the status value of the corresponding audio status in the audio status table.

[0277] Then, the audio management module sends a request to the audio device monitoring module to monitor the audio device selected on the local machine, i.e., execute step 306; at the same time, it sends a request to the audio status monitoring module to monitor the local audio status, i.e., execute step 307.

[0278] Specifically, in this embodiment, the operation of monitoring whether the audio has changed is initiated by calling the audio device monitoring module, for example, through the pre-compiled ListenDeviceChange() interface; the operation of monitoring the audio status of the PC is initiated by calling the audio status monitoring module, for example, through calling the pre-compiled ListenVolChange() interface.

[0279] For example, after the audio status monitoring module is started, it will consistently execute the operation of step 308 to monitor whether the audio status changes throughout the entire screen projection process, i.e. before the screen projection is disconnected; after the audio device monitoring module is started, it will consistently execute the operation of step 309 to monitor whether the audio device changes throughout the entire screen projection process, i.e. before the screen projection is disconnected.

[0280] Accordingly, if a change in the audio state is detected, step 310 is executed; otherwise, step 308 is executed. If a change in the audio device is detected, step 311 is executed; otherwise, step 309 is executed.

[0281] For example, due to Figure 23 In the scenario shown, before the audio device changes, the PC uses a speaker as its audio device, and the speaker's target audio state is muted. Therefore, when the audio state monitoring module detects a change in the audio state, it means that the user has clicked the audio state switching control, causing the speaker's audio state to change to non-mute. Therefore, step 310 specifically sends the state value corresponding to the non-mute state, such as "1", to the audio management module.

[0282] Next, the audio management module updates the status value of the target audio status recorded in the audio status table according to the status value "1" corresponding to the received non-silent status, and at the same time updates the status value in the mapping relationship between the target identifier codes, specifically updating the status value from 0 to 1, that is, executing step 311.

[0283] See also Figure 23 If a change in the audio device's output is detected during the screen mirroring process, such as a change from a speaker to Headphone 1, the identification number of Headphone 1 will be sent to the audio management module, i.e., step 312 will be executed.

[0284] Next, the audio management module will establish a mapping relationship between the identifier of headphone 1 and the corresponding status value of the audio status of headphone 1, such as 0, i.e., execute step 313.

[0285] At the same time, the audio management module will also notify the transmission module that the sound output device is TV1, and set the audio device currently used by the local machine through an interface for setting the audio status, such as SetDeviceMute(), for example, to mute Headphone1, that is, execute step 314 to set the local machine to a mute state.

[0286] It should be noted that in practical applications, the operation of step 314 can be performed synchronously with step 304 after the audio management module has completed step 303. This embodiment does not impose any restrictions on this.

[0287] Next, after completing step 314 above, the audio management module will obtain audio data from the channel corresponding to the headphone 1, i.e., execute step 315, and send the audio data to the transmission module, i.e., execute step 316.

[0288] Accordingly, after receiving the audio data, the transmission module will encode the audio data in the manner described in the above-mentioned step embodiment to obtain an audio data packet, that is, execute step 317.

[0289] Next, the transmission module will send the obtained audio data packets to TV 1 through the established screen mirroring link, i.e., execute step 318.

[0290] Accordingly, after receiving the audio data packet sent by the transmission module, TV 1 will perform step 319 to decode the audio data packet, thereby restoring the audio data and playing the audio data, so that TV 1 can produce sound.

[0291] This allows the PC to be muted first, then play a video before screen mirroring, with sound coming from TV 1. During the mirroring process, the audio management module monitors the audio devices and their corresponding audio statuses, dynamically updating the target identifier recorded in the audio device table and the audio status value of the corresponding audio device recorded in the audio status table. This ensures that the user's use of the mirroring function is not affected, while also allowing real-time monitoring of the user's modifications to the audio devices used on the PC and their audio statuses during mirroring. This enables the acquisition of audio data from the corresponding channel of the currently used audio device and its transmission to TV 1, ensuring that TV 1 always receives playable audio data when the sound output device is set to TV 1.

[0292] Furthermore, after the PC sends the screen mirroring information to TV 1, and TV 1 begins playing the video and audio data, if the user switches the audio output device from TV 1 to the PC, the process for setting the PC's audio status can be found in [link to relevant documentation]. Figure 24 .

[0293] For example, if the screen mirroring management module detects the user's operation of switching the audio output device and determines that the audio output device has been switched to the local machine, i.e. after the operation in step 320, the screen mirroring management module will resend the initialization command to the audio management module, i.e., execute step 321.

[0294] Accordingly, after receiving the initialization command sent by the screen mirroring management module, the audio management module will execute step 322 to perform initialization operations, such as by calling the audioContol.Init() interface.

[0295] Next, after initialization, the audio management module reads the recorded target identifier from the audio device table and reads the recorded target audio status value corresponding to the target identifier from the audio status table.

[0296] In practical applications, the target identifier and status value can be obtained from the mapping table between audio devices and status values.

[0297] For example, if the audio device corresponding to the target identifier recorded in the audio device table is headphone 1, and the status value corresponding to headphone 1 recorded in the audio status table is "1", then after initialization, the audio management module determines that the audio device is headphone 1 according to the mapping relationship and the status value of the audio status is "1", which is the content of step 323.

[0298] Next, the audio management module sets the device to non-mute state based on the read status value "1", that is, it executes the operation in step 324, and then executes the operation in step 325 to play audio data.

[0299] Specifically, since the audio device currently used by the PC is headphone 1, step 324 sets the audio state of headphone 1 to non-mute, and step 325 has headphone 1 play audio data.

[0300] For example, if the audio status value of the headset 1 recorded in the audio status table is "0", that is, the user changed the audio status of the headset 1 on the PC to mute before or during screen casting, then after initialization, the audio management module will read the recorded audio status value of "0" from the audio status table when the currently used audio device is still headset 1.

[0301] Accordingly, after the audio management module reads the status value "0" from the audio status table, it will set the headphone 1 to mute state based on the read status value "0" and then play the audio data.

[0302] Understandably, since the audio status of Headphone 1 is set to mute, even if the audio data is played by Headphone 1, no sound will be emitted during the playback process because the audio status of Headphone 1 is mute.

[0303] In addition, it should be noted that although in Figure 24 The operations performed by the audio status monitoring module and the audio device monitoring module are not shown in the diagram. However, once the audio status monitoring module and the audio device monitoring module are started, they remain active throughout the entire screen projection process. That is, during steps 320 to 325, as long as the user modifies the audio status and audio device... Figure 23 The operations from step 308 to step 313 will then be executed.

[0304] In addition, due to Figure 24 The screen mirroring scenario shown is that the user switches the audio output device from TV1 to PC. Therefore, the transmission module will no longer send audio data packets to TV1. Since TV1 cannot receive audio data packets, it will not emit sound.

[0305] Furthermore, it is understood that since the technical solution provided in this embodiment is for audio data in screen projection scenarios, the transmission of video data is not shown in the figure.

[0306] This enables the switching of the audio output device from TV 1 to PC during screen mirroring. Then, the audio management module sets the PC's audio status based on the target identifier recorded in the audio device table and the status value corresponding to the target identifier recorded in the audio status table. This ensures that after the audio output device is changed to PC, the audio device playing the audio data on the PC, as well as the audio status of the audio device playing the audio data, are consistent with the user's expectations.

[0307] Furthermore, in completing Figure 24 After the operation, the PC starts playing audio data, and TV 1 continues playing video / picture data. If the user clicks... Figure 8 For the disconnection option 10-5, the process for setting the PC's audio status can be found in [link to documentation]. Figure 25 .

[0308] For example, if the screen mirroring management module receives a screen mirroring disconnection request, i.e., the operation in step 326, the screen mirroring management module will resend the initialization command to the audio management module, i.e., execute step 327.

[0309] Accordingly, after receiving the initialization command sent by the screen mirroring management module, the audio management module will perform step 328 to perform initialization operations, such as by calling the audioContol.Init() interface.

[0310] Next, after initialization, the audio management module reads the recorded target identifier from the audio device table and reads the recorded target audio status value corresponding to the target identifier from the audio status table.

[0311] In practical applications, the target identifier and status value can be obtained from the mapping table between audio devices and status values.

[0312] For example, if the audio device corresponding to the target identifier recorded in the audio device table is Headphone 1, and the status value corresponding to Headphone 1 recorded in the audio status table is "1", then after initialization, the audio management module determines the audio device as Headphone 1 according to the mapping relationship and the status value of the audio status is "1", which is the content of step 329.

[0313] Next, the audio management module sets the device to non-mute state based on the read status value "1", that is, it executes the operation in step 330, and then executes the operation in step 331 to play audio data.

[0314] Specifically, since the audio device currently used by the PC is headphone 1, step 330 sets the audio state of headphone 1 to non-mute, and step 331 has headphone 1 play audio data.

[0315] For example, if the audio status value of the headset 1 recorded in the audio status table is "0", that is, the user changed the audio status of the headset 1 on the PC to mute before or during screen casting, then after initialization, the audio management module will read the recorded audio status value of "0" from the audio status table when the currently used audio device is still headset 1.

[0316] Accordingly, after the audio management module reads the status value "0" from the audio status table, it will set the headphone 1 to mute state based on the read status value "0" and then play the audio data.

[0317] Understandably, since the audio status of Headphone 1 is set to mute, even if the audio data is played by Headphone 1, no sound will be emitted during the playback process because the audio status of Headphone 1 is mute.

[0318] In addition, after disconnecting the screen mirroring, the audio management module will also send a request to the audio status monitoring module to stop monitoring the local audio status, i.e., execute step 332; and send a request to the audio device monitoring module to stop monitoring the audio device selected on the local machine, i.e., execute step 334.

[0319] Accordingly, after receiving a request from the audio management module to stop monitoring the local audio status, the audio status monitoring module will execute step 333 to stop listening to the local audio status.

[0320] Similarly, after receiving a request from the audio management module to stop monitoring the audio device selected by the local unit, the audio device monitoring module will execute step 335 to stop listening to the audio device selected by the local unit.

[0321] For example, the audio status monitoring module can call the pre-encapsulated StopNotifyVolChange() interface to stop listening to the local audio status; the audio device monitoring module can call the pre-encapsulated StopNotifyDeviceChange() interface.

[0322] Therefore, the screen casting solution provided in this application sets the audio status of the screen casting sending end by the audio management module according to the contents recorded in the audio status table and the audio device table during the screen casting process. In this way, no matter which protocol the screen casting application uses, the audio management module only needs to set it in a unified way, which greatly reduces the development workload and maintenance difficulty.

[0323] Method 2.1: During the screen mirroring process, the audio status of the screen mirroring sending end is set by the transmission module.

[0324] See Figure 26 For example, to better understand the technical solution provided in this embodiment, the PC is muted before screen mirroring, and then audio is played for screen mirroring (when connecting the large screen device for the first time, such as...). Figure 26 The following scenario illustrates the process of switching the sound from TV 1 to the PC, then switching back to TV 1, and finally disconnecting the screen mirroring connection.

[0325] See also Figure 26For example, when a user clicks to cast the screen, the upper-level screen casting management module (e.g., MultiCastMgr) notifies the audio management module (e.g., AudioControl) through the audioContol.Init() method; then the audio management module obtains the identifier of the currently used audio device through SaveMuteState(), saves the identifier as the initial value, and starts ListenDeviceChange() to monitor whether the audio sending device has changed.

[0326] See also Figure 26 After the PC and TV1 establish a screen mirroring connection, the audio management module will call DeviceVoiceMode() to send the channel information of the currently used audio device obtained through SaveMuteState() to the corresponding protocol layer. That is, the sound output device of the transmission module is the large screen (TV1).

[0327] Accordingly, after receiving the channel information sent by the audio management module, the transmission module retrieves the audio data from the channel indicated by the channel information, performs audio encoding and decoding on the retrieved audio data using SpeakerENcoder(), and then packages and sends it to TV 1, enabling TV 1 to decode the audio and output the sound. Simultaneously, the transmission module calls HAudioDevice() to query the currently used audio device recorded in the audio management module.

[0328] Accordingly, the audio management module will send the identifier of the queried audio device, i.e., deviceId, to the transmission module, which will then call SetDeviceMute() to mute the audio device identified by deviceId. For example, if the currently used audio device is a speaker, the speaker will be muted; if the audio device identified by deviceId is headphone 1, headphone 1 will be muted.

[0329] See also Figure 26 For example, when the sound is switched to the local device, the upper-level screen mirroring management module notifies the audio management module through the audioContol.Init() method. The upper-level product notifies the transmission module not to encode, decode, and package the audio and send it to TV 1, that is, to stop encoding and call Audiostop(). As a result, TV 1 cannot receive the audio data packets and cannot decode and obtain the audio data for playback, so TV 1 will be muted.

[0330] Accordingly, after receiving the notification from the upper layer, the transmission module will call HAudioDevice() to query the currently used audio device recorded in the audio management module.

[0331] Accordingly, the audio management module will send the identifier of the queried audio device, i.e., deviceId, to the transmission module, which will then call SetDeviceMute() to mute the audio device identified by deviceId. For example, if the currently used audio device is a speaker, the speaker will be muted; if the audio device identified by deviceId is headphone 1, headphone 1 will be muted.

[0332] Next, when switching from local audio to large screen audio, the process is similar to the process executed when the large screen is the default audio source during the initial screen mirroring connection, except that ListenDeviceChange() does not need to be called again.

[0333] See also Figure 26 For example, when the screen mirroring connection is disconnected, the audio management module notifies the transmission module to disconnect the screen mirroring connection with TV 1 via Devicestop(), and at the same time shuts down the audio status monitoring module that monitors the audio status of the PC via StopNotifyDeviceChange().

[0334] Accordingly, after receiving the notification from the upper layer, the transmission module will call HAudioDevice() to query the currently used audio device recorded in the audio management module.

[0335] Accordingly, the audio management module will send the identifier of the queried audio device, i.e., deviceId, to the transmission module, which will then call SetDeviceMute() to mute the audio device identified by deviceId. For example, if the currently used audio device is a speaker, the speaker will be muted; if the audio device identified by deviceId is headphone 1, headphone 1 will be muted.

[0336] In addition, the audio management module will also notify the transmission module not to encode, decode, and package the audio data before sending it to TV 1, i.e., stop encoding, and call Audiostop(). As a result, TV 1 will not receive audio data packets and will not be able to decode and play the audio data, so TV 1 will be muted.

[0337] Furthermore, it is understandable that during the screen mirroring process, ListenDeviceChange() will be running continuously, and when it detects a change in the audio device, it will send the changed audio device identifier to the audio management module to update the target identifier in the audio device table.

[0338] Method 2.2: During the screen mirroring process, the audio status of the screen mirroring sending end is set by the transmission module based on the audio status monitored by the audio status monitoring module.

[0339] See Figure 27For example, when a user clicks to cast the screen, the upper-level screen casting management module notifies the audio management module through the audioContol.Init() method; then the audio management module obtains the identifier and audio status of the currently used audio device through SaveMuteState(), saves the identifier and audio status of the currently used audio device as initial values, and starts ListenDeviceChange() to monitor whether the audio sending device has changed and ListenVolChange() to monitor whether the audio status of the currently used audio device has changed.

[0340] See also Figure 27 After the PC and TV1 establish a screen mirroring connection, the audio management module will call DeviceVoiceMode() to send the channel information of the currently used audio device obtained through SaveMuteState() to the corresponding protocol layer. That is, the sound output device of the transmission module is the large screen (TV1).

[0341] Accordingly, after receiving the channel information sent by the audio management module, the transmission module retrieves the audio data from the channel indicated by the channel information, performs audio encoding and decoding on the retrieved audio data using SpeakerENcoder(), and then packages and sends it to TV 1, enabling TV 1 to decode the audio and output sound. Simultaneously, the transmission module calls HAudioDevice() to query the currently used audio device and its audio status recorded in the audio management module.

[0342] Accordingly, the audio management module will send the queried audio device identifier (deviceId) and the audio status of the audio device identified by the identifier to the transmission module. The transmission module will then call SetDeviceMute() to set the audio device identified by deviceId to mute. For example, if the currently used audio device is a speaker, the speaker will be muted; if the audio device identified by deviceId is headphone 1, headphone 1 will be muted.

[0343] See also Figure 27 For example, when the sound is switched to the local device, the upper-level screen mirroring management module notifies the audio management module through the audioContol.Init() method. The upper-level product notifies the transmission module not to encode, decode, and package the audio and send it to TV 1, that is, to stop encoding and call Audiostop(). As a result, TV 1 cannot receive the audio data packets and cannot decode and obtain the audio data for playback, so TV 1 will be muted.

[0344] Accordingly, the audio management module will send the queried audio device identifier (deviceId) and the audio status of the audio device identified by the identifier to the transmission module, and the transmission module will call SetDeviceMute() to set the audio device identified by the deviceId to the corresponding audio status.

[0345] For example, if the currently used audio device is a speaker and the obtained audio status of the speaker is muted, set the speaker to mute; if the currently used audio device is a speaker and the obtained audio status of the speaker is not muted, set the speaker to not mute.

[0346] For example, if the audio device identified by deviceId is headphone 1 and the obtained audio state corresponding to the speaker is muted, then headphone 1 is muted; if the audio device identified by deviceId is headphone 1 and the obtained audio state corresponding to the speaker is not muted, then headphone 1 is not muted.

[0347] Next, when switching from local audio to large screen audio, the process is similar to the process executed when the large screen is the default audio source during the initial screen mirroring connection, except that ListenDeviceChange() and ListenVolChange() do not need to be called again.

[0348] See also Figure 27 For example, when the screen mirroring connection is disconnected, the audio management module notifies the transmission module to disconnect the screen mirroring connection with TV 1 via Devicestop(). Simultaneously, it disables the audio device monitoring module (which monitors whether the PC's audio device has changed) via StopNotifyDeviceChange(), and the audio status monitoring module (which monitors whether the audio status has changed) via StopNotifyVolChange(). At the same time, the transmission module calls HAudioDevice() to query the audio management module for the currently used audio device and its audio status.

[0349] Accordingly, the audio management module will send the queried audio device identifier (deviceId) and the audio status of the audio device identified by the identifier to the transmission module, and the transmission module will call SetDeviceMute() to set the audio device identified by the deviceId to the corresponding audio status.

[0350] In addition, the audio management module will also notify the transmission module not to encode, decode, and package the audio data before sending it to TV 1, i.e., stop encoding, and call Audiostop(). As a result, TV 1 will not receive audio data packets and will not be able to decode and play the audio data, so TV 1 will be muted.

[0351] Furthermore, it is understandable that during the screen mirroring process, ListenDeviceChange() and ListenVolChange() will be running continuously, and when a change in the audio device is detected, the changed audio device identifier will be sent to the audio management module to update the target identifier in the audio device table.

[0352] See also Figure 27 For example, if ListenDeviceChange() detects a change in the audio device currently used by the PC during screen mirroring, the audio management module can obtain the identifier of the changed audio device through GetDefaultSpeakerDevice().

[0353] Accordingly, the audio device monitoring module will send the changed audio device identifier (deviceId) to the audio management module for management.

[0354] Next, the audio management module uses the received deviceId as a query parameter and retrieves the audio status of the audio device corresponding to the deviceId from the audio status monitoring module through FindDevceMuteState().

[0355] Accordingly, the audio status management module will send the queried audio status to the audio management module via VolChangestate().

[0356] To better understand Figure 26 and 27 The following describes the screen projection processing implemented based on the technical solution provided in this embodiment in three scenarios. Figure 27 Taking the screen mirroring scenario shown as an example, where both the audio device monitoring module and the audio status monitoring module are activated simultaneously, combined with... Figures 28 to 30 Please provide a detailed explanation.

[0357] See Figure 28 For example, the screen casting sender is referred to as PC and the screen casting receiver is referred to as TV 1.

[0358] The PC has a screen mirroring application installed, which includes a screen mirroring management module, an audio management module, and a transmission module.

[0359] In addition, the system used by the PC provides an audio status monitoring module.

[0360] Assuming that when PC and TV1 establish a screen mirroring connection for the first time, the audio output device is set to TV1 by default, meaning that the sound will come from TV1.

[0361] For example, when a user clicks Figure 4 or Figure 5 When the "Connect Now" option 10-1 is selected, the screen casting management module will receive a screen casting connection request to TV 1.

[0362] See also Figure 28 For example, the technical solution provided in this embodiment includes steps 401 to 419. It is not difficult to see that steps 401 to 413 and steps 415 to 419 in this embodiment are similar to... Figure 23 Steps 301 to 311 and steps 315 to 319 in the illustrated embodiment are largely the same and will not be repeated here. The differences will be explained below.

[0363] See also Figure 28 The transmission module executes step 414 to put the local device into a silent state.

[0364] It should be noted that, since the operation of setting the local audio status is implemented by the transmission module in this embodiment, the transmission module will set the local audio status through the audio status setting interface, such as SetDeviceMute().

[0365] For example, when the transmission module sets the local audio status using SetDeviceMute(), it will set the audio device identified by the obtained identifier to a mute state.

[0366] For example, if the audio device identified by the queried identifier is Headphone 1, you can use SetDeviceMute() to set Headphone 1 to a mute state.

[0367] For example, if the audio device identified by the queried identifier is a speaker, the speaker can be muted using SetDeviceMute().

[0368] Understandably, since the audio output device is TV1, during the screen mirroring process, regardless of whether the audio device currently used by the PC is in a muted or unmute state, the audio device currently used by the PC, such as the speaker, will not produce sound. Therefore, in this scenario, the transmission modules can call the interface to set the audio state to set the local audio device, such as muting the speaker.

[0369] This allows the PC to be muted first, then play a video before screen mirroring, with sound coming from TV 1. During the mirroring process, the audio management module monitors the audio devices and their corresponding audio statuses, dynamically updating the target identifier recorded in the audio device table and the audio status value of the corresponding audio device recorded in the audio status table. This ensures that the user's use of the mirroring function is not affected, while also allowing real-time monitoring of the user's modifications to the audio devices used on the PC and their audio statuses during mirroring. This enables the acquisition of audio data from the corresponding channel of the currently used audio device and its transmission to TV 1, ensuring that TV 1 always receives playable audio data when the sound output device is set to TV 1.

[0370] Furthermore, after the PC sends the screen mirroring information to TV 1, and TV 1 begins playing the video and audio data, if the user switches the audio output device from TV 1 to the PC, the process for setting the PC's audio status can be found in [link to relevant documentation]. Figure 29 .

[0371] See Figure 29 For example, the technical solution provided in this embodiment includes steps 420 to 427. It is not difficult to see that steps 420 to 422, 424, and 427 in this embodiment are similar to... Figure 24 Steps 320 to 323 and step 325 in the illustrated embodiment are largely the same and will not be repeated here. The differences will be explained below.

[0372] See also Figure 29 In step 423, the transmission module sends an audio device and audio status query request to the audio management module.

[0373] For example, the transport module can send an audio device and audio status query request to the audio management module via a pre-compiled HAudioDevice() function.

[0374] Accordingly, the audio management module will send the queried deviceId and the audio status queried through the pre-compiled VolChangestate() to the transmission module.

[0375] In step 425, the audio management module sends the identification number and status value 1 of the headset 1 to the transmission module, that is, it sends the currently used audio device identification number deviceId and the audio status of the audio device to the transmission module.

[0376] In step 426, the transmission module sets the earphone 1 to a non-mute state.

[0377] For example, after receiving the audio device identifier and audio status from the audio module, the transmission module sets the local audio status using SetDeviceMute().

[0378] Specifically, based on the obtained identifier, the audio device identified by that identifier is set to the queried audio status.

[0379] For example, if the audio device identified by the queried identifier is Headphone 1 and the corresponding audio state is muted, then Headphone 1 is muted using SetDeviceMute(); if the audio device identified by the queried identifier is Headphone 1 and the corresponding audio state is not muted, then Headphone 1 is not muted using SetDeviceMute().

[0380] For example, if the audio device identified by the queried identifier is a speaker and the corresponding audio state is muted, SetDeviceMute() can be used to mute the speaker; if the audio device identified by the queried identifier is a speaker and the corresponding audio state is not muted, SetDeviceMute() can be used to unmute the speaker.

[0381] This enables the switching of the audio output device from TV 1 to PC during screen mirroring. Then, the audio management module sets the PC's audio status based on the target identifier recorded in the audio device table and the status value corresponding to the target identifier recorded in the audio status table. This ensures that after the audio output device is changed to PC, the audio device playing the audio data on the PC, as well as the audio status of the audio device playing the audio data, are consistent with the user's expectations.

[0382] Furthermore, in completing Figure 29 After the operation, the PC starts playing audio data, and TV 1 continues playing video / picture data. If the user clicks... Figure 8 For the disconnection option 10-5, the process for setting the PC's audio status can be found in [link to documentation]. Figure 30 .

[0383] See Figure 30 For example, the technical solution provided in this embodiment includes steps 428 to 439. It is not difficult to see that steps 428 to 435 in this embodiment are similar to... Figure 29 Steps 420 to 427 in the illustrated embodiment are largely the same, and steps 436 to 439 are the same as... Figure 25 Steps 323 to 325 in the illustrated embodiment are largely the same and will not be described again here.

[0384] Therefore, the screen mirroring solution provided in this application sets the audio status of the screen mirroring sending end through the transmission module during the screen mirroring process. This allows screen mirroring applications using different protocols to be customized according to actual needs, making the screen mirroring solution provided in this application more diversified.

[0385] Furthermore, it should be understood that the method interfaces used in the above embodiments are merely examples listed for a better understanding of the technical solutions of this embodiment, and are not intended as the sole limitation of this embodiment.

[0386] Furthermore, it should be noted that in practical application scenarios, the system projection methods provided in the above embodiments, implemented by electronic devices, can also be executed by a chip system included in the electronic device. This chip system may include a processor. The chip system can be coupled to a memory, enabling it to call computer programs stored in the memory during runtime to implement the steps executed by the electronic device. The processor in the chip system can be an application processor or a non-application processor.

[0387] In addition, this application embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the above-mentioned related method steps to implement the system screen projection method in the above embodiment.

[0388] In addition, this application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform the above-mentioned related steps to realize the system projection method in the above embodiments.

[0389] In addition, embodiments of this application also provide a chip (which may also be a component or module), which may include one or more processing circuits and one or more transceiver pins; wherein the transceiver pins and the processing circuits communicate with each other through internal connection paths, and the processing circuits execute the above-mentioned related method steps to implement the system projection method in the above embodiments, so as to control the receiving pins to receive signals and control the transmitting pins to transmit signals.

[0390] Furthermore, as can be seen from the above description, the electronic devices, computer-readable storage media, computer program products, or chips provided in the embodiments of this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0391] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A screen projection method, characterized in that, Applied to a screen mirroring sending end, wherein the screen mirroring sending end has a screen mirroring application installed, the method includes: When the screen casting sending end plays audio data through its built-in speaker, upon receiving a screen casting connection request, it obtains the speaker's identifier as the target identifier of the audio device currently used by the screen casting sending end and records it in the audio device table. In response to the screen casting connection request, the screen casting sending end establishes a screen casting connection with the screen casting receiving end, takes the current audio state of the screen casting sending end as the target audio state, and records the target identifier and the target audio state in the audio state table. After the screen mirroring sending end and the screen mirroring receiving end establish a screen mirroring connection, when the user selects the screen mirroring receiving end as the audio output device, the audio data is obtained from the channel corresponding to the speaker and sent to the screen mirroring receiving end, which then plays the audio data. The screen mirroring application automatically sets the speaker to a mute state so that the screen mirroring sending end does not make any sound. The target audio state of the speaker recorded in the audio state table remains unchanged, and the icon style corresponding to the audio state switching control of the speaker remains unchanged. However, when the user clicks the audio state switching control, the icon style corresponding to the audio state switching control changes, the target audio state of the speaker changes, and the target audio state of the speaker in the audio state table changes. After the headphones are connected to the screen projection sending end, the identification number of the headphones is obtained as the target identification number of the audio device currently used by the screen projection sending end, and recorded in the audio device table. The current audio state of the screen projection sending end is taken as the target audio state. The target identification number and the target audio state are recorded in the audio state table. The audio data is obtained from the corresponding channel of the headphones and sent to the screen projection receiving end. The screen projection receiving end plays the audio data. The screen projection application automatically sets the headphones to a mute state so that the screen projection sending end does not make any sound. The target audio state of the headphones recorded in the audio state table remains unchanged, and the icon style corresponding to the audio state switching control of the headphones remains unchanged. However, when the user clicks the audio state switching control, the icon style corresponding to the audio state switching control changes, the target audio state of the headphones changes, and the target audio state of the headphones in the audio state table changes. When the user selects the audio output device to switch to the screen projection sending end, or when the screen projection sending end disconnects from the screen projection receiving end, the target audio state of the headphones is obtained, the audio state of the headphones is set according to the target audio state, and the icon style corresponding to the audio state switching control is set.

2. The method according to claim 1, characterized in that, The display interface of the screen projection terminal shows a first audio state switching control and a first volume bar control corresponding to the speaker; The method further includes: When the built-in speaker of the screen projection terminal plays the audio data at the volume value corresponding to the first volume bar control, upon receiving a click operation on the first audio state switching control, the screen projection terminal responds to the click operation by controlling the speaker to stop playing sound and changing the icon corresponding to the first audio state switching control from a non-mute icon to a mute icon.

3. The method according to claim 2, characterized in that, After acquiring the audio data from the channel corresponding to the speaker and sending the audio data to the screen projection receiver for playback, the method further includes: Upon receiving a click operation to switch the audio output device from the screen projection receiver to the screen projection sender, the screen projection sender responds to the click operation by stopping the acquisition of audio data from the channel corresponding to the speaker and keeping the speaker silent, while the icon corresponding to the first audio state switching control remains the mute icon.

4. The method according to claim 3, characterized in that, After ceasing to acquire the audio data from the channel corresponding to the speaker and keeping the speaker silent, the method further includes: Upon receiving a click operation on the first audio state switching control, the screen projection sending terminal responds to the click operation by controlling the speaker to play the audio data at the volume value corresponding to the first volume bar control, and changing the icon corresponding to the first audio state switching control from the mute icon to the non-mute icon.

5. The method according to claim 2, characterized in that, After acquiring the audio data from the channel corresponding to the speaker and sending the audio data to the screen projection receiver for playback, the method further includes: Upon receiving a notification that the screen mirroring connection between the screen mirroring sender and the screen mirroring receiver has been disconnected, the screen mirroring sender responds to the click operation by stopping the acquisition of audio data from the channel corresponding to the speaker and keeping the speaker silent. The icon corresponding to the first audio state switching control remains the mute icon.

6. The method according to claim 2, characterized in that, The display interface of the screen projection sending terminal shows a second audio state switching control and a second volume bar control corresponding to the headphones; After acquiring the audio data from the corresponding channel of the headphones and sending the audio data to the screen projection receiver for playback by the screen projection receiver, the method further includes: Upon receiving a click operation on the second audio state switching control, the screen projection sending terminal responds to the click operation by controlling the headphones to stop playing sound and changing the icon corresponding to the second audio state switching control from a non-mute icon to a mute icon.

7. The method according to claim 6, characterized in that, After the screen projection sending end responds to the click operation by controlling the headphones to stop playing sound and changing the icon corresponding to the second audio state switching control from a non-mute icon to a mute icon, the method further includes: Upon receiving a click operation to switch the audio output device from the screen projection receiver to the screen projection sender, the screen projection sender responds to the click operation by stopping the acquisition of audio data from the corresponding channel of the headphones and keeping the headphones silent, while the icon corresponding to the second audio state switching control remains the mute icon.

8. The method according to claim 7, characterized in that, After ceasing to acquire the audio data from the corresponding channel of the headphones and keeping the headphones silent, the method further includes: Upon receiving a click operation on the second audio state switching control, the screen projection sending terminal responds to the click operation by controlling the headphones to play the audio data at the volume value corresponding to the second volume bar control, and changing the icon corresponding to the second audio state switching control from the mute icon to the non-mute icon.

9. The method according to claim 6, characterized in that, After the screen projection sending end responds to the click operation by controlling the headphones to stop playing sound and changing the icon corresponding to the second audio state switching control from a non-mute icon to a mute icon, the method further includes: Upon receiving a notification that the screen mirroring connection between the screen mirroring sender and the screen mirroring receiver has been disconnected, the screen mirroring sender responds to the click operation by stopping the acquisition of audio data from the corresponding channel of the headphones and keeping the headphones silent. The icon corresponding to the second audio state switching control remains the mute icon.

10. The method according to claim 1, characterized in that, The display interface of the screen projection sending end also displays an audio device selection list, which includes a first audio device option and a second audio device option. The first audio device option corresponds to the speaker, and the second audio device option corresponds to the headphones. After headphones are connected to the screen projection sending end, the audio data is obtained from the corresponding channel of the headphones and sent to the screen projection receiving end. After the screen projection receiving end plays the audio data, the method further includes: Upon receiving a click operation on the first audio device option, the screen projection sending end responds to the click operation by acquiring the audio data from the channel corresponding to the speaker and sending the audio data to the screen projection receiving end, which then plays the audio data.

11. An electronic device, characterized in that, The electronic device is a screen projection sending end, comprising: a memory and a processor, wherein the memory and the processor are coupled; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device performs the screen projection method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, The device includes a computer program, characterized in that, when the computer program is run on an electronic device, it causes the electronic device to perform the screen projection method as described in any one of claims 1 to 10.

Images