Screen projection method, device, wireless terminal and storage medium in wireless terminal

Abstract

The embodiments of the present application disclose a screen projection method, device, wireless terminal and storage medium in a wireless terminal, which belongs to the field of screen projection technology. The method includes: in response to a screen projection instruction, establishing a data communication link with a screen projection device; in the case where a Bluetooth link is established between the audio output terminal and the wireless terminal, transmitting the audio data in the screen projection data to the audio output terminal via the Bluetooth link, so that the audio output terminal outputs sound based on the audio data; transmitting the image data in the screen projection data to the screen projection device via the data communication link, so that the screen projection device outputs a picture based on the image data. By adopting the solution provided by the embodiments of the present application, the audio output terminal other than the screen projection device can continue to be used for audio playback during the screen projection process. For example, during the screen projection process, the Bluetooth headset connected to the terminal before the screen projection can continue to be used for audio playback, which improves the freedom of audio playback during the screen projection process and avoids the influence caused by the switching of the audio playback source during the screen projection process.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of screen projection technology, and in particular, to a screen projection method, device, wireless terminal and storage medium in a wireless terminal. Background Art

[0002] Screen projection is a technology that transmits audio and video data on the terminal side to a projection device, thereby displaying the terminal screen and playing the terminal sound through the projection device.

[0003] For example, when a mobile phone and a TV are connected to the same wireless network, the user can turn on the screen projection function of the mobile phone, and the mobile phone will transmit the audio and video data to the TV through the wireless network, so that the mobile phone's picture can be displayed on the TV and the sound of the mobile phone can be played. Summary of the invention

[0004] The embodiments of the present application provide a screen projection method, device, wireless terminal and storage medium in a wireless terminal. The technical solution is as follows:

[0005] On the one hand, an embodiment of the present application provides a screen projection method in a wireless terminal, the method comprising:

[0006] In response to the screen projection instruction, establish a data communication link with the screen projection device;

[0007] In the case where a Bluetooth link is established between the audio output terminal and the wireless terminal, the audio data in the screen projection data is transmitted to the audio output terminal through the Bluetooth link, so that the audio output terminal outputs sound based on the audio data;

[0008] The image data in the screen projection data is transmitted to the screen projection device through the data communication link, so that the screen projection device outputs the picture based on the image data.

[0009] On the other hand, an embodiment of the present application provides a screen projection device, the device comprising:

[0010] A data link establishment module, used to establish a data communication link with the screen projection device in response to a screen projection instruction;

[0011] A first transmission module is used to transmit the audio data in the screen projection data to the audio output terminal through the Bluetooth link when a Bluetooth link is established between the audio output terminal and the wireless terminal, so that the audio output terminal outputs sound based on the audio data;

[0012] The second transmission module is used to transmit the image data in the projection data to the projection device through the data communication link, so that the projection device outputs the picture based on the image data.

[0013] On the other hand, an embodiment of the present application provides a wireless terminal, which includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the screen projection method as described in the above aspects.

[0014] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one program code is stored, and the program code is loaded and executed by a processor to implement the screen projection method as described in the above aspects.

[0015] On the other hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The Bluetooth chip of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the screen projection method of the wireless terminal provided in various optional implementations of the above aspects.

[0016] The technical solution provided in the embodiments of the present application can bring the following beneficial effects:

[0017] In an embodiment of the present application, during the screen projection process of the wireless terminal to the screen projection device, if a Bluetooth link is established between the wireless terminal and the audio output terminal, the wireless terminal separates the image data and the audio data in the projection data, and transmits the audio data to the audio output terminal through the Bluetooth link, and transmits the image data through the data communication link between the wireless terminal and the screen projection device, thereby realizing the separation of devices for picture display and sound playback during the screen projection process; using the solution provided in the embodiment of the present application, audio output terminals other than the screen projection device can continue to be used for audio playback during the screen projection process. For example, audio playback can continue to be performed through the Bluetooth headset connected to the wireless terminal before screen projection during the screen projection process, thereby improving the freedom of audio playback during the screen projection process and avoiding the impact of audio playback source switching during the screen projection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic diagram showing an implementation environment provided by an exemplary embodiment of the present application is shown;

[0020] Figure 2 A flowchart of a screen projection method provided by an exemplary embodiment of the present application is shown;

[0021] Figure 3 is a schematic diagram of an implementation of a screen projection process shown in an exemplary embodiment of the present application;

[0022] Figure 4 is a flowchart of an audio data transmission process shown in an exemplary embodiment of the present application;

[0023] Figure 5 is a schematic diagram of an implementation of a screen projection process shown in another exemplary embodiment of the present application;

[0024] Figure 6 is a schematic diagram of an implementation of a switching process shown in an exemplary embodiment of the present application;

[0025] Figure 7 is a flowchart of a synchronization process of image data and audio data transmission shown in an exemplary embodiment of the present application;

[0026] Figure 8 An interactive flow chart of a screen projection process provided by an embodiment of the present application is shown;

[0027] Fig. 9 A structural block diagram of a screen projection device provided by an embodiment of the present application is shown;

[0028] Fig.10 A structural block diagram of a wireless terminal provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0030] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application, wherein the implementation environment includes a wireless terminal 110, an audio output terminal 120, and a screen projection device 130.

[0031] The wireless terminal 110 is an electronic device with Bluetooth and screen projection functions, and the electronic device can be a smart phone, a tablet computer, a personal computer, etc. Figure 1 The wireless terminal 110 is taken as a smart phone as an example for description, but this is not a limitation.

[0032] The audio output terminal 120 is an electronic device with Bluetooth and audio output functions, and the electronic device can be a Bluetooth headset, a Bluetooth speaker, etc. Figure 1 The audio output terminal 120 is taken as a Bluetooth headset as an example for description, but this is not a limitation.

[0033] After the wireless terminal 110 establishes a Bluetooth link with the audio output terminal 120, the wireless terminal 110 transmits the audio data to the audio output terminal 120 via the first Bluetooth link, and the audio output terminal 120 outputs sound based on the audio data.

[0034] The screen projection device 130 is an electronic device with a display function, which may be a smart TV, a display, a personal computer, a projector, etc. Figure 1 The screen projection device 130 is taken as a smart TV as an example for explanation.

[0035] In one possible implementation, the wireless terminal 110 and the screen projection device 130 are located in the same wireless network. When a screen projection instruction is received, the wireless terminal 110 and the screen projection device 130 establish a data communication link through the wireless network. During the screen projection process, the wireless terminal 110 transmits the projection data (image data and audio data) to the screen projection device 130 through the data communication link, and the screen projection device 130 outputs the image and outputs the audio through its own speaker.

[0036] Of course, in other possible implementations, the wireless terminal 110 and the screen projection device 130 can also establish a data communication link based on a wired connection. For example, the wireless terminal 110 and the screen projection device 130 are connected via a data line. When a screen projection instruction is received, the wireless terminal 110 and the screen projection device 130 establish a data communication link via the data line. This embodiment is not limited to this.

[0037] In the embodiment of the present application, during the screen projection process, if a Bluetooth link is established between the wireless terminal 110 and the audio output terminal 120, the wireless terminal 110 separates the audio and video data of the received screen projection data. On the one hand, the image data is transmitted to the screen projection device 130 through the data communication link for the screen projection device 130 to output the picture. On the other hand, the audio data is transmitted to the audio output terminal 120 through the Bluetooth link for the audio output terminal 120 to output the sound. That is, during the screen projection process, the audio and video of the wireless terminal can be displayed and played through different devices.

[0038] Optionally, during the screen projection process, the wireless terminal 110 also provides an audio path switching function, through which the user can switch the audio path from the audio output terminal 120 to the screen projection device 130 as needed, thereby outputting sound through the screen projection device 130.

[0039] In the related technology, during the screen projection process, the wireless terminal sends the projection data (including image data and audio data) to the screen projection device, which displays the picture and plays the sound. Even if there is a Bluetooth connection between the terminal and the audio output terminal at this time, the terminal sound will not be played through the audio output terminal.

[0040] In the solution provided in the embodiment of the present application, if screen projection is performed when there is a Bluetooth connection between the wireless terminal and the audio output terminal, the terminal separates the audio and video data of the projection data, and transmits the image data and audio data to the projection device and the audio output terminal through the data communication link and the Bluetooth link respectively, so that sound can be played through the audio output terminal during the screen projection process, avoiding the impact caused by the audio playback source switching from the audio output terminal to the projection device.

[0041] In one possible application scenario, a smartphone establishes a Bluetooth connection with a Bluetooth headset. When a user uses the smartphone to watch a video, the smartphone transmits audio data to the Bluetooth headset, which plays the audio to avoid disturbing others. When the user wants to continue watching the video on a smart TV, the screen projection function of the smartphone can be triggered to establish a data communication link between the smartphone and the smart TV.

[0042] Using the solution provided by relevant technologies, during the screen projection process, the sound of the smartphone will be played through the smart TV, and the external sound of the smart TV will cause interference to others.

[0043] By adopting the solution provided in the embodiment of the present application, after establishing a data communication link, the smart phone transmits image data to the smart TV and transmits audio data to the Bluetooth headset, so that the smart TV displays the mobile phone screen based on the received image data, and the Bluetooth headset plays the mobile phone sound based on the received audio data, thereby avoiding interference to others caused by the external sound of the smart TV when casting the screen.

[0044] Please refer to Figure 2 , which shows a flowchart of a screen projection method in a wireless terminal provided by an exemplary embodiment of the present application. In this embodiment, the method is used for Figure 1 Taking the wireless terminal 110 in the illustrated implementation environment as an example, the method includes:

[0045] Step 201, in response to the screen projection instruction, establish a data communication link with the screen projection device.

[0046] Optionally, in response to the screen projection instruction, the wireless terminal and the screen projection device establish a wireless data communication link through a wireless network, or the wireless terminal and the screen projection device establish a wired data communication link through a wired connection. For ease of description, the following embodiments are described using a wireless data communication link as an example.

[0047] In some embodiments, when there are multiple candidate screen projection devices in the wireless network, when a screen projection instruction is received, the wireless terminal displays the device identification of each candidate screen projection device, and when a device selection operation is received, establishes a data communication link with the selected screen projection device.

[0048] During the screen projection process, the wireless terminal uses the image data of its own display screen and the audio data of its own sound as the projection data, and the data communication link is used to transmit the projection data (all or part of the projection data).

[0049] Step 202: When a Bluetooth link is established between the audio output terminal and the wireless terminal, the audio data in the screen projection data is transmitted to the audio output terminal via the Bluetooth link so that the audio output terminal outputs sound based on the audio data.

[0050] In one possible implementation, when a data communication link is established between the wireless terminal and the screen projection device, it is detected whether a Bluetooth link is established with the audio output terminal. If no Bluetooth link is established, the wireless terminal adopts a conventional screen projection process to transmit the projection data containing audio and video data to the screen projection device; if a Bluetooth link is established, the wireless terminal separates the audio and video data from the projection data.

[0051] Among them, the Bluetooth link between the wireless terminal and the audio output terminal can be established before screen projection or during the screen projection process (if the Bluetooth link is not established during the screen projection process, the screen projection device displays the wireless terminal screen and plays the terminal sound).

[0052] Furthermore, for the separated audio data, the wireless terminal transmits the audio data to the audio output terminal via a Bluetooth link, so that the terminal sound can continue to be played through the audio output terminal during the screen projection process.

[0053] Indicatively, Figure 3 As shown, before screen projection, the mobile phone 31 transmits audio data to the Bluetooth headset 32 ​​through the Bluetooth link between the mobile phone 31 and the Bluetooth headset 32; after screen projection, the mobile phone 31 separates the audio and image of the projection data, and continues to transmit audio data to the Bluetooth headset 32 ​​through the Bluetooth link between the mobile phone 31 and the Bluetooth headset 32.

[0054] Step 203, transmit the image data in the projection data to the projection device through the data communication link, so that the projection device outputs the picture based on the image data.

[0055] While the wireless terminal transmits audio data to the audio output terminal, it transmits image data to the screen projection device through the established data communication link. The screen projection device parses the image data and displays the terminal screen.

[0056] It should be noted that the above steps 202 and 203 are executed synchronously to ensure that the sound output by the audio output terminal is synchronized with the picture output by the projection device.

[0057] Indicatively, Figure 3 As shown, during the screen projection process, the mobile phone 31 transmits the image data to the smart TV 33, and the smart TV 33 and the mobile phone 31 synchronously display the mobile phone screen.

[0058] It can be seen that by adopting the solution provided in the embodiment of the present application, the sound of the wireless terminal before and after screen projection is played through the audio output terminal, so as to avoid interference to others during the screen projection process due to the terminal sound being directly played through the screen projection device.

[0059] To summarize, in the embodiments of the present application, during the screen projection process of the wireless terminal to the screen projection device, if a Bluetooth link is established between the wireless terminal and the audio output terminal, the wireless terminal separates the image data and the audio data in the projection data, and transmits the audio data to the audio output terminal through the Bluetooth link, and transmits the image data through the data communication link between the screen projection device, thereby realizing the separation of devices for picture display and sound playback during the screen projection process; by adopting the scheme provided in the embodiments of the present application, audio output terminals other than the screen projection device can continue to be used for audio playback during the screen projection process. For example, audio playback can continue to be performed through the Bluetooth headset connected to the wireless terminal before the screen projection during the screen projection process, thereby improving the freedom of audio playback during the screen projection process and avoiding the influence of audio playback source switching during the screen projection process.

[0060] In one possible application scenario, before screen projection, the wireless terminal has established a Bluetooth link with the audio output terminal (that is, the Bluetooth link is established before receiving the screen projection instruction), and transmits audio data to the audio output terminal through the Bluetooth link. When the user triggers the wireless terminal to project the screen in this case, in response to the screen projection instruction, the wireless terminal separates the audio and video data of the projection data and transmits them, so that the audio output terminal can continue to play the terminal sound during the screen projection process.

[0061] In this application scenario, although the wireless terminal establishes a Bluetooth connection with the audio output terminal, the user may not be using the audio output terminal. In order to avoid continuing to play the terminal sound through the audio output terminal in this case, in a possible implementation, when a Bluetooth link is established between the wireless terminal and the audio output terminal, the terminal obtains the device status of the audio output terminal, and transmits audio data to the audio output terminal through the Bluetooth link when the audio output terminal is in use. If the audio output terminal is not in use, the terminal transmits the projection data containing audio and video data to the projection device through the data communication link.

[0062] Optionally, the device status is detected by the audio output terminal through its own sensor and reported to the wireless terminal through the Bluetooth link. For example, when the audio output terminal is a Bluetooth headset, the Bluetooth headset determines the headset posture through a built-in sensor and determines that it is in use when the headset posture is in the ear state.

[0063] In this embodiment, the wireless terminal determines whether to transmit audio data and image data separately based on the device status of the audio output terminal, thereby avoiding the problem of playing terminal sound through the audio output terminal when the audio output terminal is not in use, resulting in the need to manually switch the audio playback source again (manually switch to the projection device).

[0064] In another possible application scenario, before screen projection, the wireless terminal does not establish a Bluetooth link with the audio output terminal. If screen projection is performed at this time, the image data and audio data are transmitted to the projection device. Correspondingly, the projection device displays the terminal image and plays the terminal sound. If the wireless terminal establishes a Bluetooth connection with the audio output terminal during the screen projection process, the terminal can determine whether to play the terminal sound through the audio output terminal according to user needs. Figure 4 As shown, the above step 202 may include the following steps.

[0065] Step 202A: Establish a Bluetooth link with the audio output terminal.

[0066] Optionally, during the screen projection process, when a Bluetooth connection instruction is received, the wireless terminal establishes a Bluetooth link with the audio output terminal, wherein the audio output terminal may be a Bluetooth device that is paired with the wireless terminal for the first time, or a Bluetooth device that has been paired with the wireless terminal in the past.

[0067] Step 202B: display a first switching prompt, where the first switching prompt is used to confirm whether to output sound through the audio output terminal.

[0068] When it is detected that a Bluetooth connection is established with the audio output terminal during the screen projection process, in order to determine whether it is necessary to switch the audio from the screen projection device to the audio output terminal, the wireless terminal displays a first switching prompt to prompt the user to confirm whether to play the terminal sound through the audio output terminal.

[0069] If a confirmation instruction is received, the wireless terminal determines that the device for playing the terminal sound needs to be switched and executes step 202C; if a cancellation instruction is received, the wireless terminal determines to continue playing the terminal sound through the screen projection device.

[0070] Indicatively, Figure 5 As shown, during the screen projection process, the mobile phone 31 transmits the projection data to the TV 33, and the TV 33 displays the mobile phone screen according to the image data therein and plays the mobile phone sound according to the audio data therein. When the mobile phone 31 establishes a Bluetooth connection with the Bluetooth headset 32, the mobile phone 31 displays a first switching prompt 311, prompting whether it is necessary to switch the Bluetooth headset 32 ​​to play the mobile phone sound.

[0071] Step 202C: in response to the confirmation instruction, transmit the audio data to the audio output terminal via the Bluetooth link.

[0072] When receiving the confirmation command, the terminal separates the audio and video data from the projection data, and transmits the audio data to the audio output terminal via the Bluetooth link for playback by the audio output terminal. The confirmation command is triggered by a confirmation operation, which is received through the wireless terminal (such as clicking the confirmation option) or received through the audio output terminal (such as tapping the Bluetooth headset).

[0073] Indicatively, Figure 5 As shown, after the user clicks the confirmation option, the mobile phone 31 sends the audio data to the Bluetooth headset 32 ​​through the Bluetooth link, so that the mobile phone sound is played through the Bluetooth headset 32.

[0074] Optionally, in response to the cancellation instruction, the wireless terminal continues to transmit image data and audio data to the screen projection device through the data communication link.

[0075] In order to further improve the freedom of audio playback during screen projection, users can switch the audio playback source of the mobile phone sound as needed during screen projection. In one possible implementation, in response to the switching instruction, the wireless terminal transmits image data and audio data to the screen projection device through a data communication link, so that the screen projection device plays the terminal sound synchronously while displaying the terminal screen.

[0076] In one possible implementation, during the screen projection process, the terminal displays an audio path switching interface. When an audio path switching operation is received within the audio path switching interface, a switching instruction is triggered, and image data and audio data are transmitted to the screen projection device through a data communication link.

[0077] Indicatively, Figure 6 As shown, during the screen projection process, the TV 33 first displays the mobile phone screen and plays the mobile phone sound through the Bluetooth headset 32; when a switching instruction is received, the mobile phone 31 stops transmitting audio data to the Bluetooth headset 32, and transmits the screen projection data containing image data and audio data to the TV 33, thereby playing the mobile phone sound through the built-in speaker of the TV 33.

[0078] Optionally, when receiving a re-triggered switching instruction, the terminal transmits the audio data to the audio output terminal again through the Bluetooth link.

[0079] During screen projection, if the quality of the Bluetooth link between the wireless terminal and the audio output terminal deteriorates (for example, the terminal is too far away from the audio output terminal, or the Bluetooth link is interfered), the transmission of audio data will be affected, which will in turn affect the playback of the sound on the audio output terminal side. In this case, in order to improve the synchronization of audio and video during screen projection, the wireless terminal can prompt the user to switch the audio to the screen projection device.

[0080] In a possible implementation, during the screen projection process, the wireless terminal determines the link quality of the Bluetooth link. The link quality can be determined by measuring the received signal strength indication (RSSI), transmission rate, transmission delay or packet loss rate of the Bluetooth signal, etc. This embodiment does not limit the measurement standard of the link quality.

[0081] Furthermore, the wireless terminal detects whether the link quality of the Bluetooth link meets the transmission requirements. If so, the audio data continues to be transmitted through the Bluetooth link. If not, a second switching prompt is displayed, which is used to confirm whether to play the terminal sound through the screen projection device. In response to the confirmation operation of the second switching prompt, the wireless terminal triggers the switching instruction and transmits the image data and audio data to the screen projection device through the data communication link.

[0082] Optionally, the transmission requirement may include at least one of the following: RSSI requirement, transmission rate requirement, transmission delay requirement, and packet loss rate requirement. The embodiment of the present application does not limit the specific content of the transmission requirement.

[0083] In an illustrative example, when the wireless terminal detects that the packet loss rate of the Bluetooth link between the wireless terminal and the audio output terminal is greater than the packet loss rate threshold (for example, 5%), it determines that the link quality of the Bluetooth link does not meet the transmission requirements, and thus displays a second switching prompt to prompt the user to switch the terminal audio to the screen projection device.

[0084] In some embodiments, after the transmission path of the audio data is switched from the Bluetooth link to the data communication link, the wireless terminal continues to monitor the link quality of the Bluetooth link. When the link quality meets the transmission requirements, the wireless terminal displays a third switching prompt for the user to confirm whether to switch the transmission path of the audio data to the Bluetooth link, so as to play the terminal sound again through the audio output terminal.

[0085] In an illustrative example, the wireless terminal switches the audio data for transmission to the screen projection device because the RSSI of the Bluetooth link is lower than the RSSI threshold. When the RSSI of the subsequent Bluetooth link is greater than the RSSI threshold, the wireless terminal displays a third switching prompt, prompting the user to play the terminal sound again through the audio output terminal.

[0086] In response to the confirmation operation on the third switching prompt, the wireless terminal transmits the audio data to the audio output terminal through the Bluetooth link, thereby resuming the playing of the terminal sound at the audio output terminal.

[0087] In this embodiment, during the screen projection process, when a switching instruction is received, the wireless terminal switches the transmission link of the audio data, thereby realizing the sound playback switching between the screen projection device and the audio output terminal, further improving the freedom of sound playback during the screen projection process.

[0088] In addition, in this embodiment, the wireless terminal monitors the link quality of the Bluetooth link and provides switching prompts based on the link quality. When the link quality is poor, it prompts to switch to using the screen projection device to play the terminal sound. When the link quality is restored, it provides a switch back to the audio output terminal to play the terminal sound, thereby improving the audio playback quality during the screen projection process.

[0089] When the terminal image and terminal sound are both displayed and played through the projection device, since the image data and audio data are transmitted to the projection device through the same link, the terminal image at the projection device remains synchronized with the terminal sound; and when the terminal image is displayed by the projection device and the terminal sound is played by the audio output terminal, since the terminal image data and audio data are transmitted through different links, and the transmission delay between different links may be different, the problem of audio and video being out of sync may occur.

[0090] In order to improve the synchronization of audio and video during the screen projection process, in a possible implementation, during the screen projection process, when the terminal sound is played through the audio output terminal, the wireless terminal synchronizes the transmission of image data and audio data based on the link status of the data communication link and the Bluetooth link, thereby ensuring that in the projection data at the same timestamp, the display time of the image data is synchronized with the playback time of the audio data.

[0091] like Figure 7 As shown, the wireless terminal may include the following steps to synchronize the transmission of image data and audio data:

[0092] Step 701: determine a first transmission delay of a data communication link and a second transmission delay of a Bluetooth link.

[0093] Regarding the specific method of determining the transmission delay corresponding to the link, in one possible implementation, the wireless terminal sends a first delay measurement instruction to the screen projection device through the data communication link, and after receiving the first delay measurement instruction, the screen projection device sends a first response instruction through the data communication link. The wireless terminal determines the first transmission delay of the data communication link based on the time interval between the first delay measurement instruction and the first response instruction.

[0094] Similarly, the wireless terminal sends a second delay measurement instruction to the audio output terminal via the Bluetooth link, and after receiving the second delay measurement instruction, the audio output terminal sends a second response instruction via the Bluetooth link. The wireless terminal determines the second transmission delay of the Bluetooth link based on the time interval between the second delay measurement instruction and the second response instruction.

[0095] Schematically, when the sending time of the first delay measurement instruction is t1 and the receiving time of the first response instruction is t2, the first transmission delay of the data communication link is (t2-t1) / 2; when the sending time of the second delay measurement instruction is t1' and the receiving time of the second response instruction is t2', the second transmission delay of the Bluetooth link is (t2'-t1') / 2.

[0096] Step 702: determine a transmission time difference based on the first transmission delay and the second transmission delay.

[0097] Further, the wireless terminal determines the difference between the first transmission delay and the second transmission delay as the transmission time difference.

[0098] In combination with the example in the above steps, the transmission time difference is (t2-t1) / 2-(t2'-t1') / 2.

[0099] Step 703: synchronize the transmission of the image data and the audio data based on the transmission time difference.

[0100] In one possible implementation, the wireless terminal can delay the transmission of image data or audio based on the transmission time difference, thereby using the delay to offset the timing difference caused by the transmission of audio and video data in different links, and ensure the synchronization of image decoding in the projection device and audio decoding in the audio output terminal. For example, when the transmission time difference is 20ms (the transmission delay of the data communication link is higher than the transmission delay of the Bluetooth link), the wireless terminal will delay sending audio data to the audio output terminal by 20ms.

[0101] In this embodiment, when the terminal screen is displayed through the screen projection device and the terminal sound is played through the audio output terminal, the wireless terminal synchronizes the transmission of image data and audio data based on the transmission time difference between the data communication link and the Bluetooth link, thereby improving the synchronization of audio and video during the screen projection process.

[0102] In an exemplary embodiment, Figure 8 As shown, the wireless terminal includes the following steps when implementing screen projection through a screen projection device and an audio output terminal.

[0103] Step 801: A wireless terminal establishes a Bluetooth link with an audio output terminal.

[0104] Step 802: The wireless terminal transmits audio data to the audio output terminal via a Bluetooth link.

[0105] Step 803: the audio output terminal outputs sound based on the audio data.

[0106] Step 804, in response to the screen projection instruction, the wireless terminal establishes a data communication link with the screen projection device.

[0107] Step 805: The wireless terminal synchronizes the transmission of the image data and audio data in the projection data based on the link status of the data communication link and the Bluetooth link.

[0108] Step 806: The wireless terminal transmits the audio data in the screen projection data to the audio output terminal via the Bluetooth link.

[0109] Step 807: The audio output terminal outputs sound based on the audio data.

[0110] Step 808: The wireless terminal transmits the image data in the screen projection data to the screen projection device through the data communication link.

[0111] Step 809: the screen projection device outputs the picture based on the image data.

[0112] Step 810, in response to the switching instruction, the wireless terminal transmits image data and audio data to the screen projection device through the data communication link.

[0113] Step 811, the screen projection device outputs images based on the image data and outputs sounds based on the audio data.

[0114] In each of the above embodiments, the screen projection device and the audio output terminal are independent of each other, and there is no data communication. In one possible implementation, when the screen projection device does not have a Bluetooth function, the wireless terminal can use the solution provided in the above embodiment to achieve screen projection; when the screen projection device has a Bluetooth function, when a Bluetooth link is established between the wireless terminal and the audio output terminal, the wireless terminal sends a Bluetooth link establishment request to the screen projection device, so that the screen projection device establishes a Bluetooth link with the audio output terminal based on the Bluetooth link establishment request. During the screen projection process, the wireless terminal transmits the projection data to the screen projection device through the data communication link, and the screen projection device displays the terminal screen based on the image data, and transmits audio data to the audio output terminal through the Bluetooth link between the wireless terminal and the audio output terminal, and the audio output terminal plays the terminal sound based on the audio data.

[0115] Accordingly, when transmitting audio data to the audio output terminal through the screen projection device, the screen projection device needs to determine the transmission delay of the Bluetooth link between the screen projection device and the audio output terminal, so as to synchronize the transmission of the image data and the audio data based on the transmission delay. The process of determining the transmission delay and synchronizing the transmission based on the transmission delay can refer to the above steps 701 to 703, which will not be repeated in this embodiment.

[0116] Optionally, during the screen projection process, in response to a switching instruction, the wireless terminal sends a switching request to the screen projection device so that the screen projection device stops transmitting audio data to the audio output terminal through the second Bluetooth link and outputs sound based on the audio data.

[0117] Optionally, in response to the screen projection end instruction, the wireless terminal sends a screen projection end request to the screen projection device, so that the screen projection device disconnects the data communication link and the second Bluetooth link between the audio output terminal based on the screen projection end request.

[0118] At the same time, if the audio output terminal supports a single Bluetooth link, after receiving the screen projection end request, the screen projection device will also send the Bluetooth information of the audio output terminal to the wireless terminal, so that the wireless terminal can re-establish the Bluetooth link with the audio output terminal based on the Bluetooth information, thereby ensuring the continuity of audio playback at the audio output terminal before and after the screen projection ends.

[0119] It should be noted that in the above-mentioned embodiments, the steps with the wireless terminal as the execution subject can be independently implemented as a screen projection method on the wireless terminal side, and the steps with the screen projection device as the execution subject can be independently implemented as a screen projection method on the screen projection device side, which will not be elaborated in this embodiment.

[0120] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0121] Please refer to Fig. 9 , which shows a structural block diagram of a screen projection device provided by an embodiment of the present application. The device may include:

[0122] A data link establishing module 901 is used to establish a data communication link with the screen projection device in response to a screen projection instruction;

[0123] The first transmission module 902 is used to transmit the audio data in the screen projection data to the audio output terminal through the Bluetooth link when a Bluetooth link is established between the audio output terminal and the wireless terminal, so that the audio output terminal outputs sound based on the audio data;

[0124] The second transmission module 903 is used to transmit the image data in the projection data to the projection device through the data communication link, so that the projection device outputs the picture based on the image data.

[0125] Optionally, the device further comprises:

[0126] A synchronization module is used to synchronize the transmission of the image data and the audio data based on the link status of the data communication link and the Bluetooth link.

[0127] Optionally, the synchronization module is specifically used to:

[0128] Determining a first transmission delay of the data communication link and a second transmission delay of the Bluetooth link;

[0129] Determining a transmission time difference based on the first transmission delay and the second transmission delay;

[0130] The image data and the audio data are transmitted synchronously based on the transmission time difference.

[0131] Optionally, the first transmission module 902 includes:

[0132] A Bluetooth link establishing unit, configured to establish the Bluetooth link with the audio output terminal;

[0133] A first display unit, used for displaying a first switching prompt, wherein the first switching prompt is used to confirm whether to output sound through the audio output terminal;

[0134] A transmission unit is used to transmit the audio data to the audio output terminal through the Bluetooth link in response to a confirmation instruction.

[0135] Optionally, the device further comprises:

[0136] The third transmission module is used to transmit the image data and the audio data to the screen projection device through the data communication link in response to a cancel instruction.

[0137] Optionally, the Bluetooth link is established before receiving the screen projection instruction;

[0138] The first transmission module 902 includes:

[0139] a state acquisition unit, configured to acquire a device state of the audio output terminal when the Bluetooth link is established between the audio output terminal and the wireless terminal;

[0140] A transmission unit is used to transmit the audio data to the audio output terminal through the Bluetooth link in response to the audio output terminal being in use.

[0141] Optionally, the device comprises:

[0142] A fourth transmission module is used to transmit the image data and the audio data to the screen projection device through the data communication link in response to a switching instruction.

[0143] Optionally, the fourth transmission module includes:

[0144] A first display unit, used for displaying an audio path switching interface;

[0145] A switching transmission unit is used to trigger the switching instruction in response to the audio path switching operation in the audio path switching interface, and transmit the image data and the audio data to the projection device through the data communication link.

[0146] Optionally, the fourth transmission module includes:

[0147] A link quality determination unit, configured to determine the link quality of the Bluetooth link;

[0148] A second display unit, configured to display a second switching prompt in response to the link quality not meeting the transmission requirement, wherein the second switching prompt is used to confirm whether to output sound through the screen projection device;

[0149] A switching transmission unit is used to trigger the switching instruction in response to a confirmation operation on the second switching prompt, and transmit the image data and the audio data to the screen projection device through the data communication link.

[0150] Optionally, the device further comprises:

[0151] a prompt module, configured to display a third switching prompt in response to the link quality satisfying the transmission requirement, wherein the third switching prompt is used to confirm whether to output the sound through the audio output terminal;

[0152] A fifth transmission module is used to transmit the audio data to the audio output terminal through the Bluetooth link in response to a confirmation operation on the third switching prompt.

[0153] To summarize, in the embodiments of the present application, during the screen projection process of the wireless terminal to the screen projection device, if a Bluetooth link is established between the wireless terminal and the audio output terminal, the wireless terminal separates the image data and the audio data in the projection data, and transmits the audio data to the audio output terminal through the Bluetooth link, and transmits the image data through the data communication link between the screen projection device, thereby realizing the separation of devices for picture display and sound playback during the screen projection process; by adopting the scheme provided in the embodiments of the present application, audio output terminals other than the screen projection device can continue to be used for audio playback during the screen projection process. For example, audio playback can continue to be performed through the Bluetooth headset connected to the wireless terminal before the screen projection during the screen projection process, thereby improving the freedom of audio playback during the screen projection process and avoiding the influence of audio playback source switching during the screen projection process.

[0154] Please refer to Fig.10 , which shows a block diagram of a wireless terminal provided by an exemplary embodiment of the present application. The wireless terminal 1000 in the present application may include one or more of the following components: a processor 1010 , a memory 1020 , and a Bluetooth chip 1030 .

[0155] The processor 1010 may include one or more processing cores. The processor 1010 uses various interfaces and lines to connect the various parts of the entire wireless terminal 1000, and executes various functions and processes data of the wireless terminal 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1020, and calling data stored in the memory 1020. Optionally, the processor 1010 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1010 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1010, but may be implemented by a separate chip.

[0156] The memory 1020 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1020 includes a non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data (such as audio data, a phone book), etc., created according to the use of the wireless terminal 1000.

[0157] The Bluetooth chip 1030 is a component for implementing Bluetooth functions. The Bluetooth chip 1030 includes two parts: Host and Controller (corresponding to different Bluetooth protocol stacks). The Host and Controller can run on the same chip (single-chip architecture) or on different chips (dual-chip architecture). For example, the Host runs on the processor, and the Controller runs on the Bluetooth module; or, both the Host and the Controller run on the Bluetooth chip 1030.

[0158] In addition, those skilled in the art can understand that the structure of the wireless terminal 1000 shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the drawings, or combine certain components, or arrange the components differently. For example, the wireless terminal 1000 also includes components such as a display screen, a sensor, a speaker, a microphone, a power supply, and a WiFi component, which will not be described in detail here.

[0159] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one program code, and the program code is loaded and executed by a processor to implement the screen projection method in the wireless terminal as described in the above embodiments.

[0160] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The Bluetooth chip of the electronic device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the electronic device executes the screen projection method in the wireless terminal provided in various optional implementations of the above aspects.

[0161] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0162] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A screen projection method in a wireless terminal, characterized in that: The method comprises: In the case where a Bluetooth link is established between the audio output terminal and the wireless terminal, transmitting audio data to the audio output terminal via the Bluetooth link; In response to the screen projection instruction, establish a data communication link with the screen projection device, where the data communication link is established based on a wireless network or a wired connection; In response to the audio output terminal being in use, based on the link status of the data communication link and the Bluetooth link, the image data and the audio data in the projection data are transmitted synchronously; the audio data in the projection data is transmitted to the audio output terminal through the Bluetooth link, so that the audio output terminal outputs sound based on the audio data; and the image data in the projection data is transmitted to the projection device through the data communication link, so that the projection device outputs a picture based on the image data; In response to the audio output terminal being in an unused state, the image data and audio data in the screen projection data are transmitted to the screen projection device through the communication link.

2. The method according to claim 1, characterized in that The step of synchronizing the transmission of the image data and the audio data based on the link status of the data communication link and the Bluetooth link comprises: Determining a first transmission delay of the data communication link and a second transmission delay of the Bluetooth link; Determining a transmission time difference based on the first transmission delay and the second transmission delay; The image data and the audio data are transmitted synchronously based on the transmission time difference.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: In response to the switching instruction, the image data and the audio data are transmitted to the screen projection device through the data communication link.

4. The method according to claim 3, characterized in that The transmitting the image data and the audio data to the screen projection device through the data communication link in response to the switching instruction includes: Display the audio path switching interface; In response to the audio path switching operation in the audio path switching interface, the switching instruction is triggered, and the image data and the audio data are transmitted to the screen projection device through the data communication link.

5. The method according to claim 3, characterized in that: The transmitting the image data and the audio data to the screen projection device through the data communication link in response to the switching instruction includes: Determining a link quality of the Bluetooth link; In response to the link quality not meeting the transmission requirement, displaying a second switching prompt, where the second switching prompt is used to confirm whether to output sound through the screen projection device; In response to a confirmation operation on the second switching prompt, the switching instruction is triggered, and the image data and the audio data are transmitted to the screen projection device through the data communication link.

6. The method according to claim 5, characterized in that After the switching instruction is triggered in response to the confirmation operation on the second switching prompt, and the image data and the audio data are transmitted to the screen projection device through the data communication link, the method further includes: In response to the link quality meeting the transmission requirement, displaying a third switching prompt, wherein the third switching prompt is used to confirm whether to output sound through the audio output terminal; In response to a confirmation operation on the third switching prompt, the audio data is transmitted to the audio output terminal via the Bluetooth link.

7. A screen projection device, characterized in that: The device comprises: A first transmission module, configured to transmit audio data to the audio output terminal through a Bluetooth link when a Bluetooth link is established between the audio output terminal and the wireless terminal; A data link establishing module, used to establish a data communication link with the screen projection device in response to a screen projection instruction, wherein the data communication link is established based on a wireless network or a wired connection; The first transmission module is used to synchronize the transmission of image data and audio data in the projection data based on the link status of the data communication link and the Bluetooth link in response to the audio output terminal being in use; and transmit the audio data in the projection data to the audio output terminal through the Bluetooth link so that the audio output terminal outputs sound based on the audio data; A second transmission module, used for transmitting the image data in the screen projection data to the screen projection device through the data communication link, so that the screen projection device outputs a picture based on the image data; The second transmission module is used to transmit the image data and audio data in the screen projection data to the screen projection device through the communication link in response to the audio output terminal being in an unused state.

8. A wireless terminal, characterized in that: The wireless terminal includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the screen projection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: At least one program code is stored in the computer-readable storage medium, and the program code is loaded and executed by the processor to implement the screen projection method as described in any one of claims 1 to 6.

Images