Screen mirroring method, screen mirroring system, and electronic device

Abstract

The present application provides a screen mirroring method, a screen mirroring system, and an electronic device. The method comprises: a first electronic device may obtain a resource to be screen-mirrored and played, said resource including a local resource and an online resource; upon obtaining said resource, the first electronic device parses said resource to obtain an audio stream and a video stream of said resource; and then the first electronic device sends the video stream of said resource to a video playback device (a second electronic device) so that a video of said resource is played on the second electronic device, and sends the audio stream of said resource to an audio playback device (a third electronic device) so that an audio of said resource is played on the third electronic device. The method solves the problem in the related art that a video and an audio of a resource to be screen-mirrored and played can only be played on a second electronic device, thereby improving user experience.

Description

Screen projection method, screen projection system and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 29, 2023, with application number 2023118668859 and application name “A screen projection method, screen projection system and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of communication technology, and more specifically, to a screen projection method, a screen projection system, and an electronic device. Background Art

[0003] With the development of application display technology, more and more electronic devices support screen projection technology. Among them, screen projection technology is to project the application interface or media resources launched on an electronic device with a smaller display (such as a mobile phone) to another electronic device with a larger display (such as a large-screen device) to achieve a better viewing effect.

[0004] Currently, after a mobile phone and a large-screen device establish a screen projection connection, the mobile phone can project the resources to be projected onto the large-screen device, and the large-screen device will play the video and audio of the resources to be projected. However, when encountering a scenario where the audio does not want to be played externally during screen projection, and the large-screen device does not support headphones, while the user enjoys a better viewing effect, the audio playback will affect other users, resulting in a poor user experience.

[0005] Summary of the Invention

[0006] The present application provides a screen projection method, a screen projection system, and an electronic device. The method parses the resources to be broadcast to obtain a video stream and an audio stream, and then sends the video stream to a second electronic device and the audio stream to a third electronic device, thereby solving the problem in related technologies that video and audio can only be played on the same electronic device.

[0007] In the first aspect, the present application provides a screen projection method. The method provided in the first aspect can be executed by an electronic device, or by a module applied in the electronic device (such as a processor, chip, or chip system, etc.), or by a logical node, logical module or software that can realize all or part of the functions of the electronic device. The present application does not limit this.

[0008] Specifically, the method includes: a first electronic device obtains the resource to be broadcast, decomposes the resource to be broadcast into a video stream and an audio stream, sends the video stream to a second electronic device, the second electronic device is used to play the video corresponding to the resource to be broadcast, and sends the audio stream to a third electronic device, the third electronic device is used to play the audio corresponding to the resource to be broadcast.

[0009] The method provided in the first aspect is that a first electronic device can obtain the resource to be broadcast, and then parse the resource to be broadcast, thereby parsing out the video stream and audio stream of the resource to be broadcast, converting the complete resource to be broadcast into a resource stream, and then sending the video stream to a second electronic device, using the second electronic device to play the video of the resource to be broadcast, and sending the audio stream to a third electronic device, using the third electronic device to play the audio of the resource to be broadcast, thereby solving the problem that the complete resource can only be played on the same electronic device and improving the user experience.

[0010] In a possible implementation of the first aspect, the method also includes: the first electronic device receives a video stream with a timestamp sent by the second electronic device, and receives an audio stream with a timestamp sent by the third electronic device, and then the first electronic device synchronizes the audio corresponding to the resource to be broadcast and the video corresponding to the resource to be broadcast based on the video stream with a timestamp and the audio stream with a timestamp, so that the video corresponding to the resource to be broadcast played on the second electronic device and the audio of the resource to be broadcast played on the third electronic device are synchronized.

[0011] In this implementation, when the resource to be broadcast plays video on the second electronic device and audio on the third electronic device, the audio and video playback may be out of sync. Therefore, the second electronic device can mark the playback time of each frame of video in the video stream, and the third electronic device can mark the playback time of each frame of audio in the audio stream. The first electronic device synchronizes the audio and video based on the marked timestamps.

[0012] Optionally, the first electronic device synchronizes the audio and video based on the marked timestamp in the following two ways:

[0013] The first method: The first electronic device transmits the synchronized timestamp back to the second and third electronic devices, allowing the second electronic device to request video data at the corresponding time point based on the timestamp transmitted by the first electronic device, and the third electronic device to request audio data at the corresponding time point based on the timestamp transmitted by the first electronic device. This ensures that the video stream viewed by the user on the second electronic device and the audio stream heard on the third electronic device are played synchronously. This implementation method requires transmitting the synchronized timestamp back to the second and third electronic devices, using the second and third electronic devices to achieve audio and video synchronization.

[0014] Second, the first electronic device can use a seek command based on the synchronized timestamp to control the second electronic device to play the video stream at a specified time, and the third electronic device to play the audio stream at a specified time, thereby synchronizing the video stream viewed by the user on the second electronic device and the audio stream heard by the third electronic device. In this implementation, the first electronic device does not need to transmit the synchronized timestamp back to the second and third electronic devices. The first electronic device can directly control the audio and video synchronization of the second and third electronic devices, thereby improving the efficiency of audio and video synchronization.

[0015] In a possible implementation of the first aspect, the method further includes: the first electronic device receives control information sent by the second electronic device, and controls the playback of audio corresponding to the resource to be broadcast on the third electronic device based on the control information sent by the second electronic device.

[0016] In a possible implementation of the first aspect, the method further includes: the first electronic device receives control information sent by the third electronic device, and controls the playback of the video corresponding to the resource to be broadcast on the second electronic device based on the control information sent by the third electronic device.

[0017] In this implementation, the second electronic device can receive the user's operation on the third electronic device through the second electronic device, thereby converting the user's operation into a control instruction and sending it to the first electronic device. The first electronic device controls the third electronic device to play the resource to be cast based on the control instruction. Alternatively, the third electronic device can receive the user's operation on the second electronic device through the third electronic device, thereby converting the user's operation into a control instruction and sending it to the first electronic device. The first electronic device controls the second electronic device to play the resource to be cast based on the control instruction. This method can complete the control of the resource to be cast on either side, providing users with a more flexible and convenient experience.

[0018] Optionally, the control information includes at least one of the following: identification information of the second electronic device, identification information of the third electronic device, volume information, video playback speed, video track identification information, or audio track playback information.

[0019] It should be understood that the second electronic device (video playback device) may include multiple video track identification information, and the third electronic device (audio playback device) may include multiple audio track identification information.

[0020] For example, the display screen of the second electronic device may display the identification information of multiple electronic devices. The user can select the device to be controlled from the device identification information of multiple third electronic devices and trigger the transmission of control information to the third electronic device using the volume adjustment control of the second electronic device. In this implementation, the control information may include the identification information of the second electronic device, the identification information of the third electronic device, and the volume level.

[0021] For another example, the display screen of the second electronic device may also display multiple audio track identification information of the third electronic device. The user can select the audio track to be controlled from the multiple audio track identification information and trigger the transmission of control information for the audio track using the volume control of the second electronic device. In this implementation, the control information may include: identification information of the second electronic device, audio track identification information of the third electronic device, and audio volume.

[0022] For another example, when a third electronic device receives a user's operation to adjust the video playback speed of a second electronic device on the third electronic device, the third electronic device may send a control field to the first electronic device, and the first electronic device adjusts the video playback speed of the second electronic device based on the control field. In this implementation, the control information may include: identification information of the third electronic device, identification information of the second electronic device, and the video playback speed.

[0023] In a possible implementation of the first aspect, the audio stream includes multiple different sub-audio streams, and the third electronic device includes multiple electronic devices, which are configured to play the different sub-audio streams.

[0024] In this implementation, when the audio stream includes multiple sub-audio streams of different types, the first electronic device can broadcast the multiple sub-audio streams of different types to multiple third electronic devices respectively, providing the user with a distributed experience of audio stream separation.

[0025] Exemplarily, the multiple sub-audio streams of different types may include the treble of the audio stream, the mid-range of the audio stream, the bass of the audio stream, and may also include the left channel of the audio stream, the right channel of the audio stream, etc. The embodiment of the present application does not specifically limit the type of audio stream.

[0026] In a possible implementation of the first aspect, based on the control information sent by the second electronic device, controlling the playback of audio corresponding to the resource to be broadcast on the third electronic device includes: sending a control message carrying control information, where the control message is used to instruct the third electronic device to control the playback of audio corresponding to the resource to be broadcast on the third electronic device.

[0027] In this implementation, the first electronic device may send a control message to the third electronic device, so that the third electronic device may control the playback of the audio corresponding to the resource to be broadcast based on the user's operation on the second electronic device.

[0028] In a possible implementation of the first aspect, based on the control information sent by the third electronic device, the playback of the video corresponding to the resource to be broadcast on the second electronic device is controlled, including: sending a control message carrying control information, which is used to instruct the second electronic device to control the playback of the video corresponding to the resource to be broadcast on the second electronic device.

[0029] In this implementation, the first electronic device may send a control message to the second electronic device, so that the second electronic device may control the playback of the video corresponding to the resource to be broadcast based on the user's operation on the third electronic device.

[0030] On the second aspect, the present application also provides a screen projection system, which includes a first electronic device, a second electronic device and a third electronic device. The first electronic device is used to obtain the resources to be broadcast, the second electronic device is used to play the video stream corresponding to the resources to be broadcast, and the third electronic device is used to play the audio stream corresponding to the resources to be broadcast.

[0031] In the system provided by the second aspect, a second electronic device can play the video stream of the resource broadcast by the first electronic device, and a third electronic device can play the audio stream of the resource broadcast by the first electronic device. This solves the problem of only being able to play the entire resource on the same electronic device, and improves the user experience.

[0032] In a possible implementation of the second aspect, the first electronic device is further used to decompose the resources to be broadcast into a video stream and an audio stream; the first electronic device is further used to send the video stream to the second electronic device; the first electronic device is further used to send the audio stream to the third electronic device; the second electronic device is used to play the video corresponding to the resources to be broadcast; and the third electronic device is used to play the audio corresponding to the resources to be broadcast.

[0033] In this implementation method, the first electronic device can parse the resource to be broadcast, thereby parsing out the video stream and audio stream of the resource to be broadcast, converting the complete resource to be broadcast into a resource stream, and then sending the video stream to the second electronic device, using the second electronic device to play the video of the resource to be broadcast, and sending the audio stream to the third electronic device, using the third electronic device to play the audio of the resource to be broadcast, thereby solving the problem that the complete resource can only be played on the same electronic device and improving the user experience.

[0034] In a possible implementation of the second aspect, the second electronic device is further used to mark a timestamp on each frame of the video stream and send the video stream with the timestamp to the first electronic device; the third electronic device is further used to mark a timestamp on each frame of the audio stream and send the audio stream with the timestamp to the first electronic device; the first electronic device is further used to synchronize the audio corresponding to the resource to be broadcast and the video corresponding to the resource to be broadcast based on the video stream with the timestamp and the audio stream with the timestamp, so that the video corresponding to the resource to be broadcast played on the second electronic device and the audio corresponding to the resource to be broadcast played on the third electronic device are synchronized.

[0035] In this implementation, when the resource to be broadcast plays video on the second electronic device and audio on the third electronic device, the audio and video playback may be out of sync. Therefore, the second electronic device can mark the playback time of each frame of video in the video stream, and the third electronic device can mark the playback time of each frame of audio in the audio stream. The first electronic device synchronizes the audio and video based on the marked timestamps, thereby ensuring that the video and audio are in a synchronized playback state.

[0036] Optionally, the first electronic device synchronizes the audio and video based on the marked timestamp in the following two ways:

[0037] The first method: The first electronic device transmits the synchronized timestamp back to the second and third electronic devices, allowing the second electronic device to request video data at the corresponding time point based on the timestamp transmitted by the first electronic device, and the third electronic device to request audio data at the corresponding time point based on the timestamp transmitted by the first electronic device. This ensures that the video stream viewed by the user on the second electronic device and the audio stream heard on the third electronic device are played synchronously. This implementation method requires transmitting the synchronized timestamp back to the second and third electronic devices, using the second and third electronic devices to achieve audio and video synchronization.

[0038] Second, the first electronic device can use a seek command based on the synchronized timestamp to control the second electronic device to play the video stream at a specified time, and the third electronic device to play the audio stream at a specified time, thereby synchronizing the video stream viewed by the user on the second electronic device and the audio stream heard by the third electronic device. In this implementation, the first electronic device does not need to transmit the synchronized timestamp back to the second and third electronic devices. The first electronic device can directly control the audio and video synchronization of the second and third electronic devices, thereby improving the efficiency of audio and video synchronization.

[0039] In a possible implementation of the second aspect, the first electronic device is further used to receive control information sent by the second electronic device; the first electronic device is further used to control the playback of audio corresponding to the resource to be broadcast on the third electronic device based on the control information sent by the second electronic device.

[0040] In a possible implementation of the second aspect, the first electronic device is further used to receive control information sent by a third electronic device; the first electronic device is further used to control the playback of the video corresponding to the resource to be broadcast on the second electronic device based on the control information sent by the third electronic device.

[0041] In this implementation, the second electronic device can receive the user's operation on the third electronic device on the second electronic device, thereby converting the user's operation into a control instruction and sending it to the first electronic device. The first electronic device controls the third electronic device to play the resource to be broadcast based on the control instruction, or the third electronic device can receive the user's operation on the second electronic device on the third electronic device, thereby converting the user's operation into a control instruction and sending it to the first electronic device. The first electronic device controls the second electronic device to play the resource to be broadcast based on the control instruction.

[0042] Optionally, the control information includes at least one of the following: identification information of the second electronic device, identification information of the third electronic device, volume information, video playback speed, video track identification information, or audio track playback information.

[0043] It should be understood that the second electronic device (video playback device) may include multiple video track identification information, and the third electronic device (audio playback device) may include multiple audio track identification information.

[0044] For example, the display screen of the second electronic device may display the identification information of multiple electronic devices. The user can select the device to be controlled from the device identification information of multiple third electronic devices and trigger the transmission of control information to the third electronic device using the volume adjustment control of the second electronic device. In this implementation, the control information may include the identification information of the second electronic device, the identification information of the third electronic device, and the volume level.

[0045] For another example, the display screen of the third electronic device may display multiple audio track identification information of the third electronic device. The user can select the audio track to be controlled from the multiple audio track identification information and trigger the transmission of control information for the audio track using the volume control of the second electronic device. In this implementation, the control information may include: identification information of the second electronic device, audio track identification information of the third electronic device, and audio volume.

[0046] For another example, when a third electronic device receives a user's operation to adjust the video playback speed of a second electronic device on the third electronic device, the third electronic device may send a control field to the first electronic device, and the first electronic device adjusts the video playback speed of the second electronic device based on the control field. In this implementation, the control information may include: identification information of the third electronic device, identification information of the second electronic device, and the video playback speed.

[0047] In a possible implementation of the second aspect, the audio stream includes multiple different sub-audio streams, and the third electronic device includes multiple electronic devices, which are configured to play the different sub-audio streams.

[0048] In this implementation, when the audio stream includes multiple sub-audio streams of different types, the first electronic device can broadcast the multiple sub-audio streams of different types to multiple third electronic devices respectively, providing the user with a distributed experience of audio stream separation.

[0049] Exemplarily, the multiple sub-audio streams of different types may include the treble of the audio stream, the mid-range of the audio stream, the bass of the audio stream, and may also include the left channel of the audio stream, the right channel of the audio stream, etc. The embodiment of the present application does not specifically limit the type of audio stream.

[0050] In a possible implementation of the second aspect, the first electronic device is further configured to send a control message carrying control information, and the third electronic device is further configured to control the playback of audio corresponding to the resource to be broadcast on the third electronic device according to the control message.

[0051] In this implementation, the first electronic device may send a control message to the third electronic device, so that the third electronic device may control the playback of the audio corresponding to the resource to be broadcast based on the user's operation on the second electronic device.

[0052] In a possible implementation of the first aspect, the first electronic device is further configured to send a control message carrying control information, and the second electronic device is further configured to control playback of a video corresponding to the resource to be broadcast on the second electronic device according to the control message.

[0053] In this implementation, the first electronic device may send a control message to the second electronic device, so that the second electronic device may control the playback of the video corresponding to the resource to be broadcast based on the user's operation on the third electronic device.

[0054] In a third aspect, a communication device is provided, which includes a unit for executing each step in the above first aspect or any possible implementation of the first aspect.

[0055] In a fourth aspect, a communication device is provided, which includes at least one processor and a memory, the processor and the memory are coupled, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method in the above first aspect or any possible implementation of the first aspect is executed.

[0056] In a fifth aspect, a communication device is provided, which includes at least one processor and an interface circuit, and the at least one processor is used to execute the method in the above first aspect or any possible implementation of the first aspect.

[0057] In the sixth aspect, an electronic device is provided, which includes a processor and a memory, the memory being used to store computer programs, and the processor being used to call and run the computer programs from the memory, so that the electronic device executes the method in the above first aspect or any possible implementation of the first aspect.

[0058] In a seventh aspect, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, it is used to execute the method in the above first aspect or any possible implementation of the first aspect.

[0059] In an eighth aspect, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed, it is used to execute the method in the above first aspect or any possible implementation of the first aspect.

[0060] In the ninth aspect, a chip is provided, which includes: a processor for calling and running a computer program from a memory, so that a communication device equipped with the chip executes a method for executing the above first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] FIG1 shows a schematic diagram of screen projection according to a related technology provided by an embodiment of the present application;

[0062] FIG2 is a schematic diagram showing a scenario in which the screen projection method provided in an embodiment of the present application is applicable;

[0063] FIG3 shows another scenario diagram applicable to the screen projection method provided in an embodiment of the present application;

[0064] FIG4 shows a schematic diagram of software modules of an electronic device provided in an embodiment of the present application;

[0065] FIG5 shows a schematic flow chart of an example screen projection method 800 provided in an embodiment of the present application;

[0066] FIG6 shows a schematic flow chart of an example of decomposing a resource to be broadcast into an audio stream and a video stream according to an embodiment of the present application;

[0067] FIG7 shows a schematic flowchart of another screen projection method 1000 provided in an embodiment of the present application;

[0068] FIG8 shows a schematic flow chart of a first electronic device controlling synchronous playback of resources to be broadcast based on synchronized audio and video streams;

[0069] FIG9 shows another schematic flowchart of a first electronic device controlling synchronous playback of resources to be broadcast based on synchronized audio and video streams;

[0070] FIG10 shows a schematic diagram of an audio and video synchronization module provided in an embodiment of the present application;

[0071] FIG11 shows a schematic flow chart of another screen projection method 1200 provided in an embodiment of the present application;

[0072] FIG12 is a schematic diagram showing a structure of an electronic device 100 provided by the present application;

[0073] FIG13 shows a schematic block diagram of a communication device 1400 provided in an embodiment of the present application;

[0074] FIG14 shows a schematic block diagram of another communication device 1500 provided in an embodiment of the present application;

[0075] FIG15 shows a schematic diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0077] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification and appended claims of this application, the singular expressions "a", "an", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the embodiments of the present application, "one or more" refers to one or more (including two); "and / or" describes the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0078] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0079] The "multiple" involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first" and "second" are only used for the purpose of distinguishing the description and cannot be understood as indicating or implying relative importance or order.

[0080] With the development of Internet technology, screen projection technology has been widely used. Among them, screen projection refers to the placement of media resources on a first electronic device into a second electronic device, so that the second electronic device can also display the media resources of the first electronic device. Through screen projection technology, the media resources of devices with smaller display screens (such as mobile phones and tablets) can be projected onto large-screen display devices (such as TVs and car-mounted multimedia displays), thereby achieving better viewing effects and making it easier for users to operate using the operating devices of large-screen devices.

[0081] However, in home or work scenarios, you often encounter situations where you don't want to play audio externally when projecting the screen. For example, in a quiet environment at home, you project a video on your mobile phone to a large-screen device, but the mobile phone cannot receive audio when connected to a Bluetooth headset. If the large-screen device cannot connect to a Bluetooth headset or audio device, the user can only use the built-in speakers of the large-screen device to play audio when watching the video on the large screen. Playing audio externally will affect other family members, resulting in a poor user experience.

[0082] For example, FIG1 shows a schematic diagram of screen projection of a related technology provided by an embodiment of the present application. As shown in FIG1(a), in a home scenario, when a user is using a mobile phone 101 to watch a Huawei video, he wants to project the video screen on the mobile phone 101 onto the large-screen device 102. As shown in FIG1(b), the user can project the documentary "Brave Stride" played on the display interface 1011 of the mobile phone 101 onto the display interface 1021 of the large-screen device 102 as shown in FIG1(c). After the user projects the documentary onto the large screen, the user cannot receive audio using the Bluetooth headset connected to the mobile phone 101. If the large-screen device 102 is also unable to connect to a Bluetooth headset or audio device, the user can only use the built-in audio on the large-screen device 102 for audio playback.

[0083] To sum up, when a first electronic device (mobile phone) is projecting its screen onto a second electronic device (large-screen device), how to avoid using the large-screen device to directly play sound and affect other people is a problem that needs to be solved in the field of screen projection.

[0084] In view of this, the present application provides a screen projection method, which includes: a first electronic device decomposing the resources to be broadcast into a video stream and an audio stream, and projecting the video stream and the audio stream to a second electronic device (an electronic device that supports video playback) and a third electronic device (an electronic device that supports audio playback), respectively, thereby solving the problem in the related art that only the second electronic device can be used for video playback and audio playback at the same time, resulting in poor user experience.

[0085] Before introducing the screen projection method provided by this application, we will first introduce the application scenarios where this method is applicable.

[0086] FIG2 shows a schematic diagram of a scenario in which a screen projection method provided by an embodiment of the present application is applicable. As shown in FIG2 , in this application scenario, the first electronic device is a mobile phone, the second electronic device is a smart screen, and the third electronic device includes a Bluetooth headset, a speaker 1, and a speaker 2. The mobile phone can obtain the resources to be broadcast, parse the resources to be broadcast, decompose the resources to be broadcast into a video stream and an audio stream, and then project the video stream onto the smart screen and the audio stream onto the Bluetooth headset. The user watches the video through the smart screen and plays the audio through the Bluetooth headset.

[0087] In one possible implementation, the mobile phone can also project the audio stream to speaker 1 and speaker 2 respectively, for example, projecting the treble audio stream of the audio stream to speaker 1 and the bass audio stream of the audio stream to speaker 2, and using speaker 1 and speaker 2 to play the complete audio stream together.

[0088] In another possible implementation, the mobile phone can also project the audio stream to the Bluetooth headset, speaker 1 and speaker 2 respectively. For example, the mobile phone projects the complete audio stream to the Bluetooth headset, projects the treble audio stream of the audio stream to speaker 1, and projects the bass audio stream of the audio stream to speaker 2. The user can receive the audio through the Bluetooth headset, and other users can also hear the complete audio played by speaker 1 and speaker 2 together.

[0089] Figure 3 shows a scenario diagram applicable to another example of the screen projection method provided in an embodiment of the present application. As shown in Figure 3, in this application scenario, the first electronic device is a mobile phone, the second electronic device is a smart screen, and the third electronic device includes a Bluetooth headset, a speaker 1, and a speaker 2. The mobile phone can obtain the resources to be broadcast, and parse the resources to be broadcast, and decompose the resources to be broadcast into a video stream and an audio stream. Then, the mobile phone sends the video stream to the smart screen and the audio stream to the Bluetooth headset. The smart screen can split the video stream into frames, mark specific timestamps on all video frames, and then feed back the video frames with timestamps to the mobile phone. Similarly, the Bluetooth headset splits the audio stream into frames, marks specific timestamps on all audio frames, and then feeds back the audio frames with timestamps to the mobile phone. The mobile phone synchronizes the audio frames and video frames based on the video frames with timestamps fed back by the smart screen and the audio frames with timestamps sent by the Bluetooth headset. The mobile phone uses the synchronized timestamps to control the smart screen to play the video and the Bluetooth headset to play the audio.

[0090] The specific synchronization process is as follows:

[0091] The first one uses the video clock as the master clock and synchronizes the audio to the video. In this synchronization mode, the video playback time of the smart screen is used as a reference to make the audio played by the Bluetooth headset match the video.

[0092] The second type uses the audio clock as the master clock and synchronizes the video to the audio. In this synchronization mode, the audio playback time of the Bluetooth headset is used as a reference, and the video played on the smart screen is matched with the audio.

[0093] The third method is to use the system clock as the master clock, and synchronize audio and video to the system clock. In this embodiment of the application, the system clock can be the clock of the mobile phone. That is, the mobile phone uses its system time as a reference to match the audio and video playback to the system time.

[0094] Finally, the user watches the video resource on the smart screen and plays the audio resource through the Bluetooth headset. Compared with the example shown in Figure 2 above, because the mobile phone synchronizes the playback based on the time-stamped video frames and the time-stamped audio frames, the video and audio of the resource to be broadcast are in a synchronized playback state. It should be understood that this synchronized playback state means that the video display image and the audio playback sound are corresponding.

[0095] In one possible implementation, the mobile phone can also project the audio stream to speaker 1 and speaker 2 respectively, for example, projecting the treble audio stream of the audio stream to speaker 1 and the bass audio stream of the audio stream to speaker 2. Speaker 1 can split the treble audio stream into frames, mark specific timestamps on all corresponding audio frames in the treble audio stream, and then feed back the timestamp-marked audio frames to the mobile phone. Similarly, speaker 2 splits the bass audio stream into frames, marks specific timestamps on all corresponding audio frames in the bass audio stream, and then feeds back the timestamp-marked audio frames to the mobile phone. Based on the video frames with timestamps fed back by the smart screen, the treble audio frames with timestamps fed back by speaker 1, and the bass audio frames with timestamps fed back by speaker 2, the mobile phone synchronizes the video stream and the audio stream. Finally, the user watches the video resource through the smart screen and plays the complete audio stream together through speaker 1 and speaker 2. Compared with the example shown in Figure 2 above, the video resource and audio resource are in a synchronized playback state.

[0096] In another possible implementation, the mobile phone can also project the audio stream to the Bluetooth headset, speaker 1, and speaker 2 separately. For example, the mobile phone projects the complete audio stream corresponding to the resource to be broadcast to the Bluetooth headset, projects the treble audio stream to speaker 1, and projects the bass audio stream to speaker 2. The Bluetooth headset divides the audio stream into frames, marks all frames with specific timestamps, and then feeds the timestamped audio frames back to the mobile phone. Speaker 1 divides the treble audio stream into frames, marks all frames with specific timestamps, and then feeds the timestamped treble audio frames back to the mobile phone. Speaker 2 divides the bass audio stream into frames, marks all frames with specific timestamps, and then feeds the timestamped bass audio frames back to the mobile phone. The mobile phone synchronizes the video stream and the audio stream of the resource to be broadcast based on the time-stamped video frames fed back by the smart screen, the time-stamped audio frames fed back by the Bluetooth headset, the time-stamped high-pitched audio frames fed back by audio 1, and the time-stamped bass audio frames fed back by audio 2. Finally, the user watches the video resource through the smart screen and receives the audio resource through the Bluetooth headset. Other users can also receive the complete audio stream based on speaker 1 and speaker 2.

[0097] In another possible implementation, the user can also control the audio playback status of the Bluetooth headset, speaker 1 or speaker 2 through the smart screen, such as turning up or down the volume, pausing, etc. Or the user can also control the video playback status of the smart screen through the Bluetooth headset, speaker 1 or speaker 2, such as fast forwarding or rewinding the video playback speed.

[0098] The above are application scenarios applicable to the screen projection method provided in the embodiments of the present application. Before introducing the screen projection method provided by the present application, the software module of the electronic device to which the present application is applicable will be described in detail.

[0099] Exemplarily, FIG4 shows a schematic diagram of software modules of an electronic device provided in an embodiment of the present application. As shown in FIG4 , the first electronic device includes a media parsing module, a network transmission module, an audio and video synchronization module, and a control module.

[0100] Among them, the media analysis module is used to obtain and analyze the resources to be broadcast in the screen projection scenario of the media stream, and decompose the resources to be broadcast into a video stream and an audio stream. The network transmission module is used to transmit the video stream resources and audio stream resources decomposed by the media analysis module to the second electronic device and the third electronic device respectively. The network transmission module is also used to obtain the video frames marked with timestamps from the second electronic device and obtain the audio frames marked with timestamps from the third electronic device. The audio and video synchronization module is used to synchronize the acquired video stream marked with timestamps and the audio stream marked with timestamps, and then synchronously play the audio stream and the video stream, so as to achieve the effect of audio and video synchronization. The control module is used to adjust the playback status of the third electronic device according to the instructions of the second electronic device, or the control module is used to adjust the playback status of the second electronic device according to the instructions of the third electronic device.

[0101] For ease of understanding, the following embodiments of this application will take an electronic device having the structure shown in Figure 4 as an example, and combine the accompanying drawings and application scenarios to specifically explain the screen projection method provided in the embodiments of this application.

[0102] FIG5 shows a schematic flow chart of an example screen projection method 500 provided in an embodiment of the present application. As shown in FIG5 , the method 500 shown in FIG5 may include steps S510 to S520a and step S520b. The steps in the method 500 are described in detail below in conjunction with FIG5 .

[0103] It should be understood that in the embodiment of the present application, the method 500 is described by taking the first electronic device as the execution subject of the method 500. As an example and not a limitation, the execution subject of the method 800 may also be a chip used in the first electronic device.

[0104] S510: The first electronic device obtains a resource to be broadcast and decomposes the resource to be broadcast into a video stream and an audio stream;

[0105] In the embodiments of the present application, the first electronic device can be understood as an audio and video source device, which is used to provide video content to the video playback device, and the first electronic device is also used to provide audio content to the audio playback device. For example, in the embodiments of the present application, the audio and video source device can be, for example, a mobile phone, a tablet computer, a personal computer (PC), a personal digital assistant (PDA), a netbook, a wearable device, an augmented reality (AR) device, a virtual reality (VR) device, an in-vehicle device, a smart screen, etc. The present application does not impose any special restrictions on the specific form of the first electronic device.

[0106] It should be understood that the first electronic device can obtain the resource to be broadcast locally or obtain the online resource to be broadcast from the network. For example, when the resource to be broadcast is an online resource to be broadcast, the first electronic device can obtain the URL corresponding to the resource to be broadcast. Based on the URL, the first electronic device can obtain the resource to be broadcast from the content delivery network (CDN). The resource to be broadcast includes video resources and audio resources.

[0107] In an embodiment of the present application, in order to ensure that the video corresponding to the resource to be broadcast on the first electronic device can be played on the video playback device, and the audio corresponding to the resource to be broadcast can be played on the audio playback device, the first electronic device can decompose the resource to be broadcast into a video stream and an audio stream after obtaining the resource to be broadcast.

[0108] In one possible implementation, when the resource to be broadcast is stored locally on the first electronic device, the first electronic device can directly obtain the audio and video encoding data from the packaging container based on the location of the locally stored resource to be broadcast. The first electronic device then converts the audio encoding data into an audio digital signal and the video encoding data into a video digital signal, thereby obtaining the audio and video streams of the resource to be broadcast.

[0109] It should be understood that the encapsulation container can be understood as the resource to be broadcast, and the encapsulation container also includes the length, size, format information and audio and video coding position information of the resource to be broadcast.

[0110] In another possible implementation, when the resource to be broadcast is a file based on a streaming media protocol, the first electronic device first needs to parse the protocol, that is, the first electronic device needs to read the resource to be broadcast according to the rules of the index file. For example, the first electronic device can obtain a URL based on the streaming media protocol, and then obtain the resource to be broadcast from the CDN based on the URL. The first electronic device then extracts the audio and video encoding data based on the obtained resource to be broadcast. The audio encoding data is then converted into an audio digital signal, and the video encoding data is converted into a video digital signal, thereby obtaining the audio stream and video stream of the resource to be broadcast.

[0111] It should also be understood that the rules of the index file may refer to the location information and acquisition method of the resource to be broadcast, for example, the acquisition method is downloading, etc. The first electronic device can read the resource to be broadcast based on the location information and acquisition method.

[0112] In some embodiments, the first electronic device includes a media parsing module that is configured to obtain and parse resources to be broadcast, decomposing the resources into audio and video streams. Of course, other modules on the first electronic device may also obtain and parse resources to be broadcast. This embodiment of the present application does not specifically limit the execution module for obtaining and parsing resources to be broadcast.

[0113] For example, FIG6 shows a schematic flow chart of an example of decomposing the resources to be broadcast into audio streams and video streams provided by an embodiment of the present application. As shown in FIG6, the resources to be broadcast can be located on the network side or the local disk side. When the resources to be broadcast are located on the network device, the first electronic device obtains audio coding data and video coding data from the encapsulation container based on the streaming media protocol according to the rules of the index file, thereby converting the audio coding data into an audio digital signal to extract the audio stream, and converting the video coding data into a video data signal to extract the video stream; when the resources to be broadcast are located on the local disk, the first electronic device directly obtains the audio coding data and video coding data in the encapsulation container storing the resources to be broadcast, thereby converting the audio coding data into an audio digital signal to extract the audio stream, and converting the video coding data into a video data signal to extract the video stream.

[0114] S520a: The first electronic device sends the video stream to the second electronic device, and the second electronic device is used to play the video corresponding to the resource to be broadcast.

[0115] S520b: The first electronic device sends the audio stream to a third electronic device, and the third electronic device is configured to play the audio corresponding to the resource to be broadcast.

[0116] The first electronic device sends the decomposed video stream to the second electronic device, and the second electronic device is used to play the video corresponding to the resource to be broadcast. For example, the second electronic device in the embodiment of the present application can be a smart TV, a projection device, a computer, various portable notebooks, various tablet computers, and other electronic devices with a display screen. Optionally, the electronic device can also be a personal digital assistant (PDA) with a larger display screen, a handheld device with a larger display screen, a computing device, a vehicle-mounted device, a wearable device, an electronic device in a 5G network, or an electronic device in an evolved public land mobile communication network (PLMN), etc., and the embodiment of the present application is not limited to this.

[0117] The first electronic device sends the decomposed audio stream to a third electronic device, which is used to play the audio corresponding to the resource to be broadcast. For example, the third electronic device in the embodiment of the present application can be an audio playback device, which is used to receive the audio content sent by the first electronic device and play the audio content. Exemplarily, the third electronic device can be, for example, a wireless headset, a wireless speaker, a wearable device, an AR device, a VR device, etc., and the present application does not impose any special restrictions on the specific form of the third electronic device.

[0118] In some embodiments, the third electronic device may include an electronic device. For example, as shown in the scenario diagram of FIG2 , the first electronic device sends the audio stream to the headset, and the headset acts as the third electronic device to receive the audio stream sent by the first electronic device for playback.

[0119] In other embodiments, the third electronic device may include multiple electronic devices, the audio stream includes multiple sub-audio streams, and each sub-audio stream is of a different type. For example, as shown in the scenario diagram of Figure 3, the first electronic device sends two sub-audio streams of the audio stream to speaker 1 and speaker 2 respectively, and speaker 1 and speaker 2 act as the third electronic device to receive the sub-audio streams sent by the first electronic device for playback.

[0120] Optionally, the types of the multiple sub-audio streams may include a treble audio stream, a mid-range audio stream, and a bass audio stream, or the types of the multiple sub-audio streams may also include a left channel audio stream and a right ear channel audio stream, etc. Of course, the multiple sub-audio streams may also include other types, and the embodiments of the present application do not specifically limit this.

[0121] In a possible implementation, the first electronic device includes a network transmission module, which is used to broadcast the video stream parsed by the first electronic device to the second electronic device, and broadcast the parsed audio stream to the third electronic device.

[0122] In method 500, when performing streaming media broadcasting, the first electronic device obtains and parses the resources to be broadcasted, decomposing the resources into video streams and audio streams. The video stream is then broadcast to a video playback device, such as a large screen or car computer, and the audio stream is broadcast to an audio playback device, such as a Bluetooth device or headphones. This provides users with a distributed experience with separate audio and video streams, thereby bringing users a more flexible and customized screen broadcasting experience, thereby improving the user experience.

[0123] It should be understood that video is played frame by frame, with the image display device displaying one frame at a time. The video playback speed is determined by the frame rate, which indicates how many frames are displayed per second. Audio is also played in audio frames, with the sound playback device playing one audio frame at a time. The sound playback speed can also be determined by the frame rate. If only relying on the mechanism of playing video and audio at frame rates, then in method 500, when the second electronic device plays the video and the third electronic device plays the audio, the lack of a synchronization mechanism between the two may result in a lack of synchronization between the audio and video.

[0124] Therefore, the present application also provides another example of a screen projection method 700, which includes: after the first electronic device sends the video stream to the second electronic device and sends the audio stream to the third electronic device, the second electronic device can feedback the timestamp corresponding to each frame in the video stream to the first electronic device, and the third electronic device can feedback the timestamp corresponding to each frame in the audio stream to the first electronic device, and the first electronic device can synchronize the video stream and the audio stream based on the feedback from the second electronic device and the third electronic device. Therefore, this method can achieve time synchronization of video streams and audio streams on different electronic devices in a scenario where the audio stream and the video stream are separated, further improving the user experience.

[0125] Below, another screen projection method 700 provided in an embodiment of the present application is described in detail in combination with the accompanying drawings and application scenarios.

[0126] FIG7 shows a schematic flow chart of another example of a screen projection method 700 provided in an embodiment of the present application. As shown in FIG7 , the method 700 shown in FIG7 may include steps S710 to S750. The steps in the method 700 are described in detail below in conjunction with FIG7 .

[0127] It should be understood that in the embodiment of the present application, the method 700 is described by taking the first electronic device as the execution subject of the method 700. As an example and not a limitation, the execution subject of the method 700 may also be a chip used in the first electronic device.

[0128] S710: The first electronic device obtains a resource to be broadcast and decomposes the resource to be broadcast into a video stream and an audio stream;

[0129] S720a: The first electronic device sends the video stream to the second electronic device, and the second electronic device plays the video corresponding to the resource to be broadcasted;

[0130] S720b: The first electronic device sends the audio stream to a third electronic device, and the third electronic device is configured to play the audio corresponding to the resource to be broadcast.

[0131] The specific implementation of step S710, step S720a, and step S720b can refer to the specific implementation of step S510, step S520a, and step S520b, and will not be repeated here.

[0132] S730: The first electronic device obtains a video stream with a time stamp sent by the second electronic device and an audio stream with a time stamp sent by the third electronic device.

[0133] After the first electronic device sends the video stream corresponding to the resource to be broadcast to the second electronic device, the second electronic device parses the video stream, divides the video stream into frames, and marks each frame with a specific timestamp. The specific timestamp indicates that each frame needs to be played at the specific timestamp. The second electronic device can then send the video stream with the timestamp to the first electronic device.

[0134] Similarly, when the first electronic device sends the audio stream corresponding to the resource to be broadcast to the third electronic device, the third electronic device parses the audio stream, divides the audio stream into frames, and marks a specific timestamp on all frames. The specific timestamp indicates that each frame needs to be played according to the specific timestamp.

[0135] S740: The first electronic device performs audio and video synchronization based on the acquired video stream with the timestamp and the acquired audio stream with the timestamp.

[0136] In order to synchronize the video on the second electronic device and the audio on the third electronic device, the first electronic device performs audio and video synchronization based on the acquired video stream with a time stamp and the acquired audio stream with a time stamp.

[0137] In one possible implementation, audio and video synchronization can be achieved by establishing a master clock and using a non-master audio or video clock as a slave clock. During playback, the master clock serves as the synchronization reference, continuously determining the difference between the slave clock and the master clock, and adjusting the slave clock so that it catches up with the master clock when it lags behind, and waits for the master clock when it is ahead. It should be understood that the master clock can be the audio clock, the video clock, or the system clock.

[0138] Depending on the type of master clock, audio and video synchronization modes can be divided into the following three types:

[0139] The first is that the video clock is used as the master clock and the audio is synchronized to the video. In this synchronization mode, the time of the video playback of the second electronic device is used as a reference to make the audio played by the third electronic device match the video.

[0140] The second is that the audio clock is used as the master clock and the video is synchronized to the audio. In this synchronization mode, the audio playback time of the third electronic device is used as a reference, and the video and audio played by the second electronic device are matched.

[0141] The third method is to use the system clock as the master clock and synchronize audio and video to the system clock. In this embodiment of the application, the system clock can be the clock of the first electronic device. That is, the first electronic device uses its system time as a reference to match audio and video playback to the system time.

[0142] It should be understood that in the second approach, using the audio clock as the master clock does not adjust the playback time, playback speed, or duration of audio frames. When the audio clock is used as the master clock, the playback time of audio frames is not altered, thus avoiding the occurrence of plosives. Therefore, using the second approach for audio and video synchronization can improve the user experience.

[0143] It should be noted that in the embodiment of the present application, synchronization can be performed based on any one of the above three audio and video synchronization methods, and the embodiment of the present application does not make any specific limitations on this.

[0144] S750: The first electronic device controls the synchronous playback of the resources to be broadcast based on the synchronized audio and video streams.

[0145] After the first electronic device synchronizes the acquired video stream of the second electronic device and the audio stream of the third electronic device, it controls the synchronous playback of the resources to be broadcast based on the synchronized audio stream and video stream, so that the video stream watched by the user on the second electronic device and the audio stream heard on the third electronic device are played synchronously, further improving the user experience.

[0146] The first electronic device controls the synchronous playback of the resource to be broadcast based on the synchronized audio and video streams, including the following two implementation methods:

[0147] In one possible implementation, FIG8 shows a schematic flowchart of a first electronic device controlling the synchronous playback of resources to be broadcast based on the synchronized audio and video streams. As shown in the figure, the implementation includes steps S810a, S810b-S820a, and S810b.

[0148] S810a: The first electronic device sends the synchronized timestamp to the second electronic device.

[0149] S810b: The first electronic device sends the synchronized timestamp to the third electronic device.

[0150] Based on step S740, the first electronic device can synchronize the audio and video streams of the resource to be broadcast based on the acquired video stream with the timestamp and the audio stream with the timestamp. In steps S810a and S810b, the first electronic device sends the synchronized timestamps to the second electronic device and the third electronic device respectively.

[0151] It should be understood that the synchronized timestamp can be understood as the audio frame and video frame that need to be played at a specific timestamp.

[0152] S820a: The second electronic device obtains the video data of the corresponding time point based on the synchronized timestamp and plays it.

[0153] S820b: The third electronic device obtains the audio data of the corresponding time point based on the synchronized timestamp and plays it.

[0154] In steps S820a and S820b, after the second electronic device receives the synchronized timestamp, it obtains the video data at the corresponding time point from the video stream sent by the first electronic device and plays it. After the third electronic device receives the synchronized timestamp, it obtains the audio data at the corresponding time point from the audio stream sent by the first electronic device and plays it, so that the video viewed by the user on the second electronic device and the audio heard by the user on the third electronic device are played synchronously.

[0155] For example, after the first electronic device synchronizes the audio and video, assuming that the second frame of video data needs to be played in the second second and the third frame of audio data needs to be played in the second second to synchronize the audio and video, the second electronic device, after acquiring the video stream, plays the second frame of video data in the second second. The third electronic device, after acquiring the audio stream, plays the third frame of audio data in the second second to synchronize the audio and video.

[0156] In this implementation, the first electronic device needs to transmit the synchronized timestamp back to the second electronic device and the third electronic device, and achieve audio and video synchronization with the help of the second electronic device and the third electronic device.

[0157] In another possible implementation, FIG9 shows another schematic flowchart of a first electronic device controlling synchronous playback of resources to be broadcast based on synchronized audio and video streams. As shown in the figure, the implementation includes steps S910-S920.

[0158] S910: The first electronic device controls the second electronic device to play the video stream at a specified time based on the synchronized timestamp.

[0159] Optionally, the first electronic device may control the second electronic device to play the video stream at a specified time using a seek instruction based on the synchronized timestamp.

[0160] It should be understood that the seek instruction means that when the second electronic device reads each frame of video in the video stream file, the first electronic device controls the second electronic device to move the position of the read video frame.

[0161] For example, when the second frame of video needs to be played in the second second, the first electronic device can control the second electronic device to play the second frame of video in the second second based on the seek instruction.

[0162] Of course, the first electronic device may also control the second electronic device to play the video stream at a specified time based on other methods, and the embodiment of the present application does not specifically limit this.

[0163] S920: The first electronic device controls the third electronic device to play the audio stream at a specified time based on the synchronized timestamp.

[0164] Optionally, the first electronic device can use a seek instruction to control the second electronic device to play the video stream at a specified time based on the synchronized timestamp, so that the video stream viewed by the user on the second electronic device and the audio stream heard by the third electronic device are synchronized.

[0165] It should be understood that the seek instruction means that when the second electronic device reads each frame of video in the video stream file, the first electronic device controls the second electronic device to move the position of the read video frame.

[0166] For example, when the second frame of video needs to be played in the second second, the first electronic device can control the second electronic device to play the second frame of video in the second second based on the seek instruction.

[0167] Of course, the first electronic device may also control the second electronic device to play the video stream at a specified time based on other methods, and the embodiment of the present application does not specifically limit this.

[0168] In this implementation, the first electronic device does not need to transmit the synchronized timestamp back to the second electronic device and the third electronic device. The first electronic device can directly control the audio and video synchronization of the second electronic device and the third electronic device, thereby improving the efficiency of audio and video synchronization.

[0169] Exemplarily, Figure 10 shows a schematic diagram of an audio and video synchronization module provided in an embodiment of the present application. As shown in Figure 10, after the first electronic device obtains the resources to be broadcast, it sends the parsed audio stream and video stream to the second electronic device and the third electronic device. The second electronic device extracts multiple data packets from the video data packet queue in sequence based on the received video stream, and then sends the multiple video data packets to the decoder through the video thread for decoding. After decoding by the decoder, the second electronic device receives the decoded frame queue, and then the second electronic device marks the timestamp that needs to be displayed for each frame based on the decoded frame queue, and finally outputs the video stream marked with the timestamp.

[0170] Similarly, the third electronic device extracts multiple audio data packets from the audio data packet queue in sequence based on the received audio stream, and then sends the multiple audio data packets to the decoder through the audio thread for decoding. After decoding by the decoder, the third electronic device receives the frame queue of the decoded audio, and then the third electronic device marks the timestamp of each frame to be played based on the decoded frame queue, and finally outputs the audio stream with the timestamp marked.

[0171] Finally, the first electronic device controls the synchronous playback of the resources to be broadcast based on the timestamped video stream and the timestamped audio stream. In the example of Figure 10, the third electronic device outputs the timestamped audio stream. The first electronic device sends the synchronized timestamp to the second electronic device based on the received timestamped audio stream. The second electronic device obtains the video data at the corresponding time point based on the synchronized timestamp and plays it, so that the video played by the second electronic device and the audio played by the third electronic device are synchronized.

[0172] In method 700, after the first electronic device sends the video stream to the second electronic device and the audio stream to the third electronic device, the second electronic device can feedback the timestamp corresponding to the video stream to the first electronic device, and the third electronic device can feedback the timestamp corresponding to the audio stream to the first electronic device. The first electronic device can synchronize the video stream and the audio stream based on the feedback from the second electronic device and the third electronic device. Therefore, this method can achieve time synchronization of video streams and audio streams on different electronic devices in a scenario where the audio stream and the video stream are separated, further improving the user experience.

[0173] The present application also provides another screen projection method 1100, which includes: a first electronic device can receive control information from a second electronic device, and then control the playback of a third electronic device based on the control information of the second electronic device. Alternatively, the first electronic device can receive control information from a third electronic device, and then control the playback of the second electronic device based on the control information of the third electronic device. This method can complete the control of the resources to be projected on either side, providing a more flexible and convenient experience for users.

[0174] Below, another screen projection method 1100 provided in an embodiment of the present application is described in detail in combination with the accompanying drawings and application scenarios.

[0175] FIG11 shows a schematic flow chart of another screen projection method 1100 provided in an embodiment of the present application. As shown in FIG11 , the method 1100 shown in FIG11 may include steps S1110 to S1160. The steps in the method 1100 are described in detail below with reference to FIG11 .

[0176] It should be understood that in the embodiment of the present application, the method 1200 is described by taking the first electronic device as the execution subject of the method 1200. As an example and not a limitation, the execution subject of the method 1200 may also be a chip used in the first electronic device.

[0177] S1110: The first electronic device obtains a resource to be broadcast and decomposes the resource to be broadcast into a video stream and an audio stream;

[0178] S1120a: The first electronic device sends the video stream to the second electronic device, and the second electronic device plays the video corresponding to the resource to be broadcasted;

[0179] S1120b: The first electronic device sends the audio stream to a third electronic device, and the third electronic device is configured to play the audio corresponding to the resource to be broadcast.

[0180] S1130: The first electronic device obtains a video stream with a time stamp sent by the second electronic device and an audio stream with a time stamp sent by the third electronic device.

[0181] S1140: The first electronic device performs audio and video synchronization based on the acquired video stream with a timestamp and the acquired audio stream with a timestamp.

[0182] S1150: The first electronic device broadcasts the resource to be broadcast based on the synchronized audio and video streams.

[0183] The specific implementation of steps S1110 to S1150 can refer to the specific implementation of steps S710 to S750, which will not be repeated here.

[0184] It should be understood that steps S1130 to S1150 are optional steps.

[0185] S1160: The first electronic device receives control information from the second electronic device, and controls the playback of the third electronic device based on the control information, or the first electronic device receives control information from the third electronic device, and controls the playback of the second electronic device based on the control information.

[0186] It should be understood that in the scenario shown in Figure 5 or Figure 6, after the first electronic device, the second electronic device and the third electronic device establish a screen projection connection, the playback parameters of other electronic devices can be controlled by one of the electronic devices. The playback parameters can be volume, playback speed (for example, 1x speed, 1.5x speed, 2x speed), playback progress, playback control parameters (for example, pause, fast forward, fast rewind, switch to previous, switch to next), etc.

[0187] In some possible implementations, the control information includes at least one of the following: identification information of the second electronic device, identification information of the third electronic device, volume information, playback speed, video track identification information, or audio track identification information.

[0188] It should be understood that the second electronic device as a video playback device may include multiple video track identification information, and the third electronic device as an audio playback device may also include multiple audio track identification information.

[0189] Optionally, the control information may be a control field. For example, when the second electronic device receives a volume adjustment operation performed by a user on the second electronic device on the audio playback of the third electronic device, the second electronic device may send the control field to the first electronic device. The first electronic device obtains the device to be controlled and the size of the playback parameters based on the control field, thereby adjusting the volume of the device to be controlled.

[0190] Specifically, the display screen of the second electronic device can display the device identification information of multiple third electronic devices. The user can select the device to be controlled from the device identification information of multiple third electronic devices and trigger the sending of control information through the volume adjustment control of the second electronic device.

[0191] For example, Table 1 shows an example of a control field sent by the second electronic device to the first electronic device:

[0192] Table 1 Control fields sent from the second electronic device to the first electronic device

[0193] As can be seen from Table 1, the id in the field sent by the second electronic device to the first electronic device can represent the id of the second electronic device, volume represents the volume to be controlled, and Audio Track represents the id of the audio track to be controlled. When the first electronic device receives this field, it can adjust the volume of the third electronic device based on the control field.

[0194] Alternatively, the display screen of the second electronic device may also display multiple audio track identification information of the third electronic device. The user can select the audio track to be controlled from the multiple audio track identification information and trigger the transmission of control information for the audio track using the volume control of the second electronic device. In this implementation, the control information may include: identification information of the second electronic device, audio track identification information of the third electronic device, and volume level.

[0195] Alternatively, when a third electronic device receives a user's operation to adjust the video playback speed of a second electronic device on the third electronic device, the third electronic device may send a control field to the first electronic device, and the first electronic device may adjust the video playback speed of the second electronic device based on the control field. In this implementation, the control information may include: identification information of the third electronic device, identification information of the second electronic device, and the video playback speed.

[0196] For example, when the first electronic device is a mobile phone, the second electronic device is a large-screen device, and the third electronic device is a Bluetooth headset and a speaker, and the identification information of multiple audio playback devices can be displayed on the large-screen device, the user can select the device to be controlled based on the identification information of the Bluetooth headset and the speaker. Based on the user's operation of selecting the device to be controlled (for example, the Bluetooth headset), the mobile phone receives the control information sent by the large-screen device, and then the mobile phone sends a control message carrying the control information to the Bluetooth headset (the volume is adjusted to 10 decibels). The Bluetooth headset receives the control message and controls the audio on the Bluetooth headset to play at a volume of 10 decibels.

[0197] For another example, when the first electronic device is a mobile phone, the second electronic device is a large-screen device, and the third electronic device is a Bluetooth headset, the user can control the playback of the video on the large-screen device based on the buttons on the Bluetooth headset. When the user clicks the button on the Bluetooth headset, the mobile phone receives the control information sent by the Bluetooth headset, and then the mobile phone sends a control message carrying the control information to the large-screen device (video pause playback). The large-screen device receives the control message and controls the large-screen device to pause video playback.

[0198] Therefore, based on method 1100, a first electronic device can receive control information from a second electronic device and then control playback on a third electronic device based on the control information from the second electronic device. Alternatively, the first electronic device can receive control information from a third device and then control playback on the second electronic device based on the control information from the third electronic device. This method allows control of the resources to be broadcasted to be completed on either side, providing a more flexible and convenient experience for users.

[0199] The above description in combination with Figures 2-11 describes an embodiment of the screen projection method provided by an embodiment of the present application. The following describes an electronic device provided by an embodiment of the present application.

[0200] For example, FIG12 shows a schematic diagram of the structure of an electronic device 100 provided by the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0201] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0202] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0203] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 310 needs to use the instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0204] In an embodiment of the present application, the processor may obtain the resources to be broadcast locally or online, and parse the resources to be broadcast to obtain a video stream and an audio stream.

[0205] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 350, the wireless communication module 360, the modem processor and the baseband processor.

[0206] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0207] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0208] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) applied to the electronic device 100, such as wireless fidelity (Wi-Fi) networks, Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0209] In an embodiment of the present application, when the processor decomposes the resources to be broadcast into audio stream and video stream, the wireless communication module can receive the audio stream and video stream through the processor, and send the video stream to the second electronic device and the audio stream to the third electronic device.

[0210] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), LTE, BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS) and / or satellite-based augmentation system (SBAS).

[0211] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0212] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0213] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0214] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0215] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0216] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 is performing frequency selection, the digital signal processor is used to perform Fourier transform on the frequency energy.

[0217] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0218] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0219] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 310 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0220] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0221] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0222] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0223] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0224] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0225] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0226] Of course, the electronic device 100 may also include a charging management module, a power management module, a battery, buttons, indicators, and one or more SIM card interfaces, etc., and the embodiments of the present application do not impose any restrictions on this.

[0227] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0228] It should be understood that the specific process of the electronic device 100 executing the above corresponding steps can be referred to the relevant description of the electronic device execution steps described in each embodiment described above. For the sake of brevity, it is not repeated here.

[0229] This embodiment can also divide the electronic device into functional modules according to the above method. For example, the electronic device can be divided into functional modules corresponding to different functions, or two or more functions can be integrated into a single processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and represents only a logical functional division. In actual implementation, other division methods may be used.

[0230] It should be noted that the relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0231] The electronic device provided in the embodiment of the present application is used to perform any one of the screen projection methods provided in the above method embodiment, so that the same effect as the above implementation method can be achieved. In the case of an integrated unit, the terminal device may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the terminal device. For example, it can be used to support the terminal device to execute the steps performed by the processing unit. The storage module can be used to support the storage of program code and data, etc. The communication module can be used to support communication between the terminal device and other devices.

[0232] The processing module may be a processor or a controller. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and so on. The storage module may be a memory. The communication module may specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, or a Wi-Fi chip.

[0233] Exemplarily, Figure 13 shows a schematic block diagram of a communication device 1300 provided in an embodiment of the present application. The communication device 1300 may correspond to the first electronic device described in each embodiment of the above-mentioned methods 800 to 1200, or may be a chip or component applied to the first electronic device. Moreover, each module or unit in the communication device 1300 is respectively used to execute each action or processing process performed by the first electronic device described in each embodiment of the above-mentioned methods 800 to 1200.

[0234] As shown in FIG13 , the communication device 1300 includes a transceiver unit 1310 and a processing unit 1320. The transceiver unit 1310 is configured to perform specific signal transmission and reception under the drive of the processing unit 1320.

[0235] In some embodiments:

[0236] The processing unit 1320 is configured to obtain resources to be broadcast.

[0237] The processing unit 1320 is further configured to decompose the resources to be broadcast into a video stream and an audio stream.

[0238] The transceiver unit 1310 is configured to send the video stream to the second electronic device, and the second electronic device is configured to play the video corresponding to the resource to be broadcast.

[0239] The transceiver unit 1310 is further configured to send the audio stream to a third electronic device, and the third electronic device is configured to play the audio corresponding to the resource to be broadcast.

[0240] The communication device provided in this application can obtain resources to be broadcast, which include local resources and online resources. After obtaining the resources to be broadcast, the resources to be broadcast are parsed to obtain the audio stream and video stream of the resources to be broadcast, and then the video stream of the resources to be broadcast is sent to the video playback device (second electronic device) so that the video of the resources to be broadcast is played on the second electronic device, and the audio stream of the resources to be broadcast is sent to the audio playback device (third electronic device) so that the audio of the resources to be broadcast is played on the third electronic device. This method solves the problem in the related art that the video and audio of the resources to be broadcast can only be played simultaneously on the second electronic device, thereby improving the user experience.

[0241] Optionally, the transceiver unit 1310 is further configured to receive a video stream with a timestamp sent by a second electronic device.

[0242] Optionally, the transceiver unit 1310 is further configured to receive an audio stream with a timestamp sent by a third electronic device.

[0243] Optionally, the processing unit 1320 is further configured to synchronize the audio and video of the resource to be broadcast based on the video stream with a timestamp and the audio stream with a timestamp, so that the video played on the second electronic device and the audio played on the third electronic device are synchronized.

[0244] Optionally, the transceiver unit 1310 is further configured to receive control information sent by the second electronic device or the third electronic device.

[0245] Optionally, the processing unit 1320 is further configured to control audio playback of a third electronic device based on the control information sent by the second electronic device, or control video playback of the second electronic device based on the control information sent by the third electronic device.

[0246] Optionally, the control information includes at least one of the following: identification information of the second electronic device, identification information of the third electronic device, volume information, video track identification information, or audio track identification information.

[0247] Optionally, the audio stream includes multiple different sub-audio streams, and the third electronic device includes multiple electronic devices, and the multiple electronic devices are used to play different sub-audio streams.

[0248] Optionally, the transceiver unit 1310 is further configured to send a control message carrying the control information, where the control message is used to instruct the third electronic device to control the playback of the audio corresponding to the resource to be broadcasted on the third electronic device.

[0249] Optionally, the transceiver unit 1310 is further configured to send a control message carrying control information, where the control message is used to instruct the second electronic device to control the playback of the video corresponding to the resource to be broadcasted on the second electronic device.

[0250] Furthermore, the communication device 1300 may also include a storage unit, and the transceiver unit 1310 may be a transceiver, an input / output interface, or an interface circuit. The storage unit is used to store instructions executed by the transceiver unit 1310 and the processing unit 1320. The transceiver unit 1310, the processing unit 1320, and the storage unit are coupled to each other. The storage unit stores instructions, the processing unit 1320 is used to execute the instructions stored in the storage unit, and the transceiver unit 1310 is used to perform specific signal transmission and reception under the control of the processing unit 1320.

[0251] It should be understood that the specific process of each unit in the communication device 1300 executing the above corresponding steps can be referred to the description of the first electronic device in combination with method 500 to method 1200 and the relevant embodiments in Figures 5 to 12 in the previous text. For the sake of brevity, it is not repeated here.

[0252] It should be understood that the transceiver unit 1310 may be a transceiver, an input / output interface, or an interface circuit. The storage unit may be a memory. The processing unit 1320 may be implemented by a processor. Figure 14 shows a schematic block diagram of another example communication device 1400 provided in an embodiment of the present application. As shown in Figure 14, the communication device 1400 may include a processor 1410, a memory 1420, and a transceiver 1430.

[0253] The communication device 1300 shown in FIG13 or the communication device 1400 shown in FIG14 can implement the steps performed by the electronic device in each embodiment of the aforementioned methods 800 to 1200. Similar descriptions can refer to the descriptions of the aforementioned corresponding methods. To avoid repetition, they are not repeated here.

[0254] It should also be understood that the communication device 1300 shown in FIG. 13 or the communication device 1400 shown in FIG. 14 may be an electronic device.

[0255] An embodiment of the present application also provides a chip system, as shown in Figure 15, which includes at least one processor 1501 and at least one interface circuit 1502. The processor 1501 and the interface circuit 1502 can be interconnected via lines. For example, the interface circuit 1502 can be used to receive signals from other devices (such as the memory of the above-mentioned electronic device). For another example, the interface circuit 1502 can be used to send signals to other devices (such as the processor 1501). Exemplarily, the interface circuit 1502 can read the instructions stored in the memory and send the instructions to the processor 1501. When the instructions are executed by the processor 1501, the electronic device can execute the various steps performed by any electronic device in the above embodiments. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiment of the present application.

[0256] It should also be understood that the division of units in the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, the units in the device can all be implemented in the form of software called through processing elements; or all be implemented in the form of hardware; or some units can be implemented in the form of software called through processing elements, and some units can be implemented in the form of hardware. For example, each unit can be a separately established processing element, or it can be integrated into a certain chip of the device. In addition, it can also be stored in a memory in the form of a program, and called by a certain processing element of the device to execute the function of the unit. Here, the processing element can also be called a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each unit above can be implemented by the integrated logic circuit of the hardware in the processor element or in the form of software called through the processing element. In one example, the unit in any of the above devices may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms. For another example, when the unit in the device can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0257] The present application also provides an apparatus, included in an electronic device, that implements the electronic device behavior described in any of the above embodiments. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes at least one module or unit corresponding to the above functionality.

[0258] The present application also provides an electronic device, which includes the device provided in the above-mentioned embodiment of the present application.

[0259] The present application also provides a computer-readable storage medium for storing computer program code, wherein the computer program includes instructions for executing the steps of the electronic device performing the compilation method in any of the embodiments provided in the present application. The readable medium may be a read-only memory (ROM) or a random access memory (RAM), which is not limited in the present application.

[0260] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the electronic device executes the steps executed by the electronic device in any of the above embodiments.

[0261] The present application also provides a chip including a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer instructions to cause an electronic device to perform any of the screen projection methods provided in the embodiments of the present application.

[0262] Optionally, the computer instructions are stored in a storage unit.

[0263] Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit within the terminal located outside the chip, such as a ROM or other types of static storage devices that can store static information and instructions, random RAM, etc. Wherein, the processor mentioned in any of the above may be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned electronic projection display method. The processing unit and the storage unit may be decoupled and respectively provided on different physical devices, and connected by wired or wireless means to implement the respective functions of the processing unit and the storage unit, so as to support the system chip to implement the various functions in the above-mentioned embodiments. Alternatively, the processing unit and the memory may also be coupled on the same device.

[0264] Among them, the electronic device, device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0265] It is understandable that, in order to realize the above functions, the above-mentioned electronic devices and the like include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0266] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device etc. according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0267] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0268] The functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0269] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

[0270] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A screen mirroring method, characterized in that, The method is applied to a first electronic device, and the method includes: Obtain the resource to be projected; Decompose the resource to be projected into a video stream and an audio stream; Send the video stream to a second electronic device, where the second electronic device is used to play the video corresponding to the resource to be projected; Send the audio stream to a third electronic device, where the third electronic device is used to play the audio corresponding to the resource to be projected.

2. The method according to claim 1, wherein The method further includes: Receive the video stream with timestamps sent by the second electronic device; Receive the audio stream with timestamps sent by the third electronic device; Synchronize the audio corresponding to the resource to be projected and the video corresponding to the resource to be projected based on the video stream with timestamps and the audio stream with timestamps, so that the video corresponding to the resource to be projected played on the second electronic device and the audio corresponding to the resource to be projected played on the third electronic device are synchronized.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Receive the control information sent by the second electronic device; Based on the control information sent by the second electronic device, control the playback of the audio corresponding to the resource to be projected on the third electronic device.

4. The method according to claim 1 or 2, characterized in that, The method further includes: Receive the control information sent by the third electronic device; Based on the control information sent by the third electronic device, control the playback of the video corresponding to the resource to be projected on the second electronic device.

5. The method according to claim 3 or 4, characterized in that, The control information includes at least one of the following: identification information of the second electronic device, identification information of the third electronic device, volume information, video track identification information, or audio track identification information.

6. The method according to any one of claims 1-5, characterized in that, The audio stream includes a plurality of different sub-audio streams, the third electronic device includes a plurality of electronic devices, and the plurality of electronic devices are used to play different sub-audio streams.

7. The method according to claim 3, wherein The controlling the playback of the audio corresponding to the resource to be projected on the third electronic device based on the control information sent by the second electronic device specifically includes: Send a control message carrying the control information, where the control message is used to instruct the third electronic device to control the playback of the audio corresponding to the resource to be projected on the third electronic device.

8. The method according to claim 4, wherein, The controlling the playback of the video corresponding to the resource to be projected on the second electronic device based on the control information sent by the third electronic device specifically includes: Send a control message carrying the control information, where the control message is used to instruct the second electronic device to control the playback of the video corresponding to the resource to be projected on the second electronic device.

9. A screen mirroring system, characterized in that, The screen mirroring system includes a first electronic device, a second electronic device, and a third electronic device, The first electronic device is used to obtain the resource to be projected, the second electronic device is used to play the video stream corresponding to the resource to be projected, and the third electronic device is used to play the audio stream corresponding to the resource to be projected.

10. The system according to claim 9, characterized in that, The first electronic device is further used to: decompose the resource to be projected into a video stream and an audio stream; send the video stream to the second electronic device; send the audio stream to the third electronic device; The second electronic device is used to play the video corresponding to the resource to be projected; The third electronic device is used to play the audio corresponding to the resource to be projected.

11. The system according to claim 9 or 10, wherein the second electronic device is further configured to mark a timestamp on each frame of the video stream and send the video stream with timestamps to the first electronic device; the third electronic device is further configured to mark a timestamp on each frame of the audio stream and send the audio stream with timestamps to the first electronic device; the first electronic device is further configured to synchronize the audio corresponding to the resource to be broadcast and the video corresponding to the resource to be broadcast based on the video stream with timestamps and the audio stream with timestamps, so that the video corresponding to the resource to be broadcast played on the second electronic device and the audio corresponding to the resource to be broadcast played on the third electronic device are synchronized.

12. The system according to any one of claims 9-11, wherein the first electronic device is further configured to: receive control information sent by the second electronic device; control the playback of the audio corresponding to the resource to be broadcast on the third electronic device based on the control information sent by the second electronic device.

13. The system according to any one of claims 9-11, wherein the first electronic device is further configured to: receive control information sent by the third electronic device; control the playback of the video corresponding to the resource to be broadcast on the second electronic device based on the control information sent by the third electronic device.

14. The system according to claim 12 or 13, characterized in that, The control information includes at least one of the following: identification information of the second electronic device, identification information of the third electronic device, volume information, video track identification information, or audio track identification information.

15. The system according to any one of claims 9-14, characterized in that, The audio stream includes a plurality of different sub-audio streams, and the third electronic device includes a plurality of electronic devices, and the plurality of electronic devices are used to play different sub-audio streams.

16. The system according to claim 12, wherein the first electronic device is further configured to send a control message carrying the control information; the third electronic device is further configured to control the playback of the audio corresponding to the resource to be broadcast on the three electronic devices according to the control message.

17. The system according to claim 13, wherein the first electronic device is further configured to send a control message carrying the control information; the second electronic device is further configured to control the playback of the video corresponding to the resource to be broadcast on the two electronic devices according to the control message.

18. An electronic device, characterized in that, Comprising: a memory and a processor, the memory is used to store a computer program; the processor is configured to execute the method according to any one of claims 1-8 when calling the computer program.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by the processor, implements the method according to any one of claims 1-8.

20. A computer program product, characterized in that, When the computer program product runs on an electronic device, the electronic device is caused to execute the method according to any one of claims 1-8.

Images