Screen projection method, terminal device, screen projection system and storage medium

Abstract

The application provides a screen projection method, a terminal device, a screen projection system and a storage medium. The terminal device stores a screen projection framework. The method is connected with multiple screen projection devices based on a distributed soft bus through the terminal device; the terminal device acquires first screen projection request information sent by a first screen projection device of the multiple screen projection devices; the terminal device acquires first protocol information adapted by the first screen projection device and second protocol information adapted by a second screen projection device; the terminal device inputs first media stream information in the first screen projection request information to an application layer of the screen projection framework; the application layer inputs the first media stream information to an interface layer of the screen projection framework; the interface layer calls a preset protocol library stored in a framework layer of the screen projection framework, and is used for converting the first media stream information adapted to the first protocol information into second media stream information adapted to the second protocol information; and finally, the second media stream information is sent to the second screen projection device, and screen projection is completed. The screen projection between cross operating system devices is realized.

Description

Technical Field

[0001] This application relates to the field of distributed technology applications, and in particular to a screen projection method, terminal device, screen projection system and storage medium. Background Technology

[0002] Wireless screen mirroring displays various media information or real-time operation screens from one device to another, and is widely used in scenarios such as meetings, office work, film and television entertainment, online education, and advertising. However, when the operating systems of the mirroring device and the display screen are different, existing wireless screen mirroring methods transmit screen data by downloading applications that support data exchange between specific operating systems. However, there are many different operating systems for electronic devices, and developing corresponding applications for each different operating system requires a significant investment of time and effort from developers. Furthermore, the need for users to download and use compatible applications when mirroring across operating systems can significantly degrade the user experience. Summary of the Invention

[0003] This application provides a screen mirroring method, terminal device, screen mirroring system, and storage medium, aiming to solve the problem that users need to download and use corresponding applications when mirroring across operating systems, which greatly reduces the user experience.

[0004] Firstly, this application provides a screen mirroring method applied to a terminal device, wherein the terminal device stores a screen mirroring framework, the screen mirroring framework including an application layer, an interface layer, and a framework layer, the framework layer including a preset protocol library; the screen mirroring method includes:

[0005] It is connected to multiple projection devices based on a distributed soft bus, wherein the multiple projection devices include at least a first projection device and a second projection device;

[0006] Obtain the first screen projection request information sent by the first screen projection device, wherein the first screen projection request information includes the first media stream information;

[0007] Obtain the first protocol information adapted to the first screen projection device and the second protocol information adapted to the second screen projection device;

[0008] The first media stream information is input to the application layer, the application layer inputs the first media stream information to the interface layer, the interface layer calls the preset protocol library stored in the framework layer to convert the first media stream information adapted to the first protocol information into the second media stream information adapted to the second protocol information, and the interface layer returns the second media stream information to the application layer.

[0009] The second media stream information is sent to the second projection device to complete the projection.

[0010] In some embodiments, the first projection device and the second projection device are equipped with different operating systems; before inputting the first media stream information to the application layer, the method further includes: matching the first protocol information with the second protocol information; if the matching result is a failure, confirming that the first media stream information adapted to the first protocol information needs to be converted into the second media stream information adapted to the second protocol information.

[0011] In some embodiments, after matching the first protocol information with the second protocol information, the method further includes: if the matching result is successful, sending the first media stream information to the second projection device to complete the projection.

[0012] In some embodiments, after connecting to multiple projection devices based on a distributed soft bus, the method further includes: obtaining second projection request information sent by the second projection device, the second projection request information including third media stream information; wherein, the second projection request information is used to request that the third media stream information be projected to the first projection device.

[0013] In some embodiments, the terminal device is equipped with a first-level operating system, and the second projection device is equipped with a second-level operating system, wherein the first level is higher than the second level; the method of sending the second media stream information to the second projection device to complete projection includes: sending the second media stream information to the second projection device so that the second display screen parses the second media stream information to obtain video information and audio information, and the second display screen plays the video information and audio information; or, parsing the second media stream information to obtain the video information and audio information, sending the video information and audio information to the second projection device, and the second projection device plays the video information and audio information.

[0014] In some embodiments, when the second screen projection device is equipped with a third-level operating system, the third level is lower than the second level; the method of sending the second media stream information to the second screen projection device to complete screen projection includes: parsing the second media stream information to obtain the video information and audio information, sending the video information and audio information to the display screen, and the second display screen playing the video information and audio information.

[0015] In some embodiments, after sending the second media stream information to the second projection device and completing the projection, the method includes: obtaining first time information of the first projection device and second time information of the second projection device; correcting the second time information according to the first time information to obtain correction information; sending the correction information to the second time information, and the second projection device correcting the clock of the second projection device according to the correction information.

[0016] In some embodiments, the terminal device stores a preset projection device information database; the connection with multiple projection devices based on a distributed soft bus includes: when the distance to the projection device is detected to be less than a preset distance, obtaining the identification information of the projection device; verifying the identification information based on the preset projection device information database, and connecting to the projection device via a distributed soft bus after successful verification.

[0017] Secondly, embodiments of this application provide a terminal device, which further includes a processor, a memory, and a computer program stored in the memory and executable by the processor. The memory stores a strategy model, and when the computer program is executed by the processor, it implements the steps of the screen projection method provided in any embodiment of this application.

[0018] Thirdly, embodiments of this application provide a screen projection system, including a terminal device and multiple screen projection devices. The terminal device and the multiple screen projection devices are connected via a distributed soft bus. The terminal device stores a projection framework, which includes an application layer, an interface layer, and a framework layer. The multiple screen projection devices include at least a first screen projection device and a second screen projection device. The terminal device is used to implement the steps of the screen projection method provided in any embodiment of this application.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps of the screen projection method provided in any embodiment of this application.

[0020] This application provides a screen mirroring method, terminal device, screen mirroring system, and storage medium. The provided terminal device stores a screen mirroring framework, which includes an application layer, an interface layer, and a framework layer. The framework layer includes a preset protocol library. The provided screen mirroring method connects the terminal device to multiple screen mirroring devices via a distributed soft bus. It obtains a first screen mirroring request information sent by a first screen mirroring device, and also obtains first protocol information and second protocol information adapted to the first and second screen mirroring devices, respectively. The first media stream information in the first screen mirroring request information is input to the application layer of the screen mirroring framework. The application layer inputs the first media stream information to the interface layer. The interface layer calls the preset protocol library stored in the framework layer to convert the first media stream information into second media stream information adapted to the second protocol information. The second media stream information is then sent to the second screen mirroring device to complete the screen mirroring. The screen mirroring method provided in this application can convert the first media stream information into second media stream information adapted to the target device based on the screen mirroring framework stored in the terminal device, completing screen mirroring between devices with different operating systems without the need to install a supporting application, thus improving the user experience. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic block diagram of the distributed soft bus provided in the embodiments of this application;

[0023] Figure 2 This is a schematic block diagram of a screen projection system provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the structure of a broadcasting frame provided in an embodiment of this application;

[0025] Figure 4 This is a schematic flowchart illustrating the steps of a screen mirroring method provided in an embodiment of this application;

[0026] Figure 5 This application provides a schematic flowchart of the steps of a distributed soft bus connection method.

[0027] Figure 6 This is a schematic flowchart illustrating the steps of another screen mirroring method provided in this application embodiment;

[0028] Figure 7 This is a schematic flowchart illustrating the steps of another screen mirroring method provided in this application embodiment;

[0029] Figure 8 This is a schematic flowchart illustrating the steps of another screen mirroring method provided in this application embodiment;

[0030] Figure 9 This is a schematic flowchart illustrating the steps of a clock correction method provided in an embodiment of this application;

[0031] Figure 10 This is a schematic block diagram of a terminal device provided in an embodiment of this application.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0035] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0036] It should be understood that, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. For example, "first projection device" and "second projection device" are only used to distinguish different projection devices and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" do not necessarily imply that they are different.

[0037] It should also be understood that the term "and / or" as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0038] To facilitate understanding of the embodiments of this application, some terms involved in the embodiments of this application will be briefly explained below.

[0039] 1. Distributed soft bus: such as Figure 1 As shown, Figure 1 This is a schematic block diagram of the distributed soft bus provided in the embodiments of this application. The distributed soft bus is a virtual, "invisible" bus. It can connect all system devices developed based on the KaihongOS system within the same local area network, and has the characteristics of self-discovery, self-organizing network, high bandwidth and low latency. In addition to connecting hardware devices using the same network protocol, the soft bus technology also supports networking heterogeneous networks with different protocols.

[0040] The open-source HarmonyOS is a brand-new system designed for the era of the Internet of Everything. Building upon the capabilities of traditional single-device systems, the open-source HarmonyOS provides new distributed integrated capabilities to better adapt to next-generation application scenarios involving multiple devices, multiple tasks, and multiple data.

[0041] Distributed Soft Bus: Based on the hardware and software collaboration of Ethernet, Wi-Fi, and Bluetooth, the distributed soft bus provides device discovery and connectivity. After devices discover and connect to each other, the distributed soft bus manages the self-organizing network and topology relationships between devices. The distributed soft bus achieves unified distributed communication management capabilities among near-field devices, providing link-independent device discovery, connection, networking, and transmission capabilities. Its main functions are as follows:

[0042] 1) Discovery and Connection: Provides device discovery and connection capabilities based on communication methods such as WIFI and Bluetooth.

[0043] 2) Device networking: Provides unified device networking and topology management capabilities, and provides information on networked devices for data transmission.

[0044] 3) Data transmission: Provides data transmission channels and supports message and byte data transmission capabilities.

[0045] 2. Media Streaming Information: With the widespread adoption of the Internet, the information transmitted online is no longer limited to text and graphics. The dissemination of audio and video has brought new enjoyment to internet users. In this context, traditional film and television media from around the world have joined the Internet arena, expanding their dissemination methods.

[0046] Given the limited bandwidth and congested dial-up networks of the Internet, streaming media is the best solution for transmitting video and audio over narrowband networks. This streaming technology can provide services such as video-on-demand, audio-on-demand, MTV playback, concert playback, music appreciation, and multimedia advertising. If conditions permit, it can also provide online streaming and live television program broadcasts.

[0047] With the development of the Internet, streaming media has become increasingly popular. Streaming media refers to audio, video, or multimedia files transmitted over a network. Streaming media does not download the entire file before playback; the data stream is transmitted and played in real time, with only a slight initial delay. When a streaming media file is transmitted to a user's computer, a portion of its content is already stored in memory before playback. The technology that uses streaming transmission to enable the playback of streaming media over the Internet is called media streaming.

[0048] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0049] Current cross-operating system screen mirroring solutions primarily rely on applications to invoke screen mirroring protocols supported by different operating systems. Examples include Apple's AirPlay screen mirroring, the Wi-Fi Alliance's Miracast protocol, and DLNA push protocols from other companies. The software then handles the display of the various screen mirroring data. Screen mirroring mainly involves the following processes: device discovery; connection establishment; and transmission of screen mirroring data, including media data and control panels.

[0050] Existing cross-platform device mirroring solutions require users to install corresponding software applications on different devices. However, third-party software is not a built-in function of the operating system of the device being used, which can easily affect the user experience.

[0051] At the developer level, software needs to be adapted to different operating systems. If the quality of the operating systems to be adapted varies, such as if the software development kits (SDKs) provided by the operating system are incomplete, then developers need to spend more effort developing the software, and may even need to prioritize adapting to special hardware before developing the software.

[0052] Furthermore, existing screen mirroring solutions for cross-operating system devices mainly rely on Wi-Fi or Bluetooth protocols for device discovery and connection establishment. For example, traditional Bluetooth connections require enabling Bluetooth discovery, searching for devices, and pairing, which makes the device connection process relatively slow.

[0053] To resolve the above issues, please refer to Figure 2 , Figure 2 This is a schematic block diagram of a screen projection system 10 provided in an embodiment of this application. Figure 2The provided screen projection system 10 includes a terminal device 11 and multiple screen projection devices 12. The terminal device 11 and the multiple screen projection devices 12 are connected via a distributed soft bus. The terminal device 11 stores a projection framework, which includes an application layer, an interface layer, and a framework layer. The multiple screen projection devices 12 include at least a first screen projection device 121 and a second screen projection device 122. The terminal device 11 is used to implement the steps of the screen projection method provided in any embodiment of this application.

[0054] Specifically, the terminal device 11 and the projection device 12 are connected via a distributed soft bus, enabling the terminal device 11 to perform high-speed and stable self-discovery, connection, and data transmission with the projection device 12. The terminal device 11 self-discovers the projection device 12 via the distributed soft bus, thereby constructing a heterogeneous network and resolving the issue of interaction between devices using different protocols. When the terminal device 11 and the projection device 12 are close together, the projection device 12 will discover the terminal device 11 and automatically form a network, performing necessary information exchanges (such as device identification, supported communication media Wi-Fi, Bluetooth, or Ethernet).

[0055] In some embodiments, the terminal device 11 is equipped with a first-level operating system. The operating system can be any operating system, such as HarmonyOS or KaihongOS. This application uses KaihongOS as an example for description.

[0056] It should be noted that in some embodiments, the screen projection device 12 is equipped with KaihongOS. In some embodiments, the screen projection device 12 is not equipped with KaihongOS. Through the screen projection system 10 provided in this application embodiment, the screen projection method provided in this application embodiment can be implemented whether the screen projection device 12 is equipped with or not equipped with KaihongOS. Therefore, this application embodiment does not limit the type of operating system equipped on the screen projection device 12.

[0057] It should be noted that KaihongOS divides devices into 6 levels, from L0 to L5, with L5 being the highest level and L0 being the lowest level, i.e., a lightweight system. The level in KaihongOS running on terminal device 11 is higher than or equal to the level in KaihongOS running on screen projection device 12, which enables the constructed screen projection system 10 to implement the screen projection method provided in any embodiment of this application.

[0058] For example, terminal device 11 can be a laptop computer, a server, or an OpenHarmony controller. Terminal device 11 only needs to support KaihongOS to implement the screen projection method provided in this application embodiment. Therefore, this application embodiment does not limit the type of terminal device 11.

[0059] It should be noted that, as Figure 3 As shown, Figure 3 This is a schematic diagram of a projection framework provided in an embodiment of this application. The projection framework includes an application layer, an interface layer, and a framework layer. The framework layer includes a preset protocol library. The projection framework is a framework for accessing and sending streaming media protocols and forwarding streaming media data provided by the multimedia subsystem within KaihongOS. It has capabilities such as screen projection, media data transfer, mixing and sending, window playback, media data pulling, and pushing, and can meet the media data transfer needs between KaihongOS devices and non-KaihongOS devices.

[0060] In some embodiments, the terminal device 11 and the projection device 12 are connected via the MQTT protocol. In some embodiments, the terminal device 11 and the projection device 12 are connected via the Bluetooth protocol. In some embodiments, the terminal device 11 and the projection device 12 are connected via the WIFI protocol. Since the terminal device 11 and the projection device 12 only need to connect via a connection protocol adapted to the projection device 12 to complete cross-operating system data interaction, this application embodiment does not limit the connection protocol type between the terminal device 11 and the projection device 12.

[0061] This application provides a screen mirroring system, which includes a terminal device and a screen mirroring device. The terminal device and the screen mirroring device are connected via a distributed soft bus, and the terminal device stores a screen mirroring framework. Thus, the screen mirroring system provided by this application can achieve screen mirroring between devices with different operating systems without the need to install a supporting application.

[0062] Please see Figure 4 , Figure 4 This is a schematic flowchart illustrating the steps of a screen mirroring method provided in an embodiment of this application. The provided screen mirroring method can be applied to the screen mirroring system provided in any embodiment of this application.

[0063] like Figure 4 As shown, the provided screen projection method includes steps S101 to S104.

[0064] S101. Connects to multiple screen projection devices based on a distributed soft bus, wherein the multiple screen projection devices include at least a first screen projection device and a second screen projection device.

[0065] Specifically, after a terminal device and multiple projection devices achieve self-discovery based on a distributed soft bus, a heterogeneous network can be built based on the distributed soft bus to solve the problem of interaction between devices using different protocols. When a terminal device and a projection device are close together, the projection device will discover the terminal device and automatically form a network to perform some necessary information exchanges (such as device identification, supported communication media such as Wi-Fi, Bluetooth, or Ethernet).

[0066] In some embodiments, the terminal device stores a preset database of screen projection device information. Please refer to... Figure 5 , Figure 5 This is a schematic flowchart illustrating the steps of a distributed soft bus connection method provided in an embodiment of this application.

[0067] like Figure 5 As shown, the provided distributed soft bus connection method includes steps S101a to S101b.

[0068] S101a. When the distance to the projection device is detected to be less than the preset distance, obtain the identification information of the projection device.

[0069] S101b. Verify the identification information based on the preset projection device information database, and connect to the projection device via a distributed soft bus after successful verification.

[0070] When the distance between the terminal device and the projection device is less than the preset distance, for example, when the preset distance is 10m, the distance between the terminal device and the projection device is 5m. The projection device will detect the terminal device and automatically form a network, thereby enabling information exchange between the terminal device and the projection device.

[0071] It should be noted that the first and second screen-casting devices can run the same operating system or different operating systems. The first screen-casting device generates a first screen-casting request based on the first media stream information to be cast, and casts the first media stream information to the second screen-casting device based on the provided screen-casting system. Therefore, this application embodiment does not limit the type of operating system run on the first and second screen-casting devices.

[0072] S102. Obtain the first screen projection request information sent by the first screen projection device, the first screen projection request information including the first media stream information.

[0073] Specifically, when a terminal device receives a first screen-sharing request from a first screen-sharing device, it is considered that any of the multiple screen-sharing devices has initiated a screen-sharing request to project the first media stream information to any of the second screen-sharing devices. Since the first and second screen-sharing devices run different operating systems, directly projecting the first media stream information to the second screen-sharing device would cause the second screen-sharing device to fail to recognize the first media stream information, resulting in screen-sharing failure. Therefore, the terminal device needs to process the first media stream information before sending it to the second screen-sharing device for projection, thus enabling screen-sharing between devices with different operating systems.

[0074] In some embodiments, after connecting to multiple projection devices via a distributed soft bus, the provided method further includes: obtaining second projection request information sent by a second projection device, the second projection request information including third media stream information. The second projection request information is used to request that the third media stream information be projected to the first projection device.

[0075] In the screen mirroring method provided in this application embodiment, the first screen mirroring device can act as a screen mirroring requesting end, sending a first screen mirroring request message to request screen mirroring the first media stream information to the second screen mirroring device, and can also act as a receiving end, receiving the second screen mirroring request message sent by the second screen mirroring device. The terminal device can then process the third media stream information in the second screen mirroring request message and mirror it to the first screen mirroring device. This further enhances the flexibility of the screen mirroring method provided in this application.

[0076] S103. Obtain the first protocol information adapted to the first screen projection device and the second protocol information adapted to the second screen projection device.

[0077] Specifically, the terminal device obtains the first protocol information and the second protocol information adapted to the first projection device at the input end and the second projection device at the sending end. By comparing the protocol information, it can be ensured whether the operating systems of the first projection device and the second projection device are the same, and different processing strategies are adopted to process the first media stream information to complete the projection when the first protocol information and the second protocol information are the same or different.

[0078] For example, the first protocol information and the second protocol information can be obtained by the terminal device through establishing a distributed soft bus connection with the first projection device and the second projection device, or by the terminal device sending protocol request information to the first projection device and the second projection device respectively, and receiving the protocol information returned by the first projection device and the second projection device. This application embodiment does not limit the method of obtaining the first protocol information and the second protocol information.

[0079] S104. Input the first media stream information to the application layer. The application layer inputs the first media stream information to the interface layer. The interface layer calls the preset protocol library stored in the framework layer to convert the first media stream information adapted to the first protocol information into the second media stream information adapted to the second protocol information. The interface layer returns the second media stream information to the application layer.

[0080] Specifically, please refer to the following: Figure 3 and Figure 4 After the terminal device obtains the first media stream information to be projected, it inputs the first media stream information to the application layer. The application layer then transmits the first media stream information to the interface layer. The interface layer calls the preset protocol library stored in the framework layer, and then converts the first media stream information into second media stream information adapted to the second protocol information, thus completing the processing of the first media stream information. This enables data conversion between devices based on different operating systems.

[0081] S105. Send the second media stream information to the second projection device to complete the projection.

[0082] Specifically, the converted second media stream information is sent to the second projection device, which then completes the projection by parsing the second media stream information. This enables projection between devices operating systems without the need to install any supporting applications, thus improving the user experience.

[0083] In some embodiments, the terminal device is equipped with a first-level operating system, and the second screen-projecting device is equipped with a second-level operating system, where the first-level system is superior to the second-level system. Please refer to... Figure 6 , Figure 6 This is a schematic flowchart illustrating the steps of another screen mirroring method provided in an embodiment of this application. Figure 4 The methods provided are different, Figure 6 The provided screen mirroring method also includes step S105a.

[0084] S105a. The second media stream information is sent to the second projection device so that the second display screen can parse the second media stream information to obtain video and audio information, and the second display screen plays the video and audio information.

[0085] When both the terminal device and the second projection device are equipped with KaihongOS, if the level of KaihongOS on the terminal device is higher than that on the second projection device, for example, if the level of KaihongOS on the terminal device is L2 and the level of KaihongOS on the second projection device is L1, then the second projection device is a lightweight device that can parse the second media stream information to obtain video and audio information for playback.

[0086] In some embodiments, the terminal device is equipped with a first-level operating system, and the second screen-projecting device is equipped with a second-level operating system, where the first-level system is superior to the second-level system. Please refer to... Figure 7 , Figure 7 This is a schematic flowchart illustrating the steps of another screen mirroring method provided in an embodiment of this application. Figure 4 The methods provided are different, Figure 7 The provided screen mirroring method also includes step S105b.

[0087] S105b. Parse the second media stream information to obtain video and audio information, and send the video and audio information to the second projection device. The second projection device plays the video and audio information.

[0088] When both the terminal device and the second projection device are devices running KaihongOS, the second projection device can also receive the video and audio information parsed by the terminal device for projection.

[0089] For example, when the second projection device is equipped with a third-level operating system, the third level is lower than the second level; the second media stream information is sent to the second projection device to complete the projection, the method includes: parsing the second media stream information to obtain video information and audio information, sending the video information and audio information to the display screen, and the second display screen playing the video information and audio information.

[0090] When the KaihongOS on the second screen mirroring device is at the lowest level, namely L0, the second screen mirroring device is a lightweight display device. Due to the weak performance of lightweight display devices, it generally does not have decoding capabilities and needs to receive video and audio information parsed by the terminal device for screen mirroring.

[0091] In some embodiments, please refer to Figure 8 , Figure 8 This is a schematic flowchart illustrating the steps of another screen mirroring method provided in an embodiment of this application. Figure 4 The methods provided are different, Figure 8 The provided screen mirroring method also includes step S201.

[0092] S201. Match the first protocol information with the second protocol information. If the matching result is a failure, confirm that the first media stream information adapted to the first protocol information needs to be converted into the second media stream information adapted to the second protocol information.

[0093] Before inputting the first media stream information into the projection framework on the terminal device, it is first confirmed whether the first projection device and the second projection device support the same protocol. When the protocols adapted to the first projection device and the second projection device are different, the matching result is a failure, and it is confirmed that the first media stream information adapted to the first protocol information needs to be converted into the second media stream information adapted to the second protocol information.

[0094] For example, after matching the first protocol information with the second protocol information, the method further includes: if the matching result is successful, sending the second media stream information to the second projection device to complete the projection. When the first projection device and the second projection device support the same protocol, there is no need to process the first media stream information; simply sending it to the second projection device is sufficient to complete the projection.

[0095] It should be noted that when the first and second screen-casting devices are equipped with different operating systems, if they support the same connection protocol, the first media stream information can be directly sent to the second screen-casting device to complete the screen casting.

[0096] In some embodiments, after sending the second media stream information to the second projection device and completing the projection, please refer to... Figure 9 , Figure 9 This is a schematic flowchart illustrating the steps of a clock correction method provided in an embodiment of this application.

[0097] like Figure 9 As shown, the provided clock correction method includes steps S301 to S303.

[0098] S301. Obtain the first time information of the first screen projection device and the second time information of the second screen projection device.

[0099] S302. Correct the second time information based on the first time information to obtain correction information.

[0100] S303. The calibration information is sent to the second time information, and the second projection device calibrates its clock according to the calibration information.

[0101] The terminal device uses the clock of the first projection device as a reference clock and can correct the time information of each second projection device based on distributed soft clock technology, precisely controlling the synchronization between the first and second projection devices, achieving a distributed soft clock accuracy of 1ms. Furthermore, based on the distributed soft clock algorithm and its collaboration with the transport layer, the distributed soft clock accuracy can reach 500µs. Moreover, based on the collaboration between the distributed soft clock algorithm and the transport and physical layers, the distributed soft clock accuracy is less than 300µs. This allows the second projection device to render and play the second media stream information sent by the terminal device based on the timestamp of the video frame. Because the clocks of the first and second projection devices are synchronized, they can simultaneously display the same frame, avoiding any desynchronization issues.

[0102] This application provides a screen mirroring method. The screen mirroring method provided by this application can convert the first media stream information into the second media stream information adapted to the target device based on the projection framework stored in the terminal device. Screen mirroring between devices with different operating systems can be completed without installing a supporting application, thereby improving the user experience.

[0103] This application provides a terminal device. For example... Figure 10 As shown, Figure 10 This is a schematic block diagram of a terminal device provided in an embodiment of this application.

[0104] The terminal device may include a processor, memory, and a network interface. The processor, memory, and network interface are connected via a system bus, such as an I2C (Inter-integrated Circuit) bus.

[0105] Specifically, the processor can be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0106] Specifically, the storage device can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0107] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the controller to which the present application is applied. Specific terminal devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0108] The processor is used to run a computer program stored in the memory, and implements the corresponding steps in any of the screen projection methods provided in the embodiments of this application when executing the computer program.

[0109] The provided terminal device, as an example, is used for the following steps:

[0110] It is connected to multiple projection devices based on a distributed soft bus, and the multiple projection devices include at least a first projection device and a second projection device.

[0111] Obtain the first screen projection request information sent by the first screen projection device, wherein the first screen projection request information includes the first media stream information.

[0112] Obtain the first protocol information adapted to the first screen projection device and the second protocol information adapted to the second screen projection device.

[0113] The first media stream information is input to the application layer, the application layer inputs the first media stream information to the interface layer, the interface layer calls the preset protocol library stored in the framework layer to convert the first media stream information adapted to the first protocol information into second media stream information adapted to the second protocol information, and the interface layer returns the second media stream information to the application layer.

[0114] The second media stream information is sent to the second projection device to complete the projection.

[0115] In some embodiments, the first projection device and the second projection device are equipped with different operating systems; before inputting the first media stream information to the application layer; the method is used to: match the first protocol information with the second protocol information, and if the matching result is a failure, confirm that the first media stream information adapted to the first protocol information needs to be converted into the second media stream information adapted to the second protocol information.

[0116] In some embodiments, after matching the first protocol information with the second protocol information, the system is further configured to: if the matching result is successful, send the first media stream information to the second projection device to complete the projection.

[0117] In some embodiments, after connecting to multiple projection devices based on a distributed soft bus, the system is further configured to: obtain second projection request information sent by the second projection device, wherein the second projection request information includes third media stream information; wherein the second projection request information is used to request that the third media stream information be projected to the first projection device.

[0118] In some embodiments, the terminal device is equipped with a first-level operating system, and the second projection device is equipped with a second-level operating system, wherein the first level is higher than the second level; the step of sending the second media stream information to the second projection device to complete the projection is used to: send the second media stream information to the second projection device so that the second display screen parses the second media stream information to obtain video and audio information, and the second display screen plays the video and audio information; or, parse the second media stream information to obtain the video and audio information, send the video and audio information to the second projection device, and the second projection device plays the video and audio information.

[0119] In some embodiments, when the second screen projection device is equipped with a third-level operating system, the third level is lower than the second level; when the second screen projection device is equipped with a third-level operating system, the third level is lower than the second level, which is used to: parse the second media stream information to obtain the video information and audio information, send the video information and audio information to the display screen, and the second display screen plays the video information and audio information.

[0120] In some embodiments, after sending the second media stream information to the second projection device and completing the projection, the device is further configured to: obtain first time information of the first projection device and second time information of the second projection device; correct the second time information according to the first time information to obtain correction information; send the correction information to the second time information, and the second projection device corrects the clock of the second projection device according to the correction information.

[0121] In some embodiments, the terminal device stores a preset projection device information database; the connection with multiple projection devices based on a distributed soft bus is used to: when the distance to the projection device is detected to be less than a preset distance, obtain the identification information of the projection device; verify the identification information based on the preset projection device information database, and connect to the projection device through the distributed soft bus after successful verification.

[0122] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the computer equipment described above can be referred to the corresponding process in the aforementioned distributed screen projection method, and will not be repeated here.

[0123] This application also provides a computer-readable storage medium storing a computer program, which includes program instructions. A processor executes these program instructions to implement the screen projection method provided in the above embodiments. For example, when the computer program is loaded by a processor, it can perform the following steps:

[0124] It is connected to multiple projection devices based on a distributed soft bus, and the multiple projection devices include at least a first projection device and a second projection device.

[0125] Obtain the first screen projection request information sent by the first screen projection device, wherein the first screen projection request information includes the first media stream information.

[0126] Obtain the first protocol information adapted to the first screen projection device and the second protocol information adapted to the second screen projection device.

[0127] The first media stream information is input to the application layer, the application layer inputs the first media stream information to the interface layer, the interface layer calls the preset protocol library stored in the framework layer to convert the first media stream information adapted to the first protocol information into second media stream information adapted to the second protocol information, and the interface layer returns the second media stream information to the application layer.

[0128] The second media stream information is sent to the second projection device to complete the projection.

[0129] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0130] The computer-readable storage medium can be an internal storage unit of the computer device described in the foregoing embodiments, such as a hard disk or memory of the computer device. Alternatively, it can be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the computer device.

[0131] Since the computer program stored in the computer-readable storage medium can execute any of the screen projection methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the screen projection methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0132] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A distributed screen projection method, characterized in that, This is applied to terminal devices, which store a casting framework. The casting framework includes an application layer, an interface layer, and a framework layer. The framework layer includes a preset protocol library. The casting framework is a framework provided by the multimedia subsystem within KaihongOS for accessing and sending streaming media protocols and forwarding streaming media data. It has the capabilities of screen casting, media data transfer, mixing and sending, windowed playback, media data pulling, and pushing, meeting the media data transfer needs between KaihongOS devices and non-KaihongOS devices. The screen casting method includes: It is connected to multiple projection devices based on a distributed soft bus, wherein the multiple projection devices include at least a first projection device and a second projection device; Obtain the first screen projection request information sent by the first screen projection device, wherein the first screen projection request information includes the first media stream information; Obtain the first protocol information adapted to the first screen projection device and the second protocol information adapted to the second screen projection device; The first media stream information is input to the application layer, the application layer inputs the first media stream information to the interface layer, the interface layer calls the preset protocol library stored in the framework layer to convert the first media stream information adapted to the first protocol information into the second media stream information adapted to the second protocol information, and the interface layer returns the second media stream information to the application layer. The second media stream information is sent to the second projection device to complete projection; the first projection device and the second projection device run different operating systems; before inputting the first media stream information to the application layer, the method further includes: matching the first protocol information with the second protocol information; if the matching result is a failure, confirming that the first media stream information adapted to the first protocol information needs to be converted into the second media stream information adapted to the second protocol information; if the matching result is a success, the first media stream information is directly sent to the second projection device to complete projection.

2. The distributed screen projection method according to claim 1, characterized in that, After connecting to multiple projection devices based on a distributed soft bus, the method further includes: Obtain second screen projection request information sent by the second screen projection device, the second screen projection request information including third media stream information; wherein, the second screen projection request information is used to request that the third media stream information be projected to the first screen projection device.

3. The distributed screen projection method according to claim 1, characterized in that, The terminal device is equipped with a first-level operating system, and the second screen projection device is equipped with a second-level operating system, wherein the first level is higher than the second level; The method of sending the second media stream information to the second projection device to complete projection includes: The second media stream information is sent to the second projection device, so that the second projection device can parse the second media stream information to obtain video and audio information, and the second projection device can play the video and audio information. Alternatively, the video and audio information can be obtained by parsing the second media stream information, and the video and audio information can be sent to the second projection device, whereby the second projection device plays the video and audio information.

4. The distributed screen projection method according to claim 3, characterized in that, When the second screen-casting device is equipped with a third-level operating system, the third level is lower than the second level; the method of sending the second media stream information to the second screen-casting device to complete screen casting includes: The second media stream information is parsed to obtain the video information and audio information, and the video information and audio information are sent to the projection device, whereby the second projection device plays the video information and audio information.

5. The distributed screen projection method according to claim 1, characterized in that, After sending the second media stream information to the second projection device and completing the projection, the method includes: Obtain the first time information of the first projection device and the second time information of the second projection device; The second time information is corrected based on the first time information to obtain correction information; The correction information is sent to the second projection device, and the second projection device corrects its clock according to the correction information.

6. The distributed screen projection method according to claim 1, characterized in that, The terminal device stores a preset screen projection device information database; the connection to multiple screen projection devices based on a distributed soft bus includes: When the distance to the projection device is detected to be less than a preset distance, the identification information of the projection device is obtained; The identification information is verified based on the preset projection device information database. After successful verification, the device is connected to the projection device via a distributed soft bus.

7. A terminal device, characterized in that, The terminal device further includes a processor, a memory, and a computer program stored in the memory and executable by the processor. The memory stores a strategy model, and the computer program, when executed by the processor, implements the steps of the screen projection method as described in any one of claims 1-6.

8. A screen projection system, characterized in that, The device includes a terminal device and multiple projection devices. The terminal device and the multiple projection devices are connected via a distributed soft bus. The terminal device stores a projection framework, which includes an application layer, an interface layer, and a framework layer. The multiple projection devices include at least a first projection device and a second projection device. The terminal device is used to implement the steps of the projection method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the steps of the screen projection method as described in any one of claims 1-6.

Images