Screen projection method and device, client device, medium and computer program product

Abstract

The embodiment discloses a screen projection method and device, client equipment, a medium and a computer program product, the screen projection method is applied to the client equipment, and the method comprises the following steps: establishing a plurality of connection links with target display equipment at the same time; wherein the plurality of connection links comprise a main link and at least one standby link; screen projection is realized on the target display equipment through the main link; and switching the main link to the standby link under the condition that the communication of the main link is abnormal, and realizing screen projection on the target display equipment through the standby link. According to the method provided by the embodiment of the invention, the normal operation of the screen projection service can be ensured by switching the connection link, and the screen projection quality can be ensured.

Description

Technical Field

[0001] The present application belongs to the field of wireless communication technology, and in particular relates to a screen projection method, apparatus, client device, medium and computer program product. Background Art

[0002] The method of establishing a connection link through a LAN network to perform screen projection services is relatively simple. If the LAN network environment is affected and cannot carry normal screen projection services, some user screen projection experience effects will be sacrificed to maintain the normal operation of the screen projection service. Summary of the invention

[0003] Embodiments of the present application provide a screen projection method, apparatus, client device, medium, and computer program product.

[0004] The present application provides a screen projection method, which is applied to a client device, and the method includes:

[0005] Establishing multiple connection links with the target display device simultaneously; wherein the multiple connection links include a main link and at least one backup link;

[0006] Implementing screen projection on the target display device through the main link;

[0007] When there is a communication anomaly in the main link, the main link is switched to the backup link, and screen projection is implemented on the target display device through the backup link.

[0008] In some embodiments, the method also includes: configuring a heartbeat link and a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; switching the main link to the backup link includes: stopping the screen projection service, disconnecting the heartbeat link and the command data link; switching the main link to the backup link, and configuring the heartbeat link and the command data link based on the backup link.

[0009] It can be seen that by configuring a heartbeat link for the connection link of the screen projection service, the communication status of the current connection link can be obtained, which is conducive to ensuring the quality of screen projection; by configuring a command data link for the connection link of the screen projection service, it can be guaranteed that users can send commands based on the current connection link of the screen projection, which is conducive to improving the user's screen projection experience.

[0010] In some embodiments, the method further includes: in the event that the main link fails to switch to the backup link, rebuilding each connection link among the multiple connection links.

[0011] In some embodiments, the method also includes: configuring a heartbeat link and a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; and in the event that the main link fails to switch to the backup link, rebuilding each of the multiple connection links, including: in the event that the main link fails to switch to the backup link, stopping the screen projection service, disconnecting the heartbeat link and the command data link; and rebuilding each of the multiple connection links.

[0012] In some embodiments, the method further includes: when any connection link among the multiple connection links is successfully established, implementing screen projection on the target display device through the successfully established link.

[0013] It can be seen that the method of this embodiment is conducive to quickly restoring the screen projection service in the case of communication abnormalities and enhancing the user's screen projection experience.

[0014] In some embodiments, the method further includes: configuring a heartbeat link between the client device and the target display device; when there is a communication abnormality in the main link, before switching the main link to the backup link, the method further includes: determining the communication status of the main link based on the link status of the heartbeat link.

[0015] In some embodiments, when there is a communication anomaly in the main link, switching the main link to the backup link includes: when the packet loss rate of the heartbeat link is greater than a first threshold and / or the heartbeat link is disconnected, switching the main link to the backup link.

[0016] It can be seen that by configuring the heartbeat link, it is helpful to obtain the communication status of the connection link of the current projection screen, and it is helpful to switch to the backup link in time in the event of communication abnormalities to ensure the quality of projection screen.

[0017] In some embodiments, the method further includes: configuring a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; and in the event that there is a communication anomaly in the main link, switching the main link to the backup link includes: in the event that the command data link is disconnected, switching the main link to the backup link.

[0018] It can be seen that the connection status of the command data link is helpful for obtaining the communication status of the connection link of the current screen projection, which is helpful for timely switching to the backup link in the event of communication abnormalities to ensure the quality of screen projection.

[0019] In some embodiments, the method also includes: obtaining the air interface resource occupancy rate and / or channel utilization rate of the main link; and switching the main link to the backup link when there is a communication abnormality in the main link, including: switching the main link to the backup link when the air interface resource occupancy rate is greater than a second threshold and / or the channel utilization rate is greater than a third threshold.

[0020] It can be seen that through the method of this embodiment, it is possible to dynamically switch the connection link of the screen projection in coordination with wireless technologies such as air interface resource occupancy and channel utilization, which is beneficial to ensuring the quality of the screen projection and further beneficial to maintaining the stability of the current network.

[0021] In some embodiments, when there is a communication anomaly in the main link, before switching the main link to the backup link, the method also includes: obtaining the wireless access point (Access Point, AP) deployment point information of the network between the client device and the target display device; and obtaining the communication status of the main link based on the AP deployment point information.

[0022] In some embodiments, one of the multiple connection links is a direct link established based on a WiFi Direct signal, or one of the multiple connection links is a direct link established based on a Bluetooth connection.

[0023] It can be seen that on the basis of the connection link established based on the LAN network, a direct connection link is established based on the WiFi Direct signal and / or Bluetooth connection, which realizes the diversity of the screen projection connection link compared with the method of establishing a connection link only through the LAN network or based on the WiFi signal emitted by the target display device in the related technology. On the basis of establishing a connection link through the LAN network or establishing a connection link based on the WiFi signal emitted by the target display device, multiple connection links that can coexist at the same time can be established through WiFi Direct signals or Bluetooth connections, which is beneficial to the direct connection link established through WiFi Direct signals or Bluetooth connections not being affected in the case of abnormal communication in the LAN network, thereby improving the screen projection experience.

[0024] The embodiment of the present application also provides a screen projection device, which is applied to a client device, and the device includes:

[0025] A connection module, used to simultaneously establish multiple connection links with a target display device; wherein the multiple connection links include a main link and at least one backup link;

[0026] A processing module is used to realize screen projection on the target display device through the main link; when there is a communication anomaly in the main link, the main link is switched to the backup link, and screen projection is realized on the target display device through the backup link.

[0027] An embodiment of the present application provides a client device, the client device comprising a communication interface, a processor, and a memory for storing a computer program that can be run on the processor; wherein:

[0028] The communication interface is used to exchange information with the target display device;

[0029] The processor is used to run the computer program to execute any of the above-mentioned screen projection methods.

[0030] An embodiment of the present application provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned screen projection methods is implemented.

[0031] An embodiment of the present application provides a computer program product, including a computer program, which implements any of the above-mentioned screen projection methods when executed by a processor.

[0032] The embodiments of the present application provide a screen projection method, apparatus, client device, medium and computer program product, which simultaneously establish a main link and at least one backup link in the screen projection connection link, so that when there is a communication anomaly in the main link, it can directly switch to the established backup link for screen projection, without the need to re-establish the connection link during the switching process. Therefore, the screen projection method provided in the embodiments of the present application realizes the diversity of screen projection connection links, which helps to improve the user screen projection experience and ensure the quality of screen projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of a screen projection method provided in an embodiment of the present application;

[0034] Figure 2 A flow chart of a method for reestablishing a connection link provided in an embodiment of the present application;

[0035] Figure 3 A flow chart of a method for establishing multiple connection links provided in an embodiment of the present application;

[0036] Figure 4 A flow chart of a connection link switching method provided in an embodiment of the present application;

[0037] Figure 5 A flow chart of another connection link switching method provided in an embodiment of the present application;

[0038] Figure 6 A schematic diagram of a screen projection module of a client device provided in an embodiment of the present application;

[0039] Figure 7 A schematic diagram of the structure of a screen projection device provided in an embodiment of the present application;

[0040] Figure 8 A schematic diagram of the structure of a client device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] Multi-screen interaction and collaboration means that display devices can realize wireless multi-screen terminal intelligent interconnection with client devices such as computers and smart phones. A series of operations such as transmission, display, and reverse control of multimedia content (audio, video, pictures, etc.) can be performed on different client devices (such as different smart terminal devices based on different operating systems such as Android, MacOS, and IOS), such as tablets, mobile phones, and notebook devices. Ultimately, the displayed content can be shared on display devices at the same time, enriching the user's multimedia life.

[0042] Current methods for establishing multi-screen interaction usually include:

[0043] Method 1: The user manually confirms that the network of the client device and the target display device are in the same LAN network, and manually enters the Internet Protocol (IP) address of the target display device, or the screen projection code of the target display device, and then manually confirms the connection and projects the screen. In this way, based on the original LAN network, a single LAN network can be established for the application on the client device and the target display device to project the screen.

[0044] Method 2: The target display device itself releases a mobile hotspot (Wireless Fidelity, WiFi) signal. The user manually connects to the WiFi signal of the target display device according to the step prompts of the target display device. After confirming that the client device has successfully connected to the WiFi of the target display device and the network is reachable and interoperable, the screen projection code or target IP address is entered. In this way, based on the WiFi network released by the target display device, a single direct WiFi network is established between the application on the client device and the target display device for screen projection.

[0045] When casting screens through a relatively single LAN network or through the WiFi network of the target display device through the above method, facing a complex and changeable network environment, when encountering a relatively poor network, the commonly used solutions are:

[0046] Method 1: Dynamically adjust the parameters in the screen projection process according to the quality of the network environment, including bit rate, frame rate and other screen projection parameter values, so as to adapt to the current network and continue to maintain the user's screen projection service.

[0047] Method 2: When the network has automatically and dynamically adjusted the screen projection parameters but still cannot maintain normal screen projection services, the screen projection will be automatically exited and a corresponding prompt will be given to inform the user that the network is abnormal and the current screen projection service is automatically interrupted. Please check the network environment or try again later.

[0048] The above-mentioned screen projection method and the solution in the case of poor network communication have the following problems: users use the original LAN network or the WiFi network of the target display device to perform screen projection services. This screen projection connection method is relatively simple. If the network environment itself is affected and cannot carry normal screen projection services, it is necessary to sacrifice part of the user screen projection experience to maintain the normal operation of the screen projection service. In short, the stability of single-link screen projection is poor.

[0049] In addition, the screen projection service is an application layer service. The application layer service cannot truly perceive the connectivity of the network link. It relies on simple screen projection data retransmission logic to judge and solve abnormal situations such as service delays and interruptions. This will prolong the timeline for abnormalities to occur, and is more likely to cause unbearable service interruptions and unrecoverable situations, resulting in a significant reduction in user experience and negative effects. Furthermore, for problems such as inaccurate statistics of network quality information and slow responses, the service needs to rely on the retransmission mechanism of the Transmission Control Protocol (TCP) protocol stack to perceive faults. It usually takes 5-30 seconds or even longer to trigger the service switching link. The user experience can usually clearly perceive the screen projection abnormality.

[0050] In response to the above-mentioned problems, the embodiments of the present application provide a screen projection method, apparatus, client device, medium and computer program product, which can ensure the normal operation of the screen projection service by switching the connection link.

[0051] The following is a further detailed description of the embodiments of the present application in conjunction with the accompanying drawings and examples. It should be understood that the embodiments provided herein are only used to explain the embodiments of the present application and are not intended to limit the embodiments of the present application. In addition, the embodiments provided below are partial embodiments for implementing the present application, rather than providing all embodiments for implementing the present application. In the absence of conflict, the technical solutions recorded in the embodiments of the present application can be implemented in any combination.

[0052] It should be noted that, in the embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or device including a series of elements includes not only the elements explicitly recorded, but also includes other elements not explicitly listed, or also includes elements inherent to the implementation of the method or device. In the absence of further restrictions, an element defined by the sentence "includes..." does not exclude the presence of other related elements (such as steps in the method or units in the device, such as a unit in the device may be a part of a circuit, a part of a processor, a part of a program or software, etc.) in the method or device including the element.

[0053] The screen projection method provided in the embodiment of the present application includes a series of steps, but the screen projection method provided in the embodiment of the present application is not limited to the recorded steps. Similarly, the screen projection device provided in the embodiment of the present application includes a series of modules, but the device provided in the embodiment of the present application is not limited to including the modules explicitly recorded, and may also include modules that need to be set for obtaining relevant information or performing processing based on information.

[0054] The embodiment of the present application provides a screen projection method, which is applied to a client device, which may be a terminal device, such as a tablet computer, a mobile phone, a laptop computer, etc. Figure 1 As shown, Figure 1 The screen projection methods shown include:

[0055] Step 101: Establish multiple connection links with a target display device simultaneously; wherein the multiple connection links include a main link and at least one backup link.

[0056] In the scenario of screen projection based on the client device and the target display device, the screen projection is usually achieved by establishing a connection link based on the local area network or the WiFi network of the target display device. In the actual screen projection process, the client device is usually a single network card device, such as the tablet computer, mobile phone, laptop device, etc. mentioned above. Therefore, in the case where the client device is a single network card device, based on the method for establishing the screen projection connection link in the relevant technology, the client device can usually only support one communication mode to establish a connection link with the target display device, that is, the client device can only establish a connection link with the target display device based on the local area network, or can only establish a connection link based on the WiFi network of the target display device. Therefore, in the case of a communication anomaly in the current connection link used for screen projection, the current connection link between the target display device and the client device can only be disconnected first, and then switched to the backup connection link. For example, assuming that the current client device and the target display device establish a connection link based on the local area network for screen projection, when the current connection link communication is abnormal, it is necessary to first disconnect the connection between the client device and the target display device, and then establish a connection link through the client device based on the WiFi network of the target display device. When the connection link is successfully established, continue to project the screen.

[0057] In order to avoid the situation in which the related technology needs to re-establish the connection link when the communication is abnormal, the embodiment of the present application provides that when the client device establishes a connection link with the target display device, multiple connection links are established at the same time. In the case where the client device is a single network card device, among the multiple connection links established by the client device with the target display device, the multiple connection links may include a connection link established based on a local area network, and a connection link established based on a communication protocol different from the local area network; or, the multiple connection links may include a connection link established based on a local area network, and a connection link established without occupying the network card of the client device. In this way, it is possible to simultaneously establish at least two connection links between the client device and the target display device, and at least two connection links are connection links that can exist at the same time and do not affect each other.

[0058] In some embodiments, one of the multiple connection links is a direct link established based on a WiFi Direct signal, or one of the multiple connection links is a direct link established based on a Bluetooth connection.

[0059] Specifically, when the multiple connection links include at least two connection links, the at least two connection links include a connection link established based on a local area network, and a first link; the first link is a direct link established based on a WiFi Direct signal, or the first link is a direct link established based on a Bluetooth connection; or, the at least two connection links are any two of a connection link established based on a local area network, a direct link established based on a WiFi Direct signal, and a direct link established based on a Bluetooth connection; or, the multiple connection links include at least three connection links, and the at least three connection links include a connection link established based on a local area network, a direct link established based on a WiFi Direct signal, and a direct link established based on a Bluetooth connection.

[0060] Bluetooth technology is a low-cost short-range wireless connection; WiFi Direct is a technology that allows devices in a wireless network to connect to each other without going through a wireless router. Similar to Bluetooth technology, WiFi Direct allows wireless devices to interconnect in a point-to-point manner, and has greatly improved transmission speed and transmission distance compared to Bluetooth. A major advantage of WiFi Direct connection is that its connection does not require an intermediate device. It is a point-to-point connection with faster transmission speed and longer transmission distance; another major advantage of WiFi Direct connection is that it allows the network card to work in station (STA) mode and peer-to-peer (P2P) mode at the same time through multi-channel communication (MCC) time division multiplexing. Among them, STA mode is similar to a wireless terminal. The station itself does not accept wireless access, and it can connect to an access point AP. Any wireless network card can run in STA mode. In STA mode, the wireless network card sends a connection and authentication message to the hotspot. After the hotspot receives the completed authentication, it sends back a successful authentication message, and the network card is connected to the wireless network. The P2P mode is a peer-to-peer networking between WiFi devices, and does not require AP participation. Therefore, WiFi Direct direct link has a natural advantage in ensuring the best quality of network signals.

[0061] When there are multiple interoperable network paths between the client device and the target display device, the client device can successfully establish a connection with the target display device through the LAN network to form a connection link. For example, after the client device is started, it will automatically trigger a network environment scan, verify the specific custom information in the surrounding scanned network information, and obtain verification information. The device that passes the verification will be identified as a display device that can be used for screen projection. The specific custom information here can be the information of the display device obtained at the beginning of establishing the connection, the name of the display device obtained by parsing the character string, the Media Access Control Address (MAC) address and other information. Through the verified display device, the client device can obtain the ultrasonic information, connection code information, WiFi information, WiFi Direct information, Bluetooth connection information, etc. of the display device.

[0062] After the network scan is completed, the client device will organize and classify the information obtained, and display the list of discovered display devices to the user through the client display page. The user can try to establish a connection with the target display device on the client device in the following two ways: one way is that the user enters the connection code of the target display device on the client device and then clicks the connect button to initiate the connection; the other way is that the user double-clicks the display device list displayed on the client device display page, or clicks the connect button of the list item to connect.

[0063] After initiating the connection operation on the client device, the connectivity check of the LAN network can be performed first. The connection code of the target display device obtained in the network scanning phase is actually a mapping of the upstream IP of the target display device. The client device will first parse the connection code according to the preset rules between the client device and the target display device during the connection phase. After the successful resolution, the IP address of the target display device will be obtained. The client device will initiate a service request based on the IP address of the target display device. If a request reply of its IP address is received, it is considered that the network between the client device and the target display device is interoperable; if no reply request is received, it is considered that the two ends are not interoperable.

[0064] If the LAN network detection is interoperable, the client device considers that the connection with the target display device is successful. After the connection is successful, the client device will switch to the connection success page for display. In some cases, for example, when the current LAN network is faulty or the target display device connection is faulty, the LAN network detection result between the client device and the target display device is not interoperable.

[0065] After the client device attempts to establish a connection link with the target display device based on the LAN network, regardless of whether the connection link is successfully established, the client device will then attempt to establish a direct connection link with the target display device based on a WiFi Direct signal, or attempt to establish a Bluetooth connection with the target display device to establish a direct connection link.

[0066] The client device will then check whether the WiFi Direct signal of the target display device has been found. If there is no WiFi Direct signal, the WiFi Direct network connection will not be made, which is equivalent to the WiFi Direct network connection failure. If there is a WiFi Direct signal, it will try to initiate a WiFi Direct network connection. If the connection is successful, the WiFi Direct connection is considered successful, otherwise the WiFi Direct connection is considered failed.

[0067] The client device can then check whether the Bluetooth connection signal of the target display device has been found. If there is no Bluetooth connection signal, the Bluetooth connection will not be made, which is equivalent to the failure to establish the Bluetooth connection link. If there is a Bluetooth connection signal, it will try to initiate a Bluetooth connection. If the connection is successful, the Bluetooth connection is considered successful, otherwise the Bluetooth connection is considered failed.

[0068] Since WiFi Direct signals, Bluetooth connections, and LAN networks are three different communication methods, the corresponding communication protocols are different, and the required hardware support is also different. Therefore, even if the client device is a single network card device, it is possible to simultaneously establish three connection links between the client device and the target display device based on the three communication methods of WiFi Direct signals, Bluetooth connection signals, and LAN networks. In actual applications, in order to ensure the quality of screen projection, at least two communication methods can be selected from the WiFi Direct signals, Bluetooth connection signals, and LAN network communication methods to simultaneously establish at least two connection links. Since establishing a connection link based on the LAN network or the WiFi network of the target display device will occupy the client device's network card, a connection link can be established based on the LAN network or the WiFi network of the target device.

[0069] This step provides a method for establishing multiple connection links. When the connection links based on the LAN network, WiFi Direct signal, and Bluetooth connection signal are successfully established, multiple connection links are established between the client device and the target display device at the same time. Based on this step, the client device can also first try to establish a connection with the target display device based on the WiFi Direct signal or Bluetooth connection signal, and then try to connect based on the LAN network.

[0070] When the client device and the target display device support access to multiple LAN networks, for example, when the client device and the target display device are configured with multiple network cards, new connection links can be established based on different LAN networks on the basis of this step.

[0071] In some embodiments, the client device can also establish a connection link through the WiFi network of the target display device. The client device will check whether the WiFi network of the target display device has been found. If the WiFi network of the target display device exists, the client device will try to initiate a WiFi network connection, and establish a connection link after the connection is successful. The client device and the target display device successfully establish a connection link through the WiFi network of the target display device. At this time, the WiFi network is equivalent to a local area network. The difference from the above-mentioned local area network is that this is a local area network with two ends directly connected. At this time, it is also considered that the connection link between the client device and the target display device is successfully established, and the client device switches to the connection success page to display the results.

[0072] The above embodiment provides a method for establishing connection links based on different network forms. Based on the method provided in the above embodiment, after the client device establishes multiple connection links with the target display device, the client device can use the first established connection link as the main link and other connection links as backup links; the client device can also, according to preset rules, give priority to the connection link established through the local area network as the main link, and use the direct connection link or other connection links established based on the WiFi Direct signal, Bluetooth connection signal as the backup link; the client device can also use the direct connection link established based on the WiFi Direct signal or the Bluetooth connection signal as the main link, and use other links as backup links. In the event of an abnormal communication in the main link, you can switch to any one of the other connection links for screen projection. At this time, the switched connection link can be used as the main link for the current screen projection.

[0073] In this step, when the LAN networks of the client device and the target display device are interconnected, or when the client device and the target display device establish a connection link based on the WiFi network of the target display device, attempts will be made to establish a direct connection link based on the WiFi Direct signal and the Bluetooth connection signal, in order to achieve the purpose of establishing redundant backup links at the same time.

[0074] Step 102: Screen projection is implemented on the target display device through the main link.

[0075] When multiple connection links are established with the target display device, the client device displays the content to be projected on the target display device through the main link.

[0076] Step 103: When there is a communication anomaly in the main link, the main link is switched to the backup link, and screen projection is implemented on the target display device through the backup link.

[0077] In the event of communication anomalies in the main link, such as interruption of communication in the main link or exit of the current screen projection service, the client device will automatically switch the main link to the backup link and implement screen projection on the target device through the backup link.

[0078] This embodiment provides a screen projection method, which can ensure the normal operation of the screen projection service by establishing multiple connection links between the client device and the target display device, and switching the main link to the backup link to achieve screen projection when there is a communication anomaly in the main link. Since the direct link established based on the WiFi Direct signal and the Bluetooth connection will not affect the user's existing LAN network, the establishment of a direct link of the WiFi Direct signal and the Bluetooth connection signal on the basis of the LAN network or the WiFi network emitted by the target display device can provide a stable and reliable guarantee for the user's screen projection service. After an abnormality occurs in the main link, it can quickly switch to the backup link to ensure the continuous stability of the screen projection service.

[0079] In practical applications, steps 101 to 103 can be implemented based on a processor, and the processor can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor.

[0080] In some embodiments, the above method also includes: configuring a heartbeat link and a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; switching the main link to the backup link, including: stopping the screen projection service, disconnecting the heartbeat link and the command data link; switching the main link to the backup link, and configuring the heartbeat link and the command data link based on the backup link.

[0081] After the client device establishes a connection link with the target display device, a heartbeat link and a command data link will be established based on the current connection link. The heartbeat link is a link between the client device and the target display device, which is used to measure the real-time network quality between the client device and the target display device. Through the heartbeat link, heartbeat signals are sent regularly between the client device and the target display device. These heartbeat signals are used to obtain the operating status of the connection link between the client device and the target display device, thereby ensuring the activity of the connection link between the client device and the target display device and the availability of the system. The communication status of the connection link currently used for screen projection can be obtained through the establishment of the heartbeat link. For example, the client device will send heartbeat information in the form of a heartbeat packet to the target display device at a preset time interval. The heartbeat packet usually contains the status information, timestamp, etc. of the client device for verification by the target display device. After receiving the heartbeat packet, the target display device will parse and verify it to confirm the identity and status information of the client device. If the target display device successfully receives and verifies the heartbeat packet within the preset time, it is considered that the client device is in a normal working state and the connection link communication is normal.

[0082] After the client device establishes a connection link with the target display device, a command data link is also established based on the connection link. The client device interacts with the target display device through the command data link. The instructions for interaction between the client device and the target display device are all performed on the command data link, such as sending switching instructions, pause instructions, etc. to the target display device.

[0083] Compared with the connection link used for screen projection, the heartbeat link can be a connection link used solely for sending heartbeat packets. The heartbeat link can also share a link with the connection link for screen projection. In this case, the client device sends a heartbeat packet to the target display device through the current connection link for screen projection to detect the communication status of the current connection link for screen projection.

[0084] In the event of a communication anomaly on the main link, the client device will stop the current screen projection service and disconnect the heartbeat link and command data link of the main link before switching the main link to the backup link. After switching the link to the backup link, the heartbeat link and command data link will be configured on the backup link.

[0085] Since the backup link and the main link use different communication protocols and have different IP addresses, after switching the main link to the backup link, it is necessary to configure the heartbeat link and the command data link based on the port, IP address and other information of the backup link.

[0086] This embodiment provides a connection link switching method. When the connection link is configured with a heartbeat link and a command data link at the same time, the method of this embodiment can detect the communication status of the current screen projection connection link, which is beneficial to ensuring the quality of the screen projection service and improving the user experience during the screen projection process.

[0087] In some embodiments, the above method further includes: in the event that the main link fails to switch to the backup link, rebuilding each connection link among the multiple connection links.

[0088] In the case that the switch from the main link to all backup links fails, the client device will automatically reestablish the connection link. The reconstruction of the connection link in this embodiment is the same as the method in step 101 above. In the case that multiple connection links have been established at the same time based on the above embodiment, the client device can reestablish the connection link based on the multiple connection link establishment methods that have been implemented. At the same time, the client device can also try to find other connection methods with the target display device.

[0089] If the connection link reconstruction fails, the connection with the target display device is considered abnormally interrupted, and the screen projection is exited. The client device will show the user a corresponding prompt through the display interface.

[0090] In some embodiments, the above method also includes: configuring a heartbeat link and a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; in the event that the main link fails to switch to the backup link, each of the multiple connection links is rebuilt, including: in the event that the main link fails to switch to the backup link, stopping the screen projection service, disconnecting the heartbeat link and the command data link; and rebuilding each of the multiple connection links.

[0091] The method for configuring the heartbeat link and the command data link between the client device and the target display device in this embodiment is the same as that in the above embodiment. When the heartbeat link and the command data link are configured between the client device and the target display device, when rebuilding the connection link, it is necessary to first stop the screen projection service, disconnect the heartbeat link and the command link corresponding to the main link, and then rebuild each of the multiple connection links.

[0092] Based on the method provided in the above embodiment, when any connection link between the client device and the target display device is successfully established during the reconstruction process, the client device will configure the corresponding heartbeat link and instruction data link based on the successfully reconstructed connection link.

[0093] In some embodiments, the above method also includes: when any connection link among the multiple connection links is successfully established, screen projection is implemented on the target display device through the successfully established link.

[0094] During the connection link reconstruction process, the client device can also reconstruct the connection link in the order of priority of establishing the connection link at the beginning. For example, the order of establishing and reconstructing the connection link can be: first establish the connection link through the LAN network, then try to establish a direct link based on the WiFi Direct signal and the Bluetooth connection signal, and finally try to find the WiFi network emitted by the target display device, and try to establish a connection link with the target display device after finding the WiFi network emitted by the target display device. If any network link is reconstructed successfully, it will help the user continue to maintain the current screen projection service and ensure that the user's screen projection experience is not interrupted; if all reconstruction attempts of the client device fail, it will exit the current screen projection service and prompt the user through the display interface that the current communication is abnormal and the screen projection service is interrupted.

[0095] When the client device is establishing a connection link or projecting the screen, users can also view real-time network communication quality data and communication quality data within a preset historical time period, including network delay, packet loss rate, projection frame rate, bit rate, frame loss rate, etc. These data can be displayed intuitively and help users provide certain assistance when problems occur, which is conducive to faster location and repair of problems.

[0096] Based on the connection link reconstruction method given in the above embodiment, Figure 2 A flow chart of a method for reestablishing a connection link is shown. In practical applications, situations that may trigger the reconstruction of a connection link may also include any of the following situations: abnormal interruption and exit of a heartbeat link, abnormal interruption and exit of a WebSocke protocol data link, abnormal interruption and exit of a screen projection service, etc. Among them, WebSocke protocol data is a protocol for full-duplex communication on a single TCP connection. The reasons for abnormal interruption and exit of a WebSocke protocol data link may include heartbeat timeout, network problems, device errors, etc. The method for reestablishing a connection link includes:

[0097] Step 201: Determine whether there is any ongoing screen projection service.

[0098] If there is a screen projection service, execute step 202.

[0099] If there is no screen projection service, execute step 203.

[0100] Step 202: Disconnect the screen projection service.

[0101] After disconnecting the screen projection service, execute step 203.

[0102] Step 203: disconnect the heartbeat link; disconnect the command data link.

[0103] In the actual application of screen projection services, the WebSocket protocol, as a protocol for full-duplex communication on a single TCP connection, plays an important role. It allows a persistent connection to be established between the client device and the target display device, and transmits data content in real time. Although the WebSocket protocol itself does not directly act on the data link layer, it indirectly has a positive impact on the data link layer by optimizing the communication process between the transport layer and the application layer. Therefore, in this step, when the WebSocket protocol is applied in the screen projection service, the WebSocket protocol data link can also be disconnected to ensure that all connections between the client device and the target display device are disconnected, reducing communication interference between different connection links.

[0104] Step 204: Reestablish the connection link.

[0105] Based on the method given in the above embodiment, based on the different network forms of the client device and the target display device, the connection link is attempted to be reestablished, or based on the order in which the connection link was established last time, the connection links are reestablished in sequence.

[0106] Step 205: Determine whether the reconstruction is successful.

[0107] If the reconstruction is successful, step 206 is executed. If the reconstruction fails, step 209 is executed.

[0108] Here, when the client device successfully establishes any connection link among the multiple connection links, it is considered that the reconstruction is successful, and step 206 can be executed, and screen projection can be achieved on the target display device through the successfully established link.

[0109] Step 206: Configure the heartbeat link; configure the command data link.

[0110] After any connection link is successfully established, the client device configures a heartbeat link and a command data link based on the successfully established connection link.

[0111] Based on the method of step 203, after any connection link is successfully established, the client device can also configure a WebSocket protocol data link to optimize the transmission performance of the current connection link and provide protection for the screen projection service.

[0112] Step 207: Determine whether there is any ongoing screen projection service.

[0113] After rebuilding the connection link and configuring the heartbeat link and command data link, the client device determines whether there is currently an ongoing screen projection service, that is, whether the user initiates a new screen projection service based on the client device, or whether there was a screen projection service before the reconstruction. If there is an ongoing screen projection service, execute step 208; if there is no ongoing screen projection service, wait for the screen projection service.

[0114] Step 208: Establish screen projection service.

[0115] When there is an ongoing screen projection service, the client device implements screen projection on the target display device by reestablishing a successful link.

[0116] Step 209: The client terminates abnormally and exits.

[0117] If the link reconstruction fails, the connection between the client device and the target display device is considered to be abnormally interrupted, the screen projection service is exited, and a corresponding prompt is displayed to the user.

[0118] In some embodiments, the above method also includes: configuring a heartbeat link between the client device and the target display device; when there is a communication abnormality in the main link, before switching the main link to the backup link, the above method also includes: determining the communication status of the main link based on the link status of the heartbeat link.

[0119] After the connection link is established between the client device and the target display device, a heartbeat link will be established based on the connection link. The heartbeat link is a service channel used to measure the real-time network quality between the client device and the target display device. The client device calculates the packet loss during data transmission through this service channel, further determines the real-time status of network transmission, and decides whether to optimize the network or provide feedback to the user.

[0120] In some embodiments, when there is a communication anomaly in the main link, switching the main link to the backup link includes: when the packet loss rate of the heartbeat link is greater than a first threshold and / or the heartbeat link is disconnected, switching the main link to the backup link.

[0121] The packet loss rate of the heartbeat link indicates the proportion of heartbeat packets used to maintain the connection status in the network that are lost during the transmission process. When the packet loss rate of the heartbeat link is greater than the first threshold, it is considered that there is an abnormality in the current communication, or when the heartbeat link is disconnected, it can also be considered that there is an abnormality in the current communication. At this time, the main link is switched to the backup link to ensure the normal operation of the screen projection service. After switching the main link to the backup link, continue to configure the heartbeat link based on the backup link to monitor the communication status of the backup link.

[0122] In some embodiments, the above method also includes: configuring a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; when there is a communication anomaly in the main link, switching the main link to a backup link, including: when the command data link is disconnected, switching the main link to the backup link.

[0123] In some embodiments, the above method also includes: obtaining the air interface resource occupancy rate and / or channel utilization rate of the main link; when there is a communication abnormality in the main link, switching the main link to the backup link, including: when the air interface resource occupancy rate is greater than the second threshold and / or the channel utilization rate is greater than the third threshold, switching the main link to the backup link.

[0124] The air interface resource occupancy rate indicates the utilization degree of the air interface resources of the transmission system over a period of time. The air interface represents the wireless channel in the transmission system. The size of the air interface resource occupancy rate is one of the important indicators to measure the performance of the communication or data transmission system. Channel utilization can reflect the utilization degree of channel resources. In this embodiment, a link quality assessment module of the connection link can be built into the client device to detect in real time the status of the network where the connection link currently used for screen projection is located. In actual applications, by real-time detection of the air interface resource occupancy rate and / or channel utilization rate of different networks, the current network status of different forms such as LAN network and WiFi Direct network can be obtained, and based on the parameters of the air interface resource occupancy rate and / or channel utilization rate, the connection link used for the screen projection service can be dynamically adjusted.

[0125] The communication status of each of the multiple connection links is obtained in real time through the client device, and statistical analysis is performed based on various real-time data to decide whether dynamic and intelligent connection link switching is required. Link switching is performed autonomously without user intervention.

[0126] In this embodiment, the built-in link quality assessment module makes comprehensive decisions and dynamically adjusts the quality of screen projection data transmission to perform intelligent link switching. It can judge the quality of the user's current network by combining different custom quality standards and indicators, and quickly optimize the network for relatively poor network environments, effectively and quickly respond to situations that may occur during the screen projection service, such as freezes and delays that affect the user experience.

[0127] In some embodiments, when there is a communication anomaly in the main link, before switching the main link to the backup link, the above method also includes: obtaining the wireless access point AP deployment point information of the network between the client device and the target display device; and obtaining the communication status of the main link based on the AP deployment point information.

[0128] In practical applications, since AP can affect the coverage of wireless network and the quality of network connection, based on the role of AP in wireless network communication, the embodiment of the present application can also cooperate with the AP deployment point information in different connection links to obtain the local area network status, dynamically change the projection connection link, and maintain the stability of the projection service. Among them, the AP deployment point information includes but is not limited to AP location, coverage, channel allocation, antenna configuration, etc.

[0129] By obtaining wireless technology information such as air interface resource occupancy rate, channel utilization rate and / or AP deployment point information, the screen projection connection link can be dynamically changed. When a sudden freeze occurs, the connection link in the idle channel can be dynamically transferred to perform screen projection. This can help maintain a stable and reliable network environment and ensure the continuous stability of the screen projection service.

[0130] Based on the method given in the above embodiment, taking the establishment of multiple connection links based on the local area network and the WiFi Direct signal as an example, the following is combined with Figure 3 A method for establishing multiple connection links is provided, comprising:

[0131] Step 301: The client starts.

[0132] Step 302: Trigger network scanning.

[0133] After the client is started, it will automatically trigger a network environment scan and verify specific custom information in the surrounding scanned network information. Display devices that pass the verification will be identified as target display devices.

[0134] After triggering the network scan, the client device will synchronously execute step 303 and step 304 .

[0135] Step 303: Summarize and organize the ultrasonic information, connection code information, WiFi information, and WiFi Direct information of the display device.

[0136] The device that passes the verification in step 302 is considered as a display device and obtains verification information. The client device can also obtain ultrasonic information, connection code information, WiFi information, WiFi Direct information, etc. of the display device through the obtained verification information.

[0137] Step 304: Display device information.

[0138] After the client device obtains the verification information based on step 303, it will display the information of the display device.

[0139] Step 305: Select a target display device or manually enter a connection code.

[0140] In this step, you can try to establish a connection with the target display device on the client device in two ways: Method 1, enter the connection code of the target display device on the client and click the connect button to initiate the connection; Method 2, double-click in the display device list displayed on the client, or click the connect button of the list item to connect. Based on the method in this step, you can directly establish a connection link with the target display device through manual input without triggering a network scan.

[0141] Step 306: Determine whether the target IP network is interoperable.

[0142] After initiating the connection operation on the client device, the interoperability test of the LAN network will be performed first. The connection code of the target display device obtained in the network scanning phase can be a mapping of the upstream IP of the display device. The client device will first parse the connection code according to the preset rules in the connection phase. After the parsing is successful, the IP address of the target display device will be obtained. The client device will initiate a service request based on the IP address of the target display device. If a request reply is received for its IP address, it will be considered that the network between the client device and the target display device is interoperable; at this time, step 312 is executed; if no reply request is received, it is considered that the network between the two ends is not interoperable, and step 307 is executed.

[0143] Step 307: Determine whether the target WiFi Direct information exists.

[0144] In this step, the target WiFi Direct information indicates the WiFi Direct information of the target display device. When determining whether the target WiFi Direct information exists in this step, step 303 is also executed at the same time.

[0145] In the case that the network between the client device and the target display device is interoperable, a connection link will be established between the client device and the target display device based on the local area network. At the same time, the client device will perform this step and use the direct link established by the WiFi Direct signal as a backup link.

[0146] In this step, the client device detects whether the WiFi Direct information of the target display device is found. If no WiFi Direct information exists, the WiFi Direct network connection will not be performed, which is equivalent to the WiFi Direct network connection failure, and step 308 is executed; if WiFi Direct information exists, step 311 is executed.

[0147] Step 308: Determine whether the target WiFi information exists.

[0148] When the local area network is not interoperable and the WiFi Direct network does not exist, the target WiFi information will be detected, and the target WiFi information represents the WiFi network sent by the target display device. When determining whether the target WiFi information exists in this step, step 303 will also be executed at the same time. When the target WiFi information exists, step 309 is executed; otherwise, step 310 is executed.

[0149] Step 309: Try to establish a target WiFi network.

[0150] When the client device and the target display device successfully establish a connection through the target WiFi network of the target display device, the target WiFi network at this time is also equivalent to a local area network. The difference is that the target WiFi network is a local area network with two ends directly connected. At this time, it is considered that the connection link between the client device and the target display device is successfully established, and step 312 is executed; otherwise, step 310 is executed.

[0151] Step 310: The target display device is not connected to each other, and the client device fails to connect.

[0152] When the LAN networks are not interoperable, the WiFi Direct network fails to be established, and the target WiFi network fails to be established, it is considered that the client device and the target display device are not connected to each other and the client device connection fails.

[0153] Step 311: Try to establish a WiFi Direct network.

[0154] When WiFi Direct information exists, the client device will attempt to initiate a WiFi Direct network connection. If the connection is successful, it is considered that the WiFi Direct connection is successful, and step 312 is executed; when the WiFi Direct network connection fails, step 308 is executed.

[0155] Step 312: Confirm with the target display device that it is connected.

[0156] When the LAN networks are interoperable and the WiFi Direct network is successfully established, there are two different connection links between the client device and the target display device. The client device switches to the connection success page for display, and a redundant backup network link is established between the client device and the target display device.

[0157] When the LAN networks are interoperable but the WiFi Direct network fails to be established, there is no redundant backup network link.

[0158] When the LAN networks are not interoperable but the WiFi Direct network is successfully established, it is considered that the client device and the target display device have successfully established a connection link, and the client device switches to the connection success page for display.

[0159] If a backup link is established based on a Bluetooth connection signal, Figure 3 The given WiFi Direct network is replaced with the Bluetooth connection signal.

[0160] based on Figure 3 It can be seen from the method of establishing multiple connection links that in the method given in this embodiment, the client device can try to establish a connection link with the target display device based on multiple connection methods. When multiple connection links are established, the connection link can be switched based on the method given in the above embodiment when the main link communication is abnormal. Based on the above method, it can be seen that when some connection links fail to be established, there may be only one connection link between the client device and the target display device.

[0161] Based on the different number of connection links between the client device and the target display device, taking the establishment of multiple connection links based on the LAN network and the WiFi Direct signal communication method as an example, Figure 4 and Figure 5 Different flow charts of connection link switching methods are given respectively. The conditions for connection link switching include but are not limited to: packet loss in the heartbeat link within a limited time, and the packet loss rate is greater than the threshold, abnormal interruption and exit of the heartbeat link, abnormal interruption and exit of the WebSocket data link, abnormal interruption and exit of the screen projection service, etc. Taking multiple connection links including LAN links established based on LAN networks and direct links established based on WiFi Direct signals as examples, Figure 4 A method for switching a local area network link to a direct link is shown, comprising:

[0162] Step 401: The LAN link is abnormal.

[0163] Step 402: Determine whether there is a WiFi Direct link.

[0164] If a WiFi Direct link exists, execute step 404 ; otherwise, execute step 403 .

[0165] Step 403: Maintain the status quo.

[0166] If there is no WiFi Direct link, it means there is no redundant connection link and link switching cannot be performed. At this time, the original LAN link is maintained for screen projection.

[0167] Step 404: Determine whether there is any ongoing screen projection service.

[0168] If there is an ongoing screen projection service, execute step 405; otherwise, execute step 406.

[0169] Step 405: Disconnect the screen projection service.

[0170] After disconnecting the screen projection service, execute step 406.

[0171] Step 406: disconnect the heartbeat link; disconnect the WebSocket protocol data link.

[0172] In actual applications, all links established based on the LAN network, such as BYOM (Bring Your Own Meeting) links, command data links, etc., can also be disconnected.

[0173] Step 407: Switch to the WiFi Direct link.

[0174] Step 408: Configure the heartbeat link; configure the WebSocket protocol data link.

[0175] After switching to the WiFi Direct link, according to actual conditions, the command data link, BYOM link, etc. in the above embodiments may also be configured based on the WiFi Direct link.

[0176] Step 409: Determine whether there is any ongoing screen projection service.

[0177] If it exists, execute step 410; otherwise, wait for the screen projection service.

[0178] Step 410: Establish screen projection service.

[0179] If a screen projection service already exists, a new screen projection service will be established on the WiFi Direct link.

[0180] If a backup link is established based on a Bluetooth connection signal, Figure 4 The given WiFi Direct link is replaced with a Bluetooth connection to establish an alternative link.

[0181] Based on the method given in the above embodiment, Figure 5 A method for switching a direct link to a local area network link is also provided, comprising:

[0182] Step 501: The WiFi Direct link is abnormal.

[0183] Step 502: Determine whether there is a LAN link.

[0184] If a LAN link exists, execute step 504; otherwise, execute step 503.

[0185] Step 503: Maintain the status quo.

[0186] If there is no LAN link, it means there is no redundant connection link and link switching cannot be performed. At this time, the original WiFi Direct link is maintained for screen projection.

[0187] Step 504: Determine whether there is any ongoing screen projection service.

[0188] If there is an ongoing screen projection service, execute step 505; otherwise, execute step 506.

[0189] Step 505: Disconnect the screen projection service.

[0190] After disconnecting the screen projection service, execute step 506.

[0191] Step 506: disconnect the heartbeat link; disconnect the WebSocket protocol data link.

[0192] In actual applications, all links established based on WiFi Direct signals, such as BYOM links and command data links, can also be disconnected.

[0193] Step 507: Switch to the LAN link.

[0194] Step 508: Establish a heartbeat link; establish a WebSocket protocol data link.

[0195] After switching to the LAN link, according to actual conditions, the command data link, BYOM link, etc. in the above embodiments may also be configured based on the LAN link.

[0196] Step 509: Determine whether there is any ongoing screen projection service.

[0197] If it exists, execute step 510; otherwise, wait for the screen projection service.

[0198] Step 510: Establish screen projection service.

[0199] If a backup link is established based on a Bluetooth connection signal, Figure 5 The given WiFi Direct link is replaced with a Bluetooth connection to establish an alternative link.

[0200] Based on the method of the above embodiment, the embodiment of the present application also provides a schematic diagram of a screen projection module of a client device, such as Figure 6 As shown, Figure 6The client device screen projection module 60 shown includes a network management module 601, a display device discovery module 602, a link quality assessment module 603, and a screen projection management module 604, wherein the network management module 601 is used to manage the network connection link, including a WiFi Direct management module 6011 and a WiFi management module 6012, the WiFi Direct management module 6011 is implemented by the WiFi Direct component through point-to-point technology, and is used to manage WiFi Direct signals; the WiFi management module 6012 is composed of WiFi components and is used to manage WiFi signals. The display device discovery module 602 is used to discover display devices, including a WiFi discovery module 6021, an mDNS discovery module 6022, an ultrasonic discovery module 6023, and a Bluetooth discovery module 6024, the WiFi discovery module 6021 is used to discover display devices through WiFi signals; the mDNS discovery module 6022 is used to discover display devices through multicast Domain Name System (mDNS) technology; the ultrasonic discovery module 6023 is used to discover display devices based on ultrasonic technology; the Bluetooth discovery module 6024 is used to discover display devices based on Bluetooth technology. The link quality assessment module 603 is used to perform quality assessment on the connection link, and includes a heartbeat link management module 6031, a video stream link quality assessment module 6032, a wireless quality assessment module 6033 and a network card status detection module 6034. The heartbeat link management module 6031 is used to manage the heartbeat link and perform transmission quality assessment; the video stream link quality assessment module 6032 is used to assess the quality of the video stream link, i.e., the projection connection link; the wireless quality assessment module 6033 is used to assess the quality of the overall wireless connection link; the network card status detection module 6034 is used to detect the network card status of the client device. The screen projection management module 604 is used to parse the screen projection code, including the screen projection session management module 6041 and the screen projection reverse control link management module 6042. The screen projection session management module 6041 is composed of the screen projection software development kit (SDK) component, which is used to manage the screen projection session in the screen projection business; the screen projection reverse control link management module 6042 is composed of the screen projection reverse control component, which can provide users with a more convenient, efficient and interactive screen projection experience.

[0201] Based on the method provided in the above embodiment, an optimization method for maintaining the stability of the screen projection process in multiple scenarios is provided below.

[0202] When there are multiple interoperable network paths between the client device's own network and the target display device's network, and one of them is a direct link established based on a WiFi-Direct signal, or a direct link established based on a Bluetooth connection, the client device will successfully establish a connection with the target display device through the LAN network, and then the client device and the target display device will establish a connection link based on the WiFi Direct signal and / or the Bluetooth connection signal as a backup link. Once the LAN network itself is detected to have a poor network, the network can be switched to a WiFi-Direct direct link or a connection link established by a Bluetooth connection. The switching process takes a short time, the user's perception is low, and it does not affect the user's normal use. Similarly, if the current link is a WiFi Direct direct link or a direct link established by a Bluetooth connection, and in addition to the connection link, there is a network that is interconnected with the target display device, link switching can also be performed.

[0203] When there is only one connection path between the client device and the target display device, the client device does not meet the conditions for switching the connection link. If the connection with the target display device is abnormally interrupted during the use of the client device, it can automatically help the user to reestablish the link and help the user confirm whether it is still possible to maintain the connection between the client device and the target display device. If the link is reestablished successfully, it helps the user to continue the screen projection service. This implementation method also guarantees the user experience to a certain extent.

[0204] When the connection link switching process fails, it helps users to automatically rebuild the link. The purpose of link switching and link reconstruction is to maintain the stability of the connection with the target display device. When the network environment of the client device is complex, there is a lot of interference, or the network is abnormal, the connection link switching may fail. When the connection link switching fails, it will automatically try to rebuild the link for the user. If the reconstruction is successful, the user can perform the screen projection service normally; if the reconstruction fails, the user will be prompted that the connection with the target display device is abnormally interrupted, and needs to pay attention to whether there is a problem with the network environment.

[0205] Based on the method given in the above embodiment, the screen projection method of the embodiment of the present application establishes a direct link through a WiFi Direct direct link and a Bluetooth connection, which will not affect the user's use of the existing network. On the basis of the local area network or the WiFi network emitted by the target display device, a backup network link is established at the same time. It can bring more stable and reliable protection to the user's screen projection service. By continuously providing a backup link during the screen projection process, it can quickly adjust to the backup link after an abnormality occurs in the main link, thereby ensuring a continuous and stable screen projection experience.

[0206] The client device can add a link quality assessment module to conduct real-time quality assessment of the projection network, and make statistical analysis based on various real-time data to decide whether dynamic intelligent link switching is needed. Link switching is low-perception and can be performed autonomously without user intervention.

[0207] Through the method provided in this embodiment, the time for link switching is low and the success rate of link switching is high. The existence of the link switching mechanism provides a strong guarantee for the uninterrupted user screen projection service and provides a feasible solution for solving the interruption of screen projection service in a single network.

[0208] In the event of abnormal exit of a single connection link, or failure of the backup link switch, the client device applies a link reconstruction mechanism to help users maintain the screen projection service as much as possible to ensure that the screen projection service is not interrupted. Link reconstruction is to try to re-establish a stable and reliable network in sequence based on the information of the discovered target display device to continue to maintain the current screen projection service. If the establishment is successful, the current screen projection service will continue. If the establishment fails, the abnormal exit logic will be entered, and the user will be prompted to inform the relevant information. In a complex network environment, the target display device can also dynamically adjust the channel to avoid channels with high occupancy and poor quality, and dynamically change the screen projection channel in accordance with the air interface resource occupancy rate, channel utilization, AP deployment point information, etc. of the wireless channel; dynamically transfer to an idle channel for screen projection in the event of a sudden freeze.

[0209] Those skilled in the art will appreciate that, in the above-described specific implementation method, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0210] Based on the screen projection method proposed in the above embodiment, the present application embodiment also proposes a screen projection device, which is applied to a client device, such as Figure 7 As shown, Figure 7 A schematic diagram of the structure of a screen projection device is shown, the device comprising:

[0211] A connection module 701 is used to simultaneously establish multiple connection links with a target display device; wherein the multiple connection links include a main link and at least one backup link;

[0212] The processing module 702 is used to realize screen projection on the target display device through the main link; when there is a communication abnormality in the main link, the main link is switched to the backup link, and the screen projection is realized on the target display device through the backup link.

[0213] In practical applications, the connection module 701 and the processing module 702 can be implemented based on a processor and a communication device.

[0214] In some embodiments, the connection module 701 is also used to configure a heartbeat link and a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; the processing module 702 is specifically used to stop the screen projection service, disconnect the heartbeat link and the command data link; switch the main link to the backup link, and configure the heartbeat link and the command data link based on the backup link.

[0215] In some embodiments, the processing module 702 is further configured to reconstruct each connection link among the multiple connection links when the main link fails to switch to the backup link.

[0216] In some embodiments, the connection module 701 is also used to configure a heartbeat link and a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; the processing module 702 is specifically used to stop the screen projection service and disconnect the heartbeat link and the command data link when the main link fails to switch to the backup link; and rebuild each of the multiple connection links.

[0217] In some embodiments, the processing module 702 is further used to implement screen projection on the target display device through the successfully established link when any connection link among the multiple connection links is successfully established.

[0218] In some embodiments, the connection module 701 is further used to configure a heartbeat link between the client device and the target display device; the processing module 702 is further used to determine the communication state of the main link based on the link state of the heartbeat link.

[0219] In some embodiments, the processing module 702 is specifically configured to switch the main link to the backup link when the packet loss rate of the heartbeat link is greater than a first threshold and / or the heartbeat link is disconnected.

[0220] In some embodiments, the connection module 701 is also used to configure a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; the processing module 702 is specifically used to switch the main link to the backup link when the command data link is disconnected.

[0221] In some embodiments, the processing module 702 is also used to obtain the air interface resource occupancy rate and / or channel utilization rate of the main link; when the air interface resource occupancy rate is greater than the second threshold and / or the channel utilization rate is greater than the third threshold, the main link is switched to the backup link.

[0222] In some embodiments, the processing module 702 is further used to obtain the wireless access point AP deployment point information of the network between the client device and the target display device; and obtain the communication status of the main link based on the AP deployment point information.

[0223] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the same method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0224] It should be noted that in the embodiments of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a terminal, a server, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0225] The embodiment of the present application also provides a client device. Figure 8 A schematic diagram of the structure of a client device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the client device 80 may include:

[0226] The communication interface 801 is used to exchange information with the target display device.

[0227] The memory 802 is used to store executable instructions.

[0228] The processor 803 is connected to the communication interface 801, and is used to implement any of the above-mentioned screen projection methods when executing the executable instructions stored in the memory 802.

[0229] The processor 803 may be at least one of an ASIC, a DSP, a DSPD, a PLD, a FPGA, a CPU, a controller, a microcontroller, and a microprocessor.

[0230] In actual application, the various components in the client device 80 are coupled together through the bus system 804. It can be understood that the bus system 804 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 804 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 8 In the figure, various buses are labeled as bus system 804. The above-mentioned computer readable storage medium or memory 802 can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) or other memory; it can also be various terminals including one or any combination of the above-mentioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0231] Correspondingly, an embodiment of the present application further provides a computer storage medium, on which computer executable instructions are stored, and the computer executable instructions are used to implement any one of the screen projection methods provided in the above embodiments.

[0232] Correspondingly, an embodiment of the present application further provides a computer program product, which includes computer executable instructions, and the computer executable instructions are used to implement any screen projection method provided in the above embodiments.

[0233] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0234] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0235] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0236] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0237] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0238] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0239] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A screen projection method, characterized in that: Applied to a client device, the method comprises: Establishing multiple connection links with the target display device simultaneously; wherein the multiple connection links include a main link and at least one backup link; Implementing screen projection on the target display device through the main link; When there is a communication anomaly in the main link, the main link is switched to the backup link, and screen projection is implemented on the target display device through the backup link.

2. The method according to claim 1, characterized in that: The method further comprises: configuring a heartbeat link and a command data link between the client device and the target display device; the command data link is used to send a command to the target display device; The switching of the main link to the standby link comprises: Stop the screen projection service and disconnect the heartbeat link and the command data link; The main link is switched to the backup link, and a heartbeat link and a command data link are configured based on the backup link.

3. The method according to claim 1, characterized in that The method further comprises: In the case that the switching from the main link to the backup link fails, each connection link in the plurality of connection links is rebuilt.

4. The method according to claim 3, characterized in that The method further comprises: configuring a heartbeat link and a command data link between the client device and the target display device; the command data link is used to send a command to the target display device; In the case where the switching from the main link to the standby link fails, reconstructing each connection link among the multiple connection links comprises: In the case where the main link fails to switch to the backup link, the screen projection service is stopped, and the heartbeat link and the command data link are disconnected; Each connection link among the multiple connection links is rebuilt.

5. The method according to claim 3, characterized in that: The method further comprises: When any one of the multiple connection links is successfully established, screen projection is implemented on the target display device through the successfully established link.

6. The method according to claim 1, characterized in that The method further comprises: configuring a heartbeat link between the client device and the target display device; When there is a communication abnormality in the main link, before switching the main link to the backup link, the method further includes: The communication status of the main link is determined based on the link status of the heartbeat link.

7. The method according to claim 6, characterized in that When there is a communication abnormality in the main link, switching the main link to the standby link includes: When the packet loss rate of the heartbeat link is greater than a first threshold and / or the heartbeat link is disconnected, the main link is switched to the backup link.

8. The method according to claim 1, characterized in that The method further comprises: configuring a command data link between the client device and the target display device; the command data link is used to send commands to the target display device; When there is a communication abnormality in the main link, switching the main link to the standby link includes: When the command data link is disconnected, the main link is switched to the backup link.

9. The method according to claim 1, characterized in that: The method further comprises: Obtaining an air interface resource occupancy rate and / or a channel utilization rate of the primary link; When there is a communication abnormality in the main link, switching the main link to the standby link includes: When the air interface resource occupancy rate is greater than a second threshold and / or the channel utilization rate is greater than a third threshold, the main link is switched to the backup link.

10. The method according to claim 1, characterized in that When there is a communication abnormality in the main link, before switching the main link to the backup link, the method further includes: Acquire the wireless access point (AP) deployment point information of the network between the client device and the target display device; The communication status of the main link is obtained based on the AP deployment point information.

11. The method according to any one of claims 1 to 10, characterized in that: One of the multiple connection links is a direct connection link established based on a WiFi Direct signal, or one of the multiple connection links is a direct connection link established based on a Bluetooth connection.

12. A screen projection device, characterized in that: Applied to a client device, the device comprises: A connection module, used to simultaneously establish multiple connection links with a target display device; wherein the multiple connection links include a main link and at least one backup link; A processing module is used to realize screen projection on the target display device through the main link; when there is a communication anomaly in the main link, the main link is switched to the backup link, and screen projection is realized on the target display device through the backup link.

13. A client device, characterized in that: The client device comprises a communication interface, a processor and a memory for storing a computer program that can be run on the processor; wherein, The communication interface is used to exchange information with the target display device; The processor is configured to run the computer program to perform the method according to any one of claims 1 to 11.

14. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images