Wireless screen mirroring method and apparatus

Abstract

The present application discloses a wireless screen mirroring method and apparatus. The method comprises: a terminal device acquiring M device names, the M device names being obtained by respectively searching E display devices on the basis of N screen mirroring protocols, and M being greater than E (S310); on the basis of the M device names, generating a device name list, the device name list comprising E device names among the M device names, and the E device names respectively corresponding to the E display devices (S320); receiving an indication message input by a user, the indication message pointing to a first device name among the E device names, and the first device name corresponding to a first display device among the E display devices (S330); and the terminal device mirroring a screen to the first display device (S340). According to the present application, the complexity of selecting a display device by the user can be reduced, and the end-to-end time delay is shortened, thereby improving the user experience.

Description

Wireless screen projection method and device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on June 21, 2023, with application number 202310747672.8 and invention name “Wireless Screen Projection Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of wireless communication technology, and in particular to a wireless screen projection method and device. Background Art

[0003] Currently, wireless mirroring protocols include Miracast, developed by the Wi-Fi Alliance, and proprietary protocols developed by various companies, such as Apple's Airplay. These protocols primarily utilize technologies such as wireless access points (Wi-Fi APs) or Wi-Fi Direct to enable device discovery and data transmission.

[0004] However, when a terminal device searches for a projection device, each projection protocol has its own independent discovery process and is unaware of each other. This results in multiple device names being displayed for each projection device, making it difficult for users to select a projection device. Furthermore, if a terminal device uses the Digital Living Networking Alliance (DLNA) protocol for projection, the protocol also suffers from significant end-to-end latency, severely impacting the user experience.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a wireless screen projection method and device, which can reduce the complexity of users selecting display devices, shorten end-to-end latency, and thus improve user experience.

[0007] In the first aspect, the present application provides a wireless screen projection method, which includes: a terminal device obtains M device names, and the M device names are obtained by searching for E display devices based on N screen projection protocols, M is greater than E, and M, N and E are positive integers; a device name list is generated according to the M device names, and the device name list contains E device names among the M device names, and the E device names correspond to the E display devices respectively; an indication message input by a user is received, and the indication message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices; the terminal device projects the screen to the first display device.

[0008] From a technical perspective, this application integrates multiple device names obtained when searching for the same display device based on different protocols to obtain a device name list where one display device corresponds to one device name. Compared to the prior art where one display device corresponds to multiple device names, this can greatly reduce the complexity of user selection of display devices and effectively improve user experience.

[0009] In a feasible embodiment, the E display devices include a second display device, and generating a device name list based on the M device names includes: obtaining one or more device names corresponding to the second display device from the M device names; wherein, the one or more device names all contain the device identifier of the second display device, or the similarity of the network protocol IP addresses corresponding to any two of the one or more device names is greater than or equal to a first threshold, and at least one of the one or more device names contains the device identifier of the second display device, or the similarity of the media access control MAC addresses corresponding to any two of the one or more device names is greater than or equal to a second threshold, and at least one of the one or more device names contains the device identifier of the second display device; the device name corresponding to the second display device in the device name list is one of the one or more device names.

[0010] From the technical effect point of view, this application uses three methods to fuse the device identifier contained in the device name, or the device identifier and IP address, or the device identifier and MAC address. Directly using the device identifier for fusion generation is fast; using the device identifier and IP address / MAC address fusion method makes the fusion result more accurate.

[0011] In a feasible embodiment, the terminal device projects the screen to the first display device, including: projecting the screen to the first display device based on the first projection protocol in the projection protocol list; if the projection based on the first projection protocol fails, projecting the screen to the first display device based on the next projection protocol in the projection protocol list based on the first projection protocol; wherein, when performing the first projection, the first projection protocol is the first projection protocol in the projection protocol list, and the projection protocol list is obtained by sorting based on the delay or projection success rate of the projection protocol. The lower the delay of the projection protocol, the higher the ranking of the projection protocol in the projection protocol list, or the higher the projection success rate of the projection protocol, the higher the ranking of the projection protocol in the projection protocol list.

[0012] From a technical perspective, this application sorts the screen projection protocols according to their success rate or latency, and selects the screen projection protocols in order from high to low success rate or from low to high latency, ensuring that the optimal screen projection protocol is selected to improve user experience.

[0013] In a feasible embodiment, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method also includes: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, the terminal device stops packaging the data file sent to the first display device, and the duration of the stop packaging is a preset duration.

[0014] From a technical perspective, since the end-to-end delay is long when using the DLNA protocol, this application can consume the data files in the receiving end cache by pausing the encapsulation of data files for a period of time at the sending end, thereby shortening the end-to-end delay and improving the user's operating experience in this mirroring mode.

[0015] In a feasible embodiment, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method also includes: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, selecting a D frame from a data file containing a C frame, encapsulating the D frame and sending it to the first display device, where D is a positive integer less than C.

[0016] From a technical perspective, since the end-to-end delay is long when using the DLNA protocol, this application can reduce the data cached at the receiving end by sampling and encapsulating the sent data files, thereby shortening the end-to-end delay and improving the user experience.

[0017] In a feasible embodiment, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: when the delay between the display image on the terminal device and the display image on the first display device is greater than or equal to a third threshold, adjusting the F timestamps corresponding to the F frames in the data file, the F timestamps include a first timestamp, the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; the terminal device encapsulates the adjusted data file and sends it to the first display device.

[0018] From a technical perspective, since the end-to-end delay is long when using the DLNA protocol, this application can adjust the timestamp of the data file to speed up the playback speed of the receiving end, reduce the amount of data cached by the receiving end, and thus shorten the end-to-end delay and improve the user operation experience.

[0019] Among them, the data files encapsulated by the terminal device in this application include one or more types of files such as audio and video.

[0020] In the second aspect, the present application provides a wireless screen projection device, which includes: a search unit for obtaining M device names, the M device names being obtained by searching for E display devices based on N screen projection protocols, M being greater than E, M, N and E being positive integers; a processing unit for generating a device name list based on the M device names, the device name list containing E device names among the M device names, and the E device names corresponding to the E display devices respectively; an interaction unit for receiving an indication message input by a user, the indication message pointing to the first device name among the E device names, and the first device name corresponding to the first display device among the E display devices; and a screen projection unit for projecting the screen to the first display device.

[0021] In a feasible embodiment, the processing unit is specifically used to: obtain one or more device names corresponding to the second display device from the M device names; wherein, the one or more device names all contain the device identification of the second display device, or the similarity of the network protocol IP addresses corresponding to any two of the one or more device names is greater than or equal to a first threshold, and at least one of the one or more device names contains the device identification of the second display device, or the similarity of the media access control MAC addresses corresponding to any two of the one or more device names is greater than or equal to a second threshold, and at least one of the one or more device names contains the device identification of the second display device; the device name corresponding to the second display device in the device name list is one of the one or more device names.

[0022] In a feasible embodiment, the screen projection unit is specifically used to: project the screen to the first display device based on the first screen projection protocol in the screen projection protocol list; if the screen projection based on the first screen projection protocol fails, project the screen to the first display device based on the next screen projection protocol in the screen projection protocol list based on the first screen projection protocol; wherein, when performing the first screen projection, the first screen projection protocol is the first screen projection protocol in the screen projection protocol list, and the screen projection protocol list is obtained by sorting based on the delay or the projection success rate of the screen projection protocol. The lower the delay of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list, or the higher the projection success rate of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list.

[0023] In a feasible embodiment, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is also used to: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, stop packaging the data file sent to the first display device, and the duration of the stop packaging is a preset duration.

[0024] In a feasible embodiment, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is also used to: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, select D frame from the data file containing C frame, encapsulate the D frame and send it to the first display device, where D is a positive integer less than C.

[0025] In a feasible embodiment, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is also used to: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, adjust the F timestamps corresponding to the F frames in the data file, the F timestamps include a first timestamp, the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; encapsulate the adjusted data file and send it to the first display device.

[0026] In a third aspect, the present application provides a terminal device, which includes at least one processor, a memory and an interface circuit, wherein the memory, the interface circuit and the at least one processor are interconnected through lines, and instructions are stored in the at least one memory; when the instructions are executed by the processor, any method described in the first aspect above is implemented.

[0027] In a fourth aspect, the present application provides a chip system, which includes at least one processor, a memory and an interface circuit, wherein the memory, the interface circuit and the at least one processor are interconnected through lines, and instructions are stored in the at least one memory; when the instructions are executed by the processor, any method described in the first aspect above is implemented.

[0028] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed, the method described in any one of the above-mentioned first aspects is implemented.

[0029] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed, the method described in any one of the first to sixth aspects above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The following is an introduction to the drawings used in the embodiments of this application.

[0031] FIG1 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0032] FIG2 is a schematic diagram of a system architecture of a terminal device provided in an embodiment of the present application;

[0033] FIG3 is a schematic diagram of a flow chart of a wireless screen projection method provided in an embodiment of the present application;

[0034] FIG4 is a device search flow chart provided in an embodiment of the present application;

[0035] FIG5 is a flowchart of generating the above-mentioned device name list provided in an embodiment of the present application;

[0036] FIG6 is a flow chart of a terminal device automatically selecting a screen projection protocol for screen projection provided by an embodiment of the present application;

[0037] FIG7 is a flowchart of an adjustment process of an encapsulation control process when a terminal device performs screen projection based on the DLNA protocol, provided by an embodiment of the present application;

[0038] FIG8 is a schematic structural diagram of a wireless screen projection device provided in an embodiment of the present application;

[0039] FIG9 is a schematic structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following describes the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents or, for example, A / B can represent A or B; "and / or" in the text is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two.

[0041] The terms "first", "second", "third" and "fourth" in the specification and claims of this application and the accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. Reference to "embodiments" herein means that the specific features, structures or characteristics described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0042] Please refer to Figure 1, which is a schematic diagram of an application scenario provided in an embodiment of the present application, used to describe the scenario to which the wireless screen projection method in the present application is applicable.

[0043] As shown in Figure 1, the scenario includes terminal devices and display devices, and in some scenarios, wireless access points (APs) may also be included.

[0044] The terminal device is the initiator of wireless mirroring and the sender of the service. It is usually a portable mobile device such as a mobile phone or tablet PAD.

[0045] The display device is the receiving end of wireless mirroring and the display end of the service. It is usually a large-screen smart TV or other device.

[0046] The specific implementation of a wireless AP can be a router or a switch, etc.

[0047] A connection link can be established between the terminal device and the display device through a wireless AP or wireless direct connection Wi-Fi Direct, etc., to transmit the mirror stream (ie, data file) sent by the terminal device.

[0048] Please refer to Figure 2, which is a schematic diagram of the system architecture of a terminal device provided in an embodiment of the present application, and is used to describe the software and hardware architecture on the terminal device in Figure 1 and the embodiments below.

[0049] As shown in FIG2 , the terminal device includes a screen projection application 210 at the software level, and includes a network interface 250 , a processor 260 , a memory 270 , and a screen display & interaction module 280 at the hardware level.

[0050] The implementation process of the wireless screen projection method in this application on the terminal device is as follows:

[0051] First, the screen display & interaction module 280 is used to receive instructions input by the user, instruct the terminal device to initiate a wireless screen projection service with the instruction, and transmit the instruction to the screen projection protocol management module 211. After the screen projection protocol management module 211 receives the instruction, the device discovery module 214 searches for display devices that can be used for screen projection around the terminal device through different screen projection protocols (corresponding to the E display devices in the following embodiment), and obtains multiple device names (corresponding to the M display devices in the following embodiment). The protocol fusion module 212 then fuses the multiple device names obtained by the search to obtain a device name list. Each device name in this list corresponds to a display device, and different device names correspond to different display devices. The fused device name list is then displayed to the user, and the user inputs the display device selected for wireless mirroring through the screen display & interaction module 280 (corresponding to the first display device in the following embodiment).

[0052] The transmission management module 213 then determines the screen projection protocol for wireless mirroring (how to determine the screen projection protocol will be described in detail in the following embodiments).

[0053] Finally, the media encapsulation module 220 encapsulates the data according to the determined protocol. Specifically, the encapsulation control module 221 is used to adjust the timestamp of each frame of data and control the start and pause of the encapsulation process. The format encapsulation module 222 is used to encapsulate the data file into a corresponding bitstream based on the parameters set by the encapsulation control module 221. During the encapsulation process, the media encapsulation module 220 reads the data file to be encapsulated from the memory 270. The encapsulation process of the data file into the corresponding bitstream is executed by the processor 260. After encapsulation, the network interface 250 sends the data to the display device for display.

[0054] Optionally, during the screen projection process, the terminal device can also receive user instructions through the screen display & interaction module 280 to instruct the screen recording module 230 to perform screen recording during the screen projection process.

[0055] Among them, the process of interaction between hardware devices such as the network interface 250, processor 260, and memory 270 and the screen projection application 210 is controlled by the operating system (not shown in Figure 2) and will not be repeated here.

[0056] Please refer to Figure 3, which is a flow chart of a wireless screen projection method provided by an embodiment of the present application. The method includes steps S310, S320, S330 and S340.

[0057] Step S310: The terminal device obtains M device names, where the M device names are obtained by searching for E display devices based on N screen projection protocols. M is greater than E, and M, N, and E are positive integers.

[0058] Specifically, the terminal device receives a user input command and starts the wireless mirroring service. The terminal device searches for display devices within a certain distance range based on the N supported screen projection protocols and obtains M device names. In other words, these M device names are obtained by searching E display devices within a certain range of the terminal device.

[0059] Each of the E display devices corresponds to one or more device names in the M device names.

[0060] Among them, when searching for a display device based on a screen projection protocol, if the display device supports the screen projection protocol, a device name will be obtained. If the display device does not support the screen projection protocol, the corresponding device name cannot be obtained.

[0061] Optionally, the M device names may include the same device name, or the M device names may be different. The specific case where the M device names include the same device name is as follows:

[0062] (1) When searching for the same display device through multiple projection protocols, the device names obtained can be the same or different. That is, when searching for the same display device through different projection protocols and the same device name is obtained, the M device names mentioned above include the same device name.

[0063] (2) When a user uses multiple screen projection programs simultaneously, and when these multiple screen projection programs use the same screen projection protocol to search for display devices, for each display device, multiple identical or different device names may be obtained based on the same screen projection protocol. These multiple device names correspond one-to-one to the multiple screen projection programs. If the same device name is obtained, then the M device names will include the multiple identical device names.

[0064] The following describes the process of obtaining M device names from the perspective of the number of screen projection protocols:

[0065] (1) N is equal to 1

[0066] When a user uses multiple screen projection programs, and each of these programs searches for display devices using only one screen projection protocol, multiple device names will be obtained for each of the E display devices. These multiple device names correspond one-to-one to the multiple screen projection programs, and the multiple device names can contain the same device name or be different. In this case, each display device corresponds to multiple device names among the M device names, where M is greater than E.

[0067] (2) N is greater than 1

[0068] When a user uses multiple screen projection programs, and these multiple screen projection programs search for display devices through multiple screen projection protocols, multiple device names will be obtained for each of the E display devices. In this case, each display device corresponds to multiple device names among the M device names, where M is greater than E.

[0069] When a user uses a screen projection program, and the screen projection program searches for display devices through multiple screen projection protocols, one or more device names will be obtained for each of the E display devices. The situation in which one display device corresponds to one device name is that the display device only supports one of the multiple screen projection protocols. At this time, each display device corresponds to one or more device names in the M device names. When at least one display device supports multiple screen projection protocols, M is greater than E.

[0070] The device search (discovery) process is described below with reference to FIG. 4 .

[0071] As shown in Figure 4, a user initiates the wireless mirroring service by opening the screen mirroring application and starting a display device search. The terminal device then searches for display devices based on the N supported screen mirroring protocols, as shown in Figure 4: searching based on protocol 1, ..., and then searching based on protocol N.

[0072] As shown in FIG4 , after searching for each protocol, it is determined whether a display device is found. If so, the searched device name is saved; if not, the search process based on the protocol is terminated.

[0073] Step S320: Generate a device name list according to the M device names, where the device name list includes E device names among the M device names, and the E device names correspond to the E display devices respectively.

[0074] The process of generating the device name list (also referred to as device name fusion) is to select E device names from the M device names so that the E device names correspond one-to-one to the E display devices.

[0075] Each of the M device names corresponds to an Internet Protocol (IP) address, and the IP address is obtained during the search process.

[0076] The process of generating the above device name list is as follows, which is described by taking the second display device among the E display devices (any one of the E display devices) as an example:

[0077] First, one or more device names corresponding to the second display device in the M device names are determined. The determination method can be one of the following methods:

[0078] (1) Determined by device identification

[0079] The device names that contain the device identification of the second display device in the M device names are one or more device names corresponding to the second display device, that is, the one or more device names are obtained by searching for the second display device through one or more screen projection protocols.

[0080] The device identifier may be a device model or other identifier that can uniquely identify a display device, and this application does not limit this. For example, the device identifier may be Z6X PRO.

[0081] (2) Determined by both IP address and device identification

[0082] First, a device name that includes the device identifier of the second display device and the IP address corresponding to this device name are determined. Then, this IP address is compared with other device names in the M device names to obtain one or more device names. These one or more device names satisfy the following conditions: the similarity of the IP addresses corresponding to any two device names is greater than or equal to a first threshold. These one or more device names are the one or more device names corresponding to the second display device in the M device names. At least one device name in these one or more device names includes the device identifier of the second display device.

[0083] Optionally, the method for determining the similarity between two IP addresses may specifically include comparing characters at corresponding positions in the two IP addresses to determine whether they are identical, then calculating the proportion of positions with identical characters in the length of the IP addresses, and finally comparing the proportion with a first threshold. Furthermore, those skilled in the art may also use other methods to compare the similarity between the IP addresses, which is not limited in this application.

[0084] (3) Determined by Media Access Control (MAC) address and device identification

[0085] This method is similar to the second method described above. That is, first determine the device name containing the device identifier of the second display device and the MAC address corresponding to this device name, then compare this MAC address with other device names in the M device names to obtain one or more device names, and these one or more device names satisfy: the similarity of the MAC addresses corresponding to any two device names is greater than or equal to the first threshold. These one or more device names are the one or more device names corresponding to the second display device in the M device names. At least one device name in these one or more device names contains the device identifier of the second display device.

[0086] The MAC address corresponding to each device name is obtained by converting the corresponding IP address.

[0087] Optionally, the similarity between the two MAC addresses may be determined by comparing with a second threshold. The specific method may refer to the method for determining the similarity between IP addresses, which will not be described in detail here.

[0088] After determining one or more device names corresponding to the second display device in the above manner, a device name may be selected from the one or more device names and added to the device name list as the only device name corresponding to the second display device in the device name list.

[0089] The process of generating the above device name list is described below with reference to FIG. 5 .

[0090] As shown in Figure 5, first execute step 510: obtain the device names searched based on N protocols. Then execute step 520: determine whether at least one device name is included. If so, execute step 530: merge the searched device names; then execute step 540: display the device name list obtained after fusion, which contains E device names corresponding to E display devices. This process is to display this device name list to the user on the display screen of the terminal device (i.e., the screen display & interaction module 280 in Figure 1) for the user to select the display device for subsequent screen projection. After step 540 is executed, the above list generation process is terminated. If the judgment result of step 520 is no, the above list generation process is terminated.

[0091] Specifically, the specific execution process of each of the above steps can be found in the corresponding description of step S320 in the above embodiment, and will not be repeated here.

[0092] Step S330: receiving an instruction message input by a user, where the instruction message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices.

[0093] Specifically, after the terminal device displays the above-mentioned device name list to the user through the screen display & interaction module 280, the user clicks on the corresponding device name based on his or her own choice. This click action will generate a corresponding indication message, pointing to the first device name selected by the user. This first device name corresponds to the first device among the E display devices.

[0094] Step S340: The terminal device projects the screen onto the first display device.

[0095] Specifically, the terminal device will select a screen projection protocol from the list of supported screen projection protocols for screen projection. The specific method for selecting the screen projection protocol is:

[0096] The terminal selects the first projection protocol in the projection protocol list and projects the screen to the first display device based on the first projection protocol. If the projection is successful, the terminal device starts sending data to the first display device. If the projection fails, the terminal device projects the screen to the first display device using the next projection protocol in the projection protocol list after the first projection protocol. The terminal selects projection protocols in sequence until the projection is successful.

[0097] Among them, when performing the first screen projection, the above-mentioned first screen projection protocol is the first screen projection protocol in the screen projection protocol list.

[0098] Optionally, the projection protocols in the above projection protocol list are sorted based on preset rules. The sorting rules are: (1) the lower the delay of the projection protocol, the higher the projection protocol is ranked in the projection protocol list; or (2) the higher the success rate of the projection protocol, the higher the projection protocol is ranked in the projection protocol list. The success rate of the projection protocol is obtained based on historical data statistics; or (3) the larger the projection parameters of the projection protocol, the higher the projection protocol is ranked in the projection protocol list. Among them, the projection parameters of each projection protocol are obtained by weighted calculation of the delay and success rate of the projection protocol, and the specific weighting method is not limited in this application.

[0099] That is, this application can automatically select the optimal screen projection protocol for screen projection through the above method, improve the screen projection success rate, and enhance the user experience.

[0100] The following describes the process of the terminal device automatically selecting the screen projection protocol for screen projection in conjunction with Figure 6.

[0101] As shown in Figure 6, after the screen projection protocol selection process begins, step 610 is first executed: determining whether the first display device supports screen projection protocol 1. If the output result of step 610 is yes, the terminal device begins to establish a connection link with the first display device based on screen projection protocol 1 and executes step 620: determining whether the screen projection is successful. If the output result of step 620 is yes, step 630 is executed: the terminal device sends the data file to the first display device, and the screen projection protocol selection process ends. If the output result of step 620 or step 610 is no, step 640 is executed: determining whether the first display device supports screen projection protocol 2. If the output result of step 640 is yes, the terminal device begins to establish a connection link with the first display device based on screen projection protocol 2 and executes step 650: determining whether the screen projection is successful. If the output result of step 650 is yes, step 660 is executed: the terminal device sends the data file to the first display device, and the screen projection protocol selection process ends. If the output result of step 640 or step 650 is no, the above screen projection protocol selection process ends.

[0102] Among them, the above-mentioned process in Figure 6 describes the above-mentioned projection protocol selection process by including two projection protocols (projection protocol 1 and projection protocol 2) in the projection protocol list, and cannot be understood as a limitation on the number of projection protocols in the projection protocol list.

[0103] Among them, screen projection protocol 1 and screen projection protocol 2 are the first screen projection protocol and the second screen projection protocol in the screen projection protocol list respectively.

[0104] Optionally, if the DLNA protocol is selected and screen projection is successfully performed in the above manner, if the delay between the display image on the terminal device and the display image on the first display device is greater than or equal to the third threshold, the terminal device can adjust the encapsulation control process during data file transmission in the following manner (corresponding to the determination by the media encapsulation module 220 in Figure 2).

[0105] (1) Suspend packaging

[0106] Specifically, when the above-mentioned delay is greater than or equal to the third threshold, the terminal device suspends packaging the data file for a preset duration (set by technical personnel in this field according to the specific scenario, and is not limited in this application) to speed up the consumption of cached data files on the first display device, thereby reducing the delay between the terminal device and the display screen on the first display device, and improving the user experience.

[0107] (2) Sampling packaging

[0108] Specifically, when the delay is greater than or equal to the third threshold, the terminal device may select frame D from the data file containing frame C, encapsulate frame D, and send the encapsulated frame D to the first display device, where D is a positive integer less than C. This method accelerates the consumption of cached data files on the first display device, thereby reducing delay and improving user experience.

[0109] The sampling method may be a random sampling of C frames, a sampling of C frames at fixed time intervals, etc., which is not limited in this application.

[0110] (3) Fast forward playback

[0111] Specifically, when the delay is greater than or equal to the third threshold, the terminal device may adjust F timestamps corresponding to F frames in the data file, where the F timestamps include the first timestamp, and the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, where F is a positive integer. The terminal device then packages the adjusted data file and sends it to the first display device.

[0112] Among them, F frame is part or all of the data in the data file.

[0113] The first timestamp is any one of the F timestamps.

[0114] The timestamp corresponding to each frame is used to indicate the playback time of the frame, and the playback time is relative to the start playback time of the data file.

[0115] That is, method (3) reduces the delay and improves the user experience by speeding up the playback speed of the data file on the first display device.

[0116] In addition, the above three adjustment methods for the packaging control process can be combined arbitrarily. Specifically, the following four packaging control methods can be obtained: method (1) + method (2), method (1) + method (3), method (2) + method (3), and method (1) + method (2) + method (3). The specific implementation process of the combined solution will not be repeated here.

[0117] The following describes the adjustment process of the encapsulation control process when the terminal device performs screen projection based on the DLNA protocol in conjunction with Figure 7.

[0118] As shown in Figure 7, first execute step 710: send the data file based on the DLNA protocol. Then execute step 720: determine whether the screen projection process based on the DLNA protocol is ended. If so, end this process. If not, execute step 730: calculate the end-to-end delay, that is, the delay between the display screen on the terminal device and the display screen on the first display device. Then execute step 740: determine whether the delay is less than the third threshold. If not, execute step 750: adjust the encapsulation control process, that is, the seven methods listed in the above embodiment. If so, return to execute step 710.

[0119] Specifically, the specific execution process of the above steps can be found in the description of the above embodiments, which will not be repeated here.

[0120] Please refer to Figure 8, which is a schematic diagram of the structure of a wireless screen projection device provided by an embodiment of the present application. As shown in Figure 8, the device includes a search unit 810, a processing unit 820, an interaction unit 830 and a screen projection unit 840.

[0121] The search unit 810 is used to obtain M device names, and the M device names are obtained by searching for E display devices based on N screen projection protocols, where M is greater than E, and M, N and E are positive integers; the processing unit 820 is used to generate a device name list based on the M device names, and the device name list contains E device names among the M device names, and the E device names correspond to the E display devices respectively; the interaction unit 830 is used to receive an indication message input by the user, and the indication message points to the first device name among the E device names, and the first device name corresponds to the first display device among the E display devices; the screen projection unit 840 is used to project the screen to the first display device.

[0122] In a feasible embodiment, the processing unit 820 is specifically used to: obtain one or more device names corresponding to the second display device from the M device names; wherein, the one or more device names all contain the device identification of the second display device, or the similarity of the network protocol IP addresses corresponding to any two device names in the one or more device names is greater than or equal to a first threshold, and at least one device name in the one or more device names contains the device identification of the second display device, or the similarity of the media access control MAC addresses corresponding to any two device names in the one or more device names is greater than or equal to a second threshold, and at least one device name in the one or more device names contains the device identification of the second display device; the device name corresponding to the second display device in the device name list is one of the one or more device names.

[0123] In a feasible embodiment, the screen projection unit 840 is specifically used to: project the screen to the first display device based on the first screen projection protocol in the screen projection protocol list; if the screen projection based on the first screen projection protocol fails, project the screen to the first display device based on the next screen projection protocol in the screen projection protocol list based on the first screen projection protocol; wherein, when performing the first screen projection, the first screen projection protocol is the first screen projection protocol in the screen projection protocol list, and the screen projection protocol list is obtained by sorting based on the delay or the projection success rate of the screen projection protocol. The lower the delay of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list, or the higher the projection success rate of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list.

[0124] In a feasible embodiment, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit 840 is also used to: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, stop packaging the data file sent to the first display device, and the duration of the stop packaging is a preset duration.

[0125] In a feasible embodiment, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit 840 is also used to: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, select D frame from the data file containing C frame, encapsulate the D frame and send it to the first display device, where D is a positive integer less than C.

[0126] In a feasible embodiment, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit 840 is also used to: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, adjust the F timestamps corresponding to the F frames in the data file, the F timestamps include a first timestamp, the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; encapsulate the adjusted data file and send it to the first display device.

[0127] Specifically, the specific execution process of the above-mentioned wireless screen projection device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0128] Please refer to Figure 9, which is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application, as a specific implementation of the wireless screen projection device in Figure 8. As shown in Figure 9, the terminal device includes a processor 901, at least one memory 902, a radio frequency circuit 903, and an interface circuit 905; wherein the processor 901, the memory 902, the radio frequency circuit 903 and the interface circuit 905 are connected via a bus 904.

[0129] The radio frequency circuit 903 is a component of the antenna on the terminal device, and is used to search for E display devices based on N screen projection protocols, thereby obtaining M device names. The processor 901 is used to generate a device name list based on the M device names, and the device name list contains E device names among the M device names, and the E device names correspond to the E display devices respectively. The processor 901 is also used to receive an indication message input by the user through the screen display & interaction module on the terminal device, and the indication message points to the first device name among the E device names, and the first device name corresponds to the first display device among the E display devices. The interface circuit 905 is used to send a screen projection data file to the first display device. The memory 902 is used to store the above-mentioned M device names and the device name list.

[0130] Specifically, the specific execution process of the above-mentioned terminal device can refer to the corresponding execution process in the aforementioned method embodiment, which will not be repeated here.

[0131] An embodiment of the present application provides a chip system, which includes at least one processor, a memory and an interface circuit. The memory, the interface circuit and the at least one processor are interconnected through lines, and instructions are stored in the at least one memory. When the instructions are executed by the processor, some or all of the steps of any one of the above-mentioned wireless screen projection method embodiments are implemented.

[0132] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, some or all of the steps of any one of the above-mentioned wireless screen projection method embodiments can be implemented.

[0133] An embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by a processor, some or all of the steps of any one of the above-mentioned wireless screen projection method embodiments are implemented.

[0134] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. It should be noted that for the aforementioned method embodiments, in order to simplify the description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited to the order of the actions described, because according to this application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0136] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A wireless screen projection method, characterized in that: The method comprises: The terminal device obtains M device names, where the M device names are obtained by searching for E display devices based on N screen projection protocols, respectively, where M is greater than E, and M, N, and E are positive integers; Generate a device name list according to the M device names, the device name list including E device names among the M device names, the E device names respectively corresponding to the E display devices; receiving an indication message input by a user, wherein the indication message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices; The terminal device projects the screen onto the first display device.

2. The method according to claim 1, characterized in that The E display devices include a second display device, and generating a device name list according to the M device names includes: Obtain one or more device names corresponding to the second display device from the M device names; wherein the one or more device names all contain the device identification of the second display device, or the similarity of the network protocol IP addresses respectively corresponding to any two of the one or more device names is greater than or equal to a first threshold, and at least one of the one or more device names contains the device identification of the second display device, or the similarity of the media access control MAC addresses respectively corresponding to any two of the one or more device names is greater than or equal to a second threshold, and at least one of the one or more device names contains the device identification of the second display device; The device name corresponding to the second display device in the device name list is one of the one or more device names.

3. The method according to claim 1 or 2, characterized in that: The terminal device projects a screen onto the first display device, including: Casting the screen to the first display device based on the first screen casting protocol in the screen casting protocol list; If screen projection based on the first screen projection protocol fails, screen projection is performed to the first display device based on the next screen projection protocol of the first screen projection protocol in the screen projection protocol list; Among them, when performing screen projection for the first time, the first screen projection protocol is the first screen projection protocol in the screen projection protocol list, and the screen projection protocol list is sorted based on the delay or the projection success rate of the screen projection protocol. The lower the delay of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list, or the higher the projection success rate of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list.

4. The method according to any one of claims 1 to 3, characterized in that When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: When the time delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, the terminal device stops packaging the data file sent to the first display device, and the duration of the stop packaging is a preset duration.

5. The method according to any one of claims 1 to 3, characterized in that: When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: When the delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, a D frame is selected from a data file containing a C frame, and the D frame is encapsulated and sent to the first display device, where D is a positive integer less than C.

6. The method according to any one of claims 1 to 3, characterized in that When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: When the delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, adjusting F timestamps corresponding to F frames in the data file, wherein the F timestamps include a first timestamp, and the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; The terminal device encapsulates the adjusted data file and sends it to the first display device.

7. A wireless screen projection device, characterized in that: The device comprises: A search unit, used to obtain M device names, where the M device names are obtained by searching for E display devices based on N screen projection protocols, respectively, where M is greater than E, and M, N, and E are positive integers; a processing unit, configured to generate a device name list according to the M device names, wherein the device name list includes E device names among the M device names, and the E device names correspond to the E display devices respectively; an interaction unit, configured to receive an indication message input by a user, wherein the indication message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices; A screen projection unit, used for projecting the screen to the first display device.

8. The device according to claim 7, characterized in that The processing unit is specifically used for: Obtain one or more device names corresponding to the second display device from the M device names; wherein the one or more device names all contain the device identification of the second display device, or the similarity of the network protocol IP addresses respectively corresponding to any two of the one or more device names is greater than or equal to a first threshold, and at least one of the one or more device names contains the device identification of the second display device, or the similarity of the media access control MAC addresses respectively corresponding to any two of the one or more device names is greater than or equal to a second threshold, and at least one of the one or more device names contains the device identification of the second display device; The device name corresponding to the second display device in the device name list is one of the one or more device names.

9. The device according to claim 7 or 8, characterized in that The screen projection unit is specifically used for: Casting the screen to the first display device based on the first screen casting protocol in the screen casting protocol list; If screen projection based on the first screen projection protocol fails, screen projection is performed to the first display device based on the next screen projection protocol of the first screen projection protocol in the screen projection protocol list; Among them, when performing screen projection for the first time, the first screen projection protocol is the first screen projection protocol in the screen projection protocol list, and the screen projection protocol list is sorted based on the delay or the projection success rate of the screen projection protocol. The lower the delay of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list, or the higher the projection success rate of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list.

10. The device according to any one of claims 7 to 9, characterized in that: When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is further used for: When the time delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, the packaging of the data file sent to the first display device is stopped, and the duration of the stopping of packaging is a preset duration.

11. The device according to any one of claims 7 to 9, characterized in that: When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is further used for: When the delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, a D frame is selected from a data file containing a C frame, and the D frame is encapsulated and sent to the first display device, where D is a positive integer less than C.

12. The device according to any one of claims 7 to 9, characterized in that When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is further used for: When the delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, adjusting F timestamps corresponding to F frames in the data file, wherein the F timestamps include a first timestamp, and the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; The adjusted data file is packaged and sent to the first display device.

13. A terminal device, characterized in that: The terminal device includes at least one processor, a memory and an interface circuit. The memory, the interface circuit and the at least one processor are interconnected via lines. Instructions are stored in the at least one memory. When the instructions are executed by the processor, any method described in claims 1 to 6 is implemented.

14. A chip system, characterized in that: The chip system includes at least one processor, a memory and an interface circuit. The memory, the interface circuit and the at least one processor are interconnected by lines. Instructions are stored in the at least one memory. When the instructions are executed by the processor, any method described in claims 1-6 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 6 is implemented.

16. A computer program product, characterized in that The computer program product comprises instructions, and when the instructions are executed, the method according to any one of claims 1 to 6 is implemented.

Images