Wireless screen projection method and device

Abstract

The invention discloses a wireless screen projection method and device, and the method comprises the steps that a terminal device obtains M device names, the M device names are obtained by searching E display devices based on N screen projection protocols, and M is greater than E; an equipment name list is generated according to the M equipment names, the equipment name list comprises E equipment names in the M equipment names, and the E equipment names correspond to the E display equipment respectively; an indication message input by a user is received, the indication message points to a first device name in the E device names, and the first device name corresponds to a first display device in the E display devices; and the terminal device projects a screen to the first display device. Through the method and the device, the complexity of selecting the display equipment by the user can be reduced, the end-to-end time delay is shortened, and the user experience is further improved.

Description

Technical Field

[0001] The present application relates to the field of wireless communication technology, and in particular to a wireless screen projection method and device. Background Art

[0002] At present, the protocols used for wireless mirroring include: Miracast protocol developed by Wi-Fi Alliance, and private protocols developed by various companies (such as Airplay protocol developed by Apple). Each protocol mainly realizes device discovery and data transmission through technologies such as wireless access point (Wi-Fi Access Point, Wi-Fi AP) or wireless direct connection Wi-Fi Direct.

[0003] However, when the terminal device searches for the projection device, since each projection protocol is independent of each other, they have different discovery processes and are not aware of each other, resulting in each projection device corresponding to multiple device names being displayed, which causes trouble for users to select the projection device. In addition, if the terminal device uses the Digital Living Networking Alliance (DLNA) protocol for projection, the protocol also has the problem of large end-to-end latency, which seriously affects the user experience. Summary of the invention

[0004] The embodiments of the present application provide a wireless screen projection method and device, which can reduce the complexity of user selection of display devices, shorten end-to-end latency, and thus improve user experience.

[0005] In a first aspect, the present application provides a wireless screen projection method, the method comprising: a terminal device obtains M device names, the M device names are obtained by searching for E display devices based on N screen projection protocols, M is greater than E, and M, N and E are positive integers; a device name list is generated according to the M device names, the device name list contains E device names among the M device names, and the E device names correspond to the E display devices respectively; an indication message input by a user is received, the indication message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices; the terminal device projects the screen to the first display device.

[0006] From the perspective of technical effects, this application merges multiple device names obtained when searching for the same display device based on different protocols to obtain a device name list in which one display device corresponds to one device name. Compared with the situation in the prior art where one display device corresponds to multiple device names, this can greatly reduce the complexity of user selection of display devices and effectively improve user experience.

[0007] In a feasible implementation, the E display devices include a second display device, and generating a device name list based on the M device names includes: obtaining one or more device names corresponding to the second display device from the M device names; wherein the one or more device names all contain the device identifier of the second display device, or the similarity of the network protocol IP addresses corresponding to any two of the one or more device names is greater than or equal to a first threshold, and at least one of the one or more device names contains the device identifier of the second display device, or the similarity of the media access control MAC addresses corresponding to any two of the one or more device names is greater than or equal to a second threshold, and at least one of the one or more device names contains the device identifier of the second display device; the device name corresponding to the second display device in the device name list is one of the one or more device names.

[0008] From the technical effect point of view, this application integrates the device identifier contained in the device name, or the device identifier and IP address, or the device identifier and MAC address in three ways. Directly using the device identifier for fusion generation is fast; the fusion result is more accurate when the device identifier and IP address / MAC address are integrated.

[0009] In a feasible implementation, the terminal device projects the screen to the first display device, including: projecting the screen to the first display device based on the first projection protocol in the projection protocol list; if the projection based on the first projection protocol fails, projecting the screen to the first display device based on the next projection protocol in the projection protocol list according to the first projection protocol; wherein, when performing the first projection, the first projection protocol is the first projection protocol in the projection protocol list, and the projection protocol list is obtained by sorting based on the delay of the projection protocol or the projection success rate of the projection protocol, and the lower the delay of the projection protocol, the higher the ranking of the projection protocol in the projection protocol list, or the higher the projection success rate of the projection protocol, the higher the ranking of the projection protocol in the projection protocol list.

[0010] From a technical effect perspective, this application sorts the screen projection protocols according to their success rate or latency, and selects the projection protocols in order from high to low success rate or from low to high latency for projection, ensuring that the optimal projection protocol is selected to improve user experience.

[0011] In a feasible implementation, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: when the delay between the display image on the terminal device and the display image on the first display device is greater than or equal to a third threshold, the terminal device stops packaging the data file sent to the first display device, and the duration of the stop packaging is a preset duration.

[0012] From a technical perspective, since the end-to-end delay is long when using the DLNA protocol, this application can consume the data files in the receiving end cache by pausing the encapsulation of data files for a period of time at the sending end, thereby shortening the end-to-end delay and improving the user's operating experience in this mirroring mode.

[0013] In a feasible implementation, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: when the delay between the display image on the terminal device and the display image on the first display device is greater than or equal to a third threshold, selecting a D frame from a data file containing a C frame, encapsulating the D frame and sending it to the first display device, where D is a positive integer less than C.

[0014] From the technical effect point of view, since the end-to-end delay is long when using the DLNA protocol, this application can reduce the data cached at the receiving end by sampling and packaging the sent data files, thereby shortening the end-to-end delay and improving the user operation experience.

[0015] In a feasible implementation, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: when the delay between the displayed picture on the terminal device and the displayed picture on the first display device is greater than or equal to a third threshold, adjusting the F timestamps corresponding to the F frames in the data file, the F timestamps include a first timestamp, the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; the terminal device encapsulates the adjusted data file and sends it to the first display device.

[0016] From a technical perspective, since the end-to-end delay when using the DLNA protocol is long, this application can adjust the timestamp of the data file to speed up the playback speed of the receiving end, reduce the amount of data cached by the receiving end, thereby shortening the end-to-end delay and improving the user experience.

[0017] Among them, the data files encapsulated by the terminal device in this application include one or more types of files such as audio and video.

[0018] In the second aspect, the present application provides a wireless screen projection device, which includes: a search unit for obtaining M device names, the M device names are obtained by searching for E display devices based on N screen projection protocols, M is greater than E, M, N and E are positive integers; a processing unit for generating a device name list based on the M device names, the device name list contains E device names among the M device names, and the E device names correspond to the E display devices respectively; an interaction unit for receiving an indication message input by a user, the indication message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices; and a screen projection unit for projecting the screen to the first display device.

[0019] In a feasible implementation, the processing unit is specifically used to: obtain one or more device names corresponding to the second display device from the M device names; wherein the one or more device names all contain the device identification of the second display device, or the similarity of the network protocol IP addresses corresponding to any two of the one or more device names is greater than or equal to a first threshold, and at least one of the one or more device names contains the device identification of the second display device, or the similarity of the media access control MAC addresses corresponding to any two of the one or more device names is greater than or equal to a second threshold, and at least one of the one or more device names contains the device identification of the second display device; the device name corresponding to the second display device in the device name list is one of the one or more device names.

[0020] In a feasible implementation, the screen projection unit is specifically used to: project the screen to the first display device based on the first screen projection protocol in the screen projection protocol list; if the screen projection based on the first screen projection protocol fails, project the screen to the first display device based on the next screen projection protocol in the screen projection protocol list based on the first screen projection protocol; wherein, when performing the first screen projection, the first screen projection protocol is the first screen projection protocol in the screen projection protocol list, and the screen projection protocol list is sorted based on the delay of the screen projection protocol or the screen projection success rate. The lower the delay of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list, or the higher the projection success rate of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list.

[0021] In a feasible implementation, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is also used to: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, stop packaging the data file sent to the first display device, and the duration of the stop packaging is a preset duration.

[0022] In a feasible implementation, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is also used to: when the delay between the display image on the terminal device and the display image on the first display device is greater than or equal to a third threshold, select a D frame from a data file containing a C frame, encapsulate the D frame and send it to the first display device, where D is a positive integer less than C.

[0023] In a feasible implementation, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is also used to: when the delay between the display image on the terminal device and the display image on the first display device is greater than or equal to a third threshold, adjust the F timestamps corresponding to the F frames in the data file, the F timestamps include a first timestamp, the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; encapsulate the adjusted data file and send it to the first display device.

[0024] In a third aspect, the present application provides a terminal device, comprising at least one processor, a memory and an interface circuit, wherein the memory, the interface circuit and the at least one processor are interconnected via lines, and instructions are stored in the at least one memory; when the instructions are executed by the processor, any method described in the first aspect above is implemented.

[0025] In a fourth aspect, the present application provides a chip system, which includes at least one processor, a memory and an interface circuit, wherein the memory, the interface circuit and the at least one processor are interconnected through lines, and instructions are stored in the at least one memory; when the instructions are executed by the processor, any method described in the first aspect above is implemented.

[0026] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed, the method described in any one of the above-mentioned first aspects is implemented.

[0027] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed, the method described in any one of the first to sixth aspects above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The following is an introduction to the drawings used in the embodiments of the present application.

[0029] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0030] Figure 2 A schematic diagram of a system architecture of a terminal device provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of a flow chart of a wireless screen projection method provided in an embodiment of the present application;

[0032] Figure 4 A device search flow chart provided for an embodiment of the present application;

[0033] Figure 5 A flowchart for generating the above-mentioned device name list provided in an embodiment of the present application;

[0034] Figure 6 A flowchart of a terminal device automatically selecting a screen projection protocol for screen projection provided in an embodiment of the present application;

[0035] Figure 7 A flowchart of adjusting the encapsulation control process when a terminal device performs screen projection based on the DLNA protocol provided in an embodiment of the present application;

[0036] Figure 8 A schematic diagram of the structure of a wireless screen projection device provided in an embodiment of the present application;

[0037] Fig. 9 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0039] The terms "first", "second", "third" and "fourth" in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase at various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0040] See also Figure 1 , Figure 1 An application scenario schematic diagram provided for an embodiment of the present application is used to describe the scenario to which the wireless screen projection method in the present application is applicable.

[0041] like Figure 1 As shown, the scenario includes terminal devices and display devices, and in some scenarios may also include wireless access points (Access Point, AP).

[0042] The terminal device is the initiator of wireless mirroring and the sender of the service. It is usually a portable mobile device such as a mobile phone or tablet PAD.

[0043] The display device is the receiving end of wireless mirroring and the display end of the service. It is usually a large-screen smart TV and other devices.

[0044] The wireless AP may be implemented as a router or a switch.

[0045] A connection link can be established between the terminal device and the display device through a wireless AP or wireless direct connection Wi-Fi Direct, etc., to transmit the mirror stream (ie, data file) sent by the terminal device.

[0046] See also Figure 2 , Figure 2 A schematic diagram of a terminal system architecture provided in an embodiment of the present application is used to describe Figure 1 And the software and hardware architecture on the terminal device in the embodiments below.

[0047] like Figure 2As shown, the terminal device includes a screen projection application 210 at the software level, and includes a network interface 250, a processor 260, a memory 270, and a screen display & interaction module 280 at the hardware level.

[0048] The implementation process of the wireless screen projection method on the terminal device in this application is as follows:

[0049] First, the screen display & interaction module 280 is used to receive instructions input by the user, instruct the terminal device to initiate a wireless screen projection service with the instructions, and transmit the instructions to the screen projection protocol management module 211. After the screen projection protocol management module 211 receives the instructions, the device discovery module 214 searches for display devices that can be used for screen projection around the terminal device through different screen projection protocols (corresponding to the E display devices in the following embodiment), and obtains multiple device names (corresponding to the M display devices in the following embodiment). Then the protocol fusion module 212 merges the multiple device names obtained by the search to obtain a device name list. Each device name in this list corresponds to a display device, and different device names correspond to different display devices. The fused device name list is then displayed to the user, and the user inputs the display device selected by the user for wireless mirroring through the screen display & interaction module 280 (corresponding to the first display device in the following embodiment).

[0050] The transmission management module 213 then determines the screen projection protocol for wireless mirroring (how to determine the screen projection protocol will be described in detail in the following embodiments).

[0051] Finally, the media encapsulation module 220 performs data encapsulation according to the determined protocol. Specifically, the encapsulation control module 221 is used to adjust the timestamp of each frame of data and control the start and pause of the encapsulation process. The format encapsulation module 222 is used to encapsulate the data file into a corresponding bitstream according to the parameter setting of the encapsulation control module 221. In the encapsulation process, the media encapsulation module 220 reads the data file to be encapsulated from the memory 270, and the process of encapsulating the data file into the corresponding bitstream is executed by the processor 260. After the encapsulation is completed, it is sent to the display device by the network interface 250 for display.

[0052] Optionally, during the screen projection process, the terminal device can also receive user instructions through the screen display & interaction module 280 to instruct the screen recording module 230 to perform screen recording during the screen projection process.

[0053] The process of interaction between the network interface 250, the processor 260, the memory 270 and the screen projection application 210 is performed by the operating system ( Figure 2 The control is performed by the controller (not shown) and will not be described in detail here.

[0054] See also Figure 3 , Figure 3 A flowchart of a wireless screen projection method provided in an embodiment of the present application. The method includes step S310, step S320, step S330 and step S340. Among them,

[0055] Step S310: The terminal device obtains M device names, where the M device names are obtained by searching for E display devices based on N screen projection protocols, respectively. M is greater than E, and M, N, and E are positive integers.

[0056] Specifically, the terminal device receives the instruction input by the user and starts the wireless mirroring screen projection service. The terminal device searches for display devices within a certain distance range based on the N screen projection protocols it supports, and obtains M device names. That is, these M device names are obtained by searching for E display devices within a certain range from the terminal device.

[0057] Each of the E display devices corresponds to one or more device names in the M device names.

[0058] Among them, when searching for a display device based on a screen projection protocol, if the display device supports the screen projection protocol, a device name will be obtained. If the display device does not support the screen projection protocol, the corresponding device name cannot be obtained.

[0059] Optionally, the M device names may include the same device name, or the M device names may be different. The specific situation where the M device names include the same device name is as follows:

[0060] (1) When searching for the same display device through multiple screen projection protocols, the device names obtained can be the same or different. That is, when searching for the same display device through different screen projection protocols and the same device name can be obtained, the above M device names contain the same device name.

[0061] (2) When a user uses multiple screen projection programs at the same time, and when the multiple screen projection programs use the same screen projection protocol to search for display devices, for each display device, multiple identical or different device names can be obtained based on the same screen projection protocol, and the multiple device names correspond one-to-one to the multiple screen projection programs. If the same device name is obtained, then the M device names will include the multiple identical device names.

[0062] The following describes the process of obtaining M device names from the perspective of the number of screen projection protocols:

[0063] (1) N is equal to 1

[0064] When a user uses multiple screen projection programs, and the multiple screen projection programs search for display devices using only one screen projection protocol, multiple device names will be obtained for each of the E display devices. These multiple device names correspond to the multiple screen projection programs one by one, and the multiple device names can contain the same device name or be different. At this time, each display device corresponds to multiple device names in the M device names, and M is greater than E.

[0065] (2) N is greater than 1

[0066] When a user uses multiple screen projection programs, and the multiple screen projection programs search for display devices through multiple screen projection protocols, multiple device names will be obtained for each of the E display devices. At this time, each display device corresponds to multiple device names among the M device names, and M is greater than E.

[0067] When a user uses a screen projection program, and the screen projection program searches for display devices through multiple screen projection protocols, one or more device names will be obtained for each of the E display devices. The situation where one display device corresponds to one device name is that the one display device only supports one of the multiple screen projection protocols. At this time, each display device corresponds to one or more device names in the M device names, and when at least one display device supports multiple screen projection protocols, M is greater than E.

[0068] Combine the following Figure 4 Describe the above device search (discovery) process.

[0069] like Figure 4 As shown, the user first initiates the wireless mirroring screen casting service, which can be specifically: the user opens the screen casting application and starts the display device search. Then the terminal device searches for the display device based on the N screen casting protocols it supports, as shown in the figure. Figure 4 As shown: searching based on protocol 1, ..., searching based on protocol N.

[0070] like Figure 4 As shown, after searching based on each protocol, it is determined whether a display device is found. If so, the searched device name is saved, and if not, the search process based on the protocol is terminated.

[0071] Step S320: Generate a device name list according to the M device names, the device name list includes E device names among the M device names, and the E device names correspond to the E display devices respectively.

[0072] The generation process of the above device name list (also referred to as the fusion of device names) is to select E device names from M device names so that the E device names correspond to the E display devices one by one.

[0073] Each of the M device names corresponds to an Internet Protocol (IP) address, and the IP address is obtained during the search process.

[0074] The generation process of the above device name list is specifically as follows, and is described by taking the second display device among the E display devices (any one of the E display devices) as an example:

[0075] First, one or more device names corresponding to the second display device in the M device names are determined. The determination method may be one of the following methods:

[0076] (I) Determined by device identification

[0077] The device name that contains the device identification of the second display device among the M device names is the one or more device names corresponding to the second display device, that is, the one or more device names are obtained by searching for the second display device through one or more screen projection protocols.

[0078] The device identifier may be a device model or other identifier that can uniquely identify a display device, and this application does not limit this. For example, the device identifier may be Z6X PRO.

[0079] (ii) Determined by IP address and device identification

[0080] First, determine the device name that contains the device identification of the second display device and the IP address corresponding to the device name, then compare the IP address with other device names in the M device names to obtain one or more device names, and the one or more device names satisfy: the similarity of the IP addresses corresponding to any two device names is greater than or equal to the first threshold. The one or more device names are the one or more device names corresponding to the second display device in the M device names. At least one device name in the one or more device names contains the device identification of the second display device.

[0081] Optionally, the above method for determining the similarity of two IP addresses may specifically be: comparing the characters at corresponding positions in the two IP addresses in turn to see if they are the same, then counting the proportion of positions with the same characters in the length of the IP addresses, and finally comparing the proportion with the first threshold. In addition, those skilled in the art may also use other methods to compare the above IP address similarity, which is not limited in this application.

[0082] (III) Determined by Media Access Control (MAC) address and device identification

[0083] This method is similar to the second method described above. That is, first determine the device name that contains the device identification of the second display device, and the MAC address corresponding to this device name, and then compare this MAC address with other device names in the M device names to obtain one or more device names, and this one or more device names satisfy: the similarity of the MAC addresses corresponding to any two device names is greater than or equal to the first threshold. This one or more device names are the one or more device names corresponding to the second display device in the M device names. At least one device name in this one or more device names contains the device identification of the second display device.

[0084] The MAC address corresponding to each device name is obtained by converting the corresponding IP address.

[0085] Optionally, the similarity between the two MAC addresses may be determined by comparing with a second threshold value. The specific method may refer to the method for determining the similarity between IP addresses, which will not be described in detail here.

[0086] After determining one or more device names corresponding to the second display device in the above manner, a device name may be selected from the one or more device names and added to the above device name list as the only device name corresponding to the second display device in the device name list.

[0087] Combine the following Figure 5 To describe the process of generating the above device name list.

[0088] like Figure 5 As shown, first, step 510 is executed: obtaining the device names searched based on N protocols. Then, step 520 is executed: determining whether at least one device name is included. If so, step 530 is executed: fusing the searched device names; then step 540 is executed: displaying the device name list obtained after fusion, which includes E device names corresponding to E display devices. This process is to display the device names on the display screen of the terminal device (i.e. Figure 1 The device name list is displayed to the user on the screen display & interaction module 280 in the display module 280 for the user to select a display device for subsequent screen projection. After step 540 is executed, the above list generation process ends. If the judgment result of step 520 is no, the above list generation process ends.

[0089] Specifically, the specific execution process of each of the above steps can refer to the corresponding description in step S320 of the above embodiment, which will not be repeated here.

[0090] Step S330: receiving an indication message input by a user, where the indication message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices.

[0091] Specifically, after the terminal device displays the above device name list to the user through the screen display & interaction module 280, the user clicks on the corresponding device name based on his or her own selection. This click action will generate a corresponding indication message, pointing to the first device name selected by the user. This first device name corresponds to the first device among the E display devices.

[0092] Step S340: The terminal device projects the screen onto the first display device.

[0093] Specifically, the terminal device will select a screen projection protocol from the list of supported screen projection protocols for screen projection. The specific method for selecting the screen projection protocol is:

[0094] Select the first projection protocol in the projection protocol list, and project the screen to the first display device based on the first projection protocol. If the projection is successful, the terminal device starts to send data to the first display device; if the projection fails, the terminal device projects the screen to the first display device through the next projection protocol in the projection protocol list of the first projection protocol, and selects the projection protocol in turn until the projection is successful.

[0095] Among them, when performing the first screen projection, the above-mentioned first screen projection protocol is the first screen projection protocol in the screen projection protocol list.

[0096] Optionally, the projection protocols in the above projection protocol list are sorted based on preset rules. The sorting rules are: (1) the lower the delay of the projection protocol, the higher the ranking of the projection protocol in the projection protocol list; or (2) the higher the success rate of the projection protocol, the higher the ranking of the projection protocol in the projection protocol list. The success rate of the projection protocol is based on historical data statistics; or (3) the larger the projection parameters of the projection protocol, the higher the ranking of the projection protocol in the projection protocol list. Among them, the projection parameters of each projection protocol are obtained by weighted calculation of the delay and success rate of the projection protocol, and the specific weighting method is not limited in this application.

[0097] That is, this application can automatically select the optimal screen projection protocol for screen projection through the above method, thereby improving the screen projection success rate and improving the user experience.

[0098] Combine the following Figure 6 Describes the process of a terminal device automatically selecting a screen projection protocol for screen projection.

[0099] like Figure 6As shown, after the process of selecting the screen projection protocol starts, step 610 is first executed: determine whether the first display device supports screen projection protocol 1. If the output result of step 610 is yes, the terminal device starts to establish a connection link with the first display device based on screen projection protocol 1 and executes step 620: determine whether the screen projection is successful. If the output result of step 620 is yes, step 630 is executed: the terminal device sends a data file to the first display device, and then the screen projection protocol selection process ends. If the output result of step 620 or step 610 is no, step 640 is executed: determine whether the first display device supports screen projection protocol 2. If the output result of step 640 is yes, the terminal device starts to establish a connection link with the first display device based on screen projection protocol 2 and executes step 650: determine whether the screen projection is successful. If the output result of step 650 is yes, step 660 is executed: the terminal device sends a data file to the first display device, and then the screen projection protocol selection process ends. If the output result of step 640 or step 650 is no, the above-mentioned screen projection protocol selection process ends.

[0100] Among them, the above Figure 6 The process describes the above projection protocol selection process by including two projection protocols (projection protocol 1 and projection protocol 2) in the projection protocol list, and cannot be understood as a limitation on the number of projection protocols in the projection protocol list.

[0101] Among them, screen projection protocol 1 and screen projection protocol 2 are the first screen projection protocol and the second screen projection protocol in the screen projection protocol list respectively.

[0102] Optionally, if the DLNA protocol is selected in the above manner and the screen is successfully projected, if the delay between the display image on the terminal device and the display image on the first display device is greater than or equal to the third threshold, the terminal device can adjust the encapsulation control process during data file transmission in the following manner (corresponding to Figure 2 (determined by the media encapsulation module 220).

[0103] (1) Suspend packaging

[0104] Specifically, when the above-mentioned delay is greater than or equal to the third threshold, the terminal device suspends packaging the data file for a preset duration (set by technical personnel in this field according to the specific scenario, and is not limited in this application) to speed up the consumption of cached data files on the first display device, thereby reducing the delay between the terminal device and the display screen on the first display device, and improving the user experience.

[0105] (2) Sampling packaging

[0106] Specifically, when the delay is greater than or equal to the third threshold, the terminal device can select a D frame from the data file containing the C frame, encapsulate the D frame and send it to the first display device, where D is a positive integer less than C. In this way, the consumption of the cached data file on the first display device is accelerated, thereby reducing the delay and improving the user experience.

[0107] Among them, the above sampling method can be randomly selecting C frames, selecting C frames at fixed time intervals, etc., and this application is not limited to this.

[0108] (3) Fast forward playback

[0109] Specifically, when the delay is greater than or equal to the third threshold, the terminal device may adjust the F timestamps corresponding to the F frames in the data file, wherein the F timestamps include the first timestamp, and the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer. The terminal device then encapsulates the adjusted data file and sends it to the first display device.

[0110] Among them, F frame is part or all of the data in the data file.

[0111] The first timestamp is any one of the F timestamps.

[0112] The timestamp corresponding to each frame is used to indicate the playback time of the frame, and the playback time is relative to the start playback time of the data file.

[0113] That is, method (3) reduces latency and improves user experience by speeding up the playback speed of the data file on the first display device.

[0114] In addition, the above three methods for adjusting the packaging control process can be combined arbitrarily. Specifically, the following four packaging control methods can be obtained: method (1) + method (2), method (1) + method (3), method (2) + method (3), and method (1) + method (2) + method (3). The specific implementation process of the combined solution will not be repeated here.

[0115] Combine the following Figure 7 This section describes the process of adjusting the encapsulation control process when a terminal device performs screen projection based on the DLNA protocol.

[0116] like Figure 7As shown, first execute step 710: send the data file based on the DLNA protocol. Then execute step 720: determine whether the screen projection process based on the DLNA protocol is ended. If so, end this process. If not, execute step 730: calculate the end-to-end delay, that is, the delay between the display screen on the terminal device and the display screen on the first display device. Then execute step 740: determine whether the delay is less than the third threshold. If not, execute step 750: adjust the encapsulation control process, that is, the seven methods listed in the above embodiment. If so, return to execute step 710.

[0117] Specifically, the specific execution process of the above steps can be found in the description of the above embodiments, which will not be repeated here.

[0118] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of a wireless screen projection device provided in an embodiment of the present application. Figure 8 As shown, the device includes a search unit 810, a processing unit 820, an interaction unit 830 and a screen projection unit 840.

[0119] The search unit 810 is used to obtain M device names, where the M device names are obtained by searching for E display devices based on N screen projection protocols, respectively, where M is greater than E, and M, N, and E are positive integers; the processing unit 820 is used to generate a device name list based on the M device names, where the device name list includes E device names among the M device names, and the E device names correspond to the E display devices respectively; the interaction unit 830 is used to receive an indication message input by a user, where the indication message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices; the screen projection unit 840 is used to project the screen to the first display device.

[0120] In a feasible implementation, the processing unit 820 is specifically used to: obtain one or more device names corresponding to the second display device from the M device names; wherein the one or more device names all contain the device identification of the second display device, or the similarity of the network protocol IP addresses corresponding to any two of the one or more device names is greater than or equal to a first threshold, and at least one of the one or more device names contains the device identification of the second display device, or the similarity of the media access control MAC addresses corresponding to any two of the one or more device names is greater than or equal to a second threshold, and at least one of the one or more device names contains the device identification of the second display device; the device name corresponding to the second display device in the device name list is one of the one or more device names.

[0121] In a feasible implementation, the screen projection unit 840 is specifically used to: project the screen to the first display device based on the first screen projection protocol in the screen projection protocol list; if the screen projection based on the first screen projection protocol fails, project the screen to the first display device based on the next screen projection protocol in the screen projection protocol list based on the first screen projection protocol; wherein, when performing the first screen projection, the first screen projection protocol is the first screen projection protocol in the screen projection protocol list, and the screen projection protocol list is sorted based on the delay of the screen projection protocol or the screen projection success rate. The lower the delay of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list, or the higher the projection success rate of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list.

[0122] In a feasible implementation, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit 840 is also used to: when the delay between the display screen on the terminal device and the display screen on the first display device is greater than or equal to a third threshold, stop packaging the data file sent to the first display device, and the duration of the stop packaging is a preset duration.

[0123] In a feasible implementation, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit 840 is also used to: when the delay between the display image on the terminal device and the display image on the first display device is greater than or equal to a third threshold, select a D frame from a data file containing a C frame, encapsulate the D frame and send it to the first display device, where D is a positive integer less than C.

[0124] In a feasible implementation, when the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit 840 is also used to: when the delay between the display image on the terminal device and the display image on the first display device is greater than or equal to a third threshold, adjust the F timestamps corresponding to the F frames in the data file, the F timestamps include a first timestamp, the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; encapsulate the adjusted data file and send it to the first display device.

[0125] Specifically, the specific execution process of the above-mentioned wireless screen projection device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0126] See also Fig. 9 , Fig. 9 A schematic diagram of a terminal device provided in an embodiment of the present application, as Figure 8 A specific implementation of the wireless screen projection device. Fig. 9 As shown, the terminal device includes a processor 901, at least one memory 902, a radio frequency circuit 903, and an interface circuit 905; wherein the processor 901, the memory 902, the radio frequency circuit 903, and the interface circuit 905 are connected via a bus 904.

[0127] The radio frequency circuit 903, as a component of the antenna on the terminal device, is used to search for E display devices based on N screen projection protocols, respectively, so as to obtain M device names. The processor 901 is used to generate a device name list based on the M device names, and the device name list contains E device names among the M device names, and the E device names correspond to the E display devices respectively. The processor 901 is also used to receive an indication message input by the user through the screen display & interaction module on the terminal device, and the indication message points to the first device name among the E device names, and the first device name corresponds to the first display device among the E display devices. The interface circuit 905 is used to send a screen projection data file to the first display device. The memory 902 is used to store the above-mentioned M device names and the device name list.

[0128] Specifically, the specific execution process of the above-mentioned terminal device can refer to the corresponding execution process in the aforementioned method embodiment, which will not be repeated here.

[0129] An embodiment of the present application provides a chip system, which includes at least one processor, a memory and an interface circuit, wherein the memory, the interface circuit and the at least one processor are interconnected through lines, and instructions are stored in the at least one memory; when the instructions are executed by the processor, part or all of the steps of any one of the above-mentioned wireless screen projection method embodiments are implemented.

[0130] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, some or all of the steps recorded in any one of the above-mentioned wireless screen projection method embodiments can be implemented.

[0131] An embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by a processor, some or all of the steps recorded in any one of the above-mentioned wireless screen projection method embodiments are implemented.

[0132] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited to the described order of actions, because according to this application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0133] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0134] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A wireless screen projection method, characterized in that: The method comprises: The terminal device obtains M device names, where the M device names are obtained by searching for E display devices based on N screen projection protocols, respectively, where M is greater than E, and M, N, and E are positive integers; Generate a device name list according to the M device names, the device name list including E device names among the M device names, the E device names respectively corresponding to the E display devices; receiving an indication message input by a user, wherein the indication message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices; The terminal device projects the screen onto the first display device.

2. The method according to claim 1, characterized in that The E display devices include a second display device, and generating a device name list according to the M device names includes: Obtain one or more device names corresponding to the second display device from the M device names; wherein the one or more device names all contain the device identification of the second display device, or the similarity of the network protocol IP addresses respectively corresponding to any two of the one or more device names is greater than or equal to a first threshold, and at least one of the one or more device names contains the device identification of the second display device, or the similarity of the media access control MAC addresses respectively corresponding to any two of the one or more device names is greater than or equal to a second threshold, and at least one of the one or more device names contains the device identification of the second display device; The device name corresponding to the second display device in the device name list is one of the one or more device names.

3. The method according to claim 1 or 2, characterized in that: The terminal device projects a screen onto the first display device, including: Casting the screen to the first display device based on the first screen casting protocol in the screen casting protocol list; If screen projection based on the first screen projection protocol fails, screen projection is performed to the first display device based on the next screen projection protocol of the first screen projection protocol in the screen projection protocol list; Among them, when performing screen projection for the first time, the first screen projection protocol is the first screen projection protocol in the screen projection protocol list, and the screen projection protocol list is sorted based on the delay or the projection success rate of the screen projection protocol. The lower the delay of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list, or the higher the projection success rate of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list.

4. The method according to any one of claims 1 to 3, characterized in that When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: When the time delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, the terminal device stops packaging the data file sent to the first display device, and the duration of the stop packaging is a preset duration.

5. The method according to any one of claims 1 to 3, characterized in that: When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: When the delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, a D frame is selected from a data file containing a C frame, and the D frame is encapsulated and sent to the first display device, where D is a positive integer less than C.

6. The method according to any one of claims 1 to 3, characterized in that When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the method further includes: When the delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, adjusting F timestamps corresponding to F frames in the data file, wherein the F timestamps include a first timestamp, and the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; The terminal device encapsulates the adjusted data file and sends it to the first display device.

7. A wireless screen projection device, characterized in that: The device comprises: A search unit, used to obtain M device names, where the M device names are obtained by searching for E display devices based on N screen projection protocols, respectively, where M is greater than E, and M, N, and E are positive integers; a processing unit, configured to generate a device name list according to the M device names, wherein the device name list includes E device names among the M device names, and the E device names correspond to the E display devices respectively; an interaction unit, configured to receive an indication message input by a user, wherein the indication message points to a first device name among the E device names, and the first device name corresponds to a first display device among the E display devices; A screen projection unit, used for projecting the screen to the first display device.

8. The device according to claim 7, characterized in that The processing unit is specifically used for: Obtain one or more device names corresponding to the second display device from the M device names; wherein the one or more device names all contain the device identification of the second display device, or the similarity of the network protocol IP addresses respectively corresponding to any two of the one or more device names is greater than or equal to a first threshold, and at least one of the one or more device names contains the device identification of the second display device, or the similarity of the media access control MAC addresses respectively corresponding to any two of the one or more device names is greater than or equal to a second threshold, and at least one of the one or more device names contains the device identification of the second display device; The device name corresponding to the second display device in the device name list is one of the one or more device names.

9. The device according to claim 7 or 8, characterized in that The screen projection unit is specifically used for: Casting the screen to the first display device based on the first screen casting protocol in the screen casting protocol list; If screen projection based on the first screen projection protocol fails, screen projection is performed to the first display device based on the next screen projection protocol of the first screen projection protocol in the screen projection protocol list; Among them, when performing screen projection for the first time, the first screen projection protocol is the first screen projection protocol in the screen projection protocol list, and the screen projection protocol list is sorted based on the delay or the projection success rate of the screen projection protocol. The lower the delay of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list, or the higher the projection success rate of the screen projection protocol, the higher the ranking of the screen projection protocol in the screen projection protocol list.

10. The device according to any one of claims 7 to 9, characterized in that: When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is further used for: When the time delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, the packaging of the data file sent to the first display device is stopped, and the duration of the stopping of packaging is a preset duration.

11. The device according to any one of claims 7 to 9, characterized in that: When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is further used for: When the delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, a D frame is selected from a data file containing a C frame, and the D frame is encapsulated and sent to the first display device, where D is a positive integer less than C.

12. The device according to any one of claims 7 to 9, characterized in that When the selected screen projection protocol is the Digital Living Network Alliance DLNA protocol, the screen projection unit is further used for: When the delay between the picture displayed on the terminal device and the picture displayed on the first display device is greater than or equal to a third threshold, adjusting F timestamps corresponding to F frames in the data file, wherein the F timestamps include a first timestamp, and the time indicated by the adjusted first timestamp is earlier than the time indicated by the first timestamp, and F is a positive integer; The adjusted data file is packaged and sent to the first display device.

13. A terminal device, characterized in that: The terminal device includes at least one processor, a memory and an interface circuit. The memory, the interface circuit and the at least one processor are interconnected via lines. Instructions are stored in the at least one memory. When the instructions are executed by the processor, any method described in claims 1 to 6 is implemented.

14. A chip system, characterized in that: The chip system includes at least one processor, a memory and an interface circuit. The memory, the interface circuit and the at least one processor are interconnected by lines. Instructions are stored in the at least one memory. When the instructions are executed by the processor, any method described in claims 1-6 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 6 is implemented.

16. A computer program product, characterized in that The computer program product comprises instructions, and when the instructions are executed, the method according to any one of claims 1 to 6 is implemented.

Images