Screen projection data transmission method, device, electronic device and storage medium

The method addresses network instability in screen projection data transmission by determining a target receiving time and adjusting encoding parameters based on network detection data, ensuring stable data transmission and improved user experience.

CN115278319BActive Publication Date: 2024-12-03HISENSE VISUAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210824415.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-13
Publication Date
2024-12-03
Estimated Expiration
2042-07-13

AI Technical Summary

Technical Problem

Existing methods for solving network status monitoring in screen projection data transmission technologies fail to address the issue of network status monitoring in screen projection data transmission, leading to unstable conditions like delay, jitter, and packet loss, which cause video playback interruptions and poor user experience.

Method used

A method for screen projection data transmission that includes receiving network detection data, determining a target receiving time based on a preset number of times, and adjusting encoding parameters to ensure stable data transmission by evaluating the relationship between the target and preset receiving times, without the need for additional thread-based network monitoring, thus conserving system resources.

Benefits of technology

This approach ensures stable screen projection data transmission by dynamically adjusting encoding parameters, reducing the occurrence of network delays and interruptions, thereby enhancing user experience without wasting system resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115278319B_ABST
    Figure CN115278319B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a screen projection data transmission method, apparatus, electronic device and storage medium, including: receiving screen projection data sent by a first device, wherein the screen projection data at least includes network detection data; determining a target receiving time for receiving the network detection data within a preset number of times; based on the relationship between the target receiving time and the preset receiving time, determining an adjustment strategy for the screen projection data encoding parameters to ensure the normal transmission of the screen projection data between the first device and the second device, and the network detection data is sent based on the screen projection channel established by the first device and the second device. Therefore, there is no need to additionally start a thread on the second device to perform network status detection, thereby avoiding waste of system resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of display technology, and in particular to a method, device, electronic device and storage medium for screen projection data transmission. Background Art

[0002] With the development of technology in the field of computer technology, the emergence of the Internet has brought great convenience to modern life. The rapid development of the field of mobile communication terminals has made the forms of communication terminals more and more diversified, providing more convenience for people's lives. Users can interact with information by using various types of terminal devices. For example, the picture displayed on the mobile phone can be projected onto the TV screen. With the help of network transmission, the picture information displayed on the mobile phone can be sent to the TV in the format of frames. After the TV receives the picture information transmitted by the mobile phone, it will play the picture, thereby realizing the function of screen projection.

[0003] Usually, the transmission of screen projection data between the screen projection data source and the video playback end is data transmission under the local area network. During the data transmission process, there will be unstable conditions such as delay, jitter, packet loss, etc. caused by the network, which will cause video playback to freeze or even screen projection to be interrupted, giving users a very bad experience.

[0004] In the prior art, in order to dynamically monitor the network status, most of them start a thread on the video playback end, continuously call the Ping command (Packet Internet Groper), Ping the IP address (Internet Protocol Address) of the screen projection data source, obtain the return time of the Ping command, and determine whether the network status is abnormal by judging the return time of the Ping command.

[0005] However, in the prior art, the above detection method will occupy system resources and cause waste of system resources. Summary of the invention

[0006] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a screen projection data transmission method, device, electronic device and storage medium.

[0007] In a first aspect, an embodiment of the present disclosure provides a screen projection data transmission method, including:

[0008] Receiving screen projection data sent by the first device, wherein the screen projection data at least includes network detection data;

[0009] Determining a target receiving time for receiving the network detection data within a preset number of times;

[0010] Based on the relationship between the target receiving time and the preset receiving time, an adjustment strategy for the projection data encoding parameters is determined.

[0011] As an implementable method, optionally, determining a target receiving time for receiving the network detection data within a preset number of times includes:

[0012] When the number of times the network detection data is received is equal to a preset number, sequentially obtaining the receiving time of each of the network detection data;

[0013] Based on the obtained reception time of each of the network detection data, determine the time difference between the reception time when the network detection data is received for the Nth time and the reception time when the network detection data is received for the N-1th time, where N is an integer greater than or equal to 1;

[0014] The target receiving time is determined according to the time difference and the preset number of times.

[0015] As an implementable method, optionally, determining the target receiving time according to the time difference and the preset number of times includes:

[0016] The target time difference is determined by summing the time difference between the time when the network detection data is received for the Nth time and the time when the network detection data is received for the N-1th time;

[0017] The target time difference is divided by a preset number of times to determine the target receiving time.

[0018] As an implementable method, optionally, the screen projection data encoding parameters include a data encoding rate and a data resolution;

[0019] The step of determining an adjustment strategy for screen projection data encoding parameters according to a relationship between the target receiving time and the preset receiving time includes:

[0020] When the target receiving time is greater than the preset receiving time, an instruction is sent to the first device to instruct the first device to reduce the data encoding rate and / or data resolution.

[0021] As an implementable method, optionally, when sending the instruction to the first device, the method further includes:

[0022] Sending prompt information to the first device and / or the second device so that the prompt information is displayed through the first device and / or the second device.

[0023] As an implementable method, optionally, before sending the instruction to the first device, the method further includes:

[0024] Acquire a network bandwidth of a communication channel between the second device and the first device;

[0025] When the network bandwidth of the communication channel is 2.4G, controlling the second device to switch to a communication channel with a network bandwidth of 5.0G;

[0026] Determine the relationship between the target receiving time and the preset receiving time of receiving the network detection data;

[0027] The sending of the instruction to the first device includes:

[0028] When the target receiving time is greater than or equal to the preset receiving time, an instruction is sent to the first device to instruct the first device to reduce the data encoding rate and / or data resolution.

[0029] As an implementable method, optionally, receiving the screen projection data sent by the first device includes:

[0030] When the video data and / or audio data played by the first device is obtained as dynamic data, the screen projection data sent by the first device is received.

[0031] In a second aspect, an embodiment of the present disclosure provides a screen projection data transmission device, comprising:

[0032] A receiving module, configured to receive screen projection data sent by the first device, wherein the screen projection data at least includes network detection data;

[0033] A target receiving time determination module, used to determine a target receiving time for receiving the network detection data within a preset number of times;

[0034] The adjustment parameter determination module is used to determine the adjustment strategy of the projection data encoding parameters based on the relationship between the target receiving time and the preset receiving time.

[0035] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0036] one or more processors;

[0037] a storage device for storing one or more programs,

[0038] When the one or more programs are executed by the one or more processors, the one or more processors implement any method as described in the first aspect.

[0039] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any method described in the first aspect.

[0040] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0041] The embodiments of the present disclosure provide a screen projection data transmission method, device, electronic device and storage medium, which receive screen projection data sent by a first device, wherein the screen projection data at least includes network detection data; determine a target receiving time for receiving the network detection data; and determine an adjustment strategy for the encoding parameters of the screen projection data based on the relationship between the target receiving time and the preset receiving time, that is, a second device receives the screen projection data sent by the first device, wherein the screen projection data includes network detection data. After each time the second device receives the network detection data, the second device determines the target receiving time for receiving the network detection data based on the current system time of the second device. If the network between the first device and the second device is unobstructed, the time interval of each received network detection data should be within the preset receiving time range. If the network between the first device and the second device is abnormal, the time interval of each received network detection data will be greatly different from the preset receiving time. By evaluating the relationship between the target receiving time for receiving the network detection data and the preset receiving time, the network status between the first device and the second device is determined, and then the adjustment strategy for the encoding parameters of the screen projection data is determined to ensure the normal transmission of the screen projection data between the first device and the second device. The network detection data is sent based on the screen projection channel established by the first device and the second device. Therefore, there is no need to additionally start a thread on the second device to detect the network status, thereby avoiding waste of system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1A This is a system network architecture diagram of a screen projection data transmission method provided by an embodiment of the present disclosure;

[0045] Figure 1B is a schematic diagram of the structure of a terminal device provided by an embodiment of the present disclosure;

[0046] Figure 2A It is a flowchart of a screen projection data transmission method provided by an embodiment of the present disclosure;

[0047] Figure 2BIt is a signal flow diagram of a screen projection data transmission method provided by an embodiment of the present disclosure;

[0048] Figure 3A It is a flowchart of another screen projection data transmission method provided by an embodiment of the present disclosure;

[0049] Figure 3B This is a signal flow diagram of another screen projection data transmission method provided by an embodiment of the present disclosure;

[0050] Figure 3C is a schematic diagram of screen projection data provided by an embodiment of the present disclosure;

[0051] Figure 3D is a schematic diagram of another screen projection data provided by an embodiment of the present disclosure;

[0052] Figure 4A It is a flowchart of another method for transmitting screen projection data provided by an embodiment of the present disclosure;

[0053] Figure 4B This is a signal flow diagram of another method for screen projection data transmission provided by an embodiment of the present disclosure;

[0054] Figure 5A It is a flowchart of another method for transmitting screen projection data provided by an embodiment of the present disclosure;

[0055] Figure 5B This is a signal flow diagram of another method for screen projection data transmission provided by an embodiment of the present disclosure;

[0056] Fig. 6A It is a flowchart of another method for transmitting screen projection data provided by an embodiment of the present disclosure;

[0057] Figure 6B This is a signal flow diagram of another method for screen projection data transmission provided by an embodiment of the present disclosure;

[0058] Figure 7 It is a structural schematic diagram of a screen projection data transmission device provided by an embodiment of the present disclosure;

[0059] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0060] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0061] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0062] Figure 1A The network architecture diagram of the screen projection system according to the screen projection data transmission method of the embodiment of the present application is schematically shown. The network architecture diagram of the screen projection system includes a first device 10, a second device 20, a router 30 and a server 40, etc. Among them: the first device can be used as a sending end in the network architecture of the screen projection system, and is used to send multimedia content and various instructions to the second device. In an exemplary embodiment, the first device includes, but is not limited to, a smart phone, a tablet computer, a game console, a laptop computer, etc. The second device can be used as a display end in the network architecture of the screen projection system, for receiving multimedia content and various instructions of the first device, and synchronously playing the received multimedia content to the display panel of its own device or the display panel of an associated device. The second device can be a related device with a larger screen such as a virtual reality device, a set-top box, a smart TV, a vehicle terminal, a large outdoor display panel, etc., and of course, it can also be a related device with a smaller screen such as a smart phone, a tablet computer, a laptop computer, etc.

[0063] The router can be used as a gateway in the network architecture of the projection system to form a local area network and provide Internet services. The router can provide a local connection between the first device and the second device.

[0064] The server can be used as a cloud device in the projection system network architecture to provide cloud services, such as cloud account management services, etc. The server can be a rack server, blade server, tower server or cabinet server (including an independent server or a server cluster composed of multiple servers), etc.

[0065] The first device can discover the second device through the first network or the second network, and establish a network connection with the second device based on the first network or the second network. Through the established network connection, the first device can project the multimedia content to be projected to the second device, and play it through the second device. The first network can be a local area network, and the local area network can be a wired local area network or a wireless local area network. Figure 1A As shown, the wireless local area network can be a Wi-Fi network formed by router 8. The second network can be a wide area network, such as the Internet. The wide area network can include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, etc. The wide area network can also include wireless links, such as cellular links, satellite links, etc.

[0066] In an exemplary embodiment, Figure 1BAs shown, the first device and the second device include at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0067] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output.

[0068] The display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from the controller, and display video content, image content, and menu control interface components and user control UI interface.

[0069] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0070] The communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0071] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote controller, etc.).

[0072] The detector 230 is used to collect signals from the external environment or the external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0073] The external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0074] The tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0075] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0076] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0077] In some embodiments, the controller includes a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM Random Access Memory (RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0078] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

[0079] "User interface" is the medium interface for interaction and information exchange between applications or operating systems and users. It realizes the conversion between the internal form of information and the form acceptable to users. The commonly used form of user interface is the Graphical User Interface (GUI), which refers to the user interface related to computer operation displayed in a graphical way. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.

[0080] The above briefly introduces the functions of the first device and the second device.

[0081] The first device can perform screen projection operations on the second device.

[0082] However, in the prior art, during the process of projecting the screen from the first device to the second device, the transmission of the projected screen data between the first device and the second device is data transmission under the local area network. During the data transmission process, network delays, jitters, packet loss, etc. may occur, resulting in a freeze in the playback of the projected screen data or even interruption of the projected screen, which reduces the user experience. In the prior art, generally a thread is opened on the second device, and the Ping command is continuously called to Ping the IP address of the first device. Based on the return time of the Ping command, the network status between the first device and the second device is determined. However, in the prior art, since the second device needs to start a thread, the system resources of the second device will be occupied, resulting in a waste of system resources.

[0083] The embodiment of the present disclosure provides a screen projection data transmission method, which receives the screen projection data sent by the first device, wherein the screen projection data at least includes network detection data; determines a target receiving time for receiving the network detection data; and determines an adjustment strategy for the encoding parameters of the screen projection data based on the relationship between the target receiving time and the preset receiving time, that is, the second device receives the screen projection data sent by the first device, wherein the screen projection data includes the network detection data. After each time the second device receives the network detection data, the second device determines the target receiving time for receiving the network detection data based on the current system time of the second device. If the network between the first device and the second device is unobstructed, the time interval of each received network detection data should be within the preset receiving time range. If the network between the first device and the second device is abnormal, the time interval of each received network detection data will be greatly different from the preset receiving time. By evaluating the relationship between the target receiving time for receiving the network detection data and the preset receiving time, the network status between the first device and the second device is determined, and then the adjustment strategy for the encoding parameters of the screen projection data is determined to ensure the normal transmission of the screen projection data between the first device and the second device. The network detection data is sent based on the screen projection channel established by the first device and the second device. Therefore, there is no need to additionally start a thread on the second device to perform network status detection, thereby avoiding waste of system resources.

[0084] Figure 2A is a flowchart of a screen projection data transmission method provided by an embodiment of the present disclosure. Figure 2B This is a signal flow diagram of a screen projection data transmission method provided by an embodiment of the present disclosure, combined with Figure 2A and Figure 2B , the screen projection data transmission method includes:

[0085] S100. Receive screen projection data sent by the first device.

[0086] Among them, the screen projection data at least includes network detection data.

[0087] Specifically, in the scenario of screen projection data transmission, the first device sends the screen projection data to the second device. The first device and the second device will establish a communication channel based on a preset communication protocol, and send the screen projection data through the communication channel.

[0088] As an implementable embodiment, the communication protocol between the first device and the second device adopts the TCP protocol.

[0089] When the first device and the second device are executing operations in various stages such as connection, transmission and reception, and disconnection, the screen projection data exchanged between the two parties includes video data, audio data, and network detection data, among which the video data, audio data, and network detection data are sent through the same communication channel.

[0090] S200: Determine a target receiving time for receiving network detection data within a preset number of times.

[0091] The target receiving time of the network detection data refers to the average receiving time value of the network detection data received within a preset number of times.

[0092] In a specific implementation, during the process of the first device sending projection data to the second device, the user can customize the period of sending network detection data on the first device, for example, sending a network detection data every 30ms. Since the network detection data, video data and audio data are sent to the second device based on the same communication channel, when the first device triggers the sending of a video data at the first moment, then triggers the sending of a network detection data at the second moment, and continues to trigger the sending of a network detection data at the third moment, wherein the time difference between the second moment and the third moment is 30ms, the first device may trigger the sending of audio data and / or video data between the second moment and the third moment, or may not trigger the sending of audio data and / or video data. For example, when the screen of the first device does not change, the sending of video data is not triggered, and when the first device does not play sound, the sending of audio data is not triggered. When the first device triggers sending audio data and / or video data to the second device between the second moment and the third moment, if the network between the first device and the second device is abnormal, the audio data and / or video data sent between the second moment and the third moment will be delayed due to network reasons. Since the audio data, video data and network detection data are sent through a communication channel, the network delay causes the sending of audio data and / or video data to affect the second device's receipt of the network detection data sent at the third moment. According to the time when the second device receives the network detection data sent by the first device at the second moment and the time when it receives the network detection data sent by the first device at the third moment, the target receiving time for the second device to receive the network detection data can be determined.

[0093] It should be noted that in the above embodiment, the preset number of times is 1 time by way of example. In other possible implementation modes, the preset number of times may also be N times. When the preset number of times is N times, the receiving time of each network detection data is obtained in sequence; based on the obtained receiving time of each network detection data, the time difference between the receiving time when the network detection data is received for the Nth time and the receiving time when the network detection data is received for the N-1th time is determined, and then the target receiving time is determined according to the time difference and the preset number of times.

[0094] S300. Determine an adjustment strategy for screen projection data encoding parameters based on a relationship between a target receiving time and a preset receiving time.

[0095] By analyzing the relationship between the target receiving time and the preset receiving time of the network detection data received by the second device, the network status of the screen projection channel between the first device and the second device can be determined, and then the adjustment strategy of the screen projection data encoding parameters can be determined. By adjusting the screen projection data encoding parameters, it is ensured that the second device can normally receive the video data and / or audio data sent by the first device, avoiding the first device and the second device from encountering network delays, jitters, packet loss and other unstable conditions during data transmission, which may cause the video data or audio data to be played stuck, or even the screen projection to be interrupted, thereby reducing the user experience.

[0096] The preset receiving time is a period for the first device to send network detection data that is customized by the user. For example, the first device sends network detection data to the second device once every 30 ms.

[0097] The screen projection data transmission method provided by the embodiment of the present disclosure receives the screen projection data sent by the first device, wherein the screen projection data at least includes network detection data; determines the target receiving time for receiving the network detection data; and determines the adjustment strategy of the screen projection data encoding parameters based on the relationship between the target receiving time and the preset receiving time, that is, the second device receives the screen projection data sent by the first device, and the screen projection data includes network detection data. After each time the second device receives the network detection data, the target receiving time for receiving the network detection data is determined based on the current system time of the second device. If the network between the first device and the second device is unobstructed, the time interval of each received network detection data should be within the preset receiving time range. If the network between the first device and the second device is abnormal, the time interval of each received network detection data will be greatly different from the preset receiving time. By evaluating the relationship between the target receiving time for receiving the network detection data and the preset receiving time, the network status between the first device and the second device is determined, and then the adjustment strategy of the screen projection data encoding parameters is determined to ensure the normal transmission of the screen projection data between the first device and the second device. The network detection data is sent based on the screen projection channel established by the first device and the second device. Therefore, there is no need to additionally start a thread on the second device to detect the network status, thereby avoiding waste of system resources.

[0098] Figure 3A is a flowchart of another method for transmitting screen projection data provided by an embodiment of the present disclosure. Figure 3B is a signal flow diagram of another method for screen projection data transmission provided by an embodiment of the present disclosure, combined with Figure 3A and Figure 3B The specific implementation of step S200 includes:

[0099] S201. When the number of times network detection data is received is equal to a preset number, sequentially obtain the receiving time of each network detection data.

[0100] In a specific implementation, in order to avoid over-frequent calculation of the target reception time of network detection data, the number of times the second device receives the network detection data is determined. When the number of times the second device receives the network detection data is less than a preset number of times, the step of determining the target reception time for receiving the network detection data within the preset number of times is not executed, thereby avoiding frequent calculation of the target reception time of the network detection data and causing a waste of system resources.

[0101] When the number of times the second device receives the network detection data is equal to the preset number of times, at this time, the receiving time corresponding to each time the network detection data is received within the preset number of times is obtained.

[0102] Exemplarily, the preset number of times is set to 30 times. When the second device detects that the network detection data is received for the 30th time, the second device obtains the time of the previous 30 times of receiving the network detection data from the server in sequence.

[0103] S202: Based on the acquired reception time of each network detection data, determine the time difference between the reception time when the network detection data is received for the Nth time and the reception time when the network detection data is received for the N-1th time.

[0104] Wherein, N is an integer greater than or equal to 1.

[0105] By calculating the time difference between two adjacent times of receiving network detection data, that is, the time difference between the first time of receiving the network detection data and the thirtieth time of receiving the network detection data within the last preset number of times stored in the server of the second device, the time difference between the second time of receiving the network detection data and the first time of receiving the network detection data, ..., and the time difference between the thirtieth time of receiving the network detection data and the twenty-ninth time of receiving the network detection data.

[0106] Specifically, the server of the second device stores the time corresponding to each receipt of network detection data, such as Figure 3C Or as shown in 3D, when it is necessary to determine the target receiving time of receiving the network detection data within a preset number of times, the time corresponding to each reception of the network detection data within the preset number of times can be obtained from the server.

[0107] It should be noted that Figure 3C The exemplary method indicates that the target receiving time of receiving the network detection data within the preset number of times is very average, which means that the second device basically receives the data every 30 milliseconds for the data sent by the first device every 30 milliseconds. At this time, it indicates that the network between the first device and the second device is unobstructed. In another possible implementation, if Figure 3D As shown, the target receiving time for the second device to receive the network detection data within a preset number of times is discrete. For example, the target receiving time for receiving the network detection data for the first time is 130ms, the target receiving time for receiving the network detection data for the second time is 0ms,..., the target receiving time for receiving the network detection data for the fourth time is 0ms. The reason for the above phenomenon is that the first four network detection packets are received at one time after a delay of 130 milliseconds due to network congestion, so the receiving time of the second, third, and fourth data is 0 milliseconds. However, as long as the average target receiving time for receiving the network detection data each time is 30ms within the preset number of times, it can be characterized that the network between the first device and the second device is unobstructed.

[0108] S203: Determine a target receiving time according to the time difference and a preset number of times.

[0109] As a specific feasible implementation method, determining the target receiving time according to the time difference and the preset number of times includes:

[0110] The target time difference is determined by summing the time difference between the receiving time of the Nth receiving network detection data and the receiving time of the N-1th receiving network detection data; and the target time difference is divided by a preset number of times to determine the target receiving time.

[0111] Exemplary, combined Figure 3C After obtaining the time T1 corresponding to the first time network detection data is received, the time T2 corresponding to the second time network detection data is received, ..., and the time T30 corresponding to the thirtieth time network detection data is received, it can be determined that within the preset number of times, the time difference between the receiving time of the Nth time network detection data is received and the receiving time of the N-1th time network detection data is received is TN-TN-1, and then the N obtained time differences are summed to determine the target time difference T = (T1-T30') + (T2-T1) + ... + (T30-T29), where T30' is the time corresponding to the thirtieth time network detection data is received within the last preset number of times. After determining the target time difference, the target time difference is divided by the preset number of times to determine the target receiving time, that is, the target receiving time t = T / 30.

[0112] The screen projection data transmission method provided by the embodiment of the present disclosure determines the target receiving time by determining the difference between the receiving time when the network detection data is received for the Nth time and the receiving time when the network detection data is received for the N-1th time and the preset number of times based on the received time of each network detection data when the number of times the network detection data is received is equal to the preset number of times. That is, by determining the target receiving time when the preset number of times is met, it is avoided to determine a target receiving time every time a network detection data is received, thereby causing the second device to frequently calculate the second target receiving time, thereby causing a waste of system resources.

[0113] Figure 4A is a flowchart of another method for transmitting screen projection data provided by an embodiment of the present disclosure. Figure 4B is a signal flow diagram of another method for screen projection data transmission provided by an embodiment of the present disclosure. The embodiment of the present disclosure is based on the above embodiment. Figure 4A and Figure 4B As shown, the specific implementation of step S300 includes:

[0114] S303: When the target receiving time is greater than the preset receiving time, sending an instruction to the first device to instruct the first device to reduce the data coding rate and / or data resolution.

[0115] Generally speaking, the projection data encoding parameters include data encoding rate, data resolution, etc. Among them, the encoding rate refers to the size of data encoded by the encoder per second, in kbps, such as 800kbps means that the encoder generates 800kb (or 100KB) of data per second. Resolution refers to the number of pixels contained in the image per inch. The adjustment strategy of the projection data encoding parameters is a different parameter adjustment method formulated according to the estimated degree of change of the network bandwidth. For example, without affecting the user's extreme subjective clarity experience and the jamming experience, when the estimated network bandwidth is reduced to a small extent, the data encoding bit rate is adjusted first; when the estimated network bandwidth is reduced to a large extent, the encoding rate and resolution can be adjusted at the same time, so as to achieve the effect of reducing bandwidth occupancy, so as to reduce the jamming of the projection video data or audio data. The specific values ​​of each parameter adjustment can pre-establish a correlation function relationship between a parameter value and a network bandwidth change value, and determine the specific parameter adjustment value based on the function. After determining the specific data encoding parameter adjustment strategy, the first device uses the adjusted new data encoding parameters to encode the projection data that has not yet been encoded, and sends it to the second device after encoding. Of course, in the process of encoding the projection data that has not yet been encoded based on the adjusted new data encoding parameters, network detection data will also be sent to estimate the network bandwidth and adjust the data encoding parameters in time, so that the projection can be quickly restored due to uncontrollable sudden instability of the network, reducing the situation of projection freezes or even projection interruptions, and improving the user experience.

[0116] Figure 5A is a flowchart of another method for transmitting screen projection data provided by an embodiment of the present disclosure. Figure 5B is a signal flow diagram of another method for screen projection data transmission provided by an embodiment of the present disclosure. The embodiment of the present disclosure is based on the above embodiment. Figure 5A and Figure 5B As shown, when the screen projection data transmission method executes step S303, it also includes:

[0117] S304: Send prompt information to the first device and / or the second device, and display the prompt information through the first device and / or the second device.

[0118] As an implementable embodiment, exemplarily, when the second device detects that the network of the first device and the second device is stuck, a prompt message is returned to the first device through the communication channel, so that a prompt message "Network stuck is detected. You can optimize the screen projection stuck by selecting the projection data encoding parameter option" is popped up on the display screen of the first device, thereby prompting the user to choose to adjust the projection data encoding parameters on the first device. At this time, the first device re-encodes the projection data based on the adjusted projection data encoding parameters and sends it to the second device.

[0119] As another possible implementation mode, exemplarily, when the second device detects that the network of the first device and the second device is stuck, a prompt message is output, so that a prompt message "Network stuck is detected. You can optimize the screen projection stuck by selecting a low resolution option" is popped up on the display screen of the second device, thereby prompting the user to select the projection data encoding parameters corresponding to the received projection data on the second device, and return the selected projection data encoding parameters to the first device through the communication channel. After receiving the projection data encoding parameters returned by the communication channel, the first device re-encodes the projection data according to the projection data encoding parameters selected by the user and sends it to the second device.

[0120] Fig. 6A is a flowchart of another method for transmitting screen projection data provided by an embodiment of the present disclosure. Figure 6B is a signal flow diagram of another method for screen projection data transmission provided by an embodiment of the present disclosure. Figure 4A Based on the corresponding embodiments, Fig. 6A and Figure 6B As shown, before sending the instruction to the first device, the step also includes:

[0121] S301: Obtain a network bandwidth of a communication channel between a second device and a first device.

[0122] Among them, network bandwidth refers to the maximum data rate that can pass from the first device to the second device in a unit time. Network bandwidth estimation can be understood as determining the changing trend of network bandwidth in the next period of time. Network bandwidth and data transmission time have opposite changing trends. The smaller the network bandwidth, the longer the data transmission, and the larger the network bandwidth, the shorter the data transmission. In a specific embodiment, when it is determined that the target receiving time is greater than the preset receiving time, at this time, it can be determined that the network status of the screen projection channel between the first device and the second device is abnormal, and the reason for the abnormal network status may be that the network bandwidth of the communication channel between the first device and the second device is small. Therefore, the network bandwidth of the communication channel between the first device and the second device can be obtained first, and the network bandwidth corresponding to the communication channel between the first device and the second device can be determined. Whether the abnormal network status of the first device and the second device is caused by the low connection network bandwidth is determined, thereby avoiding directly adjusting the screen projection data encoding parameters so that the resolution of the screen projection data received by the second device is low and affects the user's experience.

[0123] S302: When the network bandwidth of the communication channel is 2.4G, control the second device to switch to a communication channel with a network bandwidth of 5.0G.

[0124] When the network bandwidth of the communication channel between the second device and the first device is obtained to be 2.4G, the second device can be controlled to switch to a communication channel with a 5.0G network bandwidth, thereby increasing the network bandwidth of the communication channel between the second device and the first device and improving data transmission efficiency. This can avoid directly adjusting the projection data encoding parameters to improve data transmission efficiency, which may result in a lower resolution of the projection data received by the second device, affecting the user experience.

[0125] At this time, the specific implementation of step S303 includes:

[0126] S3030. After switching to the communication channel of the 5.0G network bandwidth, determine the relationship between the target receiving time and the preset receiving time for receiving the network detection data.

[0127] After improving the transmission efficiency of the screen projection data between the first device and the second device by adjusting the network bandwidth of the communication channel, in order to avoid the problem of network abnormalities in the communication channel between the first device and the second device even if the network bandwidth is increased, after switching to the communication channel with a 5.0G network bandwidth, the relationship between the target receiving time and the preset receiving time for receiving the network detection data is determined.

[0128] S3031. When the target receiving time is greater than the preset receiving time, send an instruction to the first device to instruct the first device to reduce the data coding rate and / or data resolution.

[0129] When the target receiving time is greater than the preset receiving time, it indicates that there is still a problem of network abnormality in the communication channel after adjusting the network bandwidth. At this time, an instruction is sent to the first device to instruct the first device to reduce the data encoding rate and / or data resolution, thereby improving data transmission efficiency and avoiding problems such as freezes in the projection data, which affects the user experience.

[0130] As an implementable embodiment, optionally, when it is acquired that the video data and / or audio data played by the first device is dynamic data, the screen projection data sent by the first device is received.

[0131] In order to minimize the network burden, when the video data and / or audio data played by the first device does not change, the first device pauses screen recording and no longer generates projection data. Only when the video data and / or audio data played by the first device changes, projection data is generated, and video encoding and / or audio encoding are performed, and sent to the second device through the communication channel. At this time, the projection data sent by the first device is received, and the network status of the communication channel between the first device and the second device is detected based on the network detection data included in the projection data.

[0132] Figure 7 is a structural diagram of a screen projection data transmission device provided by an embodiment of the present disclosure, such as Figure 7 As shown, the screen projection data transmission device includes:

[0133] The receiving module 710 is used to receive the screen projection data sent by the first device, wherein the screen projection data at least includes network detection data;

[0134] The target receiving time determination module 720 is used to determine the target receiving time of receiving the network detection data within a preset number of times;

[0135] The adjustment parameter determination module 730 is used to determine the adjustment strategy of the projection data encoding parameters based on the relationship between the target receiving time and the preset receiving time.

[0136] The embodiment of the present disclosure provides a screen projection data transmission device, which receives the screen projection data sent by the first device, wherein the screen projection data at least includes network detection data; determines a target receiving time for receiving the network detection data; and determines an adjustment strategy for the encoding parameters of the screen projection data based on the relationship between the target receiving time and the preset receiving time, that is, the second device receives the screen projection data sent by the first device, wherein the screen projection data includes the network detection data. After each time the second device receives the network detection data, the second device determines the target receiving time for receiving the network detection data based on the current system time of the second device. If the network between the first device and the second device is unobstructed, the time interval of each received network detection data should be within the preset receiving time range. If the network between the first device and the second device is abnormal, the time interval of each received network detection data will be greatly different from the preset receiving time. By evaluating the relationship between the target receiving time for receiving the network detection data and the preset receiving time, the network status between the first device and the second device is determined, and then the adjustment strategy for the encoding parameters of the screen projection data is determined to ensure the normal transmission of the screen projection data between the first device and the second device. The network detection data is sent based on the screen projection channel established by the first device and the second device. Therefore, there is no need to additionally start a thread on the second device to detect the network status, thereby avoiding waste of system resources.

[0137] As an implementable method, optionally, the target reception time determination module includes:

[0138] A receiving time determining unit, configured to sequentially obtain a receiving time of each network detection data when the number of times the network detection data is received is equal to a preset number;

[0139] a time difference determination unit, configured to determine, based on the acquired reception time of each network detection data, a time difference between a reception time when the network detection data is received for the Nth time and a reception time when the network detection data is received for the Nth time-1st time, wherein N is an integer greater than or equal to 1;

[0140] The target receiving time determining unit is used to determine the target receiving time according to the time difference and the preset number of times.

[0141] As an implementable method, optionally, the specific implementation method of the target reception time determination unit includes:

[0142] The target time difference is determined by summing the time difference between the time when the network detection data is received for the Nth time and the time when the network detection data is received for the N-1th time;

[0143] The target time difference is divided by the preset number of times to determine the target receiving time.

[0144] As an implementable method, optionally, the adjustment parameter determination module includes:

[0145] The instruction sending unit is used to send an instruction to the first device when the target receiving time is greater than the preset receiving time, so as to instruct the first device to reduce the data encoding rate and / or data resolution.

[0146] As an implementable method, optionally, it further includes:

[0147] The prompt information sending module is used to send the prompt information to the first device and / or the second device so that the prompt information is displayed through the first device and / or the second device.

[0148] As an implementable method, optionally, it further includes:

[0149] A network bandwidth acquisition module, used to acquire the network bandwidth of the communication channel between the second device and the first device;

[0150] A communication channel switching module, used to control the second device to switch to a communication channel with a network bandwidth of 5.0G when the network bandwidth of the communication channel is 2.4G;

[0151] At this time, the specific implementation of the instruction sending unit includes:

[0152] After switching to the communication channel of the 5.0G network bandwidth, determine the relationship between the target receiving time and the preset receiving time for receiving the network detection data;

[0153] When the target receiving time is greater than or equal to the preset receiving time, an instruction is sent to the first device to instruct the first device to reduce the data encoding rate and / or data resolution.

[0154] The device provided by the embodiment of the present invention can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0155] It is worth noting that in the embodiment of the above-mentioned device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0156] The present disclosure further provides an electronic device, comprising: a processor, wherein the processor is configured to execute a computer program stored in a memory, wherein the computer program implements the steps of the above method embodiment when executed by the processor.

[0157] Figure 8 A schematic diagram of the structure of an electronic device provided by the present disclosure is shown in FIG. Figure 8 A block diagram of an exemplary electronic device suitable for implementing an exemplary embodiment of the present invention is shown. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0158] like Figure 8 As shown, the electronic device 800 is in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: one or more processors 810, a system memory 820, and a bus 830 connecting different system components (including the system memory 820 and the processor).

[0159] Bus 830 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0160] The electronic device 800 typically includes a variety of computer system readable media. These media can be any media that can be accessed by the electronic device 800, including volatile and non-volatile media, removable and non-removable media.

[0161] The system memory 820 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 840 and / or cache memory 850. The electronic device 800 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 860 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as "hard drives"). A disk drive for reading and writing removable non-volatile disks (such as "floppy disks"), and an optical disk drive for reading and writing removable non-volatile optical disks (such as CD-ROMs, DVD-ROMs or other optical media) may be provided. In these cases, each drive may be connected to the bus 830 via one or more data medium interfaces. The system memory 820 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the embodiments of the present invention.

[0162] A program / utility 880 having a set (at least one) of program modules 870 may be stored, for example, in system memory 820, such program modules 870 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 870 generally perform the functions and / or methods of the embodiments described in the embodiments of the present invention.

[0163] The processor 810 executes various functional applications and information processing by running at least one program among a plurality of programs stored in the system memory 820 , for example, implementing the method embodiment provided by the embodiment of the present invention.

[0164] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above method embodiment when executed by a processor.

[0165] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0166] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0167] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0168] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN) domain, or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0169] The present disclosure also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the steps of implementing the above method embodiments.

[0170] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0171] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A screen projection data transmission method, characterized in that: include: Receiving screen projection data sent by the first device, wherein the screen projection data at least includes network detection data; Determine a target receiving time for receiving the network detection data within a preset number of times; the target receiving time for the network detection data is an average receiving time value for receiving the network detection data within a preset number of times; Based on the relationship between the target receiving time and the preset receiving time, the adjustment strategy of the screen projection data encoding parameters is determined; the preset receiving time is the period for the first device to send the network detection data.

2. The method according to claim 1, characterized in that The determining of a target receiving time for receiving the network detection data within a preset number of times includes: When the number of times the network detection data is received is equal to a preset number, sequentially obtaining the receiving time of each of the network detection data; Based on the obtained reception time of each of the network detection data, determine the time difference between the reception time when the network detection data is received for the Nth time and the reception time when the network detection data is received for the N-1th time, where N is an integer greater than or equal to 1; The target receiving time is determined according to the time difference and the preset number of times.

3. The method according to claim 2, characterized in that The step of determining the target receiving time according to the time difference and the preset number of times includes: The target time difference is determined by summing the time difference between the time when the network detection data is received for the Nth time and the time when the network detection data is received for the N-1th time; The target time difference is divided by a preset number of times to determine the target receiving time.

4. The method according to claim 1, characterized in that: The projection data encoding parameters include data encoding rate and data resolution; The step of determining an adjustment strategy for screen projection data encoding parameters according to a relationship between the target receiving time and the preset receiving time includes: When the target receiving time is greater than the preset receiving time, an instruction is sent to the first device to instruct the first device to reduce the data encoding rate and / or data resolution.

5. The method according to claim 4, characterized in that The sending of the instruction to the first device also includes: Sending prompt information to the first device and / or the second device so that the prompt information is displayed through the first device and / or the second device.

6. The method according to claim 4, characterized in that Before sending the instruction to the first device, the method further includes: Acquire a network bandwidth of a communication channel between a second device and the first device; When the network bandwidth of the communication channel is 2.4G, controlling the second device to switch to a communication channel with a network bandwidth of 5.0G; Determine the relationship between the target receiving time and the preset receiving time of receiving the network detection data; The sending of the instruction to the first device includes: When the target receiving time is greater than or equal to the preset receiving time, an instruction is sent to the first device to instruct the first device to reduce the data encoding rate and / or data resolution.

7. The method according to claim 1, characterized in that The receiving the screen projection data sent by the first device includes: When the video data and / or audio data played by the first device is obtained as dynamic data, the screen projection data sent by the first device is received.

8. A screen projection data transmission device, characterized in that: include: A receiving module, configured to receive screen projection data sent by the first device, wherein the screen projection data at least includes network detection data; A target receiving time determination module is used to determine a target receiving time for receiving the network detection data within a preset number of times; the target receiving time for the network detection data is an average receiving time value for receiving the network detection data within a preset number of times; An adjustment parameter determination module is used to determine an adjustment strategy for screen projection data encoding parameters based on the relationship between the target receiving time and the preset receiving time; the preset receiving time is the period for the first device to send the network detection data.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.