Audio and video stream transmission method, system and cloud server based on streaming parameter adaptation

Abstract

The present invention relates to an audio and video stream transmission method based on streaming parameter adaptation and its system, a cloud server of a cloud machine service, and a computer-readable storage medium, including: identifying the currently running application process of a first cloud machine client; determining the current cloud application scenario through the currently running application process; querying the back-end platform whether there are streaming parameters matching the current cloud application scenario; if so, obtaining the matching streaming parameters, updating the encoding parameters of the encoder according to the matching streaming parameters, so that the encoder encodes the audio and video stream transmitted to the first cloud machine client based on the updated encoding parameters, and transmitting the encoded audio and video stream to the first cloud machine client according to the matching streaming parameters. The above method can provide the cloud machine client with streaming parameters adapted to the cloud application scenario, optimize network parameters, avoid network congestion and network packet loss problems, and provide users with audio and video streams that match their needs.

Description

Technical Field

[0001] The present invention relates to the field of audio and video stream processing, and in particular to an audio and video stream transmission method based on streaming parameter adaptation and a system thereof, a cloud server for cloud machine end services, and a computer-readable storage medium. Background Art

[0002] At present, cloud technology has been very mature and has been widely used, such as cloud computers, cloud desktops, cloud services, etc. Cloud phones proposed for mobile phones have also been fully developed. Existing cloud phones generally include clients and cloud servers. Clients can include mobile phones, tablets, computers, etc. The basic requirements are that the client APP can be installed and there is a network. The client connects to the cloud server through the network. When in use, the client runs the APP to connect to the cloud server, that is, based on the application of cloud phones, people can use mobile phones to operate several cloud phones with one client and control several mobile phone control interfaces with one client. As the roles played by users themselves are different, they need to deal with various different things in daily life. Many users carry multiple mobile phones at the same time to deal with different things according to different usage needs. For example, the multiple mobile phones carried include mobile phones specifically for making and receiving calls, mobile phones specifically for handling company affairs, and mobile phones specifically for handling private life trivia. However, carrying multiple mobile phones according to different usage needs will cause inconvenience to users, and they cannot obtain all the required files and information from one mobile phone at any time. Different mobile phones must be opened to obtain different files and information. Through cloud phones, the convenience of users can be improved and the security of user data can be protected.

[0003] When using a cloud phone, the backend platform provides the user's cloud client with the device information and streaming parameters of the cloud service. The cloud client connects to the cloud service through the device information, and the cloud service provides the cloud client with audio and video streams through fixed streaming parameters. However, when using the cloud client, users often switch between different applications, and different applications have different requirements for different networks. The cloud service uses fixed streaming parameters to provide users with audio and video streams, which may cause network congestion and network packet loss problems. The audio and video streams provided to users do not match their scenario requirements, resulting in a poor user experience. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an audio and video stream transmission method based on streaming parameter adaptation and its system, a cloud server for cloud machine services and a computer-readable storage medium, which can provide streaming parameters adapted to the cloud machine client based on cloud application scenarios, optimize network parameters, avoid network congestion and network packet loss problems, provide users with audio and video streams that match their needs, and improve user experience.

[0005] In order to solve at least one of the above technical problems, an embodiment of the present invention provides an audio and video stream transmission method based on stream parameter adaptation, which is applied to cloud machine services. The audio and video stream transmission method includes:

[0006] Identify the currently running application process of the first cloud client;

[0007] Determine the current cloud application scenario through the currently running application process;

[0008] Querying the backend platform whether there are streaming parameters matching the current cloud application scenario;

[0009] If so, obtain the matching streaming parameters, update the encoding parameters of the encoder according to the matching streaming parameters, so that the encoder encodes the audio and video stream transmitted to the first cloud client based on the updated encoding parameters, and transmits the encoded audio and video stream to the first cloud client according to the matching streaming parameters.

[0010] Preferably, when the backend platform receives the cloud machine streaming service request from the first cloud machine client, it returns the device information of the cloud machine service and the recommended streaming parameters to the first cloud machine client;

[0011] The first cloud client initiates a connection request to the cloud service according to the device information of the cloud service and the recommended streaming parameters, so that the cloud service encodes the audio and video stream according to the recommended streaming parameters and returns the encoded audio and video stream to the first cloud client.

[0012] Preferably, the streaming parameters include bit rate, resolution and frame rate;

[0013] The querying the backend platform whether there are streaming parameters matching the current cloud application scenario includes:

[0014] When the current cloud application scenario is a video application, obtaining first streaming parameters, where the first streaming parameters include a first bit rate, a first resolution, and a first frame rate;

[0015] When the current cloud application scenario is a game application, obtaining second streaming parameters, where the second streaming parameters include a second bit rate, a second resolution, and a second frame rate;

[0016] When the current cloud application scenario is a tool application, obtaining third streaming parameters, wherein the third streaming parameters include a third bit rate, a third resolution, and a third frame rate;

[0017] Among them, the first resolution is greater than the second resolution and greater than the third resolution, the second bit rate is greater than the first bit rate and greater than the third bit rate, the first frame rate is greater than the third frame rate, and the second frame rate is greater than the third frame rate.

[0018] Preferably, when the current cloud application scenario is a tool application, the audio and video stream transmission method further includes:

[0019] When it is detected that the current network quality does not meet the set condition, obtaining a fourth bit rate and a fourth resolution, wherein the fourth bit rate is lower than the third bit rate, and the fourth resolution is lower than the third resolution;

[0020] The encoding parameters of the encoder are updated using the fourth bit rate, and the encoded audio and video stream is transmitted to the first cloud client based on the fourth resolution.

[0021] Preferably, the back-end platform obtains user non-sensitive network performance data reported in real time by the first cloud client, determines the current optimal streaming parameters based on the user non-sensitive network performance data, and upon receiving a cloud streaming service request from a second cloud client, returns the device information of the cloud service and the current optimal streaming parameters to the second cloud client.

[0022] Preferably, the user's non-sensitive network performance data includes time, IP, region, specification information, networking type, RTT, real-time bit rate, frame rate, terminal decoding capability, and real-time encoding parameters of cloud-end services.

[0023] Preferably, the back-end platform receives a cloud streaming service request from a second cloud client, identifies that the second cloud client's networking type, region, specification information, and time match those of the first cloud client, and returns the device information of the cloud service and the current optimal streaming parameters to the second cloud client.

[0024] An audio and video stream transmission system based on streaming parameter adaptation, including a cloud machine client, a cloud machine service and a backend platform;

[0025] The cloud machine client sends a cloud machine streaming service request to the backend platform;

[0026] The backend platform returns the device information of the cloud machine service and the recommended streaming parameters to the cloud machine client;

[0027] The cloud client initiates a connection request to the cloud service according to the device information of the cloud service and the recommended streaming parameters;

[0028] The cloud machine service encodes the audio and video stream according to the recommended streaming parameters and returns the encoded audio and video stream to the cloud machine client;

[0029] The cloud machine service identifies the currently running application process of the machine client, determines the current cloud application scenario through the currently running application process, and queries the backend platform whether there are streaming parameters matching the current cloud application scenario;

[0030] The backend platform sends matching streaming parameters to the cloud machine service;

[0031] The cloud machine service updates the encoding parameters of the encoder according to the matching streaming parameters, so that the encoder encodes the audio and video stream transmitted to the cloud machine client based on the updated encoding parameters, and transmits the encoded audio and video stream to the cloud machine client according to the matching streaming parameters.

[0032] A cloud server for cloud machine services includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any of the above embodiments when executing the computer program.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0034] The above-mentioned method for transmitting audio and video streams based on streaming parameter adaptation and its system, cloud server of cloud machine service and computer-readable storage medium include identifying the currently running application process of the first cloud machine client; determining the current cloud application scenario through the currently running application process; querying the back-end platform whether there are streaming parameters matching the current cloud application scenario; if so, obtaining the matching streaming parameters, updating the encoding parameters of the encoder according to the matching streaming parameters, so that the encoder encodes the audio and video stream transmitted to the first cloud machine client based on the updated encoding parameters, and transmits the encoded audio and video stream to the first cloud machine client according to the matching streaming parameters. Therefore, it is possible to provide the cloud machine client with streaming parameters that are adapted to it based on the cloud application scenario, optimize network parameters, avoid network congestion and network packet loss problems, provide users with audio and video streams that match their needs, and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of an application scenario of an audio and video stream transmission system based on stream parameter adaptation in an embodiment of the present invention;

[0036] Figure 2is a flow chart of an audio and video stream transmission method based on stream parameter adaptation in an embodiment of the present invention;

[0037] Figure 3 It is a schematic diagram of an application scenario of an audio and video stream transmission system based on streaming parameter adaptation in another embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The embodiment of the present invention provides an audio and video stream transmission system based on stream parameter adaptation. Figure 1 As shown, an audio and video stream transmission system based on streaming parameter adaptation includes a cloud machine client, a cloud machine service and a backend platform. The cloud machine client sends a cloud machine streaming service request to the backend platform; the backend platform returns the device information of the cloud machine service and the recommended streaming parameters to the cloud machine client; the cloud machine client initiates a connection request to the cloud machine service based on the device information of the cloud machine service and the recommended streaming parameters; the cloud machine service encodes the audio and video stream according to the recommended streaming parameters and returns the encoded audio and video stream to the cloud machine client; the cloud machine service identifies the currently running application process of the client, determines the current cloud application scenario through the currently running application process, and queries the backend platform whether there are streaming parameters matching the current cloud application scenario; the backend platform sends matching streaming parameters to the cloud machine service; the cloud machine service updates the encoding parameters of the encoder based on the matching streaming parameters, so that the encoder encodes the audio and video stream transmitted to the cloud machine client based on the updated encoding parameters, and transmits the encoded audio and video stream to the cloud machine client according to the matching streaming parameters.

[0040] For details, see the information interaction between the cloud client, cloud service, and backend platform. Figure 1 As shown. When the cloud machine client switches to different applications, the cloud machine service will automatically detect the application process running in the foreground, and query the backend platform for the recommended streaming parameter combination corresponding to the currently running application process. If there is a match, the streaming parameters will be switched to achieve the purpose of scenario-based encoding and transmission. The cloud machine service will make corresponding encoder settings changes and stream transmission control based on the matching streaming parameter combination determined from the backend platform.

[0041] The embodiment of the present invention provides an audio and video stream transmission method based on stream parameter adaptation, which is applied to Figure 1 The application shown is for cloud machine services. Figure 2 As shown, a method for transmitting audio and video streams based on stream parameter adaptation includes the following steps:

[0042] S201, identifying the currently running application process of the first cloud client.

[0043] Specifically, when the first cloud machine client switches to different applications, the cloud machine service will automatically detect the application process running in the foreground of the first cloud machine client. The application process includes video applications, game applications, and tool applications.

[0044] In one example, before step S201, when the back-end platform receives a cloud streaming service request from the first cloud client, it returns the device information of the cloud service and recommended streaming parameters to the first cloud client; the first cloud client initiates a connection request to the cloud service based on the device information of the cloud service and the recommended streaming parameters, so that the cloud service encodes the audio and video stream according to the recommended streaming parameters and returns the encoded audio and video stream to the first cloud client.

[0045] Specifically, Figure 1 As shown, the back-end platform presets several groups of streaming parameters, which may be streaming encoding parameters. When the cloud machine client sends the cloud phone streaming service to the back-end platform, the back-end platform returns the device information of the cloud machine service and the recommended streaming parameter information to the cloud machine client. The cloud machine client connects to the cloud machine. After the cloud machine service sets the encoder parameters according to the recommended streaming parameters, it returns the encoded audio and video streaming data to the cloud machine client. The cloud machine client decodes the audio and video stream and plays the audio and video stream. At the same time, the cloud machine service periodically detects the application scenario running in the foreground of the cloud machine client and executes step S201.

[0046] S202: Determine a current cloud application scenario through the currently running application process.

[0047] Specifically, different application processes correspond to different cloud application scenarios, such as video applications, game applications, and tool applications, which correspond to different current cloud application scenarios of the first cloud client.

[0048] S203: Query the backend platform whether there are streaming parameters matching the current cloud application scenario.

[0049] Specifically, the backend platform pre-stores streaming parameters matching various application scenarios. The cloud machine service queries the backend platform whether there are streaming parameters matching the current cloud application scenario.

[0050] In one example, the streaming parameters include bit rate, resolution and frame rate; the querying the back-end platform whether there are streaming parameters matching the current cloud application scenario includes: when the current cloud application scenario is a video application, obtaining first streaming parameters, the first streaming parameters including a first bit rate, a first resolution and a first frame rate; when the current cloud application scenario is a game application, obtaining second streaming parameters, the second streaming parameters including a second bit rate, a second resolution and a second frame rate; when the current cloud application scenario is a tool application, obtaining third streaming parameters, the third streaming parameters including a third bit rate, a third resolution and a third frame rate; wherein the first resolution is greater than the second resolution and greater than the third resolution, the second bit rate is greater than the first bit rate and greater than the third bit rate, the first frame rate is greater than the third frame rate, and the second frame rate is greater than the third frame rate.

[0051] Specifically, streaming parameters include a combination of bitrate, resolution, and frame rate. In different application scenarios, the requirements for audio and video communication quality are different. For example, video applications have high requirements for clarity, game applications have high requirements for latency, and tool applications have low requirements for frame rate. Therefore, I use different streaming parameter combinations for different scenarios.

[0052] In one example, when the current cloud application scenario is a tool application, the audio and video stream transmission method also includes: when it is detected that the current network quality does not meet the set conditions, obtaining a fourth bit rate and a fourth resolution, the fourth bit rate is smaller than the third bit rate, and the fourth resolution is smaller than the third resolution; using the fourth bit rate to update the encoding parameters of the encoder, and transmitting the encoded audio and video stream to the first cloud client based on the fourth resolution.

[0053] Specifically, when the current cloud application scenario is a tool application, if the current network conditions are not good, the resolution and bit rate can be actively reduced to avoid lag in the first cloud client.

[0054] S204: if it exists, obtain the matching streaming parameters, update the encoding parameters of the encoder according to the matching streaming parameters, so that the encoder encodes the audio and video stream transmitted to the first cloud client based on the updated encoding parameters, and transmits the encoded audio and video stream to the first cloud client according to the matching streaming parameters.

[0055] Specifically, if there are streaming parameters matching the current cloud application scenario, the streaming parameters matching the current cloud application scenario sent from the backend platform are received, and the encoding parameters of the encoder are updated according to the matching streaming parameters. The encoder encodes the audio and video stream transmitted to the first cloud client based on the updated encoding parameters, and transmits the encoded audio and video stream to the first cloud client according to the matching streaming parameters.

[0056] The above-mentioned method for transmitting audio and video streams based on streaming parameter adaptation includes identifying the currently running application process of the first cloud machine client; determining the current cloud application scenario through the currently running application process; querying the back-end platform whether there are streaming parameters matching the current cloud application scenario; if so, obtaining the matching streaming parameters, updating the encoding parameters of the encoder according to the matching streaming parameters, so that the encoder encodes the audio and video stream transmitted to the first cloud machine client based on the updated encoding parameters, and transmits the encoded audio and video stream to the first cloud machine client according to the matching streaming parameters. Therefore, it is possible to provide the cloud machine client with streaming parameters that are adapted to the cloud application scenario, optimize network parameters, avoid network congestion and network packet loss problems, provide users with audio and video streams that match their needs, and improve user experience.

[0057] In one example, the backend platform obtains the user-insensitive network performance data reported in real time by the first cloud client, determines the current optimal streaming parameters based on the user-insensitive network performance data, and returns the device information of the cloud service and the current optimal streaming parameters to the second cloud client upon receiving the cloud streaming service request from the second cloud client. The user-insensitive network performance data includes time, IP, region, specification information, network type, RTT, real-time bit rate, frame rate, terminal decoding capability, and real-time encoding parameters of the cloud service.

[0058] Furthermore, when the back-end platform receives a cloud streaming service request from a second cloud client and identifies that the networking type, region, specification information and time of the second cloud client match the networking type, region, specification information and time of the first cloud client, it returns the device information of the cloud service and the current optimal streaming parameters to the second cloud client.

[0059] Specifically, Figure 3As shown, the first cloud client is cloud client A, and the second cloud client is cloud client B. During the cloud streaming, all user non-sensitive network performance data will be collected from cloud client A. Such as time, IP, region, network type, RTT, real-time bit rate, frame rate, terminal decoding capability, real-time encoding parameters of the cloud end, etc. Cloud client A regularly reports the above data to the big data module of the back-end platform for data storage, classification, and statistical analysis. Based on the above big data module, the back-end platform will identify the corresponding specific network types, regions, and times, determine the most optimized streaming parameters, and recommend them to the newly initiated client streaming request, so that cloud client B can obtain appropriate encoding transmission parameters, optimize the response to network congestion, and can timely perceive and adjust the streaming bit rate to avoid congestion, reduce the retransmission of packet loss and the increase in delay caused by congestion, and obtain an optimized balance from delay, fluency, and clarity to improve user experience.

[0060] A computer-readable storage medium is provided in an embodiment of the present invention, and an application is stored on the computer-readable storage medium. When the program is executed by a processor, an audio and video stream transmission method based on stream parameter adaptation of any one of the above embodiments is implemented. The computer-readable storage medium includes but is not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (for example, a computer, a mobile phone), which can be a read-only memory, a disk or an optical disk, etc.

[0061] An embodiment of the present invention further provides a computer application program, which runs on a computer and is used to execute an audio and video stream transmission method based on stream parameter adaptation according to any one of the above embodiments.

[0062] An embodiment of the present invention also provides a cloud server for cloud machine services, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of any one of the above-mentioned audio and video stream transmission methods based on streaming parameter adaptation are implemented.

[0063] In addition, the above describes in detail an audio and video stream transmission method based on streaming parameter adaptation and its system, a cloud server for cloud machine service, and a computer-readable storage medium provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for transmitting audio and video streams based on stream parameter adaptation, characterized in that: Applied to cloud machine services, the audio and video stream transmission method includes: Identify the currently running application process of the first cloud client; Determine the current cloud application scenario through the currently running application process; Querying the backend platform whether there are streaming parameters matching the current cloud application scenario; If so, obtaining the matching streaming parameters, updating the encoding parameters of the encoder according to the matching streaming parameters, so that the encoder encodes the audio and video stream transmitted to the first cloud client based on the updated encoding parameters, and transmitting the encoded audio and video stream to the first cloud client according to the matching streaming parameters; The back-end platform obtains the user-insensitive network performance data reported in real time by the first cloud client, determines the current optimal streaming parameters corresponding to the first cloud client according to the user-insensitive network performance data, and returns the device information of the cloud client service and the current optimal streaming parameters to the second cloud client upon receiving the cloud streaming service request from the second cloud client; the user-insensitive network performance data includes time, IP, region, specification information, networking type, RTT, real-time bit rate, frame rate, terminal decoding capability and real-time encoding parameters of the cloud client service; The back-end platform receives a cloud streaming service request from a second cloud client, identifies that the second cloud client's networking type, region, specification information, and time match the first cloud client's networking type, region, specification information, and time, and returns the device information of the cloud service and the current optimal streaming parameters to the second cloud client, so that the second cloud client obtains the encoding transmission parameters based on the current optimal streaming parameters.

2. The audio and video stream transmission method according to claim 1, characterized in that: When the backend platform receives the cloud streaming service request from the first cloud client, it returns the device information of the cloud service and the recommended streaming parameters to the first cloud client; The first cloud client initiates a connection request to the cloud service according to the device information of the cloud service and the recommended streaming parameters, so that the cloud service encodes the audio and video stream according to the recommended streaming parameters and returns the encoded audio and video stream to the first cloud client.

3. The audio and video stream transmission method according to claim 1, characterized in that: The streaming parameters include bit rate, resolution and frame rate; The querying the backend platform whether there are streaming parameters matching the current cloud application scenario includes: When the current cloud application scenario is a video application, obtaining first streaming parameters, where the first streaming parameters include a first bit rate, a first resolution, and a first frame rate; When the current cloud application scenario is a game application, obtaining second streaming parameters, where the second streaming parameters include a second bit rate, a second resolution, and a second frame rate; When the current cloud application scenario is a tool application, obtaining third streaming parameters, wherein the third streaming parameters include a third bit rate, a third resolution, and a third frame rate; Among them, the first resolution is greater than the second resolution and greater than the third resolution, the second bit rate is greater than the first bit rate and greater than the third bit rate, the first frame rate is greater than the third frame rate, and the second frame rate is greater than the third frame rate.

4. The audio and video stream transmission method according to claim 3, characterized in that: When the current cloud application scenario is a tool application, the audio and video stream transmission method further includes: When it is detected that the current network quality does not meet the set condition, obtaining a fourth bit rate and a fourth resolution, wherein the fourth bit rate is lower than the third bit rate, and the fourth resolution is lower than the third resolution; The encoding parameters of the encoder are updated using the fourth bit rate, and the encoded audio and video stream is transmitted to the first cloud client based on the fourth resolution.

5. An audio and video stream transmission system based on stream parameter adaptation, characterized in that: Including cloud machine client, cloud machine service and backend platform; The cloud machine client sends a cloud machine streaming service request to the backend platform; The backend platform returns the device information of the cloud machine service and the recommended streaming parameters to the cloud machine client; The cloud client initiates a connection request to the cloud service according to the device information of the cloud service and the recommended streaming parameters; The cloud machine service encodes the audio and video stream according to the recommended streaming parameters and returns the encoded audio and video stream to the cloud machine client; The cloud machine service identifies the currently running application process of the machine client, determines the current cloud application scenario through the currently running application process, and queries the backend platform whether there are streaming parameters matching the current cloud application scenario; The backend platform sends matching streaming parameters to the cloud machine service; The cloud machine service updates the encoding parameters of the encoder according to the matched streaming parameters, so that the encoder encodes the audio and video stream transmitted to the cloud machine client based on the updated encoding parameters, and transmits the encoded audio and video stream to the cloud machine client according to the matched streaming parameters; The back-end platform obtains the user-insensitive network performance data reported in real time by the first cloud client, determines the current optimal streaming parameters corresponding to the first cloud client according to the user-insensitive network performance data, and returns the device information of the cloud client service and the current optimal streaming parameters to the second cloud client upon receiving the cloud streaming service request from the second cloud client; the user-insensitive network performance data includes time, IP, region, specification information, network type, RTT, real-time bit rate, frame rate, terminal decoding capability and real-time encoding parameters of the cloud client service; The back-end platform receives a cloud streaming service request from a second cloud client, identifies that the second cloud client's networking type, region, specification information, and time match the first cloud client's networking type, region, specification information, and time, and returns the device information of the cloud service and the current optimal streaming parameters to the second cloud client, so that the second cloud client obtains the encoding transmission parameters based on the current optimal streaming parameters.

6. A cloud server for cloud machine end services, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images