Data stream processing method and device, electronic equipment and medium

Abstract

The embodiment of the invention discloses a data stream processing method and device, electronic equipment and a medium. When the method is applied to a streaming media server, the method comprises the following steps: acquiring data stream associated data of target equipment connected with the streaming media server; comparing the data flow associated data obtained in the current adjustment period with target data flow associated data, and determining a comparison result; the target data flow associated data comprises data flow associated data acquired in the previous adjustment period and / or a preset value of the data flow associated data; and according to a comparison result, determining a processing mode of the streaming media server on the data stream and / or prompting the stream sending equipment to determine the processing mode on the data stream. According to the scheme, the problem of data stream playing lagging under the condition of network fluctuation or playing equipment processing efficiency fluctuation can be solved, the playing equipment can smoothly receive and play the data stream, and lagging is avoided.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data stream processing method, device, electronic device and medium. Background Art

[0002] In current data transmission, streaming media is increasingly used. Streaming media refers to a media format that transmits audio, video and multimedia files in a streaming manner over the network. Unlike the current online playback format that must be downloaded in full before viewing, streaming media can compress continuous audio and video information and put it on the network server, so that users can watch while downloading without having to wait for the entire file to be downloaded. Streaming media technology is widely used in video on demand, video conferencing, distance education, telemedicine and online live broadcast systems. The streaming media server is the core system of streaming media applications and is the key platform for operators to provide video services to users. The main function of the streaming media server is to collect, cache, schedule, transmit and play streaming media content.

[0003] The streaming device that sends the data stream to the streaming server may experience network instability. In addition, the playback device that receives the data stream forwarded by the streaming server for playback may also experience network instability, which will cause the data stream playback to be stuck and affect the user experience. Summary of the invention

[0004] The embodiments of the present application provide a data stream processing method, device, electronic device and medium to solve the problem of playback jamming caused by data stream transmission or parsing problems.

[0005] According to one aspect of the present application, a data stream processing method is provided, which is applied to a streaming media server, and the method includes:

[0006] In the current adjustment cycle, obtaining data stream associated data of a target device connected to the streaming media server;

[0007] The target device includes a data stream sending device and / or a playback device; the data stream associated data of the data stream sending device includes the network strength of the data stream sending device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device;

[0008] Compare the data flow associated data obtained in the current adjustment cycle with the target data flow associated data to determine a comparison result; wherein the target data flow associated data includes the data flow associated data obtained in the previous adjustment cycle and / or a preset fixed value of the data flow associated data;

[0009] According to the comparison result, the streaming media server determines a processing method for the data stream and / or prompts the streaming device to determine a processing method for the data stream.

[0010] According to another aspect of the present application, a data stream processing method is provided, which is applied to a stream sending device, and the method includes:

[0011] Sending the network strength to the streaming media server, so that the streaming media server determines the processing mode of the data stream by the streaming media server and / or sends prompt information to the streaming device according to the comparison result of the data stream associated data obtained in the current adjustment period and the target data stream associated data;

[0012] The data stream associated data includes the network strength sent by the streaming device and / or the data stream associated data of the playback device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device; the target data stream associated data includes the data stream associated data obtained in the previous adjustment cycle and / or the preset fixed value of the data stream associated data;

[0013] If the prompt information sent by the streaming media server is received, or if it is detected that the data stream is not successfully sent to the streaming media server, a processing method for the data stream is determined.

[0014] According to one aspect of the present application, a data stream processing device is provided, which is configured in a streaming media server, and the device includes:

[0015] A data stream associated data acquisition module, used to acquire data stream associated data of a target device connected to the streaming media server within a current adjustment period;

[0016] The target device includes a data stream sending device and / or a playback device; the data stream associated data of the data stream sending device includes the network strength of the data stream sending device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device;

[0017] A comparison result determination module, used to compare the data flow associated data obtained in the current adjustment cycle with the target data flow associated data to determine the comparison result; wherein the target data flow associated data includes the data flow associated data obtained in the previous adjustment cycle and / or a preset fixed value of the data flow associated data;

[0018] The processing mode determining module is used to determine the processing mode of the streaming media server for the data stream and / or prompt the streaming device to determine the processing mode for the data stream according to the comparison result.

[0019] According to one aspect of the present application, a data stream processing device is provided, which is configured in a stream sending device, and the device includes:

[0020] A network strength sending module, used for sending the network strength to the streaming media server, so that the streaming media server determines the processing mode of the data stream by the streaming media server and / or sends prompt information to the streaming device according to the comparison result of the data stream associated data obtained in the current adjustment cycle and the target data stream associated data;

[0021] The data stream associated data includes the network strength sent by the streaming device and / or the data stream associated data of the playback device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device; the target data stream associated data includes the data stream associated data obtained in the previous adjustment cycle and / or the preset fixed value of the data stream associated data;

[0022] The processing mode determination module is used to determine the processing mode of the data stream if a prompt message sent by the streaming media server is received or if it is detected that the data stream is not successfully sent to the streaming media server.

[0023] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:

[0024] at least one processor; and

[0025] a memory communicatively connected to at least one processor; wherein,

[0026] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the data stream processing method applied to the streaming media server of any embodiment of the present application, or execute the data stream processing method applied to the streaming device of any embodiment of the present application.

[0027] According to another aspect of the present application, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a processor to implement the data stream processing method applied to a streaming media server according to any embodiment of the present application, or to implement the data stream processing method applied to a streaming device according to any embodiment of the present application when executed.

[0028] The technical solution of the embodiment of the present application, when applied to a streaming media server, obtains data stream associated data of a target device connected to the streaming media server within a current adjustment cycle; wherein the target device includes a streaming device and / or a playback device of the data stream; the data stream associated data of the streaming device includes a network strength of the streaming device; the data stream associated data of the playback device includes a network strength of the playback device and / or a decoding rate of the playback device for the data stream; the data stream associated data obtained within the current adjustment cycle is compared with the target data stream associated data to determine a comparison result; wherein the target data stream associated data includes the data stream associated data obtained within a previous adjustment cycle and / or a preset fixed value of the data stream associated data; according to the comparison result, determines a processing method of the streaming media server for the data stream and / or prompts the streaming device to determine a processing method for the data stream. The above scheme can monitor the network strength of the streaming device and / or the network strength and decoding rate of the playback device in real time, and adaptively adjust the streaming media server's processing method for the data stream and / or prompt the streaming device to adjust the processing method for the data stream according to the monitoring results, thereby ensuring the normal transmission and normal decoding and playback of the data stream, and solving the problem of data stream playback jamming.

[0029] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 It is a flowchart of a data stream processing method applied to a streaming media server provided according to Embodiment 1 of the present application;

[0032] Figure 2 is a flow chart of a data stream processing method applied to a streaming media server provided according to Embodiment 2 of the present application;

[0033] Figure 3 is a flow chart of a data stream processing method applied to a streaming media server provided according to Embodiment 3 of the present application;

[0034] Figure 4 It is a flowchart of a data stream processing method applied to a stream sending device provided according to Embodiment 4 of the present application;

[0035] Figure 5 This is a flowchart of a specific implementation method of a data stream processing method provided in Embodiment 5 of the present application;

[0036] Figure 6 It is a structural diagram of a data stream processing device configured in a streaming media server according to Embodiment 6 of the present application;

[0037] Figure 7 It is a structural schematic diagram of a data stream processing device configured on a stream sending device according to Embodiment 7 of the present application;

[0038] Figure 8 It is a structural diagram of an electronic device provided according to Embodiment 8 of the present application. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0040] It should be noted that the terms "first", "second", "third", "fourth", "actual", "preset", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] Embodiment 1

[0042] Figure 1This is a flowchart of a data stream processing method applied to a streaming media server according to the first embodiment of the present application. The embodiment of the present application can be applied to the case where the data stream is forwarded and played through the streaming media server. Typically, it can be applied to the case where the data stream is parsed and played when the network is unstable and / or the data stream parsing rate is unstable. The method can be executed by a data stream processing device, which can be implemented in the form of hardware and / or software. The data stream processing device can be configured in an electronic device, which can be a streaming media server. Figure 1 As shown, the method includes:

[0043] S110: within a current adjustment period, obtaining data stream associated data of a target device connected to the streaming media server.

[0044] Among them, the target device includes a data stream sending device and / or a playback device; the data stream associated data of the streaming device includes the network strength of the streaming device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device.

[0045] Among them, the current adjustment cycle can be any cycle for executing the embodiment of the present application. The target device can be a streaming device that provides a data stream to the streaming media server, or it can be a playback device that receives the data stream sent by the streaming media server for playback. When the target device is a streaming device, the data stream associated data can be the network strength of the streaming device, and when the target device is a playback device, the data stream associated data can be the network strength of the playback device and / or the decoding rate of the playback device to the data stream. The network can be a 4G network, a 5G network, a wifi network, etc., which are not limited here. The unit of network strength is dBm, and the network strength can be expressed as "value + dBm". The streaming media server can only obtain the network strength of the streaming device, or only obtain the network strength of the playback device and / or the decoding rate of the data stream, or obtain the network strength of the streaming device, and also obtain the network strength of the playback device and / or the decoding rate of the data stream. The streaming media server can actively obtain the data stream associated data from the target device, or the target device can actively send the data stream associated data to the streaming media server.

[0046] Exemplarily, the streaming device can monitor its own data stream associated data, such as monitoring the network strength. If the network strength changes, such as becoming stronger or weaker, or the network strength change is greater than a preset threshold, the network strength of the streaming device is sent to the streaming server. The streaming device can also send the network strength to the streaming server at a preset frequency. The playback device can monitor its own data stream associated data, such as monitoring the network strength and / or the decoding rate of the data stream. If the data stream associated data changes, or the change is greater than a preset threshold, the data stream associated data is sent to the streaming server. The playback device can also send the data stream associated data to the streaming server at a preset frequency.

[0047] S120: Compare the data flow associated data acquired in the current adjustment period with the target data flow associated data to determine a comparison result.

[0048] The target data flow associated data includes the data flow associated data acquired in the last adjustment cycle and / or a preset fixed value of the data flow associated data.

[0049] The previous adjustment cycle is a cycle that is earlier than and adjacent to the current adjustment cycle. The preset fixed value of the data stream associated data can be determined according to actual conditions, for example, it can be set to a conventional network strength that can meet the normal transmission of the data stream and / or a conventional decoding rate that ensures smooth playback of the data stream.

[0050] Exemplarily, comparing the data stream association data obtained in the current adjustment cycle with the data stream association data obtained in the previous adjustment cycle can reflect the change of the data stream association data obtained in the current adjustment cycle compared with the previous adjustment cycle, so as to facilitate subsequent adjustments according to the changes. Comparing the data stream association data obtained in the current adjustment cycle with the preset fixed value of the data stream association data can reflect whether the data stream association data obtained in the current adjustment cycle meets the requirements for ensuring normal transmission or normal playback of the data stream, so as to adaptively perform subsequent adjustments to ensure smooth transmission and playback of the data stream.

[0051] S130: Determine, according to the comparison result, a processing method of the streaming media server for the data stream and / or prompt the streaming device to determine a processing method for the data stream.

[0052] Exemplarily, based on the comparison result, it can be predicted whether the current playback device will experience a freeze when playing the data stream, whether it will be smoother, etc., and the streaming media server can adjust the processing method of the data stream accordingly and / or prompt the streaming device to adjust the processing method of the data stream. The processing method of the streaming device to process the data stream can be to directly send the processing method of the streaming device to the streaming device, or to send the comparison result to the streaming device so that the streaming device determines the processing method of the data stream. For example, based on the comparison result, it is determined that at least one of the following situations occurs: the network strength of the streaming device is reduced, the network strength of the playback device is reduced, the decoding rate of the playback device is reduced, etc., and it is predicted that the reduction in network strength and / or decoding rate will cause freezes when the playback device plays the data stream. The streaming media server can adaptively reduce the reading rate of the data stream and / or prompt the streaming device to reduce the clarity of the image or video data stream and the frame rate of the data stream transmission to ensure the smoothness of the data stream transmission.

[0053] For example, if it is determined based on the comparison results that at least one of the following situations occurs: the network strength of the streaming device is improved, the network strength of the playback device is improved, and the decoding rate of the playback device is improved, then in order to improve the fluency of the playback device when playing the data stream, the streaming media server can adaptively increase the reading rate of the data stream and / or prompt the streaming device to improve the clarity and the frame rate of the data stream sending, etc., while other parameters remain unchanged.

[0054] The technical solution of the embodiment of the present application, when applied to a streaming media server, obtains data stream associated data of a target device connected to the streaming media server within a current adjustment cycle; wherein the target device includes a streaming device and / or a playback device of the data stream; the data stream associated data of the streaming device includes a network strength of the streaming device; the data stream associated data of the playback device includes a network strength of the playback device and / or a decoding rate of the playback device for the data stream; the data stream associated data obtained within the current adjustment cycle is compared with the target data stream associated data to determine a comparison result; wherein the target data stream associated data includes the data stream associated data obtained within a previous adjustment cycle and / or a preset fixed value of the data stream associated data; according to the comparison result, determines a processing method of the streaming media server for the data stream and / or prompts the streaming device to determine a processing method for the data stream. The above scheme can monitor the network strength of the streaming device and / or the network strength and decoding rate of the playback device in real time, and adaptively adjust the streaming media server's processing method for the data stream and / or prompt the streaming device to adjust the processing method for the data stream according to the monitoring results, thereby ensuring the normal transmission and normal decoding and playback of the data stream, and solving the problem of data stream playback jamming.

[0055] Embodiment 2

[0056] Figure 2 This is a flow chart of a data stream processing method applied to a streaming media server according to the second embodiment of the present application. The present embodiment is optimized based on the above embodiment. For solutions not described in detail in the present embodiment, please refer to the above embodiment. Figure 2 As shown, the method of the embodiment of the present application specifically includes the following steps:

[0057] S210: within the current adjustment period, obtaining data stream associated data of a target device connected to the streaming media server.

[0058] Among them, the target device includes a data stream sending device and / or a playback device; the data stream associated data of the streaming device includes the network strength of the streaming device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device.

[0059] S220. Compare the data stream associated data obtained in the current adjustment cycle with the target data stream associated data to determine a comparison result; wherein the target data stream associated data includes the data stream associated data obtained in the previous adjustment cycle and / or a preset fixed value of the data stream associated data.

[0060] S230. If the data stream associated data obtained in the current adjustment cycle is less than the target data stream associated data, then the data stream processing parameters of the streaming media server are reduced and / or the streaming device is prompted to reduce the data stream processing parameters; wherein the data stream processing parameters of the streaming media server include a reading rate for the data stream; and the data stream processing parameters of the streaming device include at least one of a sending frame rate, an encoding bit rate, and a resolution of the data stream.

[0061] Exemplarily, if at least one of the following conditions exists: the network strength of the streaming device obtained in the current adjustment cycle is lower than the network strength of the streaming device in the target data stream associated data, the network strength of the playback device obtained in the current cycle is lower than the network strength of the playback device in the target data stream associated data, and the decoding rate of the playback device for the data stream obtained in the current cycle is lower than the decoding rate in the target data stream associated data, it is determined that the reduction in network strength will cause the data stream transmission to slow down, and the reduction in the decoding rate of the playback device for the data stream will cause the playback of the data stream to slow down, causing the playback device to produce a jam when playing the data stream, so the streaming media server can reduce the reading rate of the data stream to enable the data stream to be smoothly and continuously transmitted and played to avoid jamming. And / or an achievable solution for the streaming device: prompt the streaming device to reduce the data stream processing parameters to adaptively reduce the data volume of the data stream sent per unit time, so that the transmission process and playback process of the data stream are smoother. Exemplarily, the streaming device can reduce the frame rate of the data stream, for example, only send I frames, send I frames and some P frames (if there are B frames, send I frames, some P frames and B frames); reduce the encoding bit rate; increase the I frame interval; reduce the resolution, etc. The above method can effectively reduce the amount of data transmitted per unit time, thereby ensuring the smoothness of data transmission. Although it may reduce the clarity of the data stream or the display frame rate, it will not have a significant impact on the user's perception.

[0062] In the embodiment of the present application, if the streaming media server reduces the reading rate of the data stream, the data stream sent by the streaming device cannot be read normally by the streaming media server, and the data stream is accumulated in the buffer of the streaming device. Another possible implementation scheme for the streaming device is that if the streaming device detects that there is a buffer accumulation of the data stream, it triggers the process of reducing the data stream processing parameters.

[0063] In the embodiment of the present application, the data stream processing parameter of the streaming media server is reduced and / or the streaming device is prompted to reduce the data stream processing parameter, which is reduced on the basis that the data stream processing parameter is not lower than or equal to the preset processing parameter lower limit. If the data stream processing parameter has been reduced to the preset processing parameter lower limit, it will not be further reduced.

[0064] S240: If the data stream associated data acquired in the current adjustment period is greater than the target data stream associated data, then the data stream processing parameters of the streaming media server are increased and / or the streaming device is prompted to increase the data stream processing parameters.

[0065] Exemplarily, if at least one of the following conditions exists: the network strength of the streaming device obtained in the current adjustment cycle is higher than the network strength in the target data stream associated data, the network strength of the playback device obtained in the current cycle is higher than the network strength in the target data stream associated data, and the decoding rate of the data stream obtained by the playback device in the current cycle is higher than the decoding rate in the target data stream associated data, then it is determined that the higher network strength can make the data stream transmission smoother, and the higher decoding rate of the data stream by the playback device will improve the smoothness of the data stream playback, so the reading rate of the data stream by the streaming media server can be adaptively increased, so that the data stream can be read and sent to the playback device more quickly and smoothly, and the display frame rate of the playback device can be improved. And / or the streaming device is prompted to increase the data stream processing parameters, so that the data stream can be sent to the streaming media server at a higher rate and forwarded to the playback device, and the display frame rate and / or resolution of the playback device can be improved to obtain a better display effect. For example, on the basis of sending only I frames, sending I frames and some P frames (if there are B frames, then sending I frames, some P frames and B frames), sending all I frames, sending all I frames and P frames (if there are B frames, then sending all I frames, P frames and B frames); increasing the encoding bit rate; increasing the I frame interval; increasing the resolution, etc. In the embodiment of the present application, increasing the data stream processing parameters of the streaming media server and / or prompting the streaming device to increase the data stream processing parameters is based on the fact that the data stream processing parameters have not reached the preset processing parameter upper limit. If it has been increased to the preset processing parameter upper limit, it will not be further increased.

[0066] S250: If the data stream associated data acquired in the current adjustment period is equal to the target data stream associated data, then the data stream processing parameters are not adjusted and / or the stream sending device is prompted not to adjust the data stream processing parameters.

[0067] Exemplarily, if the data stream associated data obtained in the current adjustment cycle is equal to the target data stream associated data, it means that the data stream associated data has not changed. At this time, the data stream processing parameters may not be adjusted and / or the streaming device may be prompted not to adjust the data stream processing parameters, so that the streaming device maintains the current transmission state, the playback device maintains the current parsing state, and the current data stream playback fluency is maintained. In the embodiment of the present application, the streaming device may also be prompted not to adjust the data stream processing parameters if the prompt information is not received.

[0068] The embodiment of the present application provides a data stream processing method, if the data stream associated data obtained in the current adjustment cycle is less than the target data stream associated data, then the data stream processing parameters of the streaming media server are reduced and / or the streaming device is prompted to reduce the data stream processing parameters. If the data stream associated data obtained in the current adjustment cycle is greater than the target data stream associated data, then the data stream processing parameters of the streaming media server are increased and / or the streaming device is prompted to increase the data stream processing parameters. If the data stream associated data obtained in the current adjustment cycle is equal to the target data stream associated data, the data stream processing parameters are not adjusted and / or the streaming device is prompted not to adjust the data stream processing parameters. The above scheme can avoid the problem of playback jamming caused by unstable data stream transmission or unstable parsing, and can improve the smoothness and display quality of data stream playback when the network strength is increased and / or the decoding rate of the playback device is increased, and maintain the smoothness and display quality of data stream playback.

[0069] In an embodiment of the present application, the data stream associated data acquired in the current adjustment period is compared with the target data stream associated data to determine a comparison result, and according to the comparison result, the stream-generating device is prompted to determine a processing method for the data stream, including: if the network strength of the stream-generating device is greater than the target data stream associated data, predicting the data stream processing parameters of the stream-generating device according to the network strength of the stream-generating device and the target data stream associated data; wherein the data stream processing parameters include at least one of the sending frame rate, encoding bit rate and resolution of the data stream;

[0070] The decoding rate of the data stream by the playback device is determined according to the data stream associated data of the playback device. Exemplarily, the process of the streaming media server obtaining the decoding rate of the data stream by the playback device may be: actively obtaining the decoding rate of the data stream by the playback device from the playback device, the playback device actively sending the decoding rate of the data stream by the playback device to the streaming media server, and the streaming media server monitoring the number of times the playback device reads the data stream from the streaming media server within a preset time period.

[0071] According to the data stream processing parameters of the stream-generating device and the decoding rate of the data stream by the playback device, the stream-generating device is prompted to determine whether to increase the data stream processing parameters.

[0072] When the network strength of the streaming device is greater than the target data stream associated data, if the data stream processing parameters of the streaming media server are directly increased without considering the decoding rate of the playback device and / or the streaming device is prompted to increase its own data stream processing parameters, it is possible that the playback device has a low decoding rate and cannot decode the high-speed transmitted data stream in time and play it smoothly. In an embodiment of the present application, the data stream processing parameters of the streaming device are predicted based on the network strength and the target data stream associated data, the decoding rate of the playback device to the data stream is predicted based on the data stream associated data of the playback device, and it is determined whether the sending frame rate in the data stream processing parameters of the streaming device can match the decoding rate of the playback device to the data stream, and it is determined whether the prompting device to increase the data stream processing parameters. For example, if the sending frame rate in the data stream processing parameters of the streaming device is consistent with the decoding rate of the playback device to the data stream, the streaming device is prompted to increase the data stream processing parameters. If they are inconsistent, the streaming device is prompted not to increase the data stream processing parameters, or the sending frame rate in the data stream processing parameters of the streaming device is adjusted to be consistent with the decoding rate of the playback device to the data stream.

[0073] In an embodiment of the present application, after the streaming media server obtains the network strength of the streaming device, as well as the network strength of the playback device and / or the decoding rate of the data stream, the data stream processing parameters of the streaming device can be predicted based on the network strength of the streaming device and the target data stream association data. The decoding rate of the data stream of the playback device is determined based on the data stream association data of the playback device. According to the data stream processing parameters of the streaming device and the decoding rate of the data stream of the playback device, the streaming media server is determined to process the data stream and / or the streaming device is prompted to determine the processing method of the data stream. For example, if the sending frame rate in the data stream processing parameters of the streaming device is higher than the decoding rate of the data stream of the playback device, the data stream processing parameters are not adjusted and / or the streaming device is prompted not to adjust the data stream processing parameters, so as to avoid the playback device having difficulty in parsing the data stream transmitted at a faster rate, resulting in data stream accumulation. If the sending frame rate in the data stream processing parameters of the streaming device is lower than the decoding rate of the data stream by the playback device, the data stream processing parameters of the streaming media server are increased and / or the streaming device is prompted to increase the data stream processing parameters, so that the decoding rate of the playback device matches the rate at which the streaming media server sends the data stream, thereby improving the data stream playback quality.

[0074] In an embodiment of the present application, the data stream associated data acquired in the current adjustment period is compared with the target data stream associated data to determine a comparison result, and according to the comparison result, a processing method of the data stream by the streaming media server is determined and / or the streaming device is prompted to determine a processing method of the data stream, including: determining a ratio of a change in the data stream associated data acquired in the current adjustment period relative to the target data stream associated data to a value interval of the data stream associated data; determining a change in the reading rate of the data stream by the streaming media server according to a value interval of the reading rate of the data stream by the streaming media server and the ratio, and adjusting the current reading rate of the data stream by the streaming media server according to the change in the reading rate; and / or determining a change in a data stream processing parameter of the streaming media server according to a value interval of a data stream processing parameter of the streaming media server and the ratio, and prompting the streaming media server to adjust the data stream processing parameter according to the change in the processing parameter.

[0075] Exemplarily, the value range of the data stream associated data can be predetermined, such as the value range of the conventional network strength, the value range of the decoding rate, etc. Determine the change amount of the data stream associated data obtained in the current adjustment period relative to the target data stream associated data, and determine the ratio of the change amount to the value range of the data stream associated data, predetermine the value range of the reading rate of the streaming media server to the data stream, and take the product of the value range of the reading rate of the streaming media server to the data stream and the ratio as the change amount of the reading rate of the streaming media server to the data stream. Adjust the current reading rate of the streaming media server to the data stream according to the change amount. For example, if the network strength of the streaming device is divided into 1-10 levels, the network strength of the streaming device obtained in the current adjustment period is 6 levels, which is 2 levels lower than the network strength of 8 levels in the target data stream associated data, then the ratio of the change amount 2 levels to the value range of the data stream associated data 10 levels can be calculated as 1 / 5. Multiply the value range of the server's reading rate of the data stream by 1 / 5 to obtain the reading rate change amount, and reduce the reading rate change amount to obtain a new reading rate based on the current reading rate of the streaming media server to the data stream.

[0076] Exemplarily, for a stream-generating device, a value range of a data stream processing parameter of the stream-generating device can be predetermined, and the product of the value range of the processing parameter and the ratio calculated in the above scheme is used as the processing parameter change. The stream-generating device is prompted to adjust the data stream processing parameter according to the processing parameter change. For example, if the network strength of the stream-generating device is divided into levels 1-10, and the network strength of the stream-generating device obtained in the current adjustment cycle is level 6, which is 2 levels lower than the network strength of level 8 in the target data stream associated data, then the ratio can be calculated as 1 / 5. The value range of the encoding bit rate of the data stream of the stream-generating device is multiplied by 1 / 5 to obtain the encoding bit rate change. On the basis of the current encoding bit rate of the data stream of the stream-generating device, the encoding bit rate change is reduced to obtain a new encoding bit rate.

[0077] Embodiment 3

[0078] Figure 3 This is a flow chart of a data stream processing method applied to a streaming media server according to the third embodiment of the present application. The present embodiment is optimized based on the above embodiment. For solutions not described in detail in the present embodiment, please refer to the above embodiment. Figure 3 As shown, the method of the embodiment of the present application specifically includes the following steps:

[0079] S310: Acquire the network strength of the streaming device, determine the historical average connection time required for the streaming device to establish a data stream transmission channel with the streaming media server from historical data, and determine the average connection time of the connection device corresponding to the network strength.

[0080] The connection establishing device is a streaming device that has established a data stream transmission channel with the streaming service before the current moment. Exemplarily, before establishing the data stream transmission channel between the streaming server and the streaming device, the streaming device can send the network strength to the streaming media through the message queue telemetry transmission protocol. The message queue telemetry transmission protocol is a protocol that can also perform data transmission in a remote device with poor network status or low hardware performance.

[0081] The historical data includes information about each streaming device establishing a data stream transmission channel with a streaming media server in a historical period, including the historical average connection time. The historical average connection time required for the streaming device to establish a data stream transmission channel with the streaming media server can be obtained from the historical data, and the average connection time of all connection devices that are consistent with the network strength of the streaming device can be determined, that is, the ratio of the sum of the historical average connection times of the connection devices corresponding to the network strength to the total number of corresponding connection devices.

[0082] S320: Determine, according to the historical data corresponding to the flow generating device and located between the historical average connection establishment duration and the average connection establishment duration, a first expected value corresponding to the historical average connection establishment duration and a second expected value corresponding to the average connection establishment duration.

[0083] Obtain intermediate historical data of a duration between the historical average connection duration and the average connection duration and belonging to the flow generating device from the historical data. Determine a first expected value corresponding to the historical average connection duration and a second expected value corresponding to the average connection duration based on the historical data.

[0084] Specifically, the first expected value = the number of intermediate historical data with a duration close to the historical average connection duration / the total number of intermediate historical data. The second expected value = 1 - the first expected value. The historical data with a duration close to the historical average connection duration is the intermediate historical data with a duration between the historical average connection duration and the midpoint duration, and the midpoint duration = (historical average connection duration + average connection duration) / 2.

[0085] S330: Determine the waiting time for the streaming media server to wait for the streaming device to establish a data stream transmission channel according to the historical average connection establishment time, the average connection establishment time, the first expected value, and the second expected value.

[0086] Exemplarily, the waiting time may be determined by combining the historical average connection time, the average connection time, the first expected value and the second expected value. The waiting time is the waiting time for the streaming media server to wait for the streaming device to establish a data stream transmission channel.

[0087] In an embodiment of the present application, the waiting time for the streaming media server to wait for the streaming device to establish a data stream transmission channel is determined based on the historical average connection time, the average connection time, the first expected value, and the second expected value, including: using the first expected value as the weight of the historical average connection time, and using the second expected value as the weight of the average connection time; and performing weighted summation of the historical average connection time and the average connection time to obtain the waiting time.

[0088] Exemplarily, waiting time = A*p(A)+B*p(B), wherein A is the historical average connection establishment time, p(A) is the first expected value, B is the average connection establishment time, and p(B) is the second expected value.

[0089] S340, notifying the streaming device to establish a communication connection, and continuously waiting for a waiting time to establish a communication connection between the streaming device and the streaming server. If the streaming device and the streaming server establish a communication connection, executing S350; if the communication connection is not established, exiting the process.

[0090] S350: In the current adjustment cycle, obtain data stream associated data of the target device connected to the streaming media server. The target device includes a streaming device and / or a playback device of the data stream; the data stream associated data of the streaming device includes the network strength of the streaming device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the playback device for the data stream.

[0091] S360. Compare the data flow associated data obtained in the current adjustment cycle with the target data flow associated data to determine a comparison result; wherein the target data flow associated data includes the data flow associated data obtained in the previous adjustment cycle and / or a preset fixed value of the data flow associated data.

[0092] S370: Determine, according to the comparison result, a processing method of the streaming media server for the data stream and / or prompt the streaming device to determine a processing method for the data stream.

[0093] The embodiment of the present application provides a data stream processing method, which obtains the network strength of the streaming device, determines the historical average connection time required for the streaming device to establish a data stream transmission channel with the streaming media server from historical data, and determines the average connection time of the connection device corresponding to the network strength; determines the first expected value corresponding to the historical average connection time and the second expected value corresponding to the average connection time according to the historical data corresponding to the streaming device and located between the historical average connection time and the average connection time; determines the waiting time for the streaming media server to wait for the streaming device to establish a data stream transmission channel according to the historical average connection time, the average connection time, the first expected value and the second expected value. The above scheme can consider the network strength and comprehensively determine the waiting time of the current streaming media server based on the adaptability of the historical data of the streaming device, avoid the problem of connection failure caused by the extension of the connection time of the streaming device and the failure of the streaming media server to wait when the network strength is weakened under the condition of fixed waiting time, thereby ensuring the smooth establishment of the data stream transmission channel.

[0094] Embodiment 4

[0095] Figure 4 This is a flowchart of a data stream processing method applied to a streaming device according to the fourth embodiment of the present application. The present embodiment of the present application is applicable to the case where the streaming device sends a data stream to a streaming media server. The method can be executed by a data stream processing device, which can be implemented in the form of hardware and / or software. The data stream processing device can be configured in an electronic device, which can be a server. Details not fully described in the embodiments of the present application, such as explanations of terms, corresponding step explanations, etc., can be found in the above embodiments. Figure 4As shown, the method includes:

[0096] S410: Send the network strength to the streaming media server, so that the streaming media server determines the processing method of the data stream and / or sends prompt information to the streaming device according to the comparison result between the data stream associated data obtained in the current adjustment period and the target data stream associated data.

[0097] The data stream associated data includes the network strength sent by the streaming device and / or the data stream associated data of the playback device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device. The target data stream associated data includes the data stream associated data obtained in the previous adjustment cycle and / or the preset fixed value of the data stream associated data.

[0098] Exemplarily, after the streaming device establishes a data stream transmission channel with the streaming media server, it sends the network strength to the streaming media server through the message queue telemetry transmission protocol. The streaming media server receives the network strength sent by the streaming device, and / or obtains the data stream associated data sent by the playback device, and compares it with the target data stream associated data to determine the comparison result, and determines the processing method of the streaming media server for the data stream and / or sends prompt information to the streaming device according to the comparison result.

[0099] In an embodiment of the present application, sending network strength to a streaming media server includes: if at least one of the following conditions is met, sending the network strength to the streaming media server through a message queue telemetry transmission protocol: a change in network strength is detected; a change in network strength is detected to be greater than a preset threshold; the interval time from the last time the network strength was sent reaches a preset interval time.

[0100] Exemplarily, the streaming device can detect the network strength. If at least one of the following occurs: the network strength changes, the change in network strength is greater than a preset threshold, or the interval from the last time the network strength was sent reaches a preset interval, the network strength is sent to the streaming server through the message queue telemetry transmission protocol, so that the streaming server can promptly identify the network strength of the streaming device. The preset threshold can be determined based on actual conditions, such as a value that indicates a large change. The preset interval can be determined based on actual conditions, and the reciprocal of the preset interval is the preset frequency.

[0101] S420: If the prompt information sent by the streaming media server is received, or if it is detected that the data stream is not successfully sent to the streaming media server, determine a processing method for the data stream.

[0102] Exemplarily, if a prompt message sent by a streaming media server is received, the processing method for the data stream is determined according to the prompt message. For example, if the data stream associated data obtained in the current adjustment period is less than the target data stream associated data, the streaming media server prompts the streaming device to reduce the data stream processing parameters; wherein the data stream processing parameters of the streaming device include at least one of the data stream transmission frame rate, encoding bit rate and resolution; if the data stream associated data obtained in the current adjustment period is greater than the target data stream associated data, the streaming media server prompts the streaming device to increase the data stream processing parameters; if the data stream associated data obtained in the current adjustment period is equal to the target data stream associated data, the streaming device does not adjust the data stream processing parameters according to the prompt of the streaming media server.

[0103] Alternatively, if the streaming server reduces the reading rate of the data stream, the streaming device detects that the data stream is not successfully sent to the streaming server but is accumulated in the cache, and then adaptively determines the processing method of the data stream, for example, reducing the data stream processing parameters.

[0104] In an embodiment of the present application, if a prompt message sent by the streaming media server is received, a processing method for the data stream is determined, including: receiving data stream associated data of a playback device carried in the prompt message; wherein the playback device is a device for playing the data stream; determining a decoding rate of the data stream by the playback device according to the data stream associated data of the playback device; and determining a processing method for the data stream according to the decoding rate of the data stream by the playback device.

[0105] Exemplarily, the streaming media server may not directly send prompt information to the streaming device to instruct the streaming device to determine the processing method for the data stream, but instead send prompt information carrying the data stream associated data of the playback device to the streaming device, and the streaming device determines the decoding rate of the playback device for the data stream according to the data stream associated data of the playback device, and then determines the processing method for the data stream according to the decoding rate. For example, the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream, and the streaming device can directly obtain the decoding rate of the playback device for the data stream from the data stream associated data, or predict the decoding rate of the playback device for the data stream according to the network strength of the playback device. The streaming device compares the current sending frame rate of the data stream with the decoding rate of the playback device for the data stream. If they are consistent, the processing method of the data stream is not adjusted. If the current sending frame rate of the data stream of the streaming device is greater than the decoding rate of the data stream of the playback device, the streaming device can reduce the data stream processing parameters.

[0106] The embodiment of the present application provides a data stream processing method, which sends network strength to a streaming media server, so that the streaming media server determines the processing method of the data stream by the streaming media server and / or sends prompt information to the streaming device according to the comparison result of the data stream association data obtained in the current adjustment cycle and the target data stream association data; wherein the data stream association data envelops the network strength sent by the streaming device and / or the data stream association data of the playback device; the data stream association data of the playback device includes the network strength of the playback device and / or the decoding rate of the playback device to the data stream; the target data stream association data includes the data stream association data obtained in the previous adjustment cycle and / or the preset fixed value of the data stream association data; if the prompt information sent by the streaming media server is received, or if it is detected that the data stream is not successfully sent to the streaming media server, the processing method of the data stream is determined. The above scheme can timely and accurately determine the change of the network strength of the streaming device according to the data stream association data of the streaming device, and then adjust the processing method of the streaming device for the data stream, so as to avoid the problem of freezing when the playback device plays the data stream.

[0107] Embodiment 5

[0108] Figure 5 It is a flowchart of a specific implementation method of a data stream processing method provided in Embodiment 5 of the present application. A specific implementation method of a data stream processing method provided in the embodiment of the present application includes the following steps:

[0109] The user selects a streaming device on the playback device and clicks to play the data stream of the streaming device. The playback device notifies the streaming media server to establish a data stream transmission channel.

[0110] The streaming media server notifies the streaming device to establish a data stream transmission channel, and continues to wait for the streaming device to establish the data stream transmission channel.

[0111] The streaming device reports the network strength to the streaming server via MQTT (Message Queuing Telemetry Transport). If the network strength decreases, the streaming server proportionally reduces the reading rate of the data stream.

[0112] If the streaming device detects data stream accumulation, it adjusts the data stream processing parameters, including reducing the sending frame rate of the data stream, reducing the encoding bit rate, reducing the resolution, etc.

[0113] If the network strength reported by the streaming device to the streaming media server is improved, the streaming media server sends the increased sending frame rate of the data stream by the streaming device to the playback device.

[0114] The playback device detects whether the data stream with the increased frame rate can be parsed normally. If so, the streaming media server is notified, and the streaming media server increases the reading rate of the data stream in proportion.

[0115] If the playback device detects that the data stream with the increased sending frame rate cannot be parsed normally, the streaming media server is notified, and the streaming media server does not increase the speed of extracting the data stream.

[0116] If the network strength of the playback device decreases or the decoding rate decreases, the streaming media server is notified.

[0117] The streaming media server monitors the amount of data in the data stream read by the playback device at preset intervals and calculates the parsing rate of the data stream by the playback device.

[0118] The streaming media adjusts the reading rate of the data stream from the streaming device according to the parsing rate of the data stream by the playback device.

[0119] If the streaming media server reduces the reading rate of the data stream from the streaming device, the streaming device detects the accumulation of data stream and reduces the data stream processing parameters.

[0120] The embodiment of the present application has the same beneficial effects as any of the above embodiments.

[0121] Embodiment 6

[0122] Figure 6 This is a structural diagram of a data stream processing device configured on a streaming media server according to Embodiment 6 of the present application. The device can execute the data stream processing method provided by any embodiment of the present application and has the corresponding functional modules and beneficial effects of the execution method. Figure 6 As shown, the device comprises:

[0123] The data stream associated data acquisition module 510 is used to acquire data stream associated data of a target device connected to the streaming media server within a current adjustment period; wherein the target device includes a streaming device and / or a playback device of the data stream; the data stream associated data of the streaming device includes the network strength of the streaming device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the playback device for the data stream;

[0124] A comparison result determination module 520 is used to compare the data flow associated data obtained in the current adjustment cycle with the target data flow associated data to determine a comparison result; wherein the target data flow associated data includes the data flow associated data obtained in the previous adjustment cycle and / or a preset fixed value of the data flow associated data;

[0125] The processing mode determining module 530 is used to determine the processing mode of the streaming media server for the data stream and / or prompt the streaming device to determine the processing mode for the data stream according to the comparison result.

[0126] In the embodiment of the present application, the processing mode determination module 530 includes:

[0127] A first processing unit is configured to reduce the data stream processing parameters of the streaming media server and / or prompt the streaming device to reduce the data stream processing parameters if the data stream associated data acquired in the current adjustment period is less than the target data stream associated data; wherein the data stream processing parameters of the streaming media server include a reading rate of the data stream; and the data stream processing parameters of the streaming device include at least one of a sending frame rate, an encoding bit rate, and a resolution of the data stream;

[0128] A second processing unit is configured to increase the data stream processing parameters of the streaming media server and / or prompt the streaming device to increase the data stream processing parameters if the data stream associated data acquired in the current adjustment period is greater than the target data stream associated data;

[0129] The third processing unit is configured to not adjust the data stream processing parameters and / or prompt the stream sending device not to adjust the data stream processing parameters if the data stream associated data acquired in the current adjustment period is equal to the target data stream associated data.

[0130] In the embodiment of the present application, the processing mode determination module 530 includes:

[0131] a prediction unit, configured to predict a data stream processing parameter of the stream-generating device according to the network strength of the stream-generating device and the target data stream-generating data if the network strength of the stream-generating device is greater than the target data stream-related data; wherein the data stream processing parameter includes at least one of a sending frame rate, an encoding bit rate, and a resolution of the data stream;

[0132] A decoding rate determination unit, configured to determine a decoding rate of the data stream by the playback device according to the data stream associated data of the playback device;

[0133] The judgment unit is used to prompt the streaming device to determine whether to increase the data stream processing parameters according to the data stream processing parameters of the streaming device and the decoding rate of the data stream by the playback device.

[0134] In the embodiment of the present application, the processing mode determination module 530 includes:

[0135] A ratio determination unit, used to determine the ratio of the change amount of the acquired data flow associated data relative to the target data flow associated data in the current adjustment period to the value interval of the data flow associated data;

[0136] a first regulating unit, configured to determine a change in a reading rate of the data stream by the streaming server according to a value interval of a reading rate of the data stream by the streaming server and the ratio, and to regulate a current reading rate of the data stream by the streaming server according to the change in the reading rate; and / or

[0137] The second adjustment unit is used to determine a change amount of the data stream processing parameter of the stream generating device according to the data stream processing parameter value range of the stream generating device and the ratio, and prompt the stream generating device to adjust the data stream processing parameter according to the change amount of the processing parameter.

[0138] In the embodiment of the present application, the device further includes:

[0139] A connection establishment duration determination module is used to obtain the network strength of the streaming device, determine the historical average connection establishment duration required for the streaming device to establish a data stream transmission channel with the streaming media server from historical data, and determine the average connection establishment duration of the connection device corresponding to the network strength;

[0140] an expected value determination module, configured to determine a first expected value corresponding to the historical average connection duration and a second expected value corresponding to the average connection duration according to historical data corresponding to the flow generating device and located between the historical average connection duration and the average connection duration;

[0141] The waiting time determination module is used to determine the waiting time for the streaming media server to wait for the streaming device to establish a data stream transmission channel according to the historical average connection establishment time, the average connection establishment time, the first expected value and the second expected value.

[0142] In the embodiment of the present application, the waiting time determination module includes:

[0143] a weight determination unit, configured to use the first expected value as the weight of the historical average connection establishment time, and use the second expected value as the weight of the average connection establishment time;

[0144] A summing unit is used to perform weighted summation on the historical average connection establishment time and the average connection establishment time to obtain the waiting time.

[0145] A data stream processing device configured on a streaming media server provided in an embodiment of the present application can execute a data stream processing method applied to a streaming media server provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.

[0146] Embodiment 7

[0147] Figure 7 This is a structural diagram of a data stream processing device configured on a stream transmitting device according to Embodiment 7 of the present application. The device can execute the data stream processing method provided by any embodiment of the present application and has the corresponding functional modules and beneficial effects of the execution method. Figure 7 As shown, the device comprises:

[0148] The network strength sending module 610 is used to send the network strength to the streaming media server, so that the streaming media server determines the processing method of the data stream by the streaming media server and / or sends prompt information to the streaming device according to the comparison result of the data stream associated data obtained in the current adjustment cycle and the target data stream associated data; wherein the data stream associated data includes the network strength sent by the streaming device and / or the data stream associated data of the playback device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the playback device for the data stream; the target data stream associated data includes the data stream associated data obtained in the previous adjustment cycle and / or the preset fixed value of the data stream associated data;

[0149] The processing mode determination module 620 is used to determine the processing mode of the data stream if a prompt message sent by the streaming media server is received or if it is detected that the data stream is not successfully sent to the streaming media server.

[0150] In the embodiment of the present application, the processing mode determination module 620 includes:

[0151] A parameter reduction unit is used to reduce a data stream processing parameter; wherein the data stream processing parameter includes at least one of a sending frame rate, an encoding bit rate, and a resolution of the data stream.

[0152] In the embodiment of the present application, the network strength sending module 610 is specifically used to:

[0153] If at least one of the following conditions is met, the network strength is sent to the streaming media server via a message queue telemetry transmission protocol:

[0154] A change in network strength was detected;

[0155] It is detected that the change in network strength is greater than a preset threshold;

[0156] The interval from the last time the network strength was sent reaches the preset interval.

[0157] In the embodiment of the present application, the processing mode determination module 620 includes:

[0158] A receiving unit, configured to receive data stream associated data of a playback device carried in the prompt information; wherein the playback device is a device for playing the data stream;

[0159] A decoding rate prediction unit, configured to determine a decoding rate of the data stream by the playback device according to the data stream associated data;

[0160] The mode determination unit is used to determine the processing mode of the data stream according to the decoding rate of the data stream by the playback device.

[0161] A data stream processing device configured on a stream-generating device provided in an embodiment of the present application can execute a data stream processing method applied to a stream-generating device provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.

[0162] Embodiment 8

[0163] Figure 8 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0164] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0165] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0166] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a data stream processing method applied to a streaming media server, or a data stream processing method applied to a streaming device.

[0167] In some embodiments, the data stream processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data stream processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the data stream processing method applied to the streaming media server, or to execute the data stream processing method applied to the streaming device, by any other appropriate means (e.g., by means of firmware).

[0168] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0169] The computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data stream processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0170] In the context of the present application, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device or equipment. A computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. A more specific example of a machine readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0171] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0172] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0173] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0174] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the information expected by the technical solution of this application can be achieved, and this document is not limited here.

[0175] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A data stream processing method, characterized in that: Applied to a streaming media server, the method comprises: In the current adjustment cycle, obtaining data stream associated data of a target device connected to the streaming media server; The target device includes a data stream sending device and / or a playback device; the data stream associated data of the data stream sending device includes the network strength of the data stream sending device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device; Compare the data flow association data obtained in the current adjustment cycle with the target data flow association data to determine a comparison result; wherein the target data flow association data includes the data flow association data obtained in the previous adjustment cycle and / or a preset fixed value of the data flow association data; According to the comparison result, the streaming media server determines a processing method for the data stream and / or prompts the streaming device to determine a processing method for the data stream.

2. The method according to claim 1, characterized in that: Determining, according to the comparison result, a processing method of the streaming media server on the data stream and / or prompting the streaming device to determine a processing method on the data stream, including: If the data stream associated data acquired in the current adjustment period is less than the target data stream associated data, then the data stream processing parameters of the streaming media server are reduced and / or the streaming device is prompted to reduce the data stream processing parameters; wherein the data stream processing parameters of the streaming media server include the reading rate of the data stream; the data stream processing parameters of the streaming device include at least one of the sending frame rate, encoding bit rate and resolution of the data stream; If the data stream associated data acquired in the current adjustment period is greater than the target data stream associated data, the data stream processing parameters of the streaming media server are increased and / or the streaming device is prompted to increase the data stream processing parameters.

3. The method according to claim 2, characterized in that According to the comparison result, prompting the flow sending device to determine a processing method for the data flow includes: If the network strength of the stream-generating device is greater than the target data stream associated data, predicting the data stream processing parameters of the stream-generating device according to the network strength of the stream-generating device and the target data stream associated data; wherein the data stream processing parameters include at least one of the sending frame rate, encoding bit rate and resolution of the data stream; Determining a decoding rate of the data stream by the playback device according to the data stream associated data of the playback device; According to the data stream processing parameters of the stream-generating device and the decoding rate of the data stream by the playback device, the stream-generating device is prompted to determine whether to increase the data stream processing parameters.

4. The method according to any one of claims 1 to 3, characterized in that Determining, according to the comparison result, a processing method of the streaming media server on the data stream, including: Determine the ratio of the change amount of the acquired data stream associated data relative to the target data stream associated data in the current adjustment period to the value interval of the data stream associated data; Determining a change in the reading rate of the data stream by the streaming server according to a value interval of the reading rate of the data stream by the streaming server and the ratio, and adjusting the current reading rate of the data stream by the streaming server according to the change in the reading rate; and / or, According to the data stream processing parameter value range of the stream generating device and the ratio, a change amount of the data stream processing parameter of the stream generating device is determined, and the stream generating device is prompted to adjust the data stream processing parameter according to the change amount of the processing parameter.

5. The method according to claim 1, characterized in that: The method further comprises: Acquire the network strength of the streaming device, determine the historical average connection time required for the streaming device to establish a data stream transmission channel with the streaming media server from historical data, and determine the average connection time of the connection device corresponding to the network strength; wherein the connection device is a streaming device that has established a data stream transmission channel with the streaming media service before the current moment; Determine, according to the historical data corresponding to the flow generating device and located between the historical average connection establishment duration and the average connection establishment duration, a first expected value corresponding to the historical average connection establishment duration and a second expected value corresponding to the average connection establishment duration; The waiting time for the streaming media server to wait for the streaming device to establish a data stream transmission channel is determined according to the historical average connection establishment time, the average connection establishment time, the first expected value, and the second expected value.

6. A data stream processing method, characterized in that: Applied to a flow generating device, the method comprises: Sending the network strength to the streaming media server, so that the streaming media server determines the processing mode of the data stream by the streaming media server and / or sends prompt information to the streaming device according to the comparison result of the data stream associated data obtained in the current adjustment period and the target data stream associated data; The data stream associated data envelops the network strength sent by the streaming device and / or the data stream associated data of the playback device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device; the target data stream associated data includes the data stream associated data obtained in the previous adjustment cycle and / or the preset fixed value of the data stream associated data; If the prompt information sent by the streaming media server is received, or if it is detected that the data stream is not successfully sent to the streaming media server, a processing method for the data stream is determined.

7. A data stream transmission device, characterized in that: Configured in a streaming media server, the device comprises: A data stream associated data acquisition module, used to acquire data stream associated data of a target device connected to the streaming media server within a current adjustment period; The target device includes a data stream sending device and / or a playback device; the data stream associated data of the data stream sending device includes the network strength of the data stream sending device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device; A comparison result determination module, used to compare the data flow associated data obtained in the current adjustment cycle with the target data flow associated data to determine the comparison result; wherein the target data flow associated data includes the data flow associated data obtained in the previous adjustment cycle and / or a preset fixed value of the data flow associated data; The processing mode determining module is used to determine the processing mode of the streaming media server for the data stream and / or prompt the streaming device to determine the processing mode for the data stream according to the comparison result.

8. A data stream transmission device, characterized in that: Configured in a flow generating device, the device comprises: A network strength sending module, used for sending the network strength to the streaming media server, so that the streaming media server determines the processing mode of the data stream by the streaming media server and / or sends prompt information to the streaming device according to the comparison result of the data stream associated data obtained in the current adjustment cycle and the target data stream associated data; The data stream associated data envelops the network strength sent by the streaming device and / or the data stream associated data of the playback device; the data stream associated data of the playback device includes the network strength of the playback device and / or the decoding rate of the data stream by the playback device; the target data stream associated data includes the data stream associated data obtained in the previous adjustment cycle and / or the preset fixed value of the data stream associated data; The processing mode determination module is used to determine the processing mode of the data stream if a prompt message sent by the streaming media server is received or if it is detected that the data stream is not successfully sent to the streaming media server.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data stream transmission method applied to a streaming media server as described in any one of claims 1 to 5, or execute the data stream transmission method applied to a streaming device as described in claim 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data stream transmission method applied to a streaming media server as described in any one of claims 1 to 5, or to implement the data stream transmission method applied to a streaming device as described in claim 6 when executed.

Images