Video stream transmission method, device, electronic device and computer readable storage medium

Abstract

The present application discloses a video stream transmission method, device, electronic device and computer-readable storage medium, which are applied to a server. The video stream transmission method comprises: obtaining the buffer size information of a virtual buffer of the server, wherein the virtual buffer is used to simulate the real video transmission buffer of the client; selecting the target transmission rate required for video stream transmission to the client according to the obtained network status parameters and the buffer size information; and transmitting the video stream to the client according to the target transmission rate. The present application solves the technical problem of low stability of video stream transmission.

Description

Technical Field

[0001] The present application relates to the field of video transmission technology, and in particular to a video stream transmission method, device, electronic device and computer-readable storage medium. Background Art

[0002] At present, when transmitting video streams, the client usually needs to establish a cache area to relieve the transmission pressure of the server and speed up the loading speed of the client. The client's cache area needs to continuously feedback its real-time status to the server as the basis for the server's network transmission rate decision. However, when the real-time status is fed back from the client to the server, the network bandwidth is high, which will affect the stability of video stream transmission. Summary of the invention

[0003] The main purpose of the present application is to provide a video stream transmission method, device, electronic device and computer-readable storage medium, aiming to solve the technical problem of low stability of video stream transmission.

[0004] To achieve the above object, the present application provides a video stream transmission method, which is applied to a server, and the video stream transmission method includes:

[0005] Obtaining buffer size information of a virtual buffer on the server, wherein the virtual buffer is used to simulate a real video transmission buffer on the client;

[0006] Selecting a target transmission rate required for transmitting the video stream to the client according to the acquired network status parameters and the buffer size information;

[0007] The video stream is transmitted to the client according to the target transmission rate.

[0008] Optionally, the step of obtaining the buffer size information of the server virtual buffer includes:

[0009] Acquire the server-side video stream data, and extract the time features of the server-side video stream data according to the time feature extractor to obtain a first time feature sequence;

[0010] Receiving a second time feature sequence sent by the client, wherein the second time feature sequence is obtained by the client performing time feature extraction on the client video stream data corresponding to the server video stream data;

[0011] The buffer size information corresponding to the real video transmission buffer is determined according to the first time feature sequence and the second time feature sequence.

[0012] Optionally, the buffer size information includes a video stream time difference, the first time feature sequence includes a first target time feature corresponding to a video to which a preset time offset is applied, the second time feature sequence includes a second target time feature corresponding to the first target time feature, and the step of determining the buffer size information corresponding to the real video transmission buffer based on the first time feature sequence and the second time feature sequence includes:

[0013] Calculating a feature distance between the first target time feature and the second target time feature;

[0014] The video stream time difference is determined according to the characteristic distance and the preset time offset.

[0015] Optionally, the step of selecting a target transmission rate required for transmitting the video stream to the client according to the acquired network status parameter and the buffer size information includes:

[0016] Constructing a video transmission status feature according to the acquired network status parameters and the buffer size information;

[0017] The target transmission rate required for video stream transmission between the clients is predicted by inputting the video transmission state characteristics into a preset transmission rate prediction model.

[0018] Optionally, the step of constructing a video transmission status feature according to the acquired network status parameter and the buffer size information includes:

[0019] Acquire network status parameters, wherein the network status parameters include network bandwidth, video stream bit rate, and video stream weight coefficient, and the video stream weight coefficient is used to represent the frequency of the video stream being clicked by the user;

[0020] The buffer size information, the network bandwidth, the video stream bit rate and the video stream weight coefficient are spliced ​​into the video transmission status feature.

[0021] Optionally, obtain a video stream weight coefficient, including:

[0022] Obtain the video stream ID, the corresponding requested bit rate, the corresponding request time and the corresponding video stream parameters corresponding to the video stream to be transmitted;

[0023] splicing the video stream ID, the requested bit rate, the requested time, and the video stream parameters into a video stream weight feature;

[0024] The video stream weight feature is mapped to a corresponding video stream weight coefficient.

[0025] Optionally, after the step of predicting a target transmission rate required for video stream transmission between the clients by inputting the video transmission state feature into a preset transmission rate prediction model, the video stream transmission method further comprises:

[0026] Obtaining model parameter information of a preset transmission rate prediction model deployed on at least one video transmission thread at a current time step, wherein each of the video transmission threads is in a different video transmission environment;

[0027] Aggregating each of the model parameter information to obtain aggregated model parameter information;

[0028] The preset transmission rate prediction model is updated in real time according to the aggregated model parameter information to predict the target transmission rate of the next time step.

[0029] The present application also provides a video stream transmission method, which is applied to a client, and the video stream transmission method includes:

[0030] Acquire client video stream data, and extract time features from the client video stream data according to a time feature extractor to obtain a client time feature sequence;

[0031] The client time feature sequence is sent to the server, so that the server can transmit the video stream to the client according to the target transmission rate determined by the client time feature sequence.

[0032] The present application also provides a video stream transmission device, which is applied to a server, and the video stream transmission device includes:

[0033] An acquisition module, used to acquire buffer size information of a virtual buffer of a server, wherein the virtual buffer is used to simulate a real video transmission buffer of a client;

[0034] A transmission rate selection module, used for selecting a target transmission rate required for transmitting the video stream to the client according to the acquired network status parameters and the buffer size information;

[0035] A transmission module is used to transmit the video stream to the client according to the target transmission rate.

[0036] Optionally, the acquisition module is further used to:

[0037] Acquire the server-side video stream data, and extract the time features of the server-side video stream data according to the time feature extractor to obtain a first time feature sequence;

[0038] Receiving a second time feature sequence sent by the client, wherein the second time feature sequence is obtained by the client performing time feature extraction on the client video stream data corresponding to the server video stream data;

[0039] The buffer size information corresponding to the real video transmission buffer is determined according to the first time feature sequence and the second time feature sequence.

[0040] Optionally, the buffer size information includes a video stream time difference, the first time feature sequence includes a first target time feature corresponding to a video to which a preset time offset is applied, the second time feature sequence includes a second target time feature corresponding to the first target time feature, and the acquisition module is further used to:

[0041] Calculating a feature distance between the first target time feature and the second target time feature;

[0042] The video stream time difference is determined according to the characteristic distance and the preset time offset.

[0043] Optionally, the transmission rate selection module is further used to:

[0044] Constructing a video transmission status feature according to the acquired network status parameters and the buffer size information;

[0045] The target transmission rate required for video stream transmission between the clients is predicted by inputting the video transmission state characteristics into a preset transmission rate prediction model.

[0046] Optionally, the transmission rate selection module is further used to:

[0047] Acquire network status parameters, wherein the network status parameters include network bandwidth, video stream bit rate, and video stream weight coefficient, and the video stream weight coefficient is used to represent the frequency of the video stream being clicked by the user;

[0048] The buffer size information, the network bandwidth, the video stream bit rate and the video stream weight coefficient are spliced ​​into the video transmission status feature.

[0049] Optionally, the transmission rate selection module is further used to:

[0050] Obtain the video stream ID, the corresponding requested bit rate, the corresponding request time and the corresponding video stream parameters corresponding to the video stream to be transmitted;

[0051] splicing the video stream ID, the requested bit rate, the requested time, and the video stream parameters into a video stream weight feature;

[0052] The video stream weight feature is mapped to a corresponding video stream weight coefficient.

[0053] Optionally, the video stream transmission device is further used for:

[0054] Obtaining model parameter information of a preset transmission rate prediction model deployed on at least one video transmission thread at a current time step, wherein each of the video transmission threads is in a different video transmission environment;

[0055] Aggregating each of the model parameter information to obtain aggregated model parameter information;

[0056] The preset transmission rate prediction model is updated in real time according to the aggregated model parameter information to predict the target transmission rate of the next time step.

[0057] The present application also provides a video stream transmission device, which is applied to a client, and the video stream transmission device includes:

[0058] A feature extraction module is used to obtain client video stream data, and extract time features from the client video stream data according to a time feature extractor to obtain a client time feature sequence;

[0059] The sending module is used to send the client time feature sequence to the server, so that the server can transmit the video stream to the client according to the target transmission rate determined by the client time feature sequence.

[0060] The present application also provides an electronic device, which is a physical device, and includes: a memory, a processor, and a program of the video stream transmission method stored in the memory and executable on the processor. When the program of the video stream transmission method is executed by the processor, the steps of the video stream transmission method as described above can be implemented.

[0061] The present application also provides a computer-readable storage medium, on which is stored a program for implementing the video stream transmission method. When the program for the video stream transmission method is executed by a processor, the steps of the video stream transmission method as described above are implemented.

[0062] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned video stream transmission method when executed by a processor.

[0063] The present application provides a video stream transmission method, device, electronic device and computer-readable storage medium. Compared with the client side, which usually needs to establish a cache area to relieve the transmission pressure of the server and speed up the loading speed of the client, the client's cache area needs to continuously feedback its real-time status to the server side. As a technical means for the server's network transmission rate decision, the present application sets a virtual cache area on the server side to simulate the client's real video transmission cache area. The virtual cache area can be used to determine the cache area size information of the real video transmission cache area, and then the obtained network state parameters and the determined cache area size information can be used as the basis for selecting the transmission rate for video streaming to the client, and the target transmission rate is selected. According to the target transmission rate, the video stream is transmitted to the client. Therefore, in the present application, when performing video streaming transmission, the client's cache area does not need to feedback its real-time status to the server side in real time, which can save the interaction process between the client's cache area and the server side for buffer area status feedback, thereby saving network bandwidth, so as to overcome the technical defect that the network bandwidth is too high when sending feedback real-time status from the client to the server side, thereby affecting the stability of video streaming transmission, and improving the stability of video streaming transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0066] Figure 1 This is a flow chart of the first embodiment of the video stream transmission method of the present application;

[0067] Figure 2 A schematic diagram of a process for selecting a target transmission rate based on a state-action decision network in the video stream transmission method of the present application;

[0068] Figure 3 A schematic diagram of a model deployment for deploying a preset transmission rate prediction model in an embodiment of the present application;

[0069] Figure 4 This is a flow chart of a second embodiment of the video stream transmission method of the present application;

[0070] Figure 5 This is a flow chart of a third embodiment of the video stream transmission method of the present application;

[0071] Figure 6 This is a schematic diagram of the interaction between the server and the client when transmitting a video stream in one embodiment of the present application;

[0072] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the video stream transmission method in the embodiment of the present application.

[0073] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of this application.

[0075] At present, when transmitting video streams, the client usually needs to establish a cache area to relieve the transmission pressure of the server and speed up the loading speed of the client, and the server needs to send a detection request to the client. After receiving the detection request, the client will feedback the real-time status information of the cache area to the server. After receiving the real-time status information of the cache area, the server will inform the client that the real-time status information has been received. Therefore, when the server confirms the real-time status information of the client, the server needs to interact with the client multiple times, which seriously occupies the network bandwidth, thereby affecting the stability of video stream transmission. The process of the server detecting the real-time status information of the client cache area takes time. Due to the existence of feedback inertia, the response ability of the network will be reduced, and this part of the transmission is uncontrollable and has great volatility, which will further affect the stability of video stream transmission.

[0076] Reference Figure 1 The embodiment of the present application provides a video stream transmission method, which is applied to a server. In the first embodiment of the video stream transmission method of the present application, the video stream transmission method includes:

[0077] Step S10, obtaining buffer size information of a virtual buffer on the server side, wherein the virtual buffer is used to simulate a real video transmission buffer on the client side;

[0078] Step S20, selecting a target transmission rate required for transmitting the video stream to the client according to the acquired network status parameters and the buffer size information;

[0079] Step S30: Transmit the video stream to the client according to the target transmission rate.

[0080] In this embodiment, it should be noted that the video stream transmission method is applied to the server, the server is connected to the client, the server is used to transmit the video stream to the client, and the client is used to play the video. The server is provided with a virtual buffer, which is used to simulate the real video transmission buffer of the client, so the size of the real video buffer of the client can be determined in real time by using the virtual buffer, and the buffer size information can be a measurement value for measuring the real video buffer size of the client, for example, it can be the time difference between the video stream of the server before transmission and the video stream of the client after transmission, and the buffer size information can also be the buffer size itself, which is not limited here. The network status parameter is a parameter that characterizes the network status between the server and the client, and the network status parameter can be one or more of the network bandwidth, the video stream bit rate and the video stream weight coefficient, and the video stream weight coefficient is a weight coefficient that characterizes the importance of the video stream, for example, it can be a measurement value for measuring the frequency of video stream being clicked, and the larger the video stream weight coefficient, the higher the frequency of video stream being clicked.

[0081] As an example, step S10 to step S30 include: obtaining the cache size information of the virtual cache of the server, wherein the virtual cache is used to simulate the real video transmission cache of the client; obtaining network status parameters, wherein the network status parameters include at least one of network bandwidth, video stream bit rate and video stream weight coefficient; constructing video transmission status features according to the network status parameters and the cache size information, inputting a preset transmission rate prediction model through the video transmission status features, and predicting the target transmission rate required for video stream transmission between the clients; transmitting the video stream to the client at the target transmission rate. The preset transmission rate prediction model can be a convolutional neural network model or a reinforcement learning model, which is not limited here.

[0082] As an example, the step of constructing a video transmission status feature according to the network status parameter and the buffer size information includes:

[0083] The network status parameter and the buffer size information are concatenated into a corresponding feature vector, and the feature vector is used as the video transmission status feature.

[0084] In addition, it should be noted that when performing video stream transmission, the higher the network bandwidth occupancy, the higher the network delay during video transmission, which will easily cause black screen and stuttering when the video is switched. In the embodiment of the present application, when performing video stream transmission, the client's cache area does not need to feedback its real-time status to the server in real time, which can save the interaction process between the client's cache area and the server for cache area status feedback, save network bandwidth, thereby reducing network delay and reducing the probability of black screen and stuttering when switching videos.

[0085] The step of selecting a target transmission rate required for transmitting the video stream to the client according to the acquired network status parameter and the buffer size information comprises:

[0086] Step S21, constructing a video transmission state feature according to the acquired network state parameter and the buffer size information;

[0087] Step S22: predicting a target transmission rate required for video stream transmission between the clients by inputting the video transmission state characteristics into a preset transmission rate prediction model.

[0088] As an example, step S21 to step S22 include: obtaining network status parameters, wherein the network status parameters include at least one of network bandwidth, video stream bit rate, and video stream weight coefficient; constructing video transmission status characteristics based on the network status parameters and the cache size information; and mapping the video transmission status characteristics to the target transmission rate required for video stream transmission to the client by inputting the video transmission status characteristics into a preset transmission rate prediction model.

[0089] The step of constructing the video transmission status feature according to the acquired network status parameter and the buffer size information includes:

[0090] Step S211, obtaining network status parameters, wherein the network status parameters include network bandwidth, video stream bit rate, and video stream weight coefficient, and the video stream weight coefficient is used to represent the frequency of the video stream being clicked by the user;

[0091] Step S212: combining the buffer size information, the network bandwidth, the video stream bit rate, and the video stream weight coefficient into the video transmission status feature.

[0092] As an example, step S211 to step S212 include: obtaining network bandwidth, video stream bit rate and video stream coefficient, wherein the video stream weight coefficient is used to characterize the frequency of the video stream being clicked by the user; splicing the cache size information, the network bandwidth, the video stream bit rate and the video stream weight coefficient into a state feature vector, and using the state feature vector as a video transmission state feature. The embodiment of the present application realizes that the video transmission state feature has more feature information based on the cache size information, network bandwidth, video stream bit rate and video stream weight coefficient, and provides more decision-making basis for the prediction of the target transmission rate, thereby improving the prediction accuracy of the video stream transmission rate.

[0093] Among them, obtaining the video stream weight coefficient includes:

[0094] Step A10, obtaining a video stream ID, a corresponding requested bit rate, a corresponding request time, and corresponding video stream parameters corresponding to the video stream to be transmitted;

[0095] Step A20, splicing the video stream ID, the requested bit rate, the requested time and the video stream parameters into a video stream weight feature;

[0096] Step A30, mapping the video stream weight feature to a corresponding video stream weight coefficient.

[0097] In this embodiment, it should be noted that the video stream ID is the identity of the video stream, the requested bit rate is the video bit rate requested by the client for transmission, specifically the number of bits transmitted per second, the requested time is the transmission time required by the client for requesting transmission of the video stream, and the video stream parameters may be parameters such as the resolution or clarity of the video; the video stream to be transmitted is the video stream that the server needs to transmit to the client at the current time step.

[0098] As an example, steps A10 to A30 include: obtaining the video stream ID, the corresponding requested bit rate, the corresponding request time and the corresponding video stream parameters corresponding to the video stream to be transmitted; splicing the video stream ID, the requested bit rate, the request time and the video stream parameters into a corresponding weight feature vector, and using the weight feature vector as the video stream weight feature; mapping the video stream weight feature into a corresponding video stream weight coefficient.

[0099] As an example, the preset video stream weight prediction model may be a fully connected neural network, and the step of mapping the video stream weight feature to a corresponding video stream weight coefficient includes:

[0100] The video stream weight feature is input into a fully connected neural network to fully connect the video stream weight feature, and the video stream weight feature is mapped to a corresponding video stream weight coefficient.

[0101] As an example, the preset video stream weight prediction model may also be a decision tree model, the preset video stream weight prediction model may be a fully connected neural network, and the step of predicting the importance of the video stream to be transmitted by inputting the video stream weight feature into the preset video stream weight prediction model to obtain the video stream weight coefficient includes:

[0102] By inputting each feature value in the video stream weight feature into the decision tree model, the importance of the video stream to be transmitted is predicted to obtain a video stream weight coefficient.

[0103] After the step of predicting the target transmission rate required for video stream transmission between the clients by inputting the video transmission state feature into a preset transmission rate prediction model, the video stream transmission method further includes:

[0104] Step B10, obtaining model parameter information of a preset transmission rate prediction model deployed on at least one video transmission thread at a current time step, wherein each of the video transmission threads is in a different video transmission environment;

[0105] Step B20, aggregating each of the model parameter information to obtain aggregated model parameter information;

[0106] Step B30: updating the preset transmission rate prediction model in real time according to the aggregated model parameter information to predict the target transmission rate at the next time step.

[0107] In this embodiment, it should be noted that different preset transmission rate prediction models can be deployed on different video transmission threads, so that the model parameter information of the preset transmission rate prediction models on multiple different video transmission threads can be aggregated to obtain the aggregated model parameter information, and the preset transmission rate prediction model on each video transmission thread is updated with the aggregated model parameter information, so that the preset transmission rate prediction model can be optimized under different video transmission threads, and the robustness of the preset transmission rate prediction model can be improved, thereby improving the prediction accuracy of the preset transmission rate prediction model. The model parameter information can be any one of model loss, model gradient, model network parameter or decision value information.

[0108] As an example, step B10 to step B30 include: obtaining model gradient information of a preset transmission rate prediction model deployed on at least one video transmission thread at the current time step, wherein each of the video transmission threads is in a different video transmission environment; aggregating each of the model gradient information to obtain aggregated model gradient information, wherein the aggregation method may be weighted summation or weighted average; and updating each of the preset transmission rate prediction models in real time according to the aggregated model parameter information, so as to predict the target transmission rate of the next time step based on the preset transmission rate prediction model updated in real time.

[0109] As an example, the preset transmission rate prediction model can be a state-action decision network constructed based on reinforcement learning, the input of the state-action decision network is the video transmission state feature, the action output by the state-action decision network is to select which transmission rate to use for video streaming transmission, and the reward is the decision value information. Then, according to the reward corresponding to the action output by the state-action decision network, the network parameters of the state-action decision network can be updated and optimized in real time. Figure 2 , S t is the video transmission state feature constructed based on network bandwidth, buffer size information, video stream bit rate and video stream weight coefficient, π(θ) is the state-action decision network, a t The action output by the state-action decision network. Figure 3 , Figure 3 This is a model deployment diagram for deploying different preset transmission rate prediction models on different video transmission threads in an embodiment of the present application. Subnetwork 1, subnetwork 2, subnetwork 3 and the main network are all state-action decision networks built based on reinforcement learning. In an embodiment of the present application, subnetworks of the same structure are placed in multiple video transmission threads for synchronous training, and then the network parameters trained in the subnetworks are concentrated into the main network. After the main network obtains the network parameters of all subnetworks and completes the update, the parameters of the main network are synchronized to each subnetwork. In this way, a preset transmission rate prediction model can be constructed based on sample data of different video transmission threads, thereby improving the adaptability of the preset transmission rate prediction model to different video transmission threads.

[0110] The embodiment of the present application provides a method for transmitting video streams. Compared with the client test, which usually needs to establish a cache area to relieve the transmission pressure of the server and speed up the loading speed of the client, the cache area of ​​the client needs to continuously feedback its real-time status to the server. As a technical means for the server's network transmission rate decision, the embodiment of the present application sets a virtual cache area on the server to simulate the client's real video transmission cache area. The virtual cache area can be used to determine the cache area size information of the real video transmission cache area, and then the obtained network state parameters and the determined cache area size information can be used as the basis for selecting the transmission rate for video streaming between the server and the client, and the target transmission rate is selected. According to the target transmission rate, the video stream is transmitted to the client. Therefore, in the embodiment of the present application, when performing video streaming transmission, the client's cache area does not need to feedback its real-time status to the server in real time, which can save the interactive process between the client's cache area and the server for buffer area status feedback, thereby saving network bandwidth, so as to overcome the technical defect that the network bandwidth is too high when sending feedback real-time status from the client to the server, thereby affecting the stability of video streaming transmission, and improving the stability of video streaming transmission.

[0111] In another embodiment of the present application, the same or similar contents as those in the above embodiment can be referred to the above description, and will not be described in detail later. Figure 4 The step of obtaining the buffer size information of the virtual buffer of the server, wherein the virtual buffer is used to simulate the real video transmission buffer of the client, comprises:

[0112] Step S11, obtaining the server-side video stream data, and extracting the time features of the server-side video stream data according to the time feature extractor to obtain a first time feature sequence;

[0113] Step S12, receiving a second time feature sequence sent by the client, wherein the second time feature sequence is obtained by the client performing time feature extraction on the client video stream data corresponding to the server video stream data;

[0114] Step S13: determining the buffer size information corresponding to the real video transmission buffer according to the first time feature sequence and the second time feature sequence.

[0115] In this embodiment, it should be noted that a time difference will be generated when the video stream is transmitted from the server to the client. Usually, a cache area is set on the client as a buffer to cache the video stream data transmitted within this time difference. Therefore, the time difference between the video stream of the server before transmission and the video stream of the client after transmission can be used as the cache area size information to measure the cache area size. The virtual cache area sets a virtual cache model, which includes a time feature extractor and an estimator. The time feature extractor is used to extract the time feature sequence in the video stream data, and the estimator is used to estimate the time difference between the video stream of the server before transmission and the video stream of the client after transmission.

[0116] As an example, step S11 to step S13 include: obtaining server video stream data transmitted by the server in the previous time step, inputting the server video stream data into the time feature extractor, mapping each video data in the server video stream data to a preset feature dimension, and obtaining a first time feature sequence; receiving a second time feature sequence sent by the client, wherein the second time feature sequence is obtained by the client performing time feature extraction on the client video stream data corresponding to the server video stream data, and the client video stream data is the server video stream data transmitted by the server at the previous time step and received by the client; estimating the buffer size information corresponding to the real video transmission buffer by inputting the first time feature sequence and the second time feature sequence into the estimator.

[0117] As an example, the temporal feature extractor includes a first 3D convolutional neural network and a second 3D convolutional neural network, wherein the first 3D convolutional neural network is used to extract temporal features from image information, and the second 3D convolutional neural network is used to extract temporal features from audio information. The server-side video stream data includes at least one frame of video data, the video data includes image information and audio information, and the first temporal feature sequence includes at least a first temporal feature.

[0118] The step of inputting the server-side video stream data into the time feature extractor and mapping each video data in the server-side video stream data to a preset feature dimension to obtain a first time feature sequence comprises:

[0119] The image information in the video data is input into a first 3D convolutional neural network, and the image information is mapped to a first preset feature dimension, so as to extract the time features of the image information in the video data, and obtain the image time features; the audio information in the video data is converted into corresponding Mel-spectrogram features, and the Mel-spectrogram features are input into a second 3D convolutional neural network, and the Mel-spectrogram features are mapped to a preset second feature dimension, so as to extract the time features of the audio information in the video data, and obtain the audio time features; the image time features and the audio time features are spliced ​​to obtain the first time features corresponding to the video data.

[0120] As an example, the client video stream data includes client video data corresponding to each frame of video data in the server video stream data, and the second time feature sequence includes second time features corresponding to each frame of client video data, wherein the method for obtaining the second time features can refer to the specific implementation method of obtaining the first time features in the above-mentioned server, which will not be repeated here.

[0121] The buffer size information includes a video stream time difference, the first time feature sequence includes a first target time feature corresponding to a video to which a preset time offset is applied, the second time feature sequence includes a second target time feature corresponding to the first target time feature, and the step of determining the buffer size information corresponding to the real video transmission buffer based on the first time feature sequence and the second time feature sequence includes:

[0122] Step S131, calculating a characteristic distance between the first target time characteristic and the second target time characteristic;

[0123] Step S132: determining the video stream time difference according to the characteristic distance and the preset time offset.

[0124] In this embodiment, it should be noted that in order to determine the time difference of the video stream, when the server sends the server video stream data to the client, it selectively applies a preset time offset to at least one frame of video data in the server video stream data, so that the frame of video data is transmitted from the server to the client in advance or delayed by a preset time offset.

[0125] As an example, step S131 to step S132 include: determining a first target time feature to which a preset time offset is applied in each first time feature in the first time feature sequence, and determining a second target time feature corresponding to the first target time feature in the second time feature sequence; obtaining a feature distance between the first target time feature and the second target time feature, wherein the feature distance may be a Euclidean distance; calculating the product between the feature distance and the preset time offset to obtain the video stream time difference. For example, the preset time offset is 40ms, assuming that the server video stream data includes a first frame of video data A, a second frame of video data B, and a third frame of video data C, if the server can choose to delay the sending of the second frame of video data B by 40ms, the client will correspondingly delay the receiving of the second frame of video data B by 80ms, further assuming that the server calculates the first target time feature b1 corresponding to the second frame of video data B to be sent later, and the client calculates the second target time feature b2 corresponding to the second frame of video data B to be received later, and further assuming that the server can calculate the target time feature distance between b1 and b2 as 5, then the video stream time difference is 5*40ms.

[0126] As an example, the steps of determining a first target time feature to which a preset time offset is applied in each first time feature in the first time feature sequence, and determining a second target time feature corresponding to the first target time feature in the second time feature sequence include:

[0127] Calculate the time feature distance between each first time feature and the second time feature corresponding to each first time feature respectively; according to the value of each time feature distance, determine the time feature distance that deviates the farthest from the average value of each time feature distance as the target time feature distance; take the first time feature corresponding to the target time feature distance as the first target time feature, and take the second time feature corresponding to the target time feature distance as the second target time feature. For example, assuming that each first time feature is (a1, b1, c1) and each second time feature is (a2, b2, c2), the time feature distance between a1 and a2 is 1, the time feature distance between b1 and b2 is 1.1, and the time feature distance between c1 and c2 is 5, then it can be determined that the time feature distance 5 is the target time feature distance that deviates the farthest from the average value of each time feature distance, so that c1 is the first target time feature and c2 is the second target time feature. The embodiment of the present application applies a preset time offset to the server video data sent by the server. Since there is a time difference in video transmission, the time feature distance between the first time feature of the server video data and the corresponding second time feature of the client video data will be enlarged or reduced after the preset time offset. Therefore, the feature distance between the first target time feature and the second target time feature can be used as a reference value for measuring the multiple of the difference between the preset time offset and the actual video stream time difference. Therefore, the product of the feature distance between the first target time feature and the second target time feature and the preset time offset can be used as an estimated value of the actual video stream time difference. Therefore, the embodiment of the present application realizes efficient estimation of the video stream time difference for video stream transmission between the server and the client.

[0128] As an example, the steps of determining a first target time feature to which a preset time offset is applied in each first time feature in the first time feature sequence, and determining a second target time feature corresponding to the first target time feature in the second time feature sequence include:

[0129] According to the preset window size, each first time feature to be screened is selected from each first time feature, and the second time feature to be screened corresponding to each first time feature to be screened is selected from each second time feature; the time feature distance between each first time feature to be screened and the second time feature to be screened corresponding to each first time feature to be screened is calculated respectively; the mean of each time feature distance is calculated to obtain the mean distance; it is detected whether there is a target time feature distance between each time feature distance and the mean distance, and if so, the first time feature corresponding to the target time feature distance is used as the first target time feature, and the second time feature corresponding to the target time feature distance is used as the second target time feature; if not, return to the execution step: according to the preset window size, each first time feature to be screened is selected from each first time feature, and the second time feature to be screened corresponding to each first time feature to be screened is selected from each second time feature, until the first target time feature and the second target time feature are determined. The embodiment of the present application gradually screens and determines the first target time feature and the second target time feature in a windowed manner, without calculating the feature distances between all the first time features and the corresponding second time features at one time. This saves part of the calculation process and improves the calculation efficiency of the video stream time difference, thereby improving the video stream transmission efficiency.

[0130] The embodiment of the present application provides a method for determining the size information of the buffer area, that is, obtaining the video stream data of the server side, extracting the time features of the video stream data of the server side according to the time feature extractor, and obtaining a first time feature sequence; receiving the second time feature sequence sent by the client, wherein the second time feature sequence is obtained by the client extracting the time features of the client video stream data corresponding to the video stream data of the server side; and determining the buffer size information corresponding to the real video transmission buffer area according to the first time feature sequence and the second time feature sequence. In the embodiment of the present application, when determining the buffer size information, the client only needs to send the second time feature sequence to the server once, and there is no need for multiple interactions between the server and the client, so it overcomes the technical defect that the server needs to interact with the client multiple times when confirming the real-time status information of the client, which is more serious for the network bandwidth occupancy, so it can reduce the network bandwidth occupancy during video stream transmission.

[0131] Reference Figure 5 The embodiment of the present application provides a video stream transmission method, which is applied to a client. In the first embodiment of the video stream transmission method of the present application, the video stream transmission method includes:

[0132] Step C10, obtaining client video stream data, and extracting time features from the client video stream data according to a time feature extractor to obtain a client time feature sequence;

[0133] Step C20, sending the client time feature sequence to the server, so that the server can transmit the video stream to the client according to the target transmission rate determined by the client time feature sequence.

[0134] As an example, steps C10 to C20 include: obtaining the video stream data transmitted from the server to the client in the previous time step as the client video stream data; by inputting the client video stream data into the time feature extractor, mapping each video data in the client video stream data to the preset feature dimension, and obtaining the client time feature sequence; sending the client time feature sequence to the server, so that the server can obtain the first time feature sequence corresponding to the server video stream data transmitted in the previous time step, and use the received client time feature sequence as the second time feature sequence, and the server determines the buffer size information corresponding to the real video transmission buffer according to the first time feature sequence and the second time feature sequence, so that the server selects the target transmission rate required for video stream transmission to the client according to the buffer size information and the obtained network status parameters; according to the target transmission rate, the video stream is transmitted to the client. Among them, the specific content of the target transmission rate required for video stream transmission between the server and the client selected by the server can refer to the specific implementation content in steps S10 to S30, which will not be repeated here.

[0135] As an example, the temporal feature extractor includes a first 3D convolutional neural network and a second 3D convolutional neural network, wherein the first 3D convolutional neural network is used to extract temporal features from image information, and the second 3D convolutional neural network is used to extract temporal features from audio information. The client video stream data includes at least one frame of client video data, the client video data includes client image information and client audio information, the client temporal feature sequence includes at least a second temporal feature, and the step of inputting the client video stream data into the temporal feature extractor and mapping each video data in the client video stream data to a preset feature dimension to obtain the client temporal feature sequence includes:

[0136] The client image information is input into a first 3D convolutional neural network, and the client image information is mapped to a first preset feature dimension so as to perform time feature extraction on the client image information to obtain the client image time feature; the client audio information is converted into a corresponding Mel spectrum feature, and the Mel spectrum feature is input into a second 3D convolutional neural network, and the Mel spectrum feature is mapped to a preset second feature dimension so as to perform time feature extraction on the client audio information to obtain the client audio time feature; the client image time feature and the client audio time feature are spliced ​​to obtain the second time feature.

[0137] Reference Figure 6 , Figure 6 This is a schematic diagram of the interaction between the server and the client when transmitting a video stream in one embodiment of the present application. The server extracts audio features and video features in the server video stream data as embedded features to form a first time feature sequence. The client extracts audio features and video features in the client video stream data as embedded features to form a second time feature sequence. After receiving the second time feature sequence sent by the client, the server performs time calculation through a virtual cache model based on the first time feature sequence and the second time feature sequence to obtain a video time difference. The server inputs the video time difference and network parameters (network state parameters) into a state-action decision network constructed based on reinforcement learning, and outputs a strategy for selecting a transmission rate. The server can select the target transmission rate for the next time step according to the strategy to transmit the video stream to the client.

[0138] The embodiment of the present application provides a method for transmitting video streams. Compared with the client side, which usually needs to establish a cache area to relieve the transmission pressure of the server and speed up the loading speed of the client, the cache area of ​​the client needs to continuously feedback its real-time status to the server side. As a technical means for the server side to make decisions on the network transmission rate, the embodiment of the present application obtains the client video stream data, and extracts the time features of the client video stream data according to the time feature extractor to obtain the client time feature sequence; the client time feature sequence is sent to the server side, so that the server side can transmit the video stream to the client according to the target transmission rate determined by the client time feature sequence. When the server side selects the target transmission rate, the client side only needs to feedback the time feature sequence to the server side once, and there is no need for multiple interactions between the server side and the client side, so the network bandwidth occupancy can be reduced. Therefore, it overcomes the technical defect that when sending feedback real-time status from the client side to the server side, it not only occupies the network bandwidth, but also affects the stability of the video stream transmission, and improves the stability of the video stream transmission.

[0139] The embodiment of the present application further provides a video stream transmission device, which is applied to a server, and the video stream transmission device includes:

[0140] An acquisition module, used to acquire buffer size information of a virtual buffer of a server, wherein the virtual buffer is used to simulate a real video transmission buffer of a client;

[0141] A transmission rate selection module, used for selecting a target transmission rate required for transmitting the video stream to the client according to the acquired network status parameters and the buffer size information;

[0142] A transmission module is used to transmit the video stream to the client according to the target transmission rate.

[0143] Optionally, the acquisition module is further used to:

[0144] Acquire the server-side video stream data, and extract the time features of the server-side video stream data according to the time feature extractor to obtain a first time feature sequence;

[0145] Receiving a second time feature sequence sent by the client, wherein the second time feature sequence is obtained by the client performing time feature extraction on the client video stream data corresponding to the server video stream data;

[0146] The buffer size information corresponding to the real video transmission buffer is determined according to the first time feature sequence and the second time feature sequence.

[0147] Optionally, the buffer size information includes a video stream time difference, the first time feature sequence includes a first target time feature corresponding to a video to which a preset time offset is applied, the second time feature sequence includes a second target time feature corresponding to the first target time feature, and the acquisition module is further used to:

[0148] Calculating a feature distance between the first target time feature and the second target time feature;

[0149] The video stream time difference is determined according to the characteristic distance and the preset time offset.

[0150] Optionally, the transmission rate selection module is further used to:

[0151] Constructing a video transmission status feature according to the acquired network status parameters and the buffer size information;

[0152] The target transmission rate required for video stream transmission between the clients is predicted by inputting the video transmission state characteristics into a preset transmission rate prediction model.

[0153] Optionally, the transmission rate selection module is further used to:

[0154] Acquire network status parameters, wherein the network status parameters include network bandwidth, video stream bit rate, and video stream weight coefficient, and the video stream weight coefficient is used to represent the frequency of the video stream being clicked by the user;

[0155] The buffer size information, the network bandwidth, the video stream bit rate and the video stream weight coefficient are spliced ​​into the video transmission status feature.

[0156] Optionally, the transmission rate selection module is further used to:

[0157] Obtain the video stream ID, the corresponding requested bit rate, the corresponding request time and the corresponding video stream parameters corresponding to the video stream to be transmitted;

[0158] splicing the video stream ID, the requested bit rate, the requested time, and the video stream parameters into a video stream weight feature;

[0159] The video stream weight feature is mapped to a corresponding video stream weight coefficient.

[0160] Optionally, the video stream transmission device is further used for:

[0161] Obtaining model parameter information of a preset transmission rate prediction model deployed on at least one video transmission thread at a current time step, wherein each of the video transmission threads is in a different video transmission environment;

[0162] Aggregating each of the model parameter information to obtain aggregated model parameter information;

[0163] The preset transmission rate prediction model is updated in real time according to the aggregated model parameter information to predict the target transmission rate of the next time step.

[0164] The video stream transmission device provided by the present application adopts the video stream transmission method in the above embodiment to solve the technical problem of low stability of video stream transmission. Compared with the prior art, the beneficial effects of the video stream transmission device provided by the embodiment of the present application are the same as the beneficial effects of the video stream transmission method provided by the above embodiment, and other technical features in the video stream transmission device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0165] The embodiment of the present application further provides a video stream transmission device, which is applied to a client, and the video stream transmission device includes:

[0166] A feature extraction module is used to obtain client video stream data, and extract time features from the client video stream data according to a time feature extractor to obtain a client time feature sequence;

[0167] The sending module is used to send the client time feature sequence to the server, so that the server can transmit the video stream to the client according to the target transmission rate determined by the client time feature sequence.

[0168] The video stream transmission device provided by the present application adopts the video stream transmission method in the above embodiment to solve the technical problem of low stability of video stream transmission. Compared with the prior art, the beneficial effects of the video stream transmission device provided by the embodiment of the present application are the same as the beneficial effects of the video stream transmission method provided by the above embodiment, and other technical features in the video stream transmission device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0169] An embodiment of the present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the video stream transmission method in the above-mentioned embodiment 1.

[0170] Reference below Figure 7 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0171] like Figure 7 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0172] Typically, the following systems can be connected to the I / O interface: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and communication devices. The communication device can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0173] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0174] The electronic device provided by the present application adopts the video stream transmission method in the above embodiment to solve the technical problem of low stability of video stream transmission. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiment of the present application are the same as the beneficial effects of the video stream transmission method provided by the above embodiment 1, and other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0175] It should be understood that the various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0176] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0177] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the video stream transmission method in the above-mentioned embodiment 1.

[0178] The computer-readable storage medium provided in the embodiment of the present application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a system or a device or used in combination with it. The program code contained on the computer-readable storage medium can be transmitted with any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0179] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0180] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: obtains the cache size information of a virtual cache on the server side, wherein the virtual cache is used to simulate a real video transmission cache on the client side; selects a target transmission rate required for video stream transmission to the client based on the obtained network status parameters and the cache size information; and transmits the video stream to the client based on the target transmission rate.

[0181] Alternatively, client video stream data is obtained, and time features are extracted from the client video stream data according to a time feature extractor to obtain a client time feature sequence; the client time feature sequence is sent to a server so that the server can transmit the video stream to the client according to a target transmission rate determined by the client time feature sequence.

[0182] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0183] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0184] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0185] The computer-readable storage medium provided by the present application stores computer-readable program instructions for executing the above-mentioned video stream transmission method, which solves the technical problem of low stability of video stream transmission. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiment of the present application are the same as the beneficial effects of the video stream transmission method provided in the above-mentioned embodiment, and will not be repeated here.

[0186] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned video stream transmission method when executed by a processor.

[0187] The computer program product provided by the present application solves the technical problem of low stability of video stream transmission. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the video stream transmission method provided by the above embodiment, which will not be repeated here.

[0188] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A video stream transmission method, characterized in that: Applied to the server, the video stream transmission method includes: Obtaining buffer size information of a virtual buffer on the server, wherein the virtual buffer is used to simulate a real video transmission buffer on the client; Selecting a target transmission rate required for transmitting the video stream to the client according to the acquired network status parameters and the buffer size information; Transmitting the video stream to the client according to the target transmission rate; The step of obtaining the buffer size information of the server virtual buffer includes: Acquire the server-side video stream data, and extract the time features of the server-side video stream data according to the time feature extractor to obtain a first time feature sequence; Receiving a second time feature sequence sent by the client, wherein the second time feature sequence is obtained by the client performing time feature extraction on the client video stream data corresponding to the server video stream data; The buffer size information corresponding to the real video transmission buffer is determined according to the first time feature sequence and the second time feature sequence.

2. The video stream transmission method according to claim 1, characterized in that: The buffer size information includes a video stream time difference, the first time feature sequence includes a first target time feature corresponding to a video to which a preset time offset is applied, the second time feature sequence includes a second target time feature corresponding to the first target time feature, and the step of determining the buffer size information corresponding to the real video transmission buffer based on the first time feature sequence and the second time feature sequence includes: Calculating a feature distance between the first target time feature and the second target time feature; The video stream time difference is determined according to the characteristic distance and the preset time offset.

3. The video stream transmission method according to claim 1, characterized in that: The step of selecting a target transmission rate required for transmitting the video stream to the client according to the acquired network status parameter and the buffer size information comprises: Constructing a video transmission status feature according to the acquired network status parameters and the buffer size information; The target transmission rate required for video stream transmission between the clients is predicted by inputting the video transmission state characteristics into a preset transmission rate prediction model.

4. The video stream transmission method according to claim 3, characterized in that: The step of constructing the video transmission status feature according to the acquired network status parameter and the buffer size information comprises: Acquire network status parameters, wherein the network status parameters include network bandwidth, video stream bit rate, and video stream weight coefficient, and the video stream weight coefficient is used to represent the frequency of the video stream being clicked by the user; The buffer size information, the network bandwidth, the video stream bit rate and the video stream weight coefficient are spliced ​​into the video transmission status feature.

5. The video stream transmission method according to claim 4, characterized in that: Get the video stream weight coefficient, including: Obtain the video stream ID, the corresponding requested bit rate, the corresponding request time and the corresponding video stream parameters corresponding to the video stream to be transmitted; splicing the video stream ID, the requested bit rate, the requested time, and the video stream parameters into a video stream weight feature; The video stream weight feature is mapped to a corresponding video stream weight coefficient.

6. The video stream transmission method according to claim 3, characterized in that: After the step of predicting the target transmission rate required for video stream transmission between the clients by inputting the video transmission state feature into a preset transmission rate prediction model, the video stream transmission method further includes: Obtaining model parameter information of a preset transmission rate prediction model deployed on at least one video transmission thread at a current time step, wherein each of the video transmission threads is in a different video transmission environment; Aggregating each of the model parameter information to obtain aggregated model parameter information; The preset transmission rate prediction model is updated in real time according to the aggregated model parameter information to predict the target transmission rate of the next time step.

7. A video stream transmission method, characterized in that: Applied to the client, the video stream transmission method includes: Acquire client video stream data, and extract time features from the client video stream data according to a time feature extractor to obtain a client time feature sequence; The client time feature sequence is sent to the server, wherein the server obtains the server video stream data, performs time feature extraction on the server video stream data according to a time feature extractor to obtain a first time feature sequence, and uses the received client time feature sequence as a second time feature sequence, determines the buffer size information corresponding to the real video transmission buffer of the client according to the first time feature sequence and the second time feature sequence, and selects the target transmission rate required for video stream transmission to the client according to the obtained network status parameters and the buffer size information, and transmits the video stream to the client according to the target transmission rate.

8. A video stream transmission device, characterized in that: Applied to the server, the video stream transmission device comprises: An acquisition module is used to obtain the cache size information of a virtual cache area of ​​a server, wherein the virtual cache area is used to simulate the real video transmission cache area of ​​a client; wherein the acquisition of the cache size information of the virtual cache area of ​​the server includes: obtaining the video stream data of the server, and extracting the time features of the video stream data of the server according to the time feature extractor to obtain a first time feature sequence; receiving a second time feature sequence sent by the client, wherein the second time feature sequence is obtained by the client extracting the time features of the client video stream data corresponding to the video stream data of the server; and determining the cache size information corresponding to the real video transmission cache area according to the first time feature sequence and the second time feature sequence; A transmission rate selection module, used for selecting a target transmission rate required for transmitting the video stream to the client according to the acquired network status parameters and the buffer size information; A transmission module is used to transmit the video stream to the client according to the target transmission rate.

9. A video stream transmission device, characterized in that: Applied to a client, the video stream transmission device comprises: A feature extraction module is used to obtain client video stream data, and extract time features from the client video stream data according to a time feature extractor to obtain a client time feature sequence; A sending module is used to send the client time feature sequence to the server, wherein the server obtains the server video stream data, extracts the time features of the server video stream data according to the time feature extractor, obtains the first time feature sequence, and uses the received client time feature sequence as the second time feature sequence, determines the buffer size information corresponding to the real video transmission buffer of the client according to the first time feature sequence and the second time feature sequence, and selects the target transmission rate required for video stream transmission to the client according to the obtained network status parameters and the buffer size information, and transmits the video stream to the client according to the target transmission rate.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the video stream transmission method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for implementing the video stream transmission method, and the program for implementing the video stream transmission method is executed by a processor to implement the steps of the video stream transmission method according to any one of claims 1 to 7.

Images