Training methods for QoE evaluation models, QoE evaluation methods and equipment

Abstract

This application provides a training method for a QoE evaluation model, a QoE evaluation method, and an apparatus, relating to the field of artificial intelligence technology. In this application, an offline data training method and a score memory unit are used to pre-construct a QoE evaluation model for real-time evaluation of streaming video. When a user watches streaming video online, by collecting low-level parameters from the terminal side in real time and inputting these parameters into the QoE evaluation model, an accurate and real-time QoE evaluation can be performed.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a training method for a QoE evaluation model, a method and device for evaluating the quality of experience (QoE). Background Technology

[0002] Streaming media technology allows video files to be continuously and uninterruptedly transmitted from a server to a terminal device, enabling users to watch video files in real time. To characterize user satisfaction with streaming video services, it is typically necessary to evaluate the quality of the streaming video service.

[0003] The Quality of Experience (QoE) defined by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) is typically used as the evaluation standard for user satisfaction with streaming video services. On the one hand, streaming video commonly uses HTTP Live Streaming (HLS), an adaptive bitrate streaming protocol based on the Hypertext Transfer Protocol (HTTP), to provide video-on-demand and live streaming services. Because HLS employs a progressive and adaptive download strategy, it's impossible to simply map network performance to the QoE of streaming video; that is, QoE evaluation of streaming video cannot be achieved using simple network performance metrics. On the other hand, traditional QoE evaluation methods model offline-extracted QoE influencing factors after the user has watched the entire video to obtain a QoE score, which lacks real-time capability.

[0004] Therefore, how to perform real-time QoE evaluation on streaming video that is currently playing has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a training method for a QoE evaluation model, a QoE evaluation method, and a device, solving the technical problem of how to perform real-time QoE evaluation on streaming video that is currently playing.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] In a first aspect, embodiments of this application provide a method for training a QoE evaluation model. The QoE evaluation model includes a machine learning classifier and a score memory unit. The method includes:

[0008] Extract target offline features from offline data, which are terminal-side parameters from the initial playback time to time t obtained from the log file of the finished streaming video. These target offline features are used to evaluate the QoE at time t.

[0009] Based on the offline features of the target, a machine learning classifier is trained to obtain the mapping relationship between the offline features of the target and the preliminary QoE evaluation result at time t;

[0010] Based on multiple preliminary QoE evaluation results, a score memory unit is trained to obtain the mapping relationship between these multiple preliminary QoE evaluation results and the final QoE evaluation result at time t; wherein, the multiple preliminary QoE evaluation results include: the preliminary QoE evaluation result at time t, and the preliminary QoE evaluation result at least one time before time t.

[0011] It is understandable that, since time t can be any time, for other times before time t, a machine learning classifier can be trained based on the offline features used to evaluate the QoE at other times, and the mapping relationship between the offline features and the preliminary QoE evaluation results at other times can be obtained.

[0012] In the above scheme, by training a machine learning classifier, preliminary QoE assessments at multiple adjacent time points can be obtained. Then, based on these preliminary QoE assessments, a score memory unit is trained to obtain the mapping relationship between the multiple preliminary QoE assessments and the final QoE assessment at time t. Thus, the machine learning classifier and the score memory unit combine to form a QoE assessment model. Using this pre-built QoE assessment model, real-time QoE assessment of subsequently played streaming video can be achieved.

[0013] In one possible implementation, the target offline features are extracted from the offline data, including:

[0014] Normalize the offline data;

[0015] Extract all offline features from the normalized offline data;

[0016] Redundant offline features are removed from all offline features to obtain the target offline features.

[0017] In one possible implementation, all offline features include at least one of the following:

[0018] Global features, which are features extracted from all data in the offline data;

[0019] Window features are features extracted from a portion of the offline data, which includes data at time t and data for a preset duration prior to time t.

[0020] Other features, which are features unrelated to network data.

[0021] In one possible implementation, redundant offline features are removed from all offline features to obtain the target offline features, including:

[0022] Iterative training is performed on all offline features:

[0023] In each round of training iteration, the offline feature with the lowest importance level is deleted according to the importance level of each offline feature in all offline features, until the preset number of iterations is reached, or until the number of remaining offline features is less than or equal to the preset number.

[0024] Use the remaining offline features as the target offline features.

[0025] In one possible implementation, the multiple QoE preliminary evaluation results include: multiple preliminary prediction probabilities, each of the multiple preliminary prediction probabilities representing the probability that the machine learning classifier predicts that the streaming video is choppy at a certain moment, which is moment t or a moment at least one moment before moment t.

[0026] Accordingly, the optimization objective of the machine learning classifier is to minimize the first loss function, which represents the loss corresponding to the difference between the initial predicted probability at a given time and the initial QoE label at that time. The initial QoE label at that time is used to indicate whether the streaming video is choppy or smooth at that time.

[0027] In one possible implementation, the final QoE evaluation result at time t includes: the final prediction probability at time t, which represents the probability that the scoring memory unit predicts the streaming video will be choppy at time t.

[0028] Accordingly, the optimization objective of the score memory unit is to minimize the second loss function, which represents the loss corresponding to the degree of difference between the final predicted probability at time t and the initial QoE label at time t.

[0029] In one possible implementation, the mapping relationship between multiple preliminary QoE evaluation results and the final QoE evaluation result at time t is expressed by the following formula:

[0030]

[0031] Among them, t i Let a represent time t. j The weighting factor represents the probability of each initial prediction, T represents the time window length, and p output(t i ) represents the final predicted probability at time t;

[0032] When j = 0 The initial predicted probability at time t;

[0033] When j≠0 Represents time t before time t. i-j The initial predicted probability.

[0034] In one possible implementation, the final QoE evaluation result at time t also includes the final predicted label at time t.

[0035] If the final prediction probability at time t is greater than or equal to 0.5, then the final prediction label at time t is used to indicate that the streaming video is choppy at time t.

[0036] If the final prediction probability at time t is less than 0.5, then the final prediction label at time t is used to indicate that the streaming video is smooth at time t.

[0037] In one possible implementation, if the initial QoE label is 0, it means the streaming video is smooth; if the initial QoE label is 1, it means the streaming video is choppy.

[0038] In one possible implementation, if the initial QoE label is 1, it means that the streaming video is smooth; if the initial QoE label is 0, it means that the streaming video is choppy.

[0039] In one possible implementation, the initial QoE tags are either manually labeled or automatically generated based on log files from streaming video.

[0040] In one possible implementation, the terminal-side parameters include at least one of the following:

[0041] Transport layer parameters, which reflect the network transmission status when playing streaming video;

[0042] QoS parameters, which are used to evaluate the network's ability to serve streaming video;

[0043] Terminal parameters are the parameters of the terminal device itself when playing streaming video.

[0044] Secondly, embodiments of this application provide a QoE evaluation method. The method includes:

[0045] The target online features are extracted from the online data, which are terminal-side parameters collected in real time from the initial playback time to the current time during the playback of the target streaming video. The target online features are used to evaluate the QoE at the current time in real time.

[0046] Input the target's online features into the machine learning classifier of the QoE evaluation model to obtain the preliminary QoE evaluation result at the current time.

[0047] Multiple preliminary QoE assessment results are input into the score memory unit of the QoE assessment model to obtain the final QoE assessment result at the current time. The multiple preliminary QoE assessment results include: the preliminary QoE assessment result at the current time, and the preliminary QoE assessment result at least one time before the current time.

[0048] It is understandable that, since the current time mentioned above is any time, for other times before the current time, the online features used to evaluate the QoE of those other times in real time can also be input into a machine learning classifier to obtain a preliminary evaluation result of the QoE of those other times.

[0049] In the above scheme, because a QoE evaluation model is pre-built, during the playback of the target streaming video, by inputting the real-time collected terminal-side parameters into the machine learning classifier of the QoE evaluation model, preliminary QoE evaluation results at multiple adjacent time points can be obtained. Furthermore, by inputting these preliminary QoE evaluation results into the score memory unit of the QoE evaluation model, "spurt" samples that are misclassified as stuttering in these preliminary QoE evaluation results are filtered out, thereby achieving accurate and real-time QoE evaluation of the streaming video being played.

[0050] In one possible implementation, target online features are extracted from online data, including:

[0051] Normalize the online data;

[0052] Extract the target online features from the normalized online data.

[0053] In one possible implementation, the target online characteristics include at least one of the following:

[0054] Global features, which are features extracted from all data in the online data;

[0055] Window features are features extracted from a portion of the online data, which includes data at the current moment and data for a preset duration prior to the current moment.

[0056] Other features, which are features unrelated to network data.

[0057] In one possible implementation, the target online features include global features;

[0058] This global feature is extracted based on the following parameters:

[0059] Online data collected at the current moment;

[0060] Global features used to evaluate the QoE of the previous time step at the current time step;

[0061] Intermediate variables corresponding to the current time and the time before the current time.

[0062] In one possible implementation, the multiple QoE preliminary evaluation results include: multiple preliminary prediction probabilities, each of the multiple preliminary prediction probabilities representing the probability that the machine learning classifier predicts the target streaming video to be stuttering at a given moment, where a given moment is the current moment or a moment in at least one moment.

[0063] In one possible implementation, the final QoE evaluation result at the current moment includes: the final prediction probability at the current moment, which represents the probability that the scoring memory unit predicts the target streaming video to be stuttering at the current moment.

[0064] Multiple preliminary QoE evaluation results are input into the score memory unit of the QoE evaluation model to obtain the final QoE evaluation result at the current time step, including:

[0065] The scoring memory unit performs a weighted summation of multiple preliminary prediction probabilities according to the weight factor of each preliminary prediction probability to obtain the final prediction probability at the current time.

[0066] In one possible implementation, the final QoE evaluation result at the current moment also includes: the final predicted label at the current moment;

[0067] Multiple preliminary QoE evaluation results are input into the score memory unit of the QoE evaluation model to obtain the final QoE evaluation result at the current time, which also includes:

[0068] Determine the final predicted label for the current moment based on the final predicted probability at the current moment;

[0069] Specifically, if the final prediction probability at the current moment is greater than or equal to 0.5, then the final prediction label at the current moment is used to indicate that the streaming video is choppy at the current moment; if the final prediction probability at the current moment is less than 0.5, then the final prediction label at the current moment is used to indicate that the streaming video is smooth at the current moment.

[0070] In one possible implementation, the terminal-side parameters include at least one of the following:

[0071] Transport layer parameters, which reflect the network transmission status when playing the target streaming video;

[0072] QoS parameters, which are used to evaluate the network's ability to serve the target streaming video;

[0073] Terminal parameters are the parameters of the terminal device itself when playing the target streaming video.

[0074] In one possible implementation, the target streaming video data is obtained at the current moment through interaction between a first network and a server;

[0075] After obtaining the final QoE evaluation result at the current moment, the method also includes:

[0076] If the QoE final evaluation results at the current moment indicate that the communication quality of the first network does not meet the communication requirements, the target streaming video data will be exchanged with the server through the second network.

[0077] The communication quality of the second network is better than that of the first network.

[0078] In one possible implementation, the final QoE evaluation result at the current moment includes: the final predicted probability that the target streaming video will stutter at the current moment;

[0079] Among them, the final QoE evaluation result at the current moment indicates that the communication quality of the first network does not meet the communication requirements, including: if the final predicted probability of the target streaming video stuttering at the current moment is greater than or equal to the preset probability, then the final QoE evaluation result at the current moment indicates that the communication quality of the first network does not meet the communication requirements.

[0080] Thirdly, this application provides a training apparatus for a QoE evaluation model, which includes units / modules for performing the method described in the first aspect above. This apparatus corresponds to performing the method described in the first aspect above, and the relevant descriptions of the units / modules in this apparatus are provided in the description of the first aspect above; for the sake of brevity, they will not be repeated here.

[0081] Fourthly, this application provides a QoE evaluation apparatus, which includes units / modules for performing the method described in the second aspect above. This apparatus can correspond to performing the method described in the second aspect above. For a brief description of the units / modules in this apparatus, please refer to the description in the second aspect above; for brevity, it will not be repeated here.

[0082] Fifthly, a terminal device is provided, including a processor coupled to a memory, the processor being configured to execute a computer program or instructions stored in the memory, such that the terminal device implements a training method for a QoE evaluation model as described in any of the first aspects, or implements a QoE evaluation method as described in any of the second aspects.

[0083] In a sixth aspect, a chip coupled to a memory is provided, the chip being used to read and execute a computer program stored in the memory to implement a training method for a QoE evaluation model as described in any of the first aspects, or to implement a QoE evaluation method as described in any of the second aspects.

[0084] In a seventh aspect, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when run on a terminal device, causes the terminal device to execute a training method for a QoE evaluation model as described in any of the first aspects, or to execute a QoE evaluation method as described in any of the second aspects.

[0085] Eighthly, a computer program product is provided, which, when run on a computer, causes the computer to execute a training method for a QoE evaluation model as described in any of the first aspects, or to execute a QoE evaluation method as described in any of the second aspects.

[0086] It is understood that the beneficial effects of the third to eighth aspects mentioned above can be found in the relevant descriptions in the first and second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0087] Figure 1 This application provides a schematic diagram of the architecture of a communication system.

[0088] Figure 2 This is a schematic diagram of the overall process of the QoE evaluation method provided in the embodiments of this application;

[0089] Figure 3 A flowchart illustrating the offline training process provided in an embodiment of this application;

[0090] Figure 4 This is a schematic diagram illustrating the sorting results of some features after a certain feature sorting, as provided in an embodiment of this application.

[0091] Figure 5 A schematic diagram illustrating the feature selection process based on the LGB-RFE classifier provided for embodiments of this application;

[0092] Figure 6 A schematic diagram of a cross-entropy loss function provided in an embodiment of this application;

[0093] Figure 7 A schematic diagram of another cross-entropy loss function provided in an embodiment of this application;

[0094] Figure 8 A schematic diagram illustrating the training process of a machine learning classifier based on confidence learning, provided for an embodiment of this application;

[0095] Figure 9 A flowchart illustrating the online evaluation process provided in this application embodiment;

[0096] Figure 10 A schematic diagram illustrating the process of network acceleration based on real-time QoE evaluation results provided in this application embodiment;

[0097] Figure 11 A schematic diagram illustrating a network acceleration scenario provided in an embodiment of this application;

[0098] Figure 12 A schematic diagram of the interface for network acceleration provided in an embodiment of this application;

[0099] Figure 13 A schematic diagram of the structure of the training device for the QoE evaluation model provided in the embodiments of this application;

[0100] Figure 14 This is a schematic diagram of the structure of the QoE evaluation device provided in the embodiments of this application;

[0101] Figure 15 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0102] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0103] In the description of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In the description of this application, "and / or" is merely a way of describing the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone.

[0104] In the specification and claims of this application, the terms "first" and "second," etc., are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of the objects. For example, "first network" and "second network," etc., are used to distinguish different networks, rather than to describe a specific order of the networks.

[0105] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0106] First, some of the terms or concepts used in this application will be explained.

[0107] 1. Streaming media refers to media formats such as audio, video, or multimedia that are continuously played in real time over a network using streaming media technology. Streaming media technology is also known as streaming media technology. It involves compressing continuous video and / or audio information, uploading it to a server, and then having the server transmit the compressed files to terminal devices in real time. This allows users to watch streaming media files while downloading them, without having to wait for the entire compressed file to download to their device. In this application embodiment, when the media format of the streaming media is video, the video is referred to as streaming video or video streaming media, etc. For ease of explanation, the following embodiments use streaming video as an example and do not limit the scope of this application embodiment.

[0108] 2. QoE refers to the overall level of user satisfaction with network services or services under certain objective conditions. QoE is a user's perception of the overall quality of network services or services. It reflects the user's satisfaction with service performance by understanding the user's subjective feelings about the quality and performance of equipment, networks, systems, applications, or services.

[0109] In some embodiments, the parameters used to evaluate QoE are simply referred to as QoE parameters. During video playback, the number of times video buffering occurs and the total duration of buffering affect the video QoE evaluation. For example, frequent buffering and / or a large total duration, i.e., stuttering events occurring during video playback, will lower the user's QoE. Furthermore, factors influencing QoE evaluation may include other factors such as user expectations, user preferences and privacy, user payment costs, and the type of service offered by the application.

[0110] 3. Quality of Service (QoS) is a technical metric applied to a network to ensure or enhance network QoE quality. QoS can be used to address issues such as network latency and congestion.

[0111] In some embodiments, the parameters used to evaluate QoS are simply referred to as QoS parameters. QoS parameters may include parameters such as system throughput, signal strength, network transmission stability, reliability, transmission latency, latency jitter, packet loss rate, transmission bit rate, bit error rate, transmission failure rate, and security. Based on these QoS parameters, it is possible to assess whether the network communication quality meets the requirements of business communication.

[0112] It should be understood that the QoE and QoS parameters mentioned above are merely illustrative examples and may include other arbitrary parameters. The specific parameters can be determined according to actual usage requirements, and this application embodiment does not impose any limitations.

[0113] It should be noted that both QoE and QoS can be used to measure the overall quality of network services. QoE is related to specific services, and different services have different QoS requirements. For example, some services are more sensitive to latency metrics in QoS, such as Voice over Internet Protocol (VoIP) services; while other services are more sensitive to packet loss rate metrics in QoS, such as file transfer services.

[0114] Based on the above description, since QoE can more completely depict the user's overall level of acceptance of the network service or business used in a certain objective environment, this application adopts QoE as the evaluation standard for user satisfaction with streaming video services.

[0115] In practice, the following QoE evaluation methods are commonly used:

[0116] One method is an offline QoE assessment. During a user's viewing of a streaming video, the terminal device's log file records information about the interactions between the system and the user, automatically capturing data such as the type, content, and timing of these interactions. After the user has finished watching the video, QoE influencing factors are extracted from the log file. Then, a model is built based on these QoE influencing factors to obtain a QoE score. However, this offline assessment method can only predict and assess QoE after the user has finished watching the video, resulting in a time lag.

[0117] Another approach is real-time QoE assessment. This method includes the following steps: first, extracting the round-trip time (RTT) information of uplink data packets in the video stream and constructing an input vector; then, constructing a neural network model containing convolutional and fully connected layers; next, inputting the constructed input vector into the neural network model, extracting features, executing fully connected layers, and predicting the video QoE metric; finally, inputting the round-trip time (RTT) information of the encrypted traffic of the video to be estimated into the trained neural network model to predict the video QoE metric. The above steps involve a large amount of repetitive computation, resulting in wasted resources. Furthermore, this approach only uses RTT as an input parameter, resulting in a relatively singular parameter dimension. This approach is also limited to the Transmission Control Protocol (TCP) and is not applicable to low-latency Internet transport layer protocols such as Quick UDP Internet Connection (QUIC) based on User Datagram Protocol. In addition, because this approach does not consider the correlation of QoE scores between adjacent time points, it leads to "spikes" in the prediction results, i.e., generating more false alarms and reducing the accuracy of QoE prediction.

[0118] In view of this, embodiments of this application provide a training method for a QoE evaluation model and a QoE evaluation method: In any scenario of playing streaming video, such as video-on-demand, online live streaming, real-time video conferencing, multimedia news releases, online advertising, e-commerce, distance education, telemedicine, or internet radio, a machine learning model and a score memory unit are trained using offline data through a hierarchical training mode, thereby pre-constructing a QoE evaluation model for real-time QoE evaluation. Thus, when a user watches streaming video online, the underlying parameters can be collected in real time through the terminal device and input into the QoE evaluation model, thereby enabling real-time evaluation and prediction of QoE, such as predicting whether buffering will occur when watching streaming video in the future.

[0119] Figure 1 A schematic diagram of the communication system architecture involved in various embodiments of this application is shown. For example... Figure 1 As shown, the communication system includes a terminal device 1 and a server 2. Terminal device 1 can access a wireless local area network (WLAN) via an access point (AP) device 3, such as a Wi-Fi network via a router, and then establish a connection with server 2 to exchange data. Alternatively, terminal device 1 can access a mobile network (also known as a cellular network or mobile data network) via a network device 4, such as a mobile network via a base station, and then establish a connection with server 2 to exchange data.

[0120] The terminal device 1 can be a mobile terminal, a non-mobile terminal, a user equipment, or other devices or apparatus capable of real-time QoE assessment. For example, a mobile terminal can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle terminal, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., while a non-mobile terminal can be a personal computer (PC), smart screen, television (TV), ATM, or self-service machine, etc. One example is that the terminal device 1 can be a smartphone with a built-in Wi-Fi module, or a computer equipped with a wireless network card. This application embodiment does not impose any limitations on the specific type of terminal device.

[0121] Server 2 can be a wide area network (WAN) web server and / or a streaming media server, etc. A streaming media server is also called a streaming media server or an audio / video (A / V) server. The specific type of server is not limited in this embodiment.

[0122] In some embodiments, data transmission between terminal device 1 and server 2 can be performed via connection-oriented TCP or connectionless User Datagram Protocol (UDP).

[0123] In some embodiments, terminal device 1 and server 2 can exchange uplink and / or downlink data. For example, terminal device 1 can send playback request information for a streaming media file to server 2. As another example, server 2 can send streaming media files to terminal device 1 in real time. Yet another example is that terminal device 1 can send streaming media file data packets to other devices through server 2, and receive media file data packets sent by other devices through server 2.

[0124] In some embodiments, server 2 can be used to provide streaming media files, and terminal device 1 can be used to play the streaming media files. Taking a server including a web server and an A / V server as an example, the streaming media transmission process can specifically include:

[0125] After the A / V server obtains the original video file, it preprocesses it to compress it into a streaming media file. When a user selects a streaming media file for playback via terminal device 1, the web browser and web server exchange control information using HTTP / TCP to retrieve the necessary real-time data from the original file. The web browser then launches the audio / video helper (A / Vhelper) program, using HTTP to retrieve relevant parameters from the A / V server to initialize the helper program. These parameters include directory information, the encoding type of the A / V data, and the server address related to A / V retrieval. Afterward, the A / V Helper program and the A / V server run the streaming media protocol to exchange control information required for A / V transmission. This streaming media protocol can be HLS or Dynamic Adaptive Streaming over HTTP (DASH), providing methods for controlling playback, fast forward, rewind, pause, and recording. The A / V server uses Real-Time Transport Protocol (RTP) / UDP to transmit A / V data to the A / V client program (i.e., the A / V helper program). When terminal device 1 receives the A / V data, the A / V client program can play it. During the playback of the A / V data, low-level parameters can be collected in real time and input into a pre-built QoE evaluation model, enabling real-time evaluation and prediction of QoE.

[0126] It should be noted that the aforementioned streaming media files can be audio, video, or multimedia files, etc. This embodiment of the application illustrates the process of server 2 sending various data packets of streaming media video to terminal device 1, and performing real-time QoE evaluation using a pre-built QoE evaluation model during the playback of the streaming media video data packets on terminal device 1.

[0127] The training method of the QoE evaluation model and the specific implementation of the QoE evaluation method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0128] Figure 2 This is a schematic diagram of the overall process of the QoE evaluation method provided in the embodiments of this application.

[0129] like Figure 2 As shown, the QoE evaluation method can include two parts:

[0130] The first part is the offline training process.

[0131] The offline training process is mainly used to train a QoE evaluation model for real-time QoE evaluation based on an offline dataset.

[0132] The offline training process can include several sub-processes such as offline dataset collection, parameter normalization, offline feature calculation, feature selection, offline training of machine learning models, and offline training of score memory units.

[0133] For example, during the playback of streaming video through a user's video application (APP) (e.g., APP1 and APP2) on a terminal device, such as video-on-demand, live streaming, real-time video conferencing, multimedia news releases, online advertising, e-commerce, distance education, telemedicine, or internet radio, the terminal device's log file can record information about the interaction between the system and the user. After the streaming video playback ends, the terminal device can obtain an offline dataset based on the log file. This offline dataset includes multiple offline parameters, which are the underlying parameters input during streaming video playback, such as transport layer parameters, QoS parameters, and / or terminal parameters. The terminal device then sequentially performs parameter normalization, offline feature calculation, offline feature selection, machine learning model training, and score memory unit training on the offline dataset to construct a QoE evaluation model for real-time QoE assessment.

[0134] The QoE evaluation model mainly consists of two sub-models: a machine learning model and a score memory unit. Offline training of the machine learning model is primarily used to train the parameter weights of the machine learning model, while offline training of the score memory unit is primarily used to train the parameter weights of the score memory unit.

[0135] Specifically, the machine learning model, also known as a machine learning classifier, is mainly used to: obtain the mapping relationship between feature vectors and the preliminary QoE evaluation results at time t by training the machine learning model offline. The score memory unit, trained offline, is mainly used to: model the machine learning model output results at time t and the time period before time t based on the correlation between the preliminary QoE evaluation results of adjacent time times in the time dimension.

[0136] In some embodiments, a confidence learning (also known as confidence degree learning) algorithm is used to assist in training the machine learning model in order to improve the generalization ability of the machine learning model.

[0137] In some embodiments, the above-described offline training process trains the parameter weights of the machine learning model and the parameter weights in the score memory unit separately through a hierarchical training mode.

[0138] For example, such as Figure 3As shown, the terminal device can continuously iterate through steps such as offline feature calculation, offline feature selection, and offline training of the machine learning model until the optimal modeling result is achieved, thus completing the training of the parameter weights in the machine learning model. After training the machine learning model, the modeling result is input into the score memory unit for training, thereby training the parameter weights in the score memory unit. Finally, the machine learning model and the score memory unit together form the QoE evaluation model for real-time QoE evaluation.

[0139] The second part is the online assessment (real-time measurement) process.

[0140] The online evaluation process is mainly used to input the online dataset collected in real time during streaming media playback into a pre-built QoE evaluation model, thereby outputting the QoE evaluation result at the current moment.

[0141] The online evaluation process can include several sub-processes such as online dataset collection, parameter normalization, online feature calculation, online inference of machine learning models, and online updating of score memory units.

[0142] For example, when a user plays a target streaming video using a video-based app on a terminal device, the terminal device can acquire an online dataset in real time. This online dataset includes multiple online parameters, which are low-level parameters collected online by the terminal device during the playback of the target streaming video, such as transport layer parameters, QoS parameters, and / or terminal parameters. The terminal device then sequentially performs parameter normalization, online feature calculation, online inference of the machine learning model, and online updating of the score memory unit on the online dataset.

[0143] The parameter normalization method in the online evaluation process is the same as that in the offline training process; both are used to normalize the parameters.

[0144] Online feature computation only requires computation of certain types of features selected in the offline training process, without requiring computation of certain types of features not selected in the offline training process.

[0145] Online inference of machine learning models is used to: utilize machine learning models trained through offline training processes to map features to preliminary QoE evaluation results.

[0146] The online update of the score memory unit is used to: input the preliminary QoE evaluation results into the score memory unit, and output the preliminary QoE evaluation results from the current time t and the time period before the current time t in a linear weighted manner to obtain the final QoE evaluation result, i.e., the final QoE evaluation result.

[0147] Understandably, by training the machine learning model and the score memory unit separately using a hierarchical training approach, a QoE evaluation model for real-time QoE assessment can be pre-built. Thus, when a user watches streaming video online, the terminal device can accurately evaluate and predict the QoE in real time by inputting the underlying parameters into the QoE evaluation model.

[0148] It should be noted that, in this embodiment of the application, the preliminary QoE evaluation result is used to determine the preliminary QoE score, and the final QoE evaluation result is used to determine the final QoE score. For example, the time interval from time t1 to time t2 will be used to determine the final QoE score. i The final QoE evaluation result is input into the preset model, and a final QoE score is obtained, which can be 1, 2, 3, 4 or 5 points.

[0149] The specific implementation methods of each sub-process of the offline training process and the online evaluation process will be described below.

[0150] Figure 3 This is a flowchart illustrating the offline training process provided in an embodiment of this application.

[0151] like Figure 3 As shown, the method may include the following steps S101 to S106.

[0152] S101. The terminal device obtains the offline dataset.

[0153] The offline dataset may include multiple offline parameters, also known as offline data. These offline parameters are low-level parameters recorded by a log file during the playback of at least one streaming video and obtained by the terminal device after the playback of the at least one streaming video ends. That is, the offline parameters are terminal-side parameters obtained from the log file of the at least one streaming video, from the initial playback time of the at least one streaming video to time t.

[0154] For example, suppose the terminal device operates from time t1 to time t i If at least one streaming video has been played continuously or intermittently within a given time period, then after finishing playing the last streaming video, the terminal device can obtain the offline dataset corresponding to that at least one streaming video. Among them, sample data It can be used to represent time t i The underlying parameters and sample data are recorded in the log file. It can be used to represent time t i-1 The underlying parameters recorded in the log file, ..., sample data It can be used to represent the underlying parameters recorded by the log file at time t1.

[0155] Specifically, one example is that the terminal device can obtain data from the initial time t1 when the video app is first used to play streaming video service to the time t2 when the streaming video is last played. i All offline parameters can be used to form an offline dataset. Another example is that the terminal device can obtain data from the last time the streaming media was played, t. i At time t i Offline parameters of a streaming video service played (e.g., continuous or intermittent) using a video app at a certain point in time t1 can be obtained. For example, offline parameters of multiple playbacks of streaming video services using a video app within the past day, week, or month can be obtained. These offline parameters can form an offline dataset.

[0156] It should be noted that in data acquisition during continuous playback, the time interval between any two adjacent moments mentioned above can be the same, for example, time t1 and time x. t-1 The time interval is 1 second. Alternatively, in data collection during interval playback, the time intervals between some adjacent moments mentioned above may differ.

[0157] Optionally, the terminal-side parameters may include at least one of the following parameters:

[0158] The first type is the transport layer parameter. This parameter is the primary parameter for training machine learning models. It reflects the network transmission status when streaming video is played on a terminal device, such as the statistical characteristics of data packets per unit time.

[0159] For example, transport layer parameters may include the number of uplink TCP / UDP packets and the total packet length per unit time. As another example, transport layer parameters may include the number of downlink TCP / UDP packets, the total packet length, the RTT value per unit time, and the number and length of retransmitted packets per unit time, among other transport layer data. Here, "unit time" refers to the time interval between two equally spaced moments. Of course, transport layer parameters may also include other parameters, which are not limited in this embodiment.

[0160] The second is the QoS parameter. This QoS parameter is also used to train machine learning models. It is used to evaluate the network's ability to provide services for streaming video.

[0161] For example, QoS parameters mainly include several important environmental parameters such as cellular and Wi-Fi signal strength and Wi-Fi negotiation rate. Of course, QoS parameters can also include availability, throughput, network transmission stability, reliability, latency, latency variation, jitter, transmission bit rate, bit error rate, transmission failure rate, reliability, security, guaranteed flow bit rate (GFBR), maximum flow bit rate (MFBR), average window and aggregate maximum bit rate (AMBR), allocation and retention priority (ARP), etc.

[0162] The third type is terminal parameters. These are the parameters of the terminal device itself when playing streaming video.

[0163] For example, terminal parameters may include audio track information, video track information, brand and model, operating system, identity document (ID), network operator, network access method, and / or IP address. Among these, the audio track information and video track information are independent of network parameters and are used to indicate the playback progress of audio and video during streaming video. In this embodiment, the audio track information and video track information are used to generate the initial QoE tag.

[0164] As an optional implementation, for each video app, the terminal device performs an offline training process based on the offline parameters of the streaming videos played by each video app within a preset time period. This determines the QoE evaluation model corresponding to each video app; that is, different QoE evaluation models are pre-created for different video apps. When a user plays a streaming video online using a video app, the QoE evaluation model corresponding to that video app can be used for real-time QoE evaluation.

[0165] As an alternative implementation, for multiple video apps, the terminal device performs an offline training process based on offline parameters of streaming videos played by these apps within a preset time period. This determines a QoE evaluation model corresponding to each of the multiple video apps; in other words, a QoE evaluation model is pre-created for each video app. When a user plays a streaming video online using any of these video apps, the QoE evaluation model can be used for real-time QoE evaluation.

[0166] S102. The terminal device normalizes the offline parameters in the offline dataset.

[0167] For offline datasets Each offline parameter x in t Normalization is performed:

[0168]

[0169] Where G(·) is the normalization function.

[0170] On the terminal device, offline dataset After normalizing each offline parameter, the offline dataset is updated to... Among them, sample data It can be used to represent offline parameters Parameters after normalization, sample data It can be used to represent offline parameters Normalized parameters, ..., sample data It can be used to represent offline parameters The parameters after normalization.

[0171] Alternatively, normalization methods may include, but are not limited to, standardization, interval scaling, and discretization.

[0172] 1) Standardization Methods

[0173] If the terminal device adopts a standardized approach: Normalizing the offline parameters will normalize them to the (0,1) distribution region, making the processed offline parameters conform to a standard normal distribution. Here, μ is the expected value of all sample data, and σ is the standard deviation of all sample data.

[0174] 2) Range scaling method

[0175] Interval scaling methods are suitable for sample data that do not conform to a normal distribution. Specifically, they can include:

[0176] Range scaling method:

[0177] Logarithmic method:

[0178] Methods for taking the square root:

[0179] If the terminal device uses the range scaling method: This allows offline parameters with relatively small data differentiation to be scaled to the [0,1] range. max x is the maximum value among all sample data. min It is the minimum value among all sample data.

[0180] If the terminal device uses the interval scaling method: or This allows offline parameters with significant data differentiation and imbalance to be scaled down to a certain range.

[0181] 3) Discretization method

[0182] If the terminal device uses a discretization method, then the continuous value of the offline parameter x can be... t Bucketing to obtain discrete parameters This increases the frequency of identical parameter values, reduces outliers, and effectively filters out abnormal values.

[0183] It should be understood that regardless of which normalization method the terminal device uses to normalize the offline parameters, it can convert dimensional offline parameters into dimensionless offline parameters, i.e., scalar parameters, thereby making the distribution of different parameters more similar, and thus improving the training efficiency and robustness of the model.

[0184] S103. The terminal device calculates the offline characteristics of the normalized offline parameters.

[0185] In this embodiment of the application, the offline features of the offline parameters can be represented by feature vectors.

[0186] After normalizing the offline parameters of the offline dataset, the terminal device can process data from time t1 to time t2. i Extract the parameters from the offline parameters used to evaluate time t. i eigenvectors of QoE For example, feature vector Where R represents the real number field; N represents the number or dimension of features contained in each sample data. Feature vector It can be understood as an N-dimensional vector.

[0187] For example, used to evaluate time t i Offline features of QoE Represented as:

[0188]

[0189] Where D(·) is the offline feature extraction function.

[0190] Optionally, the offline features of the offline parameters may include, but are not limited to, window features, global features, and at least one of other features.

[0191] 1) Window features

[0192] Used to evaluate time t i The window characteristics of QoE can be represented as:

[0193]

[0194] Among them, D window (·) is the window feature extraction function.

[0195] In this embodiment of the application, window features Only with time windows [t i -T1+1,t i The input is related to offline parameters. The duration of this time window is T1, and this time window ∈ [t1, t2]. i ].

[0196] Optionally, the terminal device may employ (weighted) average, median, maximum, minimum, standard deviation, and other composite statistical methods, based on the time window [t]. i -T1+1,t i The offline parameters are input, and the window features are calculated.

[0197] It is understandable that, since the duration T1 of the time window can be considered a constant, the time complexity and space complexity of calculating the window features are both O(1), i.e., constant level. Here, time complexity refers to the amount of computation required to execute the algorithm, and space complexity refers to the memory space required to execute the algorithm.

[0198] 2) Global features

[0199] Used to evaluate time t i The global characteristics of QoE can be represented as:

[0200]

[0201] Among them, D whole (·) is the global feature extraction function.

[0202] In this application embodiment, global features With time interval [t1,t] i The input offline parameters are relevant. Among them, the time period [t1,t] is related to the input offline parameters. i [This represents the total time period corresponding to the offline dataset.]

[0203] Optionally, the terminal device may use methods such as (weighted) average, maximum, minimum, skewness, and kurtosis to determine the metric based on the time period [t1, t2]. i The input offline parameters are used to calculate global features. Among them, skewness is a feature number that characterizes the degree of asymmetry of the probability density curve relative to the mean, and kurtosis is a feature number that characterizes the height of the peak of the probability density curve at the mean.

[0204] It should be noted that, theoretically, the time complexity and space complexity for calculating global features are both O(t), which has a linear relationship with the service duration (t i - t1), that is, the longer the continuous duration of the streaming media video service, the higher the frequency of the time complexity and space complexity of the global features.

[0205] 3) Other features

[0206] Features that have nothing to do with network parameters can all be called other features. For example, the session duration of the streaming media video service, etc.

[0207] It should be noted that the time complexity and space complexity for recording the real-time attributes of the entire streaming media video service are both O(1), that is, at the constant level.

[0208] S104. The terminal device performs feature selection on the offline features.

[0209] Since the extracted feature vectors contain some features with relatively high redundancy, and these features with high redundancy will make the training of the machine learning model and the score memory unit more complex and time-consuming. Therefore, the terminal device can adopt a feature selection algorithm to remove these features with high redundancy and obtain a refined feature vector so as to reduce the training complexity and speed up the training speed. Among them, N' is used to represent the refined dimension, and N' < N. That is, the feature vector can be understood as an N'-dimensional vector.

[0210] Optionally, the feature selection algorithm may include but is not limited to: the method based on mutual information, the method based on maximum correlation - minimum redundancy, and the feature selection method based on the wrapper method, etc.

[0211] 1) Method based on mutual information

[0212] Statistically calculate the mutual information between each dimension of the N-dimensional features and the label, and select the N' dimensions of features with the highest mutual information as Among them, mutual information is a measure of information in information theory, statistics, and machine learning, which is used to represent the amount of information contained in one random variable about another random variable, or the reduction in uncertainty of one random variable due to knowing another random variable.

[0213] It should be noted that, in the embodiments of this application, "label" refers to the QoE label used to evaluate streaming video. For example, when the QoE label is 0, it means the streaming video is smooth; when the QoE label is 1, it means the streaming video is choppy. As another example, when the QoE label is 1, it means the streaming video is smooth; when the QoE label is 0, it means the streaming video is choppy.

[0214] 2) The method based on maximum correlation and minimum redundancy

[0215] In this method, a one-dimensional feature is selected from the set of features to be chosen each time. This feature has the highest correlation with the label (i.e., high mutual information) and the lowest redundancy with the already selected feature set (i.e., low mutual information). This feature is then input into the already selected feature set. This process is repeated N' times to obtain the selected feature vector.

[0216] 3) Feature selection method based on wrapper

[0217] The process of the feature selection method based on wrappers includes:

[0218] Window features, global features, and other features are obtained through offline feature computation. Then, a classifier is selected as the base classifier, and the feature vector is learned using this classifier. The mapping relationship between features and labels. Then, based on the contribution (i.e., importance) of different features to the labels given by the classifier, the feature vector is... Remove the feature with the lowest one-dimensional contribution. Repeat this process N' times until the number of features is reduced to the desired number, ultimately yielding the selected feature vector.

[0219] In some embodiments, a distributed gradient boosting machine (LGB) based on the decision tree algorithm can be used as the base classifier for multi-round training. In each round of training, all features are ranked according to their importance. Then, the features with the lowest importance are removed from the optimal feature set S. Then, the iteration continues until only v features remain in the optimal feature set S, or until a preset number of iterations is reached. Here, v is a preset value.

[0220] Calculating feature importance (also known as importance ranking) in the LGB model can be done using the following two methods:

[0221] Split method: Calculates the importance of a feature by using the total number of times it is used as a splitting attribute across all trees;

[0222] Gain method: Calculates the importance of a feature by using the total gain derived from using the feature as a partitioning attribute.

[0223] Specifically, the recursive feature elimination (RFE) method based on the LGB classification model can be used to select offline features.

[0224] For example, the LGB-RFE feature selection algorithm is as follows:

[0225] Input: A training dataset (X, Y), with features numbered. X and Y can represent different types of training data. Specifically, X = (X0, X1, ..., X...). n Y = (Y0, Y1, ..., Y) n In this set X, each data point is m-dimensional, for example, X0 = (x0, x1, ..., x...). m Each data point in set Y is 1-dimensional.

[0226] Output: Feature selection results, i.e., the sorted feature list.

[0227] process:

[0228] 1. Initialization

[0229] The existing feature subset S = [1,2,…,m], and the feature sorting list r = [].

[0230] 2. for i = 1 to m do / / Incrementing for loop, 1 is the initial value of the loop, and m is the final value of the loop.

[0231] i. Train the LGB model using the existing feature subset X = X(:,S) of all training data;

[0232] ii. Calculate the importance level a of each feature in training the LGB model in i. i / / Importance level a i It can be calculated using the split or gain method. i is a positive integer less than or equal to m.

[0233] iii. Find the feature with the lowest importance in ii, f = argmin(a); / / argmin(·) is used to represent the variable value that minimizes the objective function.

[0234] iv. Update the feature sorting list r = [s(f), r];

[0235] v. Remove the features with the lowest importance and update the existing feature subset list S = [1,…,f-1,f+1,…,m];

[0236] end for / / end for is used to indicate the end of a loop.

[0237] 3. For the final feature sorting list, restore the feature names according to the feature numbers.

[0238] Taking a certain video live stream as an example, combined with Figure 4 and Figure 5 This paper describes a method for selecting offline features based on the LGB-RFE classifier.

[0239] Figure 4 This is a schematic diagram illustrating the sorting results of some features after a certain feature sorting, as provided in an embodiment of this application. For example... Figure 4 As shown, after the terminal device sorts the features in descending order of the sum of the gains from feature splitting, the top 15 features are: feature 'dlinkPktLen_window_6_mean', feature 'uplinkRetransRate_window_10_mad', feature 'uplinkRetransRate_window_18_mad', ..., and feature 'IteRssi'. The horizontal axis represents the gain magnitude, and the vertical axis represents the feature.

[0240] Figure 5 This is a schematic diagram illustrating the feature selection process based on the LGB-RFE classifier provided in an embodiment of this application. Figure 5 As shown, after constructing features from the original features, a total of 70 features, including the original features, can be calculated. Then, the LGB-RFE classifier ranks all features based on the sum of information gain from feature splits or the total number of feature splits, as the basis for calculating the importance of different features to the label. The features with the lowest importance are then removed from the optimal feature set.

[0241] During the initial feature sorting, a sorted list ['uplinkPktNum_window_18_mean', 'uplinkPktLen_window_18_mean', ..., 'uplinkPktLen'] is generated based on the importance of each feature in the 70-dimensional feature set. Since feature 'uplinkPktLen' has the lowest importance in the initial sorting, it is removed, and the existing feature subset list is updated. Then, the updated feature subset list is trained again using the LGB-RFE classifier, and the importance of each feature is calculated.

[0242] During the second feature ranking, a ranking list ['uplinkPktNum_window_18_mean', 'uplinkPktLen_window_18_mean', ..., 'synNum'] is obtained according to the importance of each feature among the remaining 69 features. Since feature 'synNum' has the lowest importance in the second feature ranking, it is removed, and the existing feature subset list is updated. Then, the updated existing feature subset list is trained again using the LGB-RFE classifier, and the importance of each feature is calculated.

[0243] Then, the LGB-RFE classifier repeats the above process until the feature set is empty, and outputs the sorted features.

[0244] It should be noted that S101 to S104 above are for obtaining the time t used for evaluation. i eigenvectors of QoE This is used as an example for illustration. It can be understood that for time t... i The same method can be used to obtain the corresponding feature vector at any previous time point, which will not be elaborated upon in this application.

[0245] S105, Terminal devices train machine learning models.

[0246] The machine learning model described above is also called a machine learning classifier.

[0247] After obtaining the feature vector Then, the terminal device can train a machine learning classifier to obtain feature vectors. With at time t i Mapping relationship F of the preliminary QoE assessment results ml (·)

[0248] At time t i The preliminary QoE assessment results can be represented by at least one of the following:

[0249] At time t i The probability of buffering in streaming video services, and a preliminary prediction of the probability;

[0250] At time t i The label for streaming video services, i.e., the initial predicted probability;

[0251] At time t i Latency of streaming video services;

[0252] At time t i and time t iThe number of times and total duration of buffering in streaming video services over a previous period;

[0253] At time t i and time t i Packet loss rate, etc., in the previous period of time.

[0254] At time t i Preliminary prediction of the probability of buffering in streaming video services Taking the preliminary QoE evaluation result used to represent the predicted output of a machine learning classifier as an example, the following relationship exists:

[0255]

[0256]

[0257] Among them, when "When" indicates that the machine learning classifier predicts at time t. i Streaming video services will not experience buffering; when "When" indicates that the machine learning classifier predicts at time t. i Streaming video services will inevitably experience buffering or stuttering. That is, The larger the value of , the higher the probability that the machine learning classifier will predict that the streaming video service will experience buffering.

[0258] Optionally, the machine learning classifier used for training can be any of the following: decision tree, support vector machine (SVM), random forest (RF), reinforcement learning or boosting method (AdaBoost), and gradient boosting decision tree (GBDT), etc.

[0259] In this embodiment of the application, the optimization objective for training the machine learning classifier is:

[0260]

[0261] in, For time t i The initial QoE tag labeled on the playing streaming video. This initial QoE tag is either manually labeled or automatically generated based on the streaming video's log file. This initial QoE tag is also called the raw QoE tag.

[0262] Taking manually labeled initial QoE tags as an example, one possibility is that an initial QoE tag of 0 indicates that the streaming video is manually labeled as smooth, while an initial QoE tag of 1 indicates that the streaming video is manually labeled as choppy. Another possibility is that an initial QoE tag of 0 indicates that the streaming video is manually labeled as choppy, while an initial QoE tag of 1 indicates that the streaming video is manually labeled as smooth.

[0263] L(·) is the loss function, used to represent the "risk" or "loss" of a random event. express Make decisions under certain conditions The corresponding loss or risk, that is, the probability used to represent the initial prediction. and initial QoE tag The loss corresponds to the degree of difference. It should be understood that... The smaller the value, the lower the loss or risk.

[0264] "St" is an abbreviation for "subject to," used to indicate that something satisfies a constraint. In the embodiments of this application, This indicates that the loss function To find the minimum value, that is, the optimization objective of training a machine learning classifier is to minimize the initial predicted probability. and initial QoE tag To minimize the loss corresponding to the degree of difference, thereby bringing the loss or risk to a minimum.

[0265] For example, the above loss function It can be a cross entropy loss function, a hinge loss function, or an exponential loss function, etc.

[0266] The cross-entropy loss function for a single sample can be:

[0267] .

[0268] Where a is the base of the logarithmic function. For example, a can be an irrational number e.

[0269] Accordingly, the total loss function for N samples can be:

[0270]

[0271] It should be noted that the cross-entropy loss function described above is applicable to binary classification scenarios, such as scenarios where streaming video is either smooth or choppy. It should be understood that when classifying streaming video using multi-class scenarios other than binary classification, other forms of loss functions can also be used, and this application does not limit such applications.

[0272] To more intuitively understand the cross-entropy loss function mentioned above, let a = e, and combine it with... Figure 6 and Figure 7 An example is provided.

[0273] like Figure 6 As shown, when the initial QoE label At that time, the cross-entropy loss function The horizontal axis represents the probability of prediction stuttering in the output of the machine learning classifier. The ordinate represents the cross-entropy loss function L. When predicting the probability of stuttering... The closer the value is to 1, the smaller the cross-entropy loss function L becomes; when predicting the probability of stuttering... The closer it is to 0, the larger the cross-entropy loss function L becomes.

[0274] like Figure 7 As shown, when the initial QoE label At that time, the cross-entropy loss function The horizontal axis represents the probability of prediction stuttering in the output of the machine learning classifier. The ordinate represents the cross-entropy loss function L. When predicting the probability of stuttering... The closer the value is to 0, the smaller the cross-entropy loss function L becomes; when predicting the probability of stuttering... The closer it is to 1, the larger the cross-entropy loss function L becomes.

[0275] The above Figure 6 and Figure 7 The cross-entropy loss function L in the model characterizes the probability of prediction stuttering. With the initial QoE tag The difference. Because when predicting the probability of stuttering. With the initial QoE tag The smaller the difference, the smaller the cross-entropy loss function L, and the smaller the "penalty" on the machine learning model. Therefore, when training a classifier, the probability of prediction lag should be minimized as much as possible. With the initial QoE tag The gap is calculated to obtain the final machine learning classifier that minimizes the loss function.

[0276] It should be noted that S105 in the above embodiment describes the initial QoE label as the true QoE label. However, in actual implementation, since the initial QoE label is manually labeled or automatically generated based on log files, and manual labeling has error issues, while log files have non-real-time issues, there is a certain time delay error between the initial QoE label and the true QoE label, i.e., there are noisy labels. These noisy labels will interfere with the classifier's learning. In addition, even under the true QoE label background, before and after the moment when QoE changes abruptly (e.g., the QoE label at time t is stuttering, while the QoE label at time t-1 is smooth; or, the QoE label at time t is smooth, while the QoE label at time t-1 is stuttering), the difference in real-time network parameters is small, but the QoE evaluation results corresponding to these moments may be completely different. This will cause similar feature inputs to be mapped to different QoE evaluation results, thus interfering with the learning of the machine learning classifier. Therefore, the machine learning classifier trained under such circumstances does not have good robustness and generalization.

[0277] To address the issue of poor robustness and generalization of machine learning classifiers, embodiments of this application provide a confidence-based learning algorithm to assist in the training of machine learning classifiers.

[0278] like Figure 8 As shown, the training process for a machine learning classifier based on confidence learning can include:

[0279] Step 1. Input the feature vector set and QoE sets The QoE set consists of time intervals {t1, t2, ..., t}. i The initial QoE label for each time step in the sequence , ..., is composed of labels that are either manually labeled or automatically generated based on log files.

[0280] Step 2. Train the machine learning classifier and learn The mapping relationship F ml (·)

[0281] Step 3. From time {t1, t2, ... t... i Extract the time t when all QoE tags mutate from , ...} i' , and in t i' The time before and after [t] i'-T ,…,t i'+T ] feature vector set And the set of QoEs corresponding to the times of these mutations. Through mapping relationship F ml (·), calculated to obtain The set of predicted stuttering probabilities output by the machine learning classifier

[0282] Step 4. Traverse the QoE collection With the set of predicted stuttering probabilities At each time step, calculate the initial QoE label y. t With the predicted stuttering probability p t The cross-entropy. Then, in descending order of cross-entropy, the QoE set is processed. The samples are sorted.

[0283] Step 5. Change the QoE labels of the top X% of samples in the ranking results to their opposite QoE labels, for example, change the stuttering label to the smooth label or vice versa, and save the updated labels to the collection. In the middle. Where X is a preset value, such as X = 5, 10 or 20, etc.

[0284] Step 6. Repeat steps 2-5 above for the preset number of iterations. Then, use the machine learning classifier from the last round as the final machine learning classifier.

[0285] For example, consider the first iteration of a confidence-based machine learning classifier training process. Assume that at time t... i' The QoE tag undergoes a mutation, and T = 5. Table 1 below shows the mutation time t. i' The correspondence between the initial QoE labels, predicted stuttering probability, and cross-entropy at various time points before and after the event.

[0286] Table 1

[0287] time Initial QoE tag Predicting the probability of stuttering Cross-entropy The revised QoE label <![CDATA[t i'-5 ]]> 0 0.3 0.3567 0 <![CDATA[t i'-4 ]]> 0 0.2 0.2231 0 <![CDATA[t i'-3 ]]> 0 0.1 0.1054 0 <![CDATA[t i'-2 ]]> 0 0.3 0.3567 0 <![CDATA[t i'-1 ]]> 0 0.6 0.9163 1 <![CDATA[t i' ]]> 1 0.6 0.5108 1 <![CDATA[t i'+1 ]]> 1 0.9 0.1054 1 <![CDATA[t i'+2 ]]> 1 0.7 0.3567 1 <![CDATA[t i'+3 ]]> 1 0.7 0.3567 1 <![CDATA[t i'+4 ]]> 0 0.5 0.6931 0 <![CDATA[t i'+5 ]]> 0 0.1 0.1054 0

[0288] Referring to Table 1, machine learning classifiers can utilize the mapping relationship F ml (·), output and time t i'-5 t i'-4 、…、t i' 、…、t i'+4 t i'+5 The predicted stuttering probability corresponds to each time step. Then, iterate through each time step and calculate the cross-entropy between the initial QoE label and the predicted stuttering probability at each time step.

[0289] Specifically, the cross-entropy between the initial QoE label and the predicted stutter probability at each time step can be calculated according to the above embodiment. Figure 6 and Figure 7 The calculation method provided is used for calculation: when the initial QoE label At that time, the cross-entropy loss function When the initial QoE label At that time, the cross-entropy loss function This allows us to calculate the cross-entropy between the initial QoE label and the predicted stutter probability at each time step, and then sort them in descending order of cross-entropy. Assume we take the QoE labels of the top 10% of the sorted samples and modify them, i.e., take one sample from Table 1. Since time t... i'-1 The corresponding cross-entropy of 0.9163 is the highest among all cross-entropies, therefore time t is... i'-1 The initial QoE label is changed from "0" to "1". Then, the updated label is saved to the collection. In, that is, the set Tags in Change it to "1". Then, the machine learning classifier can continuously iterate through the training process until the final machine learning classifier is obtained.

[0290] It is understandable that by changing the QoE labels before and after some mutation moments, these labels are generated iteratively by the classifier itself, rather than being fixed to manually labeled labels, thus improving the generalization ability of the machine learning classifier.

[0291] It should be noted that S102-S105 above employs an iterative training method based on feature engineering and a machine learning classifier. This iterative training method assumes that the training samples are time-independent. However, in reality, there are time dependencies between training samples, which can lead to "spiking" in the machine learning classifier's output, resulting in more false alarms and reducing the accuracy of QoE prediction.

[0292] To improve the accuracy of the overall QoE evaluation model, based on the correlation between the output results of the machine learning model's predictions of adjacent time points in the time dimension, this application embodiment also proposes to train a score memory unit based on the prediction results of multiple adjacent time points output by the machine learning model.

[0293] S106. The terminal device trains the score memory unit based on the output of the machine learning model.

[0294] Suppose the machine learning model is effective at time t i The predicted probability of buffering in the streaming video is: For time t i The predicted stuttering probabilities for streaming video at various points in time over a period of time are as follows: The training process of the score memory unit is then a process of modeling these stuttering probabilities.

[0295] The preliminary QoE assessment results can be used to predict the initial probability, while the final QoE assessment results can be used to predict the final probability.

[0296] Score memory unit for time t i The final predicted probability is:

[0297] .

[0298] Among them, a j The weighting factors represent the initial predicted probabilities, T2 represents the time window length, and p output (t i ) represents time t i The final predicted probability. When j = 0, Representing time t i The initial predicted probability; when j≠0, Represents time t i Previous moment t i-j The initial predicted probability.

[0299] The preliminary QoE assessment results can be used to predict the initial probability, while the final QoE assessment results can be used to predict the final probability.

[0300] Furthermore, the final QoE evaluation result may also include the final predicted label. For example, if at time t... i If the final predicted probability is greater than or equal to 0.5, then the final predicted label is used to represent the streaming video at time t. i For stuttering; if time t i If the final predicted probability is less than 0.5, then the final predicted label is used to represent the streaming video at time t. i For smoothness.

[0301] The time window length T2 is a predefined hyperparameter. Typically, the time window length T2 is 2 or 3.

[0302] For example, when T2 = 2,

[0303] For example, when T3 = 3,

[0304] The optimization objective of the training process for the scoring memory unit is:

[0305]

[0306] in, For time t i The initial QoE tag of the streaming video being played.

[0307] It should be noted that the initial QoE label used to train the score memory unit and the initial QoE label used to train the machine learning classifier are the same label, which are either manually labeled or automatically generated based on the log file of the streaming video.

[0308] L(·) is the loss function, used to represent the "risk" or "loss" of a random event. p output (t i Making decisions under these conditions The corresponding loss or risk. That is, used to represent the final predicted probability p. output (t i ) and initial QoE tag The loss corresponds to the degree of difference. It should be understood that... The smaller the value, the lower the loss or risk.

[0309] "St" is an abbreviation for "subject to," used to indicate that something satisfies a constraint. In the embodiments of this application, This indicates that the loss function To minimize the value, that is, the optimization objective of the training score memory unit is to minimize the final predicted probability p. output (t i ) and initial QoE tag Minimize the loss corresponding to the degree of difference, thereby minimizing the loss or risk. Furthermore, since the final prediction probability p... output (t i The initial predicted probabilities are obtained by weighting and summing them according to their respective weight factors. Therefore, the optimization objective of the training score memory unit is to find the optimal weight factor so that the loss or risk approaches the minimum.

[0310] It is understandable that training the score memory unit will filter out many "spurt" samples that are misjudged as stutters, thereby improving the accuracy of the entire QoE evaluation model and reducing the sensitivity of predicting stutters.

[0311] It should be noted that the above embodiments are all illustrated by the example of the training method of the score memory unit and the QoE evaluation method being executed by the terminal device. It should be understood that in actual implementation, the training method of the score memory unit and the QoE evaluation method can also be executed by the server. The embodiments of this application are not limited to this.

[0312] In the above embodiments, the terminal device pre-constructs a QoE evaluation model for QoE evaluation of the streaming media video based on an offline dataset corresponding to at least one streaming media video, and then saves the QoE evaluation model. Thus, during the playback of the streaming media video on the terminal device, a real-time QoE evaluation of the playing streaming media video can be performed based on this QoE evaluation model. The following will combine... Figure 9 This section introduces the online evaluation process for a target streaming video that is currently playing.

[0313] Figure 9 This is a flowchart illustrating the online evaluation process provided in an embodiment of this application. Figure 9 As shown, the method may include the following steps S201-S205.

[0314] S201. The terminal device obtains online parameters.

[0315] Unlike the offline parameters obtained by the terminal device after the streaming video playback ends in the above embodiments, online parameters are terminal-side parameters collected in real time during the playback of the target streaming video, from the initial playback time to the current time. Online parameters are also called online data.

[0316] Assuming the current time is t r This means that the terminal device can start collecting online parameters in real time at the initial moment of playing the target streaming media. For example, it can collect online parameter 1 at the initial playback time t0, online parameter 2 at the next moment after the initial playback time t0, and so on, until the current moment t0. r Collect online parameters

[0317] Optionally, the terminal-side parameters may include at least one of the following parameters:

[0318] Transport layer parameters, which reflect the network transmission status when playing the target streaming video;

[0319] QoS parameters, which can be used to evaluate the network's ability to serve a target streaming video;

[0320] Terminal parameters are the parameters of the terminal device itself when playing the target streaming video.

[0321] For a description of the online parameters, please refer to the detailed description of the offline parameters in the above embodiments, which will not be repeated here.

[0322] S202, Terminal equipment normalizes online parameters.

[0323] The terminal device obtains the current time t. r Input online parameters After that, you can adjust the online parameters. Normalization is performed:

[0324]

[0325] Where G(·) is the normalization function.

[0326] It should be noted that the method for normalizing online parameters is the same as the method for normalizing offline parameters. For example, online normalization methods may include, but are not limited to, standardization, interval scaling, and discretization. Please refer to the specific descriptions in the above embodiments; they will not be repeated here.

[0327] It should be understood that during the playback of the target streaming video, by converting the real-time collected dimensional online parameters into scalar parameters, the distributions of different parameters become closer, thereby improving the prediction accuracy and robustness of the model.

[0328] S203. The terminal device performs online feature calculation on the normalized online parameters.

[0329] In this embodiment of the application, the online features of the online parameters can be represented by feature vectors.

[0330] Online features may include, but are not limited to, window features, global features, and at least one of other features.

[0331] The window feature is a feature extracted from a subset of the data, which includes the current time t. r and the current time t r Previously preset duration online data. Global features are features related to all data, which includes the data from the initial moment when the video application starts playing the target streaming video to the current moment t. r The online data. Other features are those unrelated to the network data.

[0332] In the offline training process described in the above embodiments, the terminal device first acquires feature vectors of all types corresponding to the entire time period, and then selects feature vectors of a subset of types from all feature vectors. Unlike the offline training process, the online evaluation process only calculates the feature vectors of the subset of types selected in the offline training process, without calculating all types of feature vectors. For example, in the offline training process, the terminal device first normalizes the offline data, then calculates all offline features of the normalized offline data, and then removes redundant offline features from all offline features to obtain the target offline features. In the online evaluation process, the terminal device first normalizes the online data, and then directly calculates the target online features of the normalized online data, without calculating redundant offline features.

[0333] For example, suppose that in the offline training process, the terminal device ultimately selects features from 70 dimensions, such as... Figure 5 The 15 feature types shown are: feature 'dlinkPktLen_window_6_mean', feature 'dlinkPktLen_window_6_mean', feature 'uplinkRetransRate_window_18_mad', ..., feature 'dlinkRetransRate_window_18_mean', and feature 'IteRssi'. During online evaluation, the terminal device only needs to compute these 15 feature types, eliminating the need to compute redundant features, thus reducing the computational complexity and resource consumption of the terminal device.

[0334] Furthermore, in the offline training process, the time and space complexity of window features and other features are both constant, but the time and space complexity of global features are both O(t), which is significantly different from the business duration (t). i -t1) exhibits a linear relationship. Unlike the offline training process, for the global features of the online evaluation process, this application embodiment provides an incremental feature update method: based on the current time t r Collected online data for evaluating the current time t r The global characteristics of QoE at the previous time step, and its relationship with the current time step t. r and the current time t r The intermediate variables corresponding to the previous time step are used to obtain the data for evaluating the current time step t. r The global characteristics of QoE.

[0335] Specifically, this can be expressed by the following relation:

[0336]

[0337] Where D(·) is an incremental function, t r This refers to the current moment. r-1 The previous moment before the current moment. For time t r-1 The corresponding intermediate variables. To be relative to the current time t r The corresponding intermediate variables. For time t r-1 Corresponding online features For time t r The corresponding online features.

[0338] For example, the following provides a specific incremental feature update method:

[0339]

[0340] It should be understood that in the incremental feature update algorithm, by introducing a time interval t... r-1 Corresponding intermediate variables This allows the terminal device to adjust according to time t. r-1 Corresponding online features At time t r Input online parameters The calculation yields the result at time t. r Corresponding online features and time t r-1 Corresponding intermediate variables Without needing to store online features corresponding to all times. This reduces the time and space complexity of global features to O(1), which is constant level.

[0341] S204. Terminal devices use machine learning models to predict the probability of lag online.

[0342] The machine learning model trained in the offline training process described above, i.e., the mapping relationship F between the feature vector and the preliminary QoE evaluation result, has been established. ml (·). Therefore, the result calculated using the above S203 is related to the current time t. r Corresponding online features Then, online features can be used to evaluate the current time t in real time. r QoE. For example, online features. Input this machine learning model, and it will output the result at the current time t. r Preliminary probability of buffering in streaming video services

[0343] Mapping relationship F between eigenvectors and preliminary QoE evaluation results ml (·), and the current time t r Corresponding online features And at the current time t r The probability of buffering in streaming video services The following relationship exists:

[0344]

[0345]

[0346] Among them, when "Time" indicates that the machine learning classifier predicts at the current time t. r Streaming video services will not experience buffering; when "Time" indicates that the machine learning classifier predicts at the current time t.r Streaming video services will inevitably experience buffering or stuttering. That is, The larger the value of , the higher the probability that the machine learning classifier will predict that the streaming video service will experience buffering.

[0347] S205. The terminal device updates the stuttering probability predicted online by the machine learning model to the final stuttering probability through the score memory unit.

[0348] The score memory unit, i.e., the weighted coefficients for predicting the probability of stuttering, has already been trained in the offline training process described above. Therefore, the current time t is predicted through the machine learning model. r The probability of buffering in streaming video services At time t r-1 The probability of buffering in streaming video services At time t r-2 The probability of buffering in streaming video services …These predicted stutter probabilities are output in a linearly weighted manner to obtain the final stutter probabilities:

[0349]

[0350] Among them, a j This represents the weighting factor for each preliminary QoE assessment result. T2 represents the time window length. p output (t r ) represents the current time t r The final probability of stuttering.

[0351] When j = 0 Represents the current time t r The initial stutter probability; when j≠0, Represents the current time t r Previous moment t i-j The initial probability of stuttering.

[0352] The time window length T2 is a predefined hyperparameter. Typically, the time window length T2 is 2 or 3.

[0353] For example, when T2 = 2,

[0354] For example, when T3 = 3,

[0355] Optionally, the terminal device can convert the final probability of stuttering into stuttering or smooth output.

[0356] If p output (t r If the value is less than 0.5, then the final predicted label is... The target is to output smooth streaming video.

[0357] If p output (t r If the value is greater than or equal to 0.5, then the final predicted label is... This refers to the issue of choppy output of the target streaming video.

[0358] It should be noted that S201 to S205 above refer to how to predict the current time t. r This explains the final probability of buffering. It's understandable that as streaming video plays, the terminal device can adjust the buffering rate at time t. r The probability of subsequent stuttering is predicted in real time to determine whether stuttering occurs in the streaming video.

[0359] It should be understood that when a user watches streaming video online, the terminal device can obtain the QoE evaluation results in real time by inputting the underlying parameters into a pre-built QoE evaluation model. Furthermore, based on the real-time QoE evaluation results, the terminal device can determine whether the current network quality meets communication requirements, and thus determine whether to trigger network acceleration.

[0360] Figure 10 This is a schematic diagram illustrating the process of network acceleration based on real-time QoE evaluation results, provided as an embodiment of this application.

[0361] S301. The terminal device interacts with the server via the first network to exchange streaming video service data.

[0362] For example, the first network can be as follows: Figure 1 The Wi-Fi network or cellular network shown.

[0363] In practical implementation, after the terminal device receives a user's streaming video playback request, it can send a playback request message to the server in response to the request, requesting the streaming video resource. The server can then respond to this request message by sending streaming video service data packets to the terminal device. Accordingly, the terminal device can receive the streaming video service data packets, each data packet having a corresponding sequence number.

[0364] S302. During the playback of streaming video, the terminal device performs real-time QoE evaluation on the streaming video.

[0365] During the playback of streaming video, the terminal device can collect low-level parameters such as transport layer parameters, QoS parameters, and terminal device parameters in real time, and input these parameters into a pre-built QoE evaluation model to output the QoE evaluation result at the current moment.

[0366] The online evaluation process for streaming video can be referred to the description in the above embodiments, and will not be repeated here.

[0367] S303. The terminal device determines whether the playback requirements for streaming video are met based on the real-time QoE evaluation results. If the requirements are met, proceed to S304 below; otherwise, proceed to S305 below.

[0368] For example, suppose the final QoE evaluation result at the current moment is: the final predicted probability that the target streaming video will stutter at the current moment.

[0369] When the final predicted probability of the target streaming video experiencing stuttering at the current moment is less than the preset probability, that is, the QoE real-time evaluation result is smooth, the playback requirements of the streaming video are met, and there is no need to trigger network acceleration. Therefore, the terminal device can continue to interact with the server through the first network to exchange streaming video service data.

[0370] If the predicted probability of the target streaming video experiencing stuttering at the current moment is greater than or equal to the preset probability (i.e., the QoE real-time evaluation result indicates stuttering), then the playback requirements of the streaming video are not met, and network acceleration needs to be triggered. Therefore, the terminal device can interact with the server via a second network to exchange streaming video service data. The communication quality of the second network is superior to that of the first network.

[0371] S304. The terminal device continues to interact with the server via the first network to exchange streaming video service data.

[0372] S305. The terminal device interacts with the server via the second network to exchange streaming video service data.

[0373] This application provides several possible scenarios for implementing network acceleration, as exemplarily provided in the embodiments below.

[0374] Scenario 1: The first network is a single Wi-Fi network, and the second network is multiple Wi-Fi networks.

[0375] For example, when application service data is transmitted between a terminal device and a server via a single Wi-Fi network (e.g., a 2.4GHz Wi-Fi network or a 5GHz Wi-Fi network), if the Wi-Fi network communication quality does not meet the service communication requirements, then... Figure 11 As shown in (a), application service data can be transmitted between the terminal device and the server through multiple Wi-Fi networks (e.g., 2.4GHz Wi-Fi network + 5GHz Wi-Fi network) to accelerate the network.

[0376] Scenario 2: The first network can be a Wi-Fi network, and the second network can be a cellular network.

[0377] For example, when application service data is transmitted between a terminal device and a server via a Wi-Fi network (e.g., a 2.4GHz Wi-Fi network or a 5GHz Wi-Fi network), if the Wi-Fi network communication quality does not meet the service communication requirements, then... Figure 11 As shown in (b), the terminal device and the server can switch to transmitting application service data via cellular networks (such as 5G networks and / or 4G networks) to accelerate the network.

[0378] Scenario 3: The first network can be a Wi-Fi network, and the second network can include both Wi-Fi and cellular networks. Network acceleration is achieved through multi-network collaboration.

[0379] For example, when application service data is transmitted between a terminal device and a server via a Wi-Fi network, if the network communication quality does not meet the service communication requirements, then... Figure 11 As shown in (c), terminal devices and servers can collaboratively transmit application service data through Wi-Fi networks (e.g., 2.4GHz Wi-Fi networks and / or 5GHz Wi-Fi networks) and cellular networks (e.g., 5G networks and / or 4G networks) to achieve network acceleration.

[0380] Scenario 4: The first network can be a cellular network, and the second network can include Wi-Fi and mobile networks. Through the collaboration of multiple networks, network acceleration can be achieved.

[0381] Scenario 5: The first network can be a first Wi-Fi network, and the second network can be a second Wi-Fi network, with the second Wi-Fi network operating at a higher frequency than the first Wi-Fi network.

[0382] For example, the first network can be a 2.4 GHz Wi-Fi network, and the second network can be a 5 GHz Wi-Fi network.

[0383] Scenario 6: The first network can be a first cellular network, and the second network can be a second cellular network, with the network standard of the second cellular network being higher than that of the first cellular network.

[0384] For example, the first network can be a 4G cellular network and the second network can be a 5G cellular network.

[0385] Optionally, in this embodiment, Link Turbo technology can be used to achieve collaborative acceleration of Wi-Fi network and cellular network. When the network communication quality does not meet the service communication requirements, Link Turbo technology supports collaborative acceleration of four networks: 2.4GHz+5GHz WLAN, 5G mobile data, and 4G mobile data.

[0386] For example, when terminal devices and servers transmit application service data via Wi-Fi or cellular networks, if the network communication quality does not meet the service communication requirements, then... Figure 11 As shown in (d), the terminal device and the server can simultaneously accelerate the network through the 2.4GHz Wi-Fi network + 5GHz Wi-Fi network + primary SIM card 5G network + secondary SIM card 4G network.

[0387] In addition, when the terminal device is accelerating the network, it can also display information.

[0388] For example, such as Figure 12 As shown, during live streaming on the phone, the phone displays the live streaming interface 11 and automatically enables the QoE real-time evaluation function. When the QoE real-time evaluation result determines that the playback requirements of the streaming video are not met, the notification bar icon 12 updates, indicating that network acceleration is in progress. Icon 12 indicates that the phone is in a 5G network + 5GHz Wi-Fi state. In addition, the phone can also display the prompt message 13: "Using WLAN and mobile data simultaneously," reminding that network acceleration is in progress during the streaming video playback. Network acceleration can improve the network stability of streaming video.

[0389] The above primarily describes the solutions provided in the embodiments of this application from the perspective of the terminal device. It is understood that, in order to achieve the above functions, the terminal device includes hardware structures or software modules, or a combination of both, corresponding to each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0390] This application embodiment can divide the terminal device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The following description uses the example of dividing each functional module according to each function.

[0391] Figure 13This is a schematic diagram of the structure of a training device for the QoE evaluation model provided in an embodiment of this application. This training device is used to train a QoE evaluation model for real-time evaluation. Figure 13 As shown, the training device 130 may include an acquisition module 131, a feature extraction module 132, and a training module 133.

[0392] The acquisition module 131 can be used to acquire offline data, which is terminal-side parameters obtained from the log file of the streaming video that has finished playing.

[0393] The feature extraction module 132 can be used to extract target offline features from the offline data. These target offline features are used to evaluate the QoE at time t, where time t is the moment when the streaming video is played.

[0394] Training module 133 can be used to train a machine learning classifier for the QoE evaluation model based on the offline features of the target, and obtain the mapping relationship between the offline features of the target and the preliminary QoE evaluation results at time t.

[0395] The training module 133 can also be used to train the score memory unit of the QoE evaluation model based on the preliminary QoE evaluation results, and obtain the mapping relationship between multiple preliminary QoE evaluation results and the final QoE evaluation result at time t. These multiple preliminary QoE evaluation results include: the preliminary QoE evaluation result at time t, and the preliminary QoE evaluation results at least one time before time t.

[0396] It should be understood that the training device, by pre-training and building a QoE evaluation model, facilitates real-time QoE evaluation of streaming video being played.

[0397] The training apparatus in this application embodiment can correspond to the method described in this application embodiment related to the offline training process, and for the sake of brevity, it will not be described again here.

[0398] Figure 14 This is a schematic diagram of the QoE evaluation device provided in an embodiment of this application. Figure 14 As shown, the device 140 may include an acquisition module 141, a feature extraction module 142, and an evaluation module 143.

[0399] The acquisition module 141 is used to acquire online data, which is the terminal-side parameters collected in real time from the initial playback time to time t during the playback of the target streaming video.

[0400] Feature extraction module 142 can be used to extract target online features from online data. These target online features are used to evaluate the QoE at the current time t, where time t is the time when the target streaming video is played.

[0401] Evaluation module 143 can be used to input the online features of the target into the machine learning classifier of the QoE evaluation model to obtain the preliminary QoE evaluation result at the current time t;

[0402] The evaluation module 143 can also be used to input multiple preliminary QoE evaluation results into the score memory unit of the QoE evaluation model to obtain the final QoE evaluation result at the current time t; wherein, the multiple preliminary QoE evaluation results include: the preliminary QoE evaluation result at the current time t, and the preliminary QoE evaluation result at least one time before the current time t.

[0403] It should be understood that when users watch streaming videos online, QoE evaluation results can be obtained in real time by inputting online data into a pre-built QoE evaluation model.

[0404] The QoE evaluation apparatus of this application embodiment can correspond to the method described in this application embodiment related to the online evaluation process, and for the sake of brevity, it will not be described again here.

[0405] Figure 15 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 15 As shown, the terminal device may include a processor 401, which is coupled to a memory 403. The processor 401 is used to execute computer programs or instructions stored in the memory so that the terminal device implements the methods in the above embodiments.

[0406] The terminal device may also include a communication bus 402, a communication interface 404, an output device 405, and an input device 406.

[0407] The number of processors 401 can be one or more. A processor 401 may include at least one processing unit. For example, a processor may include, Figure 15 The diagram shows at least one central processing unit (CPU). For example, the processor may also include an image signal processor (ISP), a digital signal processor (DSP), a video codec, a neural network processing unit (NPU), a graphics processing unit (GPU), an application processor (AP), a modem processor, and / or a baseband processor. In some embodiments, different processing units may be independent devices or integrated into one or more processors.

[0408] The communication bus 402 may include a path for transmitting information between the processor 401, the memory 403, and the communication interface 404.

[0409] Communication interface 404, using any transceiver-like device, is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), or wireless local area network (WLAN). In this embodiment, communication interface 404 is mainly used for communication with a server, such as transmitting data packets for streaming video.

[0410] Memory 403 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, and Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processor via a bus. Memory may also be integrated with the processor.

[0411] The memory 403 stores execution application code, such as the code for a video application, and its execution is controlled by the processor 401. The processor 401 executes the application code stored in the memory 403 to implement the training method and QoE evaluation method of the QoE evaluation model in the above embodiments. In this application embodiment, the memory can also be used for log files, machine learning models, and score memory units, etc.

[0412] Output device 405 communicates with processor 401 and can display information in various ways, such as displaying a streaming video playback interface. Output device 405 may include a display panel, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), or a quantum dot light-emitting diode (QLED), etc.

[0413] Input device 406 communicates with processor 401 and can receive user input in various ways, such as receiving user input for streaming video playback. Input device 406 can be a mouse, keyboard, touchscreen, or sensing device, etc.

[0414] It should be understood that Figure 15 The terminal device shown can correspond to Figure 13 The training device shown. Among them, Figure 15 The processor 401 in the terminal device shown can correspond to Figure 13 The training device includes the acquisition module 131, the feature extraction module 132, and the training module 133. Figure 15 The terminal equipment shown can also correspond to Figure 14 The QoE evaluation device shown is included. Figure 15 The processor 401 in the terminal device shown can correspond to Figure 14 The QoE evaluation device includes an acquisition module 141, a feature extraction module 142, and an evaluation module 143.

[0415] This application also provides a computer-readable storage medium storing computer instructions. When the computer-readable storage medium is run on a terminal device, it causes the terminal device to perform the method described above. The computer instructions can be stored in the computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access, or it can include one or more data storage devices such as servers or data centers that can be integrated using media. Available media can be magnetic media (e.g., floppy disks, hard disks, or magnetic tapes), optical media, or semiconductor media (e.g., solid-state drives (SSDs)).

[0416] This application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0417] This application also provides a chip coupled to a memory, which is used to read and execute computer programs or instructions stored in the memory to perform the methods described in the above embodiments. This chip can be a general-purpose processor or a dedicated processor.

[0418] It should be noted that the chip can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits capable of performing the various functions described throughout this application.

[0419] The terminal device, training device, QoE evaluation device, computer-readable storage medium, computer program product, and chip provided in the above embodiments of this application are all used to execute the methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects corresponding to the methods provided above, and will not be repeated here.

[0420] It should be understood that the above description is merely to help those skilled in the art better understand the embodiments of this application, and is not intended to limit the scope of the embodiments of this application. Based on the examples given above, those skilled in the art can obviously make various equivalent modifications or changes. For example, some steps in the various embodiments of the above detection method may be unnecessary, or new steps may be added. Alternatively, any combination of two or more of the above embodiments may be used. Such modifications, changes, or combinations also fall within the scope of the embodiments of this application.

[0421] It should also be understood that the above description of the embodiments of this application focuses on highlighting the differences between the various embodiments. Any similarities or differences not mentioned can be referred to each other. For the sake of brevity, they will not be repeated here.

[0422] It should also be understood that the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0423] It should also be understood that in the embodiments of this application, "pre-setting" or "pre-defining" can be achieved by pre-saving the corresponding code, table or other means that can be used to indicate relevant information in the device (e.g., including terminal device), and this application does not limit the specific implementation method.

[0424] It should also be understood that the methods, situations, categories, and classifications of embodiments in this application are for the convenience of description only and should not constitute a special limitation. Various methods, categories, situations, and features in embodiments can be combined without contradiction.

[0425] It should also be understood that, in the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced by each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0426] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for an Experience Quality of Experience (QoE) assessment model, characterized in that, The QoE evaluation model includes a machine learning classifier and a score memory unit, and the method includes: Target offline features are extracted from offline data, which are terminal-side parameters from the initial playback time to time t obtained from the log file of a streaming video that has finished playing. The target offline features are used to evaluate the QoE at time t. Based on the target offline features, the machine learning classifier is trained to obtain the mapping relationship between the target offline features and the preliminary QoE evaluation result at time t; Based on multiple preliminary QoE evaluation results, the score memory unit is trained to obtain the mapping relationship between the multiple preliminary QoE evaluation results and the final QoE evaluation result at time t; wherein, the multiple preliminary QoE evaluation results include: the preliminary QoE evaluation result at time t, and the preliminary QoE evaluation result at least one time before time t.

2. The method of claim 1, wherein, The extraction of target offline features from offline data includes: The offline data is normalized. Extract all offline features from the normalized offline data; Redundant offline features are removed from all the offline features to obtain the target offline features.

3. The method of claim 2, wherein, The complete offline features include at least one of the following: Global features, which are features extracted from all data in the offline data; Window features, which are features extracted from a portion of the offline data, including data at time t and data for a preset duration prior to time t; Other features, which are features unrelated to network data.

4. The method of claim 2, wherein, The step of removing redundant offline features from all offline features to obtain the target offline features includes: Iterative training is performed on all the offline features: In each round of iterative training, based on the importance level of each offline feature among all the offline features, the offline feature with the lowest importance level is deleted until a preset number of iterations is reached, or until the number of remaining offline features is less than or equal to a preset number. The remaining offline features are used as the target offline features.

5. The method according to any one of claims 1 to 4, characterized in that, The multiple preliminary QoE evaluation results include: multiple preliminary prediction probabilities, each of which represents the probability that the machine learning classifier predicts the streaming video to be choppy at a given moment, where a given moment is moment t or at least one moment prior to moment t; the optimization objective of the machine learning classifier is to minimize a first loss function, which represents the loss corresponding to the degree of difference between the preliminary prediction probability at a given moment and the initial QoE label at that moment; the initial QoE label at a given moment indicates whether the streaming video is choppy or smooth at that moment. The final QoE evaluation result at time t includes: the final prediction probability at time t, which represents the probability that the scoring memory unit predicts the streaming video will be stuttering at time t; the optimization objective of the scoring memory unit is to minimize the second loss function, which represents the loss corresponding to the degree of difference between the final prediction probability at time t and the initial QoE label at time t.

6. The method of claim 5, wherein, The mapping relationship between the multiple preliminary QoE evaluation results and the final QoE evaluation result at time t is expressed by the following formula: in, Represents the time t, This represents the weighting factor for each of the initial predicted probabilities, where T represents the time window length. Represents the final predicted probability at time t; when hour, The initial predicted probability represents the time t. when hour, Represents the time before the stated time t. The initial predicted probability.

7. The method according to claim 5, characterized in that, The final QoE evaluation result at time t also includes: the final predicted label at time t; If the final prediction probability at time t is greater than or equal to 0.5, then the final prediction label at time t is used to indicate that the streaming video is stuttering at time t. If the final prediction probability at time t is less than 0.5, then the final prediction label at time t is used to indicate that the streaming video is smooth at time t.

8. The method according to claim 5, characterized in that, The initial QoE tags are either manually labeled or automatically generated based on the log files of the streaming video.

9. The method according to any one of claims 1 to 4, characterized in that, The terminal-side parameters include at least one of the following: Transport layer parameters, which are used to reflect the network transmission status when playing the streaming video; Quality of Service (QoS) parameters, which are used to evaluate the network's ability to provide services for the streaming video; Terminal parameters, which are the parameters of the terminal device itself when playing the streaming video.

10. A QoE evaluation method, characterized in that, The method includes: The target online features are extracted from the online data, which are terminal-side parameters collected in real time from the initial playback time to the current time during the playback of the target streaming video. The target online features are used to evaluate the QoE at the current time in real time. The target online features are input into the machine learning classifier of the QoE evaluation model to obtain the preliminary QoE evaluation result at the current time. Multiple preliminary QoE assessment results are input into the score memory unit of the QoE assessment model to obtain the final QoE assessment result at the current time; wherein, the multiple preliminary QoE assessment results include: the preliminary QoE assessment result at the current time, and the preliminary QoE assessment result at least one time before the current time.

11. The method according to claim 10, characterized in that, The extraction of target online features from online data includes: The online data is normalized. Extract the target online features from the normalized online data.

12. The method according to claim 10, characterized in that, The target online characteristics include at least one of the following: Global features, which are features extracted from all data in the online data; Window features, which are features extracted from a portion of the online data, including data at the current moment and data for a preset duration prior to the current moment; Other features, which are features unrelated to network data.

13. The method according to claim 12, characterized in that, The target online features include global features; The global features are extracted based on the following parameters: Online data collected at the current moment; Global features used to evaluate the QoE of the previous time step at the current time step; Intermediate variables corresponding to the current time and the previous time.

14. The method according to any one of claims 10 to 13, characterized in that, The multiple QoE preliminary evaluation results include: multiple preliminary prediction probabilities, each of the multiple preliminary prediction probabilities being used to represent the probability that the machine learning classifier predicts the target streaming video is stuttering at a certain moment, wherein the certain moment is the current moment or at least one moment before the current moment; The final QoE evaluation result at the current moment includes: the final prediction probability at the current moment, which represents the probability that the scoring memory unit predicts that the target streaming video is stuttering at the current moment; The step of inputting multiple preliminary QoE evaluation results into the score memory unit of the QoE evaluation model to obtain the final QoE evaluation result at the current time includes: The score memory unit performs a weighted summation of the multiple preliminary prediction probabilities according to the weight factor of each preliminary prediction probability to obtain the final prediction probability at the current moment.

15. The method according to claim 14, characterized in that, The final QoE evaluation result at the current moment also includes: the final predicted label at the current moment; The step of inputting multiple preliminary QoE evaluation results into the score memory unit of the QoE evaluation model to obtain the final QoE evaluation result at the current time also includes: Based on the final prediction probability at the current moment, determine the final prediction label at the current moment; Wherein, if the final prediction probability at the current moment is greater than or equal to 0.5, the final prediction label at the current moment is used to indicate that the target streaming video is stuttering at the current moment; if the final prediction probability at the current moment is less than 0.5, the final prediction label at the current moment is used to indicate that the target streaming video is smooth at the current moment.

16. The method according to any one of claims 10 to 13, characterized in that, The terminal-side parameters include at least one of the following: Transport layer parameters, which are used to reflect the network transmission status when playing the target streaming video; Quality of Service (QoS) parameters, which are used to evaluate the network's ability to provide services for the target streaming video; Terminal parameters, which are the parameters of the terminal device itself when playing the target streaming video.

17. The method according to any one of claims 10 to 13, characterized in that, The data of the target streaming video is obtained at the current moment through interaction between the first network and the server; After obtaining the final QoE evaluation result at the current moment, the method further includes: If the final QoE evaluation result at the current moment indicates that the communication quality of the first network does not meet the communication requirements, the target streaming video data is exchanged with the server through the second network. The communication quality of the second network is better than that of the first network.

18. The method according to claim 17, characterized in that, The final QoE evaluation result at the current moment includes: the final predicted probability that the target streaming video will stutter at the current moment; The final QoE evaluation result at the current moment indicates that the communication quality of the first network does not meet the communication requirements, including: if the final predicted probability of the target streaming video experiencing stuttering at the current moment is greater than or equal to a preset probability, then the final QoE evaluation result at the current moment indicates that the communication quality of the first network does not meet the communication requirements.

19. A terminal device, characterized in that, The device includes a processor coupled to a memory, the processor being configured to execute a computer program or instructions stored in the memory, such that the terminal device implements the training method of the QoE evaluation model as described in any one of claims 1 to 9, or implements the QoE evaluation method as described in any one of claims 10 to 18.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a terminal device, causes the terminal device to perform a training method for the QoE evaluation model as described in any one of claims 1 to 9, or to perform a QoE evaluation method as described in any one of claims 10 to 18.

Images