Decoding capability determination method, apparatus and device, and storage medium and product

Abstract

Provided in the embodiments of the present application are a decoding capability determination method, apparatus and device, and a storage medium and a product. The technical solution provided in the embodiments of the present application involves performing analysis processing on device basic data, decoding capability data, real-time performance data and device static data by means of a decoding time prediction model to obtain a prediction result for decoding time per frame, and determining a target decoding time per frame on the basis of the prediction result for the decoding time per frame, wherein the decoding time prediction model is obtained by performing training on the basis of device basic sample data, decoding capability sample data, real-time performance sample data and device static sample data of the decoding capability of a sample device in different changepoint intervals in a time series and sample data of decoding time per frame corresponding to the changepoint intervals. By means of the decoding time prediction model, a corresponding prediction result for decoding time per frame when the decoding capability of a device changes can be accurately predicted, and the target decoding time per frame can be accurately determined, thereby effectively improving a prediction effect on the decoding capability of the device.

Description

A method, device, equipment, storage medium and product for determining decoding capability

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 5, 2024, with application number 202410250344.1, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of computer technology, and in particular to a method, apparatus, device, storage medium, and product for determining decoding capability. Background Art

[0003] With the rapid development of audio and video technologies, video quality and bitrate in video-on-demand services have significantly improved, placing correspondingly higher demands on device decoding capabilities. With the advancement of computer vision technology, real-time rendering techniques such as beautification and filters have become an essential component of video-on-demand services. Different video processing methods also place varying degrees of load on device computing power. Since video-on-demand users may originate from countries or regions with varying levels of development, and their network environments and device performance vary significantly, accurately estimating the highest video quality that a user's device can decode in real time is crucial for improving the user experience.

[0004] Current decoding capability estimation algorithms generally treat decoding capability as a static property of the device. In actual business scenarios, device performance may vary significantly over time. For example, if the device system overheats and triggers processor frequency reduction, the predictive effect of historical data on the device's current decoding capability is greatly reduced, resulting in poor prediction of the device's decoding capability. Summary of the Invention

[0005] The embodiments of the present application provide a decoding capability determination method, apparatus, device, storage medium, and product to address the technical problem in related technologies that decoding capability is regarded as a static attribute of a device, resulting in poor prediction of the device's decoding capability, and can effectively improve the prediction of the device's decoding capability.

[0006] In a first aspect, an embodiment of the present application provides a method for determining decoding capability, comprising:

[0007] Obtain device basic data, decoding capability data, real-time performance data, and device static data;

[0008] Inputting the device basic data, the decoding capability data, the real-time performance data, and the device static data into a trained decoding time prediction model, analyzing and processing the device basic data, the decoding capability data, the real-time performance data, and the device static data through the decoding time prediction model to obtain a per-frame decoding time prediction result, wherein the decoding time prediction model is trained based on the device basic sample data, decoding capability sample data, real-time performance sample data, and device static sample data of the sample devices at different change point intervals in the decoding capability in a time series, as well as the per-frame decoding time sample data corresponding to the change point intervals;

[0009] The target per-frame decoding time is determined according to the per-frame decoding time prediction result.

[0010] In a second aspect, an embodiment of the present application provides a decoding capability determination device, including a data acquisition module, a time prediction module, and a time determination module, wherein:

[0011] The data acquisition module is configured to acquire device basic data, decoding capability data, real-time performance data and device static data;

[0012] The decoding time prediction module is configured to input the basic device data, the decoding capability data, the real-time performance data, and the static device data into a trained decoding time prediction model, and analyze and process the basic device data, the decoding capability data, the real-time performance data, and the static device data through the decoding time prediction model to obtain a prediction result of the decoding time for each frame, wherein the decoding time prediction model is trained based on the basic device sample data, the decoding capability sample data, the real-time performance sample data, and the static device sample data of the decoding capability of the sample device at different change point intervals in the time series, as well as the decoding time sample data for each frame corresponding to the change point intervals;

[0013] The time determination module is configured to determine a target per-frame decoding time according to the per-frame decoding time prediction result.

[0014] In a third aspect, an embodiment of the present application provides a decoding capability determination device, including: a memory and one or more processors;

[0015] The memory is used to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the decoding capability determination method as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer-executable instructions, which, when executed by a computer processor, are used to perform the decoding capability determination method as described in the first aspect.

[0018] In the fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the decoding capability determination method as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG1 is a flowchart of a method for determining decoding capability provided by an embodiment of the present application;

[0020] FIG2 is a flowchart of another method for determining decoding capability provided by an embodiment of the present application;

[0021] FIG3 is a schematic diagram of a decoding time prediction model training process provided by an embodiment of the present application;

[0022] FIG4 is a schematic structural diagram of a decoding capability determination device provided in an embodiment of the present application;

[0023] FIG5 is a schematic structural diagram of a decoding capability determination device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only some, but not all, of the contents related to the present application are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0025] The decoding capability determination method provided in this application can be applied to determining decoding capabilities in video application scenarios. For example, in short video application scenarios, the decoding capability of a device is determined, and the media backend determines the video level to be sent to the device based on the device's decoding capability. The method aims to analyze and process basic device data, decoding capability data, real-time performance data, and static device data through a decoding time prediction model to obtain a prediction result for each frame's decoding time. The method then determines the target decoding time per frame based on the prediction result, and updates the target decoding time per frame in real time based on changes in the device's decoding capability.

[0026] In existing decoding capability determination schemes, decoding capability is generally regarded as a static attribute of the device, for example, the corresponding decoding capability is determined according to the device model or the decoding capability is determined according to the historically selected video gear. However, in actual business scenarios, device performance may change significantly over time. For example, after the device system triggers processor frequency reduction due to overheating, the predictive effect of static historical data on the current decoding capability of the device is greatly reduced, and the prediction effect on the device decoding capability is poor. Based on this, a decoding capability determination method of an embodiment of the present application is provided to solve the technical problem that the existing decoding capability determination scheme regards decoding capability as a static attribute of the device and has a poor prediction effect on the device decoding capability.

[0027] Figure 1 shows a flowchart of a decoding capability determination method provided in an embodiment of the present application. The decoding capability determination method provided in an embodiment of the present application can be executed by a decoding capability determination device, which can be implemented by hardware and / or software and integrated into a decoding capability determination device.

[0028] The following description is based on an example of a method for determining decoding capability performed by a decoding capability determination device. Referring to FIG1 , the method for determining decoding capability includes:

[0029] S110: Obtain device basic data, decoding capability data, real-time performance data, and device static data.

[0030] Exemplarily, the basic device data, decoding capability data, real-time performance data and device static data corresponding to the decoding capability determination device are obtained in real time. For example, when the decoding capability determination device uses a short video application, after each short video is played, the basic device data, decoding capability data, real-time performance data and device static data corresponding to the short video are collected, that is, each time a short video is played, a set of basic device data, decoding capability data, real-time performance data and device static data is corresponding. Optionally, after the player in the decoding capability determination device completes the playback of a short video, the corresponding basic device data, decoding capability data, real-time performance data and device static data can be uploaded to a set database (such as a Hive database) for unified storage.

[0031] In one embodiment, the device basic data provided by this solution may include one or more combinations of device identification information (device_id), the timestamp of the collected data (rtime), and the number of video frames of the played video (num_frames). The decoding capability data may be the average decoding latency per frame corresponding to the video playback (average_decoding_latency, i.e., the average time taken to decode each video frame in each short video). The real-time performance data may be a combination of one or more of the current load information of the decoding capability determination device, the current number of threads of the device (num_threads), memory usage information (ram_usage), the current power of the device (battery), the current heating status of the device (thermal_status), the duration of the current heating status of the device (thermal_status_time), and the cooling down time (cooldown_time) triggered by the device. The static data of the device may be a combination of one or more of the device model (phone_model), the maximum frequency of the processor (cpu_freq), and the physical memory size of the device (ram_size).

[0032] S120: Input the basic data of the device, the decoding capability data, the real-time performance data and the static data of the device into the trained decoding time prediction model, and analyze and process the basic data of the device, the decoding capability data, the real-time performance data and the static data of the device through the decoding time prediction model to obtain the decoding time prediction result of each frame.

[0033] The decoding time prediction model provided by this solution is trained based on basic device sample data, decoding capability sample data, real-time performance sample data, and static device sample data at different change point intervals in the decoding capability of sample devices in a time series, as well as per-frame decoding time sample data corresponding to the change point intervals. This allows the decoding time prediction model to determine a per-frame decoding time prediction result for a corresponding device based on the input device basic data, decoding capability data, real-time performance data, and static device data, and the per-frame decoding time prediction result can reflect whether the device has experienced a change in decoding capability. Optionally, the per-frame decoding time sample data can be determined based on a weighted average of the average per-frame decoding time of samples in the corresponding change point interval.

[0034] Among them, after completing the training of the decoding time prediction model, the decoding time prediction model can be configured in the decoding capability determination device (for example, configured on the short video application installed on the decoding capability determination device), or configured in the cloud. The decoding capability determination device can request the cloud to call the decoding time prediction model based on the device basic data, decoding capability data, real-time performance data and device static data.

[0035] Exemplarily, the basic device data, decoding capability data, real-time performance data, and static device data obtained above are input into a decoding time prediction model. The decoding time prediction model analyzes and processes the basic device data, decoding capability data, real-time performance data, and static device data to obtain a per-frame decoding time prediction result. The per-frame decoding time prediction result can be understood as a prediction of the time required for the decoding capability determination device to decode a single image frame.

[0036] S130: Determine a target per-frame decoding time according to the per-frame decoding time prediction result.

[0037] For example, after determining the per-frame decoding time prediction result, a target per-frame decoding time reflecting the decoding capability assessment result of the decoding capability determination device can be determined based on the per-frame decoding time prediction result. For example, the per-frame decoding time prediction result can be determined as the latest target per-frame decoding time, or a weighted average (e.g., an exponentially weighted moving average, EWMA) of the current per-frame decoding time prediction result and the previously determined target per-frame decoding time can be used as the latest target per-frame decoding time.

[0038] In a possible embodiment, the decoding capability determination method provided by this solution may further include, after determining the target decoding time per frame based on the prediction result of the decoding time per frame, sending the target decoding time per frame to the media background, for determining the video downlink gear based on the target decoding time per frame through the media background.

[0039] Exemplarily, after determining the target per-frame decoding time, the target per-frame decoding time is sent to the media backend. After receiving the target per-frame decoding time from the decoding capability determination device, the media backend can determine the video delivery level based on the target per-frame decoding time and deliver the video corresponding to the delivery level to the decoding capability determination device, effectively balancing video clarity and rendering smoothness. In one embodiment, a video level with a frame rate that does not exceed the reciprocal of the target per-frame decoding time can be used as the video delivery level.

[0040] For example, when a user's phone cools down and triggers processor frequency reduction, the estimated decoding capability of the user's phone decreases accordingly (the predicted frame decoding time decreases). The player will then send the target frame decoding time determined based on the predicted frame decoding time to the media backend, which will continue to deliver lower bitrate videos based on the target frame decoding time until the user's phone recovers its decoding capability. This solution determines the video delivery level based on the target frame decoding time and promptly updates the video delivery level when the video decoding capability changes, effectively balancing video clarity and rendering smoothness.

[0041] As described above, the decoding time prediction model is used to analyze and process the device basic data, decoding capability data, real-time performance data and device static data to obtain the decoding time prediction results for each frame, and the target decoding time for each frame is determined based on the decoding time prediction results for each frame. The decoding time prediction model is based on the device basic sample data, decoding capability sample data, real-time performance sample data and device static sample data of the decoding capability of the sample device in different change point intervals in the time series, as well as the decoding time sample data for each frame corresponding to the change point interval. The decoding time prediction model can accurately predict the corresponding decoding time prediction results for each frame when the decoding capability of the device changes, accurately determine the target decoding time for each frame, and effectively improve the prediction effect of the decoding capability of the device.

[0042] Based on the above embodiment, FIG2 shows a flowchart of another decoding capability determination method provided by an embodiment of the present application, which is a specific embodiment of the above decoding capability determination method. Referring to FIG2, the decoding capability determination method includes:

[0043] S210: Obtain device basic data, decoding capability data, real-time performance data, and device static data.

[0044] S220: Input the basic data of the device, the decoding capability data, the real-time performance data and the static data of the device into the trained decoding time prediction model, and analyze and process the basic data of the device, the decoding capability data, the real-time performance data and the static data of the device through the decoding time prediction model to obtain the decoding time prediction result of each frame.

[0045] The decoding time prediction model is trained based on basic device sample data, decoding capability sample data, real-time performance sample data, and static device sample data for different time-series change points of the decoding capabilities of sample devices, as well as per-frame decoding time sample data corresponding to the change points. Optionally, the decoding time prediction model provided in this solution can be a random forest model.

[0046] In one possible embodiment, as shown in FIG3 , a decoding time prediction model training process diagram, the decoding time prediction model training process provided by this solution includes:

[0047] S201: Obtain device basic sample data, decoding capability sample data, real-time performance sample data, and device static sample data of decoding capabilities of multiple sample devices.

[0048] S202: Determine different change point intervals of the decoding capability of each sample device in a time series and sample data of per-frame decoding time corresponding to each change point interval based on the decoding capability sample data and the device basic sample data.

[0049] S203: Train a decoding time prediction model using device basic sample data, decoding capability sample data, real-time performance sample data, and device static sample data of different change point intervals of the decoding capability of each sample device in a time series, as well as per-frame decoding time sample data corresponding to the change point intervals.

[0050] Exemplarily, basic device sample data, decoding capability sample data, real-time performance sample data, and device static sample data corresponding to multiple sample devices each time a short video is played are collected as sample data for training a decoding time prediction model.

[0051] Corresponding to the device basic data, decoding capability data, real-time performance data and device static data, the device basic sample data provided by this solution may include one or more combinations of sample device identification information, sample timestamps of collected sample data and sample video frame numbers of the sample video played. The decoding capability sample data may be the average decoding time per frame of the corresponding sample when playing the sample video. The real-time performance sample data may be one or more combinations of the current sample load information of the sample device, the current number of sample threads of the sample device, sample memory usage information, the current power of the sample device, the current heating state of the sample device, the duration of the current heating state of the sample device and the cooling and frequency reduction time triggered by the sample device. The device static sample data may be one or more combinations of the sample device model, the maximum frequency of the sample processor and the physical memory size of the sample device.

[0052] In one embodiment, the sample data can be grouped according to the device identification information corresponding to each group and sorted according to the timestamp information. Based on the decoding capability sample data and device basic sample data obtained above, the different change point intervals of the decoding capability of each sample device in the time series and the sample data of each frame decoding time corresponding to each change point interval are determined.

[0053] The change point of the decoding capability of the sample device in the time series can be understood as the time point at which the change in decoding capability reaches a set threshold. After determining multiple change points of the decoding capability of the sample device in the time series, the sample data can be divided into multiple continuous change point intervals in the time series based on the time starting point, time ending point, and each change point of the corresponding sample data. Each change point interval corresponds to sample data in the corresponding time interval. The sample data of each change point interval includes the average per-frame decoding time of the corresponding time interval. The per-frame decoding time sample data of the corresponding time interval can be determined based on the weighted average of the average per-frame decoding time of the samples in each time interval.

[0054] In one embodiment, sample data is grouped according to different change-point intervals in a time series. A decoding time prediction model is trained using the basic device sample data, decoding capability sample data, real-time performance sample data, and static device sample data of each sample device's decoding capability at different change-point intervals in the time series, as well as the per-frame decoding time sample data corresponding to the change-point intervals. For example, the per-frame decoding time sample data corresponding to the change-point interval is used as output, and the decoding time prediction model is trained using the basic device sample data, decoding capability sample data, real-time performance sample data, and static device sample data of the corresponding change-point intervals as input. This solution trains the decoding time prediction model using the basic device sample data, decoding capability sample data, real-time performance sample data, and static device sample data of the decoding capabilities of multiple sample devices, as well as the per-frame decoding time sample data of each sample device at different change-point intervals in the time series. The decoding time prediction model can accurately predict the per-frame decoding time prediction results corresponding to changes in the device's decoding capability, accurately determining the target per-frame decoding time.

[0055] In one embodiment, this scheme determines the different change point intervals of the decoding capability of each sample device in the time series, and the decoding time sample data of each frame corresponding to each change point interval based on the decoding capability sample data and the device basic sample data. It can be configured as follows: based on the set window length information, and the difference function of the loss function centered on the window, at the center of the window and at both ends, the different change point intervals of the decoding capability of the sample device in the time series are determined, and the loss function is determined based on the average decoding time sample data in the decoding capability sample data and the number of sample video frames in the device basic sample data.

[0056] Exemplarily, for the sample data corresponding to each sample device, based on the set window length information, the difference function of the loss function centered on the window, at the center of the window and at both ends (left and right), is determined, and the window is slid on the time series according to the difference function determined above to determine the different change point intervals of the decoding capability of the sample device in the time series. Among them, the length corresponding to the set window length information can be set to half the window length (i.e., the length of half the window). For example, the decoding capability of the device can be considered to be half of the minimum time window in the steady state, such as determining half of the minimum time window as the window length information. Optionally, the difference function can be the difference between the loss function at the center of the window and the loss function at the left end of the window and the loss function at the right end of the window. This solution improves the decoding time prediction effect of the decoding time prediction model by determining the different change point intervals of the decoding capability of the sample device in the time series based on the set window length information.

[0057] In one embodiment, different change point intervals of the decoding capability of the sample device in the time series are determined based on the loss function and a set change point determination algorithm (such as a sliding window algorithm (WIN algorithm)). For example, the sample data is grouped according to the sample device identifier and arranged according to the sample timestamp. For each sample data group, it is assumed that the average decoding time per frame of each sample is r1, r2, ..., r n , the sample video frames are f1, f2, ..., f n , assuming the loss function c(i,j):

[0058] Among them, f k is the number of sample video frames in the sample data group, r k is the average decoding time per frame of the kth sample in the sample data group, For r i ,…,r j The weighted average of the sample, that is, the average decoding time per frame r i to r j The weighted average of .

[0059] For the set window length information w, define the difference function:

[0060] d w (k)=c(kw,k+w)-c(kw,k)-c(k,k+w),w <k<n-w

[0061] c(kw,k+w) is the loss function at the center of the window, c(kw,k) is the loss function at the left end of the window, c(k,k+w) is the loss function at the right end of the window, and d w (k) is the difference function of the loss function at the center and both ends of the window centered on the window.

[0062] Using the sliding window-based change point determination algorithm, the average decoding time per frame of the input sample is r1, r2, ..., r n , the sample video frames are f1, f2, ..., f n , loss function c(.), window length information (half window length) k, use the set peak search algorithm, initialize the array Z←(0,0,…,0), search for one or more peaks corresponding to the average decoding time per frame of multiple samples in each sample data, and determine the time point corresponding to the peak as the change point.

[0063] The sample data is grouped according to the change point, and the weighted average of the average per-frame decoding time of the samples in the change point interval is used to replace the per-frame decoding time of a single video in each group:

[0064] Among them, i, j are adjacent change points. Using sample data as input features, r l As the target value, the decoding time prediction model is trained.

[0065] S230: Determine a time determination strategy according to the prediction result of the decoding time per frame, and determine a target decoding time per frame based on the time determination strategy.

[0066] Exemplarily, after determining the prediction result of the per-frame decoding time, a timing determination strategy is determined based on the prediction result of the per-frame decoding time. For example, when the prediction result of the per-frame decoding time is the first prediction result or differs significantly from the previously determined target per-frame decoding time, the corresponding timing determination strategy may be to replace the target per-frame decoding time with the prediction result of the per-frame decoding time. When the prediction result of the per-frame decoding time differs slightly from the previously determined target per-frame decoding time, the corresponding timing determination strategy may be to weight the sum of the prediction result of the per-frame decoding time and the previously determined target per-frame decoding time.

[0067] In one possible embodiment, the decoding capability determination method provided by the present solution determines the target decoding time per frame based on the prediction result of the decoding time per frame, and can be configured as follows: when the prediction result of the decoding time per frame is the first prediction result, or the difference between the prediction result of the decoding time per frame and the previous prediction result of the decoding time per frame exceeds a set difference threshold, the prediction result of the decoding time per frame is determined as the target decoding time per frame.

[0068] Exemplarily, after determining the prediction result of the decoding time per frame, determine whether the current prediction result of the decoding time per frame is the first prediction result (for example, determine whether there is a cached prediction result of the decoding time per frame. If there is a cached prediction result of the decoding time per frame, it is not the first prediction result; if there is no cached prediction result of the decoding time per frame, it is the first prediction result). If the current prediction result of the decoding time per frame is the first prediction result, the currently determined prediction result of the decoding time per frame is determined as the target decoding time per frame.

[0069] If the current per-frame decoding time prediction result is not the first prediction result, the difference between the per-frame decoding time prediction result and the previous per-frame decoding time prediction result is determined, and when the difference between the per-frame decoding time prediction result and the previous per-frame decoding time prediction result exceeds the set difference threshold, the currently determined per-frame decoding time prediction result is determined as the target per-frame decoding time.

[0070] This solution determines the predicted result of per-frame decoding time as the target per-frame decoding time when the predicted result of per-frame decoding time is the first predicted result, or the difference with the previous predicted result of per-frame decoding time exceeds the set difference threshold. This allows timely detection of changes in the decoding capability of the device and accurate determination of the target per-frame decoding time that reflects the real-time decoding capability of the device.

[0071] In one possible embodiment, the decoding capability determination method provided by the present solution determines the target decoding time per frame based on the prediction result of the decoding time per frame, and can be configured as follows: when the difference between the prediction result of the decoding time per frame and the previous prediction result of the decoding time per frame is within a set difference threshold, the weighted average result of the prediction result of the decoding time per frame and the previous target decoding time per frame is determined as the target decoding time per frame.

[0072] Exemplarily, when the difference between the prediction result of the per-frame decoding time and the previous prediction result of the per-frame decoding time is within a set difference threshold, a weighted average result of the prediction result of the per-frame decoding time and the previous target per-frame decoding time is determined, and the weighted average result is determined as the latest target per-frame decoding time. This solution uses the weighted average result of the current per-frame decoding time prediction result and the previous target per-frame decoding time as the target per-frame decoding time when the difference between the frame decoding time prediction result and the previous per-frame decoding time prediction result is within a set difference threshold, and smoothly transitions the target per-frame decoding time when the decoding capability of the device changes slightly, thereby achieving a smooth transition of the decoding capability prediction, reducing the situation where the user experience is reduced due to excessive changes in the video gear sent to the user, and ensuring the user experience.

[0073] In one possible embodiment, the decoding capability determination method provided by this solution determines the target per-frame decoding time as a weighted average of the per-frame decoding time prediction result and the previous per-frame decoding time prediction result, which can be configured as follows:

[0074] S231: Determine the first weighting coefficient corresponding to the previous target per-frame decoding time and the second weighting coefficient corresponding to the current per-frame decoding time prediction result based on the number of video frames corresponding to the previous target per-frame decoding time. The larger the number of video frames corresponding to the previous target per-frame decoding time, the smaller the first weighting coefficient.

[0075] S232: Perform weighted averaging on the predicted result of the per-frame decoding time and the previous target per-frame decoding time according to the first weighting coefficient and the second weighting coefficient, and determine the weighted averaging result as the target per-frame decoding time.

[0076] Exemplarily, when the difference between the prediction result of the per-frame decoding time and the previous prediction result of the per-frame decoding time is within a set difference threshold, the first weighting coefficient corresponding to the previous target per-frame decoding time and the second weighting coefficient corresponding to the current per-frame decoding time prediction result are determined based on the number of video frames corresponding to the previous target per-frame decoding time, wherein the sum of the first weighting coefficient and the second weighting coefficient is 1, and the first weighting coefficient is negatively correlated with the corresponding number of video frames, that is, the larger the number of video frames corresponding to the previous target per-frame decoding time, the smaller the first weighting coefficient.

[0077] In one embodiment, the target decoding time per frame can be taken as the exponentially weighted moving average of the predicted result of the decoding time per frame and the previous target decoding time per frame. Correspondingly, when determining the first weighting coefficient corresponding to the previous target decoding time per frame based on the number of video frames corresponding to the previous target decoding time per frame, the set power of the set parameter can be used as the first weighting coefficient, and the set number is the number of video frames corresponding to the previous target decoding time per frame.

[0078] In one embodiment, after determining the first and second weighting coefficients, a weighted average is performed on the predicted per-frame decoding time and the previous target per-frame decoding time based on the first and second weighting coefficients, and the weighted average result is determined as the target per-frame decoding time. For example, the product of the first weighting coefficient and the previous target per-frame decoding time is summed with the product of the second weighting coefficient and the predicted per-frame decoding time, and the sum is determined as the weighted average result. For example, the target per-frame decoding time can be expressed as: in, is the target decoding time per frame, α is a setting parameter with a value range of (0, 1), and f is the number of video frames corresponding to the last target decoding time per frame. is the prediction result of the decoding time per frame, and r is the previous target decoding time per frame.

[0079] This solution determines the first weighting coefficient and the second weighting coefficient based on the number of video frames corresponding to the previous target per-frame decoding time, and uses the first weighting coefficient and the second weighting coefficient to perform weighted averaging on the predicted result of per-frame decoding time and the previous target per-frame decoding time to obtain the target per-frame decoding time, thereby achieving a smooth transition in the prediction of decoding capability, reducing the situation where the user experience is reduced due to excessive changes in the video gear sent to the user, and ensuring the user experience.

[0080] As described above, the decoding time prediction model is used to analyze and process the device basic data, decoding capability data, real-time performance data and device static data to obtain the decoding time prediction results for each frame, and the target decoding time for each frame is determined based on the decoding time prediction results for each frame. The decoding time prediction model is based on the device basic sample data, decoding capability sample data, real-time performance sample data and device static sample data of the decoding capability of the sample device in different change point intervals in the time series, as well as the decoding time sample data for each frame corresponding to the change point interval. The decoding time prediction model can accurately predict the corresponding decoding time prediction results for each frame when the decoding capability of the device changes, accurately determine the target decoding time for each frame, and effectively improve the prediction effect of the decoding capability of the device. At the same time, when the prediction result of the decoding time per frame is the first prediction result, or the difference with the previous prediction result of the decoding time per frame exceeds the set difference threshold, the prediction result of the decoding time per frame is determined as the target decoding time per frame, so as to timely discover the changes in the decoding capability of the device, accurately determine the target decoding time per frame that reflects the real-time decoding capability of the device, and when the difference between the frame decoding time prediction result and the previous prediction result of the decoding time per frame is within the set difference threshold, use the weighted average result of the current prediction result of the decoding time per frame and the previous target decoding time per frame as the target decoding time per frame, and smoothly transition the target decoding time per frame when the decoding capability of the device changes slightly, thereby achieving a smooth transition of the decoding capability prediction, reducing the situation where the user experience is reduced due to excessive changes in the video gear sent to the user, and ensuring the user experience.

[0081] FIG4 is a schematic diagram of a decoding capability determination device according to an embodiment of the present invention. Referring to FIG4 , the decoding capability determination device includes a data acquisition module 41 , a time prediction module 42 , and a time determination module 43 .

[0082] Among them, the data acquisition module 41 is configured to obtain device basic data, decoding capability data, real-time performance data and device static data; the time prediction module 42 is configured to input the device basic data, decoding capability data, real-time performance data and device static data into the trained decoding time prediction model, and analyze and process the device basic data, decoding capability data, real-time performance data and device static data through the decoding time prediction model to obtain the prediction result of each frame decoding time. The decoding time prediction model is based on the device basic sample data, decoding capability sample data, real-time performance sample data and device static sample data of the sample device's decoding capability at different change point intervals in the time series, as well as the each frame decoding time sample data corresponding to the change point interval. The time determination module 43 is configured to determine the target each frame decoding time according to the each frame decoding time prediction result.

[0083] As described above, the decoding time prediction model is used to analyze and process the device basic data, decoding capability data, real-time performance data and device static data to obtain the decoding time prediction results for each frame, and the target decoding time for each frame is determined based on the decoding time prediction results for each frame. The decoding time prediction model is based on the device basic sample data, decoding capability sample data, real-time performance sample data and device static sample data of the decoding capability of the sample device in different change point intervals in the time series, as well as the decoding time sample data for each frame corresponding to the change point interval. The decoding time prediction model can accurately predict the corresponding decoding time prediction results for each frame when the decoding capability of the device changes, accurately determine the target decoding time for each frame, and effectively improve the prediction effect of the decoding capability of the device.

[0084] In a possible embodiment, the time determination module 43 determines the target per-frame decoding time according to the per-frame decoding time prediction result, and is configured as follows:

[0085] When the prediction result of the per-frame decoding time is the first prediction result, or the difference between the prediction result of the per-frame decoding time and the previous prediction result of the per-frame decoding time exceeds a set difference threshold, the prediction result of the per-frame decoding time is determined as the target per-frame decoding time.

[0086] In a possible embodiment, the time determination module 43 determines the target per-frame decoding time according to the per-frame decoding time prediction result, and is configured as follows:

[0087] When the difference between the predicted result of each frame decoding time and the previous predicted result of each frame decoding time is within the set difference threshold, the weighted average result of the predicted result of each frame decoding time and the previous target each frame decoding time is determined as the target each frame decoding time.

[0088] In one possible embodiment, the time determination module 43 determines the target per-frame decoding time by taking the weighted average of the per-frame decoding time prediction result and the previous per-frame decoding time prediction result, and is configured as follows:

[0089] Determining a first weighting coefficient corresponding to a previous per-frame decoding time prediction result and a second weighting coefficient corresponding to a current per-frame decoding time prediction result according to the number of video frames corresponding to the previous per-frame decoding time prediction result, wherein the larger the number of video frames corresponding to the previous per-frame decoding time prediction result, the smaller the first weighting coefficient;

[0090] A weighted average processing is performed on the prediction result of the decoding time per frame and the previous prediction result of the decoding time per frame according to the first weighting coefficient and the second weighting coefficient, and the weighted average result is determined as the target decoding time per frame.

[0091] In a possible embodiment, the decoding capability determination device further includes a model training module, and the model training module is configured to:

[0092] Obtain device basic sample data, decoding capability sample data, real-time performance sample data, and device static sample data of decoding capabilities of multiple sample devices;

[0093] Determine, based on the decoding capability sample data and the device basic sample data, different change point intervals of the decoding capability of each sample device in the time series, and sample data of the decoding time per frame corresponding to each change point interval;

[0094] The decoding time prediction model is trained using device basic sample data, decoding capability sample data, real-time performance sample data, and device static sample data of the decoding capabilities of each sample device at different change point intervals in the time series, as well as the decoding time sample data of each frame corresponding to the change point interval.

[0095] In one possible embodiment, the model training module determines different change point intervals of the decoding capability of each sample device in a time series, and sample data of the decoding time per frame corresponding to each change point interval, based on the decoding capability sample data and the device basic sample data, and is configured as follows:

[0096] Based on the set window length information and the difference function of the loss function centered on the window, at the center of the window and at both ends, the different change point intervals of the decoding capability of the sample device in the time series are determined. The loss function is determined based on the average decoding time sample data in the decoding capability sample data and the number of sample video frames in the device basic sample data.

[0097] In a possible embodiment, the decoding capability determination device also includes a time feedback module, which is configured to send the target decoding time per frame to the media background, and is used to determine the video downlink gear according to the target decoding time per frame through the media background.

[0098] It is worth noting that in the embodiment of the above-mentioned decoding capability determination device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of this application.

[0099] The embodiment of the present application also provides a decoding capability determination device, which can integrate the decoding capability determination apparatus provided by the embodiment of the present application. Figure 5 is a structural diagram of a decoding capability determination device provided by the embodiment of the present application. Referring to Figure 5, the decoding capability determination device includes: an input device 53, an output device 54, a memory 52 and one or more processors 51; the memory 52 is used to store one or more programs; when the one or more programs are executed by one or more processors 51, the one or more processors 51 implement the decoding capability determination method provided by the above embodiment. The decoding capability determination device, equipment and computer provided above can be used to execute the decoding capability determination method provided by any of the above embodiments, and have corresponding functions and beneficial effects.

[0100] The embodiment of the present application also provides a non-volatile storage medium for storing computer-executable instructions, which are used to perform the decoding capability determination method provided in the above embodiment when executed by a computer processor. Of course, the non-volatile storage medium for storing computer-executable instructions provided in the embodiment of the present application, whose computer-executable instructions are not limited to the decoding capability determination method provided above, can also perform the relevant operations in the decoding capability determination method provided in any embodiment of the present application. The decoding capability determination device, equipment and storage medium provided in the above embodiment can execute the decoding capability determination method provided in any embodiment of the present application. For technical details not described in detail in the above embodiment, please refer to the decoding capability determination method provided in any embodiment of the present application.

[0101] Based on the above embodiments, the embodiments of the present application also provide a computer program product. The essence of the technical solution of the present application or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes a number of instructions for enabling a computer device, a mobile terminal or the processor therein to execute all or part of the steps of the decoding capability determination method provided in each embodiment of the present application.

Claims

1. A method for determining decoding capability, wherein: include: Obtain device basic data, decoding capability data, real-time performance data, and device static data; Inputting the device basic data, the decoding capability data, the real-time performance data, and the device static data into a trained decoding time prediction model, analyzing and processing the device basic data, the decoding capability data, the real-time performance data, and the device static data through the decoding time prediction model to obtain a per-frame decoding time prediction result, wherein the decoding time prediction model is trained based on the device basic sample data, decoding capability sample data, real-time performance sample data, and device static sample data of the sample devices at different change point intervals in the decoding capability in a time series, as well as the per-frame decoding time sample data corresponding to the change point intervals; The target per-frame decoding time is determined according to the per-frame decoding time prediction result.

2. The method for determining decoding capability according to claim 1, wherein: The determining of the target per-frame decoding time according to the per-frame decoding time prediction result includes: When the per-frame decoding time prediction result is the first prediction result, or the difference between the per-frame decoding time prediction result and the previous per-frame decoding time prediction result exceeds a set difference threshold, the per-frame decoding time prediction result is determined as the target per-frame decoding time.

3. The method for determining decoding capability according to claim 1, wherein: The determining of the target per-frame decoding time according to the per-frame decoding time prediction result includes: When the difference between the predicted result of the per-frame decoding time and the previous predicted result of the per-frame decoding time is within the set difference threshold, the weighted average result of the predicted result of the per-frame decoding time and the previous target per-frame decoding time is determined as the target per-frame decoding time.

4. The method for determining decoding capability according to claim 3, wherein: The step of determining a weighted average of the prediction result of the per-frame decoding time and the previous prediction result of the per-frame decoding time as a target per-frame decoding time includes: determining, according to the number of video frames corresponding to the previous per-frame decoding time prediction result, a first weighting coefficient corresponding to the previous per-frame decoding time prediction result, and a second weighting coefficient corresponding to the current per-frame decoding time prediction result, wherein the greater the number of video frames corresponding to the previous per-frame decoding time prediction result, the smaller the first weighting coefficient; A weighted average processing is performed on the prediction result of the per-frame decoding time and the previous prediction result of the per-frame decoding time according to the first weighting coefficient and the second weighting coefficient, and the weighted average result is determined as the target per-frame decoding time.

5. The method for determining decoding capability according to claim 1, wherein: The training process of the decoding time prediction model includes: Obtain device basic sample data, decoding capability sample data, real-time performance sample data, and device static sample data of decoding capabilities of multiple sample devices; Determining, based on the decoding capability sample data and the device basic sample data, different change point intervals of the decoding capability of each of the sample devices in a time series, and sample data of per-frame decoding time corresponding to each of the change point intervals; The decoding time prediction model is trained using the device basic sample data, decoding capability sample data, real-time performance sample data and device static sample data of the decoding capabilities of each sample device at different change point intervals in the time series, as well as the decoding time sample data of each frame corresponding to the change point intervals.

6. The method for determining decoding capability according to claim 5, wherein: Determining, based on the decoding capability sample data and the device basic sample data, different change point intervals of the decoding capability of each of the sample devices in a time series, and sample data of per-frame decoding time corresponding to each of the change point intervals, includes: Based on the set window length information and the difference function of the loss function at the center and both ends of the window centered on the window, the different change point intervals of the decoding capability of the sample device in the time series are determined. The loss function is determined based on the average decoding time sample data in the decoding capability sample data and the number of sample video frames in the device basic sample data.

7. The method for determining decoding capability according to any one of claims 1 to 6, wherein: After determining the target per-frame decoding time according to the per-frame decoding time prediction result, the method further includes: The target decoding time per frame is sent to the media background, so that the media background can determine the video delivery gear according to the target decoding time per frame.

8. A decoding capability determination device, wherein: It includes a data acquisition module, a time prediction module and a time determination module, among which: The data acquisition module is configured to acquire device basic data, decoding capability data, real-time performance data and device static data; The decoding time prediction module is configured to input the basic device data, the decoding capability data, the real-time performance data, and the static device data into a trained decoding time prediction model, and analyze and process the basic device data, the decoding capability data, the real-time performance data, and the static device data through the decoding time prediction model to obtain a prediction result of the decoding time for each frame, wherein the decoding time prediction model is trained based on the basic device sample data, the decoding capability sample data, the real-time performance sample data, and the static device sample data of the decoding capability of the sample device at different change point intervals in the time series, as well as the decoding time sample data for each frame corresponding to the change point intervals; The time determination module is configured to determine a target per-frame decoding time according to the per-frame decoding time prediction result.

9. A decoding capability determination device, wherein: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the decoding capability determination method according to any one of claims 1 to 7.

10. A non-volatile storage medium storing computer-executable instructions, wherein: When the computer executable instructions are executed by a computer processor, the computer executable instructions are used to perform the decoding capability determination method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program, wherein When the computer program is executed by a processor, the decoding capability determination method according to any one of claims 1 to 7 is implemented.

Images