A code rate allocation method, storage method, device, equipment and storage medium

Abstract

The present invention discloses a bit rate allocation method, storage method, device, equipment and storage medium. The present invention obtains target video data, the target video data includes the viewing probability of all video clips in the target transmission video, wherein the introduction of the viewing probability is conducive to improving the viewing quality of users; the video clips are allocated bit rates according to the viewing probability so that the total utility of the target transmission video meets the target value; and the total utility of the target transmission video is determined according to the user viewing quality of the video clips and the transmission loss, the user viewing quality is determined according to the non-freshness factor, and the non-freshness factor is determined according to the storage status of the video clips at the edge node and the viewing probability. When the total utility of the target transmission video meets the target value, a corresponding bit rate allocation result is obtained, which can achieve the goal of improving the user viewing quality of the video clips while reducing the transmission loss. The present invention can be widely applied to the field of communication technology.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to a code rate allocation method, storage method, device, equipment and storage medium. Background Art

[0002] Due to the excellent support of 5G networks for HD video, HD video is gradually being applied in multiple industries such as surveillance and security, remote conferencing, e-commerce live streaming, VR games / videos, etc., and is favored by content providers and consumers. The transmission process of HD video requires a large amount of bandwidth and storage resources. Usually, HD video is stored by servers, resulting in large losses during the transmission process. To this end, edge nodes closer to users can be used to cache HD video. Users can directly obtain cached HD video from the edge node without having to obtain it from the server, reducing losses during the transmission process. However, as the time HD video is stored on the edge node increases, the value of the HD video to the user will decrease. Under limited bandwidth conditions, if the HD video maintains the original bit rate, it will affect other HD videos, resulting in increased overall HD video transmission losses and affecting the user viewing experience. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to provide a code rate allocation method, storage method, device, equipment and storage medium that reduce transmission loss.

[0004] To achieve the above-mentioned purpose, an embodiment of the present application provides a bit rate allocation method, including: obtaining target video data; the target video data includes the viewing probability of all video segments in the target transmission video; allocating bit rates for the video segments according to the viewing probability so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined based on the user viewing quality and transmission loss of the video segment, the user viewing quality is determined based on the non-freshness factor, and the non-freshness factor is determined based on the storage status of the video segment at the edge node and the viewing probability.

[0005] An embodiment of the present application also proposes a storage method, including: obtaining the viewing probability and bit rate allocation results of all video clips in the target transmission video through the bit rate allocation method; predicting the expected effective quality and expected transmission loss based on the bit rate allocation results and the viewing probability; performing storage optimization processing based on the expected effective quality, the expected transmission loss and the maximum capacity of the edge node to obtain storage decision information; the storage decision information includes a third storage decision value for each of the video clips, and the third storage decision value represents whether the video clip needs to be stored in the edge node; and storing the video clips according to the storage decision information.

[0006] An embodiment of the present application also proposes a bit rate allocation device, including: an acquisition module, used to acquire target video data; the target video data includes the viewing probability of all video segments in the target transmission video; a processing module, used to allocate bit rates to the video segments according to the viewing probability, so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined based on the user viewing quality of the video segment and the transmission loss, the user viewing quality is determined based on the non-freshness factor, and the non-freshness factor is determined based on the storage status of the video segment at the edge node and the viewing probability.

[0007] The embodiment of the present application also proposes a storage device, including: a determination module for determining the viewing probability and bit rate allocation result of all video segments in the target transmission video; a prediction module for predicting the expected effective quality and the expected transmission loss based on the bit rate allocation result and the viewing probability; an optimization module for performing storage optimization processing based on the expected effective quality, the expected transmission loss and the maximum capacity of the edge node to obtain storage decision information; the storage decision information includes a third storage decision value for each of the video segments, and the third storage decision value represents whether the video segment needs to be stored in the edge node; a storage module for performing storage optimization processing based on the storage decision information. The decision information is used to store the video clips; the viewing probability and bit rate allocation result of all video clips in the target transmission video are determined, specifically including: obtaining target video data; the target video data includes the viewing probability of all video clips in the target transmission video; bit rates are allocated to the video clips according to the viewing probability so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user viewing quality and transmission loss of the video clip, the user viewing quality is determined according to the non-freshness factor, and the non-freshness factor is determined according to the storage status of the video clip at the edge node and the viewing probability.

[0008] An embodiment of the present application also proposes an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the above-mentioned bit rate allocation method or storage method.

[0009] An embodiment of the present application also proposes a computer-readable storage medium, which stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-mentioned bit rate allocation method or storage method.

[0010] The embodiments of the present application propose a bit rate allocation method, storage method, apparatus, device and storage medium, which obtain target video data, wherein the target video data includes the viewing probability of all video segments in the target transmission video, wherein the introduction of the viewing probability is conducive to improving the user's viewing quality; bit rates are allocated to the video segments according to the viewing probability so that the total utility of the target transmission video meets the target value; and the total utility of the target transmission video is determined according to the user viewing quality of the video segment and the transmission loss, wherein the user viewing quality is determined according to the non-freshness factor, and the non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability. When the total utility of the target transmission video meets the target value, a corresponding bit rate allocation result is obtained, which can achieve the goal of improving the user viewing quality of the video segment while reducing the transmission loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flowchart of the steps of the bit rate allocation method for this application;

[0012] Figure 2 This is a flowchart of the steps of the storage method according to a specific embodiment of the present application;

[0013] Figure 3 This is a flowchart of a bit rate allocation method and a storage method for one of the application scenarios of a specific embodiment of the present application;

[0014] Figure 4 This is a flowchart of a code rate allocation method and a storage method for another application scenario of a specific embodiment of the present application;

[0015] Figure 5 This is a schematic diagram of a bit rate allocation device according to a specific embodiment of the present application;

[0016] Figure 6 A schematic diagram of a storage device according to a specific embodiment of the present application;

[0017] Figure 7 This is a schematic diagram of an electronic device according to a specific embodiment of the present application. DETAILED DESCRIPTION

[0018] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0019] In the subsequent description, suffixes such as "module," "component," or "unit" used to represent elements are used only to facilitate the description of the present application and have no specific meaning. Therefore, "module," "component," or "unit" may be used interchangeably.

[0020] Example 1

[0021] like Figure 1As shown, the embodiment of the present application provides a code rate allocation method, which at least includes but is not limited to steps S100-S200:

[0022] S100: Acquire target video data.

[0023] In an embodiment of the present application, the target video data includes the viewing probability of all video segments in the target transmission video. Optionally, the target video data is the data of the video that the user needs to watch, that is, the relevant data of the target transmission video, which can be obtained by the user end by generating a video request based on the user's input instruction. It should be noted that the cloud end (or server end) can block the target transmission video to generate multiple video segments in different spatial positions. Optionally, the target transmission video with a duration of T is encoded into K quality levels in the cloud, and the video of each quality level is divided into video segments of length Δt seconds (including but not limited to 2 seconds) in time, and each video segment of length Δt seconds is block-processed into M×N video segments in space, where M and N are the number of video segments in the horizontal and vertical directions, respectively. In an embodiment of the present application, in order to obtain the user's region of interest (RoI) for the video segment, the viewing probability of each video segment can be determined by the trajectory of the user's focus point, and the sum of the viewing probabilities within the user's RoI is 1, that is, it satisfies m,n are the index positions of the video segments, and p(m,n) is the viewing probability corresponding to the video segment (m,u) (the probability of being within the user's RoI). A higher viewing probability indicates greater user attention. It should be noted that when the target video is immersive, such as VR (Virtual Reality) and AR (Augmented Reality) videos, users can view 360-degree video content by wearing a head-mounted display (HMD). In this case, the RoI area is the user's actual viewable field of view (FoV) in the HMD, and the viewing probability can be determined by eye movement trajectories. When the target video is other types of video, the viewing probability can be calculated by acquiring the frequency of real-time mouse operation areas or by predicting the importance of the user's likely attention areas through video image processing. For example, a saliency map can be used to predict video salient areas, and the grayscale value ratio of different regions can be used to determine the importance of the viewing area and thus determine the viewing probability.

[0024] S200 : Allocate bit rates for video segments according to viewing probabilities, so that the total utility of the video segments meets a target value.

[0025] In this embodiment of the present application, bitrate allocation refers to determining the target transmission bitrate for each video clip, that is, the bitrate at which the edge node transmits each video clip to the user end; the total utility of the target transmitted video is determined based on the user viewing quality of the video clip and the transmission loss. Optionally, the total utility of the target transmitted video is represented by a total utility function:

[0026]

[0027] Among them, Utility t is the total utility at time t, that is, the rate allocation system utility at time t, QoP t (m,n) is the user viewing quality of the video segment (m,n) at time t, TC t (m,n) is the transmission loss of the video segment (m,n) at time t, and δ is the weight of the transmission loss. It can be seen from the formula that the utility of each video segment can be obtained by subtracting the product of the transmission loss and the weight of the transmission loss from the user viewing quality corresponding to each video segment. The sum of the utilities of all video segments is the total utility of the target transmission video. Optionally, if a video segment is stored in the cloud (cloud server), the transmission loss of a video segment includes the communication loss of the edge node obtaining the video segment from the cloud and the communication loss of the edge node transmitting the video segment to the user end; if a video segment is stored in the edge node, the transmission loss of the video segment includes the communication loss of the edge node transmitting the video segment to the user end, or also includes the transcoding loss of the edge node. It should be noted that the total utility of the target transmission video meets the target value, including but not limited to making the total utility function reach the maximum value (MAX Utility t ); or, first set a total utility target threshold. When the total utility calculated using the total utility function is greater than or equal to the total utility target threshold, it is considered that the target value is met. When it is determined that the total utility meets the target value, the user viewing quality of the target transmission video can be improved and the transmission communication loss of obtaining video clips from the cloud can be reduced, thereby improving the quality of the target transmission video viewed by users under the condition of limited user-side bandwidth.

[0028] In the embodiment of the present application, the example of satisfying the target value to maximize the total utility function is used to illustrate the rate allocation problem, which can be described as follows:

[0029]

[0030]

[0031]

[0032]

[0033] Among them, B maxis the maximum bandwidth transmitted between the edge node and the user end, is the bit rate at time t, specifically the transmission bit rate allocated to the video segment (m, n) at time t for the edge node to transmit the video segment (m, n) to the user end, c t-1 (m,n) is the storage decision value at time t-1. When the storage decision value is 1, it means that the video segment (m,n) is stored in the edge node, and when the storage decision value is 0, it means that the video segment (m,n) is not stored in the edge node; is the bit rate at time t-1, specifically the storage bit rate when the video clip (m, n) is stored by the edge node at time t-1. In the embodiment of the present application, the bandwidth constraint condition limits all The sum cannot exceed the maximum bandwidth B max ; For the video clips stored in the edge nodes (ie c t-1 The bit rate constraint condition (m,n)=1) stipulates that the bit rate assigned to the stored video clip at time t cannot be higher than This allows users to obtain video clips directly from edge nodes, reducing the additional communication loss caused by obtaining video clips from the cloud. is the quality level assigned to the video segment (m,n) at time t, q K is the Kth quality level, which can be measured by the ffprobe tool (a tool that can be used to view file format information in FFmpeg). The quality level is related to the bit rate and content complexity of the video clip. t (m,n) are closely related, that is, F() represents a mapping relationship. It should be noted that when the total utility is calculated by formula (2), the complexity of integer programming is O(K M·N ), then the target quality level of each video clip can be determined to solve the maximum total utility, and in the embodiment of the present application, the computational complexity of the integer programming problem is O(K M·N ), the execution time is too long, so a greedy algorithm is used to obtain an approximate optimal solution to reduce the computational complexity, so that the total utility meets the target value and the target quality level of each video clip and the corresponding target transmission bit rate are determined to achieve bit rate allocation.

[0034] Optionally, in the embodiment of the present application, in order to determine that the total utility function of formula (2) is maximized, bitrate allocation is performed on the video segments according to the viewing probability in step S200, including steps B201 and B202, and B203 and / or B204:

[0035] B201. Obtain a new utility value and an old utility value of the total utility of the target transmission video, and obtain a quality level and a bit rate of the video segment.

[0036] In the embodiment of the present application, the new utility value is the initial value of the total utility of the target transmission video, that is, the initial value of the total utility function in formula (1), which can be a randomly determined value or a set value, denoted as newU t , such as newU t = 0. The old utility value can be the current value of the total utility of the target transmission video determined according to the quality level and bit rate, recorded as oldU t The current moment is t, and the quality level and bit rate can be the quality level and bit rate currently assigned to the video segment, or can also be set values. In the embodiment of the present application, the quality level is used as the set value for explanation, and no specific limitation is given. Specifically, the quality level of each video segment (m, n) is set to the lowest quality level 1, which is recorded as Construct the quality level matrix of all video segments (m,n) The corresponding bit rate can be determined by the quality level or the initial bit rate can be set as Thus, the bit rate matrix of all video clips (m,n) is obtained Then, by substituting it into formula (1) or (2), we can calculate oldU t .

[0037] B202. Determine whether the sum of the bit rates is less than or equal to the bandwidth capacity.

[0038] Specifically, determine whether the sum of the bit rates of all video clips is less than or equal to the bandwidth capacity, that is, determine whether the sum of the bit rates of all video clips is less than or equal to the bandwidth capacity. Is it true?

[0039] B203. When the sum of the bit rates is less than or equal to the bandwidth capacity, traverse the video segments and perform quality level increase processing. According to the increase processing result, the new utility value and the old utility value, update the quality level and bit rate of the video segments, and return to the step of determining whether the sum of the bit rates is less than or equal to the bandwidth capacity, until the sum of the bit rates is greater than the bandwidth capacity, and obtain the target transmission bit rate for each video segment.

[0040] In the embodiment of the present application, when the sum of the bit rates of all video clips is The video segments are traversed to increase the quality level. For example, the traversal process starts with the Nth video segment in the Mth column of the video segment matrix composed of M×N video segments as the first video segment until the traversal reaches the 1st video segment in the 1st column. During the traversal process, the quality level of the video segments is increased. For example, the quality level of the video segments is increased. After the addition process is performed, the result of the addition process is obtained. The value of the increase can be adjusted as needed. This application uses 1 as an example and does not impose a specific limit on the value of the increase. Then, after the increase process, the quality level and bit rate of the video clip are updated according to the increase process result, the new utility value and the old utility value, and the process returns to step B202 until Thus, the target transmission bit rate of each video clip is obtained.

[0041] B204. When the sum of the bit rates is not less than or equal to the bandwidth capacity, traverse the video segments to reduce the quality level, update the quality level and bit rate of the video segments based on the reduction processing results, the new utility value and the old utility value, and return to the step of determining whether the sum of the bit rates is less than or equal to the bandwidth capacity, until the sum of the bit rates is less than or equal to the bandwidth capacity, and obtain the target transmission bit rate for each video segment.

[0042] Specifically, when the sum of the bit rates is not less than or equal to the bandwidth capacity, that is, The video segments are traversed to reduce the quality level. For example, the traversal process starts with the Nth video segment in the Mth column of the video segment matrix composed of M×N video segments as the first video segment until the traversal reaches the 1st video segment in the 1st column. During the traversal process, the quality level of the video segments is reduced. For example, the quality level of the video segments is reduced. After the reduction process, the reduction result is obtained Equivalent to the quality level after reduction; similarly, the value of reduction can be adjusted as needed. This application takes 1 as an example and does not impose a specific limit on the value of reduction. Then, after the reduction process, the quality level and bit rate of the video clip are updated according to the reduction process result, the new utility value and the old utility value, and the process returns to step B202 until Thus, the target transmission bit rate of each video clip is obtained.

[0043] It should be noted that after executing step B202 or B203, when the quality level and bit rate of one or more video clips are updated, an updated video clip matrix consisting of M×N video clips and an updated quality level matrix are obtained, and the target transmission bit rate of the video clip is determined based on the updated quality level matrix. It should be noted that the target transmission bit rate refers to the final determined quality level of the video clip at time t. Recorded as The final target transmission rate matrix is

[0044] Optionally, in step B203, updating the quality level and bit rate of the video clip according to the addition processing result, the new utility value and the old utility value includes step B211 or B212:

[0045] B211. When the added processing result is less than or equal to the quality level threshold, the new utility value is updated according to the added processing result, the first difference between the updated new utility value and the old utility value is calculated, the first impact factor of the video clip is determined, the video clip corresponding to the maximum value of the first impact factor is used as the first updated video clip, the quality level of the first updated video clip is updated to the added processing result corresponding to the first updated video clip, and the bit rate of the first updated video clip is updated according to the added processing result corresponding to the first updated video clip.

[0046] Optionally, the quality level threshold is quality level K. When the added processing result is less than or equal to the quality level threshold, Assume that m=M, n=N during the traversal process, the quality level of the current video clip The quality level matrix obtained after the addition process is The updated new utility value can be calculated using formula (1) or formula (2), that is, the updated newU t Then, according to the updated newU t and the old utility value oldU t The first difference of the current video segment (m = M, n = N) determines the first impact factor diff t (m,n). It should be noted that, before the traversal begins, an initial first impact factor matrix can be generated, and then each initial first impact factor in the initial first impact factor matrix can be updated during the traversal process. It can be understood that in the traversal process from m=M and n=N to m=1 and n=1, each video segment can obtain the updated first impact factor corresponding to each video segment through the same processing process, and the first impact factor matrix is ​​recorded as First Impact Factor Matrix Each video clip in corresponds to a first impact factor, which indicates the degree of change in the total utility of the video clip after the addition processing.

[0047] Optionally, after the traversal of the video clips is completed, the first impact factor matrix is ​​determined Then, find the position index of the maximum value For example, if the position of the video segment with the largest first impact factor is (M-3, N-3), then m=M-3, n=N-3, and the video segment at the position (M-3, N-3) in the video segment matrix is ​​used as the first updated video segment, and the quality level of the first updated video segment is updated to the increased processing result corresponding to the first updated video segment. and updating the bit rate of the first updated video segment according to the increase processing result corresponding to the first updated video segment, Or it can be expressed as: a new quality level matrix is ​​obtained according to the added processing result corresponding to the first updated video segment. Thus determining the updated code rate matrix Then return to step B202, where F() represents mapping.

[0048] B212: When the added processing result is greater than the quality level threshold, the quality level of the video segment is used as the updated quality level and the bit rate of the video segment is used as the updated bit rate.

[0049] In the embodiment of the present application, for example, the current video clip is the video clip of the Mth row and the Nth column. At this time, the quality level and bit rate of the video clip are kept unchanged, that is, the quality level of the video clip is used as the updated quality level, and the bit rate of the video clip is used as the updated bit rate. Then, the next video clip of the video clip in the Mth row and Nth column is traversed to determine the relationship between the increased processing result and the quality level threshold.

[0050] Optionally, in step B211, the new utility value is updated according to the addition processing result, including steps B221-B226:

[0051] B221. Calculate the transmission loss at the current moment and calculate the non-freshness at the current moment.

[0052] Specifically, the calculation steps of the transmission loss at the current moment include B2211-B2212, and B2212 includes step A1 or A2:

[0053] B2211. Determine a first storage decision value based on the storage status of the video clip at the edge node at a previous moment.

[0054] In the embodiment of the present application, the current moment is recorded as moment t, and the previous moment is recorded as moment t-1. In other embodiments, the previous moment may be moment t-2 or other moments, without specific limitation.

[0055] Optionally, the storage status of the video clip and the edge node at the previous moment includes being stored in the edge node or not being stored in the edge node. When stored in the edge node, the corresponding first storage decision value is 1, that is, the first storage decision value c t-1 (m,n) = 1 indicates that the video clip is stored in the edge node; on the contrary, when the first storage decision value c t-1 When (m,n)=0, it indicates that the video clip is not stored in the edge node. In this case, the edge node needs to obtain the video clip from the cloud. It should be noted that the first storage decision value refers to the storage decision value corresponding to the storage status of the video clip at the previous moment.

[0056] A1. When the first storage decision value indicates that the video clip is stored at the edge node, obtain the storage bit rate of the video clip, calculate the bit rate difference between the storage bit rate and the bit rate of the video clip, determine the first loss based on the first storage decision value, the bit rate difference, and the first loss coefficient, and determine the second loss based on the second loss coefficient and the bit rate of the video clip. Determine the transmission loss at the current moment based on the sum of the first loss and the second loss.

[0057] A2. When the first storage decision value indicates that the video clip is not stored in the edge node, the third loss is determined based on the third loss coefficient and the bit rate of the video clip, and the second loss is determined based on the second loss coefficient and the bit rate of the video clip. The transmission loss at the current moment is determined based on the sum of the third loss and the second loss.

[0058] Optionally, for ease of description, assume that δ = 1. In this case, the transmission loss of the video segment (m, n) at time t in formula (2) is expressed as TC t (m,n), the formula is:

[0059]

[0060] Wherein, c1 is the first loss coefficient, c2 is the second loss coefficient, and c3 is the third loss coefficient. It should be noted that when calculating the transmission loss of a video clip, c t-1 (m,n) is the first stored decision value of the video segment at time t-1, is the bit rate of the video clip at time t, is the bit rate of the video segment at time t-1; the first loss coefficient, the second loss coefficient, and the third loss coefficient can be adjusted as needed. It can be understood that by calculating the sum of the transmission losses of each video segment, the transmission loss of the target transmission video can be obtained. Specifically, from formula (6), it can be seen that the transmission loss is divided into three parts. Taking one of the video segments (m, n) as an example:

[0061] 1. That is, the first loss, which represents the transcoding loss of the edge node;

[0062] 2. That is, the second loss,represents the communication loss of the edge node transmitting the video clip to the user end;

[0063] 3. That is, the third loss, which represents the communication loss of the edge node obtaining the video clips from the cloud.

[0064] Therefore, in step A1, when c t-1 When (m,n)=1, the video clip is stored in the edge node and thus only the first loss or the second loss may occur. Greater than or equal to When , the edge node can directly transmit the video clip to the user end to improve storage efficiency, otherwise when Less than When , it still needs to be retrieved from the cloud. equal When , the first loss is minimum and is 0; when Greater than When the edge node needs to transcode the video clip into the bit rate assigned by the system before transmitting it to the user end. Specifically, the storage bit rate is That is, the video clip has been Stored in the edge node, the bit rate difference is calculated Thus, the first loss is determined, and the second loss coefficient c2 and the bit rate of the video clip are used to calculate the loss. Determine the second loss, and determine the transmission loss at the current moment based on the sum of the first loss and the second loss:

[0065] Similarly, in step A2, when c t-1 When (m,n)=0, the video clip is not stored in the edge node. The edge node obtains the video clip from the cloud and stores it in the edge node. Storage, at this time there is no first loss, but there are second and third losses. Specifically, according to the third loss coefficient c3 and the bit rate of the video segment Determine the third loss and calculate the value of the third loss coefficient c2 based on the bit rate of the video segment. Determine the second loss, and determine the transmission loss at the current moment based on the sum of the third loss and the second loss

[0066] Optionally, calculating the non-freshness at the current moment in step B221 includes steps B2213 to B2215, and step B2215 includes step B1 or B2:

[0067] B2213. Determine the non-freshness at the previous moment.

[0068] Similarly, let the previous moment be time t-1. If the video clip at the previous moment is stored in the edge node, the non-freshness of the previous moment is calculated by the non-freshness formula uf t-1 (m,n); if the video clip at the previous moment is not stored in the edge node, the non-freshness uf t-1 (m, n) is 0. In other embodiments, the non-freshness can be other values, such as a smaller value close to 0.

[0069] B2214. Determine a second storage decision value based on the storage status of the video clip at the edge node at the current moment.

[0070] Specifically, as in step B2211, a second storage decision value is determined. When the second storage decision value is 1, it indicates that the video clip at the previous moment is stored in the edge node. When the second storage decision value is 0, it indicates that the video clip at the previous moment is not stored in the edge node. It should be noted that the second storage decision value refers to the storage decision value corresponding to the storage status of the video clip at the current moment.

[0071] B1. When the second storage decision value indicates that the video clip is stored at the edge node, the product of the preset freshness growth factor and the viewing probability is calculated, and the non-freshness at the current moment is determined based on the sum of the product and the non-freshness at the previous moment.

[0072] B2. Alternatively, when the second storage decision value indicates that the video clip is not stored in the edge node, the non-freshness at the current moment is determined to be 0.

[0073] In the embodiment of the present application, the calculation formula of non-freshness is:

[0074]

[0075] Among them, uf t (m, n) is the non-freshness of the video clip (m, n) at the current moment, i.e., at moment t. When the second storage decision value is 0, the non-freshness is 0; when c t When (m,n) is 1, p is calculated by the above formula (7). t (m,n) is the viewing probability of video clip (m,n); α is the preset freshness growth factor, uf t-1 (m,n) is the non-freshness of the video clip at the previous moment. It can be understood that the non-freshness of each video clip can be calculated by formula (7); optionally, uf0(m,n) is 0, and the non-freshness of the previous moment can be calculated by formula (7). It should be noted that non-freshness can be used to reflect the degree of obsolescence of the video clip. The video clip content stored in the edge node has a low freshness. The outdated video clip content will weaken the user's viewing experience quality. In addition, from formula (7), it can be known that uf t (m,n) and the viewing probability p of the video segment t (m,n) and the storage status of the video clips at the edge nodes. When users are sensitive to the picture quality of the clips in their viewing field of view, the picture content in the user's RoI needs to be continuously updated to present the user with the freshest content. Therefore, p t The larger the (m,n) is, the faster the freshness of the video clip will be lost.

[0076] B222. Determine the effective quality based on the non-freshness, viewing probability, and quality level at the current moment.

[0077] In the embodiment of the present application, the calculation formula of effective mass is:

[0078]

[0079] Among them, EQ t (m,n) is the effective quality of the video segment (m,n) at time t, and μ is the factor of the effective quality. It should be noted that when calculating the effective quality in each sub-step of step B221, the quality level at the current moment is It is the quality level in step B201 or the result of the added processing in step B203. It is understandable that the effective quality of each video clip can be calculated by the above formula (8). In the embodiment of the present application, the purpose of setting the effective quality is to assign a higher quality level to the video clips in the user's area of ​​interest and to ensure that the loss of the freshness of the video clips is small. Because if the quality level of the video clip is high, but the content is outdated, the effective quality of the video clip will deteriorate. It is necessary to assign a high quality level to the video clips in the user's area of ​​interest to ensure that the user can clearly see the content of the video clips in the area of ​​interest.

[0080] It should be noted that, in the embodiment of the present application, non-freshness is used as a non-freshness factor to calculate the effective quality, and the effective quality is used to determine the user viewing quality, that is, the user viewing quality is determined according to the non-freshness factor. In the related art, taking into account the advantages of mobile edge computing (MEC) nearby services, multimedia content such as popular videos are usually cached in the MEC server at the base station in advance to achieve the purpose of enhancing the user viewing experience. However, these strategies are applicable to videos that already exist, that is, before the user sends a content request, the edge node has already obtained information on the probability of the video being viewed, so it cannot be applied to real-time videos such as live videos, because the information of real-time videos (such as content, probability of being viewed) is updated in real time, recorded while playing, and cannot be obtained in advance. However, some video content is usually most valuable when it is fresh. For example, in autonomous driving, it is essential to know the position, direction and speed of the motor vehicle in real time. When the video clip is stored in the edge node for a long time, the freshness of the video clip decreases, which will affect the user's viewing quality. Therefore, the non-freshness factor is introduced in the embodiment of the present application to describe the degree of content obsolescence of the video clips stored in the edge node and the video clips where no edge node exists. Applying the non-freshness factor to the storage method of bit rate allocation and storage decision-making can improve the quality of the video viewed by the user and reduce transmission loss.

[0081] B223. Obtain the quality level of the video clip at the previous moment, calculate the first subjective quality at the current moment and the second subjective quality at the previous moment based on the quality level at the current moment, the viewing probability, and the quality level at the previous moment, and determine the temporal quality loss based on the difference between the first subjective quality and the second subjective quality.

[0082] In the embodiment of the present application, the calculation formula of subjective quality is:

[0083]

[0084] The calculation formula for time quality loss is:

[0085] TQ t (m,n)=|Q t (m,n)-Q t-1 (m,n)| (10)

[0086] Among them, Q t (m,n) is the subjective quality (i.e., the first subjective quality) of the video segment (m,n) at time t (current time), TQ t (m,n) is the temporal quality loss of the video segment (m,n) at time t, Q t-1 (m,n) is the subjective quality (i.e., the second subjective quality) of the video clip (m,n) at time t-1 (the previous moment). Therefore, the first subjective quality can be calculated by formula (9), and the second subjective quality can be calculated by formula (9) through the quality of the video clip at the previous moment and the viewing probability at the previous moment (the viewing probability at the current moment can be used approximately). Then, the temporal quality loss of the video clip can be calculated by combining formula (10). It should be noted that the subjective quality representation of the video clip depends on the user's sensitivity to the video clip. In the embodiment of the present application, the viewing probability factor of the video clip is normalized to the range of 0-1 by a mathematical method with e as the base, Q t (m,n) and p t (m, n) is inversely proportional. If the importance of a video segment is low, even if a lower Subjective quality Q t (m,n) can still be accepted by users. In addition, since users are sensitive to the quality difference of adjacent time pictures, if the user's subjective quality varies greatly at adjacent times, then even if a higher quality is assigned to the video clip, the quality difference will still have a negative impact on the user's viewing experience. Therefore, the temporal quality loss is defined as formula (10).

[0087] B224. Calculate a subjective quality mean value based on the first subjective quality and the total number of video clips, and determine a spatial quality loss based on a difference between the first subjective quality and the subjective quality mean value.

[0088] In the embodiment of the present application, the calculation formula for space quality loss is:

[0089]

[0090] Among them, SQ t (m,n) is the spatial quality loss of the video segment (m,n) at time t, is the subjective quality mean, M·N is the total number of video clips, Q at time t t (m,n) is the first subjective quality mentioned above. It should be noted that in addition to the poor quality of adjacent video segments, which will weaken the user's viewing experience, the subjective quality difference of video segments in space will also affect the user's viewing experience, so spatial quality loss needs to be considered.

[0091] B225. Obtain user viewing quality based on the effective quality, temporal quality loss, spatial quality loss, and corresponding preset weights.

[0092] Optionally, the user viewing quality QoP in formula (1) and formula (2) is t The calculation formula for (m,n) is:

[0093] QoP t (m,n)=β1·EQ t (m,n)-β2·TQ t (m,n)-β3·SQ t (m,n) (12)

[0094] It should be noted that the effective quality, temporal quality loss, spatial quality loss, and the corresponding preset weights may be the same or different. In the embodiment of the present application, the corresponding preset weight of the effective quality is a preset first weight β1, the corresponding preset weight of the temporal quality loss is a preset second weight β2, and the corresponding preset weight of the spatial quality loss is a preset third weight β3. In the embodiment of the present application, the user viewing quality corresponding to each video clip can be calculated separately using formula (12).

[0095] B226. Obtain an updated new utility value based on the difference between the user's viewing quality and the transmission loss at the current moment.

[0096] Specifically, the updated new utility value can be calculated by formula (1), that is, the updated newU t It should be noted that, in the embodiment of the present application, δ is taken as an example. When δ is not 1, when calculating the difference between the user viewing quality and the transmission loss at the current moment, the difference refers to the difference between the user viewing quality and the product of δ and the transmission loss at the current moment. The updated new utility value can be obtained according to the sum of the differences.

[0097] Optionally, in step B204, updating the quality level and bit rate of the video clip according to the reduction processing result, the new utility value, and the old utility value includes step B231 or B232:

[0098] B231. When the reduction processing result is greater than or equal to a preset threshold value, the new utility value is updated according to the reduction processing result, the second difference between the updated new utility value and the old utility value is calculated, the second impact factor of the video clip is determined, the video clip corresponding to the maximum value of the second impact factor is used as the second updated video clip, the quality level of the second updated video clip is updated to the reduction processing result corresponding to the second updated video clip, and the bit rate of the second updated video clip is updated according to the reduction processing result corresponding to the second updated video clip.

[0099] Optionally, the preset threshold can be set as needed. In the embodiment of the present application, the preset threshold is 1, which is the lowest quality level. Greater than or equal to 1, according to the reduction processing result The new utility value newU in step B201 t Update, and determine the second impact factor diff of the video clip based on the second difference between the updated new utility value and the old utility value t ′(m,n), the quality level corresponding to the maximum value of the second impact factor is updated to the reduction processing result, the video segment corresponding to the maximum value of the second impact factor is used as the second updated video segment, the quality level of the second updated video segment is updated to the reduction processing result corresponding to the second updated video segment, and then the bit rate of the second updated video segment is updated according to the reduction processing result corresponding to the second updated video segment. It should be noted that the second impact factor diff t ′(m,n) determination method and first impact factor diff t (m,n) is similar, by traversing the video clips according to the updated new utility value and the old utility value oldU t The difference between the two video clips is used to determine the second impact factor diff of each video clip. t ′(m,n), thus obtaining the new second impact factor matrix diff t ' M×n , the second impact factor matrix diff t ' M×N Each video clip corresponds to a second impact factor, which indicates the degree of change in the total utility of the video clip after the reduction process. The position index m,n of the maximum value is found = SearchMax(diff t ' M×NFor example, if m=M-3 and n=N-3, the video segment at position (M-3, N-3) in the video segment matrix is ​​used as the second updated video segment, and the quality level of the second updated video segment is updated to the reduced processing result corresponding to the second updated video segment. and updating the bit rate of the second updated video segment according to the reduction processing result corresponding to the second updated video segment, Or it can be expressed as: a new quality level matrix is ​​obtained according to the added processing result corresponding to the second updated video segment. Thus determining the updated code rate matrix Then return to step B202, where F() represents mapping.

[0100] B232: When the reduction processing result is less than the preset threshold, the quality level of the video segment is used as the updated quality level and the bit rate of the video segment is used as the updated bit rate.

[0101] In the embodiment of the present application, for example, the current video clip is the video clip of the Mth row and the Nth column. At this time, the quality level and bit rate of the video clip are kept unchanged, that is, the quality level of the video clip is used as the updated quality level, and the bit rate of the video clip is used as the updated bit rate. Then, the next video clip of the video clip in the Mth row and Nth column is traversed to determine the relationship between the reduction processing result and the quality level threshold.

[0102] In the embodiment of the present application, the detailed process of code rate allocation is as follows:

[0103]

[0104]

[0105] Among them, the final output / return The final quality level matrix is Utility() means to use formula (1) or (2) to calculate. It should be noted that when Execute lines 6 to 11 of the above code in the process of traversing the video clip, execute lines 12 to 14 after one traversal is completed, and then return to line 5 for judgment until the judgment in line 5 is completed. Output the final quality level matrix And when Execute lines 17 to 22 of the above code in the process of traversing the video clip, execute lines 23 to 25 after one traversal is completed, and then return to line 16 for judgment until the judgment in line 16 is completed. Output the final quality level matrix

[0106] Example 2

[0107] like Figure 2 As shown, the embodiment of the present application further provides a storage method, which includes at least but not limited to steps S300-S600:

[0108] S300: Obtain viewing probabilities and bit rate allocation results of all video segments in the target transmission video through a bit rate allocation method.

[0109] It should be noted that the bit rate allocation method refers to the bit rate allocation method in the embodiment, and the bit rate allocation result includes the target transmission bit rate of each video segment (m, n) and target quality level

[0110] S400: Predicting expected effective quality and expected transmission loss based on the bit rate allocation result and the viewing probability.

[0111] In the embodiment of the present application, when making storage decisions for edge nodes, it is assumed that time t is the current time. Since the effective quality and transmission loss of the video clip at time t+1 (the next time) are unknown, the bit rate allocation result at time t, the video clip, and the target quality level are used for prediction. and They represent the expected effective quality and expected transmission loss of the video segment (m,n), respectively, and predict the impact of the video segment stored at time t on the effective quality of the video segment at time t+1 and the transmission loss of obtaining the video segment from the cloud.

[0112] Optionally, step S400 includes steps B401-B403:

[0113] B401. Obtain the non-freshness at the current moment and the fourth stored decision value of the video clip at the current moment, and determine the predicted non-freshness based on the non-freshness at the current moment, the viewing probability, and the fourth stored decision value.

[0114] Specifically, predicting non-freshness The formula is:

[0115]

[0116] Among them, uf t (m,n) is the non-freshness at the current moment, c t (m,n) is the fourth storage decision value, α is the growth factor of non-freshness, p t (m, n) is the viewing probability. It should be noted that the fourth storage decision value refers to the storage decision value corresponding to the storage state of the video segment at the current moment, which is the same as the second storage decision value.

[0117] B402. Determine the expected effective quality based on the predicted non-freshness, viewing probability, and target quality level.

[0118] Specifically, the expected effective mass The formula is:

[0119]

[0120] Where μ is the effective mass factor, The target quality level.

[0121] B403. Determine the expected transmission loss based on the target transmission code rate, the fourth stored decision value, the second loss coefficient, and the third loss coefficient.

[0122] Specifically, the expected transmission loss The formula is:

[0123]

[0124] Among them, c2 is the second loss coefficient, c3 is the third loss coefficient, is the target transmission bit rate, c t (m,n) is the fourth storage decision value.

[0125] S500: Perform storage optimization processing based on expected effective quality, expected transmission loss, and maximum capacity of edge nodes to obtain storage decision information.

[0126] In the embodiment of the present application, the storage decision information includes a third storage decision value c for each video segment (m, n). t ′(m,n), and form the third storage decision value matrix c t ' M×N The third storage decision value indicates whether the video clip should be stored on the edge node. For example, a third storage decision value of 1 indicates that the video clip should be stored on the edge node, while a third storage decision value of 0 indicates that the video clip should not be stored on the edge node. It should be noted that the third storage decision value refers to the storage decision value corresponding to the video clip obtained through storage optimization processing.

[0127] Alternatively, the goal of the storage method is to improve the expected effective quality and reduce the expected transmission loss, so the storage optimization problem can be expressed as:

[0128]

[0129]

[0130]

[0131] Among them, β1 is the preset first weight, s t(m,n) is the size of the video segment (m,n), C max is the maximum storage capacity of the edge node, i.e., the maximum capacity, and δ is the weight of the transmission loss. It can be understood that by solving the as well as Then, the storage optimization process can be performed using formula (16). The constraint condition stipulates that the sum of the sizes of the stored video clips cannot exceed the storage capacity of the edge node, and the cache strategy to be solved for the storage problem is a binary variable matrix. In the embodiment of the present application, the third storage decision value is solved by a branch and bound algorithm.

[0132] Optionally, step B500 includes steps B511-B513:

[0133] B511. Determine the expected utility based on the expected effective quality and the expected transmission loss, determine the storage gain based on the expected utilities corresponding to the video clip being stored in the edge node and not being stored in the edge node at the current moment, and determine the video value of each video clip based on the storage gain.

[0134] Alternatively, the expected utility is There are two cases. The expected utility of the first case is recorded as the first expected utility DC t (m,n), that is, the expected utility of the video clip stored in the edge node at the current moment. The expected utility of the second case is recorded as the second expected utility DC t ′(m,n), that is, the expected utility of the video clip not being stored in the edge node at the current moment, specifically:

[0135] 1) When the video clip is stored in the edge node at time t (i.e. the current time), the fourth storage decision value c t (m,n)=1:

[0136]

[0137] Among them, μ is the effective quality factor, β1 is the preset first weight, δ is the weight of transmission loss, α is the growth factor of non-freshness, and c2 is the second loss coefficient.

[0138] 2) When the video clip is not stored in the edge node at time t (i.e., the current time), the fourth storage decision value c t (m,n)=0:

[0139]

[0140] Wherein, c3 is the third loss coefficient.

[0141] In the embodiment of the present application, the calculation formula of the video value is:

[0142]

[0143] Among them, v t (m,n) is the video value of the video segment (m,n), DC t (m,n)-DC′ t (m,n) is the storage gain. It should be noted that in the embodiment of the present application, the 0-1 knapsack problem is used to solve the optimal solution through the 0-1 branch and bound algorithm to obtain the storage decision information, so it is assumed that C max is the size of the backpack, and the video clips are equivalent to the items that need to be put into the backpack. The corresponding size and value are defined as w t (m,n) and v t (m,n), where w t (m,n)=s t (m,n), the goal of finding the optimal solution to obtain storage decision information is to make the size of the video clip less than or equal to C max And the sum of the values ​​reaches the maximum value as much as possible.

[0144] B512. Obtain the sizes of all video segments and determine the value density based on the sizes of the video segments and the storage gain.

[0145] In the embodiment of the present application, the value density ds t The (m,n) formula is:

[0146]

[0147] B513. Obtain a third storage decision value for each video segment through a branch and bound algorithm according to the video value and the value density.

[0148] In the embodiment of the present application, the third storage decision value calculated by the branch and bound algorithm can be the first value or the second value, for example, the first value is 1 and the second value is 0. Specifically: when the third storage decision value of the video clip is c t When (m,n) is 1, it means that the video clip needs to be stored in the edge node, and when the third storage decision value of the video clip is c t When (m,n) is 0, it means that the video clip does not need to be stored in the edge node

[0149] Specifically, the branch and bound algorithm actually generates a binary tree with multiple nodes. The processing flow is:

[0150] 1) Arrange the video clips according to their value density (e.g. from high to low: nodesList = upToLow(ds t (m,n))), through nodesList and C maxInitialize the upper bound value upprofit, so that upprofit is (nodesList, C max ) to obtain the initial upper bound.

[0151] 2) Initialize queue pq:

[0152] It should be noted that the initialization queue pq has a node initialized to 0 (referred to as the initial node). The data type of the node in the binary tree is a structure containing variables such as storage quality (such as weight, value, etc.) and assigned bit rate, and the storage quality of the initial node is 0. It is assigned to the parent node during initialization. In subsequent traversals, the head node of the queue pq is returned through pq.get() as the new parent node, the head node is deleted, and the remaining nodes in the queue pq are moved forward. The left and right nodes are initialized to 0 in each round of iteration. Each time it is determined whether the left and right nodes may have a solution, a new node will be added to the queue, and the parent node will be updated in turn.

[0153] 3) Iterative process:

[0154] Specifically, after initialization, the initial node is used as the parent node, and then the left node is determined based on the video clip with the largest value density in nodesList: a fictitious left node is generated, and the video clip with the largest value density is used as the to-be-confirmed clip. When the to-be-confirmed clip is added to the backpack, the total capacity is less than C. max , where the total capacity after adding the backpack refers to the sum of the sizes of all nodes on the branch where the fictitious left node in the binary tree is located. For example, the total capacity at this time is the sum of the size of the fragment to be confirmed and the size of the initial node. When the total capacity after adding the backpack is less than C max , at this time, the fictitious left node is determined to be the actual generated left node and the fourth storage decision value of the segment to be confirmed is marked as the first value, and the left node is added to the queue pq. At this time, the left node stores the size of the segment to be confirmed and the video value of the segment to be confirmed; then the left node is used to update the value threshold, that is, the upper limit value upprofit, and updated to generate a new value threshold upprofit. Among them, the upprofit value threshold represents the maximum video value that the current edge node can accommodate, and is the maximum value of the remaining nodes that can be loaded into the backpack calculated by the greedy algorithm during the solution of the 0-1 backpack problem. It should be noted that when the segment to be confirmed is added to the backpack, the total capacity is greater than or equal to C max , that is, the sum of the size of the fragment to be confirmed and the size of the initial node is greater than or equal to C max , at this time no actual left node is generated and the upper bound value upprofit is not updated.

[0155] Optionally, after judging the left node, the video clip with the largest value density is also used as the clip to be confirmed, and the clip to be confirmed is used to judge the right node: generate a fictitious right node, and judge whether the maximum utility value of the clip to be confirmed without being added to the backpack is greater than the updated upper limit value upprofit. If the maximum utility value of the clip to be confirmed without being added to the backpack is greater than the updated upper limit value upprofit, then the fictitious right node is determined to be the actual generated right node. At this time, the right node stores the size of the clip to be confirmed and the video value of the clip to be confirmed. Then, the fourth storage decision value of the clip to be confirmed is marked as the second value and the right node is added to the queue pq. If the maximum utility value of the clip to be confirmed without being added to the backpack is less than or equal to the updated upper limit value upprofit, then the actual right node is not generated. It should be noted that the maximum utility value refers to the sum of the video values ​​of all nodes on the branch where the fictitious right node is located in the binary tree. For example, in the above process, the maximum utility value is the sum of the video values ​​of the clip to be confirmed and the initial node.

[0156] It should be noted that in the embodiment of the present application, the queue uses the Python queue data structure Queue(pq), which follows a first-in-first-out order. pq.get() returns the head node of queue pq, deletes the head node, and then determines a new parent node from queue pq. In iterative process 3), the parent node is deleted, and the first node in the queue is used as the new parent node. The above-mentioned left node determination and left node determination steps are performed. Each time a segment to be confirmed is determined, the video segment that has not been used as a segment to be confirmed and has the highest value density is used as the new segment to be confirmed and the left node determination and left node determination steps are performed continuously. In the iterative process, when an actual left node is generated, the corresponding segment to be confirmed is marked with a first value, and when an actual right node is generated, the corresponding segment to be confirmed is marked with a second value. The iteration continues until the queue pq is empty, and the third stored decision value for each video segment is obtained.

[0157] In the embodiment of the present application, in order to make the storage gain non-negative, the storage decision value matrix is ​​updated, further comprising step B514:

[0158] B514. Update the third storage decision value corresponding to the video segment whose third storage decision value is the first value and whose storage gain is less than 0 to the second value to obtain storage decision information.

[0159] Specifically, the third stored decision values ​​of the M·N video clips obtained in step B513 are searched, and c t (m,n)=1 and storage gain DC t (m,n)-DC′t The third stored decision value c of the video segment where (m,n)<0 t (m,n) is updated to 0, and the other third storage decision values ​​remain unchanged, and the storage decision information is obtained, that is, the target storage matrix c t ' M×N .

[0160] S600: Store the video clip according to the storage decision information.

[0161] Specifically, the edge node can store the decision information according to the target storage matrix c t ' M×N The third storage decision value in determines the storage strategy. For example: the video clip with the third storage decision value of 1 is stored in the edge node, and the storage content of the edge node is updated for the service when the user terminal requests to obtain the target transmission video at the next moment, that is, in the future. In an embodiment of the present application, when the target transmission video obtained by the user terminal is a real-time video, the bit rate allocation method and storage method of the embodiment of the present application can respond to the request information of the user terminal in real time as an online algorithm, and the storage content of the edge node updated at the current moment can serve the user to obtain video clips in the future.

[0162] In the embodiment of the present application, the process of obtaining storage decision information in the storage method is as follows:

[0163]

[0164] Among them, when the judgment of line 7 is executed, if the queue is not empty, lines 8 to 15 are executed, and then the judgment of line 7 is returned until the queue is empty, and then lines 16 to 20 are executed, and the output is It should be noted that is the storage decision matrix at time t, return That is

[0165] The following describes the bit rate allocation method and storage method of the embodiment of the present application using two specific application scenarios:

[0166] like Figure 3As shown, when the target transmission video is an immersive video, the server performs video slicing, i.e., block processing, to obtain video clips, and then records the movement trajectory of the user's focus point; when the user sends a request through the user terminal, the server calculates the probability that the video clip is located in the user's F0V based on the movement trajectory of the user's focus point, obtains the viewing probability, and then performs bit rate allocation (refer to step S200 for details), and determines whether all video clips (i.e., video clips) have been sent to the user: if so, a storage decision is made, i.e., the above storage method is executed, thereby updating the video clips (i.e., video clips) stored in the edge node; if not, it is determined whether the video clip is stored in the edge node. If the video clip (i.e., video clip) is not stored in the edge node, the request needs to be forwarded to the cloud, so that the cloud server in the cloud sends the video clip to the edge node, and the edge node then sends the video clip (i.e., video clip) to the user terminal. If the video clip (i.e., video clip) is stored in the edge node, the user can read the content of the video clip (i.e., video clip) from the edge node.

[0167] like Figure 4 As shown, when the target transmission video is a video for a cloud desktop application scenario, the server performs video slicing, i.e., block processing, to obtain video segments, and then records the user's interaction trajectory with the cloud desktop (for example, the frequency of the operation area of ​​the mouse or touch screen); when the user sends a request through the user terminal, the server calculates the probability that the video segment is located in the user's ROI based on the cloud desktop interaction trajectory, obtains the viewing probability, and then performs bit rate allocation (see step S200 for details), and determines whether all video segments (i.e., video segments) have been sent to the user: if so, a storage decision is made, i.e., the above storage method is executed to update the video segments stored in the edge node; if not, it is determined whether the video segment (i.e., video segment) is stored in the edge node. If the video segment (i.e., video segment) is not stored in the edge node, it is necessary to forward the request to the cloud, so that the cloud server in the cloud sends the video segment (i.e., video segment) to the edge node, and the edge node then sends the video segment (i.e., video segment) to the user terminal. If the video segment (i.e., video segment) is stored in the edge node, the user can read the content of the video segment (i.e., video segment) from the edge node.

[0168] It should be noted that the bitrate allocation method in the embodiments of the present application is executed by a user-side terminal, and in other embodiments, it can also be executed by a cloud server in the cloud. The storage method can be executed by an edge node, or after being executed by a cloud server in the cloud, the edge node stores the video clips based on the storage decision information. The edge node refers to a network node in the Internet of Things that is close to the user and has caching capabilities.

[0169] like Figure 5 As shown, the embodiment of the present application further provides a code rate allocation device, including:

[0170] An acquisition module is used to acquire target video data; the target video data includes the viewing probability of all video segments in the target transmission video;

[0171] The processing module is used to allocate bit rates to video clips based on viewing probabilities so that the total utility of the target transmitted video meets the target value; the total utility of the target transmitted video is determined based on the user viewing quality of the video clips and the transmission loss, the user viewing quality is determined based on the non-freshness factor, and the non-freshness factor is determined based on the storage status of the video clips at the edge node and the viewing probability.

[0172] like Figure 6 As shown, the embodiment of the present application further provides a storage device, including:

[0173] A determination module, used to determine the viewing probability and bit rate allocation results of all video clips in the target transmission video;

[0174] A prediction module, configured to predict expected effective quality and expected transmission loss based on the bitrate allocation result and viewing probability;

[0175] an optimization module configured to perform storage optimization processing based on expected effective quality, expected transmission loss, and maximum capacity of the edge node to obtain storage decision information; the storage decision information includes a third storage decision value for each video segment, the third storage decision value indicating whether the video segment needs to be stored in the edge node;

[0176] A storage module, used for storing video clips according to storage decision information;

[0177] Obtain target video data; the target video data includes viewing probabilities of all video segments in the target transmission video;

[0178] The bitrate of video clips is allocated according to the viewing probability so that the total utility of the target transmitted video meets the target value; the total utility of the target transmitted video is determined by the user viewing quality of the video clip and the transmission loss. The user viewing quality is determined by the non-freshness factor, which is determined by the storage status of the video clip at the edge node and the viewing probability.

[0179] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0180] like Figure 7As shown, an embodiment of the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the bit rate allocation or storage method of the aforementioned embodiment. The electronic devices of the embodiment of the present application include, but are not limited to, mobile phones, tablet computers, computers, in-vehicle computers, servers, and the like.

[0181] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0182] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction, at least one program, code set or instruction set is stored. The at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the bit rate allocation or storage method of the aforementioned embodiment.

[0183] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the bit rate allocation or storage method of the aforementioned embodiment.

[0184] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0185] In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0186] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present application shall fall within the scope of the present application.

Claims

1. A code rate allocation method, characterized in that: include: Get target video data; The target video data includes viewing probabilities of all video segments in the target transmission video; The bit rate of the video clip is allocated according to the viewing probability so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user viewing quality and transmission loss of the video clip, the user viewing quality is determined according to the non-freshness, and the non-freshness is determined according to the storage status of the video clip at the edge node and the viewing probability.

2. The code rate allocation method according to claim 1, wherein: The allocating bit rates to the video segments according to the viewing probability includes: Obtaining a new utility value and an old utility value of the total utility of the target transmission video, and obtaining a quality level and a bit rate of the video segment; Determining whether the sum of the bit rates is less than or equal to the bandwidth capacity; When the sum of the bit rates is less than or equal to the bandwidth capacity, the quality level is increased by traversing the video segments, and the quality level and bit rate of the video segments are updated according to the increase processing result, the new utility value, and the old utility value. The process returns to the step of determining whether the sum of the bit rates is less than or equal to the bandwidth capacity, until the sum of the bit rates is greater than the bandwidth capacity, thereby obtaining a target transmission bit rate for each video segment; the increase processing result is the increased quality level; When the sum of the bit rates is not less than or equal to the bandwidth capacity, the quality level is reduced by traversing the video segments, and the quality level and bit rate of the video segments are updated according to the reduction processing result, the new utility value, and the old utility value. The process returns to the step of determining whether the sum of the bit rates is less than or equal to the bandwidth capacity until the sum of the bit rates is less than or equal to the bandwidth capacity, thereby obtaining a target transmission bit rate for each of the video segments; the reduction processing result is the reduced quality level.

3. The code rate allocation method according to claim 2, wherein: The updating of the quality level and the bit rate of the video segment according to the addition processing result, the new utility value and the old utility value includes: When the added processing result is less than or equal to the quality level threshold, the new utility value is updated according to the added processing result, the first difference between the updated new utility value and the old utility value is calculated, the first impact factor of the video clip is determined, the video clip corresponding to the maximum value of the first impact factor is used as the first updated video clip, the quality level of the first updated video clip is updated to the added processing result corresponding to the first updated video clip, and the bit rate of the first updated video clip is updated according to the added processing result corresponding to the first updated video clip.

4. The code rate allocation method according to claim 3, wherein: The updating of the new utility value according to the increase processing result includes: Calculate the transmission loss at the current moment and calculate the non-freshness at the current moment; determining effective quality according to the non-freshness at the current moment, the viewing probability, and the quality level at the current moment; Obtaining a quality level of the video clip at a previous moment, calculating a first subjective quality at the current moment and a second subjective quality at the previous moment based on the quality level at the current moment, the viewing probability, and the quality level at the previous moment, and determining a temporal quality loss based on a difference between the first subjective quality and the second subjective quality; calculating a subjective quality mean according to the first subjective quality and the total number of the video clips, and determining a spatial quality loss according to a difference between the first subjective quality and the subjective quality mean; Obtaining user viewing quality according to the effective quality, the temporal quality loss, the spatial quality loss, and corresponding preset weights; An updated new utility value is obtained according to the difference between the user viewing quality and the transmission loss at the current moment.

5. The code rate allocation method according to claim 4, wherein: The calculating of the transmission loss at the current moment includes: determining a first storage decision value according to a storage state of the video clip at the edge node at the previous moment; When the first storage decision value indicates that the video clip is stored at an edge node, obtaining a storage bit rate of the video clip, calculating a bit rate difference between the storage bit rate and the bit rate of the video clip, determining a first loss based on the first storage decision value, the bit rate difference, and a first loss coefficient, determining a second loss based on a second loss coefficient and the bit rate of the video clip, and determining the transmission loss at the current moment based on a sum of the first loss and the second loss; Alternatively, when the first storage decision value indicates that the video clip is not stored in the edge node, a third loss is determined based on a third loss coefficient and a bit rate of the video clip, a second loss is determined based on the second loss coefficient and the bit rate of the video clip, and the transmission loss at the current moment is determined based on the sum of the third loss and the second loss; Among them, the first loss is the transcoding loss of the edge node, the second loss is the communication loss of the edge node transmitting the video clip to the user end, and the third loss is the communication loss of the edge node obtaining the video clip from the cloud.

6. The code rate allocation method according to claim 4, wherein: The calculation of the non-freshness at the current moment includes: Determine the non-freshness at the previous moment; determining a second storage decision value according to a storage state of the video clip at the edge node at the current moment; When the second storage decision value indicates that the video clip is stored at an edge node, calculating a product of a preset freshness growth factor and the viewing probability, and determining the non-freshness at a current moment based on a sum of the product and the non-freshness at a previous moment; Alternatively, when the second storage decision value indicates that the video clip is not stored in the edge node, the non-freshness at the current moment is determined to be 0.

7. The code rate allocation method according to claim 2, wherein: The updating of the quality level and the bit rate of the video segment according to the reduction processing result, the new utility value, and the old utility value includes: When the reduction processing result is greater than or equal to a preset threshold, the new utility value is updated according to the reduction processing result, a second difference between the updated new utility value and the old utility value is calculated, the second impact factor of the video clip is determined, the video clip corresponding to the maximum value of the second impact factor is used as the second updated video clip, the quality level of the second updated video clip is updated to the reduction processing result corresponding to the second updated video clip, and the bit rate of the second updated video clip is updated according to the reduction processing result corresponding to the second updated video clip.

8. A storage method, characterized in that: include: Determine the viewing probability and bitrate allocation results of all video clips in the target transmission video; predicting expected effective quality and expected transmission loss based on the bit rate allocation result and the viewing probability; performing storage optimization processing based on the expected effective quality, the expected transmission loss, and the maximum capacity of the edge node to obtain storage decision information; the storage decision information includes a third storage decision value for each of the video clips, the third storage decision value indicating whether the video clip needs to be stored in the edge node; storing the video clip according to the storage decision information; The step of determining the viewing probability and bitrate allocation results of all video segments in the target transmission video specifically includes: Obtaining target video data; the target video data includes viewing probabilities of all video segments in the target transmission video; The bit rate of the video clip is allocated according to the viewing probability so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user viewing quality and transmission loss of the video clip, the user viewing quality is determined according to the non-freshness, and the non-freshness is determined according to the storage status of the video clip at the edge node and the viewing probability.

9. The storage method according to claim 8, characterized in that: The bitrate allocation result includes a target transmission bitrate for each of the video segments, each target transmission bitrate having a corresponding target quality level, and predicting an expected effective quality and an expected transmission loss based on the bitrate allocation result and the viewing probability, including: Obtaining the non-freshness at a current moment and a fourth stored decision value of the video clip at the current moment, and determining a predicted non-freshness based on the non-freshness at the current moment, the viewing probability, and the fourth stored decision value; determining the expected effective quality according to the predicted non-freshness, the viewing probability, and the target quality level; determining the expected transmission loss according to the target transmission code rate, the fourth stored decision value, the second loss coefficient, and the third loss coefficient; The second loss is the communication loss when the edge node transmits the video clip to the user end, and the third loss is the communication loss when the edge node obtains the video clip from the cloud.

10. The storage method according to claim 8, characterized in that: The bitrate allocation result includes a target transmission bitrate for each of the video clips, each of the target transmission bitrates having a corresponding target quality level. The storage optimization process is performed based on the expected effective quality, the expected transmission loss, and the maximum capacity of the edge node to obtain storage decision information, including: Determining an expected utility based on the expected effective quality and the expected transmission loss, determining a storage gain based on the expected utilities corresponding to the video clip being stored at the edge node and not being stored at the edge node at the current moment, and determining a video value of each video clip based on the storage gain; Obtaining the size of the video segment, and determining the value density of the video segment according to the size of the video segment and the storage gain; A third storage decision value of each video clip is obtained through a branch and bound algorithm based on the video value and the value density; the third storage decision value is a first value or a second value, the first value indicates that the video clip needs to be stored in the edge node, and the second value indicates that the video clip does not need to be stored in the edge node.

11. The storage method according to claim 10, characterized in that: The method further comprises: The third storage decision value corresponding to the video segment whose third storage decision value is the first value and whose storage gain is less than 0 is updated to the second value to obtain storage decision information.

12. A code rate allocation device, characterized in that: include: An acquisition module, used to acquire target video data; The target video data includes viewing probabilities of all video segments in the target transmission video; A processing module is used to allocate bit rates to the video segments according to the viewing probability so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined based on the user viewing quality and transmission loss of the video segment, the user viewing quality is determined based on the non-freshness, and the non-freshness is determined based on the storage status of the video segment at the edge node and the viewing probability.

13. A storage device, characterized in that: include: A determination module, used to determine the viewing probability and bit rate allocation results of all video clips in the target transmission video; A prediction module, configured to predict expected effective quality and expected transmission loss based on the bit rate allocation result and the viewing probability; an optimization module, configured to perform storage optimization processing based on the expected effective quality, the expected transmission loss, and the maximum capacity of the edge node to obtain storage decision information; the storage decision information includes a third storage decision value for each of the video clips, the third storage decision value indicating whether the video clip needs to be stored in the edge node; a storage module, configured to store the video clip according to the storage decision information; Determining the viewing probability and bit rate allocation results of all video segments in the target transmission video specifically includes: Obtaining target video data; the target video data includes viewing probabilities of all video segments in the target transmission video; The bit rate of the video clip is allocated according to the viewing probability so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user viewing quality and transmission loss of the video clip, the user viewing quality is determined according to the non-freshness, and the non-freshness is determined according to the storage status of the video clip at the edge node and the viewing probability.

14. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the bit rate allocation method according to any one of claims 1 to 7 or the storage method according to any one of claims 8 to 11.

15. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the bit rate allocation method according to any one of claims 1 to 7 or the storage method according to any one of claims 8 to 11.

Images