Unified video quality score assessment using fuzzy logic

Abstract

The present invention provides a method for using fuzzy logic to evaluate service quality metrics based on content delivery and to provide a unified quality score that quantifies perceived quality (QoE), as well as a non-temporary computer-readable storage medium and device. [Solution] The method receives multiple metric values ​​for service quality based on delivered content in a content delivery system, converts each metric value into a fuzzy value from among multiple feasible fuzzy values ​​to determine multiple fuzzy values, applies the fuzzy rule to the fuzzy values ​​when each condition for the fuzzy rule is met by one or more of the multiple fuzzy values, and calculates a unified quality score from the fuzzy output.

Description

[[Background Art]]

[0001]

[0001] Many metrics may be used to evaluate streaming performance. Current approaches have limitations. For example, a single quality of service (QoS) metric such as average bit rate may be used to evaluate video quality. However, this metric may not accurately evaluate perceived quality. Furthermore, different methods of aggregating several quality of service metrics into a score may fail to capture the perceived quality experienced by viewers. Previous solutions have often produced discrete or binary results, which can lead to considerable score fluctuations near threshold values and potential errors. Furthermore, these methods assume a linear relationship between all influencing factors and the final result, and therefore may result in inaccurate evaluation in some cases. These scores may not accurately reflect subtle changes in quality of service metrics, and thus may lead to an unreliable assessment of overall streaming quality that will be experienced by viewers. [[Summary of the Invention]]

[0002]

[0002] The accompanying drawings are for illustrative purposes and serve only to provide examples of feasible structures and operations for the disclosed systems, apparatuses, methods, and computer program products of the present invention. These drawings are not intended to limit in any way any changes to form and detail that may be made by those skilled in the art without departing from the spirit and scope of the disclosed implementations. [[Brief Description of the Drawings]]

[0003] [Figure 1]

[0003] FIG. 1 is a diagram depicting a simplified system for generating a unified quality score in accordance with some embodiments. [Figure 2]

[0004] FIG. 2 is a diagram depicting a more detailed example of a metric generation system in accordance with some embodiments. [Figure 3]

[0005] A diagram illustrating a simplified flowchart of a method for generating a unified quality score, according to several embodiments. [Figure 4]

[0006] A diagram illustrating examples of membership functions in several embodiments. [Figure 5]

[0007] A diagram illustrating examples of adjustments based on a unified quality score, according to several embodiments. [Figure 6]

[0008] A diagram showing an example of a computing device according to several embodiments. [Modes for carrying out the invention]

[0004]

[0009] This specification describes techniques for content delivery systems. The following description includes numerous examples and specific details to provide a complete understanding of several embodiments for illustrative purposes. As defined by the claims, some embodiments may include some or all of the features described in these examples alone or in combination with other features described below, and may also include modifications and equivalents of the features and concepts described herein.

[0005]

[0010] System Overview

[0011] The system uses fuzzy logic to evaluate quality of service metrics based on content delivery and provides a unified quality score that quantifies perceived quality (QoE). Perceived quality, observed by both individual viewers and experienced evaluators, has an intuitive but ambiguous standard for evaluating overall streaming quality. For example, a session with high clarity might be considered acceptable even if it contains slight stuttering, while a session with smooth, high-quality playback is generally considered a good experience. The system bridges the gap between subjective user perception and objective measurement by leveraging fuzzy logic that captures common observations and experiences to translate ambiguous judgments of quality of service (QoS) metrics into precise numerical evaluations.

[0006] The system integrates various quality of service metrics, such as video startup time, rebuffer ratio, rebuffer count, and average bit transmission rate, into a unified quality score. The unified quality score provides a single, comprehensive measure of perceived quality. The unified quality score enables system identification and dynamic adjustments to improve the overall user experience. Directly analyzing all quality of service metrics to evaluate perceived quality can be difficult because quality of service metrics are numerous, dynamic, and fluctuate over time. Furthermore, trade-offs between metrics make it difficult to evaluate overall performance—for example, a higher bit transmission rate may improve video quality, but it may also lead to more rebuffering. Monitoring all metrics in real time and identifying problems becomes extremely complex. The unified quality score addresses this by aggregating quality of service metrics using defined logic and balanced trade-offs, providing a single, comprehensive measure of overall perceived quality. This simplifies monitoring, and trends in the unified score can indicate when and where adjustments are needed. The unified quality score can be monitored to obtain trends such as a downward trend, and then used to identify problems such as encoding issues or network distribution issues.

[0007]

[0012] Solutions based on fuzzy logic differ from previous methods in their ability to represent and process service quality metrics in a more nuanced and continuous manner, thus enabling more human-like reasoning and decision-making. This results in more accurate and reliable assessments of perceived quality, which can easily be adapted to different service quality metrics and delivery conditions without requiring extensive computing resources or significant modifications to existing frameworks. Fuzzy logic may use fewer computing resources compared to processing all service quality metrics using rules.

[0008]

[0013] The Unified Quality Score offers numerous advantages. For example, it provides a continuous and accurate assessment of perceived quality. Furthermore, the Unified Quality Score has improved scalability and adaptability to different service quality metrics and delivery conditions. Due to its continuous nature compared to binary results, the Unified Quality Score has an enhanced ability to identify areas for adjustment and improvement to optimize the user experience. In addition, the Unified Quality Score reduces subjectivity and complexity in assessing perceived quality by using fuzzy logic instead of subjective human observation.

[0009]

[0014] In some embodiments, the system includes a fuzzifier system, a fuzzy rule system, a defuzzifier system, and an adjustment system. The fuzzifier system can convert a number of service quality metric values ​​into fuzzy values ​​that represent human thought. The output of the fuzzifier may be a fuzzy value. The fuzzy rule system can receive fuzzy values ​​and apply fuzzy rules to them. The fuzzy rule system generates a fuzzy output. The defuzzifier system then converts the fuzzy output into a unified quality score. The unified quality score may be a limited number or category. The adjustment system can then use the unified quality score to make adjustments to the system.

[0010]

[0015] system

[0016] Figure 1 shows a simplified system 100 for generating a unified quality score in several embodiments. System 100 includes a server system 102 and client devices 104. A single example of server system 102 is shown, but multiple examples of server system 102 may also be accepted. For example, multiple client devices 104 may request content from a single server system 102 or multiple server systems 102.

[0011]

[0017] The server system 102 includes a content management system 106 that can facilitate the distribution of content to client devices 104. For example, the content management system 106 can communicate with a content distribution network (not shown) to distribute content to a number of client devices 104. The content distribution network includes servers that can distribute content to the client devices 104. The content may be video, audio, or other types of content. Video may be used for discussion purposes, but other types of content may be used instead of video. In some embodiments, the content distribution network distributes a segment of video to the client devices 104. The segment may be a portion of a video, such as a 6-second video. The video may be encoded in a number of profile levels, corresponding to different levels that may be different levels of bit transmission rate or quality (e.g., resolution). The client device 104 can request a segment of video in one of the profile levels based on current network conditions. For example, the client device 104 may use an adaptive bit transmission rate algorithm to select a profile level for the video based on estimated current available bandwidth and other network conditions.

[0012]

[0018] The client device 104 may include a mobile phone, smartphone, set-top box, television, living room device, tablet device, or other computing device. The client device 104 may also include a media player 112 displayed on interface 110. The media player 112 or the client device 104 can request content from the content distribution network.

[0013]

[0019] The metric generation system 108 receives feedback from the delivery of content to the client device 104. From the feedback, the metric generation system 108 determines quality of service metrics such as video startup time, rebuffer ratio / rebuffer count, and average bit transmission rate. The metric generation system 108 uses the input to evaluate perceived quality. Perceived quality may be an evaluation of the perceived quality that the user experiences when the content is delivered to the client device 104. Quality of service may be metrics based on the delivery of content to the client device 104.

[0014]

[0020] The metric generation system 108 can use fuzzy logic to convert service quality metrics and overall viewing quality assessments into a unified streaming quality assessment metric score (called a unified quality score). For example, fuzzy logic can be used to adapt service quality metrics to ambiguous, human-like standards for evaluating overall perceived quality, and then convert them into numerical assessments of the unified quality score. The unified quality score may differ from previous solutions that relied on rule-based methods or mathematical models using linear weighting. The use of fuzzy logic to represent and process service quality metrics allows for a more nuanced and continuous evaluation of perceived quality based on human-like reasoning. This can result in a unified quality score that may be more practical to use for adjusting the system when delivering content. As mentioned above, service quality metrics are dynamic and fluctuate over time, making it difficult to directly evaluate overall system performance. Furthermore, there are inherent trade-offs between metrics; for example, while improved bit transmission speed may improve video quality, it may also lead to more rebuffering. When one metric improves while another deteriorates, it becomes difficult to determine whether the system is functioning correctly or encountering problems. A unified quality score is a unified metric that provides a comprehensive and balanced representation of perceived quality. It simplifies the complexity of monitoring numerous fluctuating service quality metrics and eliminates trade-offs between them.

[0015]

[0021] In some embodiments, the metric generation system 108 can use quality of service metrics for multiple client devices 104 to generate a unified quality score. However, the unified quality score may also be generated for a single client device 104.

[0016]

[0022] Here, we will describe fuzzy logic in more detail.

[0017]

[0023] Fuzzy logic system

[0024] Figure 2 shows a more detailed example of the metric generation system 108 according to several embodiments. The fuzzy maker system 202 receives quality of service metrics as input. The quality of service metrics may describe metrics based on the delivery of content to one or more client devices 104. The following quality of service metrics may be used, but other metrics may also be accepted. For example, quality of service metrics may include rebuffer metrics, video quality metrics, response time metrics, video start failure metrics, video playback failure metrics, or other metrics.

[0018]

[0025] Rebuffering can have a significant impact on perceived quality from different perspectives. For example, the frequency and duration of buffering events can affect perceived quality. Frequent rebuffering disrupts the continuous playback of content, thereby causing interruptions in the viewing experience. Since users expect smooth, uninterrupted content delivery, frequent rebuffering events can lead to frustration and complaints. The duration of rebuffering events directly and strongly impacts perceived quality. Short pauses may be acceptable, but longer interruptions can disrupt the flow of content delivery and viewing, reducing user satisfaction. Rebuffering-related metrics include rebuffering frequency, rebuffering count, and rebuffering duration.

[0019]

[0026] It is important to evaluate the impact of video quality on perceived quality. Different video quality assessment methods may be used, such as peak signal to noise ratio (PSNR), structural similarity index measure (SSIM), and video multi-method assessment fusion (VMAF).

[0020]

[0027] Different metrics for response time may also be additionally used. Video startup time (VST) may be an initial loading time, and may refer to the time taken for content to start after a user initiates playback. A shorter video startup time creates a positive first impression, while a longer video startup time may lead to user frustration and, in some cases, abandonment of the stream. Interaction delay refers to the speed of response to user interactions such as pausing, seeking, or changing settings, and enhances the user's sense of control and satisfaction. Delay in these responses can disrupt the viewing experience and lead to negative perception of the responsiveness of the service.

[0021]

[0028] Video start failure measures the number of failures that occur when starting to play back content. In addition, video playback failure measures the number of failures that occur during playback of content.

[0022]

[0029] The fuzzifier system 202 is capable of converting complex unknown mathematical problems into fuzzy human-like thinking. Unlike binary logic that converts inputs to either "0" or "1", fuzzy logic represents transition states between input values using degrees of membership. The fuzzifier system 202 converts inputs into outputs of fuzzy language. A fuzzy language or membership function may be used to determine the output.

[0023]

[0030] Fuzzy language refers to linguistic terminology that describes variable states in a qualitative manner rather than using precise numerical values. In fuzzy logic, input variables are categorized into different fuzzy sets, such as "low," "medium," or "high," instead of using exact numbers. For example, instead of stating that the temperature is exactly 30°C, fuzzy language would describe it as "warm" or "slightly hot." This approach reflects human reasoning, and judgments are often made using approximate terminology. Human reasoning can translate service quality metrics into fuzzy values ​​that approximate human judgments of perceived quality.

[0024]

[0031] The membership function defines how each input or output value is mapped to a degree of belonging within the fuzzy set, ranging from 0 to 1. The degree of belonging represents the extent to which a particular value belongs to the fuzzy set. For example, in the case of temperature, a value of 30°C might have a degree of belonging of 0.7 in the "warm" fuzzy set and a degree of belonging of 0.3 in the "hot" fuzzy set.

[0025]

[0032] The fuzzy rule system 204 receives fuzzy values ​​from the fuzzifier system 202. The fuzzy rule system 204 can apply fuzzy rules to the fuzzy values ​​to generate a fuzzy output that includes a degree of belonging. The fuzzy rules may be defined differently depending on the evaluation of the user experience; that is, the fuzzy rules may attempt to capture the user's evaluation of the quality experience. In some embodiments, the fuzzy rules may be in the form of a condition, such as "if A then B". The fuzzy rules may be applied when the conditions of the fuzzy rules are met. Further other formats of fuzzy rules may be used, such as the fuzzy rules incorporating other operators, such as Boolean operators, to combine multiple conditions. The fuzzy rules can consider the fuzzy values ​​and determine a fuzzy output that includes a degree of belonging based on each rule applied to the fuzzy values. In some embodiments, the fuzzy output may be a number of fuzzy values, each including a degree of belonging. The fuzzy rule system 204 then outputs the fuzzy output. The determination of the fuzzy output will be described in more detail below.

[0026]

[0033] The defugifier system 206 converts the fuzzy output into a unified quality score. For further processing, the defugifier system 206 maps the fuzzy output to a limited numerical value or category. In some embodiments, the numerical value may be a continuous value within a range such as 0 to 100. Alternatively, the defugifier system 206 may map the output to a specific category such as "System Running Well" or "Issue Detected" in order to grade the fuzzy output. The defugifier system 206 may use a continuous value to determine one of the categories or to determine attribution within a category. In contrast to binary values ​​of zero and one, continuous values ​​can provide more insight into perceived quality. For example, instead of zero or one, the unified quality score may be 20, 30, or 40 to define different levels of granularity for lower perceived quality. Furthermore, the unified quality score may be 70, 80, or 90 to define different levels of granularity for higher perceived quality. In contrast, uniform quality scores of 20, 30, and 40 may be graded using a value of 0, and uniform quality scores of 70, 80, and 90 may be graded using a value of 1.

[0027]

[0034] The adjustment system 208 can receive a unified quality score and make adjustments to the delivery process. For example, adjustments may be made to the delivery of content over the content delivery network, to the encoding process, or to playback on the client device 104, as described below. In some embodiments, the adjustments may be made dynamically after the unified quality score has been received. These adjustments can improve the delivery of content. Furthermore, adjustments may be made with a finer level of granularity compared to receiving a binary value of zero or one. For example, adjustments with different levels of granularity may be made based on the unified quality score, compared to only two adjustments when the output is zero or one.

[0028]

[0035] The following describes in more detail an example of generating a unified quality score. Figure 3 shows a simplified flowchart 300 of a method for generating a unified quality score according to several embodiments. In 302, the fuzzy maker 202 receives quality of service metrics for video distribution. In some embodiments, the quality of service metrics may be scores for different quality of service metrics received from the content distributed to the client device 104. In some embodiments, the quality of service metrics may be graded into categories such as rebuffer, video quality, and response time. For example, rebuffer-related metrics may include rebuffer frequency, rebuffer count, and rebuffer duration. Video quality-related metrics may include PSNR, SSIM, and VMAF. The metrics within each category may then be aggregated into a single score on a range such as 0 to 100. By using a single score, processing efficiency can be improved because inputting all quality of service metrics into fuzzy logic can cause a rule explosion in the fuzzy logic. The metric generation system 108 aggregates the input of multiple dimensions into a unified quality score. To reduce the fuzzy logic required, service quality metrics may be aggregated into a single score for each category, and then the category scores may be aggregated into a unified quality score. In other embodiments, aggregation into categories may not be performed. Aggregation of service quality metrics into category scores may be performed in different ways, such as using fuzzy logic, weighted averaging, averaging, or other aggregation methods.

[0029]

[0036] In 304, the fuzzifier 202 determines fuzzy values ​​based on quality of service metrics using a membership function. Table 1 shows examples of fuzzy languages ​​in several embodiments.

[0030] [Table 1]

[0031]

[0037] The inputs in Table 1 include the rebuffer score and the video quality score. The output may be a fuzzy output. The fuzzy value may be low, medium, or high, but other values ​​may be used. The fuzzy value does not have to be the same for all inputs or outputs; rather, different inputs and outputs may have different fuzzy values. The attribution score may indicate the percentage of inputs to which each fuzzy value belongs. For example, a rebuffer score of 65 may have an attribution score of 0.0 for low, 0.875 for medium, and 0.125 for high.

[0032]

[0038] In 306, the fuzzy rule system 204 applies fuzzy rules to fuzzy values ​​to determine the fuzzy output. Table 2 shows examples of rules according to several embodiments.

[0033] [Table 2]

[0034]

[0039] Rules R1, R2, and R3 describe different conditions that must be met for each rule to be applied. In some embodiments, one or more rules may be applied depending on whether the conditions are met. Furthermore, only one rule may be applied to the fuzzy value. In this example, the rule may be A then B, where A is one or more conditions and B is a fuzzy output (for example, one of the fuzzy values). The conditions in this example may include Boolean operators that combine fuzzy values ​​from different inputs. A degree of belonging may be used to evaluate the rules, as will be discussed below.

[0035]

[0040] In 308, the defurifier 206 converts the fuzzy output into a unified quality score. Different methods of defuzzying may be used. For example, the defurifier 206 uses a method that combines the rules to be applied with a degree of belonging to determine the unified quality score. In some embodiments, the defurifier 206 uses a centroid method to calculate the centroid of the fuzzy output using a degree of belonging based on the shape of the fuzzy output, and uses this as the defuzzy output. The centroid is calculated as a weighted average of the area between the fuzzy output curve and the horizontal axis. Furthermore, the defurifier 206 uses a maximum value method to select the x-coordinate of the highest point on the fuzzy output curve as the defuzzy output of the unfuzzy quality score. Furthermore, the defurifier 206 uses a weighted average method to calculate a weighted average as the defuzzy output of the unfuzzy output by multiplying each point on the fuzzy output curve by its corresponding weight. The weights may be determined based on the requirements of a particular application.

[0036]

[0041] example

[0042] An example will be provided below. For the purpose of discussing the fuzzy logic used to generate a unified quality score, the rebuffer score and video quality score can be used as inputs. However, other quality of service metric scores may be used. The inputs could be a rebuffer score of 65 and a video quality score of 65. The fuzzy generator system 202 determines the assignment of fuzzy values ​​for each input. For example, Table 3 shows the degree of assignment for fuzzy values.

[0037] [Table 3]

[0038]

[0043] The degrees of belonging to the rebuffer score and video quality score are [0, 0.875, 0.125] for low, medium, and high fuzzy values, respectively. A membership function may be used to determine the degrees of belonging. Figure 4 shows an example of a membership function according to several embodiments. Each input may have a different membership function. In this example, both inputs have the same membership function, but this example is not limited to using the same membership function, and each input may have a different associated membership function.

[0039]

[0044] The Y-axis represents the degree of belonging, and the X-axis represents the input score. Fuzzy values ​​can be represented by lines in the graph membership function. For example, in 402, the membership function for low fuzzy values ​​is shown as a dashed line, in 404 as a solid line, and in 406 as a dashed and dotted line for high fuzzy values. Depending on the input value, the fuzzy values ​​may have different belonging values ​​for each fuzzy value. For example, the value of 65 corresponds to a value of 0.125 in 408 for high fuzzy values ​​and a value of 0.875 in 410 for medium fuzzy values. The low fuzzy value membership function is zero for the value of 65. This results in the values ​​shown in Table 3.

[0040]

[0045] The fuzzy rule system 204 uses a degree of belonging to determine the fuzzy output. Table 4 shows the calculation of the fuzzy output's belonging.

[0041] [Table 4]

[0042]

[0046] Based on the degree of attribution for each input, rules 1 and 2 from Table 2 are satisfied. Rule 1 includes the conditions that the rebuffer score is high AND the video quality score is high. Rule 2 includes the conditions that the rebuffer score is high AND the video quality score is medium OR the rebuffer score is medium AND the video quality score is high OR the rebuffer score is medium AND the video quality score is medium. Rule 3 does not apply because the rebuffer score is not low and the video quality score is not low. The next step is to calculate the intensity of each fuzzy rule. If the links in the metrics in the rule conditions are "AND", the minimum value method is used. If the links are "OR", the maximum value method is used. However, other evaluation methods may be used. As a result, for high fuzzy values, Rule 1 yields a minimum value of 0.125 and for medium fuzzy values ​​a maximum value of 0.875. The fuzzy output is 0.125 for high for Rule 1 and 0.875 for medium for Rule 2.

[0043]

[0047] Next, the defagifier system 206 converts the fuzzy output into a unified quality score. Different methods may be used. For example, using the centroid method, the unified quality score may be 50.25. The centroid method involves calculating a weighted average of the output values ​​based on their degree of belonging, and then normalizing the result to obtain a numerical value within a range of achievable output values.

[0044]

[0048] Once a unified quality score is determined, adjustments may be made for different systems.

[0045]

[0049] adjustment

[0050] Figure 5 illustrates examples of adjustments based on a unified quality score according to several embodiments. As described above, the service quality metric is received by the metric generation system 108, which generates a unified quality score. The adjustment system 208 then uses the unified quality score to perform the adjustments.

[0046]

[0051] The following describes some possible adjustments, but other adjustments may also be permitted. For example, adjustments may be made to the Content Delivery Network (CDN) system 502-1, the encoder system 502-2, the media player system 502-3, or other systems 502-4. The Content Delivery Network system 502-1 can adjust its content delivery strategy using a unified quality score. For example, the adjustment system 208 may adjust to use higher-performing servers or change the caching strategy based on the unified quality score. In some cases, servers associated with a better unified quality score may be used more frequently to deliver content. Furthermore, servers with a lower unified quality score may have more additional cached content to reduce content retrieval time. In some cases, the unified quality score indicates reduced playback quality in certain geographical areas due to high latency from the Content Delivery Network servers. The adjustment system 208 can group the Content Delivery Network by region and analyze the quality of service metrics for the groups. The adjustment system 208 automatically redirects client devices within its area to a less congested content delivery network server. This improves content delivery by reducing latency and buffering, thereby improving the overall perceived quality.

[0047]

[0052] The adjustment system 208 can use a unified quality score to adjust the encoding of content in the encoder system 502-2. For example, the adjustment system 208 can optimize encoding parameters such as bit transmission rate, resolution, or codec settings to maintain a balance between quality and resource usage. For instance, the unified quality score might indicate that the prevalence of 4K and high dynamic range (HDR) playback is low in certain areas. The encoding bit transmission rate for 4K and HDR content might be set too high for the average available network bandwidth in those areas. For example, if the unified quality score is consistently lower in certain areas where 4K and HDR content is available, the adjustment system 208 can analyze quality-of-service metrics such as bit transmission rate, rebuffering, and session completion rate to determine whether high bit transmission rate 4K and HDR content is resulting in an unsatisfactory playback experience due to bandwidth limitations.

[0048]

[0053] The adjustment system 208 can adjust encoder settings to include supplemental 4K and HDR content streams with lower bit transmission rates as an additional option. This ensures that more users in this area can access and view 4K and HDR content, thereby improving perceived quality without requiring a significant increase in bandwidth.

[0049]

[0054] The adjustment system 208 can adjust the media player settings in the media player system 502-3 using a unified quality score. For example, the adaptive bit rate algorithm may be adjusted to improve playback smoothness and responsiveness. For instance, a low unified quality score may cause the adaptive bit rate algorithm to be less aggressive when switching to a higher bit rate. Conversely, a higher unified quality score may cause the adaptive bit rate algorithm to become more aggressive, allowing it to switch to a higher bit rate profile.

[0050]

[0055] After adjustment, the service quality metrics are received by the real-time optimization system 504, which provides the service quality metrics as feedback to the metric generation system 108 so that real-time adjustments can be made. For example, feedback used to determine new service quality metrics may be received. An evaluation is then performed to determine a new unified quality score, and the adjustment system 208 determines adjustments to one or more systems 502-1 through 502-4. Thus, dynamic adjustments to the systems can be made.

[0051]

[0056] conclusion

[0057] The unified quality score may be generated based on a more nuanced, human-like assessment of service quality metrics. For example, instead of a strict threshold for video startup time, the unified quality score may use fuzzy settings to weight and interpret boundary cases more flexibly, thereby providing a unified quality score that better reflects the overall user experience. The unified quality score can capture coarse differences in playback quality across multiple service quality dimensions. A unified quality score using fuzzy logic provides a more accurate and intuitive reflection of the user experience.

[0052]

[0058] Therefore, the metric generation system 108 uses fuzzy logic to approximate user judgments to a unified quality score. The use of fuzzy logic allows for approximation of user judgments regarding experience quality, resulting in a more accurate unified quality score. For example, the unified quality score provides a continuous output. Unlike binary or discrete step-change evaluations, the unified quality score provides a continuous quality score, ensuring that small improvements or deteriorations in service quality metrics are accurately reflected in the unified quality score. Furthermore, the unified quality score differs from previous solutions that relied on rule-based methods or mathematical models using linear weighting. The use of fuzzy logic to represent and process service quality metrics enables a more nuanced and continuous assessment of perceived quality based on human-like reasoning. Previous use of discrete or binary results could lead to significant score fluctuations around threshold values ​​and potential errors. The unified quality score provides a continuous and accurate quality score that reflects subtle changes in service quality metrics, thereby ensuring a more reliable assessment of overall perceived quality. This could lead to a unified quality score that is more realistic for quantifying human perception.

[0053]

[0059] A unified quality score further offers efficiency in terms of time and effort. A unified quality score further reduces subjectivity and complexity. Solutions heavily reliant on subjective user ratings and extensive testing can be time-wasting and resource-intensive. By leveraging fuzzy rules derived from user judgments, unified quality score generation minimizes the need for subjective testing, thereby simplifying the evaluation process. Unlike subjective modeling that relies on extensive subjective user ratings and complex experiments, the metric generation system 108 uses fuzzy rules to translate intuitive assessments into precise numerical values. This significantly reduces the need for large-scale user participation and associated logistical effort. Furthermore, storage for remembering user ratings is avoided. This process is highly scalable and can easily adapt to different service quality metrics and delivery conditions.

[0054]

[0060] The unified quality score further includes low computational complexity and low computational cost. The fuzzy logic calculations performed by the metric generation system 108 can be computations that do not require significant computing resources, such as finding minimums, applying membership functions, and calculating weighted averages. This results in lower computational overhead compared to complex mathematical modeling. The method for determining the unified quality score is also highly scalable and can be easily adapted to different quality of service metrics and delivery conditions without requiring extensive computing resources or significant modifications to the existing framework.

[0055]

[0061] system

[0062] Figure 6 shows an example of a computing device according to several embodiments. According to various embodiments, a system 600 suitable for carrying out the embodiments described herein includes a processor 601, memory 603, a storage device 605, an interface 611, and a bus 615 (e.g., a PCI bus or other interconnection structure). System 600 can operate as a metric generation system 108, or various other devices such as any other device or services described herein. Although a specific configuration is described, various alternative configurations are achievable. The processor 601 can perform operations such as those described herein. Instructions for performing such operations may be implemented in memory 603, on one or more non-temporary computer-readable media, or on some other storage device. Various specially configured devices may also be used instead of or in addition to the processor 601. Memory 603 may be random access memory (RAM) or other dynamic storage devices. The storage device 605 may include a non-transient computer-readable storage medium that holds information, instructions, or any combination thereof, for example, instructions such that, when executed by the processor 601, the processor 601 is configured or operable to perform one or more operations in the manner described herein. Buses 615 or other communication components may support the transmission of information within the system 600. Interface 611 may be connected to bus 615 and may be configured to send and receive data packets over a network. Examples of supported interfaces include, but are not limited to, Ethernet®, High Speed ​​Ethernet, Gigabit Ethernet, Frame Relay, Cable, Digital Subscriber Line (DSL), Token Ring, Asynchronous Transfer Mode (ATM), High Speed ​​Serial Interface (HSSI), and Fiber Optic Distributed Data Interface (FDDI). These interfaces may include ports adapted for communication with the appropriate medium.They may further include a separate processor and / or volatile RAM. The computer system or computing device may include a monitor, printer, or other suitable display for providing the user with any of the results described herein.

[0056]

[0063] Any of the disclosed implementations may be embodied in various types of hardware, software, firmware, computer-readable media, and combinations thereof. For example, some of the techniques disclosed herein may be implemented, at least in part, by non-temporary computer-readable media, including program instructions, state information, etc., for constructing a computing system for performing the various services and operations described herein. Examples of program instructions include both machine code, such as that produced by a compiler, and higher-level code that can be executed via an interpreter. Instructions may be embodied in any preferred language, such as Java®, Python, C++, C, HTML, any other markup language, JavaScript®, ActiveX, VBScript, or Perl. Examples of non-temporary computer-readable media include, but are not limited to, magnetic media such as hard disks and magnetic tapes, optical media such as flash memory, compact discs (CDs), or digital versatile discs (DVDs), magneto-optical media, and other hardware devices such as read-only memory ("ROM") devices and random access memory ("RAM") devices. Non-temporary computer-readable media may be any combination of such storage devices.

[0057]

[0064] In the aforementioned specifications, for clarity, various techniques and mechanisms may be described in the singular. However, unless otherwise stated, note that some embodiments may involve multiple repetitions of a technique or multiple examples of a mechanism. For example, a system may use a processor in various contexts, but unless otherwise stated, a number of processors may be used while remaining within the scope of this disclosure. Similarly, various techniques and mechanisms may be described to include a connection between two entities. However, since various other entities (e.g., bridges, controllers, gateways, etc.) may exist between the two entities, the connection does not necessarily mean a direct, unhindered connection.

[0058]

[0065] Some embodiments may be implemented in a non-temporary computer-readable storage medium for use by or in connection with an instruction execution system, device, system, or machine. The computer-readable storage medium includes instructions for controlling the computer system to perform the methods described in some embodiments. The computer system may include one or more computing devices. The instructions may be configured or operable to perform the things described in some embodiments when executed by one or more computer processors.

[0059]

[0066] In this specification and throughout the following claims, “a,” “an,” and “the” include plural references unless the context clearly indicates otherwise. Furthermore, in this specification and throughout the following claims, the meaning of “in” includes both “in” and “on,” unless the context clearly indicates otherwise.

[0060]

[0067] The above description illustrates various embodiments, along with examples of how certain aspects of the embodiments may be carried out. The above examples and embodiments should not be considered merely embodiments, but are presented to illustrate the flexibility and advantages of some embodiments as defined by the following claims. Based on the above disclosure and the following claims, other configurations, embodiments, implementations, and equivalents may be adopted without departing from the scope of this specification as defined by the claims.

Claims

1. Receiving multiple metric values ​​for service quality based on delivered content in a content delivery system, Each metric value is converted to a fuzzy value among several possible fuzzy values, and in this process, several fuzzy values ​​are determined. The process involves determining multiple fuzzy rules, and in this process, associating the fuzzy rules with fuzzy values. To determine one or more fuzzy rules that satisfy the conditions, apply each of the conditions for the fuzzy rules among the plurality of fuzzy rules to the plurality of fuzzy values, wherein the fuzzy rule is applied when each of the conditions for the fuzzy rule is satisfied by one or more of the plurality of fuzzy values. Determining a fuzzy output from one or more fuzzy values ​​associated with one or more fuzzy rules that satisfy the conditions, Based on the aforementioned fuzzy output, a unified quality score for perceived quality is calculated, A method comprising outputting the aforementioned unified quality score, wherein the aforementioned unified quality score is used to adjust the content distribution system.

2. Classifying numerous metric values ​​into categories, The method according to claim 1, further comprising determining a metric value from among the plurality of metric values ​​for the category based on the plurality of metric values.

3. The method according to claim 2, wherein the metric values ​​for multiple categories are used to determine the multiple fuzzy values.

4. Converting each metric value to one of the aforementioned fuzzy values ​​among the multiple achievable fuzzy values ​​is The method according to claim 1, further comprising applying a membership function to each of the metric values ​​in order to determine the attribute value for the fuzzy value.

5. Converting each metric value to one of the aforementioned fuzzy values ​​among the multiple achievable fuzzy values ​​is The method according to claim 4, further comprising determining an attribute value for each of the plurality of achievable fuzzy values ​​based on the membership function.

6. Determining the aforementioned attribute value is The method according to claim 5, further comprising determining the attribute value for each of the plurality of achievable fuzzy values.

7. Applying each of the conditions for the fuzzy rules among the aforementioned plurality of fuzzy rules to the aforementioned plurality of fuzzy values, The method according to claim 1, further comprising determining which of the plurality of fuzzy rules has conditions that apply to the plurality of fuzzy values.

8. The method according to claim 7, wherein a majority of the aforementioned plurality of fuzzy rules are satisfied.

9. Applying each of the conditions for the fuzzy rules among the aforementioned plurality of fuzzy rules to the aforementioned plurality of fuzzy values, The method according to claim 7, comprising determining the degree of belonging to a number of fuzzy rules that are satisfied among the plurality of fuzzy rules.

10. The degree of belonging is associated with a large number of fuzzy values ​​among the plurality of fuzzy values, The method according to claim 9, wherein the degree of assignment for a number of fuzzy values ​​is used to determine the fuzzy output.

11. The unified quality score is calculated based on the aforementioned fuzzy output. The method according to claim 1, further comprising converting the fuzzy output into a numerical value within a continuous range of values.

12. The method according to claim 11, wherein the unified quality score is a single numerical value.

13. The unified quality score is calculated based on the aforementioned fuzzy output. The method according to claim 1, further comprising using a degree of belonging to a number of fuzzy rules among the number of fuzzy rules applied to the number of fuzzy values ​​in order to determine a single numerical value for the unified quality score.

14. The method according to claim 1, further comprising adjusting the system parameters in the content distribution system based on the unified quality score.

15. The method according to claim 1, further comprising adjusting the distribution of content via the content distribution network in the content distribution system based on the unified quality score.

16. The method according to claim 1, further comprising adjusting the parameters of an encoder in the content distribution system to encode the content differently based on the aforementioned unified quality score.

17. The method according to claim 1, further comprising adjusting the playback of content in a media player based on the unified quality score.

18. A non-temporary computer-readable storage medium on which computer-executable instructions are stored, wherein when the computer-executable instructions are executed by a computing device, the computing device... Receiving multiple metric values ​​for service quality based on delivered content in a content delivery system, Each metric value is converted to a fuzzy value among several possible fuzzy values, and in this process, several fuzzy values ​​are determined. The process involves determining multiple fuzzy rules, and in this process, associating the fuzzy rules with fuzzy values. To determine one or more fuzzy rules that satisfy the conditions, apply each of the conditions for the fuzzy rules among the plurality of fuzzy rules to the plurality of fuzzy values, wherein the fuzzy rule is applied when each of the conditions for the fuzzy rule is satisfied by one or more of the plurality of fuzzy values. Determining a fuzzy output from one or more fuzzy values ​​associated with one or more fuzzy rules that satisfy the conditions, Based on the aforementioned fuzzy output, a unified quality score for perceived quality is calculated, A non-temporary computer-readable storage medium that is operable to output the aforementioned unified quality score, wherein the aforementioned unified quality score is used to adjust the content distribution system.

19. Converting each metric value to one of the aforementioned fuzzy values ​​among the multiple achievable fuzzy values ​​is A non-temporary computer-readable storage medium according to claim 18, comprising applying a membership function to each of the metric values ​​in order to determine the attribute value for the fuzzy value.

20. One or more computer processors, A device comprising a computer-readable storage medium, wherein the computer-readable storage medium is Receiving multiple metric values ​​for service quality based on delivered content in a content delivery system, Each metric value is converted to a fuzzy value among several possible fuzzy values, and in this process, several fuzzy values ​​are determined. The process involves determining multiple fuzzy rules, and in this process, associating the fuzzy rules with fuzzy values. To determine one or more fuzzy rules that satisfy the conditions, apply each of the conditions for the fuzzy rules among the plurality of fuzzy rules to the plurality of fuzzy values, wherein the fuzzy rule is applied when each of the conditions for the fuzzy rule is satisfied by one or more of the plurality of fuzzy values. Determining a fuzzy output from one or more fuzzy values ​​associated with one or more fuzzy rules that satisfy the conditions, Based on the aforementioned fuzzy output, a unified quality score for perceived quality is calculated, A device comprising instructions for controlling one or more computer processors to be operable to output the aforementioned unified quality score, wherein the unified quality score is used to adjust the content distribution system.

Images