Performance evaluation model determination method and device, equipment, medium and program product

Abstract

The invention discloses a performance evaluation model determination method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring an operation data set of a streaming media server, wherein each operation data in the operation data set comprises a resource usage amount of the streaming media server, a pull stream throughput corresponding to each pull stream protocol in the streaming media server and a back source throughput of the streaming media server; obtaining a constructed performance evaluation model, the performance evaluation model being used for representing a change relationship between a first variable and a second variable, the first variable comprising a resource usage amount of the streaming media server, and the second variable comprising a pull stream throughput corresponding to each pull stream protocol in the streaming media server and a back source throughput of the streaming media server; and based on the operation data set, determining a pull stream weight corresponding to each pull stream throughput, a back-to-source weight corresponding to the back-to-source throughput and a basic resource quantity in the performance evaluation model, thereby improving the accuracy of performance evaluation of the streaming media server.

Description

Technical Field

[0001] This relates to the field of computer technology, and in particular to a method, apparatus, device, medium, and program product for determining a performance evaluation model. Background Technology

[0002] With the rapid development of computer technology, performance evaluation of streaming media servers is often necessary to measure the service cost of streaming media services. Current methods typically involve stress testing in a laboratory environment. However, existing stress testing methods cannot simulate real-world network environments, resulting in low accuracy in performance evaluation. Summary of the Invention

[0003] This article provides a method, apparatus, device, medium, and program product for determining performance evaluation models to improve the accuracy of streaming media server performance evaluation.

[0004] In one scenario, this paper provides a method for determining a performance evaluation model, including: Obtain a set of operational data for the streaming media server. Each piece of operational data in the set includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol in the streaming media server, and the source throughput of the streaming media server. Obtain the constructed performance evaluation model, which is used to characterize the relationship between the first variable and the second variable. The first variable includes the resource usage of the streaming media server, and the second variable includes the streaming throughput corresponding to each streaming protocol in the streaming media server and the streaming media server's back-to-origin throughput. Based on the aforementioned set of operational data, the pull weight, the origin weight, and the basic resource quantity corresponding to each type of pull throughput in the performance evaluation model are determined. The pull weight is used to characterize the amount of resources used per unit of pull throughput, the origin weight is used to characterize the amount of resources used per unit of origin throughput, and the basic resource quantity is used to characterize the amount of resources used by non-streaming tasks in the streaming media server.

[0005] In one scenario, this paper also provides a performance evaluation model determination apparatus, comprising: The runtime data set acquisition module is used to acquire the runtime data set of the streaming media server. Each runtime data in the runtime data set includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol in the streaming media server, and the source throughput of the streaming media server. The performance evaluation model acquisition module is used to acquire the constructed performance evaluation model, which is used to characterize the relationship between the first variable and the second variable. The first variable includes the resource usage of the streaming media server, and the second variable includes the streaming throughput corresponding to each streaming protocol in the streaming media server and the streaming media server's back-to-origin throughput. The model parameter determination module is used to determine the pull weight, the return weight, and the basic resource quantity corresponding to each pull throughput in the performance evaluation model based on the running data set. The pull weight is used to characterize the resource quantity used per unit pull throughput, the return weight is used to characterize the resource quantity used per unit return throughput, and the basic resource quantity refers to the resource quantity used by non-streaming tasks in the streaming media server.

[0006] In one instance, this document also provides an electronic device comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the performance evaluation model determination method as described in any of these documents.

[0007] In one instance, this document also provides a storage medium containing computer-executable instructions that, when executed by a computer processor, are used to perform the performance evaluation model determination method as described herein.

[0008] In one instance, this document also provides a computer program product, including a computer program that, when executed by a processor, implements the performance evaluation model determination method as described herein.

[0009] By acquiring a set of operational data from a streaming media server in a real network environment, each data point includes the server's resource usage, streaming throughput for each streaming protocol, and origin throughput. A performance evaluation model is then constructed to characterize the relationship between the server's resource usage and the streaming throughput and origin throughput for each protocol. Based on this actual operational data, the streaming weight, origin weight, and basic resource quantity for each streaming throughput in the performance evaluation model can be accurately determined. The streaming weight characterizes the resources used per unit of streaming throughput, the origin weight characterizes the resources used per unit of origin throughput, and the basic resource quantity characterizes the resources used by non-streaming tasks on the streaming media server. By utilizing the streaming weight and origin weight, the performance of each streaming protocol and origin response on the streaming media server can be effectively and accurately evaluated, thereby improving the accuracy of streaming media server performance evaluation. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments described herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0011] Figure 1 A schematic diagram of the system structure is determined for a given performance evaluation model in one scenario. Figure 2 This is a flowchart illustrating a method for determining a performance evaluation model under one specific scenario. Figure 3 A flowchart illustrating an alternative performance evaluation model determination method for one scenario; Figure 4 This is a flowchart illustrating another method for determining a performance evaluation model under one specific scenario. Figure 5 This is a schematic diagram of a performance evaluation model determination device provided under one scenario; Figure 6 This is a schematic diagram of the structure of an electronic device provided in one scenario. Detailed Implementation

[0012] The embodiments will now be described in more detail with reference to the accompanying drawings. While some embodiments are shown in the drawings, it should be understood that this document can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the technical solutions. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the technical solutions.

[0013] It should be understood that the steps described in the method implementation may be performed in different orders and / or in parallel. Furthermore, the method implementation may include additional steps and / or omit the steps shown. The scope of this document is not limited in this respect.

[0014] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one situation" means "at least one situation"; the term "another situation" means "at least one additional situation"; the term "some situations" means "at least some situations". Definitions of other terms will be given in the following description.

[0015] It should be noted that the concepts of "first" and "second" mentioned are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions performed by these devices, modules or units or their interdependencies.

[0016] It should be noted that the terms "one" and "more" are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0017] The names of messages or information exchanged between the various devices in this document are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0018] It is understandable that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws, regulations and related provisions.

[0019] The technical solution in this article can be applied to... Figure 1The performance evaluation model shown is used to determine the system. In practical applications, this performance evaluation model determination system may include a client 101 and a server 102. The client 101 may include, but is not limited to, personal mobile terminals such as smartphones, tablets, and personal computers, or other client terminals. Various applications, such as media content publishing applications and session applications, are deployed on the client 101. The server 102 may be one or more servers providing various interfaces. That is, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; furthermore, it can be a server in a distributed system, a server integrating blockchain technology, a cloud server, or an intelligent cloud computing server or intelligent cloud host deployed with machine learning models, etc.

[0020] In the technical solution described herein, the client 101 can interact with the server 102, such as receiving or sending data. For example, in this paper, the server 102 can receive the running data set of the streaming media server sent by the client 101, so as to determine the performance evaluation model based on the running data set, and send the determined performance evaluation model to the client 101 for data display.

[0021] It should be noted that in the technical solution of this paper, the performance evaluation model determination method can be executed on either client 101 or server 102. For example, the running data set of the streaming media server can be imported into client 101 or server 102, and client 101 or server 102 can determine the performance evaluation model based on the imported running data set. Alternatively, it can be executed by both client 101 and server 102, with different functional parts of the corresponding performance evaluation model determination device deployed on client 101 and server 102 respectively. Specifically, the front-end interaction module and local data acquisition module of the device are deployed on client 101, and the back-end data processing module and data storage module are deployed on server 102. Client 101 and server 102 achieve data interaction and functional collaboration through network communication. It should be understood that... Figure 1 The number of clients and servers shown is for illustrative purposes only. Any number of clients and servers can be configured to meet specific implementation requirements.

[0022] Figure 2 This is a flowchart illustrating a performance evaluation model determination method for one scenario, applicable to performance evaluation of streaming media servers. This method can be executed by a performance evaluation model determination device, which can be implemented in software and / or hardware, optionally through an electronic device such as a mobile terminal, PC, or server. Figure 2 As shown, the method for determining the performance evaluation model may specifically include the following steps: S210. Obtain the running data set of the streaming media server. Each piece of running data in the running data set includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol in the streaming media server, and the source throughput of the streaming media server.

[0023] In this context, a streaming media server can be a server designed for audio and video streaming services, providing streaming media services. Streaming media servers can process audio and video streams based on streaming protocols, including receiving, transcoding, scheduling, and distributing them. They are also the core carriers of streaming media services such as live streaming, video-on-demand, and video conferencing. A streaming media server can refer to any streaming media server whose performance needs to be evaluated. For example, a streaming media server can refer to an edge server in a Content Delivery Network (CDN). A CDN can be a distributed network architecture that deploys a large number of edge nodes (i.e., edge servers) at the edge of the internet, distributing streaming media resources from the origin server to these edge nodes for caching. Clients can pull streams from these edge nodes, thereby improving distribution efficiency and reducing network latency. Edge nodes in a CDN can be used as streaming media servers for performance evaluation of streaming media services.

[0024] The resource usage of a streaming media server can refer to the total amount of hardware resources used by the server during actual operation. It can also refer to the total amount of hardware resources occupied by specific processes within the streaming media server. Resource usage can be used to characterize the system resource usage of a streaming media server. Specifically, it can refer to the total amount of any hardware resource used by the server, allowing for the evaluation of the usage of any hardware resource and thus assessing the server's performance. For example, resource usage may include, but is not limited to, CPU (Central Processing Unit) usage, memory usage, or disk read / write usage. CPU usage can refer to the total number of CPU cores used by the server during operation. Memory usage can refer to the total memory bandwidth used by the server during operation. Disk read / write usage can refer to the total disk I / O read / write speed used by the server during operation. During the actual operation of the streaming media server, resource usage can be obtained through methods such as event tracking.

[0025] A streaming protocol refers to the streaming transport protocol that a terminal device, acting as a client, must follow to retrieve media streams from a streaming media server. For example, streaming protocols may include, but are not limited to: RTMP (Real-Time Messaging Protocol), RTM (Real-Time Media Transport Protocol), and HLS (HTTP Live Streaming). A streaming media server can support at least one streaming protocol. Clients on the same streaming media server can use different streaming protocols to retrieve streams. The streaming throughput corresponding to a streaming protocol can refer to the throughput when using that protocol to retrieve streams from the streaming media server. Throughput can be used to measure the total amount of data transmitted by the streaming media server per unit of time. For example, streaming throughput can include: streaming bandwidth or packets per second (PPS). Streaming bandwidth can refer to the number of bytes or bits transmitted per second when using that protocol to retrieve streams from the streaming media server. Packets per second (PPS) can refer to the number of network packets transmitted per second when using that protocol to retrieve streams from the streaming media server. There is a one-to-one correspondence between streaming protocols and streaming throughput. During the actual operation of a streaming media server, the actual streaming throughput of each streaming protocol supported by the streaming media server can be obtained through methods such as instrumentation.

[0026] "Origin pull" refers to the process where, when the required media stream is not stored on the streaming media server, the streaming media server sends a media stream retrieval request to the origin server to obtain and cache the necessary media stream. Origin pull throughput refers to the throughput of the streaming media server when retrieving the media stream from the origin server. Specifically, origin pull throughput can be the total amount of data transmitted per unit time during this process. For example, origin pull throughput can include origin pull bandwidth or origin pull packets per second. Origin pull bandwidth can refer to the number of bytes or bits transmitted per second during origin pull. Origin pull packets per second can refer to the number of network data packets transmitted per second during origin pull. In the actual operation of the streaming media server, the actual origin pull throughput can be obtained through methods such as instrumentation.

[0027] It should be noted that in addition to considering the streaming loss of terminal devices, the resource consumption of streaming media servers also takes into account the source loss of streaming media servers, thereby further improving the accuracy of streaming media server performance evaluation.

[0028] For example, the runtime data set of a streaming media server includes multiple runtime data points. Each runtime data point can characterize the actual operating status of the streaming media server at a specific point in time. Each runtime data point includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol supported by the streaming media server, and the source throughput of the streaming media server. When collecting the actual runtime data of the streaming media server, to avoid individual differences in single-machine hardware, multiple streaming media servers with identical hardware resources can be selected for runtime data collection, such as multiple streaming media servers with exactly the same CPU specifications. When collecting runtime data, at least 24 hours of runtime data can be collected to ensure that the collected runtime data covers both peak and off-peak periods, thereby ensuring the accuracy of performance evaluation. All collected runtime data are aggregated, and noise points such as system restarts and sudden abnormal events are removed to obtain a high-quality runtime data set.

[0029] For example, the streaming media server's operational data set includes first operational data and second operational data. The first operational data refers to operational data collected during peak request periods, and the second operational data refers to operational data collected during non-peak request periods. The peak request period can be the time period when the number of streaming requests is greater than or equal to a preset peak. The non-peak request period can be the time period when the number of streaming requests is less than the preset peak, such as periods when the number of streaming requests is at its lowest or lowest point. For example, the peak request period could be 20:00–21:00, and the non-peak request periods could be 0:00–20:00 and 21:00–24:00.

[0030] For example, the runtime data set of the streaming media server includes runtime data generated by at least two streaming media servers, where each streaming media server has the same hardware resource information. By collecting actual runtime data from at least two streaming media servers with identical hardware resources, individual differences in single-machine hardware can be avoided, further improving the accuracy of performance evaluation.

[0031] S220. Obtain the constructed performance evaluation model. The performance evaluation model is used to characterize the relationship between the first variable and the second variable. The first variable includes the resource usage of the streaming media server, and the second variable includes the streaming throughput corresponding to each streaming protocol in the streaming media server and the streaming media server's origin throughput.

[0032] The performance evaluation model can be a quantitative description of the resource consumption of the streaming media server. The first variable can be the dependent variable, and the second variable can be the independent variable, thus constructing the performance evaluation model. Specifically, the resource usage of the streaming media server can be used as the dependent variable, and the streaming throughput and origin throughput corresponding to each streaming protocol can be used as independent variables. This constructs a performance evaluation model that characterizes the relationship between resource usage and the streaming throughput and origin throughput of each protocol, i.e., a performance evaluation model that simultaneously includes streaming throughput and origin throughput, in order to evaluate the streaming performance and origin performance of the streaming media server. It should be noted that the performance evaluation model constructed in step S220 is only a general variation relationship model; the specific values ​​of the coefficients in the model have not yet been determined. Therefore, subsequent analysis of the constructed performance evaluation model is needed to determine a specific performance evaluation model used to characterize the performance of the streaming media server.

[0033] S230. Based on the running data set, determine the pull weight, the return weight, and the basic resource quantity corresponding to each pull throughput in the performance evaluation model. The pull weight is used to characterize the resource quantity used per unit pull throughput, the return weight is used to characterize the resource quantity used per unit return throughput, and the basic resource quantity is used to characterize the resource quantity used by non-streaming tasks in the streaming media server.

[0034] In this context, the streaming weight can be considered a coefficient of streaming throughput in the performance evaluation model, used to characterize the amount of resources consumed per unit of streaming throughput. Unit streaming throughput can refer to the amount of data transmitted per unit when streaming from a streaming media server using a streaming protocol. For example, when streaming throughput is streaming bandwidth, unit streaming throughput can be unit bandwidth, which can refer to the number of bytes or bits transmitted per unit when streaming from a streaming media server using a streaming protocol. When streaming throughput is the number of packets per second, unit streaming throughput can be the number of packets per second, which can refer to a single packet transmitted per unit when streaming from a streaming media server using a streaming protocol. There is a one-to-one correspondence between streaming protocols and streaming throughput. There is also a one-to-one correspondence between streaming throughput and streaming weight. The streaming weight corresponding to each streaming throughput can refer to the streaming weight of the streaming throughput corresponding to each streaming protocol supported by the streaming media server. The streaming weight corresponding to a streaming protocol can be used to characterize the amount of resources consumed per unit of throughput when streaming from a streaming media server using that particular streaming protocol. Pull weights can be used to characterize the pull performance of a corresponding pull protocol. For example, a smaller pull weight indicates higher pull performance.

[0035] Origin pull weight can be considered a coefficient of origin pull throughput in a performance evaluation model, used to characterize the amount of resources consumed per unit of origin pull throughput. Unit origin pull throughput can refer to the amount of data transmitted per unit during origin pull by the streaming media server. For example, when origin pull throughput is origin pull bandwidth, unit origin pull throughput can be unit bandwidth, which can refer to the number of bytes or bits transmitted per unit during origin pull. When origin pull throughput is packets per second, unit origin pull throughput can be unit packets per second, which can refer to a single packet transmitted per unit during origin pull. Origin pull throughput corresponds to origin pull weight. The origin pull weight corresponding to origin pull throughput can be used to characterize the amount of resources consumed per unit of throughput during origin pull by the streaming media server. Origin pull weight can be used to characterize the origin pull performance of the streaming media server. For example, the smaller the origin pull weight, the higher the corresponding origin pull performance.

[0036] In a streaming media server, non-streaming tasks refer to background tasks that run on the server and are unrelated to the streaming media service, such as background computation tasks and data query tasks. Non-streaming tasks can be considered as persistent tasks unrelated to the streaming media service. Basic resource quantities can be used to characterize the resource consumption of persistent tasks on a streaming media server. Introducing basic resource quantities into the performance evaluation model can further improve the accuracy of evaluating streaming pull performance and origin pull performance on a streaming media server.

[0037] For example, based on the least squares method and the running data set, regression analysis is performed on the performance evaluation model to determine the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the performance evaluation model; or, based on the gradient descent method and the running data set, regression analysis is performed on the performance evaluation model to determine the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the performance evaluation model.

[0038] By using least squares or gradient descent analysis, and performing regression analysis on the constructed performance evaluation model based on the actual operational data set of the streaming media server, the pull weight, origin pull weight, and basic resource quantity corresponding to each pull throughput, can be accurately determined. Each operational data point in the operational data set can be considered as a specific value of the first and second variables. Therefore, by performing regression analysis based on the operational data set, all unknown coefficient weights in the performance evaluation model can be obtained, namely, the pull weight, origin pull weight, and basic resource quantity corresponding to each pull protocol. Furthermore, based on the determined pull weights for each pull protocol, the performance of each pull protocol in the streaming media server can be accurately evaluated, and based on the determined origin pull weights, the origin pull performance in the streaming media server can be accurately evaluated.

[0039] For example, the performance evaluation model constructed in step S220 may include a linear regression term. The linear regression term is used to characterize the linear relationship between the first variable and the second variable. That is, the linear regression term can be used to characterize the linear relationship between the resource usage of the streaming media server and the linear relationship between each type of pull-in throughput and source-out throughput in the streaming media server. A linear relationship can mean that the rate of change of the independent variable is consistent with the rate of change of the dependent variable. The rate of change of each type of pull-in throughput and source-out throughput is consistent with the rate of change of resource usage.

[0040] For example, the linear regression term is constructed based on each type of streaming throughput, streaming weight, origin throughput, origin weight, and basic resource quantity in the streaming media server. The construction process of the linear regression term can be as follows: the streaming throughput and the streaming weight corresponding to each type of streaming throughput in the streaming media server are weighted and summed to obtain the total streaming term; the origin throughput and the origin weight corresponding to the origin throughput are multiplied to obtain the origin term; the total streaming term, the origin term, and the basic resource quantity are added together to obtain the linear regression term, thereby completing the construction of the linear regression term.

[0041] For example, the performance evaluation model can be constructed as follows: ,in, This refers to the resource usage of the streaming media server. For the first The streaming throughput corresponding to each streaming protocol For the first The streaming weight of the streaming throughput corresponding to each streaming protocol, where n is the number of streaming protocols supported by the streaming media server. For the origin throughput of the streaming media server, The origin-following weight corresponding to the origin-following throughput. Given the basic resource quantity, the linear regression term in the above performance evaluation model is: The total flow term is: The source item is: By performing regression analysis on the constructed performance evaluation model based on the runtime dataset, the streaming weight corresponding to each streaming protocol in the performance evaluation model can be accurately determined. Source weight and basic resource quantity This allows us to obtain a specific performance evaluation model for characterizing the performance of streaming media servers.

[0042] The above method obtains a set of operational data of the streaming media server in a real network environment. Each piece of operational data in the set includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol, and the origin throughput of the streaming media server. A performance evaluation model is then constructed to characterize the relationship between the resource usage of the streaming media server and the streaming throughput and origin throughput corresponding to each streaming protocol. Based on the actual operational data set, the streaming weight, origin weight, and basic resource quantity corresponding to each streaming throughput, as well as the basic resource quantity, can be accurately determined in the performance evaluation model. The streaming weight characterizes the resource quantity used per unit of streaming throughput, the origin weight characterizes the resource quantity used per unit of origin throughput, and the basic resource quantity characterizes the resource quantity used by non-streaming tasks on the streaming media server. By utilizing the streaming weight and origin weight, the performance of each streaming protocol and the origin response of the streaming media server can be effectively and accurately evaluated, thereby improving the accuracy of streaming media server performance evaluation.

[0043] In one scenario, the origin pull throughput can include: a first origin pull throughput and a second origin pull throughput; wherein, the first origin pull throughput is used to characterize the throughput of the streaming media server as a client pulling the stream from the first server; and the second origin pull throughput is used to characterize the throughput of the streaming media server as a server being pulled from the second server.

[0044] It should be understood that there are two ways for a server to perform a pull request to the origin server. The first is for the server to pull the stream directly from the origin server, and the second is for the server to pull the stream from a nearby server, thereby improving the efficiency of the pull request. For the same server, it can act as a client to pull the stream from the origin server, and it can also act as a server to support other servers to pull the stream from the origin server. In other words, the resource consumption of the server in pulling the stream to the origin server has two scenarios: one is the inbound pull request consumption when acting as a client, and the other is the outbound pull request consumption when acting as a server.

[0045] In this context, "first server" can refer to the server that performs origin pull when the streaming media server acts as a client, and "second server" can refer to the server that receives origin pull when the streaming media server acts as a server. In other words, the streaming media server pulls the stream from the first server (inbound origin pull) and the second server pulls the stream from the streaming media server (outbound origin pull). "First origin pull throughput" can refer to the total amount of data transmitted per unit time when the streaming media server retrieves the media stream from the first server. "Second origin pull throughput" can refer to the total amount of data transmitted per unit time when the second server retrieves the media stream from the streaming media server. "First origin pull throughput" can refer to the inbound origin pull throughput of the streaming media server, and "second origin pull throughput" can refer to the outbound origin pull throughput of the streaming media server. For example, "first origin pull throughput" can include "first origin bandwidth" or "first origin packets per second." "First origin bandwidth" can refer to the number of bytes or bits transmitted per second when the streaming media server pulls the stream from the first server. "First origin packets per second" can refer to the number of network data packets transmitted per second when the streaming media server pulls the stream from the first server. Second-source throughput can also include: second-source bandwidth or second-source packets per second. Second-source bandwidth can refer to the number of bytes or bits transmitted per second by the second server when retrieving data from the streaming media server. Second-source packets per second can refer to the number of network data packets transmitted per second by the second server when retrieving data from the streaming media server.

[0046] For example, the streaming media server is an edge server in a content delivery network. The first server includes: the source server or other edge servers belonging to the same cluster as the streaming media server; the second server includes: other edge servers belonging to the same cluster as the streaming media server.

[0047] It should be understood that in a content delivery network (CDN), streaming media servers, acting as edge servers, can exist in a cluster. Multiple edge servers form a cluster. If an edge server needs to retrieve content from the origin server, it can first retrieve the content from other edge servers in its cluster. If all other edge servers in the cluster are caching the required media stream, then it retrieves the content from the origin server. In a CDN, for the same edge server (i.e., the streaming media server), there are instances where it retrieves content from other edge servers in its cluster and also from the origin server. Therefore, this edge server incurs inbound resource consumption for retrieving content from the origin server; the first server can include the origin server or other edge servers belonging to the same cluster. Simultaneously, other edge servers in the same cluster also retrieve content from this edge server; therefore, this edge server incurs outbound resource consumption for retrieving content from the origin server; the second server includes other edge servers belonging to the same cluster as the streaming media server.

[0048] It should be noted that by further dividing the source backhaul in the streaming media server into inbound source backhaul and outbound source backhaul, the resource consumption of inbound and outbound source backhaul of the streaming media server can be evaluated in a more granular manner, thereby enabling a more accurate assessment of the resource consumption of the streaming media server by the source backhaul.

[0049] For example, when the streaming media server's back-to-origin throughput includes a first back-to-origin throughput and a second back-to-origin throughput, the constructed performance evaluation model can be used to characterize the relationship between the streaming media server's resource usage and the changes in each type of streaming throughput, the first back-to-origin throughput, and the second back-to-origin throughput. The back-to-origin weight corresponding to the back-to-origin throughput can include: a first back-to-origin weight corresponding to the first back-to-origin throughput and a second back-to-origin weight corresponding to the second back-to-origin throughput. The first back-to-origin weight characterizes the amount of resources used per unit of back-to-origin throughput. The first unit of back-to-origin throughput can refer to the amount of data transmitted per unit when the streaming media server retrieves the media stream from the first server. The first back-to-origin weight characterizes the amount of resources consumed per unit of throughput when the streaming media server retrieves the media stream from the first server. The second back-to-origin weight characterizes the amount of resources used per unit of back-to-origin throughput. The second unit of back-to-origin throughput can refer to the amount of data transmitted per unit when the second server retrieves the media stream from the streaming media server. The second back-to-origin weight characterizes the amount of resources consumed per unit of throughput when the second server retrieves the media stream from the streaming media server. The first origin-back weight can be used to evaluate the inbound origin-back performance of a streaming media server, while the second origin-back weight can be used to evaluate the outbound origin-back performance. For example, a smaller first origin-back weight indicates higher inbound origin-back performance, and a smaller second origin-back weight indicates higher outbound origin-back performance.

[0050] For example, the performance evaluation model can be constructed as follows: ,in, This refers to the resource usage of the streaming media server. For the first The streaming throughput corresponding to each streaming protocol For the first The streaming weight of the streaming throughput corresponding to the streaming protocol. This represents the first-round throughput of the streaming media server. The first source weight corresponding to the first source throughput. For the second source throughput of the streaming media server, The second source weight corresponds to the second source throughput. This represents the basic resource quantity. By performing regression analysis on the constructed performance evaluation model based on actual operational data sets, the streaming weight corresponding to each streaming protocol in the performance evaluation model can be accurately determined. First source weight Second source weight and basic resource quantity This allows us to obtain a specific performance evaluation model for characterizing the performance of streaming media servers.

[0051] Figure 3 This is a flowchart illustrating another method for determining a performance evaluation model under one scenario. Based on the above embodiments, the constructed performance evaluation model may further include a nonlinear correction term, and the process of determining the performance evaluation model is described in detail. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here. Figure 3 As shown, the method for determining the performance evaluation model can specifically include the following steps: S310. Obtain the running data set of the streaming media server. The running data set includes a first running data set and a second running data set.

[0052] The actual operational data set of the streaming media server can be divided into a first operational data set and a second operational data set. Both sets include multiple different operational data points, each with different parameter values. Each operational data point in both sets can include the streaming media server's resource usage, the streaming throughput corresponding to each streaming protocol, and the streaming media server's origin throughput. The first operational data set can be used as a training set for model training to determine the streaming weights corresponding to each streaming throughput, the origin throughput corresponding to each origin throughput, and the basic resource quantities in the performance evaluation model. The second operational data set can be used as a validation set for model validation to verify whether the accuracy of the determined streaming weights corresponding to each streaming throughput, the origin throughput corresponding to each origin throughput, and the basic resource quantities meets the requirements.

[0053] For example, the first set of operational data includes the set of operational data collected in a first time period, which includes peak request periods and non-peak request periods; the second set of operational data includes the set of operational data collected in a second time period, which includes peak request periods and non-peak request periods; wherein, the first time period is different from the second time period.

[0054] The first time and the second time are two different collection times, but both the first time and the second time cover the peak request period and the non-peak request period. This ensures that both the first running data set and the second running data set include running data collected during the peak request period and running data collected during the non-peak request period, thereby guaranteeing the diversity of training samples and validation samples and improving the accuracy of performance evaluation model determination.

[0055] S320. Obtain the constructed performance evaluation model, which includes a linear regression term.

[0056] For example, a performance evaluation model containing a linear regression term can be constructed. In this case, the performance evaluation model is used to characterize the linear relationship between the first and second variables. The first variable includes the resource usage of the streaming media server, and the second variable includes the streaming throughput corresponding to each streaming protocol in the streaming media server and the streaming media server's origin throughput. The origin throughput can also include both the first and second origin throughput.

[0057] S330. Based on the first set of running data, the performance evaluation model is trained to obtain the first performance evaluation model, which is the trained performance evaluation model.

[0058] For example, during the model training phase, the least squares method or gradient descent method can be used to perform regression analysis on the constructed performance evaluation model based on the first set of running data to solve for the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the performance evaluation model. The specific values ​​obtained are then substituted into the constructed performance evaluation model to complete the model training and obtain the specific performance evaluation model, namely the first performance evaluation model after training.

[0059] It should be noted that the first performance evaluation model is a performance evaluation model that includes a linear regression term. For example, the first performance evaluation model can be expressed as: Alternatively, the first performance evaluation model can also be expressed as: .

[0060] S340. Based on the second set of running data, the first performance evaluation model is validated to obtain the first residual corresponding to the first performance evaluation model.

[0061] For example, during the model validation phase, the streaming throughput corresponding to each streaming protocol in each set of runtime data and the streaming media server's origin throughput are substituted into the first performance evaluation model to obtain the corresponding predicted resource usage values. Based on the predicted resource usage values ​​and the actual resource usage in the runtime data, a first residual can be determined. The first residual can be the difference between the predicted resource usage value and the actual resource usage. The first residual can be used to characterize the determination accuracy of the first performance evaluation model. For example, the larger the first residual, the lower the determination accuracy of the first performance evaluation model.

[0062] S350. In response to the first residual being greater than a preset threshold, a constructed nonlinear correction term is obtained, and a nonlinear correction term is added to the first performance evaluation model to obtain a second performance evaluation model. The nonlinear correction term is used to characterize the nonlinear change relationship between the first variable and the second variable.

[0063] The nonlinear correction term can be used to characterize the nonlinear relationship between the resource usage of the streaming media server and the throughput of each type of pull and source connections within the server. This nonlinear correction term can compensate for the performance degradation of the protocol stack under high load. For example, it can compensate for the additional nonlinear resource overhead caused by interrupt storms, context switching, and memory bandwidth bottlenecks under high bandwidth throughput.

[0064] For example, the non-linear correction term can be constructed based on the pull throughput and origin throughput of each streaming protocol in the streaming media server. The variables in the non-linear correction term can include the pull throughput corresponding to each streaming protocol in the streaming media server, or the origin throughput of the streaming media server.

[0065] For example, the construction process of the nonlinear correction term can be as follows: Add the streaming throughput corresponding to each streaming protocol in the streaming media server to the source throughput of the streaming media server to obtain the total throughput; construct the nonlinear correction term based on the total throughput, the total throughput weight, and the order of the total throughput, where the order of the total throughput is greater than or equal to second order. For example, the constructed nonlinear correction term can be expressed as: Alternatively, the nonlinear correction term can also be expressed as: ,in, As the weight of total throughput, This represents the order of total throughput.

[0066] For example, if the first residual corresponding to the first performance evaluation model is greater than a preset threshold, and since the amount of data in the first running dataset used as the training set is sufficiently large, the model accuracy may be low due to nonlinear relationships. Therefore, a nonlinear correction term is constructed and added to the first performance evaluation model to obtain a second performance evaluation model that includes a linear regression term and a nonlinear correction term. For example, the second performance evaluation model can be expressed as: Alternatively, the first performance evaluation model can also be expressed as: .

[0067] It should be noted that if the first residual corresponding to the first performance evaluation model is less than or equal to the preset threshold, it indicates that the determination accuracy of the first performance evaluation model meets the requirements. At this time, it can be determined that the training of the first performance evaluation model has ended, and the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the first performance evaluation model are directly determined as the final performance evaluation indicators.

[0068] S360. Iteratively train the second performance evaluation model until the second residual corresponding to the second performance evaluation model is less than or equal to the preset threshold, and obtain the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the second performance evaluation model.

[0069] For example, the second performance evaluation model with added nonlinear correction terms can be retrained and validated based on the first and second running data sets until the second residual corresponding to the trained second performance evaluation model is less than or equal to a preset threshold. When the second residual corresponding to the trained second performance evaluation model is less than or equal to the preset threshold, it indicates that the training of the second performance evaluation model has ended. At this time, the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the second performance evaluation model can be used as the final determined performance evaluation indicators.

[0070] For example, step S360 may include: training a second performance evaluation model based on a first set of running data to obtain a trained second performance evaluation model; validating the trained second performance evaluation model based on the second set of running data to obtain a second residual corresponding to the trained second performance evaluation model; and adjusting the order and / or weight of the nonlinear correction term in the second performance evaluation model in response to the second residual being greater than a preset threshold, and retraining the adjusted second performance evaluation model until the second residual is less than or equal to the preset threshold.

[0071] Specifically, the least squares method or gradient descent method can be used to perform regression analysis on the second performance evaluation model based on the first set of running data. This allows for the determination of the pull weights for each pull throughput, the return weights for each return throughput, and the basic resource quantities in the second performance evaluation model. The calculated values ​​are then substituted into the second performance evaluation model to obtain the trained model. Substituting the pull throughput for each pull protocol and the return throughput of the streaming media server in each set of running data into the second performance evaluation model yields the corresponding predicted resource usage values. Based on these predicted values ​​and the actual resource usage in the running data, the second residual of the second performance evaluation model can be determined. The second residual can be used to characterize the accuracy of the second performance evaluation model. If the second residual is greater than a preset threshold, the order and / or weights of the nonlinear correction term in the second performance evaluation model can be adjusted, such as increasing or decreasing the order of the nonlinear correction term. And / or, increase or decrease the weight of the nonlinear correction term. Based on the first and second running data sets, the adjusted second performance evaluation model is retrained and validated until the second residual corresponding to the trained second performance evaluation model is less than or equal to the preset threshold. This indicates that the determination accuracy of the second performance evaluation model meets the requirements. At this point, the training of the second performance evaluation model can be determined to be over. The pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the second performance evaluation model at this time are taken as the final determined performance evaluation indicators.

[0072] The above method adds a nonlinear correction term to the first performance evaluation model when the first residual corresponding to the first performance evaluation model is greater than a preset threshold, to obtain a second performance evaluation model. The obtained second performance evaluation model is then iteratively trained until the second residual corresponding to the second performance evaluation model is less than or equal to the preset threshold. This results in more accurate pull-in weights, back-out weights, and basic resource quantities, thereby ensuring the accuracy of the performance evaluation model under different business pressures, avoiding distortion of linear evaluation in extreme cases, and improving the generalization ability of the model.

[0073] It should be noted that the final performance evaluation model can not only assess the streaming performance of each streaming protocol in the streaming media server, but also accurately evaluate the resource consumption of the streaming media server by the origin pull logic, providing a quantitative basis for origin pull optimization within the content delivery network. By introducing more granular first and second origin pull throughput into the performance evaluation model, and treating them as independent variables for origin pull evaluation, the resource consumption of inbound and outbound origin pulls can be accurately quantified. Through the streaming weight and basic resource quantity in the performance evaluation model, it is possible to clearly reflect whether the system noise floor of the streaming media server is too high or the logic of a certain streaming protocol is too heavy, thereby accurately locating the performance bottleneck.

[0074] Figure 4 This is a flowchart illustrating another method for determining a performance evaluation model under one scenario. Based on the above embodiments, it details the process of evaluating and comparing the performance of the first object before and after optimization. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0075] like Figure 4 As shown, the method for determining the performance evaluation model specifically includes the following steps: S410, Obtain the runtime data set of the streaming media server.

[0076] S420, Obtain the constructed performance evaluation model.

[0077] S430. Based on the running data set, determine the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the performance evaluation model.

[0078] S440. Obtain the first weight corresponding to the first object. The first object is any pull protocol or origin return method in the streaming media server. The first weight is the weight corresponding to the first object in the first performance evaluation model. The first performance evaluation model is determined based on the set of running data before the first object is optimized.

[0079] The first object can refer to the object whose performance needs to be compared before and after optimization. For example, the first object can refer to any streaming protocol supported by the streaming media server, so as to compare the performance of the same streaming protocol before and after optimization. The first object can also refer to the streaming media server's origin pull method, which refers to the origin pull logic when the streaming media server pulls the media stream from the origin. The origin pull method can include a first origin pull method and a second origin pull method. The first origin pull method can refer to the inbound origin pull logic when the streaming media server pulls the media stream from the first server. The second origin pull method can refer to the outbound origin pull logic when the second server pulls the media stream from the streaming media server.

[0080] When the first object is a pull streaming protocol, the first weight is the pull streaming throughput corresponding to that protocol in the first performance evaluation model. When the first object is a back-to-origin method, the first weight is the back-to-origin throughput corresponding to that method in the first performance evaluation model. For example, when the first object is a first back-to-origin method, the first weight is the first back-to-origin throughput corresponding to that method in the first performance evaluation model. When the first object is a second back-to-origin method, the first weight is the second back-to-origin throughput corresponding to that method in the first performance evaluation model.

[0081] The first performance evaluation model can be constructed by analyzing the actual operating data set before the first object is optimized, in order to determine the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the performance evaluation model, and then using the specific performance evaluation model after determining the parameters as the first performance evaluation model. For example, before optimizing the first object in the streaming media server, the first performance evaluation model before the first object is optimized is determined by executing steps S410-S430, and then the first weight corresponding to the first object in the first performance evaluation model is obtained.

[0082] S450. Obtain the second weight corresponding to the first object. The second weight is the weight corresponding to the first object in the second performance evaluation model. The second performance evaluation model is determined based on the optimized running data set of the first object.

[0083] Specifically, when the first object is a pull streaming protocol, the second weight is the pull streaming throughput corresponding to that protocol in the second performance evaluation model. When the first object is a back-to-origin method, the second weight is the back-to-origin throughput corresponding to that method in the second performance evaluation model. For example, when the first object is a first back-to-origin method, the second weight is the first back-to-origin throughput corresponding to that method in the second performance evaluation model. When the first object is a second back-to-origin method, the second weight is the second back-to-origin throughput corresponding to that method in the second performance evaluation model.

[0084] The second performance evaluation model can be constructed by analyzing the actual operating data set after the optimization of the first object, in order to determine the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the performance evaluation model, and then using the specific performance evaluation model after determining the parameters as the second performance evaluation model. For example, after optimizing the first object in the streaming media server, the second performance evaluation model after the optimization of the first object is determined by executing steps S410-S430, and then the second weight corresponding to the first object in the second performance evaluation model is obtained.

[0085] S460. Based on the first weight and the second weight, determine the weight change amount corresponding to the first object. The weight change amount is used to characterize the optimization degree of the first object.

[0086] It should be understood that the difference between the first weight and the second weight is taken as the weight change for the first object. A larger weight change indicates a greater degree of optimization for the first object. For example, if the first weight is greater than the second weight, it indicates that the performance of the first object has improved after optimization, and the greater the weight change, the greater the performance improvement. For instance, when the resource quantity is the number of CPU cores, a decrease in the weight of the first object means that, with the same number of CPU cores on the streaming media server, it can handle more throughput (such as bandwidth), or, with the same throughput, it can reduce the number of machines, thereby reducing costs.

[0087] By determining the weight change of the first object, the performance optimization effect of the first object can be quantitatively and accurately described. This allows for a clearer understanding of the optimization direction and improvement plan. Furthermore, the allocation ratio of different streaming protocols or the number of machines can be adjusted based on the weight change, thus achieving an accurate assessment of the object's performance optimization.

[0088] Figure 5 This is a schematic diagram of a performance evaluation model determination device provided in one scenario, such as... Figure 5 As shown, the device may include: a running data set acquisition module 510, a performance evaluation model acquisition module 520, and a model parameter determination module 530.

[0089] The system includes a runtime data set acquisition module 510, which acquires a runtime data set of the streaming media server. Each runtime data set includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol in the streaming media server, and the origin throughput of the streaming media server. A performance evaluation model acquisition module 520 is used to acquire a constructed performance evaluation model, which characterizes the relationship between a first variable and a second variable. The first variable includes the resource usage of the streaming media server, and the second variable includes the streaming throughput corresponding to each streaming protocol in the streaming media server and the origin throughput of the streaming media server. A model parameter determination module 530, based on the runtime data set, determines the streaming weight corresponding to each streaming throughput, the origin weight corresponding to the origin throughput, and the basic resource quantity in the performance evaluation model. The streaming weight characterizes the resource quantity used per unit of streaming throughput, the origin weight characterizes the resource quantity used per unit of origin throughput, and the basic resource quantity refers to the resource quantity used by non-streaming media tasks in the streaming media server.

[0090] Based on the aforementioned apparatus, by acquiring a set of operational data of the streaming media server in a real network environment, each piece of operational data in the set includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol in the streaming media server, and the streaming throughput back to the origin in the streaming media server; a constructed performance evaluation model is acquired, which characterizes the relationship between the resource usage of the streaming media server and the streaming throughput and back to the origin corresponding to each streaming protocol; based on the actual operational data set, the streaming weight corresponding to each streaming throughput, the back-to-origin weight corresponding to the back-to-origin throughput, and the basic resource quantity in the performance evaluation model can be accurately determined. The streaming weight characterizes the resource quantity used per unit of streaming throughput, the back-to-origin weight characterizes the resource quantity used per unit of back-to-origin throughput, and the basic resource quantity characterizes the resource quantity used by non-streaming tasks in the streaming media server. By utilizing the streaming weight and back-to-origin weight, the performance of each streaming protocol and the back-to-origin operation in the streaming media server can be effectively and accurately evaluated, thereby improving the accuracy of streaming media server performance evaluation.

[0091] Optionally, the constructed performance evaluation model includes a linear regression term; the linear regression term is constructed based on each type of streaming throughput, streaming weight, origin throughput, origin weight, and basic resource quantity in the streaming media server; The linear regression term is used to characterize the linear relationship between the first variable and the second variable.

[0092] Optionally, the running data set includes a first running data set and a second running data set; Model parameter determination module 530 includes: The model training unit is used to train the performance evaluation model based on the first set of running data to obtain a first performance evaluation model, wherein the first performance evaluation model is the trained performance evaluation model. The model validation unit is used to validate the first performance evaluation model based on the second running data set to obtain the first residual corresponding to the first performance evaluation model. A nonlinear correction term addition unit is used to obtain a constructed nonlinear correction term in response to the first residual being greater than a preset threshold, and to add the nonlinear correction term to the first performance evaluation model to obtain a second performance evaluation model. The nonlinear correction term is used to characterize the nonlinear change relationship between the first variable and the second variable. The model iteration unit is used to iteratively train the second performance evaluation model until the second residual corresponding to the second performance evaluation model is less than or equal to a preset threshold, thereby obtaining the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the second performance evaluation model.

[0093] Optionally, the nonlinear correction term is constructed based on the streaming throughput corresponding to each streaming protocol in the streaming media server and the source throughput of the streaming media server.

[0094] Optionally, the device may also include: The nonlinear correction term construction module is used to add the streaming throughput corresponding to each streaming protocol in the streaming media server and the source throughput of the streaming media server to obtain the total throughput; based on the total throughput, the total throughput weight and the order of the total throughput, the nonlinear correction term is constructed, wherein the order of the total throughput is greater than or equal to the second order.

[0095] Optional, model iteration unit, specifically used for: Based on the first set of running data, the second performance evaluation model is trained to obtain the trained second performance evaluation model. Based on the second set of running data, the trained second performance evaluation model is validated to obtain the second residual corresponding to the trained second performance evaluation model. In response to the second residual being greater than a preset threshold, the order and / or weight of the nonlinear correction term in the second performance evaluation model are adjusted, and the adjusted second performance evaluation model is retrained until the second residual is less than or equal to the preset threshold.

[0096] Optionally, the first set of operational data includes the set of operational data collected within a first time period, where the first time period includes peak request periods and non-peak request periods; The second set of operational data includes the set of operational data collected in the second time period, which includes peak request periods and non-peak request periods. The first time is different from the second time.

[0097] Optionally, the origin pull throughput includes: a first origin pull throughput and a second origin pull throughput; wherein, the first origin pull throughput is used to characterize the throughput of the streaming media server as a client pulling streams from the first server; the second origin pull throughput is used to characterize the throughput of the streaming media server as a server being pulled streams from the second server. The back-source weights corresponding to the back-source throughput include: the first back-source weight corresponding to the first back-source throughput and the second back-source weight corresponding to the second back-source throughput.

[0098] Optionally, the streaming media server is an edge server in a content delivery network; The first server includes: an origin server or other edge servers belonging to the same cluster as the streaming media server; The second server includes other edge servers that belong to the same cluster as the streaming media server.

[0099] Optionally, the device may also include: The first weight acquisition module is used to acquire the first weight corresponding to the first object, where the first object is any pull streaming protocol or origin return method in the streaming media server, and the first weight is the weight corresponding to the first object in the first performance evaluation model, which is determined based on the set of running data of the first object before optimization. The second weight acquisition module is used to acquire the second weight corresponding to the first object. The second weight is the weight corresponding to the first object in the second performance evaluation model. The second performance evaluation model is determined based on the optimized running data set of the first object. The weight change determination module is used to determine the weight change amount corresponding to the first object based on the first weight and the second weight, wherein the weight change amount is used to characterize the optimization degree of the first object.

[0100] Optionally, the resource usage includes: CPU usage, memory usage, or disk read usage; The streaming throughput includes: streaming bandwidth or streaming packets per second; The origin throughput includes origin bandwidth or origin packets per second.

[0101] The performance evaluation model determination apparatus provided herein can execute the performance evaluation model determination method provided in any embodiment of this invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0102] It is worth noting that the various units and modules included in the above-mentioned device are divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection.

[0103] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one scenario. See below for reference. Figure 6 It shows an electronic device suitable for implementing this technical solution (e.g., Figure 6 The diagram below shows the structure of the terminal device or server 500. The terminal device in this document may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic devices shown are merely examples and should not impose any limitations on their functionality or scope of use.

[0104] like Figure 6 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0105] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0106] In particular, according to embodiments herein, the processes described in the above-referenced flowcharts can be implemented as computer software programs. For example, embodiments herein include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 509, or installed from storage device 508, or installed from ROM 502. When the computer program is executed by processing device 501, the aforementioned functions defined in the methods are performed.

[0107] The names of messages or information exchanged between multiple devices in the embodiments herein are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0108] The electronic device provided in this text and the performance evaluation model determination method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail can be found in the above embodiments, and they have the same beneficial effects as the above embodiments.

[0109] The text provides a computer storage medium on which a computer program is stored, which, when executed by a processor, implements the performance evaluation model determination method provided in the above embodiments.

[0110] It should be noted that the aforementioned computer-readable medium can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. A computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0111] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0112] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0113] The aforementioned computer-readable medium carries one or more programs. When the aforementioned one or more programs are executed by the electronic device, the electronic device causes the following: It acquires a set of operational data for a streaming media server, wherein each piece of operational data in the set includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol in the streaming media server, and the origin throughput of the streaming media server; it acquires a constructed performance evaluation model, which characterizes the relationship between a first variable and a second variable, wherein the first variable includes the resource usage of the streaming media server, and the second variable includes the streaming throughput corresponding to each streaming protocol in the streaming media server and the origin throughput of the streaming media server; and based on the set of operational data, it determines the streaming weight corresponding to each streaming throughput, the origin weight corresponding to the origin throughput, and the basic resource quantity in the performance evaluation model, wherein the streaming weight characterizes the resource quantity used per unit of streaming throughput, the origin weight characterizes the resource quantity used per unit of origin throughput, and the basic resource quantity characterizes the resource quantity used by non-streaming media tasks in the streaming media server.

[0114] Computer program code for performing the operations described herein may be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The text also provides a computer program product, including a computer program that, when executed by a processor, implements the performance evaluation model determination method provided in the above embodiments.

[0116] The computer program product includes a computer program carried on a non-transitory computer-readable medium, which contains program code for performing a method for determining a performance evaluation model. The program code can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this document. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0118] The units described herein can be implemented in software or hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0119] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0120] In the context of this document, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0121] The above description is merely a preferred embodiment and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure herein is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed herein that have similar functions.

[0122] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this document. Certain features described in the context of individual implementations may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0123] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for determining a performance evaluation model, comprising: Obtain a set of operational data for the streaming media server. Each piece of operational data in the set includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol in the streaming media server, and the source throughput of the streaming media server. Obtain the constructed performance evaluation model, which is used to characterize the relationship between the first variable and the second variable. The first variable includes the resource usage of the streaming media server, and the second variable includes the streaming throughput corresponding to each streaming protocol in the streaming media server and the streaming media server's origin throughput. Based on the aforementioned set of operational data, the pull weight, the origin weight, and the basic resource quantity corresponding to each type of pull throughput in the performance evaluation model are determined. The pull weight is used to characterize the amount of resources used per unit of pull throughput, the origin weight is used to characterize the amount of resources used per unit of origin throughput, and the basic resource quantity is used to characterize the amount of resources used by non-streaming tasks in the streaming media server.

2. The performance evaluation model determination method according to claim 1, wherein the constructed performance evaluation model includes a linear regression term; the linear regression term is constructed based on each type of streaming throughput, streaming weight, origin throughput, origin weight, and basic resource quantity in the streaming media server; The linear regression term is used to characterize the linear relationship between the first variable and the second variable.

3. The performance evaluation model determination method according to claim 2, wherein the running data set includes a first running data set and a second running data set; Based on the operational data set, the determination of the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the performance evaluation model includes: Based on the first set of running data, the performance evaluation model is trained to obtain a first performance evaluation model, which is the trained performance evaluation model. Based on the second set of running data, the first performance evaluation model is validated to obtain the first residual corresponding to the first performance evaluation model. In response to the first residual being greater than a preset threshold, a constructed nonlinear correction term is obtained, and the nonlinear correction term is added to the first performance evaluation model to obtain a second performance evaluation model. The nonlinear correction term is used to characterize the nonlinear change relationship between the first variable and the second variable. The second performance evaluation model is iteratively trained until the second residual corresponding to the second performance evaluation model is less than or equal to a preset threshold, thereby obtaining the pull weight corresponding to each pull throughput, the return weight corresponding to the return throughput, and the basic resource quantity in the second performance evaluation model.

4. The performance evaluation model determination method according to claim 3, wherein the nonlinear correction term is constructed based on the streaming throughput corresponding to each streaming protocol in the streaming media server and the source throughput of the streaming media server.

5. The performance evaluation model determination method according to claim 4, constructing the nonlinear correction term based on the streaming throughput corresponding to each streaming protocol in the streaming media server and the origin throughput of the streaming media server, includes: The total throughput is obtained by adding the streaming throughput corresponding to each streaming protocol in the streaming media server and the origin throughput of the streaming media server. Based on the total throughput, the total throughput weight, and the order of the total throughput, the nonlinear correction term is constructed, wherein the order of the total throughput is greater than or equal to second order.

6. The performance evaluation model determination method according to claim 3, wherein iteratively training the second performance evaluation model until the second residual corresponding to the second performance evaluation model is less than or equal to a preset threshold includes: Based on the first set of running data, the second performance evaluation model is trained to obtain the trained second performance evaluation model. Based on the second set of running data, the trained second performance evaluation model is validated to obtain the second residual corresponding to the trained second performance evaluation model. In response to the second residual being greater than a preset threshold, the order and / or weight of the nonlinear correction term in the second performance evaluation model are adjusted, and the adjusted second performance evaluation model is retrained until the second residual is less than or equal to the preset threshold.

7. The performance evaluation model determination method according to claim 3, wherein the first set of running data includes the set of running data collected in a first time period, and the first time period includes peak request periods and non-peak request periods; The second set of operational data includes the set of operational data collected in the second time period, which includes peak request periods and non-peak request periods. in, The first time is different from the second time.

8. The performance evaluation model determination method according to claim 1, wherein the back-to-source throughput includes: First source throughput and second source throughput; wherein, the first source throughput is used to characterize the throughput of the streaming media server as a client pulling streams from the first server; the second source throughput is used to characterize the throughput of the streaming media server as a server being pulled streams from the second server. The back-source weights corresponding to the back-source throughput include: the first back-source weight corresponding to the first back-source throughput and the second back-source weight corresponding to the second back-source throughput.

9. The performance evaluation model determination method according to claim 8, wherein the streaming media server is an edge server in a content delivery network; The first server includes: an origin server or other edge servers belonging to the same cluster as the streaming media server; The second server includes other edge servers that belong to the same cluster as the streaming media server.

10. The performance evaluation model determination method according to claim 1, further comprising: Obtain the first weight corresponding to the first object, where the first object is any pull streaming protocol or origin return method in the streaming media server, and the first weight is the weight corresponding to the first object in the first performance evaluation model, which is determined based on the set of running data of the first object before optimization. Obtain the second weight corresponding to the first object. The second weight is the weight corresponding to the first object in the second performance evaluation model. The second performance evaluation model is determined based on the optimized running data set of the first object. Based on the first weight and the second weight, the weight change amount corresponding to the first object is determined, and the weight change amount is used to characterize the optimization degree of the first object.

11. The method for determining the performance evaluation model according to any one of claims 1-10, wherein the resource usage includes: CPU usage, memory usage, or disk read usage; The streaming throughput includes: streaming bandwidth or streaming packets per second; The origin throughput includes origin bandwidth or origin packets per second.

12. A performance evaluation model determination apparatus, comprising: The runtime data set acquisition module is used to acquire the runtime data set of the streaming media server. Each runtime data in the runtime data set includes the resource usage of the streaming media server, the streaming throughput corresponding to each streaming protocol in the streaming media server, and the source throughput of the streaming media server. The performance evaluation model acquisition module is used to acquire the constructed performance evaluation model, which is used to characterize the relationship between the first variable and the second variable. The first variable includes the resource usage of the streaming media server, and the second variable includes the streaming throughput corresponding to each streaming protocol in the streaming media server and the streaming media server's back-to-origin throughput. The model parameter determination module is used to determine the pull weight, the return weight, and the basic resource quantity corresponding to each pull throughput in the performance evaluation model based on the running data set. The pull weight is used to characterize the resource quantity used per unit pull throughput, the return weight is used to characterize the resource quantity used per unit return throughput, and the basic resource quantity refers to the resource quantity used by non-streaming tasks in the streaming media server.

13. An electronic device, the electronic device comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the performance evaluation model determination method as described in any one of claims 1-11.

14. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the performance evaluation model determination method as described in any one of claims 1-11.

15. A computer program product comprising a computer program that, when executed by a processor, implements the performance evaluation model determination method as described in any one of claims 1-11.

Images