Method for evaluating device performance, electronic device, storage medium, and program product

Abstract

The present disclosure relates to a device performance evaluation method, an electronic device, a storage medium and a program product. The method can include: collecting network performance data of at least two network performance indicators in a target time period, and determining a theoretical delay duration range according to the collected network performance data. And, obtaining a first actual delay duration set of the device to be tested in the target time period, the first actual delay duration set containing at least one first actual delay duration data. Then, at least according to the comprehensive difference between each first actual delay duration data in the first actual delay duration set and the theoretical delay duration range, a performance evaluation result of the device to be tested is generated. Through the technical scheme of the present disclosure, the specific influence of at least two network environment factors on delay can be effectively quantified, and the efficiency and operation convenience of device performance evaluation are improved.

Description

Technical Field

[0001] This disclosure relates to the field of performance evaluation technology, and in particular to a method for evaluating device performance, electronic devices, storage media, and program products. Background Technology

[0002] In applications such as video playback, online gaming, and real-time communication, network latency is a key factor affecting user experience. Excessive latency directly impacts the smoothness of displayed or transmitted content on the device, degrading the overall user experience. Therefore, accurately assessing a device's latency performance is crucial.

[0003] In related technologies, simple latency testing tools are typically used to obtain latency data for the device under test (DUT) and a reference device at the same time, and these latency data are manually compared to evaluate the latency performance of the DUT. This approach not only requires manual intervention for data collection and analysis, which is cumbersome and time-consuming, but also fails to finely distinguish the specific impact of various network environmental factors (such as bandwidth limitations, network jitter, packet loss rate, etc.) on latency. It merely mixes all interfering factors together for evaluation, resulting in inaccurate final evaluation results and making it difficult to provide valuable performance optimization suggestions. Summary of the Invention

[0004] This disclosure provides a method for evaluating device performance, an electronic device, a storage medium, and a program product to address the shortcomings of related technologies.

[0005] According to a first aspect of the present disclosure, a method for evaluating device performance is provided, comprising:

[0006] Collect network performance data for at least two network performance metrics within the target time period, and determine the theoretical latency range based on the collected network performance data;

[0007] Obtain a first set of actual delay durations of the device under test during the target time period, wherein the first set of actual delay durations contains at least one set of first actual delay duration data;

[0008] The performance evaluation result of the device under test is generated based on at least the comprehensive difference between each of the first actual delay duration data in the first actual delay duration set and the theoretical delay duration range.

[0009] Optionally, determining the theoretical latency range based on the collected network performance data includes: determining the weight coefficients corresponding to each network performance data; performing a weighted calculation on the collected network performance data based on the weight coefficients; and using the sum of the weighted calculation result and the preset basic latency as the theoretical latency range.

[0010] Optionally, determining the weight coefficients corresponding to each network performance data includes: determining the network performance index to which each network performance data belongs, and using the index weight coefficients corresponding to each network performance index as the weight coefficients of the network performance data under the corresponding network performance index; or, determining the target data category to which each network performance data belongs based on the data classification standard of the network performance index to which each network performance data belongs, and using the category weight coefficients corresponding to the target data category as the weight coefficients of the corresponding network performance data; the index weight coefficients and the category weight coefficients are obtained by analyzing historical network performance data and its corresponding historical actual latency data.

[0011] Optionally, the display screen of the device under test displays first actual delay duration data. The step of obtaining the first actual delay duration set of the device under test in the target time period includes: obtaining screen recording data of the device under test in the target time period; parsing the screen recording data of the device under test frame by frame to obtain multiple frames of images; extracting the first actual delay duration data contained in each frame of images from the multiple frames of images, and constructing the first actual delay duration set of the device under test based on the extracted first actual delay duration data.

[0012] Optionally, generating the performance evaluation result of the device under test based at least on the comprehensive difference between each first actual delay duration data in the first actual delay duration set and the theoretical delay duration range includes: filtering target first actual delay duration data from the first actual delay duration set, wherein the target first actual delay duration data includes first actual delay duration data within the theoretical delay duration range; calculating the proportion of the target first actual delay duration data in the first actual delay duration set; and determining the performance evaluation result based on the performance level corresponding to the calculated proportion.

[0013] Optionally, it further includes: determining the continuity of each remaining first actual delay duration data based on the timestamps corresponding to each remaining first actual delay duration data, wherein the remaining first actual delay duration data refers to the data in the first actual delay duration set excluding the target first actual delay duration data; determining the performance evaluation result based on the performance level corresponding to the calculated proportion includes: determining the performance evaluation result of the device under test as abnormal when the continuity indicates that the number of time-continuous remaining first actual delay durations reaches a preset number threshold; and determining the performance evaluation result based on the performance level corresponding to the calculated proportion when the continuity indicates that the number of time-continuous remaining first actual delay durations does not reach the preset number threshold.

[0014] Optionally, it further includes: obtaining a second set of actual latency durations for a reference device during the target time period, wherein the reference device and the device under test are connected to the same network; generating a performance evaluation result for the device under test based at least on the comprehensive difference between each first actual latency duration data in the first set of actual latency durations and the theoretical latency duration range includes: generating an actual latency duration difference based on the first set of actual latency durations of the device under test and the second set of actual latency durations of the reference device; and generating the performance evaluation result based on the actual latency duration difference and the comprehensive difference.

[0015] Optionally, generating the performance evaluation result based on the actual latency difference and the overall difference includes: determining the actual latency score corresponding to the actual latency difference based on the correspondence between latency difference and latency score; determining a first weight corresponding to the overall difference, a second weight corresponding to the actual latency score, and a third weight corresponding to the statistic, wherein the statistic is obtained based on the overall difference and the actual latency score; performing a weighted calculation on the overall difference, the actual latency score, and the statistic based on the first weight, the second weight, and the third weight; and generating the performance evaluation result based on the weighted calculation result, wherein the performance evaluation result includes the performance level or potential risk corresponding to the weighted calculation result.

[0016] Optionally, it may also include: displaying the network performance data and a first set of actual latency durations of the device under test on the client of the evaluation platform.

[0017] According to a second aspect of the present disclosure, an electronic device is provided, comprising:

[0018] processor;

[0019] Memory used to store processor-executable instructions;

[0020] The processor is configured to implement the method described in the embodiments of the first aspect above.

[0021] According to a third aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in the embodiments of the first aspect above.

[0022] According to a fourth aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in the embodiments of the first aspect above.

[0023] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0024] As can be seen from the above embodiments, this disclosure can obtain a first set of actual latency durations of the device under test (DUT) within a target time period, as well as network performance data for at least two network performance indicators within the target time period. Then, a theoretical latency duration range is determined based on the collected network performance data. Furthermore, a performance evaluation result for the DUT is generated based at least on the comprehensive difference between the first set of actual latency durations and the theoretical latency duration range.

[0025] Since the theoretical latency range is generated based on network performance data from at least two network performance metrics within the same network environment, it defines the range of optimal latency that a device should maintain under that environment. Therefore, by using the theoretical latency range as a benchmark for measuring device latency performance and comparing the first set of actual latency times with the theoretical latency range, the specific impact of at least two network environment factors on latency can be effectively quantified. In other words, this method not only identifies network performance metrics that significantly affect latency but also helps determine how these metrics collectively affect overall latency performance, thus significantly improving the accuracy of performance evaluation results. Furthermore, this evaluation method can automate the collection of network performance data and actual latency data without manual intervention, improving the efficiency and ease of use of device performance evaluation.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating an evaluation of device performance in a related art according to embodiments of the present disclosure.

[0029] Figure 2 This is a schematic diagram of the architecture of a device performance evaluation platform according to an embodiment of the present disclosure.

[0030] Figure 3 This is a schematic flowchart illustrating a method for evaluating device performance according to an embodiment of the present disclosure.

[0031] Figure 4 This is a schematic diagram of a hardware architecture for evaluating device performance, according to an embodiment of the present disclosure.

[0032] Figure 5 This is a schematic flowchart illustrating an evaluation of device performance according to an embodiment of the present disclosure.

[0033] Figure 6 This is a schematic block diagram illustrating an apparatus for evaluating device performance according to an embodiment of the present disclosure.

[0034] Figure 7 This is a schematic block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0035] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0036] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. The singular forms “a” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0037] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of embodiments of this disclosure, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0038] For the sake of brevity and ease of understanding, this document uses the terms "greater than" or "less than", "higher than" or "lower than" to describe size relationships. However, it will be understood by those skilled in the art that the term "greater than" also includes the meaning of "greater than or equal to", and "less than" also includes the meaning of "less than or equal to"; the term "higher than" also includes the meaning of "higher than or equal to", and "lower than" also includes the meaning of "lower than or equal to".

[0039] In related technologies, simple latency testing tools are typically relied upon to acquire latency data of the device under test (DUT) and a reference device over the same time period, and these latency data are manually compared to evaluate the latency performance of the DUT. For example, Figure 1 This is a schematic flowchart illustrating the delay performance of a detection device in a related art, provided as an exemplary embodiment. Please refer to... Figure 1 Assuming a gaming scenario, the process of testing the latency performance of the device under test includes the following steps:

[0040] S101: The device under test and the reference device start the same game simultaneously, and screen recordings are performed on both devices within the same time period to obtain video data.

[0041] In this embodiment, the device under test and the reference device are connected to the same network.

[0042] S102: Game latency detection for the device under test.

[0043] The video data obtained in S101 is imported into the delay detection tool for analysis to obtain the delay data of the device under test and the delay data of the reference device.

[0044] S103: Generate delayed scores.

[0045] Through manual statistical analysis, and based on a pre-defined scoring template, the latency score of the device under test is determined using its latency data. Similarly, the latency score of the reference device is obtained.

[0046] S104: Compare whether the delay score of the device under test is greater than the delay score of the reference device.

[0047] If yes, proceed to S105, the delay performance of the device under test has passed the test; if no, proceed to S106, the delay performance of the device under test has failed the test.

[0048] S105: The delay performance of the device under test has passed the test.

[0049] This indicates that the device under test has good performance.

[0050] S106: The delay performance of the device under test failed the test.

[0051] This indicates that the device under test has poor performance.

[0052] Clearly, the above process fails to finely differentiate the specific impacts of various network environmental factors on latency; it merely mixes all interfering factors together for evaluation, ultimately yielding only a rough assessment result of either passing or failing the test, resulting in poor accuracy. Furthermore, this process requires manual intervention, making it cumbersome and inefficient.

[0053] In view of this, this disclosure proposes a method for evaluating device performance, which can automatically evaluate device performance and effectively quantify the specific impact of various network environmental factors on latency, thereby improving the accuracy of the evaluation results.

[0054] The following describes one or more embodiments of this disclosure in detail.

[0055] The embodiments of this disclosure can be applied to electronic devices, including but not limited to smartphones, desktop computers, tablets, laptops, e-book readers, smartwatches, and smart bracelets. One or more embodiments of this disclosure are not intended to limit this. During operation, the electronic device can run a device performance evaluation platform to generate performance evaluation results for the device under test. The application for the device performance evaluation platform can be pre-installed on the electronic device, allowing the platform to be started and run. Alternatively, when using technologies such as HTML5, it is not necessary to install the corresponding application on the electronic device to obtain and run the device performance evaluation platform.

[0056] In another embodiment, the device performance evaluation platform may include, for example, Figure 2 The diagram shows a server 21, a network 22, and an electronic device 23. During operation, server 21 can run the server-side program of the device performance evaluation platform to perform related data processing functions. Meanwhile, electronic device 23 can run the client-side program of the device performance evaluation platform to perform related data acquisition and human-computer interaction functions. Thus, the server 21 and electronic device 23 work together to implement the device performance evaluation scheme. It should be noted that server 21 can also communicate with multiple electronic devices to perform performance evaluations on multiple electronic devices.

[0057] Server 21 can be a physical server containing a single host, or it can be a virtual server hosted by a host cluster. Electronic device 23 can be any type of device, such as a PC, tablet, laptop, PDA (Personal Digital Assistants), mobile phone, wearable device (such as smart glasses, smartwatches, etc.), etc., and this specification does not limit this to any of the embodiments described herein. The network 22 for interaction between electronic device 23 and server 21 can include various types of wired or wireless networks. In one embodiment, network 22 can include the Public Switched Telephone Network (PSTN) and the Internet.

[0058] The following example illustrates how an evaluation platform can be used to perform the device performance evaluation method provided in this disclosure. Figure 3 This is a schematic flowchart illustrating a method for evaluating device performance according to embodiments of the present disclosure. Figure 3 As shown, the method may include the following steps:

[0059] S301: Collect network performance data for at least two network performance metrics within the target time period, and determine the theoretical latency range based on the collected network performance data.

[0060] First, it should be noted that this disclosed solution is applicable to evaluating device performance in scenarios requiring low latency and high stability, such as smart homes, online games, telemedicine, and high-definition video streaming. This embodiment does not specifically limit the length of the target time period or the specific moments involved in the target time period.

[0061] Network performance metrics refer to key parameters used to evaluate the quality of network services, and may include, but are not limited to, network bandwidth, network jitter, packet loss rate, duty cycle, wireless signal quality, and server response time. The network performance data for any network performance metric can be understood as its specific numerical value; for example, bandwidth network performance data could be 5 Mbps (megabits per second). The evaluation platform can collect network performance data for at least two network performance metrics within a target time period. This disclosure does not limit the specific method of collecting network performance data. For example, bandwidth monitoring tools can be used to measure network bandwidth network performance data in real time. Alternatively, the client of the evaluation platform can periodically send data packets (such as ping packets) to the server of the evaluation platform to obtain server response time network performance data. Another example is that duty cycle network performance data can be obtained by analyzing network load in real time.

[0062] Furthermore, the theoretical latency range is determined based on the collected network performance data. This theoretical latency range defines the interval of optimal latency under the network environment reflected by the collected network performance data. For example, a machine learning model can be established and trained to predict the theoretical latency range under different network environment conditions. The input to the machine learning model is network performance data for multiple network performance indicators, and the output is the theoretical latency range. During the training phase of the machine learning model, machine learning algorithms, regression analysis, and other methods can be used to learn the relationship or trend between network performance data and the theoretical latency range from historical network performance data and the actual latency range corresponding to the historical network performance data. Then, the collected network performance data is input into the trained machine learning model to obtain the theoretical latency range. Alternatively, historical network performance data and expert experience / knowledge in the field of network performance research can be analyzed to formulate empirical formulas or rules. For example, for certain types of network applications, there is a certain proportional relationship between the known theoretical latency range and bandwidth and packet loss rate; therefore, formulas or rules can be constructed based on this proportional relationship. The above are merely examples, and this disclosure does not limit the specific method of determining the theoretical latency range.

[0063] In one embodiment, different network performance data have varying impacts on latency. For example, a high packet loss rate and significant jitter typically lead to higher latency. Therefore, different weighting coefficients can be assigned to each network performance data point. This weighting coefficient indicates the degree of influence of the corresponding network performance data on the theoretical latency range; a larger weighting coefficient indicates a greater degree of influence on the theoretical latency range. It should be noted that statistical analysis can be performed on a large amount of historical network performance data to obtain the weighting coefficients corresponding to each network performance data point. Alternatively, the weighting coefficients corresponding to each network performance data point can be set independently based on human experience or expert knowledge; this disclosure does not impose any limitations on this.

[0064] After determining the weight coefficients corresponding to each network performance data, the collected network performance data are weighted according to these weight coefficients, and the weighted calculation result is added to the preset base delay duration to obtain the theoretical delay duration range. The base delay duration can be used to cover any uncertainties or errors that may exist in the weighted calculation result, thereby improving the reliability of the theoretical delay duration range. The base delay duration can be obtained through statistical analysis of a large amount of historical network performance data, or it can be set by human experience or expert knowledge. For example, the base delay duration can be assumed to be ±0.5ms. The calculation principle of the theoretical delay duration range can be referred to the following formula (1):

[0065]

[0066] In formula (1), Theoretical Delay represents the theoretical delay range, Base Delay represents the preset base delay, and Interference Factor... i The weight represents the network performance data of the i-th data point (1≤i≤n, where i and n are both positive integers). i The weight coefficients that represent the i-th network performance data.

[0067] In this embodiment, the impact of different network performance data on network latency is dynamically adjusted by weighting coefficients, thus obtaining an accurate theoretical latency range. This provides a more accurate latency benchmark for subsequent performance evaluation, thereby improving the accuracy of performance evaluation results.

[0068] In one embodiment, different network performance metrics have varying impacts on latency. Therefore, the weighting coefficient for each network performance data point can be determined based on the network performance metric to which it belongs. For example, first, the network performance metric to which each network performance data point belongs is determined. Then, the weighting coefficients corresponding to each network performance metric are used as the weighting coefficients for the network performance data under that metric. For instance, the weighting coefficient for packet loss rate can be determined as the weighting coefficient for the network performance data under the packet loss rate metric. This method of determining weighting coefficients accurately reflects the importance of each network performance metric to the performance evaluation results, thereby distinguishing the specific impact of various network performance metrics on latency, and generating valuable performance optimization suggestions based on these specific impacts.

[0069] Furthermore, since network performance data under the same network performance metric can vary, specific data classification standards can be set for each network performance metric. The method for setting data classification standards can be referenced in the aforementioned basic latency duration, and will not be elaborated here. For example, for the network performance metric of bandwidth, it can be classified into high-bandwidth network performance data and low-bandwidth network performance data based on the numerical value. For the network performance metric of server response time, it can be classified into fast-response network performance data, medium-response network performance data, and slow-response network performance data based on the numerical value. Alternatively, it can be classified into stable-response network performance data and fluctuating-response network performance data based on the quality of service (QoS). The above are merely examples, and this disclosure does not impose any specific limitations on the data classification standards for each network performance metric.

[0070] Thus, multiple categories of network performance data can exist under the same network performance metric. Different categories of network performance data have varying degrees of impact on network latency; for example, high duty cycle network performance data leads to longer latency, while low duty cycle network performance data leads to shorter latency. Therefore, category weight coefficients can be set for each data category. It should be noted that the "metric weight coefficients" and "category weight coefficients" mentioned in this embodiment can be determined through statistical analysis of historical network performance data and the corresponding historical actual latency data. For example, machine learning models can be used to learn the correlation between historical network performance data and historical actual latency data to determine the metric weight coefficients and category weight coefficients from this correlation.

[0071] For any network performance data, we can first determine the network performance index to which the data belongs, and then, based on the data classification criteria of that index, determine the target data category to which the data belongs. Then, the category weight coefficient corresponding to the target data category is used as the weight coefficient for the network performance data. Since network performance data under the same index can vary significantly, this method, by classifying and assigning different category weight coefficients to different data categories, achieves a more refined data segmentation. This more accurately reflects the impact of differences between different network performance data on the theoretical latency range, thus contributing to more precise performance evaluation results.

[0072] S302: Obtain the first actual delay duration set of the device under test in the target time period, wherein the first actual delay duration set contains at least one first actual delay duration data.

[0073] The device under test (DUT) can be any device, and this disclosure does not limit the model, quality, or other aspects of the DUT. For example, the DUT and the evaluation platform can be connected to the same network. This avoids additional latency caused by differences in network environment and allows for more accurate detection of the DUT's network performance. For instance, the DUT and the evaluation platform can be connected to the same wireless access point (AP), the same switch, or other means to achieve the same network connection. There are many ways to obtain the first actual latency set of the DUT within a target time period. For example, network monitoring software such as Wireshark or SolarWinds can be deployed on the DUT to obtain the first actual latency data. Alternatively, a custom script can be written to periodically send data packets and measure the round-trip time to obtain the first actual latency data.

[0074] In one embodiment, the following function can be integrated into the device under test (DUT): displaying first actual delay duration data in real time on the DUT's display screen. In this case, the evaluation platform can acquire screen recording data of the DUT during a target time period, which includes images displayed on the DUT's display screen within the target time period. Then, the evaluation platform can parse the acquired screen recording data frame by frame to obtain multiple frames of images. Each frame of these multiple frames contains the first actual delay duration data. Then, the first actual delay duration data contained in each frame can be extracted from these multiple frames of images, and the extracted first actual delay duration data constitutes the first actual delay duration set of the DUT. This disclosure does not specifically limit the extraction method of the first actual delay duration data; for example, it can be extracted using optical character recognition (OCR) and computer vision technologies. This method can obtain the first actual delay duration data of the DUT without requiring a high level of technical theoretical foundation, and the solution has a wider range of applications. On the other hand, screen recording of the device under test can capture changes in every frame, thus obtaining minute changes in the first actual latency data in greater detail, improving the accuracy and comprehensiveness of the first actual latency data, and facilitating more accurate performance evaluation results in the future.

[0075] Alternatively, the first actual latency data can be obtained from the latency log of the device under test (DUT). The DUT automatically generates detailed latency logs during operation, recording the first actual latency data for each frame. Therefore, the latency log of the DUT can be accessed through relevant interfaces, and the first actual latency data for the target time period can be extracted from the latency log to obtain the first actual latency set. This method eliminates the need for screen recording, making the operation simpler.

[0076] It should be emphasized that this disclosure does not impose any special restrictions on the execution order of S301 and S302.

[0077] S303: Generate a performance evaluation result for the device under test based at least on the comprehensive difference between each of the first actual delay duration data in the first actual delay duration set and the theoretical delay duration range.

[0078] Each actual delay duration data point in the first actual delay duration set is compared with the theoretical delay duration range to obtain the difference between each actual delay duration data point and the theoretical delay duration range. Based on these differences, a comprehensive difference between the first actual delay duration set and the theoretical delay duration range is determined. For example, the difference between each actual delay duration data point and the upper limit of the theoretical delay duration range can be calculated separately, and the statistical measures of these differences (such as the mean, weighted average, median, etc.) can be used as the comprehensive difference. Furthermore, the performance evaluation results of the device under test can be generated based on the comprehensive difference.

[0079] In the above embodiments, since the theoretical latency range is generated based on network performance data from at least two network performance indicators, this theoretical latency range defines the range of optimal latency that the device should maintain in this network environment. Therefore, by using the theoretical latency range as a benchmark for measuring device latency performance and comparing the first set of actual latency with the theoretical latency range, the specific impact of at least two network environmental factors on latency can be effectively quantified. In other words, the above method not only identifies network performance indicators that significantly affect latency but also helps determine how these network performance indicators collectively affect overall latency performance, thereby significantly improving the accuracy of performance evaluation results. Furthermore, the above evaluation method can automatically collect network performance data and actual latency data without manual operation, improving the efficiency and ease of operation of device performance evaluation.

[0080] In one embodiment, the first actual delay duration data contained in the first actual delay duration set of the device under test can be filtered to select the first actual delay duration data within the theoretical delay duration range (hereinafter referred to as "target first actual delay duration data"). Then, the proportion of the target first actual delay duration data in the first actual delay duration set is calculated, and the performance evaluation result of the device under test is determined according to the performance level corresponding to the calculated proportion. A higher proportion indicates more first actual delay duration data within the theoretical delay duration range, meaning better performance of the device under test. For example, multiple performance levels can be set, each corresponding to a proportion range of the target first actual delay duration data. Table 1 is a schematic table of performance levels and proportion ranges provided in an exemplary embodiment, where P represents the proportion of the target first actual delay duration data. Please refer to Table 1:

[0081] Table 1

[0082] Percentage range performance level 0＜P≤20％ Performance Level 1 20％＜P≤60％ Performance level 2 60％＜P≤80％ Performance level 3 80％＜P≤1 Performance level 4

[0083] According to Table 1, the higher the performance level, the better the performance of the device under test.

[0084] In this embodiment, the performance level of the device under test (DUT) is determined by calculating the proportion of the target's first actual latency data in the first actual latency set, and the performance evaluation result is determined based on the performance level. Compared with the two coarse evaluation results of passing and failing the test in related technologies, this embodiment introduces multiple performance levels. This refined classification method can more accurately reflect the actual performance level of the DUT, providing more detailed and comprehensive performance evaluation results, which helps to provide strong support for subsequent device performance optimization and troubleshooting.

[0085] In one embodiment, each first actual delay duration data has a corresponding timestamp, which is used to characterize the acquisition time of the first actual delay duration data. All first actual delay duration data in the first actual delay duration set can be sorted according to the timestamps of each first actual delay duration data. In this embodiment, the data in the first actual delay duration set excluding the target first actual delay duration data is referred to as "remaining first actual delay duration data". When generating the performance evaluation results of the device under test, the continuity of the remaining first actual delay duration data can be determined based on the timestamps of the remaining first actual delay durations. "Continuity" means that when two remaining first actual delay duration data are in adjacent positions after sorting, it means that these two remaining first actual delay duration data are continuous in time.

[0086] If the continuity indicates that the number of consecutive remaining actual delay durations reaches a preset threshold, it means that the actual delay duration of the device under test (DUT) is not within the theoretical delay duration range within a certain time interval. In this case, the performance evaluation result of the DUT can be directly determined as abnormal. If the continuity indicates that the number of consecutive remaining actual delay durations does not reach a preset threshold, the performance evaluation result is further determined based on the performance level corresponding to the proportion of the target actual delay duration data.

[0087] In this embodiment, the performance of the device under test is first determined based on the continuity of the remaining first actual delay duration data. If the continuity indicates normal performance, the performance evaluation result is further determined based on the proportion of the target first actual delay duration data. This hierarchical evaluation method helps to comprehensively and accurately determine the performance evaluation result and avoids interference from sudden data on the performance evaluation result.

[0088] In one embodiment, a reference device may be introduced. The reference device can be understood as a standardized device whose performance and characteristics have been rigorously defined and verified, and can serve as a benchmark or reference standard during technical research, development, or testing.

[0089] In this embodiment, the reference device and the device under test (DUT) are connected to the same network. A second set of actual latency durations for the reference device during the target time period is obtained. This second set of actual latency durations contains at least one second actual latency duration data point. The method for obtaining the second set of actual latency durations can be referred to in relevant embodiments of the DUT, and will not be elaborated here. Then, the second set of actual latency durations for the reference device is compared with the first set of actual latency durations for the DUT to obtain the actual latency duration difference. For example, the second actual latency duration data at any given time can be compared with the first actual latency duration data. Based on the comparison result, it can be determined whether the performance of the DUT at that time is superior to that of the reference device. Then, the actual latency duration difference is determined based on the performance superiority / inferiority at each time point within the target time period. Alternatively, statistical measures (such as median, standard deviation, mean, etc.) can be taken from the first set of actual latency durations and the second set of actual latency durations respectively, and the difference between these two statistical measures can be used as the actual latency duration difference. The above are merely examples, and this disclosure does not limit the specific method of generating the actual latency duration difference.

[0090] Furthermore, performance evaluation results can be generated based on the aforementioned differences in actual latency duration and overall differences. For example, if the actual latency duration difference indicates that the performance of the device under test (DUT) is inferior to that of the reference device, and the overall differences indicate that the first actual latency duration data of the DUT is significantly higher than the theoretical latency duration range, then the performance evaluation result of the DUT is "very low performance." If the actual latency duration difference indicates that the performance of the DUT is inferior to that of the reference device, and the overall differences indicate that the difference between the first actual latency duration data of the DUT and the theoretical latency duration range is small, then the performance evaluation result of the DUT is "low performance," and further analysis of the specific reasons is needed.

[0091] This embodiment introduces a reference device, comparing the first set of actual latency durations of the device under test (DUT) with the second set of actual latency durations of the reference device. Simultaneously, the first set of actual latency durations of the DUT is compared with the theoretical latency duration range, achieving a comprehensive performance evaluation from different levels / angles. The theoretical latency duration range represents the latency duration under optimal expected performance; comparing it with the theoretical latency duration range clearly shows the gap between the DUT's performance and its optimal expected performance. The second set of actual latency durations of the reference device provides a more realistic benchmark; comparing it with the second set of actual latency durations of the reference device reveals the DUT's performance relative to similar devices, offering more practical guidance. Combining these two comparison processes yields more accurate and reliable performance evaluation results.

[0092] In one embodiment, the delay duration difference can be presented in the form of a score. For example, a correspondence between the delay duration difference and the delay score can be established. For instance, the larger the delay duration difference, the smaller the corresponding delay score. This disclosure does not limit the specific correspondence. Then, the actual delay score corresponding to the actual delay duration difference can be determined according to the aforementioned correspondence. Similarly, the comprehensive difference can also be presented in the form of a score. Furthermore, the score of the comprehensive difference and the statistical measure of the actual delay score are calculated. The statistical measure can be the median, standard deviation, weighted average, etc. The first weight corresponding to the comprehensive difference, the second weight corresponding to the actual delay score, and the third weight corresponding to the statistical measure are determined. The score of the comprehensive difference, the actual delay score, and the statistical measure are weighted according to these three weights, and the weighted calculation result is used as the risk score of the hardware and software system of the device under test. The calculation principle can be referred to the following formula (2):

[0093] R = w1 * P th +w2*s+w3*(σ(P th )+σ(s))(2)

[0094] In formula (2), R represents the weighted calculation result, i.e., the risk score (maximum score is 100%). w1, w2, and w3 represent the first weight, the second weight, and the third weight, respectively. P th The score representing the overall difference. σ(P) th ) represents the standard deviation of the overall difference score. s represents the actual latency score. σ(s) represents the standard deviation of the actual latency score. It should be noted that the first, second, and third weights can be obtained through statistical analysis of a large amount of historical network performance data and corresponding historical performance data, or they can be set by relevant testers according to actual needs. This disclosure does not impose any restrictions on this.

[0095] Furthermore, a performance evaluation result is generated based on the risk score. It is important to emphasize that in this embodiment, the performance evaluation result refers to the risk assessment result of the software and hardware system in the device under test (DUT). This risk assessment result can include the current risk level or potential risk of the software and hardware system. In other words, the risk status of the current software and hardware system of the DUT can be determined based on the risk score. For example, if the risk score is greater than 15% (out of 100%), the current software and hardware system of the DUT can be determined to have a high risk, belonging to a high-risk software and hardware system. If the risk score is less than or equal to 15% but greater than 5%, the current software and hardware system of the DUT can be determined to be at a medium risk level, belonging to a medium-risk software and hardware system. If the risk score is less than or equal to 5%, the current software and hardware system of the DUT can be determined to have a low risk, belonging to a low-risk software and hardware system. In this way, the risk status of the software and hardware system can be monitored, and corresponding optimizations can be performed when the system risk is high.

[0096] In this embodiment, by dynamically adjusting the weights to adjust the impact of the overall difference and the actual latency difference on the performance evaluation results, the deviation between the overall difference and the actual latency difference can be balanced to a certain extent, which helps to obtain more flexible, accurate and comprehensive performance evaluation results.

[0097] In one embodiment, the evaluation platform's client can display data involved in the evaluation process to the user in real time, such as collected network performance data, the first actual latency set of the device under test, comprehensive differences, performance evaluation results, etc. For example, the data involved in the evaluation process can be displayed through trend charts, line charts, bar charts, or other forms (such as videos, text, tables). Additionally, the performance evaluation results of the device under test can be reported to the evaluation platform's server to achieve automated management of the performance evaluation results.

[0098] Through real-time visualization, users can intuitively understand the network performance data and the first actual latency set of the device under test involved in the evaluation process, which improves the transparency and understandability of various data and helps to optimize the user experience.

[0099] Figure 4 This is a schematic diagram of a hardware architecture for evaluating device performance, provided as an exemplary embodiment. Please refer to [link / reference]. Figure 4 Taking a gaming scenario as an example, the device under test (DUT), the reference device, and the evaluation platform are connected to the same access point (AP) to ensure consistency and fairness of network conditions. The DUT and the reference device are each deployed with the same latency detection model, which is used to obtain a first set of actual latency durations for the DUT and a second set of actual latency durations for the reference device. The evaluation platform's server-side deploys a network performance model to output the theoretical latency range. The evaluation platform's client-side deploys a risk assessment model to generate performance evaluation results for the DUT based on the first set of actual latency durations and the theoretical latency range.

[0100] Combination Figure 4 The hardware architecture shown is Figure 5 This is a schematic flowchart illustrating an exemplary embodiment for evaluating device performance. Figure 5 As shown, the process includes the following steps:

[0101] S501: Obtain the first actual delay duration set of the device under test and the second actual delay duration set of the reference device within the target time period using the delay detection models deployed on the device under test and the reference device, respectively.

[0102] In a gaming scenario, the device under test and the reference device are configured with the same version of the game, and the actual latency data will be displayed in real time on the game screen. In this embodiment, this step can be divided into two sub-steps, S501-1 and S501-2. Among them,

[0103] S501-1: Perform game latency detection on the device under test and the reference device respectively.

[0104] For example, the game is started simultaneously on the device under test and the reference device, and the screen of both devices is recorded to obtain the screen recording data of the device under test and the reference device during the target time period. The screen recording data of the device under test and the reference device are then input into their respective latency detection models.

[0105] S501-2: Obtain the set of actual delay durations output by the delay detection model.

[0106] Taking the device under test (DUT) as an example, the latency detection model deployed on the DUT parses the input screen recording data frame by frame, obtaining multiple frames of images. Then, it extracts the first actual latency duration data contained in each frame of these multiple frames. The extracted first actual latency duration data are then combined to form the first actual latency duration set of the DUT and output. Similarly, the second actual latency duration set of the reference device can be obtained through the latency detection model deployed on the reference device.

[0107] S502: The evaluation platform's server determines the theoretical latency range within the target time period.

[0108] In this embodiment, this step can be divided into two sub-steps, S502-1 and S502-2. Among them,

[0109] S502-1: Collect network performance data for at least two network performance indicators within the target time period and input the collected network performance data into the network performance model.

[0110] S502-2: Obtain the theoretical latency range output by the network performance model.

[0111] S503: The device under test and the reference device send their respective actual delay duration sets to the client of the evaluation platform, and the server of the evaluation platform sends the theoretical delay duration range to the client of the evaluation platform.

[0112] S504: The client of the assessment platform uses the risk assessment model to output the performance assessment results of the device under test.

[0113] The theoretical delay range, the first set of actual delay durations, and the second set of actual delay durations are input into the risk assessment model to obtain the performance evaluation results of the device under test. The internal operating logic of the risk assessment model may include the following five sub-steps (S504-1 to S504-5):

[0114] S504-1: Determine whether the proportion of the target first actual delay duration data of the device under test in the first actual delay duration set is greater than the preset proportion threshold.

[0115] The target first actual latency data refers to the first actual latency data within the theoretical latency range. The risk assessment model calculates the proportion of the target first actual latency data in the set of first actual latency data of the device under test and compares this proportion with a preset proportion threshold. If the proportion is greater than or equal to the preset proportion threshold, it jumps to S504-2; if the proportion is less than the preset proportion threshold, it jumps to S504-3.

[0116] S504-2: Determine whether the actual delay score of the device under test is greater than the actual delay score of the reference device.

[0117] The risk assessment model determines the actual latency score of the device under test (DUT) based on a first set of actual latency durations, and determines the actual latency score of the reference device based on a second set of actual latency durations. For example, the risk assessment model can learn from a large set of historical latency durations to obtain a mapping relationship between latency scores and latency duration sets. Then, during the assessment process, the actual latency scores of the DUT and the reference device are determined based on this mapping relationship.

[0118] If the actual delay score of the device under test is greater than or equal to the actual delay score of the reference device, proceed to S505. If the actual delay score of the device under test is less than the actual delay score of the reference device, proceed to S504-4 for risk analysis.

[0119] S504-3: Determine whether the actual delay score of the device under test is greater than the actual delay score of the reference device.

[0120] If the actual latency score of the device under test is greater than or equal to the actual latency score of the reference device, proceed to S504-4 for risk analysis. If the actual latency score of the device under test is less than the actual latency score of the reference device, create a JIRA task.

[0121] S504-4: Risk analysis is conducted based on the comparison results of the actual delay scores of the device under test and the reference device, as well as the comparison results of the proportion of the target first actual delay duration data with the preset proportion threshold.

[0122] Table 2 is a schematic table illustrating a risk analysis provided in an exemplary embodiment. Please refer to Table 2:

[0123] Table 2

[0124]

[0125]

[0126] S504-5: Generate corresponding JIRA tasks based on the risk analysis results of S504-4.

[0127] S505: Report the JIRA task and / or risk analysis results to the server of the evaluation platform so that the server can generate the performance evaluation results of the device under test.

[0128] Corresponding to the embodiments of the aforementioned equipment performance evaluation method, this disclosure also provides embodiments of an equipment performance evaluation apparatus.

[0129] Please see Figure 6 , Figure 6 This is a schematic block diagram of a device performance evaluation apparatus 600 provided in an exemplary embodiment. The apparatus may include: a data acquisition unit 601, an acquisition unit 602, and a generation unit 603.

[0130] The acquisition unit 601 is configured to acquire network performance data of at least two network performance indicators within a target time period, and determine the theoretical latency range based on the acquired network performance data.

[0131] The acquisition unit 602 is configured to acquire a first set of actual delay durations of the device under test during the target time period, wherein the first set of actual delay durations includes at least one set of first actual delay duration data.

[0132] The generation unit 603 is configured to generate a performance evaluation result of the device under test based at least on the comprehensive difference between each first actual delay duration data in the first actual delay duration set and the theoretical delay duration range.

[0133] Optionally, the acquisition unit 601 is specifically used to: determine the weight coefficients corresponding to each network performance data; perform weighted calculations on the acquired network performance data based on the weight coefficients; and use the sum of the weighted calculation results and the preset basic delay duration as the theoretical delay duration range.

[0134] Optionally, the acquisition unit 601 is specifically used for: determining the weight coefficients corresponding to each network performance data, including: determining the network performance index to which each network performance data belongs, and using the index weight coefficients corresponding to each network performance index as the weight coefficients of the network performance data under the corresponding network performance index; or, determining the target data category to which each network performance data belongs according to the data classification standard of the network performance index to which each network performance data belongs, and using the category weight coefficients corresponding to the target data category as the weight coefficients of the corresponding network performance data; the index weight coefficients and the category weight coefficients are obtained by analyzing historical network performance data and its corresponding historical actual latency data.

[0135] Optionally, the display screen of the device under test displays first actual delay duration data, and the acquisition unit 602 is specifically used to: acquire screen recording data of the device under test during the target time period; parse the screen recording data of the device under test frame by frame to obtain multiple frames of images; extract the first actual delay duration data contained in each frame of images from the multiple frames of images, and construct the first actual delay duration set of the device under test based on the extracted first actual delay duration data.

[0136] Optionally, the generation unit 603 is specifically used to: filter out target first actual latency data from the first actual latency set, the target first actual latency data including first actual latency data within the theoretical latency range; calculate the proportion of the target first actual latency data in the first actual latency set; and determine the performance evaluation result based on the performance level corresponding to the calculated proportion.

[0137] Optional, also includes:

[0138] The continuity determination unit 604 is configured to determine the continuity of each remaining first actual delay duration data according to the timestamps corresponding to each remaining first actual delay duration data, wherein the remaining first actual delay duration data is the data in the first actual delay duration set excluding the target first actual delay duration data;

[0139] The step of determining the performance evaluation result based on the performance level corresponding to the calculated percentage includes: when the number of consecutive remaining first actual delay durations in the continuity condition reaches a preset number threshold, determining the performance evaluation result of the device under test as performance abnormal; when the number of consecutive remaining first actual delay durations in the continuity condition does not reach the preset number threshold, determining the performance evaluation result based on the performance level corresponding to the calculated percentage.

[0140] Optionally, the acquisition unit 602 is further configured to acquire a second set of actual delay durations of the reference device during the target time period, wherein the reference device and the device under test are connected to the same network;

[0141] The generation unit 603 is specifically used to: generate an actual delay duration difference based on the first actual delay duration set of the device under test and the second actual delay duration set of the reference device; and generate the performance evaluation result based on the actual delay duration difference and the comprehensive difference.

[0142] Optionally, generating the performance evaluation result based on the actual latency difference and the overall difference includes: determining the actual latency score corresponding to the actual latency difference based on the correspondence between latency difference and latency score; determining a first weight corresponding to the overall difference, a second weight corresponding to the actual latency score, and a third weight corresponding to the statistic, wherein the statistic is obtained based on the overall difference and the actual latency score; performing a weighted calculation on the overall difference, the actual latency score, and the statistic based on the first weight, the second weight, and the third weight; and generating the performance evaluation result based on the weighted calculation result, wherein the performance evaluation result includes the performance level or potential risk corresponding to the weighted calculation result.

[0143] Optionally, the device further includes:

[0144] The display unit 605 is configured to display the network performance data and a first set of actual latency durations of the device under test on a client of the evaluation platform.

[0145] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments of the relevant methods, and will not be elaborated upon here.

[0146] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0147] Figure 7This is a schematic block diagram illustrating an electronic device according to embodiments of the present disclosure. For example, the electronic device 700 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0148] Reference Figure 7 The electronic device 700 may include one or more of the following components: a processing component 702, a memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0149] Processing component 702 typically controls the overall operation of electronic device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.

[0150] Memory 704 is configured to store various types of data to support the operation of electronic device 700. Examples of this data include instructions for any application or method operating on electronic device 700, contact data, phonebook data, messages, pictures, videos, etc. Memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0151] Power supply component 706 provides power to various components of electronic device 700. Power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 700.

[0152] Multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0153] Audio component 710 is configured to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) configured to receive external audio signals when electronic device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 704 or transmitted via communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.

[0154] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0155] Sensor assembly 714 includes one or more sensors for providing state assessments of various aspects of electronic device 700. For example, sensor assembly 714 can detect the on / off state of electronic device 700, the relative positioning of components such as the display and keypad of electronic device 700, changes in position of electronic device 700 or a component of electronic device 700, the presence or absence of user contact with electronic device 700, orientation or acceleration / deceleration of electronic device 700, and temperature changes of electronic device 700. Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 714 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0156] Communication component 716 is configured to facilitate wired or wireless communication between electronic device 700 and other devices. Electronic device 700 can access wireless networks based on communication standards, such as WiFi, 2G or 3G, 4G LTE, 7G NR, or combinations thereof. In one exemplary embodiment, communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 716 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0157] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described in any of the above embodiments.

[0158] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, which can be executed by a processor 720 of an electronic device 700 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0159] In an exemplary embodiment, this disclosure also provides a computer program product including a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any of the above embodiments.

[0160] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0161] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for evaluating equipment performance, characterized in that, The method includes: Collect network performance data for at least two network performance metrics within the target time period, and determine the theoretical latency range based on the collected network performance data; Obtain a first set of actual delay durations of the device under test during the target time period, wherein the first set of actual delay durations contains at least one set of first actual delay duration data; The performance evaluation result of the device under test is generated based on at least the comprehensive difference between each of the first actual delay duration data in the first actual delay duration set and the theoretical delay duration range.

2. The method according to claim 1, characterized in that, The process of determining the theoretical latency range based on the collected network performance data includes: Determine the weighting coefficients for each network performance data point; The collected network performance data is weighted and calculated based on the aforementioned weighting coefficients. The sum of the weighted calculation result and the preset basic delay duration is used as the theoretical delay duration range.

3. The method according to claim 2, characterized in that, The determination of the weight coefficients corresponding to each network performance data includes: Determine the network performance index to which each network performance data belongs, and use the index weight coefficient corresponding to each network performance index as the weight coefficient of the network performance data under the corresponding network performance index; or... Based on the data classification standard of the network performance index to which each network performance data belongs, determine the target data category to which each network performance data belongs, and use the category weight coefficient corresponding to the target data category as the weight coefficient of the corresponding network performance data. The index weight coefficient and the category weight coefficient are obtained by analyzing historical network performance data and their corresponding historical actual latency data.

4. The method according to claim 1, characterized in that, The display screen of the device under test shows the first actual delay duration data. The acquisition of the first actual delay duration set of the device under test in the target time period includes: Acquire the screen recording data of the device under test during the target time period; The screen recording data of the device under test is analyzed frame by frame to obtain multiple frames of images; The first actual delay duration data contained in each of the multiple frames of images is extracted, and the first actual delay duration set of the device under test is constructed based on the extracted first actual delay duration data.

5. The method according to claim 1, characterized in that, The step of generating a performance evaluation result for the device under test based at least on the comprehensive difference between each of the first actual delay duration data in the first actual delay duration set and the theoretical delay duration range includes: Select target first actual delay duration data from the first actual delay duration set, wherein the target first actual delay duration data includes first actual delay duration data within the theoretical delay duration range; Calculate the proportion of the target first actual delay duration data in the first actual delay duration set; The performance evaluation result is determined based on the performance level corresponding to the calculated percentage.

6. The method according to claim 5, characterized in that, It also includes: determining the continuity of each remaining first actual delay duration data according to the timestamp corresponding to each remaining first actual delay duration data, wherein the remaining first actual delay duration data is the data in the first actual delay duration set excluding the target first actual delay duration data; Determining the performance evaluation result based on the performance level corresponding to the calculated percentage includes: If the number of remaining first actual delay durations that are consecutive in time in the continuity condition reaches a preset threshold, the performance evaluation result of the device under test is determined to be abnormal. If the number of remaining first actual delay durations that are consecutive in time in the continuity condition does not reach the preset number threshold, the performance evaluation result is determined based on the performance level corresponding to the calculated percentage.

7. The method according to claim 1, characterized in that, It also includes: obtaining a second set of actual delay durations of a reference device during the target time period, wherein the reference device and the device under test are connected to the same network; The step of generating a performance evaluation result for the device under test based at least on the comprehensive difference between each of the first actual delay duration data in the first actual delay duration set and the theoretical delay duration range includes: Based on the first actual delay duration set of the device under test and the second actual delay duration set of the reference device, an actual delay duration difference is generated; The performance evaluation result is generated based on the difference in actual latency and the overall difference.

8. The method according to claim 7, characterized in that, The step of generating the performance evaluation result based on the actual latency difference and the overall difference includes: Based on the correspondence between the delay duration difference and the delay score, the actual delay score corresponding to the actual delay duration difference is determined; Determine the first weight corresponding to the comprehensive difference, the second weight corresponding to the actual delay score, and the third weight corresponding to the statistic, wherein the statistic is obtained based on the comprehensive difference and the actual delay score; Based on the first weight, the second weight, and the third weight, the comprehensive difference, the actual delay score, and the statistic are weighted and calculated. The performance evaluation result is generated based on the weighted calculation result, and the performance evaluation result includes the performance level or potential risk corresponding to the weighted calculation result.

9. The method according to claim 1, characterized in that, Also includes: The network performance data and the first set of actual latency durations of the device under test are displayed on the client side of the evaluation platform.

10. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor implements the method as described in any one of claims 1 to 9 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1 to 9.

12. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 9.

Images