Testing methods, apparatus and computer equipment for heterogeneous network computing platforms

Abstract

This application relates to a testing method, apparatus, computer equipment, and storage medium for a heterogeneous network computing platform. The method includes: acquiring all test indicators of the heterogeneous network computing platform and all sub-test indicators contained in each test indicator; all test indicators include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities; acquiring the evaluation score corresponding to each sub-test indicator contained in each test indicator; and performing a weighted summation of the evaluation scores corresponding to each sub-test indicator in each test indicator to obtain the evaluation score corresponding to each test indicator. By testing and verifying the heterogeneous network computing platform from six dimensions, a comprehensive test of the heterogeneous network computing platform is achieved, thereby improving the accuracy of the test results.

Description

Technical Field

[0001] This application relates to the field of network heterogeneous computing platform technology, and in particular to a network heterogeneous computing platform testing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the rapid development of technologies such as machine learning, artificial intelligence, and industrial simulation, and facing the demands of diversified computing types and exponential growth in computing volume, network heterogeneous computing platforms have become the preferred solution for building large-scale data center computing power. Since the 1980s, heterogeneous computing has undergone a debate among various routes, including CPU+GPU and FPGA. With the development of virtualization and next-generation network technologies, distributed network heterogeneous computing platforms using fiber optic networks and Ethernet as data exchange methods have emerged as a powerful force, becoming an important foundation for applications related to the Internet of Things, blockchain, and artificial intelligence.

[0003] To improve the computational efficiency of heterogeneous computing platforms, they are typically tested and validated before actual deployment. Related technologies test and validate heterogeneous computing platforms from any one of three aspects: computational performance, network performance, and algorithm optimization. However, testing and validating a platform from only one aspect has limitations and specialization, leading to inaccurate test results. Summary of the Invention

[0004] Therefore, it is necessary to provide a testing method, apparatus, computer equipment, and computer-readable storage medium for heterogeneous computing platforms that can improve the accuracy of test results for such platforms, addressing the aforementioned technical problems.

[0005] Firstly, this application provides a method for testing heterogeneous computing platforms in networks. The method includes:

[0006] This document describes the acquisition of all test metrics for a heterogeneous computing platform, including all sub-test metrics within each metric. The test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The resource utilization and parallel scheduling capabilities sub-test metrics include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The intelligent computing framework and component support capabilities sub-test metrics include support capabilities for preset frameworks and heterogeneous resource support capabilities. The intelligent data and service management support capabilities sub-test metrics include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. The data and model privacy protection capabilities sub-test metrics include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. The intelligent algorithm parallel optimization capabilities sub-test metrics include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. The general intelligent service capabilities sub-test metrics include intelligent voice interaction capabilities, visual object recognition capabilities, and natural language processing capabilities.

[0007] Obtain the evaluation score corresponding to each sub-test indicator included in each test indicator;

[0008] The evaluation scores corresponding to each sub-test index within each test index are weighted and summed to obtain the evaluation score for each test index.

[0009] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0010] The first scheduling duration of the computing task when it is scheduled through the resource management system of the network heterogeneous computing platform and the second scheduling duration of the computing task when it is not scheduled through the resource management system are obtained respectively. The first ratio between the second scheduling duration and the first scheduling duration is calculated, and the first difference between 1 and the first ratio is obtained. Based on the first difference, the evaluation score corresponding to the heterogeneous resource fusion scheduling capability is determined.

[0011] Determine the total number of computing resources deployed by the network heterogeneous computing platform based on the horizontal scaling deployment strategy when it receives a first preset number of computing test requests, and determine the first reciprocal of the total number of computing resources; calculate the first power of the negative first reciprocal of the natural constant, and determine the evaluation score corresponding to the automatic scaling capability of the task cluster based on the first power, where the first preset number is greater than the trigger threshold of the horizontal scaling deployment strategy.

[0012] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0013] Obtain the second ratio between the total number of all deep learning frameworks and the total number of deep learning frameworks supported by the network heterogeneous computing platform; determine the evaluation score corresponding to the support capability of the preset framework based on the second reciprocal of the second ratio;

[0014] Obtain the third ratio between the total number of all heterogeneous computing resources and the total number of heterogeneous computing resources supported by the network heterogeneous computing platform; determine the evaluation score corresponding to the heterogeneous resource support capability based on the third reciprocal of the third ratio.

[0015] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0016] Obtain the accuracy of pre-labeling test data for heterogeneous computing platforms and use it as an evaluation score corresponding to the multi-mode data pre-labeling capability; pre-labeling refers to labeling the attributes of test data;

[0017] Obtain the maximum number of data entities that the network heterogeneous computing platform can manage, including data entity storage, data entity processing, and data entity application; determine the fourth reciprocal of the maximum number; calculate the second power of the negative fourth reciprocal of the natural constant, and use the second power as the evaluation score corresponding to the data management capability;

[0018] Obtain the rating of each user on the interactive intelligent application of the network heterogeneous computing platform from a second preset number of users; the rating of each user on the interactive intelligent application ranges from 0 to 1; obtain the fourth ratio between each user's rating on the interactive intelligent application and the second preset number, and sum the fourth ratios to obtain the first summation result; use the first summation result as the evaluation score corresponding to the interactive intelligent modeling capability.

[0019] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0020] Obtain the second summation result between the encryption time of the network heterogeneous computing platform for each 100MB of data and 1; determine the fifth reciprocal of each second summation result, and sum the fifth reciprocals to obtain the third summation result; obtain the fourth summation result between the decryption time of the network heterogeneous computing platform for each 100MB of data and 1; determine the sixth reciprocal of each fourth summation result, and sum the sixth reciprocals to obtain the fifth summation result; wherein, the number of encryption processes and the number of decryption processes are both the third preset number of times;

[0021] Obtain the sixth summation result between the third and fifth summation results, obtain the fifth ratio between the sixth summation result and twice the third preset quantity, and use the fifth ratio as the evaluation score corresponding to the model's data privacy protection capability;

[0022] Obtain the total number of algorithm types that user data supporting homomorphic encryption in the network heterogeneous computing platform depends on. Obtain the sixth ratio between the total number of algorithm types that all homomorphically encrypted user data depends on and the total number of algorithm types that user data supporting homomorphic encryption in the network heterogeneous computing platform depends on. Use the seventh reciprocal of the sixth ratio as the evaluation score corresponding to the user data privacy protection capability.

[0023] Obtain the second difference between the accuracy of aggregated training on the heterogeneous computing platform and the accuracy of centralized training on the heterogeneous computing platform. Calculate the seventh ratio between the second difference and the accuracy of aggregated training. Use the seventh ratio as the evaluation score corresponding to the privacy-preserving data aggregation capability.

[0024] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0025] Obtain the eighth ratio between the total number of all machine learning algorithm libraries and the total number of machine learning algorithm libraries supported by the network heterogeneous computing platform, and use the eighth reciprocal of the eighth ratio as the evaluation score corresponding to the intelligent algorithm library adaptation capability.

[0026] Obtain the runtime of the preset algorithm on a single computing node in the heterogeneous computing platform; obtain the total runtime of the preset algorithm on a fourth preset number of computing nodes; calculate the product between the total runtime and the fourth preset number; calculate the ninth ratio between the runtime and the product; and use the ninth ratio as the evaluation score corresponding to the parallel optimization efficiency of the intelligent algorithm.

[0027] The first detection time and the second detection time of the optimized model are obtained when the preset model of the heterogeneous computing platform is detected. The optimized model is obtained by optimizing the preset model. The third difference between the first detection time and the second detection time is calculated. The tenth ratio between the third difference and the first detection time is calculated, and the tenth ratio is used as the evaluation score corresponding to the ability to improve the training efficiency of deep learning.

[0028] Obtain the first test accuracy of the heterogeneous computing platform under the generated sample training strategy and the second test accuracy of the heterogeneous computing platform under the conventional training strategy. Calculate the fourth difference between the first and second test accuracies. Calculate the eleventh ratio between the fourth difference and the first test accuracy, and use the eleventh ratio as the evaluation score corresponding to the improvement ability of deep learning training effect.

[0029] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0030] Get the rating of each user on the intelligent voice interaction service of the network heterogeneous computing platform from the fifth preset number of users; the rating of each user on the intelligent voice interaction service is in the range of 0 to 1; get the twelfth ratio between each user's rating on the intelligent voice interaction service and the fifth preset number; sum the twelfth ratios to obtain the seventh summation result, and use the seventh summation result as the evaluation score corresponding to the intelligent voice interaction capability.

[0031] Obtain the recognition accuracy of visual targets on the heterogeneous computing platform and use the recognition accuracy as the evaluation score corresponding to the visual target recognition capability.

[0032] Obtain the ninth reciprocal of the number of languages ​​supported by the heterogeneous computing platform for machine translation; calculate the third power of the negative ninth reciprocal of the natural constant, and use the third power as the evaluation score corresponding to the natural language processing capability.

[0033] Secondly, this application also provides a network heterogeneous computing platform testing device. The device includes:

[0034] The first acquisition module is used to acquire all test metrics of the heterogeneous computing platform and all sub-test metrics included in each test metric. All test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The sub-test metrics for resource utilization and parallel scheduling capabilities include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The sub-test metrics for intelligent computing framework and component support capabilities include support capabilities for preset frameworks and heterogeneous resource support capabilities. The sub-test metrics for intelligent data and service management support capabilities include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. The sub-test metrics for data and model privacy protection capabilities include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. The sub-test metrics for intelligent algorithm parallel optimization capabilities include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. The sub-test metrics for general intelligent service capabilities include intelligent voice interaction capabilities, visual target recognition capabilities, and natural language processing capabilities.

[0035] The second acquisition module is used to acquire the evaluation score corresponding to each sub-test indicator included in each test indicator;

[0036] The determination module is used to perform a weighted summation of the evaluation scores corresponding to each sub-test index included in each test index to obtain the evaluation score corresponding to each test index.

[0037] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0038] This document describes the acquisition of all test metrics for a heterogeneous computing platform, including all sub-test metrics within each metric. The test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The resource utilization and parallel scheduling capabilities sub-test metrics include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The intelligent computing framework and component support capabilities sub-test metrics include support capabilities for preset frameworks and heterogeneous resource support capabilities. The intelligent data and service management support capabilities sub-test metrics include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. The data and model privacy protection capabilities sub-test metrics include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. The intelligent algorithm parallel optimization capabilities sub-test metrics include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. The general intelligent service capabilities sub-test metrics include intelligent voice interaction capabilities, visual object recognition capabilities, and natural language processing capabilities.

[0039] Obtain the evaluation score corresponding to each sub-test indicator included in each test indicator;

[0040] The evaluation scores corresponding to each sub-test index within each test index are weighted and summed to obtain the evaluation score for each test index.

[0041] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0042] This document describes the acquisition of all test metrics for a heterogeneous computing platform, including all sub-test metrics within each metric. The test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The resource utilization and parallel scheduling capabilities sub-test metrics include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The intelligent computing framework and component support capabilities sub-test metrics include support capabilities for preset frameworks and heterogeneous resource support capabilities. The intelligent data and service management support capabilities sub-test metrics include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. The data and model privacy protection capabilities sub-test metrics include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. The intelligent algorithm parallel optimization capabilities sub-test metrics include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. The general intelligent service capabilities sub-test metrics include intelligent voice interaction capabilities, visual object recognition capabilities, and natural language processing capabilities.

[0043] Obtain the evaluation score corresponding to each sub-test indicator included in each test indicator;

[0044] The evaluation scores corresponding to each sub-test index within each test index are weighted and summed to obtain the evaluation score for each test index.

[0045] The aforementioned testing methods, apparatus, computer equipment, and storage media for heterogeneous network computing platforms acquire all test indicators of the platform and all sub-test indicators within each indicator. All test indicators include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The sub-test indicators for resource utilization and parallel scheduling capabilities include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The sub-test indicators for intelligent computing framework and component support capabilities include the support capabilities of preset frameworks and heterogeneous resource support capabilities. The sub-test indicators for intelligent data and service management support capabilities include multi-mode data pre-... The test evaluates the network heterogeneous computing platform across six dimensions: annotation capabilities, data management capabilities, and interactive intelligent modeling capabilities; data and model privacy protection capabilities, encompassing all sub-test indicators including model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities; intelligent algorithm parallel optimization capabilities, encompassing all sub-test indicators including intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities; and general intelligent service capabilities, encompassing all sub-test indicators including intelligent voice interaction capabilities, visual object recognition capabilities, and natural language processing capabilities. The evaluation score for each sub-test indicator within each test indicator is obtained. The evaluation scores for each sub-test indicator within each test indicator are then weighted and summed to obtain the evaluation score for each test indicator. By testing and verifying the network heterogeneous computing platform from six dimensions—resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities—a comprehensive test of the network heterogeneous computing platform is achieved, thereby improving the accuracy of the test results. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating a testing method for a heterogeneous network computing platform in one embodiment;

[0047] Figure 2 This is a structural block diagram of the test system corresponding to a network heterogeneous computing platform in one embodiment;

[0048] Figure 3 This is a structural block diagram of a network heterogeneous computing platform test device in one embodiment;

[0049] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] With the rapid development of technologies such as machine learning, artificial intelligence, and industrial simulation, and facing the demands of diversified computing types and exponential growth in computing volume, network heterogeneous computing platforms have become the preferred solution for building large-scale data center computing power. Since the 1980s, heterogeneous computing has undergone a debate among various routes, including CPU+GPU and FPGA. With the development of virtualization and next-generation network technologies, distributed network heterogeneous computing platforms using fiber optic networks and Ethernet as data exchange methods have emerged as a powerful force, becoming an important foundation for applications related to the Internet of Things, blockchain, and artificial intelligence.

[0052] To improve the computational efficiency of heterogeneous computing platforms, they are typically tested and validated before actual deployment. Related technologies test and validate heterogeneous computing platforms from any one of three aspects: computational performance, network performance, and algorithm optimization. However, testing and validating a platform from only one aspect has limitations and specialization, leading to inaccurate test results.

[0053] It is understood that the terms "first," "second," etc., used in this application may be used to describe various technical terms, but unless otherwise specified, these technical terms are not limited to these terms. These terms are only used to distinguish one technical term from another. For example, without departing from the scope of this application, the first preset quantity and the second preset quantity may be the same or different.

[0054] To address the problems existing in the aforementioned related technologies, embodiments of the present invention provide a method for testing a heterogeneous computing platform. This method can be applied to servers, terminals, and systems including terminals and servers, and is implemented through the interaction between the terminals and servers. The server can be a standalone server or a server cluster composed of multiple servers. Terminals can be, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. It should be noted that the quantities "multiple" mentioned in the embodiments of this application all refer to "at least two," for example, "multiple" means "at least two."

[0055] In one embodiment, such as Figure 1 As shown, a method for testing a heterogeneous computing platform is provided. This embodiment uses the method applied to a server as an example for illustration, and includes the following steps:

[0056] 101. Obtain all test metrics for the heterogeneous computing platform and all sub-test metrics included in each test metric. All test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. Resource utilization and parallel scheduling capabilities include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. Intelligent computing framework and component support capabilities include preset framework support capabilities and heterogeneous resource support capabilities. Intelligent data and service management support capabilities include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. Data and model privacy protection capabilities include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. Intelligent algorithm parallel optimization capabilities include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. General intelligent service capabilities include intelligent voice interaction capabilities, visual target recognition capabilities, and natural language processing capabilities.

[0057] Among them, the testing system for heterogeneous network computing platforms can be as follows: Figure 2 As shown. It should be noted that, Figure 2 The ability to support mainstream frameworks corresponds to the ability to support preset frameworks; the ability to support heterogeneous resources corresponds to the ability to support heterogeneous resources; deep learning training efficiency corresponds to the ability to improve deep learning training efficiency; and deep learning training effect corresponds to the ability to improve deep learning training effect.

[0058] 102. Obtain the evaluation score corresponding to each sub-test indicator included in each test indicator;

[0059] 103. The evaluation scores corresponding to each sub-test index included in each test index are weighted and summed to obtain the evaluation score corresponding to each test index.

[0060] The aforementioned testing method for heterogeneous computing platforms involves acquiring all test metrics of the platform and all sub-test metrics within each metric. All test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The resource utilization and parallel scheduling capabilities include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The intelligent computing framework and component support capabilities include pre-defined framework support capabilities and heterogeneous resource support capabilities. The intelligent data and service management support capabilities include multi-mode data pre-labeling capabilities, data... The test evaluates the network heterogeneous computing platform across six dimensions: management capabilities and interactive intelligent modeling capabilities; data and model privacy protection capabilities, encompassing all sub-test indicators including model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities; intelligent algorithm parallel optimization capabilities, encompassing all sub-test indicators including intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities; and general intelligent service capabilities, encompassing all sub-test indicators including intelligent voice interaction capabilities, visual object recognition capabilities, and natural language processing capabilities. The evaluation score for each sub-test indicator is obtained, and the weighted sum of these scores yields the final evaluation score for each test indicator. By testing and verifying the network heterogeneous computing platform across these six dimensions—resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities—a comprehensive test of the network heterogeneous computing platform is achieved, thereby improving the accuracy of the test results.

[0061] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0062] The first scheduling duration of the computing task when it is scheduled through the resource management system of the network heterogeneous computing platform and the second scheduling duration of the computing task when it is not scheduled through the resource management system are obtained respectively. The first ratio between the second scheduling duration and the first scheduling duration is calculated, and the first difference between 1 and the first ratio is obtained. Based on the first difference, the evaluation score corresponding to the heterogeneous resource fusion scheduling capability is determined.

[0063] The specific process for determining the evaluation score corresponding to the heterogeneous resource fusion scheduling capability can be shown in the following formula (1), which is not specifically limited in this embodiment of the present invention:

[0064] R = (1 - T2 / T1) * 100%; (1)

[0065] It should be noted that in formula (1), R is the evaluation score corresponding to the heterogeneous resource fusion scheduling capability, T2 is the first scheduling time of the computing task when it is scheduled through the resource management system of the network heterogeneous computing platform, T1 is the second scheduling time of the computing task when it is not scheduled through the resource management system, and T2 / T1 is the first ratio. Among them, the closer R is to 1, the higher the heterogeneous resource fusion scheduling capability of the network heterogeneous computing platform.

[0066] Furthermore, this invention does not limit the specific process of determining the evaluation score corresponding to the heterogeneous resource fusion scheduling capability based on the first difference, including but not limited to: obtaining the percentage of the first difference and using the percentage of the first difference as the evaluation score corresponding to the heterogeneous resource fusion scheduling capability. Converting the first difference into a percentage makes the evaluation score corresponding to the heterogeneous resource fusion scheduling capability more intuitive.

[0067] Determine the total number of computing resources deployed by the network heterogeneous computing platform based on the horizontal scaling deployment strategy when it receives a first preset number of computing test requests, and determine the first reciprocal of the total number of computing resources; calculate the first power of the negative first reciprocal of the natural constant, and determine the evaluation score corresponding to the automatic scaling capability of the task cluster based on the first power, where the first preset number is greater than the trigger threshold of the horizontal scaling deployment strategy.

[0068] The specific process for determining the evaluation score corresponding to the automatic expansion capability of the task cluster can be shown in the following formula (2), which is not specifically limited in this embodiment of the present invention:

[0069]

[0070] It should be noted that in formula (2), T is the evaluation score corresponding to the automatic scaling capability of the task cluster, e is a natural constant, and N1 is the total number of computing resources deployed by the network heterogeneous computing platform based on the horizontal scaling deployment strategy when it receives the first preset number of computing test requests. Among them, the closer T is to 1, the higher the automatic scaling capability of the task cluster of the network heterogeneous computing platform.

[0071] Furthermore, this invention does not limit the specific process of determining the evaluation score corresponding to the auto-scaling capability of the task cluster based on the first power, including but not limited to: obtaining the percentage of the first power and using the percentage of the first power as the evaluation score corresponding to the auto-scaling capability of the task cluster. Converting the first power to a percentage makes the evaluation score corresponding to the auto-scaling capability of the task cluster more intuitive.

[0072] Accordingly, the specific process for determining the evaluation scores corresponding to resource utilization and parallel scheduling capabilities can be shown in the following formula (3), which is not specifically limited in this embodiment of the present invention:

[0073] S1=α1*R+β1*T; (3)

[0074] It should be noted that in formula (3), S1 is the evaluation score corresponding to resource utilization and parallel scheduling capability. The closer the value of S1 is to 1, the higher the resource utilization and parallel scheduling capability of the heterogeneous computing platform. α1 is the weight corresponding to the heterogeneous resource fusion scheduling capability, and β1 is the weight corresponding to the automatic expansion capability of the task cluster. α1 + β1 = 1. The values ​​of α1 and β1 can be determined according to the specific application scenario and actual needs of the heterogeneous computing platform.

[0075] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0076] Obtain the second ratio between the total number of all deep learning frameworks and the total number of deep learning frameworks supported by the network heterogeneous computing platform; determine the evaluation score corresponding to the support capability of the preset framework based on the second reciprocal of the second ratio;

[0077] The specific process of determining the evaluation score corresponding to the support capability of the preset framework can be shown in the following formula (4), which is not specifically limited in this embodiment of the present invention:

[0078]

[0079] It should be noted that in formula (4), Q1 is the evaluation score corresponding to the support capability of the preset framework, and N2 is the total number of deep learning frameworks supported by the heterogeneous computing platform. The total number of all deep learning frameworks is 10, and this invention does not specifically limit this. Among them, the closer Q1 is to 1, the higher the support capability of the preset framework of the heterogeneous computing platform.

[0080] Furthermore, this invention does not limit the specific process of determining the evaluation score corresponding to the support capability of the preset framework based on the second reciprocal of the second ratio, including but not limited to: obtaining a percentage of the second reciprocal and using that percentage as the evaluation score corresponding to the support capability of the preset framework. Converting the second reciprocal to a percentage makes the evaluation score corresponding to the support capability of the preset framework more intuitive.

[0081] Obtain the third ratio between the total number of all heterogeneous computing resources and the total number of heterogeneous computing resources supported by the network heterogeneous computing platform; determine the evaluation score corresponding to the heterogeneous resource support capability based on the third reciprocal of the third ratio.

[0082] The specific process for determining the evaluation score corresponding to the heterogeneous resource support capability can be shown in the following formula (5), which is not specifically limited in this embodiment of the present invention:

[0083]

[0084] It should be noted that in formula (5), Q2 is the evaluation score corresponding to the heterogeneous resource support capability, and N3 is the total number of heterogeneous computing resources supported by the network heterogeneous computing platform. The total number of all heterogeneous computing resources is 5, and this invention does not specifically limit this. Among them, the closer Q2 is to 1, the higher the heterogeneous resource support capability of the network heterogeneous computing platform.

[0085] Furthermore, this invention limits the specific process of determining the evaluation score corresponding to heterogeneous resource support capability based on the third reciprocal of the third ratio, including but not limited to: obtaining a percentage of the third reciprocal and using that percentage as the evaluation score corresponding to heterogeneous resource support capability. Converting the third reciprocal to a percentage makes the evaluation score corresponding to heterogeneous resource support capability more intuitive.

[0086] Accordingly, the specific process for determining the evaluation scores corresponding to the intelligent computing framework and component support capabilities can be shown in the following formula (6), which is not specifically limited in this embodiment of the present invention:

[0087] S2=α2*Q1+β2*Q2; (6)

[0088] It should be noted that in formula (6), S2 is the evaluation score corresponding to the intelligent computing framework and component support capability. The closer the value of S2 is to 1, the higher the intelligent computing framework and component support capability of the network heterogeneous computing platform. α2 is the weight corresponding to the support capability of the preset framework, and β2 is the weight corresponding to the support capability of heterogeneous resources. α2 + β2 = 1. The values ​​of α2 and β2 can be determined according to the specific application scenario and actual needs of the network heterogeneous computing platform.

[0089] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0090] Obtain the accuracy of pre-labeling test data for heterogeneous computing platforms and use it as an evaluation score corresponding to the multi-mode data pre-labeling capability; pre-labeling refers to labeling the attributes of test data;

[0091] Obtain the maximum number of data entities that the network heterogeneous computing platform can manage, including data entity storage, data entity processing, and data entity application; determine the fourth reciprocal of the maximum number; calculate the second power of the negative fourth reciprocal of the natural constant, and use the second power as the evaluation score corresponding to the data management capability;

[0092] The specific process for determining the evaluation score corresponding to data management capability can be shown in the following formula (7), which is not specifically limited in this embodiment of the present invention:

[0093]

[0094] It should be noted that in formula (7), Q3 is the evaluation score corresponding to data management capability, and N4 is the maximum number of data entities that the network heterogeneous computing platform can support managing. Among them, the closer Q3 is to 1, the higher the data management capability of the network heterogeneous computing platform.

[0095] Obtain the rating of each user on the interactive intelligent application of the network heterogeneous computing platform from a second preset number of users; the rating of each user on the interactive intelligent application ranges from 0 to 1; obtain the fourth ratio between each user's rating on the interactive intelligent application and the second preset number, and sum the fourth ratios to obtain the first summation result; use the first summation result as the evaluation score corresponding to the interactive intelligent modeling capability.

[0096] The specific process for determining the evaluation score corresponding to the interactive intelligent modeling capability can be shown in the following formula (8), which is not specifically limited in this embodiment of the present invention:

[0097]

[0098] It should be noted that in formula (8), Q4 is the evaluation score corresponding to the interactive intelligent modeling capability, and P... i Let Q4 be the rating of the i-th user for the interactive intelligent application of the heterogeneous computing platform. The value 10 indicates that the second preset quantity is 10. The second preset quantity is not less than 5. Since the accuracy of the evaluation score corresponding to the interactive intelligent modeling capability changes little when the second preset quantity is greater than 10, in this embodiment, the second preset quantity is set to 10 to reduce computational load. The closer Q4 is to 1, the higher the interactive intelligent modeling capability of the heterogeneous computing platform.

[0099] Accordingly, the specific process for determining the evaluation score corresponding to the intelligent data and service management support capability can be shown in the following formula (9), which is not specifically limited in this embodiment of the present invention:

[0100] S3=α3*ε+β3*Q3+γ3*Q4; (9)

[0101] It should be noted that in formula (9), S3 is the evaluation score corresponding to the intelligent data and service management support capability. The closer the value of S3 is to 1, the higher the intelligent data and service management support capability of the network heterogeneous computing platform. ε is the evaluation score corresponding to the multi-mode data pre-labeling capability, α3 is the weight corresponding to the multi-mode data pre-labeling capability, β3 is the weight corresponding to the data management capability, and γ3 is the weight corresponding to the interactive intelligent modeling capability. α3+β3+γ3=1. The values ​​of α3, β3, and γ3 can be determined according to the specific application scenario and actual needs of the network heterogeneous computing platform.

[0102] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0103] Obtain the second summation result between the encryption time of the network heterogeneous computing platform for each 100MB of data and 1; determine the fifth reciprocal of each second summation result, and sum the fifth reciprocals to obtain the third summation result; obtain the fourth summation result between the decryption time of the network heterogeneous computing platform for each 100MB of data and 1; determine the sixth reciprocal of each fourth summation result, and sum the sixth reciprocals to obtain the fifth summation result; wherein, the number of encryption processes and the number of decryption processes are both the third preset number of times;

[0104] Obtain the sixth summation result between the third and fifth summation results, obtain the fifth ratio between the sixth summation result and twice the third preset quantity, and use the fifth ratio as the evaluation score corresponding to the model's data privacy protection capability;

[0105] The specific process for determining the evaluation score corresponding to the model's data privacy protection capability can be shown in the following formula (10), which is not specifically limited in this embodiment of the present invention:

[0106]

[0107] It should be noted that in formula (10), Q5 is the evaluation score corresponding to the model's data privacy protection capability, and A y B is the encryption duration of the y-th encryption of 100MB of data on the heterogeneous computing platform. y Let Q5 be the decryption time of the y-th iteration of a 100MB data set by the heterogeneous computing platform. The third preset value is 10. Since the accuracy of the evaluation score corresponding to the model's data privacy protection capability changes little when the third preset value is greater than 10, in this embodiment, the third preset value is 10 to reduce computational load. Specifically, the closer Q5 is to 1, the higher the model's data privacy protection capability of the heterogeneous computing platform.

[0108] Obtain the total number of algorithm types that user data supporting homomorphic encryption in the network heterogeneous computing platform depends on. Obtain the sixth ratio between the total number of algorithm types that all homomorphically encrypted user data depends on and the total number of algorithm types that user data supporting homomorphic encryption in the network heterogeneous computing platform depends on. Use the seventh reciprocal of the sixth ratio as the evaluation score corresponding to the user data privacy protection capability.

[0109] The specific process for determining the evaluation score corresponding to the user data privacy protection capability can be shown in the following formula (11), which is not specifically limited in this embodiment of the present invention:

[0110]

[0111] It should be noted that in formula (11), Q6 is the evaluation score corresponding to the user data privacy protection capability, and N5 is the total number of algorithm types relied upon by user data supporting homomorphic encryption in the heterogeneous computing platform. The total number of algorithm types relied upon by all homomorphically encrypted user data is 6. Among them, the closer Q6 is to 1, the higher the user data privacy protection capability of the heterogeneous computing platform.

[0112] Obtain the second difference between the accuracy of aggregated training on the heterogeneous computing platform and the accuracy of centralized training on the heterogeneous computing platform. Calculate the seventh ratio between the second difference and the accuracy of aggregated training. Use the seventh ratio as the evaluation score corresponding to the privacy-preserving data aggregation capability.

[0113] The specific process for determining the evaluation score corresponding to the privacy protection data aggregation capability can be shown in the following formula (12), which is not specifically limited in this embodiment of the present invention:

[0114]

[0115] It should be noted that in formula (12), Q7 is the evaluation score corresponding to the privacy-preserving data aggregation capability, Z2 is the accuracy of the aggregation training of the heterogeneous computing platform, and Z1 is the accuracy of the centralized training of the heterogeneous computing platform. Among them, the closer Q7 is to 1, the higher the privacy-preserving data aggregation capability of the heterogeneous computing platform.

[0116] Accordingly, the specific process for determining the evaluation score corresponding to the data and model privacy protection capabilities can be shown in the following formula (13), which is not specifically limited in this embodiment of the present invention:

[0117] S4=α4*Q5+β4*Q6+γ4*Q7; (13)

[0118] It should be noted that in formula (13), S4 is the evaluation score corresponding to the data and model privacy protection capability. The closer the value of S4 is to 1, the higher the data and model privacy protection capability of the heterogeneous computing platform. α4 is the weight corresponding to the model data privacy protection capability, β4 is the weight corresponding to the user data privacy protection capability, and γ4 is the weight corresponding to the privacy protection data aggregation capability. α4+β4+γ4=1. The values ​​of α4, β4, and γ4 can be determined according to the specific application scenario and actual needs of the heterogeneous computing platform.

[0119] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0120] Obtain the eighth ratio between the total number of all machine learning algorithm libraries and the total number of machine learning algorithm libraries supported by the network heterogeneous computing platform, and use the eighth reciprocal of the eighth ratio as the evaluation score corresponding to the intelligent algorithm library adaptation capability.

[0121] The specific process for determining the evaluation score corresponding to the adaptability of the intelligent algorithm library can be shown in the following formula (14), which is not specifically limited in this embodiment of the present invention:

[0122]

[0123] It should be noted that in formula (14), Q8 is the evaluation score corresponding to the intelligent algorithm library adaptation capability, and N6 is the total number of machine learning algorithm libraries supported by the heterogeneous computing platform. The total number of all machine learning algorithm libraries is 5. Among them, the closer Q8 is to 1, the higher the intelligent algorithm library adaptation capability of the heterogeneous computing platform.

[0124] Obtain the runtime of the preset algorithm on a single computing node in the heterogeneous computing platform; obtain the total runtime of the preset algorithm on a fourth preset number of computing nodes; calculate the product between the total runtime and the fourth preset number; calculate the ninth ratio between the runtime and the product; and use the ninth ratio as the evaluation score corresponding to the parallel optimization efficiency of the intelligent algorithm.

[0125] The specific process for determining the evaluation score corresponding to the parallel optimization efficiency of the intelligent algorithm can be shown in the following formula (15), which is not specifically limited in this embodiment of the present invention:

[0126]

[0127] It should be noted that in formula (15), Q9 is the evaluation score corresponding to the parallel optimization efficiency of the intelligent algorithm, t2 is the total running time of the preset algorithm on the fourth preset number of computing nodes, t1 is the running time of the preset algorithm of the network heterogeneous computing platform on a single computing node, and X is the fourth preset number. Among them, the closer Q9 is to 1, the higher the parallel optimization efficiency of the intelligent algorithm of the network heterogeneous computing platform.

[0128] The first detection time and the second detection time of the optimized model are obtained when the preset model of the heterogeneous computing platform is detected. The optimized model is obtained by optimizing the preset model. The third difference between the first detection time and the second detection time is calculated. The tenth ratio between the third difference and the first detection time is calculated, and the tenth ratio is used as the evaluation score corresponding to the ability to improve the training efficiency of deep learning.

[0129] The specific process for determining the evaluation score corresponding to the ability to improve deep learning training efficiency can be shown in the following formula (16), which is not specifically limited in this embodiment of the present invention:

[0130]

[0131] It should be noted that in formula (16), Q 10 The evaluation score corresponds to the ability to improve deep learning training efficiency. t3 is the first detection time when the preset model of the heterogeneous computing platform is detected, and t4 is the second detection time when the optimized model is detected. Where Q... 10 The closer it is to 1, the higher the efficiency improvement capability of deep learning training on the heterogeneous computing platform.

[0132] Obtain the first test accuracy of the heterogeneous computing platform under the generated sample training strategy and the second test accuracy of the heterogeneous computing platform under the conventional training strategy. Calculate the fourth difference between the first and second test accuracies. Calculate the eleventh ratio between the fourth difference and the first test accuracy, and use the eleventh ratio as the evaluation score corresponding to the improvement ability of deep learning training effect.

[0133] The specific process for determining the evaluation score corresponding to the improvement ability of deep learning training effect can be shown in the following formula (17), and the embodiments of the present invention do not specifically limit this:

[0134]

[0135] It should be noted that in formula (17), Q 11The evaluation score corresponds to the ability to improve the training effect of deep learning. acc2 represents the first test accuracy of the heterogeneous computing platform under the generated sample training strategy, and acc1 represents the second test accuracy of the heterogeneous computing platform under the conventional training strategy. Where Q... 11 The closer it is to 1, the higher the ability of the heterogeneous computing platform to improve the deep learning training effect.

[0136] Accordingly, the specific process for determining the evaluation score corresponding to the parallel optimization capability of the intelligent algorithm can be shown in the following formula (18), and the embodiments of the present invention do not specifically limit this:

[0137] S5=α5*Q8+β5*Q9+γ5*Q 10 +δ5*Q 11 (18)

[0138] It should be noted that in formula (18), S5 is the evaluation score corresponding to the parallel optimization capability of intelligent algorithms. The closer the value of S5 is to 1, the higher the parallel optimization capability of intelligent algorithms of the heterogeneous computing platform. α5 is the weight corresponding to the adaptation capability of intelligent algorithm library, β5 is the weight corresponding to the parallel optimization efficiency of intelligent algorithms, γ5 is the weight corresponding to the ability to improve deep learning training efficiency, and δ5 is the weight corresponding to the ability to improve deep learning training effect. α5+β5+γ5+δ5=1. The values ​​of α5, β5, γ5 and δ5 can be determined according to the specific application scenario and actual needs of the heterogeneous computing platform.

[0139] In one embodiment, obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes:

[0140] Get the rating of each user on the intelligent voice interaction service of the network heterogeneous computing platform from the fifth preset number of users; the rating of each user on the intelligent voice interaction service is in the range of 0 to 1; get the twelfth ratio between each user's rating on the intelligent voice interaction service and the fifth preset number; sum the twelfth ratios to obtain the seventh summation result, and use the seventh summation result as the evaluation score corresponding to the intelligent voice interaction capability.

[0141] The specific process for determining the evaluation score corresponding to the intelligent voice interaction capability can be shown in the following formula (19), which is not specifically limited in this embodiment of the present invention:

[0142]

[0143] It should be noted that in formula (19), Q 12 C is the evaluation score corresponding to the intelligent voice interaction capability. jFor the j-th user's rating of the intelligent voice interaction service on the heterogeneous computing platform, the fifth preset quantity is set to 10. Since the accuracy of the evaluation score corresponding to the intelligent voice interaction capability changes little when the fifth preset quantity is greater than 10, in this embodiment, the fifth preset quantity is set to 10 to reduce computational load. Wherein, Q... 12 The closer it is to 1, the higher the intelligent voice interaction capability of the heterogeneous computing platform.

[0144] Obtain the recognition accuracy of visual targets on the heterogeneous computing platform and use the recognition accuracy as the evaluation score corresponding to the visual target recognition capability.

[0145] Obtain the ninth reciprocal of the number of languages ​​supported by the heterogeneous computing platform for machine translation; calculate the third power of the negative ninth reciprocal of the natural constant, and use the third power as the evaluation score corresponding to the natural language processing capability.

[0146] The specific process for determining the evaluation score corresponding to natural language processing ability can be shown in the following formula (20), which is not specifically limited in this embodiment of the present invention:

[0147]

[0148] It should be noted that in formula (20), Q 13 Q represents the evaluation score corresponding to natural language processing capability, and N7 represents the number of languages ​​supported by the heterogeneous network computing platform for machine translation. 13 The closer the value is to 1, the higher the natural language processing capability of the heterogeneous computing platform.

[0149] Accordingly, the specific process for determining the evaluation score corresponding to the general intelligent service capability can be shown in the following formula (21), and the embodiments of the present invention do not specifically limit this:

[0150] S6=α6*Q 12 +β6*k+γ6*Q 13 ; (twenty one)

[0151] It should be noted that in formula (21), S6 is the evaluation score corresponding to the general intelligent service capability. The closer the value of S6 is to 1, the higher the general intelligent service capability of the heterogeneous computing platform. α6 is the weight corresponding to the intelligent voice interaction capability, β6 is the weight corresponding to the visual target recognition capability, k is the evaluation score corresponding to the visual target recognition capability, and γ6 is the weight corresponding to the natural language processing capability. α6+β6+γ6=1. The values ​​of α6, β6, and γ6 can be determined according to the specific application scenario and actual needs of the heterogeneous computing platform.

[0152] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0153] Based on the same inventive concept, this application also provides a network heterogeneous computing platform testing device for implementing the network heterogeneous computing platform testing method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more network heterogeneous computing platform testing device embodiments provided below can be found in the limitations of the network heterogeneous computing platform testing method described above, and will not be repeated here.

[0154] In one embodiment, such as Figure 3 As shown, a network heterogeneous computing platform testing device is provided, comprising: a first acquisition module 301, a second acquisition module 302, and a determination module 303, wherein:

[0155] The first acquisition module 301 is used to acquire all test indicators of the heterogeneous computing platform and all sub-test indicators included in each test indicator. All test indicators include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The sub-test indicators included in resource utilization and parallel scheduling capabilities include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The sub-test indicators included in intelligent computing framework and component support capabilities include the support capabilities of preset frameworks and heterogeneous resource support capabilities. The sub-test indicators included in intelligent data and service management support capabilities include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. The sub-test indicators included in data and model privacy protection capabilities include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. The sub-test indicators included in intelligent algorithm parallel optimization capabilities include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. The sub-test indicators included in general intelligent service capabilities include intelligent voice interaction capabilities, visual target recognition capabilities, and natural language processing capabilities.

[0156] The second acquisition module 302 is used to acquire the evaluation score corresponding to each sub-test indicator included in each test indicator;

[0157] The determination module 303 is used to perform a weighted summation of the evaluation scores corresponding to each sub-test index included in each test index to obtain the evaluation score corresponding to each test index.

[0158] In one embodiment, the second acquisition module 302 includes:

[0159] The first acquisition unit is used to acquire the first scheduling duration of the computing task when it is scheduled through the resource management system of the network heterogeneous computing platform and the second scheduling duration of the computing task when it is not scheduled through the resource management system, calculate the first ratio between the second scheduling duration and the first scheduling duration, obtain the first difference between 1 and the first ratio, and determine the evaluation score corresponding to the heterogeneous resource fusion scheduling capability based on the first difference.

[0160] The determining unit is used to determine the total number of computing resources deployed by the network heterogeneous computing platform based on the horizontal scaling deployment strategy when it receives a first preset number of computing test requests, and to determine the first reciprocal of the total number of computing resources; to calculate the first power of the negative first reciprocal of the natural constant, and to determine the evaluation score corresponding to the automatic scaling capability of the task cluster based on the first power, wherein the first preset number is greater than the trigger threshold of the horizontal scaling deployment strategy.

[0161] In one embodiment, the second acquisition module 302 includes:

[0162] The second acquisition unit is used to acquire a second ratio between the total number of all deep learning frameworks and the total number of deep learning frameworks supported by the heterogeneous computing platform; and to determine the evaluation score corresponding to the support capability of the preset framework based on the second reciprocal of the second ratio.

[0163] The third acquisition unit is used to acquire the third ratio between the total number of all heterogeneous computing resources and the total number of heterogeneous computing resources supported by the network heterogeneous computing platform; and to determine the evaluation score corresponding to the heterogeneous resource support capability based on the third reciprocal of the third ratio.

[0164] In one embodiment, the second acquisition module 302 includes:

[0165] The fourth acquisition unit is used to acquire the accuracy of the pre-labeling of test data on the heterogeneous computing platform and use it as the evaluation score corresponding to the multi-mode data pre-labeling capability; pre-labeling refers to labeling the attributes of the test data.

[0166] The fifth acquisition unit is used to acquire the maximum number of data entities that the network heterogeneous computing platform can support managing. The management items include data entity storage, data entity processing, and data entity application; determine the fourth reciprocal of the maximum number; calculate the second power of the negative fourth reciprocal of the natural constant, and use the second power as the evaluation score corresponding to the data management capability;

[0167] The sixth acquisition unit is used to acquire the rating of each user on the interactive intelligent application of the network heterogeneous computing platform from the second preset number of users; the rating of each user on the interactive intelligent application is in the range of 0 to 1; acquire the fourth ratio between the rating of each user on the interactive intelligent application and the second preset number, and sum the fourth ratios to obtain the first summation result; use the first summation result as the evaluation score corresponding to the interactive intelligent modeling capability.

[0168] In one embodiment, the second acquisition module 302 includes:

[0169] The seventh acquisition unit is used to acquire the second summation result between the encryption time of the network heterogeneous computing platform for each 100 Mbps data and 1; determine the fifth reciprocal of each second summation result, and superimpose each fifth reciprocal to obtain the third summation result; acquire the fourth summation result between the decryption time of the network heterogeneous computing platform for each 100 Mbps data and 1; determine the sixth reciprocal of each fourth summation result, and superimpose each sixth reciprocal to obtain the fifth summation result; wherein, the number of encryption processes and the number of decryption processes are both the third preset number of times;

[0170] The eighth acquisition unit is used to acquire the sixth summation result between the third summation result and the fifth summation result, acquire the fifth ratio between the sixth summation result and twice the third preset quantity, and use the fifth ratio as the evaluation score corresponding to the model's data privacy protection capability.

[0171] The ninth acquisition unit is used to acquire the total number of algorithm types that user data supporting homomorphic encryption in the network heterogeneous computing platform depends on, acquire the sixth ratio between the total number of algorithm types that user data supporting homomorphic encryption depends on and the total number of algorithm types that user data supporting homomorphic encryption in the network heterogeneous computing platform depends on, and use the seventh reciprocal of the sixth ratio as the evaluation score corresponding to the user data privacy protection capability.

[0172] The tenth acquisition unit is used to acquire the second difference between the accuracy of aggregated training on the heterogeneous computing platform and the accuracy of centralized training on the heterogeneous computing platform, calculate the seventh ratio between the second difference and the accuracy of aggregated training, and use the seventh ratio as the evaluation score corresponding to the privacy-preserving data aggregation capability.

[0173] In one embodiment, the second acquisition module 302 includes:

[0174] The eleventh acquisition unit is used to obtain the eighth ratio between the total number of all machine learning algorithm libraries and the total number of machine learning algorithm libraries supported by the network heterogeneous computing platform, and the eighth reciprocal of the eighth ratio is used as the evaluation score corresponding to the intelligent algorithm library adaptation capability.

[0175] The twelfth acquisition unit is used to acquire the runtime of the preset algorithm on a single computing node of the network heterogeneous computing platform, acquire the total runtime of the preset algorithm on a fourth preset number of computing nodes; calculate the product between the total runtime and the fourth preset number; calculate the ninth ratio between the runtime and the product, and use the ninth ratio as the evaluation score corresponding to the parallel optimization efficiency of the intelligent algorithm.

[0176] The thirteenth acquisition unit is used to acquire the first detection time when the preset model of the network heterogeneous computing platform is detected and the second detection time when the optimized model is detected. The optimized model is obtained by optimizing the preset model. The third difference between the first detection time and the second detection time is calculated. The tenth ratio between the third difference and the first detection time is calculated, and the tenth ratio is used as the evaluation score corresponding to the deep learning training efficiency improvement capability.

[0177] The fourteenth acquisition unit is used to acquire the first test accuracy of the network heterogeneous computing platform under the generated sample training strategy and the second test accuracy of the network heterogeneous computing platform under the conventional training strategy, calculate the fourth difference between the first test accuracy and the second test accuracy, calculate the eleventh ratio between the fourth difference and the first test accuracy, and use the eleventh ratio as the evaluation score corresponding to the improvement ability of deep learning training effect.

[0178] In one embodiment, the second acquisition module 302 includes:

[0179] The fifteenth acquisition unit is used to acquire the rating of each user among the fifth preset number of users for the intelligent voice interaction service of the network heterogeneous computing platform; the rating of each user for the intelligent voice interaction service is in the range of 0 to 1; acquire the twelfth ratio between each user's rating for the intelligent voice interaction service and the fifth preset number; sum the twelfth ratios to obtain the seventh summation result, and use the seventh summation result as the evaluation score corresponding to the intelligent voice interaction capability;

[0180] The sixteenth acquisition unit is used to acquire the recognition accuracy of visual targets by the heterogeneous computing platform of the network, and uses the recognition accuracy as the evaluation score corresponding to the visual target recognition capability.

[0181] The seventeenth acquisition unit is used to acquire the ninth reciprocal of the number of languages ​​supported by the heterogeneous computing platform for machine translation; calculate the third power of the negative ninth reciprocal of the natural constant, and use the third power as the evaluation score corresponding to the natural language processing capability.

[0182] Each module in the aforementioned heterogeneous computing platform test device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0183] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores all test metrics of the network heterogeneous computing platform and all sub-test metrics included in each test metric. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a testing method for the network heterogeneous computing platform.

[0184] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0185] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0186] This document describes the acquisition of all test metrics for a heterogeneous computing platform, including all sub-test metrics within each metric. The test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The resource utilization and parallel scheduling capabilities sub-test metrics include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The intelligent computing framework and component support capabilities sub-test metrics include support capabilities for preset frameworks and heterogeneous resource support capabilities. The intelligent data and service management support capabilities sub-test metrics include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. The data and model privacy protection capabilities sub-test metrics include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. The intelligent algorithm parallel optimization capabilities sub-test metrics include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. The general intelligent service capabilities sub-test metrics include intelligent voice interaction capabilities, visual object recognition capabilities, and natural language processing capabilities.

[0187] Obtain the evaluation score corresponding to each sub-test indicator included in each test indicator;

[0188] The evaluation scores corresponding to each sub-test index within each test index are weighted and summed to obtain the evaluation score for each test index.

[0189] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: acquiring all test metrics of a heterogeneous computing platform and all sub-test metrics included in each test metric; all test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities; all sub-test metrics included in resource utilization and parallel scheduling capabilities include heterogeneous resource fusion scheduling capabilities and task cluster automatic expansion capabilities; all sub-test metrics included in intelligent computing framework and component support capabilities include preset frameworks. The test includes: support capabilities for the framework and heterogeneous resources; intelligent data and service management support capabilities, encompassing all sub-test indicators including multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities; data and model privacy protection capabilities, encompassing all sub-test indicators including model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities; intelligent algorithm parallel optimization capabilities, encompassing all sub-test indicators including intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities; and general intelligent service capabilities, encompassing all sub-test indicators including intelligent voice interaction capabilities, visual target recognition capabilities, and natural language processing capabilities.

[0190] Obtain the evaluation score corresponding to each sub-test indicator included in each test indicator;

[0191] The evaluation scores corresponding to each sub-test index within each test index are weighted and summed to obtain the evaluation score for each test index.

[0192] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0193] This document describes the acquisition of all test metrics for a heterogeneous computing platform, including all sub-test metrics within each metric. The test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The resource utilization and parallel scheduling capabilities sub-test metrics include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The intelligent computing framework and component support capabilities sub-test metrics include support capabilities for preset frameworks and heterogeneous resource support capabilities. The intelligent data and service management support capabilities sub-test metrics include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. The data and model privacy protection capabilities sub-test metrics include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. The intelligent algorithm parallel optimization capabilities sub-test metrics include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. The general intelligent service capabilities sub-test metrics include intelligent voice interaction capabilities, visual object recognition capabilities, and natural language processing capabilities.

[0194] Obtain the evaluation score corresponding to each sub-test indicator included in each test indicator;

[0195] The evaluation scores corresponding to each sub-test index within each test index are weighted and summed to obtain the evaluation score for each test index.

[0196] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0197] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A testing method for a heterogeneous network computing platform, characterized in that, The method includes: This document describes the acquisition of all test metrics for a heterogeneous computing platform and all sub-test metrics within each metric. The test metrics include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The resource utilization and parallel scheduling capabilities sub-test metrics include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The intelligent computing framework and component support capabilities sub-test metrics include support capabilities for preset frameworks and heterogeneous resource support capabilities. The intelligent data and service management support capabilities sub-test metrics include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. The data and model privacy protection capabilities sub-test metrics include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. The intelligent algorithm parallel optimization capabilities sub-test metrics include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. The general intelligent service capabilities sub-test metrics include intelligent voice interaction capabilities, visual target recognition capabilities, and natural language processing capabilities. Obtain the evaluation score corresponding to each sub-test indicator included in each test indicator; The evaluation scores corresponding to each sub-test index included in each test index are weighted and summed to obtain the evaluation score corresponding to each test index. The step of obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes: The first scheduling duration of the computing task when it is scheduled through the resource management system of the heterogeneous computing platform and the second scheduling duration of the computing task when it is not scheduled through the resource management system are obtained respectively. The first ratio between the second scheduling duration and the first scheduling duration is calculated, and the first difference between 1 and the first ratio is obtained. Based on the first difference, the evaluation score corresponding to the heterogeneous resource fusion scheduling capability is determined. The total number of computing resources deployed by the heterogeneous computing platform based on the horizontal scaling deployment strategy when it receives a first preset number of computing test requests is determined, and the first reciprocal of the total number of computing resources is determined; the negative first reciprocal of the natural constant is calculated to obtain the first power; based on the first power, the evaluation score corresponding to the automatic scaling capability of the task cluster is determined, wherein the first preset number is greater than the trigger threshold of the horizontal scaling deployment strategy.

2. The method according to claim 1, characterized in that, The step of obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes: Obtain a second ratio between the total number of all deep learning frameworks and the total number of deep learning frameworks supported by the heterogeneous computing platform; determine the evaluation score corresponding to the support capability of the preset framework based on the second reciprocal of the second ratio; Obtain the third ratio between the total number of all heterogeneous computing resources and the total number of heterogeneous computing resources supported by the network heterogeneous computing platform; determine the evaluation score corresponding to the heterogeneous resource support capability based on the third reciprocal of the third ratio.

3. The method according to claim 1, characterized in that, The step of obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes: The accuracy of the pre-labeling of test data for the heterogeneous computing platform is obtained and used as the evaluation score corresponding to the multi-mode data pre-labeling capability; the pre-labeling refers to labeling the attributes of the test data. The maximum number of data entities that the heterogeneous computing platform supports managing is obtained, and the management items include data entity storage, data entity processing, and data entity application; the fourth reciprocal of the maximum number is determined; the negative fourth reciprocal of the natural constant is calculated to obtain the second power; and the second power is used as the evaluation score corresponding to the data management capability. Obtain the rating of each user from a second preset number of users for the interactive intelligent application of the heterogeneous computing platform; the rating of each user for the interactive intelligent application is in the range of 0 to 1; obtain the fourth ratio between each user's rating for the interactive intelligent application and the second preset number, and sum the fourth ratios to obtain the first summation result; use the first summation result as the evaluation score corresponding to the interactive intelligent modeling capability.

4. The method according to claim 1, characterized in that, The step of obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes: Obtain the second summation result between the encryption time of the heterogeneous computing platform for each 100MB of data and 1; determine the fifth reciprocal of each second summation result, and sum the fifth reciprocals to obtain the third summation result; obtain the fourth summation result between the decryption time of the heterogeneous computing platform for each 100MB of data and 1; determine the sixth reciprocal of each fourth summation result, and sum the sixth reciprocals to obtain the fifth summation result; wherein, the number of encryption processes and the number of decryption processes are both a third preset number of times; Obtain a sixth summation result between the third summation result and the fifth summation result, obtain a fifth ratio between the sixth summation result and twice the third preset quantity, and use the fifth ratio as the evaluation score corresponding to the model's data privacy protection capability; Obtain the total number of algorithm types that user data supporting homomorphic encryption in the heterogeneous computing platform, obtain the sixth ratio between the total number of algorithm types that user data supporting homomorphic encryption in the heterogeneous computing platform and the total number of algorithm types that user data supporting homomorphic encryption in the heterogeneous computing platform, and use the seventh reciprocal of the sixth ratio as the evaluation score corresponding to the user data privacy protection capability. Obtain a second difference between the accuracy of the aggregated training on the heterogeneous computing platform and the accuracy of the centralized training on the heterogeneous computing platform. Calculate a seventh ratio between the second difference and the accuracy of the aggregated training. Use the seventh ratio as the evaluation score corresponding to the privacy-preserving data aggregation capability.

5. The method according to claim 1, characterized in that, The step of obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes: Obtain the eighth ratio between the total number of all machine learning algorithm libraries and the total number of machine learning algorithm libraries supported by the network heterogeneous computing platform, and use the eighth reciprocal of the eighth ratio as the evaluation score corresponding to the adaptability of the intelligent algorithm library. The runtime of the preset algorithm on a single computing node of the heterogeneous computing platform is obtained, and the total runtime of the preset algorithm on a fourth preset number of computing nodes is obtained; the product between the total runtime and the fourth preset number is calculated; the ninth ratio between the runtime and the product is calculated, and the ninth ratio is used as the evaluation score corresponding to the parallel optimization efficiency of the intelligent algorithm. The first detection time when the preset model of the heterogeneous computing platform is detected and the second detection time when the optimized model is detected are obtained, wherein the optimized model is obtained by optimizing the preset model; the third difference between the first detection time and the second detection time is calculated; the tenth ratio between the third difference and the first detection time is calculated, and the tenth ratio is used as the evaluation score corresponding to the deep learning training efficiency improvement capability; Obtain the first test accuracy of the network heterogeneous computing platform under the generated sample training strategy and the second test accuracy of the network heterogeneous computing platform under the conventional training strategy when it is tested. Calculate the fourth difference between the first test accuracy and the second test accuracy. Calculate the eleventh ratio between the fourth difference and the first test accuracy, and use the eleventh ratio as the evaluation score corresponding to the improvement capability of the deep learning training effect.

6. The method according to claim 1, characterized in that, The step of obtaining the evaluation score corresponding to each sub-test indicator included in each test indicator includes: Obtain the rating of each user from a fifth preset number of users for the intelligent voice interaction service of the network heterogeneous computing platform; the rating of each user for the intelligent voice interaction service is in the range of 0 to 1; obtain the twelfth ratio between each user's rating for the intelligent voice interaction service and the fifth preset number; sum the twelfth ratios to obtain the seventh summation result, and use the seventh summation result as the evaluation score corresponding to the intelligent voice interaction capability; The accuracy rate of visual target recognition by the heterogeneous computing platform is obtained, and the recognition accuracy rate is used as the evaluation score corresponding to the visual target recognition capability. Obtain the ninth reciprocal of the number of language types supported by the heterogeneous computing platform for machine translation; calculate the negative ninth reciprocal of the natural constant to obtain the third power; use the third power as the evaluation score corresponding to the natural language processing capability.

7. A testing device for a network heterogeneous computing platform, characterized in that, The device includes: The first acquisition module is used to acquire all test indicators of the heterogeneous computing platform and all sub-test indicators included in each test indicator. All test indicators include resource utilization and parallel scheduling capabilities, intelligent computing framework and component support capabilities, intelligent data and service management support capabilities, data and model privacy protection capabilities, intelligent algorithm parallel optimization capabilities, and general intelligent service capabilities. The resource utilization and parallel scheduling capabilities include heterogeneous resource fusion scheduling capabilities and automatic task cluster expansion capabilities. The intelligent computing framework and component support capabilities include preset framework support capabilities and heterogeneous resource support capabilities. The intelligent data and service management support capabilities include multi-mode data pre-labeling capabilities, data management capabilities, and interactive intelligent modeling capabilities. The data and model privacy protection capabilities include model data privacy protection capabilities, user data privacy protection capabilities, and privacy-protected data aggregation capabilities. The intelligent algorithm parallel optimization capabilities include intelligent algorithm library adaptation capabilities, intelligent algorithm parallel optimization efficiency, deep learning training efficiency improvement capabilities, and deep learning training effect improvement capabilities. The general intelligent service capabilities include intelligent voice interaction capabilities, visual target recognition capabilities, and natural language processing capabilities. The second acquisition module is used to acquire the evaluation score corresponding to each sub-test indicator included in each test indicator; The determination module is used to perform a weighted summation of the evaluation scores corresponding to each sub-test index included in each test index to obtain the evaluation score corresponding to each test index. The second acquisition module includes: The first acquisition unit is used to acquire the first scheduling duration of the computing task when it is scheduled through the resource management system of the heterogeneous computing platform and the second scheduling duration of the computing task when it is not scheduled through the resource management system, calculate the first ratio between the second scheduling duration and the first scheduling duration, obtain the first difference between 1 and the first ratio, and determine the evaluation score corresponding to the heterogeneous resource fusion scheduling capability based on the first difference. The determining unit is configured to determine the total number of computing resources deployed by the heterogeneous computing platform based on the horizontal scaling deployment strategy when it receives a first preset number of computing test requests; determine the first reciprocal of the total number of computing resources; calculate the negative first reciprocal of the natural constant to obtain the first power; and determine the evaluation score corresponding to the automatic scaling capability of the task cluster based on the first power, wherein the first preset number is greater than the trigger threshold of the horizontal scaling deployment strategy.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images