An AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility

Abstract

The present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, which belongs to the field of digital computing device technology, including: automatically collecting the first data of the target device, generating a performance report of the target device based on the first data; determining the first operator demand vector and the second operator demand vector of the target device based on the performance report; determining the candidate operator vector and the candidate operator vector that match the performance report of the target device, executing the operator and generating an optimization strategy; dynamically adjusting the optimization strategy based on the collected real-time operation performance data. It can reduce manual intervention and optimization, ensure the timeliness and accuracy of data, reduce technical barriers and development costs, refine operator adaptation and optimization, achieve the optimal deployment of operators, and keep the device at the best performance at all times, thereby greatly improving the running speed and resource utilization of the AI ​​model, improving the portability and compatibility of the AI ​​model, and improving the stability and response speed of the overall system.

Description

Technical Field

[0001] The present invention relates to the technical field of digital computing devices, and in particular to an intelligent adaptation and optimization method for AI operators compatible with heterogeneous computing power. Background Art

[0002] Existing AI operator optimization technologies include operator fusion, matrix partitioning, automatic mixed precision training, low-precision quantization, automatic operator optimization tools, hardware-specific optimization, and loop optimization. These technologies significantly improve the execution efficiency and performance of AI algorithms by reducing the number of memory accesses, improving parallel processing capabilities, and data locality. For example, operator fusion accelerates calculations by reducing intermediate data storage and memory access; automatic mixed precision training increases calculation speed and reduces memory usage without sacrificing model accuracy. However, these technologies also have some shortcomings, such as hardware compatibility issues, large optimization workload, risk of precision loss, and limitations of automatic optimization technology.

[0003] The current challenges in the field of AI operator optimization include the lack of versatility that may result from hardware-specific optimizations, and the underestimated potential of compiler optimizations in AI workloads. Although automatic operator optimization tools such as AutoKernel can reduce development costs and improve optimization efficiency, they cannot fully reach the level of manual optimization by senior engineers, especially in complex optimization scenarios. In addition, customized operator development has increased development costs in the domestic AI chip ecosystem, brought about technical standardization and compatibility issues, and caused industry fragmentation. In addition, existing technologies may lack cross-platform adaptability and optimization capabilities, as well as high threshold requirements for operator development and optimization. More intelligent, automated, and compatible optimization solutions are needed to achieve wider hardware adaptation and more efficient performance improvements.

[0004] Therefore, the present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility. Summary of the invention

[0005] The present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility. The method determines the performance report of a target device through the collected first data, determines the first operator demand vector and the second operator demand vector according to the performance report, determines the candidate operator vector and the candidate operator vector that match the performance report of the target device, determines the execution operator and generates an optimization strategy, and dynamically adjusts the optimization strategy according to the collected real-time operation performance data. This method can reduce manual intervention and optimization, ensure the timeliness and accuracy of data, lower technical barriers and development costs, refine operator adaptation and optimization, achieve optimal deployment of operators, and enable the device to always maintain optimal performance, thereby greatly improving the operation speed and resource utilization of the AI ​​model, improving the portability and compatibility of the AI ​​model, and improving the stability and response speed of the overall system.

[0006] The present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, including:

[0007] 101: Automatically collect first data of a target device, and generate a performance report of the target device based on the first data;

[0008] 102: Determine a first operator requirement vector of the target device based on the performance report, and determine a second operator requirement vector of the target device based on the first operator requirement vector;

[0009] 103: Determine a candidate operator vector and a candidate operator vector that match the performance report of the target device, determine an execution operator of the target device and generate an optimization strategy;

[0010] 104: Collect real-time operation performance data of the target device, and dynamically adjust the optimization strategy based on the real-time operation performance data.

[0011] According to an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility provided by the present invention, the first data includes the model, manufacturer and benchmark performance data of the target device;

[0012] Automatically collecting first data of a target device and generating a performance report of the target device based on the first data, including:

[0013] Determining a target device based on an automated tool, and determining a model and manufacturer of the target device;

[0014] Performing a benchmark test on the target device to determine benchmark performance data, and analyzing the benchmark performance data to extract performance characteristics of the target device;

[0015] A performance report of the target device is generated based on the model, manufacturer, and performance characteristics of the target device.

[0016] According to an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility provided by the present invention, a first operator demand vector of the target device is determined based on a performance report, including:

[0017] Determine a sub-operator demand vector associated with each performance characteristic in the performance report of the target device, and determine a weight of each sub-operator demand in the sub-operator demand vector;

[0018] Counting the number of occurrences of each sub-operator requirement based on the sub-operator requirement vectors of all performance characteristics, and determining the comprehensive importance of each sub-operator requirement to the target device based on the number of occurrences of each sub-operator requirement and the weight in the corresponding multiple sub-operator requirement vectors;

[0019] Sorting the comprehensive importance of all sub-operator requirements to the target device from large to small, drawing an operator line graph based on the comprehensive importance of all sub-operator requirements to the target device after sorting, and determining a descending inflection point of the operator line graph;

[0020] Determine a first adjustment factor based on a position of the descending inflection point in the operator line graph and the number of sub-operator demands in the operator line graph;

[0021] The number of operator requirements of the target device is determined based on the descending inflection point and the first adjustment factor, the first operator requirement vector of the target device is determined based on the number of operator requirements and the operator line graph, and the weights of all operator requirements in the first operator requirement vector are determined based on the comprehensive importance of the sub-operator requirements corresponding to each operator requirement in the first operator requirement vector.

[0022] According to an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility provided by the present invention, a second operator demand vector of the target device is determined based on a first operator demand vector, including:

[0023] constructing a correlation matrix based on the correlation value between every two operator requirements in the first operator requirement vector, and determining positively correlated operator pairs and negatively correlated operator pairs based on the signs of all correlation values;

[0024] Determine a correlation value threshold based on the number of correlation values ​​in the correlation matrix and the values ​​of all correlation values, and identify a strong correlation operator pair based on the correlation value threshold, wherein the strong correlation operator pair includes a strong positive correlation operator pair and a strong negative correlation operator pair;

[0025] Determine a second adjustment factor based on the correlation values ​​of all strongly positively correlated operator pairs, extract operator demands that appear in all strongly positively correlated operator pairs to determine a positive operator demand vector, count the number of times all operator demands appear in the positive operator demand vector, and determine a first adjustment weight of the corresponding operator demand based on the second adjustment factor, the number of occurrences of each operator demand in the positive operator demand vector, the weight, and the correlation values ​​of multiple strongly positively correlated operator pairs;

[0026] The third adjustment factor is determined based on the correlation values ​​of all strongly negatively correlated operator pairs, and the positions of the two operator requirements corresponding to each strongly negatively correlated operator pair in the operator line graph are compared to extract

[0027] For each strongly negatively correlated operator pair, the operator demand whose operator demand is at the rear in the operator line graph determines a negative operator demand vector based on all extracted operator demands, counts the number of occurrences of all operator demands in the negative operator demand vector, and determines a second adjustment weight of the corresponding operator demand based on the third adjustment factor, the number of occurrences of each operator demand in the negative operator demand vector, the weight, and the correlation values ​​of the multiple strongly negatively correlated operator pairs;

[0028] Determining adjustment weights of all operator demands in the first operator demand vector based on weights of all operator demands in the first operator demand vector, first adjustment weights of all operator demands in the positive operator demand vector, and second adjustment weights of all operator demands in the negative operator demand vector;

[0029] All operator requirements in the first operator requirement vector are sorted from large to small according to the adjustment weight of each operator requirement, and the sorted operator requirements are determined as the second operator requirement vector.

[0030] According to an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility provided by the present invention, a candidate operator vector and a candidate operator vector to be matched with the performance report of the target device are determined, including:

[0031] Constructing a query condition based on the second operator demand vector, the adjustment weights of all operator demands in the second operator demand vector, the strong positive correlation operator pair, and the strong negative correlation operator pair;

[0032] Generate query statements based on query conditions, and query the dynamic operator library based on the query statements;

[0033] Based on the query result and the second operator requirement vector, a candidate operator vector and a candidate operator vector that match the performance report of the target device are calculated, wherein the candidate operator vector includes multiple candidate operators, the candidate operator vector includes multiple candidate operators, and any candidate operator in the candidate operator vector is different from any candidate operator in the candidate operator vector.

[0034] According to an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility provided by the present invention, based on the query result and the second operator demand vector, a candidate operator vector and a candidate operator vector matching the performance report of the target device are calculated, including:

[0035] Based on the query result and the second operator requirement vector, calculating a candidate operator vector that matches the performance report of the target device;

[0036] Where N1 represents the number of operators in the query result. Indicates the similarity value between the i-th operator and the second operator demand vector in the query result, represents the feature vector corresponding to the i-th operator in the query result, is the second operator demand vector, represents the first fitting parameter of the second operator demand vector, N2 represents the number of operator demands in the second operator demand vector, represents the weight of the sub-operator demand corresponding to the j-th operator demand in the second operator demand vector, represents the adjustment weight of the jth operator demand in the second operator demand vector, Indicates the value of the jth operator requirement in the second operator requirement vector of the i-th operator in the query result. represents a positive parameter, represents the similarity value threshold of the candidate operator, represents the candidate operator vector, Represents the i-th operator in the query result;

[0037] Based on the query result, the candidate operator vector and the second operator requirement vector, calculating a candidate operator vector that matches the performance report of the target device;

[0038] Among them, RE represents the fitting term, Indicates the value of the jth operator requirement in the second operator requirement vector for the kth operator that is in the query result but not in the candidate operator vector. represents the value of the jth operator demand in the second operator demand vector, represents the second fitting parameter of the second operator requirement vector, N3 represents the number of operators in the query result but not in the candidate operator vector, represents the score value of the kth operator in the query result but not in the candidate operator vector, represents the weight of the sub-operator demand corresponding to the k-th operator demand in the second operator demand vector, represents the adjustment weight of the kth operator demand in the second operator demand vector, Indicates the score threshold of the candidate operator. represents the candidate operator vector, Indicates the kth operator that is in the query result but not in the candidate operator vector.

[0039] According to an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility provided by the present invention, the execution operator of the target device is determined and an optimization strategy is generated, including:

[0040] Collecting user-specific requirements to determine a requirement optimization target, and determining an execution operator vector of the target device based on the requirement optimization target, the candidate operator vector, and the candidate operator vector;

[0041] Execute each operator in the operator vector as a node of a computation graph, and construct a computation graph of the target device based on a deep learning framework;

[0042] The constructed computation graph is parsed, an optimization strategy for the execution operator is generated based on the parsing result, and the computation graph is converted and parallelized based on the optimization strategy.

[0043] According to an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility provided by the present invention, the execution operator of the target device is determined and an optimization strategy is generated, and the method also includes:

[0044] The security and compliance of the optimization strategy are checked based on automated tools, and the optimized computational graph is converted into a cross-platform intermediate representation.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The performance report of the target device is determined through the collected first data, the first operator demand vector and the second operator demand vector are determined according to the performance report, the candidate operator vector and the candidate operator vector that match the performance report of the target device are determined, the execution operator is determined and the optimization strategy is generated, and the optimization strategy is dynamically adjusted according to the collected real-time operation performance data. This can reduce manual intervention and optimization, ensure the timeliness and accuracy of data, lower technical barriers and development costs, refine operator adaptation and optimization, achieve the optimal deployment of operators, and enable the device to always maintain optimal performance, thereby greatly improving the running speed and resource utilization of the AI ​​model, improving the portability and compatibility of the AI ​​model, and improving the stability and response speed of the overall system. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 It is a flow chart of an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Embodiment 1:

[0051] The embodiment of the present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, such as Figure 1 As shown, including:

[0052] 101: Automatically collect first data of a target device, and generate a performance report of the target device based on the first data;

[0053] 102: Determine a first operator requirement vector of the target device based on the performance report, and determine a second operator requirement vector of the target device based on the first operator requirement vector;

[0054] 103: Determine a candidate operator vector and a candidate operator vector that match the performance report of the target device, determine an execution operator of the target device and generate an optimization strategy;

[0055] 104: Collect real-time operation performance data of the target device, and dynamically adjust the optimization strategy based on the real-time operation performance data.

[0056] In this embodiment, the dynamic operator library includes implementations of multiple operators, which may be optimized according to different hardware characteristics, such as CUDA kernel functions for a specific GPU architecture.

[0057] In this embodiment, the performance report includes the model, manufacturer, and performance characteristics of the target device.

[0058] In this embodiment, real-time operating performance data represents the current working status of the device. These data may include key indicators such as CPU and GPU usage, memory usage, energy consumption, temperature, execution speed, latency, etc. The real-time collected data can reflect the actual performance of the device at the current time point.

[0059] The beneficial effects of the above technical solution are as follows: determining the performance report of the target device through the collected first data, determining the first operator demand vector and the second operator demand vector according to the performance report, determining the candidate operator vector and the candidate operator vector that match the performance report of the target device, determining the execution operator and generating the optimization strategy, and dynamically adjusting the optimization strategy according to the collected real-time operation performance data, which can reduce manual intervention and optimization, ensure the timeliness and accuracy of data, lower technical barriers and development costs, refine operator adaptation and optimization, achieve the optimal deployment of operators, and enable the device to always maintain optimal performance, thereby greatly improving the running speed and resource utilization of the AI ​​model, improving the portability and compatibility of the AI ​​model, and improving the stability and response speed of the overall system.

[0060] Embodiment 2:

[0061] An embodiment of the present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, wherein the first data includes the model, manufacturer, and benchmark performance data of the target device;

[0062] Automatically collecting first data of a target device and generating a performance report of the target device based on the first data, including:

[0063] Determining a target device based on an automated tool, and determining a model and manufacturer of the target device;

[0064] Performing a benchmark test on the target device to determine benchmark performance data, and analyzing the benchmark performance data to extract performance characteristics of the target device;

[0065] A performance report of the target device is generated based on the model, manufacturer, and performance characteristics of the target device.

[0066] In this embodiment, the benchmark performance data includes indicators such as processor speed, memory bandwidth, and number of cores.

[0067] In this embodiment, the performance characteristics include hardware architecture, computing power (floating-point computing power, integer computing power), vectorized processing capability, memory bandwidth, cache size, and specific types of computing limitations and hardware characteristics.

[0068] In this embodiment, for example: the automated tool determines that the target device is a new GPU, and determines the model and manufacturer of the new GPU. When performing performance analysis on the GPU, after running a series of graphics and computing-intensive benchmark tests, the speed and efficiency of processing large-scale matrix operations are determined as benchmark performance data. By analyzing the benchmark performance data, the performance characteristics of the target device are determined to be a GPU processor with strong computing power, strong vectorized processing capabilities, limited memory bandwidth, and no support for specific instruction sets (such as AVX, SIMD, CUDA) or accelerators. A corresponding performance report is generated based on the model, manufacturer, and performance characteristics of the GPU.

[0069] The beneficial effects of the above technical solution are as follows: by automatically collecting the first data of the target device and generating a performance report of the target device based on the first data, manual intervention can be reduced, the timeliness and accuracy of the data can be ensured, and the operating status and performance bottlenecks of the target device can be fully understood.

[0070] Embodiment 3:

[0071] An embodiment of the present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, which determines a first operator demand vector of the target device based on a performance report, including:

[0072] Determine a sub-operator demand vector associated with each performance characteristic in the performance report of the target device, and determine a weight of each sub-operator demand in the sub-operator demand vector;

[0073] Counting the number of occurrences of each sub-operator requirement based on the sub-operator requirement vectors of all performance characteristics, and determining the comprehensive importance of each sub-operator requirement to the target device based on the number of occurrences of each sub-operator requirement and the weight in the corresponding multiple sub-operator requirement vectors;

[0074] Sorting the comprehensive importance of all sub-operator requirements to the target device from large to small, drawing an operator line graph based on the comprehensive importance of all sub-operator requirements to the target device after sorting, and determining a descending inflection point of the operator line graph;

[0075] Determine a first adjustment factor based on a position of the descending inflection point in the operator line graph and the number of sub-operator demands in the operator line graph;

[0076] The number of operator requirements of the target device is determined based on the descending inflection point and the first adjustment factor, the first operator requirement vector of the target device is determined based on the number of operator requirements and the operator line graph, and the weights of all operator requirements in the first operator requirement vector are determined based on the comprehensive importance of the sub-operator requirements corresponding to each operator requirement in the first operator requirement vector.

[0077] In this embodiment, each performance feature corresponds to a sub-operator requirement vector. The closer the position of the sub-operator requirement in the sub-operator requirement vector is, the more important the operator requirement is to the corresponding performance feature in the operator requirement vector. For example, if the performance report of the target device includes 10 performance features, there are 10 sub-operator requirement vectors.

[0078] In this embodiment, the sub-operator demand vectors of all performance characteristics are summarized, the number of occurrences of each sub-operator demand is counted, and for each sub-operator demand, its comprehensive importance to the target device is calculated based on its weight in multiple vectors. For example, a sub-operator demand appears in the sub-operator demand vectors corresponding to three performance characteristics. Based on the weight of the sub-operator demand in the three sub-operator demand vectors and the number of occurrences 3, the comprehensive importance of the sub-operator demand to the target device is calculated.

[0079] In this embodiment, all sub-operator requirements are sorted from large to small according to their comprehensive importance to the target device, and based on the sorting result, the impact ranking of the sub-operator requirements on the target device can be identified.

[0080] In this embodiment, the horizontal axis of the operator line graph represents all sorted sub-operator requirements, and the vertical axis represents the comprehensive importance corresponding to the sub-operator requirements. The broken line in the operator line graph is a descending line.

[0081] In this embodiment, the descending inflection point of the operator line graph represents a place where the importance decreases rapidly, which can indicate the transition of the importance of the operator requirement from significant to secondary.

[0082] In this embodiment, the position of the descending inflection point is determined (such as the number of sub-operator demands in the operator line graph), and the first adjustment factor is calculated according to the inflection point position and the total number of sub-operator demands, which may be a certain ratio or proportion value.

[0083] In this embodiment, the first adjustment factor and the inflection point position are used to estimate the number of operator requirements of the target device, for example: the first few sub-operator requirements are selected as the final operator requirements based on the calculated ratio value, and the first operator requirement vector is determined based on the selected operator requirements.

[0084] The beneficial effect of the above technical solution is that the first operator requirement vector of the target device is determined according to the performance report, which can provide a data basis for determining the second operator requirement vector and refine the operator adaptation.

[0085] Embodiment 4:

[0086] An embodiment of the present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, which determines a second operator demand vector of the target device based on a first operator demand vector, including:

[0087] constructing a correlation matrix based on the correlation value between every two operator requirements in the first operator requirement vector, and determining positively correlated operator pairs and negatively correlated operator pairs based on the signs of all correlation values;

[0088] Determine a correlation value threshold based on the number of correlation values ​​in the correlation matrix and the values ​​of all correlation values, and identify a strong correlation operator pair based on the correlation value threshold, wherein the strong correlation operator pair includes a strong positive correlation operator pair and a strong negative correlation operator pair;

[0089] Determine a second adjustment factor based on the correlation values ​​of all strongly positively correlated operator pairs, extract operator demands that appear in all strongly positively correlated operator pairs to determine a positive operator demand vector, count the number of times all operator demands appear in the positive operator demand vector, and determine a first adjustment weight of the corresponding operator demand based on the second adjustment factor, the number of occurrences of each operator demand in the positive operator demand vector, the weight, and the correlation values ​​of multiple strongly positively correlated operator pairs;

[0090] The third adjustment factor is determined based on the correlation values ​​of all strongly negatively correlated operator pairs, and the positions of the two operator requirements corresponding to each strongly negatively correlated operator pair in the operator line graph are compared to extract

[0091] For each strongly negatively correlated operator pair, the operator demand whose operator demand is at the rear in the operator line graph determines a negative operator demand vector based on all extracted operator demands, counts the number of occurrences of all operator demands in the negative operator demand vector, and determines a second adjustment weight of the corresponding operator demand based on the third adjustment factor, the number of occurrences of each operator demand in the negative operator demand vector, the weight, and the correlation values ​​of the multiple strongly negatively correlated operator pairs;

[0092] Determining adjustment weights of all operator demands in the first operator demand vector based on weights of all operator demands in the first operator demand vector, first adjustment weights of all operator demands in the positive operator demand vector, and second adjustment weights of all operator demands in the negative operator demand vector;

[0093] All operator requirements in the first operator requirement vector are sorted from large to small according to the adjustment weight of each operator requirement, and the sorted operator requirements are determined as the second operator requirement vector.

[0094] In this embodiment, for example, there are Nu operator requirements in the first operator requirement vector, and the correlation matrix is ​​a Nu×Nu matrix, and the correlation matrix is ​​a symmetric matrix.

[0095] In this embodiment, each element in the correlation matrix represents the correlation value between the corresponding two operator requirements. The correlation value can be a positive number or a negative number. If the correlation value is a positive number, the sign of the correlation value is positive, and the two operator requirements corresponding to the correlation value are a positively correlated operator pair; if the correlation value is a negative number, the sign of the correlation value is negative, and the two operator requirements corresponding to the correlation value are a negatively correlated operator pair.

[0096] In this embodiment, a strongly positively correlated operator pair among the strongly correlated operator pairs is identified based on all positively correlated operator pairs and a correlation value threshold. If the correlation value of the positively correlated operator pair is greater than the positive correlation threshold, the positively correlated operator pair is determined to be a strongly positively correlated operator pair, wherein the positive correlation threshold is a correlation value threshold with a positive sign added.

[0097] In this embodiment, a strongly negatively correlated operator pair among the strongly correlated operator pairs is identified based on all negatively correlated operator pairs and a correlation value threshold. If the correlation value of the negatively correlated operator pair is less than the negative correlation threshold, the negatively correlated operator pair is determined to be a strongly negatively correlated operator pair, wherein the negative correlation threshold is a correlation value threshold with a negative sign added.

[0098] In this embodiment, a second adjustment factor is calculated from all pairs of strong positive correlation operators based on their correlation values. The second adjustment factor may be the average value or weighted average value of these correlation values, aiming to reflect the overall influence of these pairs of strong positive correlation operators.

[0099] In this embodiment, a third adjustment factor is calculated from all pairs of strong negatively correlated operators based on their correlation values. The third adjustment factor may be the average or weighted average of these correlation values, aiming to reflect the overall influence of these pairs of strong positively correlated operators.

[0100] In this embodiment, for operator demands that only appear in the positive operator demand vector, positive adjustments are made according to the weight of the operator demand in the first operator demand vector and the first adjustment weight, and the weight of the operator demand in the first operator demand vector and the first adjustment weight are weighted adjusted, adjusted proportionally, or other suitable methods are used, depending on the requirements of the application scenario.

[0101] In this embodiment, for operator demands that only appear in the negative operator demand vector, a negative adjustment is performed based on the weight of the operator demand in the first operator demand vector and the second adjustment weight, and the weight of the operator demand in the first operator demand vector and the second adjustment weight are weighted adjusted, adjusted proportionally, or other suitable methods are used, depending on the requirements of the application scenario.

[0102] In this embodiment, for the operator demand that appears in both the positive operator demand vector and the negative operator demand vector, a compound adjustment is performed based on the weight of the operator demand in the first operator demand vector, the first adjustment weight, and the second adjustment weight. The weight of the operator demand in the first operator demand vector, the first adjustment weight, and the second adjustment weight are weighted adjusted, adjusted proportionally, or other suitable methods are used, depending on the requirements of the application scenario.

[0103] In this embodiment, for an operator demand that does not appear in the positive operator demand vector and does not appear in the negative operator demand vector, the weight of the operator demand in the first operator demand vector is determined as the corresponding adjustment weight.

[0104] In this embodiment, all operator requirements in the first operator requirement vector are sorted from large to small according to all adjustment weights, and the sorted operator requirements constitute the second operator requirement vector.

[0105] The beneficial effect of the above technical solution is as follows: determining the second operator requirement vector of the target device according to the first operator requirement vector can provide a data basis for determining the candidate operator vector and the candidate operator vector that match the performance report of the target device, further refine the operator adaptation, realize the optimal deployment of the operator, and improve the device performance.

[0106] Embodiment 5:

[0107] The embodiment of the present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, which determines a candidate operator vector and a candidate operator vector that matches the performance report of the target device, including:

[0108] Constructing a query condition based on the second operator demand vector, the adjustment weights of all operator demands in the second operator demand vector, the strong positive correlation operator pair, and the strong negative correlation operator pair;

[0109] Generate query statements based on query conditions, and query the dynamic operator library based on the query statements;

[0110] Based on the query result and the second operator requirement vector, a candidate operator vector and a candidate operator vector that match the performance report of the target device are calculated, wherein the candidate operator vector includes multiple candidate operators, the candidate operator vector includes multiple candidate operators, and any candidate operator in the candidate operator vector is different from any candidate operator in the candidate operator vector.

[0111] In this embodiment, the operator requirements with large adjustment weights in the second operator requirement vector are used as the main query conditions. For example, if the adjustment weights of some operators in the second operator requirement vector are large, matching items of these operators can be added to the query conditions to ensure that the query results include these operators with large adjustment weights.

[0112] In this embodiment, for pairs of strongly positively correlated operators, they can be used as joint query conditions; for pairs of strongly negatively correlated operators, the combination of related operators can be excluded or restricted. For example, positively correlated operators can be combined through "AND" logic, and negatively correlated operators can be restricted through "NOT" or "EXCLUDE" logic.

[0113] In this embodiment, the query condition can be understood as a filtering rule for identifying which operators should be considered and which should be excluded or restricted. In the query condition, a strongly positively associated operator pair represents an operator that should be included in the query under certain conditions, while a strongly negatively associated operator pair represents an operator that can be excluded or restricted under certain conditions.

[0114] In this embodiment, after the query condition is formed, it is converted into a specific query statement, which is usually a SQL query or a similar structured query, for accessing the dynamic operator library.

[0115] In this embodiment, the candidate operator vector is a group of operators that are closest to the requirements of the target device and are screened out from the query results.

[0116] In this embodiment, the candidate operator vectors are used to further optimize the candidate operator vectors, which may be to select a group of operators with higher scores from the operators that fail to fully meet the similarity threshold.

[0117] The beneficial effects of the above technical solution are as follows: by determining the candidate operator vectors and the candidate operator vectors that match the performance report of the target device, a data basis can be provided for determining the execution operator of the target device and generating an optimization strategy to achieve the optimal deployment of the operator, so that the device always maintains the best performance, thereby greatly improving the running speed and resource utilization of the AI ​​model.

[0118] Embodiment 6:

[0119] The embodiment of the present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, which calculates a candidate operator vector and a candidate operator vector that matches the performance report of the target device based on the query result and the second operator demand vector, including:

[0120] Based on the query result and the second operator requirement vector, calculating a candidate operator vector that matches the performance report of the target device;

[0121] Where N1 represents the number of operators in the query result. Indicates the similarity value between the i-th operator and the second operator demand vector in the query result, represents the feature vector corresponding to the i-th operator in the query result, is the second operator demand vector, represents the first fitting parameter of the second operator demand vector, N2 represents the number of operator demands in the second operator demand vector, represents the weight of the sub-operator demand corresponding to the j-th operator demand in the second operator demand vector, represents the adjustment weight of the jth operator demand in the second operator demand vector, Indicates the value of the jth operator requirement in the second operator requirement vector of the i-th operator in the query result. represents a positive parameter, represents the similarity value threshold of the candidate operator, represents the candidate operator vector, Represents the i-th operator in the query result;

[0122] Based on the query result, the candidate operator vector and the second operator requirement vector, calculating a candidate operator vector that matches the performance report of the target device;

[0123] Among them, RE represents the fitting term, Indicates the value of the jth operator requirement in the second operator requirement vector for the kth operator that is in the query result but not in the candidate operator vector. represents the value of the jth operator demand in the second operator demand vector, represents the second fitting parameter of the second operator requirement vector, N3 represents the number of operators in the query result but not in the candidate operator vector, represents the score value of the kth operator in the query result but not in the candidate operator vector, represents the weight of the sub-operator demand corresponding to the k-th operator demand in the second operator demand vector, represents the adjustment weight of the kth operator demand in the second operator demand vector, Indicates the score threshold of the candidate operator. represents the candidate operator vector, Indicates the kth operator that is in the query result but not in the candidate operator vector.

[0124] In this embodiment, the fitting parameters are used to control the influence of all operator requirements in the second operator requirement vector on the operator similarity value in the query result.

[0125] In this embodiment, the positive parameter is a very small positive value used to avoid calculation errors of zero values, for example: .

[0126] In this embodiment, The weight of the sub-operator demand corresponding to the j-th operator demand It has a small impact on the similarity value between the i-th operator and the second operator's demand vector in the query result. The nonlinear amplification of the adjustment weights is performed using a logarithmic function to further emphasize that the adjustment weights have a greater impact on the similarity values ​​of the demand vectors of the i-th operator and the second operator in the query results.

[0127] In this embodiment, It reflects the performance or attribute of the i-th operator in the query result on the j-th operator requirement in the second operator requirement vector.

[0128] In this embodiment, the candidate operator vector includes all operators in the query result whose similarity values ​​with the second operator requirement vector are greater than a similarity value threshold.

[0129] In this embodiment, Indicates the performance or attribute of the kth operator in the query result but not in the candidate operator vector on the jth operator requirement in the second operator requirement vector.

[0130] In this embodiment, Represents the performance or attribute of the j-th operator requirement in the second operator requirement vector.

[0131] In this embodiment, the fitting term represents a fitting term for calculating the score value of the operator in the query result but not in the candidate operator vector, and is used to control the influence of all operator requirements in the second operator requirement vector on the value score.

[0132] In this embodiment, the candidate operator vector includes all operators that are in the query result but not in the candidate operator vector and whose score values ​​are less than or equal to the score value threshold.

[0133] The beneficial effects of the above technical solution are as follows: based on the query results and the second operator demand vector, the candidate operator vector and the candidate operator vector that match the performance report of the target device are calculated, which can further refine the operator adaptation, achieve the optimal deployment of the operator, and greatly improve the running speed and resource utilization of the AI ​​model.

[0134] Embodiment 7:

[0135] The embodiment of the present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, which determines the execution operator of the target device and generates an optimization strategy, including:

[0136] Collecting user-specific requirements to determine a requirement optimization target, and determining an execution operator vector of the target device based on the requirement optimization target, the candidate operator vector, and the candidate operator vector;

[0137] Execute each operator in the operator vector as a node of a computation graph, and construct a computation graph of the target device based on a deep learning framework;

[0138] The constructed computation graph is parsed, an optimization strategy for the execution operator is generated based on the parsing result, and the computation graph is converted and parallelized based on the optimization strategy.

[0139] In this embodiment, through interaction with users, their specific needs are clarified. These needs may include performance requirements, computing resource limitations, algorithm accuracy, response time, etc. By understanding these needs, one or more demand optimization goals can be determined.

[0140] In this embodiment, according to the demand optimization target, a set of final execution operator vectors is determined by comprehensively considering the operators of the candidate operator vectors and the operators to be candidate operator vectors. The final execution operator vectors will include those operators that best meet user needs and can run efficiently on the target device.

[0141] In this embodiment, in the deep learning framework, the computational graph is a structural diagram that describes the model execution process, each node represents an operator, and the edges between nodes represent the flow and dependencies of data.

[0142] In this embodiment, parsing and constructing the computational graph includes analyzing each node and edge in the graph, understanding the dependencies between operators, data flow, critical paths, and possible bottlenecks, and formulating optimization strategies for the computational graph based on the information obtained from the analysis. The optimization strategies may include adjusting the execution order of operators, merging or decomposing operators, reducing data transmission, instruction selection, data layout optimization, etc., to achieve the optimization goal.

[0143] The beneficial effects of the above technical solution are as follows: by determining the execution operator of the target device and generating an optimization strategy, the need for manual optimization can be reduced, the technical threshold and development costs can be lowered, the running speed and resource utilization of the AI ​​model can be greatly improved, and the stability and response speed of the overall system can be improved.

[0144] Embodiment 8:

[0145] The embodiment of the present invention provides an AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, which determines the execution operator of the target device and generates an optimization strategy, and also includes:

[0146] The security and compliance of the optimization strategy are checked based on automated tools, and the optimized computational graph is converted into a cross-platform intermediate representation.

[0147] In this embodiment, before applying the optimization policy, it is necessary to use an automated tool to perform a security check on it, which includes evaluating whether the policy introduces potential security vulnerabilities, such as out-of-bounds access, unhandled exceptions, resource leakage, or sensitive data exposure. The automated tool can help detect these problems and provide corresponding repair suggestions.

[0148] In this embodiment, in addition to security, it is also necessary to ensure that the optimization strategy complies with relevant regulations and standards. For example, whether data processing complies with privacy protection regulations (such as GDPR), whether the use of algorithms follows industry standards, or in the medical field, whether model optimization complies with the requirements of regulatory agencies such as the FDA. Automated tools can automatically verify these compliance requirements and reduce the burden of manual inspection.

[0149] In this embodiment, after applying the optimization strategy and passing the security and compliance checks, an optimized computation graph is obtained, which is adjusted to improve performance, reduce resource consumption, and meet other optimization goals.

[0150] In this embodiment, in order to ensure that the computation graph can run on different platforms (such as different operating systems, hardware architectures or computing environments), it needs to be converted into a cross-platform intermediate representation. This representation can abstract platform-related details, so that the computation graph has good portability between different platforms.

[0151] In this embodiment, a deep learning framework or other tools are used to convert the optimized computational graph into a cross-platform intermediate representation, and functional verification and performance verification are performed. For example, ONNX (Open Neural Network Exchange) is a common intermediate representation format that supports model conversion between multiple deep learning frameworks. This intermediate representation can be loaded and run in different execution environments, making the model more widely applicable.

[0152] The beneficial effects of the above technical solution: security and compliance checks on the optimization strategy, and conversion of the optimized computational graph into a cross-platform intermediate representation, can improve the portability and compatibility of the AI ​​model, and improve the stability and response speed of the overall system.

[0153] The method embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AI operator intelligent adaptation and optimization method for heterogeneous computing power compatibility, characterized in that: include: 101: Automatically collect first data of a target device, and generate a performance report of the target device based on the first data; 102: Determine a first operator requirement vector of the target device based on the performance report, and determine a second operator requirement vector of the target device based on the first operator requirement vector; 103: Determine a candidate operator vector and a candidate operator vector that match the performance report of the target device, determine an execution operator of the target device and generate an optimization strategy; 104: Collecting real-time operation performance data of the target device, and dynamically adjusting the optimization strategy based on the real-time operation performance data; The first data includes the model, manufacturer and benchmark performance data of the target device; The step of automatically collecting first data of a target device and generating a performance report of the target device based on the first data includes: Determining a target device based on an automated tool, and determining a model and manufacturer of the target device; Performing a benchmark test on the target device to determine benchmark performance data, and analyzing the benchmark performance data to extract performance characteristics of the target device; Generate a performance report of the target device based on the model, manufacturer and performance characteristics of the target device; The step of determining the first operator requirement vector of the target device based on the performance report includes: Determine a sub-operator demand vector associated with each performance characteristic in the performance report of the target device, and determine a weight of each sub-operator demand in the sub-operator demand vector; Counting the number of occurrences of each sub-operator requirement based on the sub-operator requirement vectors of all performance characteristics, and determining the comprehensive importance of each sub-operator requirement to the target device based on the number of occurrences of each sub-operator requirement and the weight in the corresponding multiple sub-operator requirement vectors; Sorting the comprehensive importance of all sub-operator requirements to the target device from large to small, drawing an operator line graph based on the comprehensive importance of all sub-operator requirements to the target device after sorting, and determining a descending inflection point of the operator line graph; Determine a first adjustment factor based on a position of the descending inflection point in the operator line graph and the number of sub-operator demands in the operator line graph; Determine the number of operator requirements of the target device based on the descending inflection point and the first adjustment factor, determine a first operator requirement vector of the target device based on the number of operator requirements and the operator line graph, and determine the weights of all operator requirements in the first operator requirement vector based on the comprehensive importance of the sub-operator requirements corresponding to each operator requirement in the first operator requirement vector; Wherein, determining the second operator requirement vector of the target device based on the first operator requirement vector includes: constructing a correlation matrix based on the correlation value between every two operator requirements in the first operator requirement vector, and determining positively correlated operator pairs and negatively correlated operator pairs based on the signs of all correlation values; Determine a correlation value threshold based on the number of correlation values ​​in the correlation matrix and the values ​​of all correlation values, and identify a strong correlation operator pair based on the correlation value threshold, wherein the strong correlation operator pair includes a strong positive correlation operator pair and a strong negative correlation operator pair; Determine a second adjustment factor based on the correlation values ​​of all strongly positively correlated operator pairs, extract operator demands that appear in all strongly positively correlated operator pairs to determine a positive operator demand vector, count the number of times all operator demands appear in the positive operator demand vector, and determine a first adjustment weight of the corresponding operator demand based on the second adjustment factor, the number of occurrences of each operator demand in the positive operator demand vector, the weight, and the correlation values ​​of multiple strongly positively correlated operator pairs; The third adjustment factor is determined based on the correlation values ​​of all strongly negatively correlated operator pairs, and the positions of the two operator requirements corresponding to each strongly negatively correlated operator pair in the operator line graph are compared to extract For each strongly negatively correlated operator pair, the operator demand whose operator demand is at the rear in the operator line graph determines a negative operator demand vector based on all extracted operator demands, counts the number of occurrences of all operator demands in the negative operator demand vector, and determines a second adjustment weight of the corresponding operator demand based on the third adjustment factor, the number of occurrences of each operator demand in the negative operator demand vector, the weight, and the correlation values ​​of the multiple strongly negatively correlated operator pairs; Determining adjustment weights of all operator demands in the first operator demand vector based on weights of all operator demands in the first operator demand vector, first adjustment weights of all operator demands in the positive operator demand vector, and second adjustment weights of all operator demands in the negative operator demand vector; For all operator demands in the first operator demand vector, sort them from large to small according to the adjustment weight of each operator demand, and determine the sorted operator demands as the second operator demand vector; The step of determining a candidate operator vector and a candidate operator vector that match the performance report of the target device includes: Constructing a query condition based on the second operator demand vector, the adjustment weights of all operator demands in the second operator demand vector, the strong positive correlation operator pair, and the strong negative correlation operator pair; Generate query statements based on query conditions, and query the dynamic operator library based on the query statements; Based on the query result and the second operator requirement vector, a candidate operator vector and a candidate operator vector that match the performance report of the target device are calculated, wherein the candidate operator vector includes multiple candidate operators, the candidate operator vector includes multiple candidate operators, and any candidate operator in the candidate operator vector is different from any candidate operator in the candidate operator vector.

2. According to claim 1, the AI ​​operator intelligent adaptation and optimization method for heterogeneous computing power compatibility is characterized in that: Based on the query result and the second operator requirement vector, calculating a candidate operator vector and a candidate operator vector that match the performance report of the target device includes: Based on the query result and the second operator requirement vector, calculating a candidate operator vector that matches the performance report of the target device; Where N1 represents the number of operators in the query result. Indicates the similarity value between the i-th operator and the second operator demand vector in the query result, represents the feature vector corresponding to the i-th operator in the query result, is the second operator demand vector, represents the first fitting parameter of the second operator demand vector, N2 represents the number of operator demands in the second operator demand vector, represents the weight of the sub-operator demand corresponding to the j-th operator demand in the second operator demand vector, represents the adjustment weight of the jth operator demand in the second operator demand vector, Indicates the value of the jth operator requirement in the second operator requirement vector of the i-th operator in the query result. represents a positive parameter, represents the similarity value threshold of the candidate operator, represents the candidate operator vector, Represents the i-th operator in the query result; Based on the query result, the candidate operator vector and the second operator requirement vector, calculating a candidate operator vector that matches the performance report of the target device; Among them, RE represents the fitting term, Indicates the value of the jth operator requirement in the second operator requirement vector for the kth operator that is in the query result but not in the candidate operator vector. represents the value of the jth operator demand in the second operator demand vector, represents the second fitting parameter of the second operator requirement vector, N3 represents the number of operators in the query result but not in the candidate operator vector, represents the score value of the kth operator in the query result but not in the candidate operator vector, represents the weight of the sub-operator demand corresponding to the k-th operator demand in the second operator demand vector, represents the adjustment weight of the kth operator demand in the second operator demand vector, Indicates the score threshold of the candidate operator. represents the candidate operator vector, Indicates the kth operator that is in the query result but not in the candidate operator vector.

3. According to claim 1, the AI ​​operator intelligent adaptation and optimization method for heterogeneous computing power compatibility is characterized in that: Determining the execution operator of the target device and generating an optimization strategy, including: Collecting user-specific requirements to determine a requirement optimization target, and determining an execution operator vector of the target device based on the requirement optimization target, the candidate operator vector, and the candidate operator vector; Execute each operator in the operator vector as a node of a computation graph, and construct a computation graph of the target device based on a deep learning framework; The constructed computation graph is parsed, an optimization strategy for the execution operator is generated based on the parsing result, and the computation graph is converted and parallelized based on the optimization strategy.

4. According to claim 3, the AI ​​operator intelligent adaptation and optimization method for heterogeneous computing power compatibility is characterized in that: Determining the execution operator of the target device and generating an optimization strategy also includes: The security and compliance of the optimization strategy are checked based on automated tools, and the optimized computational graph is converted into a cross-platform intermediate representation.

Images