A code processing method and related apparatus

Abstract

A code processing method is applied to a computing device including multiple computing units. In the code processing method, multiple instructions capable of being executed in parallel by the multiple computing units are set in the code of an operator, and corresponding performance information of the code at runtime is collected. In this way, by comparing the performance information of the code and the performance information of the operator before the code is implemented using other instruction proportions, it can be determined whether the data processing performance of the operator reaches an optimal value. When the data processing performance of the operator does not reach the optimal value, the proportion of the multiple instructions in the code is continuously adjusted based on the performance information of the code to continuously optimize the code. Moreover, by referring to the congestion degree of each computing unit indicated in the performance information, the adjustment direction of the instruction proportion can be well determined, the code can be quickly adjusted, the optimal code can be determined without traversing all combinations, and it is ensured that the processor resources can be maximally utilized when data is processed by executing the code.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a code processing method and related apparatus. Background Technology

[0002] Current processor architectures typically include various computational units, such as scalar units, vector units, and matrix units. These different types of computational units are usually computational modules designed for different data granularities. Specifically, scalar units are hardware components used to perform operations on single data elements; vector units are hardware components specifically designed for vector operations, supporting the simultaneous execution of the same operation on multiple data elements; and matrix units are dedicated to performing matrix-related operations, such as matrix multiplication, matrix addition, and matrix transpose.

[0003] In most data processing systems, data processing is achieved by invoking various operators to perform operations on the data. Generally, one operator is used to perform a specific type of processing on the data, and different operators correspond to different data processing methods. Taking a processor architecture that includes scalar units and vector units as an example, in current data processing systems, a single operator is often implemented using either pure scalar or pure vector methods. That is, a single operator is implemented using only scalar instructions or only vector instructions. Therefore, when a scalar operator implemented using scalar instructions is executed, the vector unit is often idle; when a vector operator implemented using vector instructions is executed, the scalar unit is idle. This results in the processor's computing resources not being utilized to their fullest potential, leading to a waste of processor resources.

[0004] Therefore, there is an urgent need for a method to improve processor resource utilization during data processing. Summary of the Invention

[0005] This application provides a code processing method and related apparatus that can improve the utilization of processor resources.

[0006] Firstly, a code processing method is provided, applied to a computing device including multiple computing units. The code processing method includes: the computing device executing first code of a first operator in an application, the first code including multiple instructions that can be executed in parallel by multiple computing units, and the ratio of the multiple instructions being a first ratio. That is, in the first code, the multiple instructions can be executed in parallel by multiple computing units in the computing device, and each computing unit executes a corresponding instruction in the first code.

[0007] Then, the computing device collects the first performance information of the first operator during the execution of the first code. The first performance information includes the congestion level of each computing unit and the data processing performance of the computing device when the computing device executes the first code. The data processing performance is used to indicate the amount of data processed by the computing device per unit time when the computing device executes the first code.

[0008] Secondly, the computing device determines whether the data processing performance of the first operator has reached its optimal value based on the data processing performance in the first performance information and the data processing performance of the first operator collected at least once before this run. That is, by comparing the data processing performance corresponding to the first code with the data processing performance corresponding to at least one code previously executed by the computing device for the first operator, it can be determined whether the data processing performance of the first operator has reached its optimal value during the code adjustment process.

[0009] If the computing device determines that the data processing performance of the first operator has not reached its optimal value, the computing device adjusts the ratio of various instructions to a second ratio based on the first performance information, generates second code according to the second ratio, and runs the second code.

[0010] If the computing device determines that the data processing performance of the first operator has reached its optimal value, the computing device will use the code corresponding to the optimal value as the code for the first operator to run on the computing device.

[0011] In this scheme, the first code of the first operator contains multiple instructions that can be executed in parallel by multiple computing units, and the first performance information corresponding to the execution of the first code is collected. By comparing the first performance information of the first code with the performance information of the first operator when using code implementations with different instruction ratios, it can be confirmed whether the data processing performance of the first operator has reached its optimal value. If the data processing performance of the first operator has not reached its optimal value, the ratio of multiple instructions in the code is adjusted based on the first performance information of the first code, thereby generating the second code corresponding to the first operator to achieve continuous code optimization. Furthermore, by referring to the congestion level of each computing unit indicated in the first performance information, the direction of instruction ratio adjustment can be clearly identified, enabling rapid code adjustment. The optimal processing performance code can be determined without traversing all ratios, ensuring that processor resources are maximized when processing data by executing the code.

[0012] In one possible implementation, if the data processing performance of the first operator in at least n previous runs is greater than the data processing performance in the first performance information, it means that continuous code adjustments can no longer achieve better data processing performance. The computing device then determines that the data processing performance of the first operator has reached its optimal value, and takes the highest data processing performance among the at least n data processing performances as the optimal value. Here, n is a value greater than 1, such as a pre-set threshold of 2, 3, or 4.

[0013] Alternatively, if the difference between two adjacent data processing performance values ​​in the m consecutively collected data processing performances of the first operator is less than a preset difference value, it means that the data processing performance of the last m consecutively adjusted codes is very close. In this case, the computing device determines that the data processing performance of the first operator has reached the optimal value, and takes the highest data processing performance among at least m data processing performances as the optimal value. Here, m is a value greater than 2, such as a preset threshold like 3 or 4.

[0014] In this solution, by comparing the data processing performance of multiple sets of code that have been adjusted and are running, it is possible to determine whether the current code adjustment process has reached a convergence state, and then determine whether further code adjustments are needed, ensuring that code adjustments can be terminated in a timely manner and saving resource overhead incurred during the code adjustment process.

[0015] In one possible implementation, when adjusting the ratio of multiple instructions, the computing device can adjust the ratio of multiple instructions to a second ratio based on the differences in congestion levels between the various computing units in the first performance information. That is, the goal of the computing device in adjusting the ratio of multiple instructions in the code is to minimize the differences in congestion levels between the various computing units so that the various computing units can execute instructions simultaneously and in parallel without the phenomenon of computing units waiting for each other.

[0016] In one possible implementation, the multiple computing units are divided into two types. When adjusting the proportion of multiple instructions, provided that the data processing performance in the first performance information is not lower than the data processing performance of the first operator collected before this run, the proportion of instructions executed by the first computing unit among the multiple instructions is increased based on the difference in congestion levels among the various computing units in the first performance information, resulting in a second proportion. The first computing unit is the computing unit with the lowest congestion level among the multiple computing units. That is, the computing device can increase the proportion of instructions executed by the computing unit with a lower congestion level, thereby enabling that computing unit to process more instructions and avoiding long waiting times for other computing units.

[0017] If the data processing performance in the first performance information is lower than the data processing performance of the first operator in the previous run, then based on the difference in congestion levels among the various computing units in the first performance information, the proportion of instructions executed by the second computing unit among the various instructions is reduced to obtain a second proportion. The second computing unit is the computing unit with the highest congestion level among the various computing units. That is, the computing device can reduce the proportion of instructions executed by the computing unit with a high degree of congestion, thereby enabling that computing unit to process fewer instructions and avoiding other computing units having to wait for that computing unit for a long time.

[0018] In one possible implementation, during the generation of the second code, the computing device may first obtain the code corresponding to each computing unit executing a single instruction (i.e., the basic implementation code of various instructions). Then, based on the code corresponding to each computing unit executing a single instruction and a second ratio, the computing device generates the second code using a code generation method. The code corresponding to each computing unit executing a single instruction may be provided in advance by the user or generated in advance by the computing device based on the operational logic of the first operator.

[0019] In this solution, based on pre-written code for implementing a single instruction, the ratio of multiple instructions is used as a constraint. A code generation method is employed to generate code with a specific instruction ratio, which enables rapid online adjustment of the code, reduces the initial development cost of the code, and ensures that the code with the best execution performance on the current computing device can be obtained.

[0020] In one possible implementation, the computing devices include at least two of the following computing units: scalar units, vector units, and matrix units.

[0021] Secondly, a code processing apparatus is provided, applied to a computing device including multiple computing units. The code processing apparatus includes: a code execution module for executing first code of a first operator in an application, the first code including multiple instructions that can be executed in parallel by multiple computing units, and the ratio of the multiple instructions is a first ratio; a processing module for collecting first performance information of the first operator during the execution of the first code, the first performance information including the congestion level of each computing unit when the computing device executes the first code and the data processing performance of the computing device, the data processing performance being used to indicate the amount of data processed per unit time when the computing device executes the first code; the processing module is further used to determine whether the data processing performance of the first operator has reached an optimal value based on the data processing performance in the first performance information and at least one data processing performance collected by the first operator before the current run; if the processing module determines that the data processing performance of the first operator has not reached the optimal value, the processing module is further used to adjust the ratio of the multiple instructions to a second ratio based on the first performance information, and generate second code according to the second ratio, and run the second code; if the processing module determines that the data processing performance of the first operator has reached the optimal value, the processing module is further used to use the code corresponding to the optimal value as the code for the first operator to run on the computing device.

[0022] In one possible implementation, if the data processing performance of the first operator collected at least n times before this run is greater than the data processing performance in the first performance information, the processing module is further used to determine that the data processing performance of the first operator has reached the optimal value, and the highest data processing performance among the at least n data processing performances is taken as the optimal value.

[0023] Alternatively, if the difference between two adjacent data processing performance values ​​in the m consecutively collected data processing performances of the first operator is less than a preset difference value, the processing module is further used to determine that the data processing performance of the first operator has reached the optimal value, and the highest data processing performance among at least m data processing performances is taken as the optimal value.

[0024] In one possible implementation, the processing module is further configured to adjust the ratio of multiple instructions to a second ratio based on the differences in the congestion levels of the various computing units in the first performance information.

[0025] In one possible implementation, the multiple computing units are two types of computing units. The processing module is specifically used to: when the data processing performance in the first performance information is not lower than the data processing performance of the first operator in the previous data collected before this operation, increase the proportion of the instructions executed by the first computing unit in the multiple instructions according to the difference between the congestion levels of each computing unit in the first performance information to obtain a second proportion, wherein the first computing unit is the computing unit with the lowest congestion level among the multiple computing units.

[0026] In one possible implementation, the processing module is specifically used to: when the data processing performance in the first performance information is lower than the data processing performance of the first operator in the previous data collection before this run, reduce the proportion of instructions executed by the second computing unit among the various instructions based on the difference in the congestion level between the various computing units in the first performance information, to obtain a second proportion, wherein the second computing unit is the computing unit with the highest congestion level among the various computing units.

[0027] In one possible implementation, the processing module is specifically used to: obtain the code corresponding to each computing unit when executing a single instruction; and generate a second code based on the code corresponding to each computing unit when executing a single instruction and a second ratio.

[0028] In one possible implementation, the computational units include at least two of the following: scalar units, vector units, and matrix units.

[0029] Thirdly, a code processing apparatus is provided, comprising: a processor and a memory; the memory is used to store computer instructions, which, when executed by the processor, cause the code processing apparatus to perform the methods of any of the above aspects.

[0030] Fourthly, a computer-readable storage medium is provided that stores instructions which, when executed on a computer, cause the computer to perform the methods of any of the above aspects.

[0031] Fifthly, a computer program product containing instructions is provided, which, when executed on a computer, enable the computer to perform the methods described above.

[0032] In a sixth aspect, a chip system is provided, the chip system including a processor and a communication interface for communicating with a module other than the chip shown, the processor for running computer programs or instructions such that an apparatus on which the chip system is mounted can perform the methods of any of the above aspects.

[0033] In a seventh aspect, a computing device is provided, the computing device including a code processing apparatus of the third aspect or a chip system of the sixth aspect, wherein the code processing apparatus or the chip system in the computing device is used to implement the operational steps of the method of any of the above aspects.

[0034] Eighthly, a computing device cluster is provided, comprising at least one computing device, wherein any one computing device is used to run a computer program or instructions, such that the computing device cluster can perform the methods of any of the above aspects. Alternatively, some or all of the computing devices are used together to run a computer program or instructions, such that the computing device cluster can perform the methods of any of the above aspects.

[0035] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0036] Figure 1 A schematic diagram of a system architecture provided for this application;

[0037] Figure 2 A flowchart illustrating a code processing method provided in this application;

[0038] Figure 3 A schematic diagram of an online label-vector hybrid architecture provided in this application;

[0039] Figure 4 A flowchart illustrating a code adjustment process provided in this application;

[0040] Figure 5 This application provides a schematic diagram illustrating the performance information required to be collected during code execution.

[0041] Figure 6 A flowchart illustrating an adjustment of instruction allocation in code based on recorded performance information provided in this application;

[0042] Figure 7 A schematic diagram illustrating a function registration method provided in this application;

[0043] Figure 8 A schematic diagram of code generation provided for this application;

[0044] Figure 9 An illustration of another code generation method provided for this application;

[0045] Figure 10 A schematic diagram of the structure of a code processing device provided in this application;

[0046] Figure 11 A schematic diagram of the structure of a computing device provided in this application;

[0047] Figure 12 This is a schematic diagram of the structure of a computer-readable storage medium provided in this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application are described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some, and not all, of the embodiments of this application. Those skilled in the art will recognize that, with the emergence of new application scenarios, the technical solutions provided by this application are also applicable to similar technical problems.

[0049] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such descriptions can be used interchangeably where appropriate to allow embodiments to be implemented in a sequence other than that illustrated or described in this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps appearing in this application does not imply that the steps in the method flow must be performed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical purpose, as long as the same or similar technical effect is achieved. The division of units in this application is a logical division. In practical applications, there may be other division methods. For example, multiple units may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the shown or discussed mutual coupling, direct coupling, or communication connection may be through some interface, and the indirect coupling or communication connection between units may be electrical or other similar forms, none of which are limited in this application. Furthermore, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed among multiple circuit units. Some or all of the units can be selected to achieve the purpose of the solution in this application according to actual needs.

[0050] To facilitate understanding, some technical terms used in this application will be introduced below.

[0051] (1) Scalar Instructions

[0052] Scalar instructions are a type of instruction in computer architecture that processes only a single data element at a time. Generally, scalar instructions are executed by scalar units in the processor.

[0053] (2) Vector Instruction

[0054] Vector instructions are a core implementation of the Single Instruction Multiple Data (SIMD) paradigm in computer architecture. Essentially, they are instructions that can perform the same operation on a single, ordered set of data elements (i.e., a vector). For example, a vector instruction can perform addition on four single-precision floating-point numbers simultaneously. Generally, vector instructions are executed by vector units within the processor.

[0055] (3) Scalar unit

[0056] A scalar unit is a hardware component in a processor used to perform operations on individual data elements, such as the arithmetic and logic unit (ALU).

[0057] (4) Vector Unit

[0058] A vector unit is a hardware component in a processor specifically designed for vector operations, enabling the simultaneous execution of the same operation on multiple data elements. Vector units include, for example, floating-point based single-instruction multiple-data units (FSUs).

[0059] (5) Operators

[0060] In the field of data processing technology, an operator is a basic unit that performs specific mathematical or logical operations.

[0061] (6) Code Generation

[0062] Code generation technology is a technique that uses automated tools or algorithms to transform abstract requirements (such as natural language descriptions, design documents, data structures, etc.) into concrete executable code. Specifically, based on code generation technology, executable code can be automatically generated according to input constraints, thereby improving software development efficiency and enhancing software maintainability and scalability.

[0063] Specifically, code generation technology is a runtime technology that dynamically converts source code or intermediate representation (IR) into target machine code (i.e., binary code) through Just-In-Time (JIT) compilation. Code generation technology can optimize for the current code runtime environment, achieving efficient instruction selection, register allocation, and code generation, thereby improving code performance.

[0064] (7) Intermediate representation

[0065] Intermediate representation, also known as intermediate code, is an intermediate representation form in programming languages. It acts as an intermediate layer between the source code and the target machine code. Intermediate representations are used by compilers to optimize and transform code, and are also used in static analysis and code generation.

[0066] The design goal of intermediate representations is to provide a middle layer between high-level languages ​​and low-level machine languages, allowing compilers to perform various optimizations at this level. By converting source code into an intermediate representation, compilers can better understand the meaning of the code and perform optimizations to improve program efficiency and performance.

[0067] The applicant's research revealed that in current data processing systems, even when processor architectures include multiple computational units, individual operators often operate using only one type of computational unit. Taking a processor architecture that includes scalar and vector units as an example, individual operators are often implemented using pure scalar or pure vector methods, which can easily lead to underutilization of processor resources and waste of resources. Specifically, the ratio of scalar to vector units in different processing architectures is often different, and the execution time of different operators' computational logic in scalar and vector units often differs. Therefore, if a hybrid scalar and vector approach is used to implement an operator, the ratio of scalar to vector units in the processing architecture, as well as the operator's computational logic itself, will affect the implementation of the operator code. That is, the ratio of scalar instructions to vector instructions in the operator code needs to be set to an appropriate value based on the processing architecture and the operator's computational logic to achieve optimal execution performance. In particular, in some cases, different operating system environments may also affect the implementation of the operator code. Therefore, in order to simplify the implementation of operator code, related technologies often use pure scalars or pure vectors to implement operators.

[0068] Generally, in most cases, implementing a single operator using pure scalars or pure vectors does not significantly impact the performance of data processing. However, for frequently used operators, these operators may be invoked for most of the data processing. Therefore, because these frequently used operators are executed at a high frequency, implementing them using pure scalars or pure vectors can easily lead to underutilization of processor resources, resulting in wasted processor resources.

[0069] In view of this, this application provides a code processing method that sets multiple instructions in the first code of a first operator that can be executed in parallel by multiple computing units, and collects the first performance information corresponding to the runtime of the first code. By comparing the first performance information of the first code with the performance information of the first operator when using code implementations with different instruction ratios, it can be confirmed whether the data processing performance of the first operator has reached its optimal value. When the data processing performance of the first operator has not reached its optimal value, the ratio of multiple instructions in the code is adjusted based on the first performance information of the first code, thereby generating the second code corresponding to the first operator, to achieve continuous code optimization. Furthermore, by referring to the congestion level of each computing unit indicated in the first performance information, the direction of instruction ratio adjustment can be clearly identified, enabling rapid code adjustment. It is possible to determine the code with optimal processing performance without traversing all ratios, ensuring that processor resources are maximized when processing data by executing the code.

[0070] Please see Figure 1 , Figure 1 This is a schematic diagram of a system architecture provided for this application. (For example...) Figure 1 As shown, in this system architecture, the electronic device 10 includes a computing device 11 and memory 12. The electronic device 10 can be a server, physical host, smartphone, laptop, tablet, or smart car in a data center or AI cluster. In the electronic device 10, the computing device 11 is used to execute the code processing method provided in this application. The computing device 11 can be, for example, a processor with computing capabilities such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or a Neural Network Processing Unit (NPU). Furthermore, the computing device 11 can include various computing units such as scalar units 111 for executing scalar instructions, vector units 112 for executing vector instructions, and matrix units 113 for executing matrix operation instructions. The computing device 11 can have one or more scalar units 111, and the number of vector units 112 and matrix units 113 can also be one or more. Of course, the above three types of computing units are only one example provided in this application; the computing device 11 can include two or more types of computing units, and this application does not specifically limit the type of computing units.

[0071] In addition, memory 12 in the system architecture can be used to store generated code or performance information collected during the execution of code by computing device 11.

[0072] During the operation of the computing device 11, the computing device 11 can adjust the ratio of various instructions in the code to obtain code with different instruction ratios. Furthermore, the computing device 11 can run the obtained code and collect the code's performance information, thereby enabling targeted adjustment of the ratio of various instructions in the code based on the performance information during the execution of each set of code, ultimately obtaining a set of code with optimal performance.

[0073] Please see Figure 2 , Figure 2 This is a flowchart illustrating a code processing method provided in this application. Figure 2 As shown, the code processing method provided in this application is applied to a computing device that includes multiple computing units. These multiple computing units may include at least two of scalar units, vector units, and matrix units. Of course, the multiple computing units may also be combinations of other different types of computing units; this application does not limit the specific types of the multiple computing units.

[0074] For ease of description, the code processing method provided in this application will be described below using various computing units of a computing device, including scalar units and vector units, as examples. Specifically, the code processing method includes the following steps 201-205.

[0075] Step 201: Execute the first code of the first operator in the application. The first code includes multiple instructions that can be executed in parallel by multiple computing units, and the ratio of the multiple instructions is a first ratio.

[0076] In this application, the first code can specifically be the implementation code of a first operator. When executed, the first code can perform data processing based on the computational logic of the operator. For example, the first code may be the implementation code of a hash operator, which can perform hash operations on the data; or, the first code may be the implementation code of an aggregation operator, which can perform aggregation operations on the data; or, the first code may be the implementation code of a compression operator, which can perform compression operations on the data.

[0077] Furthermore, the first code includes multiple instructions that can be executed in parallel. Each instruction can be executed by one of the multiple computing units in the computing device. Therefore, the multiple instructions can be executed in parallel by the multiple computing units in the computing device, that is, each computing unit executes the corresponding instruction in the first code. Thus, when the computing device executes the first code, it can simultaneously execute multiple instructions in the first code. For example, taking the multiple computing units as including scalar units and vector units, while the scalar units in the computing device process the scalar instructions in the first code, the vector units in the computing device also simultaneously process the vector instructions.

[0078] Furthermore, in the first code, the ratio of various instructions that can be executed in parallel is a first ratio. For example, taking multiple computing units including scalar units and vector units as an example, the ratio between the number of scalar instructions and vector instructions that can be executed simultaneously can be M:N. Here, M is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

[0079] Optionally, the multiple instructions in the first code can be used to perform the same operation on the data, such as hashing, aggregation, or compression. That is, when the first code is executed, it can perform a certain operation on the data, while the multiple instructions in the first code are used to perform the same operation on different parts of the input data of the first code, thereby collaboratively completing the processing of the input data.

[0080] Step 202: Collect first performance information of the first operator during the execution of the first code. The first performance information includes the congestion level of each computing unit and the data processing performance of the computing device when the computing device executes the first code. The data processing performance is used to indicate the amount of data processed by the computing device per unit time when the computing device executes the first code.

[0081] After acquiring the first code, the computing device can execute the first code and continuously collect first performance information during the execution of the first code to obtain the execution performance of the first code. In the first performance information corresponding to the first code, the congestion level of each type of computing unit is related to the proportion of instructions processed by that type of computing unit in the first code. Taking multiple instructions, including scalar instructions and vector instructions, as an example, the higher the proportion of scalar instructions in the first code, the higher the congestion level of scalar units tends to be. Similarly, the congestion level of vector units is related to the proportion of vector instructions in the first code. The higher the proportion of vector instructions in the first code, the higher the congestion level of vector units tends to be.

[0082] The data processing performance of the first code is related to the ratio of the number of different instructions within it. Generally, the data processing performance of the first code is maximized when the ratio of different instructions is appropriate. Taking multiple instructions, including scalar instructions and vector instructions, as an example, when the number of scalar instructions and vector instructions in the first code reaches an appropriate ratio, a good balance is achieved between scalar units and vector units. This maximizes the utilization of both scalar units and vector units, thereby maximizing the amount of data processed per unit time when executing the first code.

[0083] In general, the congestion level of each computing unit characterizes how busy that unit is when processing instructions. For example, the congestion level of a scalar unit characterizes how busy it is when processing scalar instructions, and the congestion level of a vector unit characterizes how busy it is when processing vector instructions. Therefore, by combining the congestion level of each computing unit, the computing device can determine which instruction has a higher proportion in the current first code. In the first performance information, data processing performance characterizes the actual execution performance of the first code.

[0084] Step 203: Based on the data processing performance in the first performance information and the data processing performance of the first operator collected at least once before this run, determine whether the data processing performance of the first operator has reached the optimal value.

[0085] In this application, the first code is an implementation code corresponding to the first operator. Before executing the first code, the computing device also executes at least one code corresponding to the first operator (the proportion of multiple instructions in this at least one code is different from the proportion of multiple instructions in the first code), and collects the data processing performance corresponding to the at least one code, thereby obtaining at least one data processing performance of the first operator collected before this run. In this way, by comparing the data processing performance corresponding to the first code with the at least one data processing performance of the first operator collected before this run (i.e., the data processing performance corresponding to the previous at least one code), it can be determined whether the data processing performance of the first operator has reached the optimal value during the code adjustment process.

[0086] For example, if the data processing performance of the first operator in at least n previous runs is greater than the data processing performance in the first performance information, it means that the code cannot achieve better data processing performance after continuous adjustments. Therefore, it can be determined that the data processing performance of the first operator has reached the optimal value, and the highest data processing performance among the at least n data processing performances is taken as the optimal value. Here, n is a value greater than 1, such as a pre-set threshold of 2, 3, or 4.

[0087] Alternatively, if the difference between two adjacent data processing performance values ​​in the m consecutively collected data processing performance of the first operator is less than the preset difference value, it means that the data processing performance of the last m consecutively adjusted codes is very close. Therefore, it can be determined that the data processing performance of the first operator has reached the optimal value, and the highest data processing performance among at least m data processing performances is taken as the optimal value.

[0088] Step 204: If it is determined that the data processing performance of the first operator has not reached the optimal value, then the ratio of various instructions is adjusted to a second ratio based on the first performance information, and second code is generated and executed according to the second ratio.

[0089] If the data processing performance of the first operator is not optimal, the computing device may determine that it is necessary to further adjust the proportions of various instructions in the code of the first operator to obtain code with optimal data processing performance. At this point, the computing device can adjust the proportions of various instructions to a second proportion based on the first performance information, generate second code (i.e., the new code corresponding to the first operator) according to the second proportion, and run the second code. Furthermore, after running the second code, the computing device can continue to collect second performance information of the first operator during the execution of the second code, in order to determine whether the data processing performance of the first operator has reached its optimal value by referring to the above steps.

[0090] For example, the computing device can adjust the ratio of various instructions to a second ratio based on the differences in congestion levels between the various computing units in the first performance information. That is, the goal of the computing device in adjusting the ratio of various instructions in the code is to minimize the differences in congestion levels between the various computing units so that the various computing units can execute instructions simultaneously and in parallel without the phenomenon of computing units waiting for each other.

[0091] Specifically, the multiple computing units can be of two types. When adjusting the proportion of multiple instructions, provided that the data processing performance in the first performance information is not lower than the data processing performance of the first operator in the previous run, the proportion of instructions executed by the first computing unit among the multiple instructions is increased based on the difference in congestion levels among the various computing units in the first performance information, resulting in a second proportion. The first computing unit is the computing unit with the lowest congestion level among the multiple computing units. That is, the computing device can increase the proportion of instructions executed by the computing unit with a lower congestion level, thereby enabling that computing unit to process more instructions and avoid waiting for other computing units for a long time.

[0092] If the data processing performance in the first performance information is lower than the data processing performance of the first operator in the previous run, then based on the difference in congestion levels among the various computing units in the first performance information, the proportion of instructions executed by the second computing unit among the various instructions is reduced to obtain a second proportion. The second computing unit is the computing unit with the highest congestion level among the various computing units. That is, the computing device can reduce the proportion of instructions executed by the computing unit with a high degree of congestion, thereby enabling that computing unit to process fewer instructions and avoiding other computing units having to wait for that computing unit for a long time.

[0093] Optionally, in the process of generating the second code, the computing device may first obtain the code corresponding to each computing unit executing a single instruction (i.e., the basic implementation code of various instructions). Then, based on the code corresponding to each computing unit executing a single instruction and a second ratio, the computing device generates the second code using a code generation method. The code corresponding to each computing unit executing a single instruction may be provided in advance by the user or generated in advance by the computing device based on the operational logic of the first operator.

[0094] In this system, the code corresponding to each computing unit executing a single instruction can be source code or an intermediate representation, while the second code is binary code. The intermediate representation is an abstract code representation, situated between source code and binary code, allowing the compiler to more easily optimize and transform the code. In other words, the actual executable second code is generated through a code generation method; it does not require pre-deployment of executable code on the computing device to facilitate code adjustments based on the code generation method.

[0095] In this way, based on pre-written code to implement a single instruction, and using the ratio of multiple instructions as constraints, code generation methods are employed to generate code with a specific instruction ratio. This enables rapid online adjustment of the code, reduces upfront development costs, and ensures that the code with the best execution performance on current computing devices can be obtained.

[0096] Step 205: If it is determined that the data processing performance of the first operator has reached the optimal value, then the code corresponding to the optimal value is used as the code for the first operator to run on the computing device.

[0097] When the data processing performance of the first operator reaches its optimal value, the computing device can select the code corresponding to the optimal data processing performance as the code for the first operator to run on the computing device, thereby completing the code optimization.

[0098] In simple terms, in the application, after collecting performance information of the first code, the computing device can adjust the proportions of various instructions in the first code based on pre-set tuning goals, thereby obtaining new code with different proportions of various instructions. Furthermore, when adjusting the code, the computing device's tuning goal is actually to reduce the differences in congestion levels between various computing units while improving the data processing performance corresponding to the code.

[0099] Specifically, the purpose of the computing device adjusting the code corresponding to the first operator is actually to obtain a set of codes with optimal data processing performance. Furthermore, since the data processing performance of a code is determined by the ratio of the number of various instructions that can be executed simultaneously within the code, the process of adjusting the code is essentially adjusting this ratio. Therefore, to quickly find the optimal ratio of various instructions, the computing device in this application can guide the code adjustment based on the congestion level of each computing unit. That is, each time the computing device adjusts the code, it aims to reduce the differences in congestion levels between different computing units, thereby making the ratio of various instructions in the adjusted code increasingly closer to the optimal instruction ratio, ensuring that the code adjustment is completed with as few adjustments as possible, and improving the efficiency of code adjustment.

[0100] Furthermore, to ensure that while obtaining code with superior performance is achieved, the resource consumption associated with code adjustments is minimized, the computing device can continuously perform multiple adjustments to the code corresponding to the first operator based on the optimization target, until the adjustment termination condition is met. That is, after the adjustment termination condition is met, the computing device stops adjusting the code corresponding to the first operator. The adjustment termination condition includes one or more of the following: the data processing performance of the code obtained from the last adjustment is less than the data processing performance of the n sets of code already obtained; the number of adjustments to the first code reaches a preset number; or the difference in data processing performance between the codes obtained after n consecutive adjustments is less than a preset difference value.

[0101] Specifically, during the adjustment of the first code, if the data processing performance of the code obtained by the last adjustment is less than the data processing performance of the n sets of code obtained so far, it means that the code adjustment direction has moved in the direction of negative optimization, that is, the code with the best performance has been obtained. At this time, the computing device can stop adjusting the code corresponding to the first operator.

[0102] When the code corresponding to the first operator has been adjusted a preset number of times, it means that the code corresponding to the first operator has been adjusted sufficiently, and the computing device can stop adjusting the code corresponding to the first operator to avoid excessive resource consumption during the code adjustment process. The preset number of times is, for example, 10, 15, or 20, and can be determined according to the actual application scenario; no specific limit is set here.

[0103] During the adjustment process of the code corresponding to the first operator, if the difference in data processing performance between the codes obtained after m consecutive adjustments is less than a preset difference value, the computing device can compare the data processing performance difference between the adjusted set of code and the previous set of code. If the data processing performance difference between m consecutive sets of code and the previous set of code is less than a preset percentage (such as 2%, 3%, or 5%), it means that the code adjustment has stabilized, and the computing device can stop adjusting the code corresponding to the first operator. Here, m is an integer greater than 1.

[0104] To facilitate understanding, the following uses various computational units, including scalar units and vector units, as examples to explain in detail the process by which a computing device adjusts the first code.

[0105] Specifically, during the process of the computing device adjusting the code corresponding to the first operator, the computing device obtains the performance information during the execution of the i-th adjustment result. The i-th adjustment result is the code obtained when the i-th adjustment is performed on the code corresponding to the first operator, where i is an integer greater than or equal to 1.

[0106] Then, based on the performance information during the execution of the i-th adjustment result, the computing device increases or decreases the number of scalar instructions or vector instructions in the i-th adjustment result to obtain the (i+1)-th adjustment result. That is, the (i+1)-th adjustment result is actually obtained based on the i-th adjustment result. The computing device can obtain the (i+1)-th adjustment result by further adjusting the number of scalar instructions or vector instructions in the i-th adjustment result.

[0107] Specifically, since the result of the i-th adjustment is itself executable code, the performance information during the execution of the i-th adjustment result will include the congestion level of scalar units and vector units. Based on the reduction of the difference between the congestion levels of scalar units and vector units indicated in the tuning objective, the computing device can adjust the number of scalar instructions or vector instructions in the i-th adjustment result to obtain the (i+1)-th adjustment result.

[0108] In this solution, during the code adjustment process, the performance information of the previous code adjustment result is referenced to determine the method of the next code adjustment. This allows the code adjustment process to proceed in a regular manner along a predetermined direction, which helps the computing device quickly locate the code with the best execution performance and improves the efficiency of code adjustment.

[0109] For example, if the data processing performance of the i-th adjustment result is not lower than that of the (i-1)-th adjustment result, it means that the code execution performance has improved after the i-th adjustment. Therefore, the computing device continues to increase the number of scalar instructions or vector instructions in the code based on the congestion level of the scalar unit and the vector unit corresponding to the i-th adjustment result, to obtain the (i+1)-th adjustment result. For instance, if the congestion level of the scalar unit corresponding to the i-th adjustment result is less than that of the vector unit, it means that there are relatively more vector instructions, so the computing device can increase the number of scalar instructions in the code; if the congestion level of the scalar unit corresponding to the i-th adjustment result is not less than that of the vector unit, it means that there are relatively more scalar instructions, so the computing device increases the number of vector instructions in the code.

[0110] If the data processing performance of the i-th adjustment result is lower than that of the (i-1)-th adjustment result, it means that the execution performance of the code deteriorates after the i-th adjustment. Therefore, the computing device can reduce the number of scalar instructions or vector instructions in the code based on the congestion level of the scalar units and vector units corresponding to the i-th adjustment result to obtain the (i+1)-th adjustment result. For example, if the congestion level of the scalar units corresponding to the i-th adjustment result is less than that of the vector units, it means that there are relatively more vector instructions, so the computing device can reduce the number of vector instructions in the code; if the congestion level of the scalar units corresponding to the i-th adjustment result is not less than that of the vector units, it means that there are relatively more scalar instructions, so the computing device can reduce the number of scalar instructions in the code.

[0111] In this solution, by referring to the performance changes after the previous code adjustment, it is determined whether to increase or decrease the number of instructions in the code. This enables the regular adjustment of the number of instructions in the code, which helps to determine the range of instruction ratios in the code with the best execution performance. It also facilitates the quick location of the code with the best execution performance and improves the efficiency of code adjustment.

[0112] The above describes how computing devices adjust the number of instructions in code to achieve optimal execution performance. The following will describe the scenarios in which computing devices adjust code execution.

[0113] In one possible example, the computing device could be used to adjust the code during code testing before it is officially put into use, so as to obtain code that can be officially put into use.

[0114] Specifically, the aforementioned electronic devices, including computing devices, can be code testing devices. The processing architecture (i.e., hardware architecture) of these electronic devices, as well as the processing architecture and operating system of the actual running device when the code is officially deployed, are the same to ensure that the code's execution performance is identical in both the electronic devices and the actual running device. In other words, the electronic devices are essentially devices deployed in an offline environment, and their deployment is based on the actual running device of the code to ensure that the code's execution performance is the same during testing and official deployment.

[0115] In another possible example, electronic devices could involve adjusting the code as it is put into use, ultimately determining the code to be used continuously thereafter.

[0116] Specifically, for code that includes both scalar and vector instructions, the execution performance is affected by the number and performance of scalar and vector units in the processing architecture on which the code runs, as well as the operating system. Therefore, changes in the runtime environment can lead to changes in code performance, potentially resulting in different instruction ratios for different environments. To improve code applicability and prevent code that performs well in offline testing from only being applicable to specific devices, this application proposes adaptive adjustments based on the actual runtime environment before deployment, ensuring optimal performance on every device.

[0117] In other words, the code corresponding to the first operator is deployed on the electronic device as the original code of the target function. When the electronic device needs to call the target function to process data, it can do so by calling and executing the code corresponding to the first operator. Furthermore, based on the actual runtime performance information of the code corresponding to the first operator, the electronic device adjusts the code corresponding to the first operator online to obtain the set of code with optimal data processing performance.

[0118] In this solution, the original code corresponding to the objective function is deployed to an electronic device, and then the electronic device adjusts the original code during actual operation. This ensures that the adjusted code can adapt to the actual operating environment, thereby guaranteeing that the adjusted code can achieve optimal execution performance in the actual operating environment and improving the applicability of the code.

[0119] The execution process of the code processing method provided in this application has been described above. For ease of understanding, the following will describe in detail the implementation process of this code processing method in practical applications, assuming that the computing device only includes scalar units and vector units.

[0120] Please see Figure 3 , Figure 3This is a schematic diagram of an online label-vector hybrid architecture provided in this application. Figure 3 As shown, this application can deploy an online vector hybrid architecture on an electronic device to implement the above-mentioned code processing method online, thereby achieving code adjustment.

[0121] Specifically, the online label-vector hybrid architecture includes multiple modules operating at different stages: a function registration module and a code generation module operating at compile time; a code execution module and a performance monitoring module operating at runtime; and a performance recording module and a tuning module operating at optimization time. The function registration module registers the basic implementation code of functions with the code generation module, enabling the code generation module to generate label-vector hybrid code with different instruction ratios. The code execution module executes the code generated by the code generation module, while the performance monitoring module monitors performance information during code execution. The performance recording module records the performance information corresponding to the code under various instruction ratios, allowing the tuning module to perform code tuning based on this performance information.

[0122] For example, please refer to Figure 4 , Figure 4 This is a flowchart illustrating a code adjustment process provided for this application. Figure 4 As shown, the process of implementing code adjustments online on computing devices includes the following steps 401-407.

[0123] Step 401: The computing device executes the source code.

[0124] Specifically, the computing device can generate raw code based on an initial instruction ratio (e.g., a 1:1 ratio of scalar instructions to vector instructions) using a code generation method, and then execute the raw code.

[0125] Step 402: Calculate the performance information during the execution of the device monitoring code.

[0126] To determine the execution performance of code under various instruction ratios, computing devices can monitor and collect performance information during code execution. For details, please refer to... Figure 5 , Figure 5 This is a schematic diagram illustrating the performance information required to be collected during the execution of inference code, as provided in this application. Figure 5As shown, to evaluate code execution performance, computing devices can consider that the better the mixed execution of scalar and vector instructions in the code, the more efficient the parallelism and pipelined execution of scalar and vector units, and therefore the lower the congestion levels of scalar and vector units. Taking the ALU as the scalar unit and the FSU as the vector unit as an example, the congestion level of the scalar unit can be represented by the ALU bottleneck, and the congestion level of the vector unit can be represented by the FSU bottleneck. Therefore, during code execution, the performance information that computing devices need to collect can include data processing performance, ALU bottleneck, FSU bottleneck, and the number of instructions processed per clock cycle (IPC).

[0127] For example, Table 1 shows examples of performance information collected by the computing device for different generated codes.

[0128] Table 1

[0129] Metafunctions Scalar instructions Vector instructions load Execution time Data processing performance FSU bottleneck ALU bottleneck IPC Hash function 3 1 3000 20 150 20％ 4％ 3.1 Hash function 0 2 20000 200 100 32％ 3％ 2.29 aggregate functions 1 0 62000 300 210 0％ 33％ 1.5 aggregate functions 1 2 4000 10 400 1％ 19％ 3.4

[0130] In Table 1, load represents the amount of data processed during code execution, specifically the number of data items or the total size of the data. Execution time represents the execution duration of the code, indicating the time it takes to execute the code to perform data processing. Therefore, based on load and execution time, the computing device can calculate the data processing performance corresponding to the code. Furthermore, the FSU bottleneck is specifically calculated as the clock cycles of blocked instructions in the FSU queue divided by the total clock cycles of code execution; the ALU bottleneck is specifically calculated as the clock cycles of blocked instructions in the ALU queue divided by the total clock cycles of code execution. In general, a higher FSU bottleneck indicates more vector instructions are blocked, meaning a higher degree of congestion in vector units. A higher ALU bottleneck indicates more scalar instructions are blocked, meaning a higher degree of congestion in scalar units.

[0131] Step 403: The computing device records performance information during code execution.

[0132] Specifically, in order to facilitate the determination of how to adjust the instruction ratio of the code based on the performance information during the execution of each generated code, the computing device can record the performance information during the execution of the generated code. For example, the performance information corresponding to the code with different instruction ratios used to implement the same function can be uniformly recorded in the same tuning table.

[0133] Step 404: The computing device determines whether the code adjustment has met the adjustment termination condition.

[0134] Based on the recorded performance information during code execution, the computing device can further determine whether the code tuning process has reached the tuning termination condition. Specifically, the tuning termination condition may include at least one of the following three conditions: 1. The data processing performance of the code is stable after m consecutive tunings (for example, for the code after m consecutive tunings, the difference in data processing performance between two adjacent sets of code does not exceed 2%); 2. The data processing performance of the tuned code is lower than the data processing performance of the previously generated n sets of code (i.e., the latest tuning result is worse than all previous tuning results); 3. The number of code tunings exceeds a preset number (e.g., 10 times).

[0135] Step 405: If the code adjustment does not meet the adjustment termination condition, the computing device adjusts the instruction ratio in the code based on the recorded performance information.

[0136] For details, please refer to Figure 6 , Figure 6 This application provides a flowchart illustrating how to adjust the instruction allocation in code based on recorded performance information. For example... Figure 6 As shown, the computing device first obtains the recorded performance information corresponding to the code implementing the same function from the tuning table, and determines whether the code adjustment has reached the adjustment termination condition. If the code adjustment has not reached the adjustment termination condition, the computing device determines whether the data processing performance corresponding to the code after the last adjustment has decreased, that is, whether the data processing performance corresponding to the code after the i-th adjustment is lower than the data processing performance corresponding to the code after the (i-1)-th adjustment.

[0137] If the data processing performance of the code corresponding to the last adjustment degrades, the computing device continues to determine whether the FSU bottleneck of the code corresponding to the last adjustment is greater than the ALU bottleneck. If the FSU bottleneck of the code corresponding to the last adjustment is greater than the ALU bottleneck, then the computing device linearly reduces the number of vector instructions (e.g., reduces the number of vector instructions by 1) based on the code of the last adjustment. Furthermore, if reducing the number of vector instructions by 1 would lead to redundant tuning (i.e., the ratio of vector instructions to scalar instructions remains the same as the previous adjustment), then the computing device keeps the number of vector instructions unchanged and multiplies the number of scalar instructions by 2. If the FSU bottleneck of the code corresponding to the last adjustment is not greater than the ALU bottleneck, then the computing device linearly reduces the number of scalar instructions (e.g., reduces the number of scalar instructions by 1) based on the code of the last adjustment. Furthermore, if reducing the number of scalar instructions by 1 would lead to redundant tuning, then the computing device increases the number of vector instructions by 1, multiplies the number of scalar instructions by 2, and then decreases it by 1 again.

[0138] If the data processing performance of the code corresponding to the last adjustment has not decreased, the computing device continues to determine whether the FSU bottleneck of the code corresponding to the last adjustment is greater than the ALU bottleneck. If the FSU bottleneck of the code corresponding to the last adjustment is greater than the ALU bottleneck, then the computing device doubles the number of scalar instructions (e.g., multiplies the number of scalar instructions by 2) based on the code of the last adjustment. In addition, if multiplying the number of scalar instructions by 2 would lead to repeated tuning, then the computing device keeps the number of vector instructions unchanged and multiplies the number of scalar instructions by 2 and then adds 1. If the FSU bottleneck of the code corresponding to the last adjustment is not greater than the ALU bottleneck, then the computing device linearly increases the number of vector instructions (e.g., adds 1) based on the code of the last adjustment. In addition, if adding 1 to the number of vector instructions would lead to repeated tuning, then the computing device adds 1 to the number of vector instructions and multiplies the number of scalar instructions by 2 and then subtracts 1.

[0139] In summary, the logic for instruction ratio optimization can be shown in Table 2.

[0140] Table 2

[0141]

[0142] It's important to note that compared to scalar units, vector units process more data at a time, but they also execute instructions more slowly. Therefore, during processor operation, while scalar units process less data per instruction, their processing time is faster, allowing scalar instructions to grow faster (i.e., multiply). Conversely, while vector units process more data per instruction, their processing time is slightly slower, resulting in a slower growth rate for vector instructions (i.e., linear growth).

[0143] Step 406: The computing device generates adjusted code based on the adjusted instruction ratio using a code generation method and executes the adjusted code.

[0144] After the adjusted instruction ratio is determined, the computing device can generate adjusted code based on the adjusted instruction ratio using a code generation method. The ratio between scalar instructions and vector instructions in the adjusted code is the adjusted instruction ratio obtained by the computing device in step 405.

[0145] To achieve efficient code generation, the computing device can pre-register the basic implementation code of each function with the code generation module. This allows the code generation module to generate code with a specific instruction set (i.e., the adjusted code described above) based on the basic implementation code and instruction ratio of each function. Please refer to... Figure 7 , Figure 7 This is a schematic diagram illustrating a function registration method provided in this application. For example... Figure 7 As shown, the computing device can register the basic implementation code of various functions provided by the user (i.e., the code corresponding to a single scalar instruction and the code corresponding to a single vector instruction) with the code generation module through the function registration module. For example, the computing device can register the basic implementation code of some functions used in data processing, such as hash functions, encoding functions, aggregation functions, compression functions, or encryption functions, with the code generation module so that the code generation module can know the basic implementation code of each function.

[0146] Once the code generation module obtains the basic implementation code of each function, it can then generate the code for each function based on the input instruction ratio, ensuring that the ratio of scalar instructions to vector instructions in the generated code matches the input instruction ratio. Please refer to [link to relevant documentation]. Figure 8 , Figure 8 This is a schematic diagram illustrating a code generation method provided in this application. For example... Figure 8 As shown, the code generation module can obtain the scalar-based implementation code of the hash function (i.e., the scalar hashing method in the diagram) and the vector-based implementation code (i.e., the vector hashing method in the diagram), with an instruction ratio of M:N between scalar and vector instructions. At this point, the code generation module can determine that the required code needs to include M scalar hash instructions and N vector hash instructions that can be executed simultaneously, and generate the code. Figure 8 The specific code is shown below. Figure 8 As shown in the code generated by the code generation module, scalar_hash represents a scalar hash instruction, and a single scalar hash instruction processes one piece of data at a time; vec_hash represents a vector hash instruction, and a single vector hash instruction processes vec pieces of data at a time.

[0147] Furthermore, if the implementation logic of a function is complex (e.g., a function has multiple implementation logics), the computing device can call a scalar-vector hybrid library to generate scalar and vector hybrid code, without requiring developers to provide both scalar and vector basic implementation code. The scalar-vector hybrid library can provide both scalar and vector basic implementation code for various functions.

[0148] For example, please refer to Figure 9 , Figure 9 A schematic diagram illustrating another code generation method provided for this application. For example... Figure 9As shown, the code generation module can obtain the implementation code of complex nested metafunctions from the scalar-vector hybrid library, specifically including the decoded scalar-vector hybrid IR and the hashed scalar-vector hybrid IR. At this point, the code generation module can generate scalar-vector hybrid code by combining the decoded IR and the hashed IR, ensuring that the ratio of scalar instructions to vector instructions is M:N. That is, the code generation module can use one scalar decoding instruction and one scalar hash instruction as a scalar instruction combination to generate M scalar instruction combinations; and the code generation module can use one vector decoding instruction and one vector hash instruction as a vector instruction combination to generate N vector instruction combinations.

[0149] Step 407: Once the code adjustment has reached the termination condition, the computing device outputs code for stable operation.

[0150] Once the code adjustments have reached the termination criteria, the computing device can select the set of code with the highest data processing performance from among the generated sets as the code for subsequent stable execution. In this way, when the computing device subsequently calls the target function, it can achieve the function call by calling and executing the output stable-running code.

[0151] Based on the code adjustment process shown in steps 401-407 above, this application takes aggregate functions as an example to adjust the code corresponding to aggregate functions. The computing device generated four sets of code with different instruction ratios based on the above code adjustment process, and the performance information during the execution of these four sets of code is shown in Table 3.

[0152] Table 3

[0153] Metafunctions Scalar instructions Vector instructions load Execution time Data processing performance FSU bottleneck ALU bottleneck IPC aggregate functions 1 1 3000 20 150 20％ 3％ 3.1 aggregate functions 2 1 2500 10 250 16％ 10％ 3.4 aggregate functions 4 1 62000 300 210 12％ 20％ 1.5 aggregate functions 3 1 2300 10 230 10％ 16％ 2.7

[0154] As shown in Table 3, the code adjustment process for the aggregate functions is described below.

[0155] 1. Generate the first set of code.

[0156] In the first set of code, the ratio of scalar instructions to vector instructions is 1:1. Furthermore, during the execution of the first set of code on the computing device, 3000 lines of data are processed in a total time of 20 milliseconds (ms), resulting in a data processing performance of 150 lines / ms.

[0157] 2. Adjust the first group of code to obtain the second group of code.

[0158] When adjusting the first group of code, it is assumed that the performance optimization effect of the first group of code has improved. Since the FSU bottleneck is greater than the ALU bottleneck when the first group of code is executed, the computing device determines that the adjustment method is to double the scalar instructions, so that the ratio of scalar instructions to vector instructions in the second group of code is 2:1.

[0159] Furthermore, during the execution of the second set of code on the computing device, 2500 lines of data were processed in a total time of 10 milliseconds (ms), resulting in a data processing performance of 250 lines / ms.

[0160] 3. Adjust the second group of code to obtain the third group of code.

[0161] When adjusting the second group of code, it was found that the data processing performance of the second group of code was greater than that of the first group of code (i.e., the performance optimization effect was better), and the FSU bottleneck was still greater than the ALU bottleneck. Therefore, the computing device determined that the adjustment method was to double the scalar instructions, so that the ratio of scalar instructions to vector instructions in the third group of code was 4:1.

[0162] Furthermore, during the execution of the third set of code on the computing device, the data processing performance was 210 lines / ms, with the ALU bottleneck being greater than the FSU bottleneck.

[0163] 4. Adjust the third group of code to obtain the fourth group of code.

[0164] When adjusting the third group of code, it was found that the data processing performance of the third group of code was lower than that of the second group of code (i.e., the performance tuning effect was worse), and a serious ALU bottleneck appeared. Therefore, the computing device determined that the adjustment method was to linearly reduce the number of scalar instructions, so that the ratio of scalar instructions to vector instructions in the fourth group of code was 3:1.

[0165] Finally, after three rounds of optimization, the optimal code set was found to have an instruction ratio of 2 scalar instructions and 1 vector instruction. At this point, the computing device stopped iterating, thus ending the code optimization process. Furthermore, the computing device adopted the code with the 2 scalar instructions and 1 vector instruction ratio as the stable running code for the aggregation function, ensuring that subsequent calls to the aggregation function would always use this code set to perform data aggregation processing.

[0166] The method provided in this application has been described in detail above. Next, the device provided in this application for performing the above method will be described.

[0167] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a code processing device provided in this application. Figure 10As shown, the code processing device is applied to a computing device including multiple computing units. The code processing device includes: a code execution module 1001, used to execute first code of a first operator in an application, the first code including multiple instructions that can be executed in parallel by multiple computing units, and the ratio of the multiple instructions is a first ratio; a processing module 1002, used to collect first performance information of the first operator during the execution of the first code, the first performance information including the congestion level of each computing unit when the computing device executes the first code and the data processing performance of the computing device, the data processing performance being used to indicate the amount of data processed by the computing device per unit time when executing the first code; the processing module 100... 2. It is also used to determine whether the data processing performance of the first operator has reached the optimal value based on the data processing performance in the first performance information and the data processing performance collected at least once before the current operation of the first operator; if the processing module 1002 determines that the data processing performance of the first operator has not reached the optimal value, the processing module 1002 is also used to adjust the ratio of various instructions to a second ratio based on the first performance information, generate second code according to the second ratio, and run the second code; if the processing module 1002 determines that the data processing performance of the first operator has reached the optimal value, the processing module 1002 is also used to use the code corresponding to the optimal value as the code for the first operator to run on the computing device.

[0168] In one possible implementation, if the data processing performance of the first operator collected at least n times before this run is greater than the data processing performance in the first performance information, the processing module 1002 is further used to determine that the data processing performance of the first operator has reached the optimal value, and the highest data processing performance among the at least n data processing performances is taken as the optimal value.

[0169] Alternatively, if the difference between two adjacent data processing performance values ​​of the first operator in the m consecutively collected data processing performances is less than a preset difference value, the processing module 1002 is further used to determine that the data processing performance of the first operator has reached the optimal value, and the highest data processing performance among at least m data processing performances is taken as the optimal value.

[0170] In one possible implementation, the processing module 1002 is further configured to adjust the ratio of multiple instructions to a second ratio based on the difference in congestion levels between the various computing units in the first performance information.

[0171] In one possible implementation, the multiple computing units are converted into two types of computing units, and the processing module 1002 is specifically used for:

[0172] If the data processing performance in the first performance information is not lower than the data processing performance of the first operator in the previous data collection before this operation, the proportion of the instructions executed by the first computing unit in the various instructions is increased according to the difference in the congestion level of each computing unit in the first performance information to obtain the second proportion, and the first computing unit is the computing unit with the lowest congestion level among the various computing units.

[0173] In one possible implementation, the processing module 1002 is specifically used for:

[0174] If the data processing performance in the first performance information is lower than the data processing performance of the first operator in the previous run, the proportion of instructions executed by the second computing unit among the various instructions is reduced according to the difference in the congestion level of each computing unit in the first performance information, to obtain a second proportion, and the second computing unit is the computing unit with the highest congestion level among the various computing units.

[0175] In one possible implementation, the processing module 1002 is specifically used to: obtain the code corresponding to each computing unit when executing a single instruction; and generate a second code based on the code corresponding to each computing unit when executing a single instruction and a second ratio.

[0176] In one possible implementation, the computational units include at least two of the following: scalar units, vector units, and matrix units.

[0177] Both the code execution module 1001 and the processing module 1002 can be implemented in software or in hardware. For example, the implementation of the processing module 1002 will be described below. Similarly, the implementation of the code execution module 1001 can be referenced from the implementation of the processing module 1002.

[0178] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 11 The electronic device 1100 shown can be used to execute the code processing method provided in this embodiment. For example... Figure 11 As shown, the electronic device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate with each other via the bus 1102. It should be understood that this application does not limit the number of processors and memories in the electronic device 1100.

[0179] Bus 1102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 11 The bus 1102 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1102 may include a path for transmitting information between various components of the electronic device 1100 (e.g., memory 1106, processor 1104, communication interface 1108).

[0180] The processor 1104 includes various computing units specifically used to execute the code processing methods provided in this application. The processor 1104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0181] The memory 1106 may include volatile memory, such as random access memory (RAM). The processor 1104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0182] The memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the functions of the aforementioned code execution module and processing module, thereby implementing the code processing method described above. That is, the memory 1106 stores instructions for executing the code processing method.

[0183] The communication interface 1108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the electronic device 1100 and other devices or communication networks.

[0184] This application also provides a chip comprising a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in a storage unit to cause the chip to perform the methods described in the above embodiments. Optionally, the storage unit may be an in-chip storage unit, such as a register or cache. Alternatively, the storage unit may be an external storage unit located within a wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0185] Please refer to Figure 12 , Figure 12 This is a schematic diagram of a computer-readable storage medium provided in this application. This application also provides a computer-readable storage medium in some embodiments, wherein the above-described... Figure 2 The disclosed method can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0186] Figure 12 A conceptual partial view of an example computer-readable storage medium arranged according to at least some of the embodiments shown herein is illustrated schematically. The example computer-readable storage medium includes a computer program for executing computer processes on a computing device.

[0187] In one embodiment, the computer-readable storage medium 1200 is provided using a signal bearer medium 1201. The signal bearer medium 1201 may include one or more program instructions 1202, which, when executed by one or more processors, can provide the above-mentioned... Figure 2 The described function or part of the function.

[0188] In some examples, signal carrying medium 1201 may include computer-readable medium 1203, such as, but not limited to, hard disk drive, compact disc (CD), digital video disc (DVD), digital magnetic tape, memory, ROM or RAM, etc.

[0189] In some embodiments, the signal-bearing medium 1201 may comprise a computer-recordable medium 1204, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some embodiments, the signal-bearing medium 1201 may comprise a communication medium 1205, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). Therefore, for example, the signal-bearing medium 1201 may be transmitted by a wireless communication medium 1205 (e.g., a wireless communication medium conforming to the IEEE 1102.X standard or other transmission protocols).

[0190] One or more program instructions 1202 may be, for example, computer-executable instructions or logical implementation instructions. In some examples, the computing device may be configured to provide various operations, functions, or actions in response to one or more program instructions 1202 conveyed to the computing device via a computer-readable medium 1203, a computer-recordable medium 1204, and / or a communication medium 1205.

[0191] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0192] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods of the various embodiments of this application.

[0193] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0194] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, computer instructions can be transferred from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A code processing method, characterized in that, The method is applied to a computing device comprising multiple computing units, and the method includes: The first code of the first operator in the execution application includes multiple instructions that can be executed in parallel by the multiple computing units, and the ratio of the multiple instructions is a first ratio; Collect first performance information of the first operator during the execution of the first code. The first performance information includes the congestion level of each computing unit when the computing device executes the first code and the data processing performance of the computing device. The data processing performance is used to indicate the amount of data processed by the computing device per unit time when it executes the first code. Based on the data processing performance in the first performance information and the data processing performance of the first operator collected at least once before this operation, determine whether the data processing performance of the first operator has reached the optimal value; If it is determined that the data processing performance of the first operator has not reached the optimal value, the ratio of the various instructions is adjusted to a second ratio based on the first performance information, and second code is generated and executed according to the second ratio. If it is determined that the data processing performance of the first operator has reached its optimal value, then the code corresponding to the optimal value is used as the code for the first operator to run on the computing device.

2. The method according to claim 1, characterized in that, The step of determining whether the data processing performance of the first operator has reached its optimal value based on the data processing performance in the first performance information and the data processing performance of the first operator collected at least once before this operation includes: If the data processing performance of the first operator in at least n data processing tests collected before this run is greater than the data processing performance in the first performance information, then it is determined that the data processing performance of the first operator has reached the optimal value, and the highest data processing performance among the at least n data processing performance tests is taken as the optimal value. Alternatively, if the difference between two adjacent data processing performance values ​​in the m consecutively collected data processing performances of the first operator is less than a preset difference value, then the data processing performance of the first operator is determined to have reached the optimal value, and the highest data processing performance among the at least m data processing performances is taken as the optimal value.

3. The method according to claim 1 or 2, characterized in that, The step of adjusting the ratio of the various instructions to a second ratio based on the first performance information includes: Based on the differences in the congestion levels of the various computing units in the first performance information, the ratio of the multiple instructions is adjusted to a second ratio.

4. The method according to claim 3, characterized in that, The multiple computing units refer to two types of computing units. Adjusting the ratio of the multiple instructions to a second ratio based on the differences in congestion levels among the various computing units in the first performance information includes: If the data processing performance in the first performance information is not lower than the data processing performance of the first operator in the previous data collection before this operation, the proportion of the instructions executed by the first computing unit in the various instructions is increased according to the difference in the congestion level of each computing unit in the first performance information to obtain the second proportion, and the first computing unit is the computing unit with the lowest congestion level among the various computing units.

5. The method according to claim 4, characterized in that, The step of adjusting the ratio of the various instructions to a second ratio based on the differences in congestion levels among the various computing units in the first performance information includes: If the data processing performance in the first performance information is lower than the data processing performance of the first operator in the previous run, the proportion of instructions executed by the second computing unit among the various instructions is reduced according to the difference in congestion level between the various computing units in the first performance information, to obtain the second proportion, whereby the second computing unit is the computing unit with the highest congestion level among the various computing units.

6. The method according to any one of claims 1-5, characterized in that, The step of generating the second code according to the second ratio includes: Obtain the code corresponding to each computing unit when executing a single instruction; The second code is generated based on the code corresponding to the execution of a single instruction by each computing unit and the second ratio.

7. The method according to any one of claims 1-6, characterized in that, The various computational units include at least two of the following: scalar units, vector units, and matrix units.

8. A code processing device, characterized in that, The device is applied to a computing device including multiple computing units, and the device includes: A code execution module is used to execute the first code of the first operator in the application, the first code including a variety of instructions that can be executed in parallel by the various computing units, and the ratio of the various instructions is a first ratio; The processing module is used to collect first performance information of the first operator during the execution of the first code. The first performance information includes the congestion level of each computing unit when the computing device executes the first code and the data processing performance of the computing device. The data processing performance is used to indicate the amount of data processed by the computing device per unit time when it executes the first code. The processing module is further configured to determine whether the data processing performance of the first operator has reached the optimal value based on the data processing performance in the first performance information and the data processing performance of the first operator collected at least once before this operation. If the processing module determines that the data processing performance of the first operator has not reached the optimal value, the processing module is further configured to adjust the ratio of the multiple instructions to a second ratio based on the first performance information, generate second code according to the second ratio, and run the second code; If the processing module determines that the data processing performance of the first operator has reached the optimal value, the processing module is further configured to use the code corresponding to the optimal value as the code for the first operator to run on the computing device.

9. The apparatus according to claim 8, characterized in that, If the data processing performance of the first operator in at least n data collections prior to this run is greater than the data processing performance in the first performance information, then the processing module is further configured to determine that the data processing performance of the first operator has reached the optimal value, and take the highest data processing performance among the at least n data processing performances as the optimal value. Alternatively, if the difference between two adjacent data processing performance values ​​of the first operator in the m consecutively collected data processing performances is less than a preset difference value, the processing module is further used to determine that the data processing performance of the first operator has reached the optimal value, and the highest data processing performance among the at least m data processing performances is taken as the optimal value.

10. The apparatus according to claim 8 or 9, characterized in that, The processing module is also configured to adjust the ratio of the various instructions to a second ratio based on the differences in the congestion levels of the various computing units in the first performance information.

11. The apparatus according to claim 10, characterized in that, The multiple computing units are of two types, and the processing module is specifically used for: If the data processing performance in the first performance information is not lower than the data processing performance of the first operator in the previous data collection before this operation, the proportion of the instructions executed by the first computing unit in the various instructions is increased according to the difference in the congestion level of each computing unit in the first performance information to obtain the second proportion, and the first computing unit is the computing unit with the lowest congestion level among the various computing units.

12. The apparatus according to claim 11, characterized in that, The processing module is specifically used for: If the data processing performance in the first performance information is lower than the data processing performance of the first operator in the previous run, the proportion of instructions executed by the second computing unit among the various instructions is reduced according to the difference in congestion level between the various computing units in the first performance information, to obtain the second proportion, whereby the second computing unit is the computing unit with the highest congestion level among the various computing units.

13. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the operational steps of the method as described in any one of claims 1 to 7.

14. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed by the computer, cause the computer to perform the method according to any one of claims 1 to 7.

15. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1 to 7.

Images