Hybrid SIMD library and programming method based on hybrid SIMD library

Abstract

The present invention discloses a hybrid SIMD library and a programming method based on the hybrid SIMD library, wherein the hybrid SIMD library comprises: at least two SIMD libraries; a hybrid SIMD library interface, wherein the hybrid SIMD library interface is implemented based on a Vector template class, a universal function and each SIMD library template class, wherein the parameters of the Vector template class include the element type of Vector, the parameters of the universal function include the parameters of the Vector template class and the SIMD library template class, the Vector template class encapsulates the intrinsic vector type shared by all SIMD libraries, and the formal parameters and return types of each function implemented in each SIMD library template class are Vector vector types. The present invention can flexibly use different SIMD libraries in the programming process to improve the running efficiency of the program.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer software engineering, and in particular relates to a hybrid SIMD library and a programming method based on the hybrid SIMD library. Background Art

[0002] Single Instruction Multiple Data (SIMD) is a parallel computing technology that performs the same operation in one instruction but on different data. This technology has a wide range of applications in computer science, especially in data-intensive or computation-intensive applications such as artificial intelligence, digital image processing, large-scale scientific computing, audio and video encoding and decoding. SIMD technology can greatly improve data processing efficiency.

[0003] With the development of hardware technology, SIMD technology has received more and more attention. At present, most modern microprocessors support some form of SIMD instruction set, such as Intel's MMX, SSE, AVX series instruction sets, AMD's 3DNow! instruction set, and ARM's NEON instruction set. However, there are differences between different hardware platforms and instruction sets. Therefore, code optimized for a specific hardware platform or instruction set may not achieve the same performance on other platforms. This makes it complicated and difficult to write and maintain code for different platforms.

[0004] To solve this problem, some libraries have emerged to simplify SIMD programming, such as Google's highway library and various open source SIMD libraries, such as mipp, highway, vcl, eve, etc. These libraries provide a series of functions and macros, making it easier for developers to use SIMD instructions. Developers do not need to care about the underlying hardware details, they only need to call the functions or macros provided by these libraries to use SIMD technology for parallel computing.

[0005] However, even with these libraries, developers still need to face a problem, that is, how to choose the most suitable library. Since different SIMD libraries may have large differences in performance when processing different operations, using a single SIMD library for programming may not achieve the best results. Each library has its own advantages and disadvantages. For certain specific operations, a library may have better performance than other libraries. Therefore, how to choose the best library according to specific operations in programming is a problem that needs to be solved.

[0006] In practical applications, some operations are frequently executed and have a significant impact on the overall performance. Finding these operations with poorer performance in a certain SIMD library compared to other SIMD libraries and performing targeted optimizations is crucial to improving the efficiency of the program. However, when programming with SIMD libraries, identifying these operations with poor performance and determining better libraries and parameters often requires a lot of experiments and debugging, which undoubtedly increases the difficulty and complexity of development. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes a hybrid SIMD library and a programming method based on the hybrid SIMD library. The programming method can improve the running efficiency of the program by flexibly using different SIMD libraries.

[0008] The technical solution of the present invention includes:

[0009] A hybrid SIMD library, comprising:

[0010] At least two SIMD libraries;

[0011] A hybrid SIMD library interface, wherein the hybrid SIMD library interface is implemented based on a Vector template class, a universal function, and each SIMD library template class, wherein the parameters of the Vector template class include the element type of Vector, the parameters of the universal function include the parameters of the Vector template class and the SIMD library template class, the Vector template class encapsulates an intrinsic vector type shared by all SIMD libraries, and the formal parameters and return types of each function implemented in each SIMD library template class are Vector vector types;

[0012] Wherein, in the case where the general function defines parameters of the Vector template class and the SIMD library template class, the corresponding SIMD library template class converts the corresponding Vector vector type into the vector type of the corresponding SIMD library to implement the vector function call of the corresponding SIMD library.

[0013] Furthermore, the element types in the intrinsics vector type include: int and / or float.

[0014] Furthermore, the implementation type of the SIMD library includes: EVE, HIGHWAY, MIPP or VCL.

[0015] A programming method based on the hybrid SIMD library, the method comprising:

[0016] Selecting a SIMD library implementation in the hybrid SIMD library for each operation, obtaining an initial individual of the genetic algorithm, and forming an initial population of the genetic algorithm based on the initial individual;

[0017] Testing the performance data of each individual in the current population, and selecting individuals from the current population based on the performance data of the individuals, and performing genetic mutation to obtain the next generation population;

[0018] Under the condition that the genetic algorithm cutoff condition is met, the final individual of each operation is obtained;

[0019] A vector function call of a SIMD library is performed according to the final entity to perform programming.

[0020] Furthermore, the step of selecting a SIMD library implementation for each operation in the hybrid SIMD library to obtain an initial individual of the genetic algorithm includes:

[0021] For each operation, a SIMD library implementation is randomly selected in the hybrid SIMD library;

[0022] or,

[0023] Based on the training data, determine whether the SIMD library corresponding to any operation has insufficient performance for the operation;

[0024] After excluding SIMD libraries with insufficient performance for the operation, a SIMD library implementation for the operation is selected from the remaining SIMD libraries.

[0025] Furthermore, judging whether a SIMD library corresponding to any operation has insufficient performance in the operation based on the training data includes:

[0026] Based on the training data, the corresponding final sample population is obtained, and the performance data of all final individual samples in the final sample population are recorded;

[0027] Keep the SIMD library implementation of other operations unchanged, and form a new individual sample by changing the SIMD library implementation of a final individual sample in the final population of samples;

[0028] Calculate the difference between the individual performance data of the new individual sample and the performance data of the final individual sample;

[0029] When the performance data difference is greater than a set threshold, it is determined whether the SIMD library corresponding to the operation has insufficient performance for the operation.

[0030] Furthermore, the selecting of individuals from the current population based on the performance data of the individuals and performing genetic mutation to obtain the next generation population includes:

[0031] Perform crossover operation on the selected individuals, and implement the SIMD library to interchange the operations on the two operations to obtain new individuals after crossover;

[0032] Adding the new individuals after the crossover to the next generation population;

[0033] and / or,

[0034] Perform crossover operation on the selected individuals, and implement the SIMD library to interchange the operations on the two operations to obtain new individuals after crossover;

[0035] Perform mutation operation on the new individuals after crossover according to the set probability to obtain the mutated new individuals;

[0036] The mutated new individuals are added to the next generation population.

[0037] Furthermore, the operation includes: loading, storing and / or adding.

[0038] A programming device based on a hybrid SIMD library, the device comprising:

[0039] A hybrid SIMD library, the hybrid SIMD library comprising: at least two SIMD libraries and a hybrid SIMD library interface, the hybrid SIMD library interface being implemented based on a Vector template class, a universal function and each SIMD library template class, the parameters of the Vector template class comprising the element type of Vector, the parameters of the universal function comprising the parameters of the Vector template class and the SIMD library implementation type, the Vector template class encapsulating an intrinsic vector type shared by all SIMD libraries, and the formal parameters and return types of the functions implemented in each SIMD library template class being Vector vector types; wherein, in the case where the universal function defines the parameters of the Vector template class and the SIMD library template class, the corresponding SIMD library template class converts the corresponding Vector vector type into the corresponding SIMD library vector type to implement a vector function call of the corresponding SIMD library;

[0040] An initial population construction module is used to select a SIMD library implementation for each operation in the hybrid SIMD library to obtain an initial individual of the genetic algorithm, and to form an initial population of the genetic algorithm based on the initial individual;

[0041] The genetic calculation module is used to test the performance data of each individual in the current population, and select individuals from the current population based on the performance data of the individuals, and then perform genetic mutation to obtain the next generation population; when the genetic algorithm cutoff condition is met, the final individual of each operation is obtained;

[0042] The function calling module is used for calling the vector function of the SIMD library according to the final individual to perform programming.

[0043] A computer device, characterized in that the computer device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the programming method based on the hybrid SIMD library as described in any one of claims 4 to 8 is implemented.

[0044] Compared with the prior art, the technical solution provided by the present invention includes at least the following beneficial effects:

[0045] The present invention can make full use of the advantages of each library and avoid its disadvantages by mixing different SIMD libraries and dynamically selecting the optimal library according to specific operations, thereby achieving optimal performance. Moreover, through the interface testing method, operations with poor performance in a specific SIMD library can be more intuitively and accurately found and optimized in a targeted manner. This can not only improve the operating efficiency of the program, but also reduce the difficulty and complexity of programming.

[0046] In addition, the method of the present invention is a general method, which is applicable not only to the current SIMD library, but also to new SIMD libraries that may appear in the future. With the continuous development of hardware technology and software requirements, new SIMD libraries and new operations will continue to appear. The method of the present invention can cope with these changes well and ensure that the program can always achieve the best performance.

[0047] In the present invention, various combinatorial optimization algorithms can be applied to find the optimal SIMD library combination for each function call point, thereby obtaining the optimal solution for performance. This method provides more sophisticated control and greater flexibility by dynamically selecting different SIMD library implementations, thereby maximizing the performance of the program.

[0048] In general, the present invention provides a programming and interface testing method based on a hybrid SIMD library, which is flexible and efficient and provides a new solution to challenges in the field of parallel computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is the design diagram of the existing SIMD library.

[0050] Figure 2 It is a design diagram of the hybrid SIMD library of the present invention.

[0051] Figure 3 This is the design diagram of the template function being the LOAD function.

[0052] Figure 4 It is an implementation diagram of the LOAD function of the present invention.

[0053] Figure 5 It is a flow chart of the programming method based on hybrid SIMD library of the present invention.

[0054] Figure 6 It is a flow chart of calculating the insufficient performance of SIMD library of the present invention. DETAILED DESCRIPTION

[0055] This section will describe the specific embodiments of the present invention in detail so that those skilled in the art can fully understand and implement the present invention.

[0056] The programming and interface testing method based on hybrid SIMD library of the present invention aims to optimize computing efficiency, provide a more flexible programming method, and facilitate the identification and optimization of hot functions in the program. The core idea is: different SIMD libraries are mixed in programming, and a special interface testing method is used to dynamically select the optimal SIMD library according to the specific operation. In terms of implementation, a unified interface is first provided, so that different SIMD libraries can be mixed in programming in one application. Specifically, by defining a special vector type `Vector <t>`, and implement a series of operations for it, such as `Load_Unaligned`, `Add`, `Store_Unaligned`, etc. The implementation of these operations can dynamically select different SIMD libraries, so as to select the optimal library for operation according to the specific situation at runtime.

[0057] The implementation of the interface testing method is: in a hybrid SIMD library, the SIMD library implementation of a certain operation can be dynamically specified to facilitate finding those operations with poor performance in a specific SIMD library. Specifically, first, all operations are specified as the implementation of the same SIMD library, and the performance of the program in this case is recorded. Then, one of the operations is changed to the implementation of other SIMD libraries, and the performance of the program is recorded again. By comparing the performance data recorded twice, the present invention can obtain the performance change after changing the SIMD library implementation. If the performance is greatly improved, it means that the original SIMD library has insufficient performance when processing the operation. Then, based on this information, a more suitable SIMD library can be selected to implement the operation, thereby improving the overall performance of the program and providing optimization guidance for library developers.

[0058] Specifically, before implementing the present invention, it is necessary to first prepare the required SIMD libraries, such as MIPP, HIGHWAY, VCL, etc. These libraries all provide a series of SIMD operations, such as load, store, addition, etc. The present invention needs to select appropriate libraries as needed and integrate them into the programming environment of the present invention. Specifically, the implementation of these libraries is achieved by packaging intrinsic vector types. Then, taking this as a breakthrough, since they are all implemented by packaging intrinsic vector types, the present invention can use this vector type as a bridge to connect different SIMD libraries. For example, the underlying functions of the HIGHWAY library are actually operated on intrinsic vector types, and the same is true for the other three. The difference between these four SIMD libraries is that the functions they implement operate on intrinsic vector types in different ways. Although they can all output the same results, different implementation methods may result in performance differences. Then the present invention can create a unified vector type based on the intrinsic vector type to connect various SIMD libraries. The implementation details are: Vector <t>It contains intrinsic vector types of different vector instruction sets, such as __m256 type of AVX, etc. The present invention operates this intrinsic vector type through the operation of different SIMD libraries (Vector <t>The operation is actually the operation of the internal intrinsic vector type). In fact, the type of operation changes as follows: Vector <t>--->__m256--->Vector <t>.

[0059] For example, Vector <t>z_simd = Add<T, Library>(x_simd, y_simd), where Add is for Vector <t>The operation is actually an operation on the internal __m256, and the result is also of type __m256. Then the result of type __m256 is converted to Vector <t>Type of result.

[0060] On the basis of the hybrid SIMD library, the first step to implement the present invention is to create a unified interface so that at the application level, the programmer does not need to care which SIMD library function is used. This unified interface can be implemented by creating a new vector type using C++ template technology and overloading technology. The new vector type is, for example, `Vector <t>`, instances of whose type represent a SIMD vector, where `T` is the type of the vector elements, such as `int`, `float`, etc.

[0061] This `Vector <t>The key to the design of ` is that its bottom layer directly depends on the intrinsics type at the hardware level, which is also the common bottom layer implementation of all SIMD libraries, which provides a bridge for the present invention to achieve the unification of these libraries.

[0062] Next, the present invention needs to implement a series of operations, such as `Load_Unaligned`, `Store_Unaligned`, `Add`, etc. The implementation of these operations can dynamically select different SIMD libraries, so that the optimal library can be selected for operation according to the specific situation at runtime. The key technology here is the template technology and overloading technology of C++. For example, the present invention can define a template function `Load_Unaligned`<T,Library> `, where `T` is a data type and `Library` is a SIMD library type. Then, the present invention implements a specialized version for each SIMD library, such as `Load_Unaligned<int,VCL> ` and `Load_Unaligned<int,MIPP> In this way, the present invention can dynamically select different SIMD library implementations by changing the template parameter `Library`.

[0063] Assume that the present invention has a function `axpy`, which needs to perform vector addition and scaling operations, wherein the axpy function contains a for loop, and the for loop contains two loads, a multiplication, an addition and a store operation, wherein each operation is implemented through a template function, and the parameters in the template function include an element type T and a SIMD library implementation type Library, which can be one of EVE, HIGHWAY, MIPP, VCL and the like. When an operation selects EVE as a template parameter, the back end of the operation will select the implementation method of the EVE library, and so on.

[0064] In one embodiment, the steps for completing the design of the hybrid SIMD library include:

[0065] 1. Design the Vector class template.

[0066] (1) Define the Vector class template, with the template parameter being the element type T.

[0067] (2) The Vector class contains a member variable of the built-in vector type.

[0068] (3) Implement the Vector constructor, with parameters being vector types of different SIMD libraries, which are used to convert to Vector types.

[0069] (4) Implement other necessary methods, such as getting size and accessing elements.

[0070] 2. Create template classes for each SIMD library.

[0071] (1) Create a template class for each SIMD library, such as HighwayVector.

[0072] (2) Encapsulate the vector function of the library in a class template, with the parameter and return types being Vector types.

[0073] (3) Implement the conversion function that converts Vector into the vector type of the SIMD library.

[0074] (4) Implement the method of calling the SIMD library vector function.

[0075] 3. Create a general function that acts on Vector.

[0076] (1) Define general function templates such as LOAD and STORE.

[0077] (2) The function template has a SIMD library type template parameter.

[0078] (3) Inside the function, call the corresponding SIMD library implementation according to the template type parameter.

[0079] (4) When using, pass in different libraries, such as HighwayVector, to select the implementation to use.

[0080] In summary, the designs of existing SIMD libraries are mostly designed by encapsulating intrinsic vector types and intrinsic functions through C++ templates, exposing many functions for users to use, such as Figure 1 load, store, etc., these functions encapsulate intrinsic functions to act on intrinsic vector types to achieve acceleration. So the SIMD library essentially encapsulates a layer of interface on top of intrinsic functions and intrinsic vector types for user convenience.

[0081] After finding the essence of many SIMD library designs, a bridge between SIMD libraries can be established through intrinsic vector types and intrinsic functions to achieve mixed use. Therefore, the present invention designs a unified programming interface, such as Figure 2 The present invention also exposes multiple interfaces such as LOAD, STORE, etc. These interfaces of the present invention are implemented by encapsulating functions corresponding to different SIMD libraries, such as Figure 3 Our LOAD function is designed as a template function, which provides a template parameter to select the implementation of different SIMD libraries. Since each SIMD library is implemented through an intrinsic function, our LOAD function is essentially an operation on the intrinsic vector type by the internal intrinsic function of the load implementation of different SIMD libraries, such as Figure 4 .

[0082] Based on the above hybrid SIMD library, the present invention also discloses a programming method. Since manually selecting the optimal SIMD library implementation usually requires a lot of trial and error, in order to solve this problem, the present invention requires an automated method to find the optimal SIMD library implementation combination. First, the present invention defines a default SIMD library implementation for each operation, represented by a special tag, such as "AUTO". Then, the present invention uses a genetic algorithm to find the optimal tag combination.

[0083] A genetic algorithm is a search algorithm that is inspired by natural selection and genetics and searches for the optimal solution through an iterative process. During the iterative process, the genetic algorithm maintains a population in which each individual represents a possible solution, that is, a marker combination. Then, the genetic algorithm selects excellent individuals from the population based on the fitness function and generates new individuals through crossover and mutation operations. In the scenario of the present invention, the fitness function is calculated based on the performance data of the program, and the excellent individuals are those marker combinations that make the program perform better.

[0084] To implement the genetic algorithm, Figure 5 As shown, the present invention first randomly generates a group of initial populations, then applies these populations to the SIMD library implementation of each operation, executes the program, and records the performance data. Then, the present invention sorts the population and selects the best performing individuals as the parent generation. Next, the present invention performs crossover operations and genetic mutations to generate new individuals, and then performs mutation operations on the new individuals to increase the diversity of the population. This process will continue to iterate until the optimal solution that meets the preset conditions of the present invention is found. Finally, the vector function call of the SIMD library is performed based on the final individual of the optimal solution to perform programming.

[0085] In addition, finding those operations where the performance of a certain SIMD library is insufficient compared to other SIMD libraries is also an important part of the present invention. Figure 6 As shown, the present invention can perform this test by following the steps below:

[0086] 1. First, the present invention selects a default SIMD library implementation for all operations, then executes the program and records the performance data.

[0087] 2. Then, the present invention changes the SIMD library implementation of each operation one by one. For example, assume that the present invention has three operations: load, store, and add. The present invention first changes the SIMD library implementation of the load operation, then executes the program and records the performance data. Then, the present invention restores the SIMD library implementation of the load operation to the default, then changes the SIMD library implementation of the store operation, executes the program, and records the performance data. According to this method, the present invention changes the SIMD library implementation of all operations one by one and records the performance data.

[0088] 3. Finally, the present invention analyzes the performance data to find those operations whose performance is greatly improved after changing the SIMD library implementation. These operations are those operations where the performance of a certain SIMD library is insufficient compared to other SIMD libraries.

[0089] Through such testing, the present invention can find the performance bottlenecks in the program and then optimize these performance bottlenecks in a targeted manner.

[0090] This programming and interface testing method based on the hybrid SIMD library not only provides a unified interface so that programmers do not need to worry about the specific SIMD library when programming, but also provides an interface performance testing method so that programmers can quickly identify and optimize performance bottlenecks in the program, thereby greatly improving the performance of the program.

[0091] In summary, the present invention can dynamically select between different SIMD libraries, so that the program can select the optimal library for operation according to the specific situation. At the same time, the present invention also provides an interface testing method, which can observe the performance changes by specifying and changing the SIMD library implementation of a certain operation, thereby finding and optimizing the hotspot functions in the program. Such a method can provide more sophisticated control and greater flexibility, so that the performance of the program is maximized.

[0092] A person skilled in the art may modify or make equivalent substitutions to the technical solution of the present invention, and the protection scope of the present invention shall be subject to the claims.< / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t>

Claims

1. A programming method based on a hybrid SIMD library, characterized in that: The method comprises: For each operation, a SIMD library is selected in the hybrid SIMD library to implement, and an initial individual of the genetic algorithm is obtained, and an initial population of the genetic algorithm is formed according to the initial individual; wherein the hybrid SIMD library comprises: at least two SIMD libraries and a hybrid SIMD library interface, the hybrid SIMD library interface is implemented based on a Vector template class, a universal function and each SIMD library template class, the parameters of the Vector template class include the element type of Vector, the parameters of the universal function include the parameters of the Vector template class and the SIMD library implementation type, the Vector template class encapsulates the intrinsic vector type shared by all SIMD libraries, and the formal parameters and return types of the functions implemented in each SIMD library template class are Vector vector types; in the case where the universal function defines the parameters of the Vector template class and the SIMD library template class, the corresponding SIMD library template class converts the corresponding Vector vector type into the corresponding SIMD library vector type to implement the vector function call of the corresponding SIMD library; Testing the performance data of each individual in the current population, and selecting individuals from the current population based on the performance data of the individuals, and performing genetic mutation to obtain the next generation population; Under the condition that the genetic algorithm cutoff condition is met, the final individual of each operation is obtained; A vector function call of a SIMD library is performed according to the final entity to perform programming.

2. The method according to claim 1, characterized in that The step of selecting a SIMD library implementation for each operation in the hybrid SIMD library to obtain an initial individual of the genetic algorithm includes: For each operation, a SIMD library implementation is randomly selected in the hybrid SIMD library; or, Based on the training data, determine whether the SIMD library corresponding to any operation has insufficient performance for the operation; After excluding SIMD libraries with insufficient performance for the operation, a SIMD library implementation for the operation is selected from the remaining SIMD libraries.

3. The method according to claim 2, characterized in that The step of judging, based on the training data, whether the SIMD library corresponding to any operation has insufficient performance in the operation includes: Based on the training data, the corresponding final sample population is obtained, and the performance data of all final individual samples in the final sample population are recorded; Keep the SIMD library implementation of other operations unchanged, and form a new individual sample by changing the SIMD library implementation of a final individual sample in the final population of samples; Calculate the difference between the individual performance data of the new individual sample and the performance data of the final individual sample; When the performance data difference is greater than a set threshold, it is determined whether the SIMD library corresponding to the operation has insufficient performance for the operation.

4. The method according to claim 1, characterized in that After selecting individuals from the current population based on the performance data of the individuals, genetic mutation is performed to obtain the next generation population, including: Perform crossover operation on the selected individuals, and implement the SIMD library to interchange the operations on the two operations to obtain new individuals after crossover; Adding the new individuals after the crossover to the next generation population; and / or, Perform crossover operation on the selected individuals, and implement the SIMD library to interchange the operations on the two operations to obtain new individuals after crossover; Perform mutation operation on the new individuals after crossover according to the set probability to obtain the mutated new individuals; The mutated new individuals are added to the next generation population.

5. The method according to claim 1, characterized in that The operations include: loading, storing and / or adding.

6. A programming device based on a hybrid SIMD library, characterized in that: The device comprises: A hybrid SIMD library, the hybrid SIMD library comprising: at least two SIMD libraries and a hybrid SIMD library interface, the hybrid SIMD library interface being implemented based on a Vector template class, a universal function and each SIMD library template class, the parameters of the Vector template class comprising the element type of Vector, the parameters of the universal function comprising the parameters of the Vector template class and the SIMD library implementation type, the Vector template class encapsulating an intrinsic vector type shared by all SIMD libraries, and the formal parameters and return types of the functions implemented in each SIMD library template class being Vector vector types; wherein, in the case where the universal function defines the parameters of the Vector template class and the SIMD library template class, the corresponding SIMD library template class converts the corresponding Vector vector type into the corresponding SIMD library vector type to implement a vector function call of the corresponding SIMD library; An initial population construction module is used to select a SIMD library implementation for each operation in the hybrid SIMD library to obtain an initial individual of the genetic algorithm, and to form an initial population of the genetic algorithm based on the initial individual; The genetic calculation module is used to test the performance data of each individual in the current population, and select individuals from the current population based on the performance data of the individuals, and then perform genetic mutation to obtain the next generation population; when the genetic algorithm cutoff condition is met, the final individual of each operation is obtained; The function calling module is used for calling the vector function of the SIMD library according to the final individual to perform programming.

7. A computer device, characterized in that: The computer device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the programming method based on the hybrid SIMD library as claimed in any one of claims 1 to 5 is implemented.

Images