Halton sequence optimization method and system based on RISC-V vector extension and application

Abstract

The invention relates to a Halton sequence optimization method and system based on RISC-V vector extension and application. The Halton sequence optimization method comprises the following steps: 1) identifying mathematical characteristics generated by a Halton sequence; a Halton sequence is generated; after continuous index values are partitioned and grouped, simultaneous processing of the multiple index values is achieved through a parallelized grouping calculation method in combination with vectorization operation; 2) carrying out vectorization optimization by using an RISC-V vector instruction; in a Halton sequence generation process, carrying out parallel processing on the grouped index values by using an RISC-V vector extension instruction; 3) packaging into a dynamic link library; and 4) integration and calling with a Hadoop framework. According to the Halton sequence generation method, the number of loop iterations is remarkably reduced, and the Halton sequence generation efficiency is greatly improved. According to the method, resource waste in traditional serial computing is avoided, meanwhile, branch jump and condition judgment expenses are reduced, and the utilization rate of hardware computing resources is increased.

Description

Technical Field

[0001] The present invention relates to the field of low-discrepancy sequence generation and high-performance computing, and in particular to a Halton sequence optimization method, system and application based on RISC-V vector extension. Background Art

[0002] Halton sequence is a low-discrepancy sequence widely used in fields such as Monte Carlo simulation, financial modeling, and computer graphics. The traditional Halton sequence generation algorithm generates sequences one by one, and the computational efficiency is limited by the change of sequence index, and each index may require different computational steps. As the data scale increases, this one-by-one calculation method performs poorly in performance, especially in large-scale parallel computing environments such as Hadoop.

[0003] Hadoop is a distributed computing framework designed to process large-scale data sets. However, the traditional Halton sequence generation algorithm is difficult to fully utilize its parallel computing advantages in this environment because its generation process is essentially serial.

[0004] In the existing technology, the synergy between hardware acceleration algorithms and distributed computing frameworks is limited. For example, even if some hardware optimization technologies are introduced into distributed computing tasks, it is difficult to transparently and efficiently integrate the optimization algorithms into the distributed framework due to the lack of appropriate dynamic library encapsulation and framework interfaces (such as JNI). In addition, most existing hardware optimization methods lack adaptation to mainstream distributed systems, further reducing their practical application value.

[0005] RISC-V vector extension provides vector processing instructions, making batch operations on data efficient. By identifying the consistency of the number of loops in the calculation of the number of consecutive bases during the generation of the Halton sequence, RISC-V vector instructions can be used for vectorized processing, thereby improving the computational efficiency of sequence generation. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a method for optimizing the traditional Halton sequence generation algorithm using RISC-V vector extension instructions, and at the same time, the optimized Halton generation method is applied to the Hadoop distributed computing framework to improve the computing performance of large-scale distributed tasks (such as random number sampling, numerical simulation, Monte Carlo simulation, etc.). The present invention combines software and hardware collaborative optimization with distributed parallel computing architecture, and is widely applicable to high-performance computing, artificial intelligence, data science, engineering simulation and other fields.

[0007] The present invention optimizes the traditional Halton sequence generation algorithm by using RISC-V vector extension instructions, solving the problems of serial computing bottleneck, insufficient hardware acceleration capability and poor compatibility with distributed computing framework in the prior art of Halton sequence generation. The present invention provides a Halton sequence optimization method and system based on RISC-V vector extension, and through the seamless integration of dynamic library encapsulation and Hadoop framework, it can significantly improve the computing efficiency in distributed computing tasks.

[0008] Terminology explanation:

[0009] vle64_v_f64m4 instruction, vle64_v_f64m4 instruction, vfmul_vf_f64m4 instruction, vfadd_vv_f64m4 instruction: all are instruction functions in RISC-V Vector1.0 (RVV1.0). They are intrinsic functions of RVV1.0 and can be introduced in C / C++ programming.<riscv_vector.h> Header file and call it.

[0010] Among them, for the vle64_v_f64m4 instruction, "vle" stands for vector load element, that is, "load vector element"; "64" means that the data width to be operated is 64 bits; "v" means it is a vector operation; "f64" means double-precision floating-point type; "m4" means setting LMUL=4, that is, using 4 consecutive vector registers as a group, and a group is regarded as a logical vector register. Therefore, the meaning of this instruction is: load a 64-bit double-precision floating-point number from memory to a vector register.

[0011] For the vfmul_vf_f64m4 instruction, "vf" stands for vector-float operation, "mul" stands for multiplication, "vf" stands for vector-float scalar, "f64" stands for double-precision floating-point type, and m4 has the same meaning as above. The meaning of this instruction is: multiply each element in the vector register by a scalar floating-point number.

[0012] vfadd_vv_f64m4 instruction, "vf" stands for vector-float operation, "add" stands for addition, "vv" stands for vector-vector, that is, calculation between vectors, "f64" stands for double-precision floating-point type, and m4 has the same meaning as above. The meaning of this instruction is: add the corresponding elements in two vector registers.

[0013] Java Native Interface: JNI is part of the Java platform and is used for Java code to interact with native code written in other programming languages ​​(such as C / C++). It is a framework provided by the Java Virtual Machine (JVM) that can call native methods in Java applications and call Java methods from native code. It is a bridge for the interoperability between the Java ecosystem and native code. Specific operations: first declare a native method in a Java program, then compile the program to generate the corresponding JNI C / C++ header file, then use C / C++ to implement the corresponding function in the header file, and finally compile the C / C++ file into a dynamic library, so that Java can call the function implemented in C / C++. Reasons for using JNI: Since Java does not support the direct use of RVV features, C / C++ is used to write the relevant RVV calculation code, and the Java program can be optimized by calling it through the Java program.

[0014] The technical solution of the present invention is:

[0015] A Halton sequence optimization method based on RISC-V vector extension, comprising:

[0016] 1) Identify the mathematical properties of Halton sequence generation

[0017] Generate Halton sequence; after dividing the continuous index values ​​into blocks, use the parallel grouping calculation method combined with vectorized operation to realize the simultaneous processing of multiple index values;

[0018] 2) Use RISC-V vector instructions for vectorization optimization

[0019] During the Halton sequence generation process, RISC-V vector extension instructions are used to parallelize the grouped index values, including:

[0020] Data loading: load the grouped index values ​​into the vector register;

[0021] Vectorized calculation: parallel multiplication of index values ​​to complete the carry processing of Halton values;

[0022] Accumulation generation: accumulate the calculation results of multiple index values ​​to generate Halton sequence value;

[0023] 3) Encapsulate as a dynamic link library

[0024] Encapsulate the optimized Halton sequence generation algorithm in steps 1) and 2) as a dynamic link library;

[0025] 4) Integration and calling with Hadoop framework

[0026] The optimized Halton sequence generation algorithm is integrated into the MapReduce framework of Hadoop through the JNI interface and is used as a random number generation source or a low-discrepancy sequence generator in the Hadoop distributed computing environment.

[0027] Preferably, for a Halton sequence H with a base as base, an index value starting at index, and a sequence length of length base (i) Halton sequence H base (i) The generating formula is:

[0028]

[0029] Where: H base (i) represents the i-th value of the Halton sequence with base b; base is the selected prime base; d k (i) is the kth digit of the integer i after expanding it to the base b; i is the input index value; n is the number of digits of the integer i in the base b.

[0030] Preferably, a parallelized group calculation method is combined with a vectorized operation to realize simultaneous processing of multiple index values; including:

[0031] 1) For the index values ​​from index to index+length-1, the integers are grouped in blocks by base, with each group length being base. Different loading and parallel processing strategies are designed based on whether index and base are aligned, as follows:

[0032] When index is an integer multiple of base, the entire group is directly loaded into the vector register;

[0033] When index is not an integer multiple of base, the first r = base - (index% base) data are calculated separately, and the subsequent data are processed in parallel;

[0034] 2) For the remaining data to be processed, if the remaining data size is not less than base, the entire set of vector registers is used for parallel processing; otherwise, the remaining data is calculated separately.

[0035] Preferably, vectorization optimization is performed using RISC-V vector instructions, including:

[0036] Use the vle64_v_f64m4 instruction in the RISC-V vector extension instruction set to load the index value group into the vector register;

[0037] Use the vfmul_vf_f64m4 instruction to perform vectorized multiplication on the index value and complete the carry processing;

[0038] The calculation results of multiple index values ​​are accumulated through the vfadd_vv_f64m4 instruction to generate multiple Halton sequence values.

[0039] Preferably, the package is a dynamic link library, including:

[0040] The optimized Halton sequence generation algorithm is written in C++ language; the optimized Halton sequence generation algorithm is compiled into a dynamic link library; the dynamic library is encapsulated through the Java Native Interface interface so that it can be called by a Java program.

[0041] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a Halton sequence optimization method based on RISC-V vector extension when executing the computer program.

[0042] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of a Halton sequence optimization method based on RISC-V vector extension are implemented.

[0043] A Halton sequence optimization system based on RISC-V vector extension, comprising:

[0044] Input module: provides the parameters required for Halton sequence generation, including: starting index index, generation length length and Halton sequence base base;

[0045] Halton optimization algorithm module: realizes efficient parallel generation of Halton sequences through hardware acceleration technology based on RISC-V vector instructions;

[0046] Dynamic library encapsulation module: encapsulates the optimized Halton sequence generation algorithm into a dynamic library and provides a standard interface for calling; JNI interface module: exposes the dynamic library function to the distributed computing framework through Java Native Interface technology;

[0047] Hadoop MapReduce framework: In distributed computing, the optimized Halton sequence generation algorithm is used for task decomposition and result integration;

[0048] Result output module: outputs low difference point sequence or Monte Carlo simulation results according to the calculation task.

[0049] The application of the above-mentioned Halton sequence optimization method based on RISC-V vector extension includes the application of the above-mentioned Halton sequence optimization method based on RISC-V vector extension in distributed computing tasks such as Monte Carlo simulation, numerical integration, and random sampling.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The generation efficiency is significantly improved. The Halton sequence generation algorithm is optimized through RISC-V vector extension instructions, which enables the simultaneous processing of multiple index values, significantly reduces the number of loop iterations, and greatly improves the efficiency of Halton sequence generation.

[0052] 2. Improved hardware resource utilization. The present invention makes full use of the hardware parallel computing capability of the RISC-V vector extension instruction set, avoids the waste of resources in traditional serial computing, reduces branch jumps and conditional judgment overhead, and improves the utilization of hardware computing resources.

[0053] 3. Adapt to the distributed computing framework. The optimized Halton sequence generation algorithm is integrated with the Hadoop framework through dynamic library encapsulation, and can be seamlessly applied to distributed computing tasks, significantly improving the computing efficiency in a distributed environment while reducing the communication overhead between task nodes.

[0054] 4. Wide range of practical application scenarios. The present invention is applicable to a variety of distributed computing tasks such as Monte Carlo simulation, numerical integration, random sampling, etc., and especially shows significant performance advantages in scenarios where large-scale low-difference point sequences need to be generated, thus broadening the application scope of Halton sequences in the field of high-performance computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a structural block diagram of the Halton sequence optimization system based on RISC-V vector extension of the present invention;

[0056] Figure 2 This is a schematic diagram of the Halton vectorization optimization algorithm flow of the present invention;

[0057] Figure 3 It is a schematic diagram of the performance improvement effect of the present invention; DETAILED DESCRIPTION

[0058] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.

[0059] Example 1

[0060] In order to better realize the technical solution of the present invention, the Halton sequence optimization method based on RISC-V vector extension and its application in a distributed computing framework will be specifically described below. The present invention designs an algorithm for efficient and low-difference point generation by combining the mathematical characteristics of the Halton sequence and the RISC-V vector extension technology, and implements dynamic library encapsulation for easy application in a distributed computing framework. Specific implementation methods of the present invention include: vectorized optimization design of the Halton sequence, data processing strategies in vector registers, dynamic library encapsulation, integration of a distributed computing framework, and expansion of actual distributed application tasks. 1. Vectorized optimization design of the Halton sequence

[0061] The present invention aims to solve the serial performance bottleneck of the traditional Halton sequence point-by-point generation method, and designs an efficient parallel computing solution by combining RISC-V vector extension technology.

[0062] A Halton sequence optimization method based on RISC-V vector extension, comprising:

[0063] 1) Identify the mathematical properties of Halton sequence generation

[0064] The mathematical characteristics of Halton sequence generation are analyzed, and it is found that the index values ​​have the same number of carry times in a continuous base range during the generation process. This characteristic is used to convert the traditional point-by-point serial algorithm into a parallelizable group calculation method; the Halton sequence is generated; after the continuous index values ​​are divided into blocks and grouped, the parallelized group calculation method is used in combination with vectorized operations to realize the simultaneous processing of multiple index values;

[0065] 2) Use RISC-V vector instructions for vectorization optimization

[0066] During the Halton sequence generation process, RISC-V vector extension instructions are used to parallelize the grouped index values, including:

[0067] Data loading: through vle <sew>_v_f <vsew> m <lmul>The instruction loads the grouped index values ​​into the vector register; ensuring that multiple index values ​​can be processed simultaneously;

[0068] Vectorized calculation: using vfmul_vf_f <vsew> m <lmul>The instruction performs parallel multiplication on the index value to complete the carry processing of the Halton value;

[0069] Accumulated generation: through vfadd_vv_f <vsew> m <lmul>The instruction accumulates the calculation results of multiple index values ​​to generate a Halton sequence value;

[0070] Through the above vectorized operations, the number of loop iterations in the traditional algorithm is significantly reduced, and the efficiency of Halton sequence generation is improved.

[0071] 3) Encapsulate as a dynamic link library

[0072] Steps 1) and 2), i.e. the optimized Halton sequence generation algorithm, are encapsulated as a dynamic link library (.so file) so that it can be integrated into the Hadoop distributed computing framework through the Java Native Interface (JNI) interface. The encapsulation of the dynamic library enables this method to be adapted to other distributed computing frameworks, improving its versatility and ease of use.

[0073] Steps 1) and 2) outline the idea of ​​optimizing the Halton sequence generation algorithm. The "optimized Halton sequence generation algorithm" here is an optimization algorithm that implements the specific operations based on the ideas of steps 1) and 2). Specifically, it can be described as follows: according to the identified mathematical characteristics, the data is first indexed and grouped, and then vectorized operations such as "data loading", "vectorized calculation", and "accumulation generation" are used on the grouped data. According to the characteristic of RVV that the vector operation length (VL) can be dynamically set, the data is processed in parallel, so that the serial algorithm has vectorized optimization.

[0074] 4) Integration and calling with Hadoop framework

[0075] The optimized Halton sequence generation algorithm is integrated into Hadoop's MapReduce framework through the JNI interface and used as a random number generation source or low-discrepancy sequence generator in the Hadoop distributed computing environment. By calling the optimized algorithm in MapReduce tasks, the efficiency of Halton sequence generation in distributed computing tasks is significantly improved, and the waiting time and communication overhead between task nodes are reduced.

[0076] Example 2

[0077] The Halton sequence optimization method based on RISC-V vector extension described in Example 1 is different in that:

[0078] Based on the mathematical model generated by Halton sequence, for the Halton sequence H with base as base, index value starting as index, and sequence length as length base (i) Halton sequence H base (i) The generating formula is:

[0079]

[0080] Where: H base (i) represents the i-th value of the Halton sequence with base b; base is a selected prime number base (such as 2, 3, 5, etc.); d k (i) is the kth digit of the integer i after expanding it to the base b; i is the input index value; n is the number of digits of the integer i in the base b.

[0081] During the Halton sequence generation process, continuous index values ​​are divided into blocks according to computing power, and each block calculates multiple index values ​​at the same time. The instruction parallelism and vector register capacity supported by RISC-V vector extension are used to eliminate the dependency of point-by-point calculation and achieve parallel optimization.

[0082] For the calculation of index values ​​in Halton sequence, the present invention designs a method for block processing in combination with vector registers, and realizes simultaneous processing of multiple index values ​​by a parallelized group calculation method combined with vectorized operation; including:

[0083] 1) For the index values ​​from index to index+length-1, the integers are grouped in blocks by base, with each group length being base. Different loading and parallel processing strategies are designed based on whether index and base are aligned, as follows:

[0084] When index is an integer multiple of base, the batch of data is aligned and can be directly loaded into the vector register as a whole group without separate processing.

[0085] When index is not an integer multiple of base, the first r = base - (index% base) data are calculated separately (called padding processing), and the subsequent data are processed in parallel;

[0086] 2) For the remaining data to be processed, if the remaining data size is not less than base, the entire set of vector registers is used for parallel processing; otherwise, the remaining data is calculated separately.

[0087] Based on the above logic design, such as Figure 2 As shown, the vectorized calculation process of the present invention is as follows:

[0088] a. Input parameters, including: sequence start index index, total generated length length, and prime base of Halton sequence base (supports multiple dimensions);

[0089] b. Initialize the current processing position and the remaining data length. The remaining length is length;

[0090] c. Group data according to the alignment of index and base:

[0091] If index divides base, there is no need to fill it up and it will directly enter normal grouping;

[0092] Otherwise, the data with the current starting segment length of r=base-(index%base) is calculated separately, and the index and the remaining length are updated;

[0093] d. For each remaining piece of data:

[0094] If the current remaining length is greater than or equal to base, a set of index values ​​with a length of base is directly loaded into the vector register and processed in parallel using RISC-V vector instructions;

[0095] If the remaining length is less than base, scalar calculation is performed on this part of the data separately.

[0096] Through the design of the above process, the present invention effectively utilizes the vectorization characteristics of RISC-V to realize the parallel operation of the entire set of data, and at the same time solves the boundary problem caused by the starting offset.

[0097] Use RISC-V vector instructions for vectorization optimization, including:

[0098] Use the vle64_v_f64m4 instruction in the RISC-V vector extension instruction set to load the index value group into the vector register;

[0099] Use the vfmul_vf_f64m4 instruction to perform vectorized multiplication on the index value and complete the carry processing;

[0100] The calculation results of multiple index values ​​are accumulated through the vfadd_vv_f64m4 instruction to generate multiple Halton sequence values.

[0101] To implement the above parallel optimization scheme, the present invention designs an efficient hardware affinity algorithm through the support of RISC-V vector extension instructions. Specifically, the following steps are included:

[0102] 2.1 Data loading and initialization

[0103] According to the vector register capacity, determine the index length of each operation:

[0104] 1) Configure the length of the vector register to support multi-index parallel computing;

[0105] 2) The index segments are grouped by base and loaded into registers in batches, with a fixed set of consecutive index values ​​(including non-aligned data) being operated each time.

[0106] 2.2 Parallel Computing

[0107] In view of the weighted hierarchical accumulation characteristics of the Halton sequence, batch calculations are performed through RISC-V vector registers to complete the following tasks:

[0108] Decompose each group of indexes into high-order weights and low-order weights expanded in base digits;

[0109] The multiplication and accumulation of each bit weight is completed synchronously through vector operations (solving the problem of bit-by-bit coefficient calculation);

[0110] Vector operations are fully parallelized, reducing the cycle time of bit-by-bit processing.

[0111] 2.3 Boundary Processing

[0112] After the batch parallelization is completed, scalar calculation is performed on the remaining index segments that are less than one group (less than base). Since single-point calculation and vector calculation share the same formula and processing flow, there is no need to design complex logic separately.

[0113] Packaged as a dynamic link library, including:

[0114] The optimized Halton sequence generation algorithm is written in C++ language; the optimized Halton sequence generation algorithm is compiled into a dynamic link library; the dynamic library is encapsulated through the Java Native Interface (JNI) interface so that it can be called by a Java program.

[0115] In order to realize cross-platform and cross-language calling of the optimized Halton sequence generation algorithm and facilitate its integration in a distributed computing framework, the present invention encapsulates the algorithm into a dynamic link library (such as a .so file) and implements the external interface through Java Native Interface (JNI) to ensure efficient calling and flexible expansion of the algorithm functions.

[0116] The dynamic link library packaging process of the present invention comprises the following steps:

[0117] 1) Encapsulation of core algorithms

[0118] The Halton sequence generation algorithm is developed using C++ language, and the calculation logic optimized by RISC-V vector instructions is encapsulated as a general function interface. The core function interface is as follows:

[0119] void generate_halton_sequence(int start_idx,int length,int base[],double result[]);

[0120] Parameter Description:

[0121] start_idx: the starting index value of the generated sequence;

[0122] length: the length of the generated sequence;

[0123] base[]: base array, used to support multidimensional Halton sequence generation;

[0124] result[]: Output array used to store the generated Halton sequence results.

[0125] This interface supports the generation of multi-dimensional Halton sequences and fully utilizes the parallel computing capabilities of RISC-V vector instructions to ensure high efficiency and accuracy.

[0126] 2) Compilation and generation of dynamic libraries

[0127] Compile the C++ code into a dynamic link library (.so file for Linux platform and .dll file for Windows platform) to achieve cross-platform dynamic loading and calling. For example, the dynamic library file generated in Linux is named libhalton.so.

[0128] During the compilation process, ensure that the dynamic library is compatible with mainstream hardware architectures (such as RISC-V and x86) and operating systems (such as Linux, Windows, and macOS), and optimize the hardware instruction set characteristics of different platforms.

[0129] 3) Testing and verification of dynamic libraries

[0130] After packaging is completed, the functionality, performance, and stability of the dynamic library are fully tested to ensure that it can correctly generate Halton sequences and maintain high efficiency and reliability during the calling process.

[0131] To support the integration of distributed computing frameworks (such as Hadoop, Spark, etc.), the present invention re-encapsulates the dynamic link library through Java Native Interface (JNI) to provide a calling interface for Java applications. The specific design is as follows:

[0132] 1) Java interface definition

[0133] Provides a Java native method interface, allowing Java programs to call core functions in dynamic libraries. The method signature is as follows:

[0134] public native double[]generateHaltonSequence(int startIdx,int length,int[]base);

[0135] Parameter Description:

[0136] startIdx: the starting index value of the generated sequence;

[0137] length: the length of the generated sequence;

[0138] base: base array, supports multi-dimensional Halton sequence generation;

[0139] The return value is the generated Halton sequence result array.

[0140] 2) Dynamic library loading and initialization

[0141] In a Java program, load a dynamic library through System.loadLibrary("halton"). This method automatically loads the corresponding dynamic library file according to the hardware architecture.

[0142] After loading is complete, the core algorithm in the dynamic library can be directly called through the Java interface to complete the generation of the Halton sequence.

[0143] 3) Cross-platform compatibility

[0144] Through JNI encapsulation, the dynamic library supports running in a variety of distributed frameworks and platforms, adapting to mainstream operating systems and hardware architectures, ensuring the versatility and scalability of the algorithm.

[0145] The present invention integrates the optimized Halton sequence generation algorithm into a distributed computing framework (such as Hadoop MapReduce) to support large-scale parallel generation and distributed computing tasks with low variance points. Through the combination of dynamic libraries and JNI interfaces, the present invention can efficiently generate Halton sequences in a distributed environment and significantly improve the performance of computing tasks. The integration and application in the distributed computing framework include:

[0146] 4.1Map phase call

[0147] 1) Task allocation and Halton sequence generation

[0148] In the Map phase of the MapReduce task, each Map task will independently call the Halton sequence generation algorithm in the dynamic library to generate the required low-difference point sequence.

[0149] Through the JNI interface, the Map task can directly call the generate_halton_sequence method of the dynamic library to generate a Halton sequence of a specified length.

[0150] The Halton point sequence generated by each Map task can be used as input data for subsequent calculations, such as sample points in Monte Carlo simulation, sampling points in numerical integration, etc.

[0151] 2) Advantages of parallelization

[0152] Since the Halton sequence generation algorithm has been vectorized and optimized and can fully utilize the parallel computing capabilities of the underlying hardware, the efficiency of the Map task is significantly improved when generating large-scale low-difference points.

[0153] Each Map task generates a point sequence independently, avoiding communication overhead between nodes and further improving the parallel efficiency of distributed tasks.

[0154] 4.2 Reduce Phase Aggregation of Results

[0155] In the Reduce phase, the Halton sequence generated by the Map task will be submitted to the Reduce task for integration of global results.

[0156] Experimental effects and applicable scenarios

[0157] 5.1 Experimental environment and test plan

[0158] 1) Experimental environment

[0159] Test platform: RISC-V hardware platform, combined with Hadoop MapReduce distributed computing framework.

[0160] Experimental scenario: π value calculation task in Monte Carlo simulation. This is a classic example program for Hadoop MapReduce calculation.

[0161] 2) Test plan description

[0162] Test task: In Monte Carlo simulation, random sampling is used to generate a point sequence, which is then optimized using a low-discrepancy Halton sequence to reduce random errors and improve task computational efficiency.

[0163] Comparison method: The unoptimized Halton sequence generation algorithm (original algorithm, hereinafter referred to as T original ) and the RISC-V vectorization optimization algorithm (optimization algorithm, hereinafter referred to as T) proposed in the present invention optimized ) for comparison.

[0164] Evaluation indicators: based on the consumed task computing time and computing performance improvement indicators

[0165] in:

[0166] T original : The running time required to complete the task using the original unoptimized algorithm;

[0167] T optimized : The running time required to complete the task using the method of the present invention;

[0168] η: Performance improvement percentage, used to quantify the optimization effect.

[0169] 5.2 Experimental Results

[0170] 1) Quantitative data on performance improvement

[0171] The present invention is applied in Monte Carlo simulation tasks, and experimental tests are carried out on the MapReduce distributed program for π value calculation under the same hardware platform and framework settings.

[0172] The experimental results are as follows Figure 3 As shown, the performance of the Halton sequence generation method optimized by the present invention has been significantly improved, with the performance improvement index reaching 89.32%. That is:

[0173]

[0174] The optimized algorithm greatly reduces the computation time in the Halton sequence generation process and improves the overall efficiency of MapReduce tasks, especially in the general Monte Carlo simulation model.

[0175] 2) Comparative analysis

[0176] Insufficient original method: The traditional point-by-point generation of Halton sequences cannot fully utilize the parallel performance of hardware due to serious cyclic dependencies, causing the generation process to become a bottleneck for distributed simulation computing.

[0177] In the present invention, vectorized operation: fully utilizing the vector registers provided by RISC-V hardware, greatly reducing the time complexity of point sequence generation; block and alignment optimization: by performing block processing on index values ​​and alignment processing on vector registers, the concurrency of data loading and computing operations is improved; dynamic library integration: the optimized Halton generation algorithm is easily integrated into the MapReduce framework through dynamic library encapsulation, achieving a smoother task flow.

[0178] 5.3 Application Scenario Analysis

[0179] 1) Applicable scenarios

[0180] The optimized Halton sequence generation method of the present invention has wide applicability and can be applied to a variety of practical scenarios in large-scale distributed computing tasks, such as:

[0181] Monte Carlo simulation: Generate low-difference points through Halton sequences to efficiently complete complex system simulation tasks, such as option pricing in finance and system behavior simulation in physical sciences;

[0182] Numerical integration and sampling: In high-dimensional integral problems, the accuracy of integral estimation can be improved by reducing computational errors through low-difference points;

[0183] Machine learning and model training: The low-difference points generated by the Halton sequence can be used for tasks such as data augmentation and hyperparameter search to improve training results;

[0184] Other distributed random computing: such as point sample generation in statistical simulation and data visualization.

[0185] 2) Scalability analysis

[0186] For different tasks, the cardinality array and point sequence dimensions of the Halton sequence can be flexibly adjusted, so that the method of the present invention can adapt to a variety of distributed computing requirements;

[0187] The underlying algorithm optimization (such as vectorization implementation, dynamic library encapsulation, etc.) that the present invention relies on has strong versatility and can be integrated in mainstream distributed frameworks (such as Hadoop, Spark, Flink, etc.) to adapt to more practical scenarios.

[0188] 5.4 Summary of optimization effects

[0189] Experimental results show that the present invention has a significant performance optimization effect on classic distributed computing tasks (such as Monte Carlo simulation π value calculation), with performance improvement of up to 89.32%. In addition, through the introduction of RISC-V vectorization technology, the present invention can effectively overcome the performance bottleneck of the original point-by-point generation algorithm and show high efficiency, adaptability and ease of use in large-scale distributed computing tasks. Its optimization method and technical implementation have important practical significance for the performance improvement of distributed computing tasks.

[0190] 6. Error handling and optimization

[0191] In order to ensure the stability and robustness of the present invention in various operating environments, the present invention designs a comprehensive error handling mechanism and combines it with a performance optimization strategy to achieve efficient and reliable Halton sequence generation.

[0192] 6.1 Parameter Verification Mechanism

[0193] The parameters are fully checked at the input stage to ensure that the input index value is a non-negative integer, the sequence length is within a reasonable range, and all values ​​in the base array are valid prime numbers. If the parameters do not meet the requirements, an error message is immediately returned and the calculation is terminated to avoid error propagation.

[0194] A maximum limit is set for the length of sequence generation to prevent excessively large computational tasks from causing insufficient memory or performance degradation.

[0195] 6.2 Memory Management and Resource Release

[0196] During the Halton sequence generation process, the dynamically allocated memory space will be released immediately after use to avoid resource leakage.

[0197] For the allocation and use of vector registers, ensure that they are cleaned up after the calculation is completed to release the hardware resources.

[0198] When an exception or error interrupt occurs, a resource recovery mechanism is designed to ensure that all allocated resources can be released safely.

[0199] 6.3 Runtime Error Detection and Handling

[0200] When executing vectorized instructions, the availability of the hardware vector unit is detected in real time. If the vector unit is unavailable or an execution error occurs, the present invention will automatically downgrade to a scalar calculation mode to ensure that the task can continue to be completed.

[0201] During the task running process, set up an error logging mechanism to record the time, cause, and context information of the error in detail for subsequent analysis and optimization.

[0202] 6.4 Performance Optimization Measures

[0203] When processing large-scale data, a block computing strategy is adopted to dynamically adjust the size of data blocks according to the hardware cache size, reduce memory access latency, and improve cache hit rate.

[0204] According to the characteristics of RISC-V vector instructions, the execution order of the instruction pipeline is optimized to reduce the dependency between instructions and improve the utilization of hardware computing units.

[0205] For computing tasks with high repetitiveness, a result caching mechanism is used to avoid repeated calculations and further improve performance.

[0206] 7. Security design

[0207] During the design and implementation process, the present invention fully considers data security, concurrency security and the reliability of the operating environment to ensure that the algorithm can run safely in various scenarios.

[0208] 7.1 Data Security

[0209] During data processing, a strict boundary checking mechanism is designed to ensure that no out-of-bounds access occurs when accessing arrays or buffers, thus preventing potential security vulnerabilities.

[0210] For input and output data, a verification mechanism is designed to ensure the integrity and correctness of the data. For example, after generating the Halton sequence, the result can be randomly sampled and verified to ensure that the generated point sequence meets the expected mathematical properties.

[0211] 7.2 Concurrency Safety

[0212] In a multi-threaded or distributed environment, a thread-safe resource access control mechanism is used to ensure that there is no conflict when multiple threads or nodes access shared resources at the same time. For example, when reading or writing shared data, a locking mechanism or atomic operation is used to ensure data consistency.

[0213] For the management of global state, a synchronization mechanism for initialization and destruction is designed to ensure that there will be no competition issues in resource initialization or release in a multi-threaded environment.

[0214] 7.3 Reliability of the operating environment

[0215] In a distributed computing environment, nodes may be interrupted due to hardware failure or network problems. The present invention designs a fault recovery mechanism to allow tasks to be restarted after interruption and continue execution from the interruption point to avoid repeated calculations.

[0216] For possible hardware anomalies (such as vector unit failure), a degradation strategy is designed to automatically switch to compatible mode to ensure task continuity.

[0217] 8. Compatibility design

[0218] In order to ensure that the present invention can run normally in different hardware platforms and distributed computing frameworks, multi-level compatibility support is designed.

[0219] 8.1 Hardware Compatibility

[0220] The present invention supports different versions of RISC-V architecture, including processors with vector extension instruction sets and general-purpose processors without vector extensions. When it is detected that the hardware does not support vector extensions, the algorithm automatically switches to scalar calculation mode to ensure compatibility on low-end hardware.

[0221] In order to adapt to different hardware characteristics, the present invention designs a hardware detection mechanism to automatically detect the instruction set characteristics supported by the processor at runtime and select the optimal execution path according to the hardware capabilities. For example, on hardware that supports vector extensions, vectorized algorithms are used first, while on hardware that does not support it, scalar algorithms are used.

[0222] 8.2 Backward Compatibility

[0223] The present invention takes full account of backward compatibility during the design process to ensure that it can still operate normally in future hardware or software upgrades. For example, in a new version of the RISC-V architecture, if the instruction set changes, the present invention can quickly adapt to the new architecture by updating the hardware detection module and the instruction mapping table.

[0224] For version upgrades of the distributed framework, the present invention provides an extensible adaptation interface, allowing users to quickly adapt to the new version of the framework without modifying the core algorithm.

[0225] Example 3

[0226] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the Halton sequence optimization method based on RISC-V vector extension described in Example 1 or 2 are implemented.

[0227] Example 4

[0228] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the Halton sequence optimization method based on RISC-V vector extension described in Example 1 or 2.

[0229] Example 5

[0230] A Halton sequence optimization system based on RISC-V vector extension, such as Figure 1 As shown, including:

[0231] Input module: provides the parameters required for Halton sequence generation, including: starting index index, generation length length and Halton sequence base base;

[0232] Halton optimization algorithm module: This is the core module of the present invention, which realizes efficient parallel generation of Halton sequences through hardware acceleration technology based on RISC-V vector instructions;

[0233] Dynamic library encapsulation module: encapsulates the optimized Halton sequence generation algorithm into a dynamic library and provides a standard interface for calling; JNI interface module: exposes the dynamic library function to the distributed computing framework (such as Hadoop) through Java Native Interface technology;

[0234] Hadoop MapReduce framework: In distributed computing, the optimized Halton sequence generation algorithm is used for task decomposition (Map phase) and result integration (Reduce phase);

[0235] Result output module: outputs low-difference point sequences or Monte Carlo simulation results according to the calculation task, such as the estimation of π value.

[0236] Through the collaborative work of the above modules, the efficiency bottleneck problem of Halton sequence generation in distributed computing scenarios is solved.

[0237] The application of the above-mentioned Halton sequence optimization method based on RISC-V vector extension includes the application of the above-mentioned Halton sequence optimization method based on RISC-V vector extension in distributed computing tasks such as Monte Carlo simulation, numerical integration (such as high-dimensional integral calculation), and random sampling (such as data enhancement tasks in machine learning). In Monte Carlo simulation (such as π value calculation, financial risk assessment, etc.), the performance improvement effect of the present invention is verified by calling multi-node MapReduce tasks, which significantly shortens the task completion time and improves the overall performance of the distributed computing environment.

[0238] The application of the above-mentioned Halton sequence optimization method based on RISC-V vector extension in the Hadoop framework includes: calling the optimized Halton sequence generation algorithm in Hadoop's MapReduce task; generating a large-scale low-difference point sequence in the Map stage as input for Monte Carlo simulation or numerical integration; and aggregating the calculation results in the Reduce stage to improve the overall performance of distributed computing tasks.< / lmul> < / vsew> < / lmul> < / vsew> < / lmul> < / vsew> < / sew>

Claims

1. A Halton sequence optimization method based on RISC-V vector extension, characterized in that: include: 1) Identify the mathematical properties of Halton sequence generation Generate Halton sequence; After the continuous index values ​​are divided into blocks and grouped, the parallel grouping calculation method is combined with vectorized operations to achieve simultaneous processing of multiple index values; 2) Use RISC-V vector instructions for vectorization optimization During the Halton sequence generation process, RISC-V vector extension instructions are used to parallelize the grouped index values, including: Data loading: load the grouped index values ​​into the vector register; Vectorized calculation: parallel multiplication of index values ​​to complete the carry processing of Halton values; Accumulation generation: accumulate the calculation results of multiple index values ​​to generate Halton sequence value; 3) Encapsulate as a dynamic link library Encapsulate the optimized Halton sequence generation algorithm in steps 1) and 2) into a dynamic link library; 4) Integration and calling with Hadoop framework The optimized Halton sequence generation algorithm is integrated into the MapReduce framework of Hadoop through the JNI interface and is used as a random number generation source or a low-discrepancy sequence generator in the Hadoop distributed computing environment.

2. The Halton sequence optimization method based on RISC-V vector extension according to claim 1, characterized in that: For the Halton sequence H with base as base, index value starting at index, and sequence length as length base (i) Halton sequence H base (i) The generating formula is: Where: H base (i) represents the i-th value of the Halton sequence with base b; base is the selected prime base; d k (i) is the kth digit of the integer i after expanding it to the base b; i is the input index value; n is the number of digits of the integer i in the base b.

3. The Halton sequence optimization method based on RISC-V vector extension according to claim 1, characterized in that: Through the parallel grouping calculation method, combined with vectorized operations, multiple index values ​​can be processed simultaneously; including: 1) For the index values ​​from index to index+length-1, the integers are grouped in blocks by base, with each group length being base. Different loading and parallel processing strategies are designed based on whether index and base are aligned, as follows: When index is an integer multiple of base, the entire group is directly loaded into the vector register; When index is not an integer multiple of base, the first r = base - (index% base) data are calculated separately, and the subsequent data are processed in parallel; 2) For the remaining data to be processed, if the remaining data size is not less than base, the entire set of vector registers is used for parallel processing; otherwise, the remaining data is calculated separately.

4. The Halton sequence optimization method based on RISC-V vector extension according to claim 1, characterized in that: Use RISC-V vector instructions for vectorization optimization, including: Use the vle64_v_f64m4 instruction in the RISC-V vector extension instruction set to load the index value group into the vector register; Use the vfmul_vf_f64m4 instruction to perform vectorized multiplication on the index value and complete the carry processing; The calculation results of multiple index values ​​are accumulated through the vfadd_vv_f64m4 instruction to generate multiple Halton sequence values.

5. The Halton sequence optimization method based on RISC-V vector extension according to claim 1, characterized in that: Packaged as a dynamic link library, including: The optimized Halton sequence generation algorithm is written in C++ language; the optimized Halton sequence generation algorithm is compiled into a dynamic link library; the dynamic library is encapsulated through the Java Native Interface interface so that it can be called by a Java program.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the Halton sequence optimization method based on RISC-V vector extension described in any one of claims 1-5 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the Halton sequence optimization method based on RISC-V vector extension described in any one of claims 1 to 5 are implemented.

8. A Halton sequence optimization system based on RISC-V vector extension, characterized in that: include: Input module: provides the parameters required for Halton sequence generation, including: starting index index, generation length length and Halton sequence base base; Halton optimization algorithm module: realizes efficient parallel generation of Halton sequences through hardware acceleration technology based on RISC-V vector instructions; Dynamic library encapsulation module: encapsulates the optimized Halton sequence generation algorithm into a dynamic library and provides a standard interface for calling; JNI interface module: exposes dynamic library functions to the distributed computing framework through Java Native Interface technology; Hadoop MapReduce framework: In distributed computing, the optimized Halton sequence generation algorithm is used for task decomposition and result integration; Result output module: outputs low difference point sequence or Monte Carlo simulation results according to the calculation task.

9. The application of the Halton sequence optimization method based on RISC-V vector extension according to any one of claims 1 to 5, characterized in that: Including the application of Halton sequence optimization method based on RISC-V vector extension in distributed computing tasks such as Monte Carlo simulation, numerical integration, and random sampling.

Images