Storage and calculation integrated controller-based large model application acceleration method and system

Abstract

The invention discloses a large model application acceleration method and system based on a storage and calculation integrated controller, and the method comprises the steps: integrating a calculation module and a storage module based on the storage and calculation integrated controller, and enabling the storage and calculation integrated controller to cooperate with a GPU module for data processing; a private document is preprocessed and segmented through a calculation module and then stored in a private vector knowledge base; data circulation is directly carried out between the GPU module and the storage module, and distributed storage and sample preprocessing are carried out on sample data; performing model reasoning by utilizing a calculation module according to a received query instruction to obtain vector features of a knowledge query problem; using a KV database based on hardware characteristics to carry out storage and comparison query on the vector characteristics; and the calculation module directly transmits a query result to the GPU module for model reasoning. According to the technical scheme, the large model training and reasoning speed is increased, and the architecture process, performance improvement and resource cost reduction of large model application are greatly optimized and improved.

Description

Technical Field

[0001] The present invention relates to the field of large model technology, and in particular to a large model application acceleration method based on a storage-computing integrated controller and a large model application acceleration system based on a storage-computing integrated controller. Background Art

[0002] At present, in commercial applications, large models generally rely on local / private knowledge bases due to factors such as accuracy, local deployment, and data privacy. The application of large models based on private knowledge bases will make the final question-answering effect better, more accurate, and more secure.

[0003] In general large language model applications, the entire knowledge base query consists of two parts: the Embedding model and the vector knowledge base. During initialization, private documents are preprocessed and segmented, and then the Embedding model is used to extract vector features for each segmented piece of knowledge information, and then the vector features are stored in the vector knowledge base. In the questioning phase, the question is also embedded to obtain vector features, and then the similarity is calculated with the features in the vector library to find the best knowledge information. Finally, the knowledge information found is sent to the LLM large model to obtain more accurate results.

[0004] However, traditional knowledge base queries rely on CPU or GPU and memory or video memory. If the private knowledge base is too large, it will pose a greater challenge to hardware resources. Figure 1 As shown, in the traditional architecture, the overall data flow is to first search the knowledge base and then import the information back to the CPU, and then import the information into the GPU for LLM model reasoning. Summary of the invention

[0005] In response to the above problems, the present invention provides a large model application acceleration method and system based on a storage-computing integrated controller. By integrating the computing module and the storage module in the storage-computing integrated controller, computing and storage are combined to improve the speed of large model training and reasoning. The ability of direct data communication between the storage-computing integrated controller and the graphics card GPU module is used to improve the data memory access speed, realize the end-to-end private knowledge base query complete process, and greatly optimize and improve the architectural process, performance improvement and resource cost reduction of large model applications.

[0006] To achieve the above object, the present invention provides a large model application acceleration method based on a storage-computing integrated controller, comprising:

[0007] Based on the integrated computing and storage controller, the computing module and the storage module are integrated, the integrated computing and storage controller is connected to the CPU module and the GPU module respectively, and the integrated computing and storage controller cooperates with the GPU module to perform data processing;

[0008] In the initialization phase, after the private document is preprocessed and segmented by the computing module, vector features are extracted using the Embedding model, and the vector features are stored in the private vector knowledge base of the storage module;

[0009] Data is directly transferred between the GPU module and the storage module, and distributed storage and sample preprocessing are performed on sample data in the large model training process;

[0010] According to the private knowledge base query instruction received from the CPU module, the computing module is used to perform Embedding model reasoning to obtain vector features of the knowledge query question;

[0011] Using a KV database based on hardware characteristics, the vector features are stored and compared in the private knowledge database;

[0012] The computing module directly transfers the query results or search results to the GPU module for model reasoning.

[0013] In the above technical solution, preferably, the GPU module includes a computing unit and a video memory, the computing unit is used to perform model reasoning, and data is directly transferred between the video memory and the storage-computing integrated controller.

[0014] The present invention also proposes a large model application acceleration system based on a storage-computing integrated controller, which applies the large model application acceleration method based on a storage-computing integrated controller disclosed in the above technical solution, including:

[0015] A model integration connection module, used for integrating a computing module and a storage module based on a storage-computing integrated controller, wherein the storage-computing integrated controller is connected to a CPU module and a GPU module respectively, and the storage-computing integrated controller cooperates with the GPU module to perform data processing;

[0016] The document vector storage module is used to, in the initialization phase, pre-process and segment the private document through the calculation module, extract vector features using the Embedding model, and store the vector features in the private vector knowledge base of the storage module;

[0017] A data direct flow module, used to directly transfer data between the GPU module and the storage module, and to perform distributed storage and sample preprocessing on sample data in the large model training process;

[0018] A knowledge vector reasoning module, configured to perform Embedding model reasoning using the computing module according to the private knowledge base query instruction received from the CPU module, and obtain vector features of the knowledge query question;

[0019] A knowledge storage query module, used for storing and comparing the vector features in the private knowledge database using a KV database based on hardware characteristics;

[0020] The graphics card model inference module is used to directly transfer the query results or search results from the calculation module to the GPU module for model inference.

[0021] In the above technical solution, preferably, the GPU module includes a computing unit and a video memory, the computing unit is used to perform model reasoning, and data is directly transferred between the video memory and the storage-computing integrated controller.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: by integrating the computing module and the storage module in the storage-computing integrated controller, the computing and storage are combined, the speed of large model training and reasoning is improved, and the ability of direct data communication between the storage-computing integrated controller and the graphics card GPU module is utilized to improve the data access speed, realize the end-to-end private knowledge base query complete process, and greatly optimize and improve the architectural process, performance improvement and resource cost reduction of large model applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of the data flow architecture of a large model system disclosed in an embodiment of the prior art;

[0024] Figure 2 A schematic diagram of the data flow architecture of a large model system disclosed in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of a private knowledge base query process based on a large model of a storage-computing integrated controller disclosed in an embodiment of the present invention;

[0026] Figure 4 A logical framework diagram of a large model application acceleration method based on a storage-computing integrated controller disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0029] like Figure 2and Figure 3 As shown, a large model application acceleration method based on a storage-computing integrated controller provided by the present invention includes:

[0030] The computing module and storage module are integrated based on the storage-computing integrated controller, which is connected to the CPU module and the GPU module respectively, and the storage-computing integrated controller and the GPU module cooperate to process data;

[0031] In the initialization phase, the private documents are preprocessed and segmented by the computing module, and the vector features are extracted using the Embedding model. The vector features are then stored in the private vector knowledge base of the storage module.

[0032] Data is directly transferred between the GPU module and the storage module, and sample data in the large model training process is distributedly stored and preprocessed.

[0033] According to the private knowledge base query instruction received from the CPU module, the computing module is used to perform Embedding model reasoning to obtain the vector features of the knowledge query problem;

[0034] Using the KV database based on hardware characteristics, vector features are stored and compared in the private knowledge database;

[0035] The computing module directly transfers the query results or search results to the GPU module for model inference.

[0036] like Figure 4 As shown, in this implementation, by integrating the computing module and the storage module in the storage-computing integrated controller, computing and storage are combined to improve the speed of large model training and reasoning, and by utilizing the ability of direct data communication between the storage-computing integrated controller and the graphics card GPU module, the data access speed is improved, and the end-to-end private knowledge base query complete process is realized, which greatly optimizes and improves the architectural process, performance improvement and resource cost reduction of large model applications.

[0037] Specifically, compared with the data flow method in the traditional architecture, the present invention is based on the EB1212 storage and computing integrated controller. Since the Linux system has been integrated on the storage and computing integrated chip, it can be used as the main control, directly access the graphics card through the PCIE bridge, and directly transmit the knowledge base search results to the graphics card for LLM large model reasoning.

[0038] Due to the huge number of parameters in the large model itself, the storage and writing of its parameters and the storage of training samples all rely on large-capacity SSDs for distributed storage and computing. The EB1212 storage-computing integrated controller can simultaneously complete the distributed storage and sample preprocessing of samples during the large model training process, while simplifying the data flow process of SSD-memory-video memory in the traditional architecture, directly completing the data flow from SSD to video memory, and improving the data access speed.

[0039] At the same time, the computing unit on the EB1212 storage and computing integrated controller can also assist in some logical processing and data calculation work during training and reasoning, maintaining a parallel working state with the graphics card, thereby improving the training and reasoning speed of large models.

[0040] In the above implementation, preferably, the GPU module includes a computing unit and a video memory, the computing unit is used to perform model reasoning, and data flows directly between the video memory and the storage-computing integrated controller.

[0041] Specifically, during the implementation process, the EB1212 integrated storage and computing controller is used, which integrates CPU, NPU or GPU computing modules and storage modules. The CPU module and NPU module can be used to reason about the Embedding model, and then the vector features of knowledge information / query questions can be obtained. The self-developed KV database based on hardware characteristics can be used to efficiently store and compare vector databases.

[0042] During the implementation process, the storage and computing integrated chip with EB1212 storage and computing integrated controller and NVIDIA T4 card were used to search the knowledge base respectively. The efficiency comparison is shown in the following table.

[0043]

[0044] According to the analysis of the above table, the knowledge base search performance based on the EB1212 storage and computing integrated controller is slightly higher than that based on NVIDIA T4. Therefore, this storage and computing integrated chip has a higher cost-effectiveness advantage in knowledge base search.

[0045] The present invention further proposes a large model application acceleration system based on a storage-computing integrated controller, which applies the large model application acceleration method based on a storage-computing integrated controller disclosed in the above embodiment, including:

[0046] A model integration connection module is used to integrate a computing module and a storage module based on a storage-computing integrated controller, the storage-computing integrated controller is connected to a CPU module and a GPU module respectively, and the storage-computing integrated controller and the GPU module cooperate to perform data processing;

[0047] The document vector storage module is used to, in the initialization phase, pre-process and segment the private documents through the calculation module, extract vector features using the Embedding model, and then store the vector features in the private vector knowledge base of the storage module;

[0048] The data direct transfer module is used to directly transfer data between the GPU module and the storage module, and to perform distributed storage and sample preprocessing on sample data during large model training.

[0049] The knowledge vector reasoning module is used to perform Embedding model reasoning using the computing module according to the private knowledge base query instruction received from the CPU module to obtain the vector features of the knowledge query problem;

[0050] The knowledge storage query module is used to use the KV database based on hardware characteristics to store and compare vector features in the private knowledge database;

[0051] The graphics card model inference module is used to directly transfer the query results or search results from the computing module to the GPU module for model inference.

[0052] In the above implementation, preferably, the GPU module includes a computing unit and a video memory, the computing unit is used to perform model reasoning, and data flows directly between the video memory and the storage-computing integrated controller.

[0053] According to the large model application acceleration system based on the storage and computing integrated controller disclosed in the above-mentioned implementation mode, the functions to be implemented by each module thereof correspond to the steps of the large model application acceleration method based on the storage and computing integrated controller disclosed in the above-mentioned implementation mode. During the implementation process, operations are performed with reference to the above-mentioned implementation mode, which will not be repeated here.

[0054] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A large model application acceleration method based on a storage and computing integrated controller, characterized in that: include: Based on the integrated computing and storage controller, the computing module and the storage module are integrated, the integrated computing and storage controller is connected to the CPU module and the GPU module respectively, and the integrated computing and storage controller cooperates with the GPU module to perform data processing; In the initialization phase, after the private document is preprocessed and segmented by the computing module, vector features are extracted using the Embedding model, and the vector features are stored in the private vector knowledge base of the storage module; Data is directly transferred between the GPU module and the storage module, and distributed storage and sample preprocessing are performed on sample data in the large model training process; According to the private knowledge base query instruction received from the CPU module, the computing module is used to perform Embedding model reasoning to obtain vector features of the knowledge query question; Using a KV database based on hardware characteristics, the vector features are stored and compared in the private knowledge database; The computing module directly transfers the query results or search results to the GPU module for model reasoning.

2. The large model application acceleration method based on the storage and computing integrated controller according to claim 1 is characterized in that: The GPU module includes a computing unit and a video memory. The computing unit is used to perform model reasoning, and data is directly transferred between the video memory and the storage-computing integrated controller.

3. A large model application acceleration system based on a storage and computing integrated controller, characterized in that: The large model application acceleration method based on the storage-computing integrated controller as claimed in claim 1 or 2 comprises: A model integration connection module, used for integrating a computing module and a storage module based on a storage-computing integrated controller, wherein the storage-computing integrated controller is connected to a CPU module and a GPU module respectively, and the storage-computing integrated controller cooperates with the GPU module to perform data processing; The document vector storage module is used to, in the initialization phase, pre-process and segment the private document through the calculation module, extract vector features using the Embedding model, and store the vector features in the private vector knowledge base of the storage module; A data direct flow module, used to directly transfer data between the GPU module and the storage module, and to perform distributed storage and sample preprocessing on sample data in the large model training process; A knowledge vector reasoning module, configured to perform Embedding model reasoning using the computing module according to the private knowledge base query instruction received from the CPU module, and obtain vector features of the knowledge query question; A knowledge storage query module, used for storing and comparing the vector features in the private knowledge database using a KV database based on hardware characteristics; The graphics card model inference module is used to directly transfer the query results or search results from the calculation module to the GPU module for model inference.

4. The large model application acceleration system based on the storage and computing integrated controller according to claim 1 is characterized in that: The GPU module includes a computing unit and a video memory. The computing unit is used to perform model reasoning, and data is directly transferred between the video memory and the storage-computing integrated controller.

Images