Method for reducing energy consumption based on star large model data preprocessing
Through data preprocessing and sparse fully connected computing mode based on the Star Model, combined with the space mapping computing architecture, the high temperature problem caused by the combination of artificial intelligence chips and 3D DRAM is solved, and more efficient energy consumption management and chip operation performance are achieved.
Patent Information
- Application Number
- CN202411755719.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-03
- Publication Date
- 2025-04-11
AI Technical Summary
The existing artificial intelligence chips are directly combined with high-density storage technologies such as 3D DRAM, resulting in high temperature problems. Traditional designs cannot make full use of on-chip cache resources, resulting in increased high temperatures.
The data preprocessing method based on the Star Model is adopted to extract key information of high-temperature data and match it with the Star Model vector database, and a wide data table is constructed for high-temperature weight marking and computing power. Combined with the sparse fully connected computing mode and spatial mapping computing architecture, the temperature of the memory is optimized.
By reducing the total amount of off-chip memory access, reducing peak temperature, improving the accuracy and timeliness of large-scale model retrieval, reducing the duration of analyzing text, and achieving more efficient chip operation.
Smart Images
Figure CN119806804A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of AI chip technology, and specifically relates to a method for reducing energy consumption based on star large model data preprocessing. Background Art
[0002] Neural network computing, especially fully connected neural network and recurrent neural network computing, has huge bandwidth and storage requirements. In order to meet these bandwidth and storage requirements, it is possible to consider combining high-density off-chip storage technologies such as 3D DRAM.
[0003] However, according to the JEDEC standard, once the DRAM temperature reaches 85C, the memory's self-refresh rate must double, and double for every 10 degrees increase, which will greatly reduce the reliability and performance of DRAM. Therefore, heat dissipation becomes a problem that must be solved for high-density off-chip storage. The TPUV3 chip uses 3D DRAM to meet high-bandwidth requirements. Existing artificial intelligence chips are directly combined with high-density storage technologies such as 3D DRAM. Most traditional artificial intelligence chips use a time division mapping (TDM) mechanism, and existing artificial intelligence chip designs cannot fully utilize on-chip cache resources. Sparsification does not save on-chip cache resources, which leads to high temperature problems. In order to solve the resulting high temperature problem, a water cooling system is even used. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a method for reducing energy consumption based on star large model data preprocessing in view of the deficiencies of the above-mentioned prior art.
[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:
[0006] A method for reducing energy consumption based on star large model data preprocessing, comprising:
[0007] Step 1: Use the Xingchen Big Model to obtain key information of high-temperature data and build a wide data table after matching it with the Xingchen Big Model vector database. Perform high-temperature weight marking and computational quantification on the data within the wide data table to obtain a data result set containing high-temperature weight markings.
[0008] Step 2: According to the principle that the higher the sparsity of the large model data result set is, the greater the benefit is. The spatial mapping computing architecture and the same weight algorithm for computing locality are used for the smart chip. The input data is assigned to the same weight calculation unit, and finally the high-temperature weight mark in the data result set is updated.
[0009] To optimize the above technical solutions, the specific measures taken also include:
[0010] In the above step 1, the connection operation between the key information result tables obtained by matching the database of the Xingchen big model is performed to obtain the label value of each data and the weight affecting the high temperature of the chip, and the proportion of problems corresponding to each data is calculated based on different label values and weight aggregation.
[0011] In the above step 1, the Star Big Model is used to obtain key information of high temperature data, including resource requirements, computing power quantization, external storage access frequency and cache resource utilization; the Star Big Model queries the corresponding data in the Star Big Model vector database based on the key information, constructs a dictionary data structure, and forms a data wide table.
[0012] The above dictionary data structure is: -##Start reference##Resource requirements, computing power quantification, external storage access frequency, cache resource utilization##End reference##.
[0013] The above step 2 optimizes the temperature of the memory by smoothing the memory access behavior through space partition mapping.
[0014] The above-mentioned spatial mapping computing architecture adopts a global cache based on a high-speed scratchpad memory, a multi-register cache, and a cache-oriented blocking and scheduling mechanism to maximize data reuse and adopts a zero-padding-free sparse computing method to reduce the total amount of memory access.
[0015] The above-mentioned spatial mapping computing architecture includes a partitionable and reconfigurable computing unit array, a global SPM cache, a register cache and a data flow controller; when there are K weight computing units running in the index service, the operation request is forwarded to the weight computing unit with the largest idle computing power for processing; the input data with computational locality is handed over to the same weight computing unit.
[0016] The present invention has the following beneficial effects:
[0017] The present invention aims at the high temperature problem caused by the direct combination of existing artificial intelligence chips with high-density storage technologies such as 3D DRAM. Firstly, key information is extracted from high-temperature data: four indicators, namely resource demand, computing power quantification, external storage access frequency and cache resource utilization, are matched with the Xingchen large model vector database to build a data wide table. Secondly, high-temperature weight marking and computing power quantification are performed on the data within the data wide table to improve the accuracy and timeliness of large model retrieval.
[0018] Compared with directly generating text based on original large-scale data and queries, the present invention is only based on a vector database and retrieves data within a wide table range, which helps to alleviate the hallucination problem of large models, generates text with higher accuracy, and the difficulty of extracting corresponding data values in this method is also smaller.
[0019] The present invention stores the data of the intermediate reasoning steps in a data wide table structure, which is more intuitive and easier to process than the original data table. The scale of the constructed data wide table is significantly reduced, thereby reducing the time required to analyze the text.
[0020] The present invention uses the spatial mapping computing architecture to calculate the same weight algorithm for computing locality for the smart chip according to the principle that the higher the sparsity of the sparse fully connected computing mode, the greater the benefit. The input data is assigned to the same weight computing unit (LM2-LM4) to be responsible, thereby reducing the total amount of memory access between LM1 and (LM2-LM4) and running more efficiently. Finally, the high temperature weight mark in the result set is updated. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 It is a principle block diagram of the method of the present invention;
[0022] Figure 2 The calculation process for population profile distribution analysis and indicator analysis;
[0023] Figure 3 An example of accelerating sparse fully connected computations in a compute unit. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0025] Although the steps in the present invention are arranged with numbers, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" used in this article involves and covers any and all possible combinations of one or more of the associated listed items.
[0026] like Figure 1 As shown, the present invention provides a method for reducing energy consumption based on star large model data preprocessing, comprising:
[0027] Step 1: Directly combining existing AI chips with high-density storage technologies such as 3D DRAM leads to high temperature problems. Innovatively, firstly, extract key information from high temperature data (resource demand, computing power quantification, external storage access frequency, and cache resource utilization) and match the four indicators with the Xingchen large model vector database to build a data wide table. Secondly, high temperature weight marking and computing power quantification are performed on the data within the data wide table to improve the accuracy and timeliness of large model retrieval.
[0028] First, we analyze the main reasons for high temperature problems caused by the direct combination of existing artificial intelligence chips with high-density storage technologies such as 3D DRAM, including the following three points.
[0029] First, most traditional artificial intelligence chips use the time division mapping (TDM) mechanism, for example, hierarchical time mapping. In practical applications, neural networks containing various types of network layers (such as convolutional layers, link layers, etc.) are widely used. Under the time division mapping mechanism, various network layers in the neural network are executed in sequence, and different network layers have different tendencies in resource utilization. Generally speaking, when executing the convolutional layer, the demand for computing resources is greater, and when executing the fully connected layer and the loop layer, the demand for bandwidth resources is greater. This will cause large temperature fluctuations and the peak temperature may be very high.
[0030] Second, existing AI chip designs cannot fully utilize on-chip cache resources. They usually divide the on-chip cache into three independent parts, which are used to store the input, output, and weight data of the neural network. However, the requirements for on-chip cache for different layers in the neural network are also very different. If a fixed size of on-chip cache is allocated to the input, output, and weight data, the capacity of the three on-chip caches often cannot match the actual demand well during the execution of the entire neural network, resulting in waste. If the on-chip cache cannot be fully utilized, it will cause more intensive off-chip storage access, thereby exacerbating the high temperature problem.
[0031] Third, sparsification does not save on-chip cache resources. Sparsification technology is often used to accelerate the fully connected layer in artificial intelligence chips based on single instruction multiple data flow computing units. However, the current artificial intelligence chips based on 2D computing unit arrays usually still store dense weights in the on-chip cache, that is, the pruned parts of the fully connected layer and the loop layer after sparsification are filled with 0 and then saved. This dense representation prevents the on-chip cache resources from obtaining the benefits of sparsification and does not effectively utilize the on-chip cache resources. This will also lead to intensive off-chip storage access and aggravate the high temperature problem.
[0032] Secondly, analyze the solution to the high temperature problem:
[0033] The high temperature problem in 3D DRAM includes two aspects: one is stable high temperature; the other is excessive peak temperature. The two have different causes and are related to each other, so they need to be jointly optimized to solve the problem. Stable high temperature is mainly affected by the total amount of off-chip memory access, so we can use zero-free sparse computing, global cache, cache-oriented blocking and scheduling and other optimization methods to reduce the total amount of off-chip memory access and reduce the stable temperature. Peak temperature is mainly affected by the memory access behavior pattern. We can reduce the peak temperature by utilizing the complementarity of memory access requirements between different types of network layers in the neural network, and by reasonable resource allocation and computing scheduling, smoothing the memory access at each moment.
[0034] Finally, the logical chain concept of the star model of the present invention is proposed:
[0035] Step 1: Extract key information from high-temperature data (resource demand, computing power quantization, external storage access frequency, and cache resource utilization) and match them with the Xingchen large model vector database to build a data wide table.
[0036] Step 1-1: Data wide table example: The join operation between the four result tables obtained by matching the database through the Xingchen big model (resource demand, computing power quantification, external storage access frequency, and cache resource utilization) can obtain the label value of each data and the weight that affects the high temperature of the chip. According to the two indicators of different label values and weights, the corresponding problem ratio analysis can be calculated by aggregation.
[0037] For example, if a population has 1 million people, of which 600,000 are male and 400,000 are female, then the gender distribution of the population is 60% (60 / 100) and 40% (40 / 100). Indicator analysis requires the use of aggregate functions for calculation, such as the average function AVG, the sum function SUM, and the maximum (minimum) value function MAX (MIN). Figure 2 Taking gender and online time as examples, the calculation process of population portrait distribution analysis and indicator analysis is demonstrated.
[0038] Guide the large language model: First, extract the key information of the question and convert it into a wide data table.
[0039] Step 1-2: Model extracts key information:
[0040] First, obtain data related to resource requirements, computing power quantification, external storage access frequency, and cache resource utilization. The advantage of applying a large model is that you can get all relevant associated data by entering keywords. If you don’t need it, query the database and make associations.
[0041] The computational power quantification model is as follows:
[0042]
[0043] In the formula, C br is the total computing power requirement; f(x) is the mapping function; a, B and y are the mapping ratio coefficients; q is the redundant computing power. Taking parallel computing power as an example, assuming there are three different types of parallel computing chip resources: b1, b2, and b3, then f(b j ) represents the mapping function of the parallel computing capability that the j-th parallel computing chip b can provide, and q2 represents the redundant computing power of parallel computing.
[0044] Step 1-3: The model queries the corresponding data in the data table based on the key information to build a dictionary data structure:
[0045] In the provided context candidate fragment, a piece of information related to the interference signal was found:
[0046] -##Start quoting##Resource requirements: 8 cores, 16G, 100T, computing power quantization: 1500p, external storage access frequency: MB / s, cache resource utilization: 30%##End quoting##
[0047] Step 2: Output the answers to the questions in natural language text form based on the contents of the data wide table structure
[0048] Guide the large language model (analysis engine): Secondly, answer the previous questions based on the data structure and output the question answer process.
[0049] Guide the large language model: organize the answer points generated by the model in the previous step into coherent sentences to ensure that the information is accurate and logically clear:
[0050] -##Large model dynamic parameters start##Resource requirements: 8 cores, 16G, 100T, computing power quantization: 1500p, external storage access frequency: MB / s, cache resource utilization: 30%##Large model dynamic parameters end##Data result set containing high temperature weight tags
[0051] Compared with directly generating text based on the original large-scale data and query, this method is based only on the first-level vector database. The second wide-table retrieval of data helps to alleviate the hallucination problem of large models, the generated text is more accurate, and the difficulty of extracting corresponding data values in this method is also less.
[0052] The data of the intermediate reasoning steps are stored in a data wide table structure, which is more intuitive and easier to process. Compared with the original data table, the scale of the constructed data wide table is significantly reduced, thereby reducing the time for analyzing the text.
[0053] Step 2: The large model data result set is based on the principle that the higher the sparsity of the sparse fully connected computing mode, the greater the benefit. The spatial mapping computing architecture adopts the same weight algorithm for computing locality for the smart chip, and the input data is assigned to the same weight computing unit (LM2-LM4), which is conducive to reducing the total memory access between LM1 and (LM2-LM4) and more efficient operation. Finally, the high temperature weight mark in the result set is updated. The computing architecture is optimized as follows:
[0054] The architecture mainly consists of four parts:
[0055] Partitionable and reconfigurable compute cell array, global SPM cache, register cache, and data flow controller.
[0056] When there are K weight calculation units running in the index service, the operation request is forwarded to the weight calculation unit with the largest idle computing power (obtained by computing power quantization model calculation) for processing.
[0057] According to formula (2), the probability that the two smallest elements in (LM2-LM4) of the K weight calculation units are equal is equal to the value of the number of identical elements in (LM2-LM4) divided by the number of all elements in (LM2-LM4).
[0058] It can be seen that using the RCID method for scheduling can assign input data with computational locality to the same weight calculation unit (LM2-LM4) with a high probability, which is beneficial to reducing the total memory access between LM1 and (LM2-LM4) and more efficient operation.
[0059]
[0060] In the sparse fully connected computing mode, the computing unit is executed as follows Figure 3 As shown in the figure, (a) is the original expression of the sparse fully connected operation, and (b) is its execution process on the computing unit.
[0061] In the sparse fully connected mode, the input and output are stored in the local memory LM1 and LMS of the computing unit respectively.
[0062] LM2, LM3, and LM4 work in FIFO mode to cache the non-zero weight values and corresponding column / row coordinates in the fully connected layer.
[0063] Under the control of the computational unit controller, the calculation of the sparse fully connected layer on the computational unit includes four steps:
[0064] The D calculation unit reads the non-zero weight values and their column / row coordinates from LM3, LM2, and LM4 respectively;
[0065] The computation unit uses the column coordinate to index the input operand from LM1 (in[col]), and uses the row coordinate to index the partial sum stored in LM5 (out[row]).
[0066] @The computing unit performs multiplication and accumulation operations: out[row] = in[col]xweight;
[0067] @Write the result back to LM5.
[0068] In this way, during the calculation process, the on-chip cache does not need to store zero values in the weights, and more space can be saved to store other valid values, thereby reducing the amount of access to off-chip memory. It should be noted that the coordinates of non-zero weight values also require storage space, so the higher the sparsity, the greater the benefit of this calculation method.
[0069] The terms in this article are explained as follows:
[0070] AI chips are also called AI accelerators or computing cards, which are modules specifically designed to handle large amounts of computing tasks in artificial intelligence applications (other non-computing tasks are still handled by the CPU).
[0071] Memory chips are the specific application of the concept of embedded system chips in the storage industry. Therefore, whether it is a system chip or a memory chip, it is achieved by embedding software in a single chip to achieve multi-function and high performance, as well as support for multiple protocols, multiple hardware and different applications.
[0072] Computing power (also known as hash rate) is a unit of measurement for the processing power of the Bitcoin network. It is the speed at which a computer (CPU) calculates the output of a hash function. Computing power is a measure of the total computing power of a unit that generates new blocks at a certain network cost.
[0073] China Telecom Artificial Intelligence Technology Co., Ltd. and China Telecom (601728) Artificial Intelligence Research Institute (TeleAI) released a product called Xingchen Big Model Software Factory (hereinafter referred to as Xingchen Software Factory), and said that this is the first breakthrough AI development tool in China.
[0074] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
[0075] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
Claims
1. A method for reducing energy consumption based on star large model data preprocessing, characterized in that: include: Step 1: Use the Xingchen Big Model to obtain key information of high-temperature data and build a wide data table after matching it with the Xingchen Big Model vector database. Perform high-temperature weight marking and computational quantification on the data within the wide data table to obtain a data result set containing high-temperature weight markings. Step 2: According to the principle that the higher the sparsity of the large model data result set is, the greater the benefit is. The spatial mapping computing architecture and the same weight algorithm for computing locality are used for the smart chip. The input data is assigned to the same weight calculation unit, and finally the high-temperature weight mark in the data result set is updated.
2. The method for reducing energy consumption based on star large model data preprocessing according to claim 1 is characterized in that: The step 1 connects the key information result tables obtained from the Xingchen big model matching database to obtain the label value of each data and the weight affecting the high temperature of the chip, and calculates the proportion of problems corresponding to each data based on different label values and weight aggregation.
3. The method for reducing energy consumption based on star large model data preprocessing according to claim 1 is characterized in that: The step 1 uses the Star Big Model to obtain key information of high temperature data, including resource requirements, computing power quantification, external storage access frequency and cache resource utilization; the Star Big Model queries the corresponding data in the Star Big Model vector database based on the key information, constructs a dictionary data structure, and forms a data wide table.
4. The method for reducing energy consumption based on star large model data preprocessing according to claim 3 is characterized in that: The dictionary data structure is: -##Start quoting##Resource requirements, computing power quantification, external storage access frequency, cache resource utilization##End quoting##.
5. The method for reducing energy consumption based on star large model data preprocessing according to claim 1 is characterized in that: The second step optimizes the temperature of the memory by smoothing the memory access behavior through space partition mapping.
6. The method for reducing energy consumption based on star large model data preprocessing according to claim 1 is characterized in that: The spatial mapping computing architecture adopts a global cache based on a high-speed scratchpad memory, a multi-register cache, and a cache-oriented blocking and scheduling mechanism to maximize data reuse and adopts a zero-padding-free sparse computing method to reduce the total amount of memory access.
7. The method for reducing energy consumption based on star large model data preprocessing according to claim 1 is characterized in that: The spatial mapping computing architecture includes a partitionable and reconfigurable computing unit array, a global SPM cache, a register cache, and a data flow controller; when there are K weight computing units running in the index service, the operation request is forwarded to the weight computing unit with the largest idle computing power for processing; Assign input data with computational locality to the same weight calculation unit.