Computer hardware system and artificial intelligence calculation program
By introducing different types of memory devices and processors into the computer hardware system, resource sharing is achieved, solving the problem of memory resource management in AI computing systems, improving efficiency and stability, and extending system lifespan.
Patent Information
- Application Number
- CN202510976732.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-07-17
- Filing Date
- 2025-07-16
- Publication Date
- 2026-01-20
AI Technical Summary
Existing AI computing systems have high requirements for memory resource management and optimization. In particular, the limited number of read/write cycles of solid-state drives leads to a shortened system stability and lifespan, necessitating an effective solution to improve the performance and stability of computer hardware systems.
By introducing a primary storage device, a primary processor, a secondary storage device, and an accelerator processor into the computer hardware system, which are used to store and process write-intensive and read-intensive data respectively, the advantages of different memory devices are utilized to avoid the high-frequency read and write wear of solid-state drives and achieve resource sharing.
It improves data processing efficiency, ensures the efficient operation of artificial intelligence computing programs, extends the service life of computer hardware systems, and reduces costs.
Smart Images
Figure CN121365033A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of artificial intelligence technology, in particular to a computer hardware system and an artificial intelligence computing program executed by the computer hardware system, which utilizes resource sharing to effectively improve the performance and stability of the computer hardware system and prolong the service life of the computer hardware system. BACKGROUND
[0002] With the rapid development of artificial intelligence (AI) technology, its application range is continuously expanding, from image recognition, speech recognition to natural language processing, etc. AI technology is changing our lives. However, behind these technologies, there is a huge dependence on computing resources and storage space. The training and inference process of AI operation requires a large amount of memory to store the parameter data of the AI model and the cache data generated during calculation, which not only increases the cost of equipment, but also puts higher requirements on the management and optimization of computing resources. In this case, resource sharing becomes an important way to solve the problem.
[0003] First of all, we need to understand the demand for memory in AI computing. Modern AI models, such as deep neural networks, usually contain millions or even billions of parameters. These parameters need to be stored in memory for quick access and update during training and inference. In addition, a large amount of intermediate data (i.e. cache data) will be generated during the calculation process, which also needs to be stored in memory. For example, training a large natural language processing model (such as GPT-3) may require hundreds of GB or even TB of memory space.
[0004] In the process of resource sharing, in addition to the management and allocation of computing resources, the characteristics of different memory devices also need to be considered. Because the use of solid state drives (SSD) such as NAND flash has its limit on the number of read-write cycles, long-term and high-frequency read-write operations will cause wear and tear to the memory resources, resulting in a shortened lifespan of these devices. Therefore, in the process of resource sharing, the properties of these devices need to be considered to ensure the stability and long-term operation of the system.
[0005] Therefore, it is necessary to provide a computer hardware system and an artificial intelligence computing program executed by the computer hardware system, which utilizes resource sharing to effectively improve the performance and stability of the computer hardware system at a lower cost, and prolong the service life of the computer hardware system. SUMMARY
[0006] An object of the present application is to provide a computer hardware system and an artificial intelligence computing program using the same, which solves the problem of insufficient memory space in a low-cost manner using resource sharing, and improves the performance and stability of the computer hardware system, which is particularly important for AI operations that require intensive processing of large-scale data.
[0007] To achieve the above object, in one aspect of the present application, a computer hardware system is provided for executing an artificial intelligence computing program, the computer hardware system comprising: a primary storage device, a primary processor, a secondary storage device, and at least one acceleration processor. The primary storage device is configured to store instructions and cache data for executing the artificial intelligence computing program. The primary processor is connected to the primary storage device and configured to execute a set program in the artificial intelligence computing program. The secondary storage device is configured to store artificial intelligence model data. The at least one acceleration processor is connected to the secondary storage device and the primary processor, wherein the at least one acceleration processor is configured to load the artificial intelligence model data stored in the secondary storage device, and execute a plurality of layer operations in the artificial intelligence computing program based on the artificial intelligence model data, and access the cache data in the primary storage device through the primary processor.
[0008] In another aspect of the present application, an artificial intelligence computing program is also provided, which is executed by a computer hardware system comprising a primary storage device, a primary processor, a secondary storage device, and at least one acceleration processor. The artificial intelligence computing program comprises: accessing instructions in the primary storage device using the primary processor to execute a set program; loading artificial intelligence model data from the secondary storage device to the at least one acceleration processor; and using the at least one acceleration processor to execute a plurality of layer operations in the artificial intelligence computing program based on the artificial intelligence model data, and accessing cache data in the primary storage device through the primary processor.
[0009] In yet another aspect of the present application, a computer hardware system is also provided for executing an artificial intelligence computing program, the computer hardware system comprising: a primary storage device, a primary processor, a secondary storage device, and at least one acceleration processor. The primary storage device is configured to store write-intensive data. The primary processor is connected to the primary storage device and configured to execute a set program in the artificial intelligence computing program. The secondary storage device is configured to store read-intensive and non-write-intensive data. The at least one acceleration processor is configured to load the read-intensive and non-write-intensive data stored in the secondary storage device, and access the write-intensive data in the primary storage device through the primary processor.
[0010] In summary, the computer hardware system and the artificial intelligence computing program using the computer hardware system of the present application effectively improve the performance and stability of the computer hardware system and prolong the service life of the computer hardware system by means of resource sharing.
[0011] The advantageous configurations and improvements of the present application described above and / or presented in the dependent claims can be applied individually and in any combination with each other, except in cases of mutually exclusive or incompatible alternatives. BRIEF DESCRIPTION OF DRAWINGS
[0012] The features, characteristics and advantages of the present application described above and how to achieve them are better and more clearly understood in conjunction with the following detailed description of embodiments, in which:
[0013] Figure 1 FIG. 1 is a schematic diagram of a computer hardware system of an embodiment of the present application executing an artificial intelligence computing program under an artificial intelligence operating system;
[0014] Figure 2 FIG. 2 is a flowchart of an artificial intelligence computing program of an embodiment of the present application;
[0015] Figure 3 FIG. 3 is a flowchart of a setting program of an artificial intelligence computing program of an embodiment of the present application; and
[0016] Figure 4 FIG. 4 is a flowchart of a setting program of an artificial intelligence computing program of another embodiment of the present application. DETAILED DESCRIPTION
[0017] REFERENCE Figure 1 FIG. 1 is a schematic diagram of a computer hardware system of an embodiment of the present application executing an artificial intelligence computing program under an artificial intelligence operating system. In Figure 1 In FIG. 1, under the environment of an artificial intelligence operating system 100, in order to execute an artificial intelligence computing program 110, a computer hardware system 120 is provided, which includes a main processor 124, a main storage device 125, at least one acceleration processor 128, and a secondary storage device 127. The main storage device 125 is configured to store instructions and cache data for executing the artificial intelligence computing program 110. The main processor 124 is connected to the main storage device 125 and is configured to execute a setting program in the artificial intelligence computing program 110.
[0018] In one embodiment, the artificial intelligence computing program 110 can be an artificial intelligence inference application or an artificial intelligence training application. For example, in performing the artificial intelligence inference application of the computing program 110, a setup program can be executed using the main processor 124. Specifically, the main processor 124 first defines a plurality of layer operations of a model and allocates the plurality of layer operations to at least one acceleration processor 128. Then, data is input from the input 121 and pre-processed by the main processor 124, such as normalization, standardization, feature extraction, or cleaning, to ensure that the data is suitable for the input format of the model. Then, the pre-processed data is imported into the plurality of layer operations defined in the model. After completing the setup program, the artificial intelligence model data is loaded from the secondary storage device 127 to at least one acceleration processor 128. Since existing artificial intelligence model data usually contains millions or even billions of parameters, in this embodiment, these parameters are mainly stored in the secondary storage device 127, which uses a solid state drive (SSD) such as NAND flash, a hard disk (HDD), NOR flash, RRAM, or FRAM. However, the solid state drive has a limit on the number of read-write cycles, and long-term and high-frequency read-write operations will cause wear and tear to the solid state drive, reducing its lifespan. Therefore, in this application, the secondary storage device 127 is mainly used to store data that is read-intensive and not write-intensive, taking advantage of the large capacity and low cost of SSDs while avoiding the disadvantage of limited write cycles. In one specific example, read-intensive and not write-intensive data includes artificial intelligence model data.
[0019] Then, at least one acceleration processor 128 and based on the artificial intelligence model data performs a plurality of layer operations in the artificial intelligence computing program 110, and accesses the cache data in the main storage device 125 through the main processor 124. Finally, the computer hardware system 120 outputs the results of the artificial intelligence inference application to the output 122. In this embodiment, these cache data are mainly stored in the main storage device 125, which uses memory such as DRAM, SRAM, MRAM, etc. However, such storage devices are relatively expensive, but they can withstand high-frequency and long-term write operations. Therefore, in this application, the main storage device 125 is mainly used to store write-intensive data, taking advantage of its durability and fast response. In one specific example, write-intensive data includes instructions and cache data for executing the artificial intelligence computing program. By using memory resources for different operations in this application, not only is data processing efficiency improved, but the artificial intelligence computing program can also be efficiently run.
[0020] As mentioned previously, in one embodiment, the artificial intelligence computing program 110 can be an artificial intelligence inference application or an artificial intelligence training application. For example, in the artificial intelligence training application of the artificial intelligence computing program 110, a setup program can be executed using the primary processor 124. Specifically, the primary processor 124 first defines a plurality of layer operations of a model and allocates the plurality of layer operations to at least one acceleration processor 128. Then, before the model training, data is input from the secondary storage device 127 and pre-processed by the primary processor 124, such as normalization, standardization, feature extraction, or cleaning, to ensure that the data is suitable for the input format of the model. Then, the iteration parameters of the model training are set, i.e., the number of times the model is trained on the entire data set, or the training cycle. A certain number of iterations is usually set to ensure that the model is fully learned. After the setup program is completed, the artificial intelligence model data is loaded from the secondary storage device 127 to at least one acceleration processor 128. Since existing artificial intelligence model data usually contains millions or even billions of parameters, in the present embodiment, these parameters are mainly stored in the secondary storage device 127, which uses a solid state drive (SSD) such as NAND flash, a hard disk (HDD), NOR flash, RRAM, or FRAM. However, the solid state drive has a limit on the number of read-write cycles, and long-term and high-frequency read-write operations will cause wear and tear to the solid state drive, reducing its lifespan. Therefore, in the present application, the secondary storage device 127 is mainly used to store data that is read-intensive and not write-intensive, taking advantage of the large capacity and low cost of the SSD while avoiding the disadvantage of limited write cycles. In one specific example, the data that is read-intensive and not write-intensive includes artificial intelligence model data.
[0021] Then, at least one acceleration processor 128 performs a plurality of layer operations in the artificial intelligence computing program 110 based on the artificial intelligence model data, and accesses the cache data in the primary storage device 125 through the primary processor 124. Finally, the computer hardware system 120 outputs the results of the artificial intelligence training application to the secondary storage device 127. In the present embodiment, these cache data are mainly stored in the primary storage device 125, which uses memory such as DRAM, SRAM, MRAM, etc. However, such storage devices are relatively expensive, but they can withstand high-frequency and long-term write operations. Therefore, in the present application, the primary storage device 125 is mainly used to store data that is write-intensive, taking advantage of its durability and fast response. In one specific example, the data that is write-intensive includes instructions and cache data for executing the artificial intelligence computing program. By allocating the use of memory resources for different operations in the present application, not only is the data processing efficiency improved, but also the artificial intelligence computing program can be efficiently run.
[0022] In one embodiment, the computer hardware system 120 of the present application can further comprise a storage controller 126 connected to the secondary storage device 127, which can be configured to determine whether the at least one accelerator processor 128 is accessing the secondary storage device 127.
[0023] In one embodiment, the computer hardware system 120 of the present application can further comprise an accelerator storage device 129 connected to the at least one accelerator processor 128 for the at least one accelerator processor 128 to access. In one embodiment, the accelerator storage device 129 can be considered as an extension of the primary storage device 125, i.e. using memory such as DRAM, SRAM, MRAM, etc. to primarily store write-intensive data for the at least one accelerator processor 128 to access. In one embodiment, the write-intensive data includes instructions and cache data for executing the artificial intelligence computing program. In one embodiment, the accelerator storage device 129 can also be accessed by the primary processor 124 through the at least one accelerator processor 128, fully exploiting the effect of resource sharing.
[0024] In one embodiment, the primary processor 124, the storage controller 126, and the at least one accelerator processor 128 of the computer hardware system 120 of the present application are in communication with each other through a PCIe interface 123. However, it is to be understood by those skilled in the art that the present application is not limited to the use of the PCIe interface 123.
[0025] In one embodiment, the at least one accelerator processor 128 of the computer hardware system 120 of the present application can be a GPU, NPU, TPU, ASIC, etc. which can be directly connected to the individual accelerator storage device 129, or can access the write-intensive data in the primary storage device 125 through the primary processor 124 and access the read-intensive and non-write-intensive data in the secondary storage device 127 through the storage controller 126.
[0026] With Figure 1 With the disclosed computer hardware system 120, the present application also provides an artificial intelligence computing program executed by the computer hardware system 120, wherein the computer hardware system 120 comprises a primary processor 124, a primary storage device 125, a secondary storage device 127, and at least one accelerator processor 128. Referring to Figure 2 , which is a flowchart of the artificial intelligence computing program of one embodiment of the present application. In Figure 2 , the artificial intelligence computing program of the present application comprises the following steps. First, in step S21, the primary processor 124 accesses instructions in the primary storage device 125 to execute a set program.
[0027] As mentioned previously, the artificial intelligence computing program 110 can be an artificial intelligence inference application. Figure 3 Flowchart of the setting program of the artificial intelligence computing program 110 for the artificial intelligence inference application executed by the computer hardware system 120 of the present application. In Figure 3 In one embodiment, the setting program of the artificial intelligence computing program 110 for the artificial intelligence inference application executed by the computer hardware system 120 of the present application is shown in FIG. 2. In the embodiment, first, step S211 is executed, the main processor 124 first defines the plurality of layer operations of the model and allocates the plurality of layer operations to the at least one acceleration processor 128. Next, in step S212, data is input from the input 121 and pre-processed by the main processor 124, such as normalization, standardization, feature extraction or cleaning, etc., to ensure that the data is suitable for the input format of the model. Next, in step S213, the pre-processed data is imported into the plurality of layer operations defined in the model.
[0028] In another embodiment, the artificial intelligence computing program 110 can also be an artificial intelligence training application. Figure 4 Flowchart of the setting program of the artificial intelligence computing program 110 for the artificial intelligence training application executed by the computer hardware system 120 of the present application. In Figure 4 In one embodiment, the setting program of the artificial intelligence computing program 110 for the artificial intelligence training application executed by the computer hardware system 120 of the present application is shown in FIG. 3. In the embodiment, first, step S216 is executed, the main processor 124 first defines the plurality of layer operations of the model and allocates the plurality of layer operations to the at least one acceleration processor 128. Next, in step S217, data is input from the secondary storage 127 before the model training and pre-processed by the main processor 124, such as normalization, standardization, feature extraction or cleaning, etc., to ensure that the data is suitable for the input format of the model. Next, in step S218, the iteration parameters of the model training are set, i.e. the number of times the model is trained on the entire dataset, or called training epoch. Usually a certain number of iterations is set to ensure that the model is fully learned.
[0029] Next, in step S22, the artificial intelligence model data is loaded from the secondary storage 127 to the at least one acceleration processor 128. Then, in step S23, based on the artificial intelligence model data, the at least one acceleration processor 128 executes the plurality of layer operations in the artificial intelligence computing program and accesses the cache data in the primary storage 125 through the main processor 124. In one embodiment, the at least one acceleration processor 128 can be directly connected to the accelerator storage 129 for access by the at least one acceleration processor 128. In one embodiment, the accelerator storage 129 can be considered as an extension of the primary storage 125, i.e. using memory such as DRAM, SRAM, MRAM, etc. to primarily store write-intensive data for access by the at least one acceleration processor 128. In one embodiment, the accelerator storage 129 can also be accessed by the main processor 124 through the at least one acceleration processor 128, fully realizing the effect of resource sharing.
[0030] Next, in step S24, the main processor 124 determines whether all layer operations have been completed. If so, the main processor 124 obtains the operation result of this time of artificial intelligence computing program, and outputs it from the output terminal 122 or stores it in the secondary storage device 127. If the main processor 124 determines that all layer operations have not been completed, step S22 is executed again, and the artificial intelligence model data is loaded from the secondary storage device 127 to at least one of the acceleration processors 128.
[0031] As discussed above, the computer hardware system and the artificial intelligence computing program using the computer hardware system of the present application solve the problem of insufficient memory space in a low-cost manner using resource sharing, and improve the performance and stability of the computer hardware system, which is particularly important for AI operations that require intensive processing of large-scale data.
[0032] The present application is described in detail by preferred embodiments. However, it is not limited by the disclosed examples, because the skilled person can derive other variants therefrom without departing from the scope of protection of the present application.
[0033] List of reference signs
[0034] 100 artificial intelligence operating system
[0035] 110 artificial intelligence computing program
[0036] 120 computer hardware system
[0037] 121 input terminal
[0038] 122 output terminal
[0039] 123 PCIe interface
[0040] 124 main processor
[0041] 125 main storage device
[0042] 126 storage controller
[0043] 127 secondary storage device
[0044] 128 acceleration processor
[0045] 129 accelerator storage device
[0046] S21-S25, S211-S213, S216-S218 steps
Claims
1. A computer hardware system for executing an artificial intelligence computing program, the computer hardware system comprising: a main storage device configured to store instructions for executing the artificial intelligence computing program and cache data that is write-intensive; a main processor connected to the main storage device and configured to execute a setup program in the artificial intelligence computing program and access the cache data that is write-intensive in the main storage device; a secondary storage device configured to store data that is read-intensive and not write-intensive, including artificial intelligence model data; at least one accelerator processor connected to the secondary storage device and the main processor, wherein the at least one accelerator processor is configured to load the artificial intelligence model data stored in the secondary storage device, execute a plurality of layer operations in the artificial intelligence computing program based on the artificial intelligence model data, and access the cache data in the main storage device through the main processor, wherein the setup program executed by the main processor comprises the following steps: defining the plurality of layer operations and assigning the plurality of layer operations to the at least one accelerator processor; inputting data and pre-processing the data; and importing the pre-processed data to the plurality of layer operations. 2.The computer hardware system of claim 1, further comprising a storage controller connected to the secondary storage device and configured to determine whether the at least one accelerator processor accesses the secondary storage device. 3.The computer hardware system of claim 1, further comprising an accelerator storage device configured to be connected to the at least one accelerator processor for access by the at least one accelerator processor. 4.The computer hardware system of claim 2, wherein the main processor, the at least one accelerator processor, and the storage controller communicate with each other through a PCIe interface. 5.The computer hardware system of claim 1, wherein the setup program executed by the main processor comprises the following steps: defining the plurality of layer operations and assigning the plurality of layer operations to the at least one accelerator processor; loading pre-processed data from the secondary storage device; and setting iteration parameters. 6.An artificial intelligence computing program executed by a computer hardware system, wherein the computer hardware system comprises a main storage device, a main processor, a secondary storage device, and at least one accelerator processor, and the artificial intelligence computing program comprises: accessing write-intensive instructions in the main storage device using the main processor to execute a setup program; loading data that is read-intensive and not write-intensive, including artificial intelligence model data, from the secondary storage device; and executing a plurality of layer operations in the artificial intelligence computing program using the at least one accelerator processor based on the artificial intelligence model data and accessing cache data in the main storage device through the main processor, wherein the setup program comprises the following steps: defining the plurality of layer operations and assigning the plurality of layer operations to the at least one accelerator processor; inputting data and pre-processing the data; and importing the pre-processed data to the plurality of layer operations. 7. The artificial intelligence computing program of claim 6, further comprising: using the storage controller to determine whether the at least one accelerator processor accesses the secondary storage device.
8. The artificial intelligence computing program of claim 6, further comprising providing an accelerator storage device configured to be connected to the at least one accelerator processor for access by the at least one accelerator processor.
9. The artificial intelligence computing program of claim 7, wherein the host processor, the at least one accelerator processor, and the storage controller communicate with each other through a PCIe interface.
10. The artificial intelligence computing program of claim 6, wherein the setup program executed by the host processor comprises the steps of: defining the plurality of layer operations and assigning the plurality of layer operations to the at least one accelerator processor; loading pre-processed data from the secondary storage device; and setting iteration parameters.
11. A computer hardware system for executing an artificial intelligence computing program, the computer hardware system comprising: a primary storage device configured to store write-intensive data; a host processor connected to the primary storage device and configured to execute a setup program in the artificial intelligence computing program; a secondary storage device configured to store read-intensive and non-write-intensive data; and at least one accelerator processor configured to load the read-intensive and non-write-intensive data stored by the secondary storage device and execute a plurality of layer operations in the artificial intelligence computing program based on the read-intensive and non-write-intensive data, and access the write-intensive data in the primary storage device through the host processor, wherein the setup program executed by the host processor comprises the steps of: defining the plurality of layer operations and assigning the plurality of layer operations to the at least one accelerator processor; inputting data and pre-processing the data; and importing the pre-processed data to the plurality of layer operations.
12. The computer hardware system of claim 11, wherein the write-intensive data comprises instructions and cache data for executing the artificial intelligence computing program.
13. The computer hardware system of claim 11, wherein the read-intensive and non-write-intensive data comprises artificial intelligence model data.